Why Models Break Character Limits and How to Fix It with Code

Suppose a publishing form accepts a headline of no more than 60 characters. You ask a language model to write one under 60 characters. Sometimes it succeeds. Sometimes it returns 61, 64, or 72 characters. The result may even contain a useful explanation around the headline, making the value unsuitable for direct submission.

This is not a contradiction. Natural-language generation and exact string measurement are different tasks. A model predicts likely text; your application must enforce the contract. The production pattern is simple:

1. Ask for more candidates than you need.
2. Measure every candidate in code.
3. Reject candidates that fail the exact rule.
4. Ask for a revision when no candidate passes.
5. Validate once more immediately before storing or sending the result.

Why a prompt is not a character-limit guarantee

A prompt such as “write a headline under 60 characters” is a useful instruction, but it is not a validation mechanism. Several details make exact counting difficult for a model:

• Generation is probabilistic. The model is choosing a plausible continuation, not executing a deterministic counting loop.
• Tokens are not characters. A token is a unit used by a model internally. It does not map one-to-one to a visible character.
• Unicode has edge cases. Accented letters, emoji, combining marks, and some scripts can make “what a user sees” differ from a programming language’s simple string length.
• Formatting can leak into the value. Quotes, labels such as “Headline:”, Markdown, or a trailing explanation may be counted by the receiving system.
• Different systems count differently. A form may count Unicode code points, UTF-16 code units, bytes, or grapheme clusters.

Therefore, the first engineering question is not “how do I phrase the prompt more strongly?” It is “what exact string rule does the destination enforce?”

Define the contract before generating text

Write the rule as executable requirements. For example:

{
  "field": "headline",
  "max_characters": 60,
  "min_characters": 20,
  "counting_method": "unicode_code_points",
  "allowed_output": "plain_text_only",
  "trim_outer_whitespace": true
}

This example is a configuration proposal, not a universal standard. Replace counting_method with the rule used by your actual destination. If the destination counts visible user-perceived characters, code-point length may still be the wrong measure.

Keep the limit in one configuration value. Avoid copying “60” into prompts, tests, UI code, and database checks independently. A single source of truth reduces the chance that one part of the pipeline silently uses a different limit.

A minimal filtering pipeline

The following Python example demonstrates the core method. It uses Python’s built-in len(), which counts Unicode code points for ordinary Python strings. Confirm that this matches the receiving system before using it as a production rule.

MAX_CHARS = 60
MIN_CHARS = 20

def normalize_candidate(value: str) -> str:
    """Remove only formatting that the contract explicitly permits."""
    return value.strip()

def character_count(value: str) -> int:
    return len(value)

def is_valid_candidate(value: str) -> bool:
    value = normalize_candidate(value)

    if not value:
        return False

    if character_count(value) < MIN_CHARS:
        return False

    if character_count(value) > MAX_CHARS:
        return False

    return True

candidates = [
    "A Practical Guide to Reliable Model Output",
    "How to Make Generated Headlines Fit a Strict Limit",
    "Headline: How to Make Generated Headlines Fit a Strict Limit"
]

valid = [item for item in candidates if is_valid_candidate(item)]

if not valid:
    raise ValueError("No candidate satisfies the headline contract")

headline = valid[0]
print(headline)
print(character_count(headline))

Notice what the code does not do: it does not shorten an invalid headline by slicing it at character 60. Blind truncation can cut a word, remove a closing punctuation mark, or change the meaning. Rejection is safer than silently damaging the generated text.

Generate several candidates instead of one

Asking for one answer creates a brittle all-or-nothing interaction. Asking for several candidates gives the validator alternatives. Request a machine-readable structure so the application can separate values from commentary.

{
  "task": "Write five alternative headlines about validating model output.",
  "constraints": {
    "min_characters": 20,
    "max_characters": 60,
    "output": "plain headline strings",
    "no_prefixes": true,
    "no_explanations": true
  },
  "response_format": {
    "headlines": ["string", "string", "string", "string", "string"]
  }
}

The format above is an illustrative request body. The exact field names depend on the model API or SDK you use. The important design choice is that the application receives a list it can inspect, rather than prose it must guess how to parse.

A simple selection function can preserve the original order:

def choose_first_valid(candidates: list[str]) -> str | None:
    for candidate in candidates:
        cleaned = normalize_candidate(candidate)
        if is_valid_candidate(cleaned):
            return cleaned
    return None

You can later add ranking—such as relevance, duplication checks, or editorial preferences—but the length check should remain an explicit, testable gate.

Request a revision when every candidate fails

Over-generation improves the first attempt, but it cannot guarantee success by itself. Build a bounded retry loop. Pass back the measured failures and ask for shorter replacements.

MAX_ATTEMPTS = 3

def build_revision_request(candidates: list[str]) -> str:
    failures = [
        {
            "text": item,
            "characters": character_count(normalize_candidate(item)),
            "maximum": MAX_CHARS
        }
        for item in candidates
        if not is_valid_candidate(item)
    ]

    return (
        "Rewrite every candidate so it is plain text, between "
        f"{MIN_CHARS} and {MAX_CHARS} characters. "
        "Return only a JSON object with a 'headlines' array. "
        f"Measured failures: {failures}"
    )

def select_with_revisions(generate):
    candidates = generate()
    
    for attempt in range(MAX_ATTEMPTS):
        selected = choose_first_valid(candidates)
        if selected is not None:
            return selected

        candidates = generate(build_revision_request(candidates))

    raise RuntimeError("Generation did not produce a valid headline")

The final exception is intentional. A system that cannot produce a valid value should fail visibly, use a reviewed fallback, or place the item in a human-review queue. It should not return an invalid string while pretending the contract was met.

Validate at the boundary, not only after generation

Validation belongs at every boundary where invalid data could escape:

• after parsing the model response;
• after normalization;
• before writing to storage;
• before sending the value to a third-party API or form;
• in automated tests for the complete pipeline.

The last check is especially important. A later transformation may add a prefix, replace characters, or concatenate a brand name. Validate the final value, not only the model’s raw response.

def assert_contract(value: str) -> None:
    cleaned = normalize_candidate(value)

    if cleaned != value:
        raise ValueError("Value contains disallowed outer whitespace")

    if not is_valid_candidate(cleaned):
        raise ValueError(
            f"Invalid headline length: {character_count(cleaned)} "
            f"(allowed: {MIN_CHARS}-{MAX_CHARS})"
        )

Unicode: decide what “character” means

A strict limit is incomplete until its counting method is documented. These are different measurements:

• Bytes: useful when a protocol limits encoded payload size.
• Code points: a common Unicode-oriented string measure.
• UTF-16 code units: relevant to some platform APIs.
• Grapheme clusters: closer to visible user-perceived characters.

For example, an emoji may occupy more than one code unit in some environments. A letter plus a combining mark may be displayed as one visible unit while being stored as multiple code points. These are technical examples, not a claim about the counting behavior of any particular service.

If your destination documents a counting rule, implement that rule directly and test representative inputs. If it does not document one, record the assumption and include edge cases in your verification suite.

Verification you can reproduce

Start with boundary tests. For a maximum of 60 characters, test values of 59, 60, and 61 characters. Also test empty strings, whitespace, formatting prefixes, non-ASCII text, and model responses with unexpected fields.

def test_length_boundaries():
    assert is_valid_candidate("a" * 59)
    assert is_valid_candidate("a" * 60)
    assert not is_valid_candidate("a" * 61)

def test_empty_and_whitespace_are_rejected():
    assert not is_valid_candidate("")
    assert not is_valid_candidate("   ")

def test_prefix_is_counted_when_present():
    value = "Headline: " + ("a" * 55)
    assert not is_valid_candidate(value)

Run the tests with the command used by your project, for example:

python -m pytest

This article does not report a test run for your environment. The command is a reproducible starting point; run it in the project that contains your implementation. In a real pipeline, log the measured length, contract version, attempt number, and final validation result without logging secrets or personal data.

Common failure cases

The model returns an explanation

Parse a structured response and reject fields that are not part of the contract. Do not assume the first line is always the requested value.

Normalization changes the count

Decide whether trimming, whitespace collapsing, punctuation replacement, or Unicode normalization is allowed. Apply the same transformation before every measurement.

All candidates are too long

Send a bounded revision request containing the measured lengths. If the retry budget is exhausted, use a reviewed fallback or escalate to human review.

The value passes locally but fails remotely

Compare counting methods and the exact transmitted value. A wrapper, label, encoding step, or hidden whitespace may be part of what the remote system validates.

Truncation creates poor copy

Prefer regeneration. If truncation is an explicit product requirement, implement a word-aware or domain-specific shortening strategy and validate its output again.

Limitations and practical trade-offs

This pipeline guarantees that your selected value satisfies the rule implemented by your validator. It cannot guarantee that a third-party service uses the same definition of “character” unless you have matched that definition. It also cannot guarantee that a valid headline is accurate, useful, original, or legally safe; those are separate review dimensions.

More candidates and retries can increase latency and usage. Keep the retry count bounded, measure the failure rate, and choose a fallback appropriate to the product. For high-value publishing workflows, a human-review queue may be preferable to endlessly requesting shorter text.

The reliable pattern

Treat character limits as application constraints, not writing suggestions. Define the counting rule, keep the limit in configuration, generate multiple candidates, measure them in code, filter invalid values, request bounded revisions, and validate the final string at the destination boundary.

The model remains useful for producing language and alternatives. Your code remains responsible for enforcing the contract.

Continue with the guides or look up unfamiliar terms in the glossary.