This is independent security research conducted on my own time. The findings and views expressed here are my own and do not represent my employer.
We spend enormous effort securing what models decide to do. Less attention is paid to an earlier question: what did the model believe it was looking at?
If the answer to the second question is wrong, everything secured at the first can run correctly against a reality that does not exist. The reasoning can be sound, the guardrails can hold, the policy can be enforced, and the object all of it operates on can still be a reconstruction that diverged from the application's canonical data.
I want to argue that perception is a distinct security surface, that it is not yet clearly owned end to end, and that much of the industry's attention is concentrated one layer above it.
What I mean by perception
Not vision. I mean everything the model settles before and during reasoning about what the input contains: how many entities are present, where each one starts and ends, which spans belong together, which spans are data and which are instruction, what is content and what is container, what deserves attention and what is background. The word is a functional label for that set of decisions, not a claim about experience.
Some of that structure may be represented explicitly, in prompts, schemas, role tokens, serialization, or tool interfaces. But the model still reconstructs much of the operational structure it reasons over: which boundaries dominate, which entities exist, which spans belong together, and which representation should be treated as authoritative. That reconstruction is what I mean by the perception layer.
So the problem is not that the model receives no structure. It is that structure the application considers authoritative can become just another pattern the model is free to reinterpret. This is the distinction I set out in the founding post on systemtheory: token boundaries are enforced by a component outside the model, object boundaries are inferred inside it, and the security consequences follow from which side of that line a given piece of structure sits on. Structured outputs, tool schemas and JSON mode all help, but they constrain the shape of what the model emits, not the model's reading of what arrived. A schema that guarantees a well-formed array of records says nothing about whether the records in it correspond to the ones in the source.
There is serious prior work near this boundary, and it is worth being precise about what it covers. The instruction hierarchy work from OpenAI trains models to privilege system-level instructions over instructions found in data. Microsoft's spotlighting marks or transforms untrusted spans so the model can tell them apart from the task. StruQ separates the prompt into a structured query with distinct instruction and data channels. Simon Willison's dual-LLM pattern quarantines the model that touches untrusted content from the one that acts. CaMeL, from Google DeepMind, goes furthest: it separates control flow from data and carries capabilities with each value through the pipeline, so that where a piece of data came from constrains what can be done with it. All of these are aimed at the authority axis, the question of which text is allowed to direct the model. None of them addresses the identity axis, the question of how many objects there are and where each one ends. CaMeL's provenance tracking is the closest thing I have found to half of what I am describing, and I would build on it rather than around it. But I have not found a layer that is responsible, end to end, for keeping the model's object set equal to the application's.
Why the attention went elsewhere
The asymmetry is not stupidity, it is incentive. Behavior is where failure is legible: a model that says the wrong thing produces an artifact you can screenshot, evaluate, red-team, and benchmark. So we built a large apparatus around it, including alignment training, refusal evaluation, jailbreak research, prompt engineering as a discipline, guardrail products, and injection detectors.
Perception failures are less visible. The output may contain no obvious sign that the model's internal object model diverged from the application's canonical one. It can be fluent, internally consistent, and formatted exactly as expected. What looks like a reasoning result may actually be faithful reasoning over a corrupted reconstruction.
The same asymmetry runs through capability work. Enormous effort goes into better reasoning and better data, and both are worth doing. But better reasoning over an input whose structure was never guaranteed can produce a more confident wrong answer, and better data does not help if the relationships and boundaries inside that data can be reconstructed incorrectly.
What a perception attack looks like
The clearest case I have documented is a spreadsheet with five rows, one of which carries text shaped with separators inside a single free-text cell. Asked to summarize the dataset, the assistant reports fourteen records. The extra nine are reconstructed out of that one cell, and the reported count follows a fixed function of the injected content:
reported = records_in_the_cell + (real_rows - 1)
That relationship is what separates the behavior from ordinary model noise. It is also not specific to spreadsheets. Counting the host record's own fields as one of the fragments, the same function reproduces every count in the published experiments across formats: two JSON records with five fragments in one field become six, eleven XML nodes with five become fifteen, thirty CSV rows with ten become thirty-nine, and two JSON records with eleven become twelve. The minus one is the real record being taken apart, its fields redistributed across the fabricated ones, so that it does not survive as a unit.
In one product, the fabricated rows are then written into a new sheet as persistent data. That step is worth stating plainly: records that never existed become stored artifacts that downstream systems and people will treat as real. A summary that is merely inaccurate is a quality problem. Fabricated records written into storage under the user's authority are an integrity problem, and the distinction does not depend on how the output reads.
There is no instruction anywhere in the payload. Nothing tells the model to do anything, and many injection defenses are designed around instruction or authority conflict, so this failure may not present the signal they are designed to detect. What changed is not what the model was told to do. What changed is how many things the model believed were there.
I have found the same class across formats and vendors, including mail clients, spreadsheets, cloud storage, and connector layers used by frontier assistants. The effect reproduces through different encodings of the same underlying table, and in some cases it survives export and downstream processing.
Vendors have often classified these findings as quality issues rather than security issues. That classification is itself informative: current taxonomies are much better at describing failures of behavior than failures of object reconstruction.
The distinction I care about can be expressed as a simple pipeline:
canonical input -> model-perceived representation -> reasoning -> action
Prompt injection mainly targets instruction and behavior. Classic parser bugs affect the canonical parse. Agentic failures can amplify errors originating at any stage. Structural desynchronization, the term I use for this class and document in the paper, targets the representation itself: the canonical parse and the model's reconstruction of it come apart, without either the instructions or the underlying parser being subverted. The security boundary I am interested in sits between those two representations.
A warning about encoding
Encoding tricks have a long history in this field, and much of that work is about bypass: hide an instruction in Morse, Base64, homoglyphs, escaping, or another representation a model learned during training, and attempt to slip it past a filter. That is a real technique, but it is not what I am describing here.
The interesting use of encoding in perception research is not concealment, it is segmentation and reconstruction. A payload can contain ordinary text with no hidden instruction and still alter where the model draws boundaries or how many objects it believes are present. The cleanest evidence that this is segmentation rather than recognition is in the paper: separators generated from os.urandom and from a SHA-1 over an RSA key, strings that have never appeared in any training corpus, are still promoted to record boundaries. There is nothing to recognize in them. Repetition and position are enough.
The model's learned decoding abilities widen this surface. An application may treat a sequence as an opaque string while the model recognizes internal structure, language, or separators inside it. Not every encoding creates an exploit, but the model may have a broader semantic decoder than the application supplying the data, and that gap is worth treating as a security boundary.
Perception is a surface, not a mechanism
This is the part I want to be careful about, because I tested the convenient version and it was wrong.
The tempting hypothesis is that all of this comes from one underlying mechanism, boundary collapse as a shared cause behind object inflation, prompt injection, and constraint failure. In a preliminary set of probing experiments, which I have not yet written up and which should be read with that caveat, I trained linear probes on one failure family and tested whether they transferred to the others. Cross-domain transfer was at chance. The phenomena were representationally distinct, so the unification hypothesis did not survive the experiment.
What survives is narrower and more useful. The same probes point to a localized representation associated specifically with object inflation and boundary reconstruction, and that result replicated across the three models I examined. I am deliberately not calling it a mechanism yet: probes show where a signal lives, not that it does the work, and the intervention experiments that would establish that are the next thing to run. Even at this strength, though, the picture is that perception should not be treated as a single mechanism waiting for one patch, but as a surface containing several distinct failure families, which is exactly why it needs a research program rather than a single fix.
It also explains why existing defenses can miss it. Many are designed around instruction conflict, authority inversion, or malicious intent encoded in the input, and object desynchronization can happen without any of those. The defenses are not necessarily weak. They are aimed at a different failure class.
Why this is architecturally hard
If perception were easy to secure, it would already be treated as a solved preprocessing problem. It is not, and the difficulty is structural.
Models receive rich mixtures of content, formatting, instructions, separators, schema hints, metadata, and context. Some of these are represented explicitly by the system, others are encoded only through surface form, and there is no general mechanism that makes application-level structure immutable to the model's interpretation.
That matters because the application may already know the correct structure. The spreadsheet knew there were five rows because it rendered them, the mail client knew where one message ended and another began, the connector knew which value belonged to which field. In many integrations that canonical structure is serialized into text or another model-consumable representation and then ceases to be authoritative, leaving the model free to reconstruct a different object model from the same material. Nothing in the stack is responsible for carrying that structure across the boundary as an invariant.
So the problem has two halves. The first is architectural: how should models represent and preserve boundaries, entities, provenance, and authority? The second is engineering: how should products carry already-known structure into the model without allowing content inside one object to become a peer-level object? The second half is not a fundamental model limitation. It is an integration problem.
Why this matters more now
Perception errors used to be absorbed by a human reading the answer, and that absorption is disappearing on two fronts at once.
Agents increasingly act on their reading without an intermediate human reader. And where a human is retained at an approval gate, the evidence presented to that human is often assembled downstream of the perception step, so a wrong reading can arrive at the gate as a clean and plausible summary. I have written separately about that gate placement problem; the short version is that oversight positioned after perception may never see the original structural error, and calling the downstream review a control does not make the upstream reconstruction trustworthy.
The industry is therefore reducing the amount of human friction that used to compensate for this layer, while simultaneously increasing the number of places where the layer can influence actions.
Epistemic status
The specific findings are documented and reproducible; the payloads and raw model outputs are public. The broader thesis, that perception is an undersecured layer in current AI systems, is still a hypothesis. I hold it on the strength of a bounded set of results, repeated behavior across products and models, and the way vendors currently classify these failures. I have not demonstrated it at the level I would want, and the probing results above are the least mature part of the evidence.
What would change my mind? If perception failures turn out to be reliably caught by existing behavioral defenses once those defenses are tuned, then the layer may not be distinct and this could be a subcase of existing injection research. If the failures do not generalize past the formats and products I have tested, then this may be a collection of parsing and integration bugs rather than a broader layer. And if the probing results fail to replicate on models I have not examined, or do not hold up under intervention, the representational story becomes much weaker.
I would rather publish the frame and be corrected than hold it until it is airtight. If someone has already built the thing I am describing, I want to read it.
What progress would look like
Not a filter. Filters presuppose that you know which shapes are dangerous, and many of the payloads that work here are ordinary text with no hostile signature.
Progress looks like structure that is asserted rather than merely suggested. The deterministic parse should be carried across the model boundary instead of discarded. Fields should remain isolated so that content inside one field cannot become a peer-level object regardless of what the content looks like. Counts, joins, aggregates, and structural facts should be computed from canonical data rather than read back from the model's reconstruction. Provenance should be preserved so that a reviewer can trace a generated claim to an original object without relying on the same model that reconstructed it. And agent actions should operate on canonical object identifiers rather than entities invented during free-form reconstruction.
None of this reduces what models are good for. It removes one job they should not be given implicitly: deciding which application-level objects exist.
We have built a large apparatus for asking whether a model did the right thing. We still have no dependable way to answer the question that comes before it: what did it believe it was looking at?