Instruction Format Density Benchmark

6 frontier models · 13 formats · 8 tests · 3 runs · 1,872 scored calls · March 2026

How compressed can an LLM system prompt get before the model stops following instructions?

Human-readable Compressed Inhuman Extreme
Sweet Spot
compressed_nl
88% compliance at 41% of markdown's size. Stripped grammar, zero labels, near-full compliance.
The Cliff
< 200 chars
58% pass rate. Below 200 chars, models lose formatting rules and identity. Factual accuracy partially survives.
Best Density
bitfield
515 compliance-per-size ratio. 59% pass at just 138 chars (11% of markdown).
The Compression Cliff
Average pass rate by character count band, all models
< 200 chars
58%
bitfield, hex_tagged
200 – 400
74%
emoji_code, microcode, symbol_stream
400 – 700
83%
compressed_nl, positional_grid, dsl
700 – 1100
86%
json_compact, table, yaml, b64_hybrid
1100+ chars
90%
markdown
Model × Format Heatmap
Pass rate (%) per model per format, sorted by density
Format Ch Opus 4.6 DS V3.2 Gem3 Pro GLM-5 Kimi 2.5 Qwen 3.5 AVG
hex_tagged 133 62634755654857
bitfield 138 70577735635059
emoji_code 201 75688382776876
microcode 238 73677260728071
symbol_stream 292 77658882756575
compressed_nlSWEET SPOT 503 87939090808888
positional_grid 532 75659077727576
json_compact 765 88758290807381
dsl 688 87808093838585
b64_hybrid 1048 75739292938886
table 896 87839383979289
yaml 984 90908587908788
markdown 1213 95978590928090
AVERAGE 80% 75% 82% 78% 80% 75%
Robustness: Markdown → Extreme
How much does each model degrade from markdown to extreme compression formats?
gemini-3.1-pro
62% → 85%
-23pp
kimi-k2.5
64% → 92%
-28pp
claude-opus-4.6
66% → 95%
-29pp
qwen3.5-397b
49% → 80%
-31pp
deepseek-v3.2
60% → 97%
-37pp
glm-5
45% → 90%
-45pp
What Breaks First
Biggest compliance drops from human-readable to extreme formats
CheckHuman-readableExtremeDrop
Concise responses87%17%-70pp
Max 3 paragraphs94%25%-69pp
No markdown headers89%25%-64pp
Direct tone94%36%-58pp
Inline citations76%22%-54pp
Mentions research role98%58%-40pp
Identifies as Atlas94%56%-38pp
Identifies Python96%83%-13pp
Refuses medical diagnosis93%86%-7pp
No fabrication100%97%-3pp
Density Efficiency Score
pass_rate / relative_size_vs_markdown — higher = more compliance per character
hex_tagged
517
133ch / 57%
bitfield
515
138ch / 59%
emoji_code
456
201ch / 76%
microcode
360
238ch / 71%
symbol_stream
313
292ch / 75%
compressed_nl
212
503ch / 88%
positional_grid
172
532ch / 76%
dsl
149
688ch / 85%
json_compact
129
765ch / 81%
table
121
896ch / 89%
yaml
109
984ch / 88%
b64_hybrid
99
1048ch / 86%
markdown
90
1213ch / 90%

Key Takeaways

compressed_nl remains the sweet spot for practical use: 88% compliance at 41% of markdown's size. It strips articles, pronouns, and filler while keeping semantic clarity. Every model scored 80%+ on it, with four of six above 88%.

The cliff lives at ~200 chars. Below that threshold, models lose formatting rules, identity, and citation style. Factual accuracy and refusals are the most resilient instruction types — no_fabrication still hits 97% even in extreme compression. Formatting rules (concise, paragraph limits) break first, dropping 65-70pp.

b64_hybrid was a surprise performer at 86% average, outscoring dsl (85%) and json_compact (81%) despite being the second-largest format. The base64 decoding overhead penalized it less than expected — Gemini (92%), GLM-5 (92%), and Kimi (93%) all excelled with it.

Gemini 3.1 Pro is the most resilient model under compression with only a 23pp drop from markdown to extreme formats. It led overall at 82% across all formats. DeepSeek V3.2, despite the highest markdown score (97%), dropped 37pp to extreme — strong at the top, fragile at the bottom.

GLM-5 remains the most brittle with a 45pp drop from markdown to extreme. It collapses particularly hard on bitfield (35%), dragging its average down despite strong showings on dsl (93%) and b64_hybrid (92%).

Run It Yourself
Python 3.10+ stdlib only. No dependencies.
See all generated prompts
python instruction_density_bench.py --show-prompts --no-score
Full elite benchmark (6 frontier models, 13 formats, scored)
python instruction_density_bench.py --openrouter-key YOUR_KEY
Quick smoke test (no scoring)
python instruction_density_bench.py --openrouter-key YOUR_KEY \ --models anthropic/claude-opus-4.6 z-ai/glm-5 \ --formats markdown hex_tagged \ --tests identity guardrail_medical --no-score
Just the inhuman formats
python instruction_density_bench.py --openrouter-key YOUR_KEY --inhuman-only