'You Don't Need Long Context' implementation

Quick post about the You don't need long context paper.

I built this feature into my custom agentic harness (jinn) since the technique and results seemed promising. However I ended up tuning the values to basically turn it into a last-resort compaction technique rather than as-implemented by the authors of the paper.

TL;DR: The paper doesn't work in practice.

My Concern

In the paper, the authors have a system prompt of about ~1k context, but my system prompt is approximately 28k context once tools + skills are included. I only run with 4-5 skill descriptions (not full content), but i do have 27 tools enabled, and the AGENTS.md for the project I tested this with is about 8.5k context. I adjusted the thresholds from the paper to accommodate.

Task

The #goal is a prompt macro that inserts instructions on plan generation and rules for implementation:

#goal session-cwd-ghost

the change cwd popup requires either manually typign in a path or using the external fzf integration. i would like to add a "ghosting" autocomplete that shows potential autocompletion results for a given path. pressing TAB accepts the path. use the existing fuzzy matching machinery to fuzzy match directory names as the user types and display a "ghosted" (muted text) path that the user can press TAB to automatically fill in. no recursion: just search the one directory the user is typing. should be async since it hits the disk (make new actor if needed)

Since each session generated slightly different questions in response to the initial prompt, I said to "use recommended options" for all runs and let the agent pick everything.

Results

Baseline

The baseline run doesn't have the feature at all, no compactions were run, and no context mutations occurred. So this is the "FHA" scenario described in the paper (Full-History Accumulation).

• 99% cache hit rate
• 33.2M input tokens
• 277.8k output tokens
• 02:05:00 wall time
• $0.3055 projected cost using deepseek-v4.1-flash-1017
• Task completed (no bugs)

80k/60k

This run had the context limit at 80k with the compaction threshold at 60k.

• 96% cache hit rate
• 38.3M input tokens
• 560.3k output tokens
• 03:53:20 wall time
• $0.4734 projected cost using deepseek-v4.1-flash-1017
• Task completed (with bugs)
  • Autocompletion showed things that don't exist on disk
• "Working set" context size ~29k tokens

50k/40k

This run had the context limit at 50k with the compaction threshold at 40k. However I had to up it to 60k/50k halfway through because it turned into an infinite compaction loop.

• 91% cache hit rate
• 30.9M input tokens
• 730.7k output tokens
• 03:49:50 wall time
• $0.5226 projected cost using deepseek-v4.1-flash-1017
• Task completed (with bugs)
  • Dir scanning was running in the UI thread instead of async/actor
• "Working set" context size ~15k tokens

Conclusion

I don't think it performs better than my existing context pruning strategies. I'm still keeping the feature around in jinn since I like the implementation better than my previous compaction implementation. However, I changed the config to be "working set" based with automatic threshold calculations. This way any amount of skills/tools/MCP still works fine without the limit/threshold being hard-coded.