# Day 03: Your repo does not fit in the context window. Now what?

*Handing cartographer files one at a time had been working, so I did the obvious next thing and wrote the four lines that walk a repository and concatenate every source file into one string. Large context windows exist precisely so you can stop being clever about this.*

*Then, out of habit, I printed* `len()` *on the string before sending it.*

*I never sent it. Not because it would definitely have failed — some of it would have fit — but because I could not answer either of the two questions that immediately followed. How many tokens is that, actually? And what does one run of this cost? I had been building an agent for two days with no number attached to any part of it.*

*So today's build is not a feature. It is a tape measure and a meter. What they tell you is the constraint the entire project exists to work around, and it is better to meet it with a* `print()` *than with an invoice.*

* * *

## The Short Version

**Code for this day:** [cartographer at day-03](https://github.com/Harshavardhan17/cartographer/tree/day-03)

*   A repository is measured in **characters on disk**; a model bills in **tokens**, the chunks of text it reads and writes. The ratio between them is not a constant you can look up.
    
*   **Do not estimate with a tokenizer library from a different model family.** You get a precise number for the wrong vocabulary, because each family splits text into tokens its own way, and the error is systematic rather than random.
    
*   **Two samples and a subtraction** give you the real ratio and cancel the fixed per-request overhead that a single sample would silently bake in.
    
*   The context window, the most one request can carry, is the wall you notice. **The token count is the wall you hit first**, because a transcript re-sends itself on every call.
    
*   Input and output are priced differently, so **a meter that adds the two token counts together tells you nothing**.
    
*   The conclusion is not "wait for a bigger window". It is that **the agent has to choose what to read**. That is a feature, not a workaround.
    

## Measure the repository first

Walking a repository is easy. Walking it *correctly* is where the first mistake lives.

```python
def source_files(root: pathlib.Path) -> Iterator[pathlib.Path]:
    for path in root.rglob("*"):
        if path.suffix not in SOURCE_SUFFIXES or not path.is_file():
            continue
        if SKIP_DIRECTORIES.intersection(path.relative_to(root).parts):
            continue
        yield path

def measure(root: pathlib.Path) -> tuple[int, int]:
    paths = list(source_files(root))
    return len(paths), sum(len(path.read_text(errors="ignore")) for path in paths)
```

Both live in a new module, `src/cartographer/budget.py`, which knows about files and arithmetic and nothing about HTTP. `rglob("*")` walks every path under the root. A file survives only if its extension is in `SOURCE_SUFFIXES` (Python, docs, config) and none of its folder names below the root is in `SKIP_DIRECTORIES`; the set intersection checks them all at once. `measure` returns the file count and the total characters, ignoring bytes that are not valid text.

The skip list is doing more work than the walk. Leave `.venv` and `node_modules` out of it and you are not measuring a project, you are measuring its dependencies, which is a much larger number that says nothing about the code. **Measure the code someone wrote, not everything installed beside it.**

Run that on any repository you have lying around before reading further. The character count is real and takes seconds to get.

## The estimate everybody gets wrong

Characters are not what you are billed for. So you need a ratio, and there are two popular ways to get one, both wrong.

The first is `characters // 4`. It is a folk constant, it is not specific to your model, and it is not specific to source code, which tokenizes differently from prose because of indentation, punctuation and long snake\_case identifiers.

The second is worse, because it feels rigorous: install a tokenizer library, encode the repository, report the count. That library ships one vendor's vocabulary. If you are calling a model from a different family, you have computed a precise number for a tokenizer nobody in your request is using. Precision is not accuracy, and the error is not noise you can average away — it is a systematic bias in a direction you cannot see from the outside, because you do not have the vocabulary the server used.

## Two samples and a subtraction

There is an authoritative counter available and it costs almost nothing to query: the server tells you what it counted, in `usage.prompt_tokens`, on every response.

```python
def prompt_tokens(text: str) -> int:
    body = post([{"role": "user", "content": text}], max_tokens=1)
    return int(body["usage"]["prompt_tokens"])
```

`post` is the old body of `ask` in `src/cartographer/agent.py`, now its own function so it can return the whole response and accept an optional `max_tokens`; `ask` now calls it and keeps its old signature. One request, capped at a single output token, because we are measuring input, not generating an answer.

Now the naive move is to send one sample and divide. Do not: `prompt_tokens` counts the whole rendered request, including the chat template the server wraps your message in. On a short sample that fixed overhead is a large fraction of the total, and you would fold it permanently into your ratio.

Send two samples of very different sizes instead, and take the slope. The constant cancels.

```python
def chars_per_token(count: Callable[[str], int], small: str, large: str) -> float:
    return (len(large) - len(small)) / (count(large) - count(small))
```

Two requests, two output tokens, and you have a ratio measured against the model you are actually calling. It goes stale the moment you change models, which is the argument for it being a function rather than a constant in a config file.

The test that matters is offline: give it a counter with a known overhead of 11 tokens and a known true ratio of 4, and assert the result is exactly 4.0. If the subtraction were wrong, that assertion would fail.

## Count input and output separately

The other half of today is the meter, and it exists because the two token counts are not interchangeable.

```python
@dataclass
class Meter:
    input_per_million: float = 0.0
    output_per_million: float = 0.0
    calls: int = 0
    input_tokens: int = 0
    output_tokens: int = 0

    def record(self, usage: dict[str, int]) -> None:
        self.calls += 1
        self.input_tokens += usage.get("prompt_tokens", 0)
        self.output_tokens += usage.get("completion_tokens", 0)
```

`record` adds each response's `usage` block to running totals, and `post` calls it on every response, so no request can bypass the meter. A `dollars` property multiplies each total by its own price per million, and `tests/test_budget.py` pins all three pieces offline: the calibration, the skip list and the pricing. Every line is explained in [today's dev notes](https://github.com/Harshavardhan17/cartographer/blob/main/docs/devnotes/day-03.md).

Input and output are billed at different rates on every provider I have used, so a single total is not convertible into money. Keeping them apart also tells you which lever to pull: the transcript grows and every file gets re-sent, while the output stays a paragraph.

The rates themselves come from environment variables and default to zero. No price literal goes into the source, because published rates change without anyone editing your repo, and a stale number in version control is worse than no number. On a free model the dollars are honestly zero and the token counts are still true, which is the point: tokens are the thing that runs out, against the window and against rate limits, long before money does.

## Three ways to make a repo fit

|  | Send the whole repo | Index it all up front | Let the agent choose what to read |
| --- | --- | --- | --- |
| **You pay** | the whole repo, on every call | one embedding pass (every file turned into searchable numbers), then cheap lookups | only for the files it opened |
| **Breaks when** | the repo outgrows the window, which is soon | the index goes stale on the next commit | the agent chooses badly |
| **Right when** | one question, one small repo | stable code, many repeated queries | large repo, open-ended questions |

The third column is the one this project needs, and not reluctantly. A map made from everything is not a map: it is the territory again. Choosing what matters is the actual product, and the constraint is what forces the design to be honest about it.

My numbers for the pinned evaluation repositories live in `docs/measurements.md`, with a commit SHA beside each one (click 8.5.0 came to roughly half a million tokens), because a measurement against a moving branch is not a measurement and a blog post is the wrong place to freeze it.

## What I'd do

1.  **Measure before you design.** A few seconds of `len()` will tell you whether the architecture you are imagining is even legal.
    
2.  **Calibrate against the model you call.** Two samples, subtract, never a folk constant and never another vendor's tokenizer.
    
3.  **Keep prices out of source.** Environment variables, defaulting to zero.
    
4.  **Print the meter on every run**, including free-tier runs. The habit is the point.
    
5.  **Treat selection as the feature.** The moment you accept the agent must choose what to read, tools stop being a convenience and become the product.
    

* * *

*If this was useful, a clap helps other people find it. I would like to know how other people size this up: do you estimate agent cost before you build, or find out from the first surprising bill?*

* * *

*All views in this post are my own. The scenarios are illustrative composites, and code examples are not drawn from any specific codebase.*