How this
actually works.

A tool is a function the model can ask to call, described by a JSON Schema: a name, a description, and typed parameters. The model has never seen your tool before; the schema and its description are the entire interface.

Tool design is therefore API design with an unusual constraint — the consumer reads the documentation exactly once, cannot ask clarifying questions, and will confidently invent a parameter that sounds plausible if your description leaves room for it.

The subtlety that costs real time: providers disagree about schemas. Anthropic supports a stricter dialect than the baseline; OpenAI's structured-output mode has its own rules; Bedrock differs again. Authoring the same tool several times is how those definitions drift apart, so the definitions are written once and converted per provider at request time.

Every model vendor invented a different wire format. Anthropic streams one event shape, OpenAI streams another, OpenAI's newer Responses API a third, and AWS Bedrock offers two more. Message roles, tool-call encoding, streaming deltas, stop reasons, and reasoning blocks all differ.

The methodology is an old one: normalize at the edges, keep the core single-shaped. Five separate streaming implementations all emit one internal event type. The agent loop has no idea which provider it is talking to. Adding a provider means writing one adapter, not touching the loop.

The payoff shows up in features nobody planned for. Because the core is provider-neutral, you can switch models mid-project, or run a subagent on something cheap while the main loop uses something expensive, without any of that logic knowing vendors exist.

Models have a fixed context window — the maximum number of tokens they can consider at once. Every message, every tool result, every attached file consumes it. A long agent session will exhaust any window, because tool results accumulate far faster than conversation does.

Compaction is the answer: when the conversation approaches the limit, summarize the older portion and replace it with the summary. The agent keeps working; it just remembers the distant past in less detail, the way you would remember a meeting from three hours ago.

Doing this well is harder than it sounds. Compaction itself costs a model call, so a naive implementation spirals — compact, immediately overflow, compact again. A circuit breaker is not optional.

Providers will cache a prefix of your request and charge far less to re-read it. Since an agent loop resends the whole conversation on every iteration, this is not a micro-optimization — it is the difference between a workable tool and an unaffordable one.

The rule that makes it work: caches key on an exact prefix. Anything that changes invalidates everything after it. So stable content — system prompt, tool schemas, early conversation — must come first, and volatile content must come last.

Put a timestamp near the top of your prompt and you have silently disabled caching for the entire session. Nothing will error. The bill just quietly multiplies.

To show a codebase as a graph, something has to decide what the nodes and edges are. Nodes are easy: files. Edges are the interesting problem, and it is a static analysis question — determining structure by reading code rather than running it.

The pipeline runs in two stages, and separating them is the whole trick:

1. Scan each file for import statements. Per-language pattern matching — fast, tolerant of broken syntax, cheap enough to run over a very large repository.
2. Resolve each import string to an actual file. This is where the difficulty lives, and it is unavoidably per-ecosystem: TypeScript path aliases from tsconfig.json, Go module paths, Java and C# namespaces, Python packages and re-exports, PHP autoload rules.

A full parser per language would be more precise and vastly more expensive. Scanning plus resolution gets most of the accuracy for a fraction of the cost, and degrades gracefully when it meets code it does not understand.

On top sits a service lens: detecting that one service calls another's HTTP API, publishes an event it consumes, or writes a table it reads. The hard part is precision — a naive matcher pairs any route string with any similar-looking URL and produces a hairball.

Once you have a graph you must position it. The standard approach is force-directed layout: treat edges as springs and nodes as mutually repelling particles, then simulate until it settles. It produces genuinely good results — related things cluster because the physics pulls them together.

It is also roughly O(n²) per tick. Beautiful at 2,000 nodes, unusable at 60,000.

Rather than accept a slow map on large repositories or a mediocre one everywhere, Blacksite switches algorithms at a threshold of 6,000 nodes, falling back to a linear-time phyllotaxis-style packing. This is a broadly useful methodology: an algorithm that is optimal in the common case and one that degrades gracefully in the extreme case are not competitors. Pick a threshold and ship both.

The Language Server Protocol is how editors get real semantic understanding. A language server for each language runs alongside the editor and answers structured questions: where is this symbol defined, what are its references, what type is this, what is wrong with this file.

This matters enormously for an agent. Ask a model to rename a symbol and it will pattern-match on text and miss cases. Ask a language server and you get the same answer the editor's own rename command would — because it is that command.

The second-order benefit is targeting. An edit aimed at ChatProvider.send survives the file changing around it; an edit aimed at line 412 does not.

A VS Code extension runs in the extension host, a Node.js process with filesystem and network access but no DOM. Any custom interface runs in a webview: a sandboxed browser context with a DOM but no filesystem, no require, and a strict Content Security Policy.

The two communicate only by passing serializable messages. That constraint shapes everything. State both sides need must be deliberately synchronized; anything expensive must be computed host-side and sent over; and payload size becomes a real performance concern, since every message is serialized and deserialized.

It also has a consequence for the graph: how you send it matters as much as how you draw it.

A single model call failing is a nuisance. An agent loop running for an hour across hundreds of calls will meet every transient failure mode there is: rate limits, timeouts, overloaded servers, streams that stop producing without closing, malformed tool calls.

The patterns that keep it alive are all conventional, and all necessary:

• Retry with exponential backoff on transient errors, with jitter so retries do not synchronize.
• Idle timeouts on streams — a stream that stalls without erroring will otherwise hang forever.
• Circuit breakers, so a failing subsystem stops being retried rather than consuming the whole budget.
• Bounded queues and output caps, so one enormous tool result cannot exhaust memory or context.
• Checkpointing, so a crashed session resumes rather than restarts.

Blacksite keeps this in one shared module rather than scattering try/catch through the provider layer. It is the least glamorous code in the project and probably the highest-value.