ADR 0032: Small Model Context Methodology & Platform-Native Orchestration

Status

Accepted | revised 2026-06-09 after empirical validation — see tests/e2e/reports/local-model-validation.md | revised 2026-06-10 after OpenCode config-semantics confirmation — see docs/notes/research/2026-06-construct-audit/20-opencode-ecosystem.md and 80-synthesis.md | revised 2026-06-22 after measured external-MCP schema capture — see tests/fixtures/mcp-tool-schemas/manifest.json

Context

Running local Ollama models (e.g. Qwen 7B) inside OpenCode produced severe repetition collapse ("word salad" — "client client … ver ver"). An empirical investigation (recording proxy + direct /v1 and native-API bisection, 2026-06-09) overturned the original hypotheses in this ADR and established the actual mechanics:

1. Context is not settable from opencode.json. OpenCode reaches Ollama through the OpenAI-compatible /v1 endpoint. num_ctx is forwarded in the request body but Ollama's /v1 endpoint does not honor it — the runtime window stays at Ollama's 4096 default. repeat_penalty (Ollama-specific) is likewise dropped. Only OpenAI-standard params (temperature, stop, top_p, frequency/presence_penalty) survive the boundary. The only way to set a real context window is a Modelfile PARAMETER num_ctx baked via ollama create.

2. Tool overload dominates the payload. A full Construct session serializes ~133 MCP tool schemas (~34k tokens) plus the system prompt — ~36k input tokens. Per-agent permission: "deny" and tools: {glob:false} do not remove a tool's schema from the payload (they only gate execution); only disabling a whole MCP server (mcp.\<id>.enabled:false) reduces the count.

3. The collapse is model-specific. qwen2.5-coder:7b degenerates on OpenCode's agentic system prompt regardless of context window, tool count, temperature, or repeat_penalty. The identical payload runs coherently — with correct tool calls — on qwen3-coder:32k and devstral:24b. Capability is not predictable from parameter count and must be probed empirically.

The earlier hypotheses — "cap context at 8k" and "inject repeat_penalty 1.15 via config" — are falsified: capping cannot be done over /v1, and the injected penalty is silently dropped.

Decision

A small-context local-model methodology with four pillars plus capability honesty:

1. Platform-Native Orchestration Alignment

Do not inject the static specialist roster into the system prompt on hosts with native subagent routing (OpenCode, VS Code, Cursor). They get a tool-bound micro-prompt and resolve the chain at runtime via the orchestration_policy MCP tool. Hosts without native routing (Claude Code, Codex) still receive the roster. Implemented in scripts/sync-specialists.mjs buildPrompt().

2. Real Context Windows via Modelfile Variants

For any tool-capable model lacking a baked num_ctx (size is not a gate — capability does not track parameter count), auto-provision a context-extended variant (\<model>-cx\<N>k) via ollama create with num_ctx, repeat_penalty, and ChatML stop baked in, and register the variant in place of the raw tag. Implemented in lib/ollama/provision-context.mjs, wired through platforms/opencode/sync-config.mjs.

3. Tool-Surface Reduction

Disable heavy external MCP servers (mcp.\<id>.enabled:false) — confirmed against OpenCode source (remeda deep-merge; mcp.enabled === false gate at mcp/index.ts) as the only lever that actually shrinks the serialized schema, since OpenCode exposes no per-session tool filter (chat.params carries no tools field) and scoped permission denies keep the schema (only a blanket "*": false + allowlist removes it).

Trim must be scoped to the project, not the machine (revised 2026-06-10). The original implementation disabled the heavy servers whenever Ollama models merely existed on the host (ollamaHasModels), which silently stripped context7/github from cloud-model sessions in other projects. Because a project .opencode/opencode.json can disable a globally-enabled server via deep-merge, the trim belongs in the project config and is gated on the project's resolved default model (intent), not on machine-wide Ollama presence. The decision is centralized in a pure decideTrim({scope, defaultModel, probeData}) so it is unit-testable and consistent across hosts.

4. Sampler Hygiene in the Right Place

opencode.json emits only boundary-surviving params (temperature, stop). The real sampler settings (num_ctx, repeat_penalty, stop tokens) are baked into the Modelfile variant. Never emit frequency_penalty/presence_penalty — the wrong knob for Qwen.

5. Capability Honesty

Some small models cannot do agentic tool use no matter how the context and tools are tuned. Construct provides an empirical coherence probe (provision-context.mjs --probe) and steers users toward agentic-capable local models rather than silently registering a model that will collapse.

The probe verdict must persist and be consumed, not just print (revised 2026-06-10). construct doctor --probe-local writes per-model results (verdict, ratios, model digest, timestamp) to a capability store; construct sync reads it (never re-probes — probing loads models and costs minutes) to skip Modelfile provisioning for COLLAPSED models, feed decideTrim's auto mode, and warn when a configured default model is COLLAPSED. Verdicts are digest-keyed so a re-pulled model is re-probed, and consumers warn-and-allow-override rather than silently hiding a model — a false COLLAPSED verdict must never strand a working model.

Consequences

• Positive: Capable local models (qwen3-coder, devstral) run the full Construct loop coherently; payload shrinks enough to fit a 32k window; users get an honest signal about model suitability instead of inexplicable word salad.
• Negative: Modelfile variants consume extra Ollama metadata (layers are shared, so disk cost is negligible) and depend on ollama create. Models that fail the coherence probe simply are not usable for agentic work — Construct surfaces this rather than papering over it.

Measured external-MCP schema cost (2026-06-22)

The earlier informal ~12k figure for the five heavy external servers (context7, github, memory, sequential-thinking, playwright) was understated. A reproducible tools/list capture (scripts/measure-external-mcp-schemas.mjs, fixtures in tests/fixtures/mcp-tool-schemas/) measured 37,281 schema tokens total (estimateToolTokens, TOKENS_PER_CHAR = 0.25): github 30,128 (47 tools), playwright 4,271 (23), context7 1,152 (2), sequential-thinking 1,126 (1), memory 604 (5 via cm bridge). GitHub dominates; sync-time mcp.\<id>.enabled:false trim is therefore essential for local-model windows, not optional hygiene.