Repo-scale agent workflows are exactly what Kimi K3 on Amazon Bedrock was built for: a 1M-token context window, explicit prompt caching, native tool use through Converse, and a launch post that literally names long-running coding and repository analysis as target use cases. They are also the fastest way to set a pile of money on fire if you architect them like a chatbot. This post is about the design decisions, not the API basics: how to lay out prompts so the cache actually hits, when to stuff the whole repo versus chunk it, how to keep routing inside your residency rules, where Guardrails belong, and how to stop a runaway agent before finance does.
If you need the fundamentals first (model IDs, the OpenAI-compatible endpoint, streaming), start with the AWS launch post and the What’s New entry.
Designing repo-scale agent workflows around the cache
K3 lets you place a prompt_cache_breakpoint after a reusable prefix of at least 1,024 tokens. That one fact should drive your entire prompt layout. Think of the prompt as layers ordered from least to most volatile:
- Layer 1, frozen: agent instructions, coding conventions, tool definitions. Changes on deploy.
- Layer 2, slow: the repo snapshot or its relevant slice, pinned to a commit SHA. Changes on push.
- Breakpoint.
- Layer 3, volatile: the task, the running conversation, tool results. Changes every turn.
The rules that fall out of this are boring and non-negotiable. Serialize the repo deterministically (sorted paths, stable separators, no mtime headers). Never inject timestamps, request IDs, or user names above the breakpoint. Keep tool definitions in a fixed order, because a reordered JSON schema is a different prefix. And key the cache on the commit: one repo snapshot per SHA means every agent working on that commit shares warm prefixes instead of each paying its own cache write. The Bedrock prompt caching docs cover exact placement and cache lifetime; design for the assumption that an idle cache eventually expires.
Instrument the hit rate from day one. Every Converse response returns cache read and write counters in its usage block. Emit them as metrics per workflow. If writes exceed reads over an hour, something upstream of the breakpoint is churning, and you want an alarm, not a surprised FinOps review.
Whole repo or chunks at 1M tokens?
A million tokens tempts you to skip retrieval entirely. Sometimes that’s right. Here’s the rule of thumb we use:
- Whole repo in context when the codebase fits comfortably, the agent does cross-cutting work (refactors, dependency upgrades, security sweeps), and the same snapshot serves many turns. The cache amortizes the big prefix.
- Chunked or retrieved context when the repo is larger than the window, when tasks are narrow (“fix this failing test”), or when each task hits a different area so the prefix rarely gets reused.
- Hybrid for monorepos: a cached map of the repo (tree, module summaries, interfaces) as the prefix, with the agent pulling full files through a
read_filetool on demand. This is usually the winner.
Remember that a cache hit makes a big prefix cheaper, not free, and attention over a million tokens still affects latency. Bigger is not automatically smarter. Moonshot’s claimed 2.5x scaling efficiency over Kimi K2 is nice; your eval suite is the judge.
Routing and residency with cross-Region inference
K3 is available through two cross-Region inference profiles: global.moonshotai.kimi-k3, roughly 10% cheaper, and us.moonshotai.kimi-k3, which keeps processing in US Regions. For an agent platform, don’t leave the choice to each team’s config file. Make residency a platform control. In an organization with regulated accounts, a service control policy that denies the global profile is a clean guardrail:
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "DenyGlobalKimiInRegulatedOUs",
"Effect": "Deny",
"Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
"Resource": "arn:aws:bedrock:*:*:inference-profile/global.moonshotai.kimi-k3"
}]
}Attach it to the OUs that hold regulated workloads and leave internal-tooling OUs free to take the discount. Data handling is the same on both profiles: per AWS, prompts and outputs stay within the AWS boundary, aren’t shared with Moonshot, aren’t used for training, and default to zero data retention and zero operator access.
The agent loop: tools, Guardrails, and a circuit breaker
The core loop is simple: call Converse with a toolConfig, execute whatever tool the model asks for, append the result, repeat until the model stops asking. The architecture lives in what wraps that loop. Two things belong there on day one.
Guardrails, scoped. Amazon Bedrock Guardrails attach to Converse through guardrailConfig. The trap: a guardrail evaluates input text, and guardrail usage is billed on the text it assesses. You do not want it re-scanning an 800K-token repo prefix on every turn. Wrap the untrusted user task in a guardContent block so the guardrail evaluates only that, and let output filtering do the rest.
A cost circuit breaker. Agents loop. Sometimes they loop forever. Put a hard budget on turns and tokens per task, enforced in code, and trip it before the model decides to read the entire node_modules folder one file at a time.
MAX_TURNS = 25
MAX_INPUT_TOKENS = 3_000_000
def run_task(brt, system, tools, task, guardrail_id):
messages = [{"role": "user", "content": [{"guardContent": {"text": {"text": task}}}]}]
spent = 0
for turn in range(MAX_TURNS):
r = brt.converse(
modelId="us.moonshotai.kimi-k3",
system=system,
messages=messages,
toolConfig={"tools": tools},
guardrailConfig={"guardrailIdentifier": guardrail_id, "guardrailVersion": "1"},
inferenceConfig={"maxTokens": 4096},
)
u = r["usage"]
spent += u["inputTokens"] + u.get("cacheReadInputTokens", 0) + u.get("cacheWriteInputTokens", 0)
if spent > MAX_INPUT_TOKENS:
raise RuntimeError(f"budget tripped after {turn + 1} turns")
msg = r["output"]["message"]
messages.append(msg)
if r["stopReason"] != "tool_use":
return msg
results = []
for block in msg["content"]:
if "toolUse" in block:
tu = block["toolUse"]
out = dispatch(tu["name"], tu["input"])
results.append({"toolResult": {"toolUseId": tu["toolUseId"], "content": [{"text": out}]}})
messages.append({"role": "user", "content": results})
raise RuntimeError("turn limit reached")A few design notes on that loop. The system list carries the cached layers and the breakpoint, so every turn reuses the prefix; only the growing message history is fresh input. Tool outputs should be truncated before they go back to the model, since a single grep across a monorepo can return more text than the rest of the conversation combined. And check the stopReason for guardrail_intervened so a blocked request ends cleanly instead of looking like a crash.
Backstop the in-code budget with account-level signals: CloudWatch alarms on Bedrock token metrics per model, plus model invocation logging so you can trace which workflow blew the budget.
Plugging in existing agent tools
Not every team will run your custom loop. K3 is also reachable through Bedrock’s OpenAI-compatible Chat Completions and Responses APIs, which is how integrations like OpenCode and Hermes Agent connect. Treat those as first-class tenants: same IAM roles, same SCPs, same logging. The routing and budget rules above should live at the platform layer, so they apply whether a request comes from boto3 or from an OpenAI-shaped client. The Amazon Bedrock documentation covers the endpoint details.
At Enkompass (enkompass.net), this is our home turf. We’ve built production agent platforms on Bedrock and SageMaker that deliver real results, with the cost controls to prove it. If you’d rather have seasoned cloud people handle it instead of learning the hard way at 2 AM, let’s talk. Reach out through our contact page or call us directly at +1-828-436-5667 — we’ll help you turn this from a someday-project into a done-deal.