Kimi K3 on Amazon Bedrock: The Engineering Guide

Kimi K3 on Amazon Bedrock went GA on September 18, and it is not just another model tile in the catalog. Moonshot AI’s open-weight flagship brings 2.8 trillion parameters, a 1M-token context window, native vision, and, the part that matters most to your bill, explicit prompt caching. It is the first open-weight model on Bedrock that lets you mark exactly which part of the prompt gets cached. This guide covers what you need to wire it into real code: model IDs, routing, context and image inputs, cache mechanics, the two API surfaces, streaming, and logging.

The official announcements live on the AWS Machine Learning Blog and the What’s New post. Everything below assumes you already have Bedrock model access sorted in your account.

Kimi K3 on Amazon Bedrock: model IDs and routing

There is no single-Region, on-demand model ID to memorize here. You get two cross-Region inference profiles:

• global.moonshotai.kimi-k3 routes requests across AWS commercial Regions worldwide. AWS prices it roughly 10% cheaper than the geographic profile.
• us.moonshotai.kimi-k3 keeps routing inside US Regions. You pay a bit more for the geographic fence.

Pick based on data residency, not vibes. If your compliance team has opinions about where inference happens, use the us. profile and move on. If they don’t, the global profile is cheaper and generally has more capacity to absorb spikes. The mechanics of both are covered in the cross-Region inference docs, including the IAM wrinkle: your policy has to allow bedrock:InvokeModel on the inference profile ARN and on the underlying foundation model ARNs in the destination Regions. Forget the second half and you’ll get an AccessDenied that looks like it came from a different universe.

On the data side, AWS says prompts and outputs stay within the AWS boundary, are not shared with Moonshot, and are not used for training, with zero data retention and zero operator access by default. That is the same posture as other third-party models on Bedrock, and it is the reason this model is usable in shops that would never paste code into a random hosted endpoint.

1M tokens of context and native vision

A million tokens is enough to fit a mid-sized repository, a stack of contracts, or a painfully long incident timeline in one request. Moonshot pitches K3 for long-running coding sessions, repo analysis, and document processing, and says it gets about 2.5x better scaling efficiency than Kimi K2. Treat that as a vendor claim until your own evals agree.

Vision is native, so images go in as regular content blocks alongside text. With the Converse API, that looks like this:

import boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")

with open("architecture.png", "rb") as f:
    img = f.read()

resp = brt.converse(
    modelId="us.moonshotai.kimi-k3",
    messages=[{
        "role": "user",
        "content": [
            {"image": {"format": "png", "source": {"bytes": img}}},
            {"text": "List every single point of failure in this diagram."},
        ],
    }],
    inferenceConfig={"maxTokens": 2048, "temperature": 0.2},
)

print(resp["output"]["message"]["content"][0]["text"])
print(resp["usage"])

One warning before you dump your whole monorepo in there: a 1M-token window is a capacity, not a target. Every token you send is a token you pay for and a token the model has to attend over. That brings us to caching.

Explicit prompt caching with prompt_cache_breakpoint

Most open-weight models on Bedrock either don’t cache at all or do it implicitly, which means you get whatever the serving layer decides. Kimi K3 lets you place a prompt_cache_breakpoint marker after a reusable prefix. Everything before the marker is eligible for caching; everything after is processed fresh. The prefix has to be at least 1,024 tokens, so don’t bother caching a two-line system prompt.

In Converse-style JSON, the marker sits as its own content block right after the stable content. Check the Bedrock prompt caching docs and the model card for the exact placement rules, but the shape looks like this:

system = [
    {"text": open("repo_dump.txt").read()},
    {"prompt_cache_breakpoint": {}},
]

messages = [{
    "role": "user",
    "content": [{"text": "Where is the retry logic for the payments client?"}],
}]

resp = brt.converse(
    modelId="us.moonshotai.kimi-k3",
    system=system,
    messages=messages,
)

u = resp["usage"]
print(u.get("cacheWriteInputTokens"), u.get("cacheReadInputTokens"), u["inputTokens"])

The first call writes the cache, and you’ll see a write count in the usage block. Subsequent calls with a byte-identical prefix read from it. The golden rule: anything that changes goes after the breakpoint. A timestamp, a request ID, or a reordered file list before the marker silently turns every call into a cache write. Cache reads and writes are priced differently from normal input tokens, so check the Bedrock pricing page for the current K3 rates before you model anything.

A practical pattern: put the slow-moving material first (system instructions, tool definitions, the repo snapshot), then the breakpoint, then the conversation. When the repo changes, you pay one cache write and every follow-up question rides the cached prefix again. If your usage numbers show writes on every call, diff two consecutive prefixes byte by byte. The culprit is usually a dictionary serialized in a different key order.

Converse vs the OpenAI-compatible endpoint

K3 is reachable through four doors: Converse, InvokeModel, and OpenAI-compatible Chat Completions and Responses. Here’s how to choose:

• Converse is the default for new AWS-native code. One request shape across models, first-class tool use, and Guardrails integration.
• InvokeModel is the raw, model-native body. Use it only if you need a provider-specific field that Converse doesn’t expose.
• OpenAI-compatible endpoints at https://bedrock-runtime.<region>.amazonaws.com/openai/v1 are for existing OpenAI SDK code and agent tools that only speak that dialect. Swap the base URL, use a Bedrock API key, set the model to the inference profile ID, done.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://bedrock-runtime.us-east-1.amazonaws.com/openai/v1",
    api_key=os.environ["AWS_BEARER_TOKEN_BEDROCK"],
)

r = client.chat.completions.create(
    model="us.moonshotai.kimi-k3",
    messages=[{"role": "user", "content": "Summarize this stack trace."}],
)

That compatibility is why integrations like OpenCode and Hermes Agent can point at K3 with a config change instead of a rewrite.

Streaming for long outputs

Long coding sessions produce long answers, and nobody wants to stare at a spinner for a full diff. Use converse_stream and print deltas as they land. The final metadata event carries the same usage block, including cache counters.

stream = brt.converse_stream(
    modelId="us.moonshotai.kimi-k3",
    system=system,
    messages=messages,
)

for event in stream["stream"]:
    if "contentBlockDelta" in event:
        print(event["contentBlockDelta"]["delta"].get("text", ""), end="")
    elif "metadata" in event:
        print("\n", event["metadata"]["usage"])

Turn on invocation logging before production

If you only do one ops task this week, do this. Model invocation logging ships request and response metadata (and optionally full payloads) to CloudWatch Logs or S3. It’s account-level and per-Region, so enable it in every Region your inference profile can route through if you want complete coverage.

aws bedrock put-model-invocation-logging-configuration \
  --region us-east-1 \
  --logging-config '{
    "cloudWatchConfig": {
      "logGroupName": "/bedrock/invocations",
      "roleArn": "arn:aws:iam::123456789012:role/BedrockLogging"
    },
    "textDataDeliveryEnabled": true,
    "imageDataDeliveryEnabled": false
  }'

With logs flowing, you can query cache read versus write tokens per caller and catch the one service that keeps busting the cache with a timestamp in its system prompt. There is always one.

The short version

Use the us. profile if residency matters and global. if it doesn’t. Keep a stable prefix of 1,024+ tokens behind a prompt_cache_breakpoint, put everything volatile after it, and watch the cache counters. Default to Converse for new code, use the OpenAI endpoint for tools that already speak it, stream anything long, and turn on logging before your first real user shows up. The Amazon Bedrock documentation has the rest.

At Enkompass (enkompass.net), this is our home turf. We’ve shipped production Bedrock and SageMaker workloads that actually move the needle, not just demo well. If you’d rather have seasoned cloud people handle it instead of learning the hard way at 2 AM, let’s talk. Reach out through our contact page or call us directly at +1-828-436-5667 — we’ll help you turn this from a someday-project into a done-deal.