Gemini Context Caching Guide

This guide explains how to create and reuse Gemini context caches (Context Cache) through the OpenAI-compatible Chat Completions API or the native Gemini API. Before you begin:

export API_KEY="YOUR_API_KEY"

The examples in this guide use gemini-3.6-flash. Refer to the platform’s model documentation and pricing page to confirm whether other models support Context Cache.

Use cases

When multiple requests repeatedly include the same large body of content, you can cache the stable prefix. Examples include:

A very long system prompt
A fixed knowledge base or product documentation
Stable conversation history in a multi-turn conversation
Reused tool definitions and instructions

Context Cache is ideal for requests where the content at the beginning remains unchanged while the final question changes.

Core usage

Add cache_control to the content block in the last message of the stable prefix:

{
  "type": "text",
  "text": "This is the final section of the stable prefix",
  "cache_control": {
    "type": "ephemeral",
    "ttl": "5m"
  }
}

Supported TTL values:

TTL	Meaning
`5m`	Cache for 5 minutes
`1h`	Cache for 1 hour

If ttl is omitted, it defaults to 5m.

Message structure

We recommend the following structure:

system
→ Stable long-form text or message history
→ Stable prefix boundary with cache_control
→ Current user question (not cached)

The message containing cache_control and all messages before it form the cached prefix. At least one real-time message must follow it.

OpenAI-compatible request example

curl "https://api.apimart.ai/v1/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-3.6-flash",
    "stream": false,
    "messages": [
      {
        "role": "system",
        "content": "Answer questions strictly based on the provided reference material."
      },
      {
        "role": "user",
        "content": "Place the long reference material you want to reuse here..."
      },
      {
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "I have read and understood the reference material above.",
            "cache_control": {
              "type": "ephemeral",
              "ttl": "5m"
            }
          }
        ]
      },
      {
        "role": "user",
        "content": "Summarize the three main points in the reference material."
      }
    ]
  }'

On the first request, the system attempts to create a cache and uses the new cache to complete that same request.

You do not need to call a separate cache creation endpoint. cache_control specifies both the cache boundary and the cache lifetime.

Native Gemini request example

The native Gemini generateContent endpoint also supports adding cache_control to contents[].parts[]:

curl "https://api.apimart.ai/v1beta/models/gemini-3.6-flash:generateContent" \
  -H "x-goog-api-key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "systemInstruction": {
      "parts": [
        {
          "text": "Answer questions strictly based on the provided reference material."
        }
      ]
    },
    "contents": [
      {
        "role": "user",
        "parts": [
          {
            "text": "Place the long reference material you want to reuse here..."
          }
        ]
      },
      {
        "role": "model",
        "parts": [
          {
            "text": "I have read and understood the reference material above.",
            "cache_control": {
              "type": "ephemeral",
              "ttl": "5m"
            }
          }
        ]
      },
      {
        "role": "user",
        "parts": [
          {
            "text": "Summarize the three main points in the reference material."
          }
        ]
      }
    ]
  }'

cache_control is a platform extension to the Gemini request format. After identifying the boundary, the platform removes this field before forwarding the request and automatically creates or reuses cached content (cachedContent). The streaming endpoint uses the same request body. You only need to change the URL to:

curl -N "https://api.apimart.ai/v1beta/models/gemini-3.6-flash:streamGenerateContent?alt=sse" \
  -H "x-goog-api-key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "contents": [
      {
        "role": "user",
        "parts": [
          {
            "text": "Place the long reference material you want to reuse here...",
            "cache_control": {
              "type": "ephemeral",
              "ttl": "5m"
            }
          }
        ]
      },
      {
        "role": "user",
        "parts": [
          {
            "text": "Summarize the three main points in the reference material."
          }
        ]
      }
    ]
  }'

When reusing the cache, keep systemInstruction, the contents before the boundary, the TTL, and tools unchanged. Only modify the real-time content after the boundary.

Creation and reuse flow

When you send a request with cache_control for the first time:

Identify the stable prefix
→ Create Context Cache
→ Reference the new cache in the current request
→ Return the model response

When you send the same stable prefix again:

Identify the same stable prefix
→ Reuse the unexpired Context Cache
→ Send only the current real-time content
→ Return the model response

As a result, the first request may already report a large number of cache-hit tokens. This is expected and does not require a separate warm-up request.

Reusing a cache

For subsequent requests, keep the following unchanged:

The model
All messages before cache_control
cache_control.ttl
Tool definitions (if you use tools)
systemInstruction in native Gemini requests

Only modify the real-time question after the boundary:

{
  "role": "user",
  "content": "What risks are mentioned in the reference material?"
}

The system reuses the existing cache as long as the stable prefix is identical and the cache has not expired. The following changes create a different cache:

Changing text or message order within the stable prefix
Changing the model
Changing 5m to 1h
Changing tools or tool parameter definitions
Using a different API user or channel

Python example

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apimart.ai/v1",
)

stable_messages = [
    {
        "role": "system",
        "content": "Answer questions strictly based on the provided reference material.",
    },
    {
        "role": "user",
        "content": "Place the long reference material you want to reuse here...",
    },
    {
        "role": "assistant",
        "content": [
            {
                "type": "text",
                "text": "I have read and understood the reference material above.",
                "cache_control": {
                    "type": "ephemeral",
                    "ttl": "5m",
                },
            }
        ],
    },
]

response = client.chat.completions.create(
    model="gemini-3.6-flash",
    messages=[
        *stable_messages,
        {
            "role": "user",
            "content": "Summarize the three main points in the reference material.",
        },
    ],
)

print(response.choices[0].message.content)
print(response.usage)

For subsequent requests, reuse the same stable_messages and replace only the final user message.

Checking for a cache hit

OpenAI-compatible response

Check the following fields in the response:

{
  "usage": {
    "prompt_tokens": 14430,
    "prompt_tokens_details": {
      "cached_tokens": 14420,
      "cache_write_tokens": 0
    }
  }
}

Field descriptions:

Field	Meaning
`prompt_tokens`	All input tokens for this request
`cached_tokens`	Input tokens read from the cache for this request
`cache_write_tokens`	Cache-write tokens; a value of `0` is expected

The first request may also report a large cached_tokens value because the system can create a cache and reference it within the same model call.

Native Gemini response

Check usageMetadata.cachedContentTokenCount in the response:

{
  "usageMetadata": {
    "promptTokenCount": 13926,
    "cachedContentTokenCount": 13916,
    "totalTokenCount": 13954
  }
}

Field descriptions:

Field	Meaning
`promptTokenCount`	All input tokens for this request
`cachedContentTokenCount`	Input tokens read from the cache for this request
`totalTokenCount`	Total input and output tokens for this request

streamGenerateContent returns the same usageMetadata in an SSE response frame. The client should read the frame containing this field rather than checking only the first text frame.

Recommendations

Cache only long content that is truly stable and will be reused multiple times.
Place the question that changes with each request after the cache_control boundary.
Do not include timestamps, random IDs, or dynamic user information in the stable prefix.
Use 5m when you expect repeated calls within a short period.
Use 1h when you need a longer reuse window.
If the prefix is too short, the model does not support caching, or the cache is temporarily unavailable, the request may automatically run in standard mode.
For native Gemini requests, the cache boundary must be placed in contents[].parts[], not in systemInstruction.

Frequently asked questions

Can the native Gemini request format create a cache automatically?

Yes. generateContent and streamGenerateContent use the same cache_control structure. The boundary must be placed in contents[].parts[], and at least one real-time content item must remain after the content containing the boundary.If the request explicitly provides a native cachedContent resource name, the platform prioritizes the user-provided resource and does not create a cache automatically.

Why was there no cache hit?

Common reasons include:

The stable prefix does not exactly match the previous request
The TTL has expired
The model or tools were changed
The cached content does not meet the model’s minimum token requirement
cache_control was placed on the final message, leaving no real-time question after it

Can cache_control be placed on the final message?

This is not recommended. The final message is usually the current real-time question and should not be cached. If no real-time message follows the boundary, the request runs in standard mode.

Can I set a different TTL?

No. Currently, only 5m and 1h are supported. Any other value returns HTTP 400.

Can I set multiple cache boundaries?

Yes, but all boundaries must use the same TTL, and the system uses the final boundary. In general, using only one boundary per request is recommended for a clearer structure.

Will a request fail if the cache is unavailable?

Usually not. If the conditions for creating or reusing a cache are not met, the system automatically sends a standard request. Parameter errors such as an invalid TTL or mixed TTL values are exceptions.

What happens if Context Cache is not enabled for the model?

The request automatically runs in standard mode without creating an explicit cache or incurring cache storage fees. Standard input, output, and any available implicit caching continue to be billed according to the model’s existing rules.

Why is cache_write_tokens 0?

This is expected behavior for Gemini context caching. Cache creation costs are recorded as separate cache storage fees rather than using OpenAI/Claude-style cache_write_tokens to represent the amount written to the cache.

Does cached_tokens greater than 0 always mean an explicit cache was created?

Not necessarily. The system may also produce implicit cache hits. For regular users, cache-read tokens indicate whether the current request benefited from cache reads. To verify explicit cache creation fees, check the Context Cache storage entries in the platform’s usage logs.

How is caching billed?

Creating a cache may incur a one-time cache storage fee. When the cache is used, hit tokens are billed at the cache-read price. Refer to the model pricing displayed on the platform for exact rates.

Overview

Text Series

Image Series

Video Series

Audio Series

Moderation Series

Upload Management

Task Management

Account Management

Gemini Context Caching Guide

Use cases

Core usage

Message structure

OpenAI-compatible request example

Native Gemini request example

Creation and reuse flow

Reusing a cache

Python example

Checking for a cache hit

OpenAI-compatible response

Native Gemini response

Recommendations

Frequently asked questions

​Use cases

​Core usage

​Message structure

​OpenAI-compatible request example

​Native Gemini request example

​Creation and reuse flow

​Reusing a cache

​Python example

​Checking for a cache hit

​OpenAI-compatible response

​Native Gemini response

​Recommendations

​Frequently asked questions

Use cases

Core usage

Message structure

OpenAI-compatible request example

Native Gemini request example

Creation and reuse flow

Reusing a cache

Python example

Checking for a cache hit

OpenAI-compatible response

Native Gemini response

Recommendations

Frequently asked questions