CrewAI

CrewAI supports custom OpenAI-compatible endpoints out of the box.

Install

pip install crewai litellm

Depending on your environment, litellm may not be pulled in automatically, so install it explicitly to be safe.

Configure

from crewai import Agent, Task, Crew, LLM

llm = LLM(
    model="openai/{{selectedModel.id}}",
    base_url="https://api.doubleword.ai/v1",
    api_key="{{apiKey}}",
)

agent = Agent(
    role="Assistant",
    goal="Answer questions",
    backstory="A helpful assistant.",
    llm=llm,
)

task = Task(
    description="Say hello.",
    expected_output="A greeting.",
    agent=agent,
)

crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
print(result)

The model string must be prefixed with openai/ so CrewAI routes it through its OpenAI-compatible provider. The part after the slash should match a model name configured in your Control Layer.

Prompt caching

Register a before_llm_call hook that rewrites the agent's system message into a cached content block. Add it above the Crew in the sample.

from crewai.hooks import before_llm_call


@before_llm_call
def cache_system_prompt(context):
    first = context.messages[0]
    if first["role"] == "system" and isinstance(first["content"], str):
        first["content"] = [
            {
                "type": "text",
                "text": first["content"],
                "cache_control": {"type": "ephemeral", "ttl": "5m"},
            }
        ]

The cached prefix is the agent's role, goal and backstory. Put the stable bulk of your prompt in backstory so it stays byte identical between runs. Short system prompts fall under the minimum cacheable length and the directive is ignored. The second run reports cache_read_input_tokens in its usage.

CrewAI also ships a mark_cache_breakpoint helper. Only the Anthropic provider turns that flag into a real cache_control block, so it has no effect against this endpoint.

See the prompt caching guide.