PydanticAI | Doubleword Inference API

PydanticAI supports custom OpenAI-compatible endpoints via its OpenAIProvider.

Install

pip install pydantic-ai

Configure

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "{{selectedModel.id}}",
    provider=OpenAIProvider(
        base_url="https://api.doubleword.ai/v1",
        api_key="{{apiKey}}",
    ),
)

agent = Agent(model)
result = agent.run_sync("Say hello.")
print(result.output)

Prompt caching

PydanticAI has a native CachePoint and so in order to use thie effectively, we use a small subclass to send it in the shape Doubleword expects:

from pydantic_ai import Agent
from pydantic_ai.messages import CachePoint
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider


class DoublewordModel(OpenAIChatModel):
    async def _map_user_prompt_content_item(self, item, content):
        if isinstance(item, CachePoint):
            content[-1]["cache_control"] = {"type": "ephemeral", "ttl": item.ttl}
        else:
            await super()._map_user_prompt_content_item(item, content)


agent = Agent(
    DoublewordModel(
        "{{selectedModel.id}}",
        provider=OpenAIProvider(
            base_url="https://api.doubleword.ai/v1",
            api_key="{{apiKey}}",
        ),
    )
)

result = agent.run_sync([HANDBOOK, CachePoint(ttl="1h"), "What is 2 + 2?"])
print(result.usage.cache_read_tokens)

CachePoint marks the previous block that comes before it. It cannot come first. Note that HANDBOOK needs to be greater than the 1024-token floor, else a cache will not be written. The stock OpenAIChatModel sends OpenAI's prompt_cache_breakpoint instead, which Doubleword ignores, so you will need to use the pattern above if you want to enabled caching with Pydantic. See the prompt caching guide for more information.

Async pricing with Autobatcher

For background tasks where latency is not critical, use Autobatcher to run requests on Doubleword's cheaper async (flex) tier:

pip install pydantic-ai autobatcher

from autobatcher import BatchOpenAI
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

client = BatchOpenAI(
    api_key="{{apiKey}}",
    base_url="https://api.doubleword.ai/v1",
)

model = OpenAIChatModel(
    "{{selectedModel.id}}",
    provider=OpenAIProvider(openai_client=client),
)

agent = Agent(model)
result = agent.run_sync("Say hello.")
print(result.output)

By default BatchOpenAI sends each call as an async (flex) request and polls for the result. Your code stays the same, only the client changes. The framework adds per-request headers, which the Batch API cannot carry, so completion_window="24h" fails here; use the dw CLI for 24h batch jobs.