PydanticAI | Doubleword Inference API

PydanticAI supports custom OpenAI-compatible endpoints via its OpenAIProvider.

Install

pip install pydantic-ai

Configure

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "{{selectedModel.id}}",
    provider=OpenAIProvider(
        base_url="https://api.doubleword.ai/v1",
        api_key="{{apiKey}}",
    ),
)

agent = Agent(model)
result = agent.run_sync("Say hello.")
print(result.output)

Prompt caching

PydanticAI has a native CachePoint and so in order to use thie effectively, we use a small subclass to send it in the shape Doubleword expects:

from pydantic_ai import Agent
from pydantic_ai.messages import CachePoint
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider


class DoublewordModel(OpenAIChatModel):
    async def _map_user_prompt_content_item(self, item, content):
        if isinstance(item, CachePoint):
            content[-1]["cache_control"] = {"type": "ephemeral", "ttl": item.ttl}
        else:
            await super()._map_user_prompt_content_item(item, content)


agent = Agent(
    DoublewordModel(
        "{{selectedModel.id}}",
        provider=OpenAIProvider(
            base_url="https://api.doubleword.ai/v1",
            api_key="{{apiKey}}",
        ),
    )
)

result = agent.run_sync([HANDBOOK, CachePoint(ttl="1h"), "What is 2 + 2?"])
print(result.usage.cache_read_tokens)

CachePoint marks the previous block that comes before it. It cannot come first. Note that HANDBOOK needs to be greater than the 1024-token floor, else a cache will not be written. The stock OpenAIChatModel sends OpenAI's prompt_cache_breakpoint instead, which Doubleword ignores, so you will need to use the pattern above if you want to enabled caching with Pydantic. See the prompt caching guide for more information.

Batch pricing with Autobatcher

For background tasks where latency is not critical, use Autobatcher to transparently route requests through the Batch API at reduced cost:

pip install pydantic-ai autobatcher

from autobatcher import BatchOpenAI
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

client = BatchOpenAI(
    api_key="{{apiKey}}",
    base_url="https://api.doubleword.ai/v1",
)

model = OpenAIChatModel(
    "{{selectedModel.id}}",
    provider=OpenAIProvider(openai_client=client),
)

agent = Agent(model)
result = agent.run_sync("Say hello.")
print(result.output)

BatchOpenAI collects requests and submits them as batch jobs automatically, cutting inference costs by up to 90%. Your code stays the same, only the client changes.