Repository navigation
Conversation
The effective cost used fixed cache factors (OpenAI cache reads at 0.5x input, Anthropic reads at 0.1x and writes at 1.25x). gpt-5 family cache reads are 0.1x and claude-opus-5-5 / claude-sonnet-5-5 reads are 0.05x, so cached agent steps were mispriced. The Anthropic path also read usage.cache_input_tokens, which the SDK does not set, so cache reads were priced at $0. OpenAI and Anthropic models are now priced from the llm-prices table (each model's own input, output, cache read and 5m/1h cache write prices, long-context tiers included). Models without a row keep the previous pricing path and the fixed factors.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The effective cost that study summaries report as cum_effective_cost uses fixed cache factors: OpenAI cache reads at 0.5x input, but the gpt-5 family (including the GPT_5_mini and GPT_5_nano agents in tool_use_agent.py) bills them at 0.1x, so a gpt-5 call with 100k prompt tokens, 80k of them cached, and 2k output is recorded as $0.095 instead of $0.055. On the Anthropic side the code reads usage.cache_input_tokens, which the SDK never sets (the field is cache_read_input_tokens), so cache reads are priced at $0, and the fixed 0.1x read factor would be 2x too high for claude-opus-5-5 and claude-sonnet-5-5 anyway (their reads are 0.05x). This PR prices the openai and anthropic paths with llm-prices, a dated, source-cited price table with each model's cache read and 5m/1h cache write prices and long-context tiers, and passes the real cached and cache-write counts through Price.cost_usage; models without a row (claude-3.x, fine-tunes, self-hosted base_url models) keep the previous pricing path and factors, and the openrouter and litellm paths are untouched. It is a regular runtime dependency (pure Python, no dependencies of its own, no network access) because prices change and a copied table goes stale; I maintain it (MIT). tests/llm passes with 5 new cases checked against the providers' published prices, a two-step ToolUseAgent run (gpt-5-mini, mocked client) and a claude-opus-5-5 step record effective costs that match litellm's cost_per_token and Anthropic's price list exactly, and the rest of the suite gives the same results as main.