Skip to content

Prefer Ark x-request-id for chat trace response IDs - #1230

Open
xgtcode wants to merge 4 commits into
mainfrom
feat/d9fca78d-9zex
Open

xgtcode wants to merge 4 commits into
mainfrom
feat/d9fca78d-9zex

Conversation

@xgtcode

@xgtcode xgtcode commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Preserve provider response IDs on LLM trace spans across LiteLLM, Ark Responses, Gemini, streaming, retry, and fallback paths.
  • For Ark OpenAI-compatible chat completions, prefer the response header x-request-id over the SSE/body id when recording gen_ai.response.id.
  • Keep Ark Responses API behavior unchanged: response.id from /responses events is still recorded, including resp_... IDs.
  • Add regression coverage for streaming/non-streaming response ID provenance, missing/generated IDs, cancellation, retry, and Ark header precedence.

Why

The Runtime path using Agent(model_name=...) goes through LiteLLM chat completions rather than Ark Responses API. Ark chat completions may expose the traceable provider identifier in the response header x-request-id, while the SSE/body id can use a different hex-like format.

APMPlus comparison showed:

  • PyPI latest baseline did not record gen_ai.response.id.
  • The fixed Runtime records gen_ai.response.id on call_llm / model_request.
  • For Ark chat completions, the desired correlation value should come from x-request-id.

Verification

  • uv run --extra dev pre-commit run --all-files
    • Passed
  • uv run --extra dev pytest tests/models/test_model_response_id.py -q
    • Passed: 86 passed
  • uv run --extra dev pytest tests/test_adk_tracing_integration.py tests/test_model_tpot.py -q
    • Passed: 82 passed

Live E2E

Deployed real AgentKit Runtimes and invoked the streaming runner.run_async(..., StreamingMode.SSE) path.

  • Baseline PyPI Runtime: veadk-python==1.1.28
    • Real request succeeded
    • APMPlus trace had no gen_ai.response.id
  • Fixed wheel Runtime: 1.1.27.dev25+g4afd3be59
    • Real request succeeded
    • APMPlus trace recorded gen_ai.response.id on call_llm and model_request

Also verified Ark /api/v3/responses directly returns resp_... IDs in response.id, while the current Agent(model_name=...) Runtime path uses chat completions and returns the hex-like body id unless Ark x-request-id is preferred.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant