[do not merge] feat: Span streaming & new span API

[sentrivana]

Introduce a new start_span() API with a simpler and more intuitive signature to eventually replace the original start_span() and start_transaction() APIs.

Additionally, introduce a new streaming mode (sentry_sdk.init(_experiments={"trace_lifecycle": "stream"})) that will send spans as they finish, rather than by transaction.

import sentry_sdk

sentry_sdk.init(
    _experiments={"trace_lifecycle": "stream"},
)

with sentry_sdk.traces.start_span(name="my_span"):
    ...

The new API MUST be used with the new streaming mode, and the old API MUST be used in the legacy non-streaming (static) mode.

Migration guide: getsentry/sentry-docs#16072

Notes

• The diff is huge mostly because I've optimized for easy removal of legacy code in the next major, deliberately duplicating a lot. I'll of course split it up to reviewable PRs once ready.
  • Chose to go with a new file and a new span class so that we can just remove the old Span and drop the new StreamedSpan in tracing.py as a replacement.
• The batcher for spans is a bit different from the logs and metrics batchers because it needs to batch by trace_id (we can't send spans from different traces in the same envelope).

Release Plan

• There will be prereleases for internal testing.
• We'll release the new API in a minor version as opt-in.
• In the next major, we'll drop the legacy API.

Project

https://linear.app/getsentry/project/span-first-sdk-python-727da28dd037/overview

[github-actions]

Semver Impact of This PR

⚪ None (no version bump detected)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).

---

New Features ✨

Integrations

• feat(integrations): OpenAI/OpenAI Agents detect and report the time to first token metric (TTFT) as gen_ai.response.time_to_first_token by constantinius in #5348
• feat(integrations): openai-agents streaming support by constantinius in #5291

Openai

• feat(openai): Set system instruction attribute for Responses API by alexander-alderman-webb in #5376
• feat(openai): Set system instruction attribute for Completions API by alexander-alderman-webb in #5358

Openai Agents

• feat(openai-agents): Set system instruction attribute on gen_ai.chat spans by alexander-alderman-webb in #5370
• feat(openai-agents): Set system instruction attribute by alexander-alderman-webb in #5355

Other

• feat(anthropic): Set system instruction attribute by alexander-alderman-webb in #5353
• feat(asyncio): Allow to turn task spans off by sentrivana in #5367
• feat(gen_ai): add function set_conversation_id and managing functions on the Scope and apply it on the Span on .finish() by constantinius in #5362
• feat(google-genai): Set system instruction attribute by alexander-alderman-webb in #5354
• feat(langchain): Set system instruction attribute by alexander-alderman-webb in #5357
• feat(pydantic-ai): Set system instruction attribute by alexander-alderman-webb in #5356
• feat(transport): Report 413 responses for oversized envelopes by alexander-alderman-webb in #5380

Bug Fixes 🐛

• fix(ai): Keep single content input message by alexander-alderman-webb in #5345

Internal Changes 🔧

• ci(release): Fix changelog-preview permissions by BYK in #5368
• ref: Replace set_data_normalized() with Span.set_data() for system instructions by alexander-alderman-webb in #5374
• ci: Fix path in AI integration tests by alexander-alderman-webb in #5347

Other

• [do not merge] feat: Span streaming & new span API by sentrivana in #5317

---

_{🤖 This preview updates automatically when you update the PR.}