Integrate FutureAGI OpenTelemetry-compatible exporter #1090
Conversation
Signed-off-by: Sahil Nishad <[email protected]>
WalkthroughAdds support for FutureAGI telemetry export via a new telemetry exporter implementation and configuration example. Introduces a FutureAGITelemetryExporter class with OTLP adapter integration, API key validation, and configurable resource attributes. Includes a sample configuration demonstrating observability setup for a calculator workflow. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes The changes introduce a new exporter following established patterns in the codebase. While requiring attention to API key handling, credential environment variable fallback, and resource attribute merging logic, the overall scope is contained and involves straightforward configuration and initialization code without complex control-flow modifications. Pre-merge checks and finishing touches✅ Passed checks (5 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py (1)
198-208: Let me check how the resource_attributes are used in the FutureAGI exporter implementation to confirm whether theAnytype is actually leveraged:Based on my verification, I can now generate the rewritten review comment.
The OpenTelemetry Python specification documents that Resource attributes accept Union[str, bool, int, float, Sequence[str], Sequence[bool], Sequence[int], Sequence[float]], so non-string types are legitimately supported. However, the issue is one of code consistency within this file.
Align
resource_attributestype annotation with other exporters.The FutureAGI exporter uses
dict[str, Any]forresource_attributes(line 207), while all four other exporters in this file (Langfuse, Langsmith, OtelCollector, Patronus) consistently usedict[str, str]. For maintainability and consistency across the codebase, change to:- resource_attributes: dict[str, Any] = Field(default_factory=dict, + resource_attributes: dict[str, str] = Field(default_factory=dict,
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
examples/observability/simple_calculator_observability/configs/config-futureagi.yml(1 hunks)packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py(2 hunks)
🧰 Additional context used
📓 Path-based instructions (8)
**/configs/**
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Configuration files consumed by code should be stored next to that code in a configs/ folder
Files:
examples/observability/simple_calculator_observability/configs/config-futureagi.yml
{**/*.py,**/*.sh,**/*.md,**/*.toml,**/*.y?(a)ml,**/*.json,**/*.txt,**/*.ini,**/*.cfg,**/*.ipynb}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
{**/*.py,**/*.sh,**/*.md,**/*.toml,**/*.y?(a)ml,**/*.json,**/*.txt,**/*.ini,**/*.cfg,**/*.ipynb}: Every file must start with the standard SPDX Apache-2.0 header
Confirm copyright years are up to date when a file is changed
All source files must include the SPDX Apache-2.0 header template (copy from an existing file)
Files:
examples/observability/simple_calculator_observability/configs/config-futureagi.ymlpackages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
**/*
⚙️ CodeRabbit configuration file
**/*: # Code Review Instructions
- Ensure the code follows best practices and coding standards. - For Python code, follow
PEP 20 and
PEP 8 for style guidelines.- Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
Example:def my_function(param1: int, param2: str) -> bool: pass- For Python exception handling, ensure proper stack trace preservation:
- When re-raising exceptions: use bare
raisestatements to maintain the original stack trace,
and uselogger.error()(notlogger.exception()) to avoid duplicate stack trace output.- When catching and logging exceptions without re-raising: always use
logger.exception()
to capture the full stack trace information.Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any
words listed in the
ci/vale/styles/config/vocabularies/nat/reject.txtfile, words that might appear to be
spelling mistakes but are listed in theci/vale/styles/config/vocabularies/nat/accept.txtfile are OK.Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,
and should contain an Apache License 2.0 header comment at the top of each file.
- Confirm that copyright years are up-to date whenever a file is changed.
Files:
examples/observability/simple_calculator_observability/configs/config-futureagi.ymlpackages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
examples/**/*
⚙️ CodeRabbit configuration file
examples/**/*: - This directory contains example code and usage scenarios for the toolkit, at a minimum an example should
contain a README.md or file README.ipynb.
- If an example contains Python code, it should be placed in a subdirectory named
src/and should
contain apyproject.tomlfile. Optionally, it might also contain scripts in ascripts/directory.- If an example contains YAML files, they should be placed in a subdirectory named
configs/. - If an example contains sample data files, they should be placed in a subdirectory nameddata/, and should
be checked into git-lfs.
Files:
examples/observability/simple_calculator_observability/configs/config-futureagi.yml
**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
**/*.py: In code comments use the abbreviations: nat (API namespace/CLI), nvidia-nat (package), NAT (env var prefixes); never use these abbreviations in documentation
Follow PEP 20 and PEP 8 for Python style
Run yapf with column_limit=120; yapf is used for formatting (run second)
Indent with 4 spaces (no tabs) and end each file with a single trailing newline
Use ruff (ruff check --fix) as a linter (not formatter) per pyproject.toml; fix warnings unless explicitly ignored
Respect Python naming schemes: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
When re-raising exceptions, use bare raise to preserve stack trace; log with logger.error(), not logger.exception()
When catching and logging without re-raising, use logger.exception() to capture full stack trace
Provide Google-style docstrings for every public module, class, function, and CLI command
Docstring first line must be a concise description ending with a period
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work (HTTP, DB, file I/O)
Cache expensive computations with functools.lru_cache or an external cache when appropriate
Leverage NumPy vectorized operations when beneficial and feasible
Files:
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
{src/**/*.py,packages/*/src/**/*.py}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
{src/**/*.py,packages/*/src/**/*.py}: All importable Python code must live under src/ or packages//src/
All public APIs must have Python 3.11+ type hints on parameters and return values
Prefer typing/collections.abc abstractions (e.g., Sequence over list)
Use typing.Annotated for units or metadata when useful
Treat pyright warnings as errors during development
Files:
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
packages/*/src/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
If a package contains Python code, it must have tests in a tests/ directory at the same level as pyproject.toml
Files:
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
packages/**/*
⚙️ CodeRabbit configuration file
packages/**/*: - This directory contains optional plugin packages for the toolkit, each should contain apyproject.tomlfile. - Thepyproject.tomlfile should declare a dependency onnvidia-nator another package with a name starting
withnvidia-nat-. This dependency should be declared using~=<version>, and the version should be a two
digit version (ex:~=1.0).
- Not all packages contain Python code, if they do they should also contain their own set of tests, in a
tests/directory at the same level as thepyproject.tomlfile.
Files:
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
🧬 Code graph analysis (1)
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py (3)
tests/nat/data_models/test_common.py (2)
BatchConfigMixin(158-159)CollectorConfigMixin(161-162)src/nat/builder/builder.py (1)
Builder(68-290)packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/otlp_span_adapter_exporter.py (1)
OTLPSpanAdapterExporter(25-94)
🪛 Ruff (0.14.1)
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
212-212: Unused function argument: builder
(ARG001)
223-223: Avoid specifying long messages outside the exception class
(TRY003)
🔇 Additional comments (2)
examples/observability/simple_calculator_observability/configs/config-futureagi.yml (2)
1-14: LGTM!The SPDX license header is correctly formatted with up-to-date copyright years.
17-69: Configuration looks good.The FutureAGI telemetry configuration correctly:
- Uses environment variable substitution for API credentials (lines 32-33)
- Specifies all required fields (endpoint, project, keys, version)
- Follows the established YAML configuration pattern
The workflow and function definitions are properly structured.
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
Show resolved
Hide resolved
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
Show resolved
Hide resolved
| "metadata": json.dumps({}) | ||
| } | ||
|
|
||
| merged_resource_attributes = {**default_resource_attributes, **config.resource_attributes} |
There was a problem hiding this comment.
User-provided attributes can override critical fields.
The merge strategy at line 238 allows user-provided resource_attributes to override system-generated defaults like project_name, project_type, and project_version_id, which could break FutureAGI's tracing.
Reverse the merge order to give precedence to critical system attributes:
- merged_resource_attributes = {**default_resource_attributes, **config.resource_attributes}
+ # Merge user attributes first, then overlay critical system attributes
+ merged_resource_attributes = {**config.resource_attributes, **default_resource_attributes}Or more explicitly protect critical fields:
+ # Start with user attributes
+ merged_resource_attributes = dict(config.resource_attributes)
+
+ # Override with critical system attributes that must not be changed
+ merged_resource_attributes.update({
+ "project_name": config.project,
+ "project_type": config.project_type.lower(),
+ "project_version_name": config.project_version_name,
+ "project_version_id": project_version_id,
+ })
+
+ # Add optional attributes only if not already present
+ merged_resource_attributes.setdefault("eval_tags", json.dumps([]))
+ merged_resource_attributes.setdefault("metadata", json.dumps({}))
-
- merged_resource_attributes = {**default_resource_attributes, **config.resource_attributes}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| merged_resource_attributes = {**default_resource_attributes, **config.resource_attributes} | |
| # Merge user attributes first, then overlay critical system attributes | |
| merged_resource_attributes = {**config.resource_attributes, **default_resource_attributes} |
🤖 Prompt for AI Agents
In packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
around line 238, the current merge lets user-supplied config.resource_attributes
overwrite critical system defaults; change the merge so system defaults win
(e.g., merged_resource_attributes = {**config.resource_attributes,
**default_resource_attributes}) or explicitly remove/protect keys like
"project_name", "project_type", and "project_version_id" from
config.resource_attributes before merging so those critical fields cannot be
overridden by user input.
There was a problem hiding this comment.
This is a critical issue that needs addressed
There was a problem hiding this comment.
@willkill07, I agree this is critical. Allowing user-provided resource_attributes to override system-generated fields like project_name, project_type, and project_version_id could break FutureAGI's tracing functionality.
Here's the recommended fix that explicitly protects critical system attributes:
# Start with user-provided attributes
merged_resource_attributes = dict(config.resource_attributes)
# Override with critical system attributes that must not be changed
merged_resource_attributes.update({
"project_name": config.project,
"project_type": config.project_type.lower(),
"project_version_name": config.project_version_name,
"project_version_id": project_version_id,
})
# Add optional attributes only if not already present
merged_resource_attributes.setdefault("eval_tags", json.dumps([]))
merged_resource_attributes.setdefault("metadata", json.dumps({}))This approach:
- Starts with user attributes to preserve their custom fields
- Explicitly overwrites the 4 critical fields that FutureAGI requires
- Uses
setdefault()foreval_tagsandmetadatato allow user override while providing sensible defaults
This makes the protection of critical fields explicit and maintainable. The current line should be removed and replaced with the above code block.
willkill07
left a comment
willkill07
left a comment
There was a problem hiding this comment.
Changes request due to logic errors and missing configuration fields.
In addition to the added example, you should:
- update the Observability example README with an entry for FutureAGI
- add an entry to the table in
docs/source/extend/telemetry-exporters.md
It is also strongly recommended (though not required) to:
- add a new guide to
docs/source/workflows/observe/
| futureagi: | ||
| _type: futureagi | ||
| endpoint: https://api.futureagi.com/tracer/v1/traces | ||
| project: simple_calculator |
There was a problem hiding this comment.
project is not part of the futureagi configuration?
| class FutureAGITelemetryExporter(BatchConfigMixin, CollectorConfigMixin, TelemetryExporterBaseConfig, name="futureagi"): | ||
| """A telemetry exporter to transmit traces to Future AGI service.""" | ||
|
|
||
| endpoint: str = Field(description="The Future AGI endpoint to export telemetry traces", | ||
| default="https://api.futureagi.com/tracer/v1/traces") | ||
| fi_api_key: str = Field(description="The FutureAGI API key", default="") | ||
| fi_secret_key: str = Field(description="The FutureAGI Secret key", default="") | ||
| project_type: str = Field(description="The project type for FutureAGI instrumentation", default="observe") | ||
| project_version_name: str = Field(description="The project version name", default="1.0.0") | ||
| resource_attributes: dict[str, Any] = Field(default_factory=dict, | ||
| description="The resource attributes to add to the span") |
There was a problem hiding this comment.
project is not a field here. The config data model must be updated.
|
|
||
| headers = {"x-api-key": api_key, "x-secret-key": secret_key} | ||
|
|
||
| project_version_id = str(uuid.uuid4()) |
There was a problem hiding this comment.
You have project_version_id as a config member. Use it?
uuid should not be required
| project_version_id = str(uuid.uuid4()) | ||
|
|
||
| default_resource_attributes = { | ||
| "project_name": config.project, |
There was a problem hiding this comment.
currently fails to run. config.project does not exist.
| "eval_tags": json.dumps([]), | ||
| "metadata": json.dumps({}) |
There was a problem hiding this comment.
| "eval_tags": json.dumps([]), | |
| "metadata": json.dumps({}) | |
| "eval_tags": [], | |
| "metadata": {} |
| "metadata": json.dumps({}) | ||
| } | ||
|
|
||
| merged_resource_attributes = {**default_resource_attributes, **config.resource_attributes} |
There was a problem hiding this comment.
This is a critical issue that needs addressed
willkill07
added
feature request
willkill07
changed the title
2 participants
Description
Add support for OpenTelemetry-compatible export to Future AGI
Closes #972
By Submitting this PR I confirm:
Summary by CodeRabbit