Skip to content

Reopen of #10475: dynamic modification of OTEL Resource Attributes (via Content Modifier or LUA) #11228

@ErmakovDmitriy

Description

@ErmakovDmitriy

Is your feature request related to a problem? Please describe.
This is reopening of an automatically closed issue (#10475).

I would like to be able to modify otel_resource_attributes dynamically, based on
a log message data.
This is useful in case the messages come from non-OTEL source and I want to
use FluentBit to make them OTEL-compatible and send further.

For example, if we have Kubernetes container logs, it would be nice to put some Kubernetes metadata in otel_resource_attributes
such as pod namespace, name, container name.

Describe the solution you'd like

It would be nice to be able to use content_modifier as below (example is borrowed from the closed issue):

inputs:
      # Receiving forwarded
      # messages from a remote
      # FluentBit which has input "systemd"
    - name: forward
      listen: 0.0.0.0
      port: 24224

      processors:
        logs:
          - name: opentelemetry_envelope
          - name: lua
            # Some LUA record modifications
            # which will define additional log_record.resource_attributes
            # data based on information from the Log message.
          - name: content_modifier
            context: otel_resource_attributes
            # Here it would be really nice not only extract
            # the data using RecordAccessor
            # but also "upsert" (or replace) the whole 
            # otel_resource_attributes context data.
            action: upsert
            value: log_record.resource_attributes

or somehow modify the resource.attributes from LUA.

Describe alternatives you've considered

I have considered using LUA to set OTEL resource attributes (it would actually be nice to be able to use LUA to modify any OTEL contexts (https://docs.fluentbit.io/manual/4.0/data-pipeline/processors/content-modifier#opentelemetry-contexts) but it doesn't seem to be possible now.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions