GitHub - r0mdau/fluentforwardexporter: Opentelemetry exporter for the Fluent Forward protocol

Fluent Forward Exporter

Status
Stability	development: logs
Distributions	contrib
Issues
Code Owners	@r0mdau

Forward is the protocol used by Fluentd to route message between peers.

Protocol specification: Forward protocol specification v1
Library used IBM/fluent-forward-go (MIT License)

Getting Started

Settings

Property	Default value	Type	Description
endpoint.tcp_addr		string	MANDATORY Target URL to send `Forward` log streams to
endpoint.validate_tcp_resolution	false	bool	Controls whether to validate the tcp address and fail at startup.
connection_timeout	30s	time.Duration	Maximum amount of time a dial will wait for a connect to complete
tls.insecure	true	bool	If set to true, the connection is not secured with TLS.
tls.insecure_skip_verify	false	bool	Controls whether the exporter verifies the server's certificate chain and host name. If true, any certificate is accepted and any host name. This mode is susceptible to man-in-the-middle attacks
tls.ca_file	""	string	Used for mTLS. Path to the CA cert. For a client this verifies the server certificate
tls.cert_file	""	string	Used for mTLS. Path to the client TLS cert to use
tls.key_file	""	string	Used for mTLS. Path to the client TLS key to use
shared_key	""	string	A key string known by the server, used for authorization
require_ack	false	bool	Protocol delivery acknowledgment for log streams : true = at-least-once, false = at-most-once
tag	"tag"	string	Fluentd tag is a string separated by '.'s (e.g. myapp.access), and is used as the directions for Fluentd's internal routing engine
compress_gzip	false	bool	Transparent data compression. You can use this feature to reduce the transferred payload size
default_labels_enabled	true	map[string]bool	If omitted then default labels will be added. If one of the labels is omitted then this label will be added

See the default values in the method createDefaultConfig() in factory.go file.

Example, for default_labels_enabled that will add only the timestamp attribute in the log record:

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    require_ack: true
    tag: nginx
    compress_gzip: true
    default_labels_enabled:
      timestamp: true
      level: false
      message: false

But a best practice is to have at least timestamp, level and message in the exported log record to a Fluent endpoint.

Example with TLS enabled and shared key:

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    tls:
      insecure: false
    shared_key: otelcol-dev

Example with mutual TLS authentication (mTLS):

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    tls:
      insecure: false
      ca_file: ca.crt.pem
      cert_file: client.crt.pem
      key_file: client.key.pem

Severity

OpenTelemetry uses record.severity to track log levels.

Advanced Configuration

Queued retry capabilities are enabled by default, see the Exporter Helper queuing and retry settings to fine tune them.

Example usage:

exporters:
  fluentforward:
    endpoint:
      tcp_addr: a.new.fluentforward.target:24224
    connection_timeout: 10s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 5m
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 2000