feat: OpenTelemetry.logger_provider API, ProxyLoggers, Configuration,…

… and Instrument Registry (#1725)

* WIP: Log SDK configuration

* feat: Add configuration patch for logs SDK

* style: Update spacing

* test: Add tests for logs api

* feat: Update inheritance to get tests to pass

* feat: Add Instrument Registry to LoggerProvider

Create a registry for loggers to make sure a logger with an identical
name and version is created only once and reused

* feat: Rescue NameError for OTLP logs exporter

When OTLP logs exporter not installed, rescue the error, emit a message
and set the exporter to nil.

* Remove skip instrumenting stuff

* style: Rubocop

* test: Add skip for intermittent failure

* refactor: Remove delegate, mutex from ProxyLogger

* fix: Do not emit logs if stopped

Previously, a no-op Logger was returned when LoggerProvider#logger was
called after the provider was stopped.

Now, when the provider is stopped, the on_emit method will return early
and not emit any log records.

This more closely follows the behavior in the TracerProvider.

logs_api/lib/opentelemetry-logs-api.rb

@@ -5,5 +5,37 @@
 # SPDX-License-Identifier: Apache-2.0
 
 require 'opentelemetry'
-require_relative 'opentelemetry/logs'
-require_relative 'opentelemetry/logs/version'
+require 'opentelemetry/logs'
+require 'opentelemetry/logs/version'
+require 'opentelemetry/internal/proxy_logger_provider'
+require 'opentelemetry/internal/proxy_logger'
+
+# OpenTelemetry is an open source observability framework, providing a
+# general-purpose API, SDK, and related tools required for the instrumentation
+# of cloud-native software, frameworks, and libraries.
+#
+# The OpenTelemetry module in the Logs API gem provides global accessors
+# for logs-related objects.
+module OpenTelemetry
+  @logger_provider = Internal::ProxyLoggerProvider.new
+
+  # Register the global logger provider.
+  #
+  # @param [LoggerProvider] provider A logger provider to register as the
+  #   global instance.
+  def logger_provider=(provider)
+    @mutex.synchronize do
+      if @logger_provider.instance_of? Internal::ProxyLoggerProvider
+        logger.debug("Upgrading default proxy logger provider to #{provider.class}")
+        @logger_provider.delegate = provider
+      end
+      @logger_provider = provider
+    end
+  end
+
+  # @return [Object, Logs::LoggerProvider] registered logger provider or a
+  #   default no-op implementation of the logger provider.
+  def logger_provider
+    @mutex.synchronize { @logger_provider }
+  end
+end

logs_api/lib/opentelemetry/internal/proxy_logger.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module OpenTelemetry
+  module Internal
+    # @api private
+    #
+    # {ProxyLogger} is an implementation of {OpenTelemetry::Logs::Logger}. It is returned from
+    # the ProxyLoggerProvider until a delegate logger provider is installed. After the delegate
+    # logger provider is installed, the ProxyLogger will delegate to the corresponding "real"
+    # logger.
+    class ProxyLogger < Logs::Logger
+      attr_writer :delegate
+
+      # Returns a new {ProxyLogger} instance.
+      #
+      # @return [ProxyLogger]
+      def initialize
+        @delegate = nil
+      end
+
+      def on_emit(
+        timestamp: nil,
+        observed_timestamp: nil,
+        severity_number: nil,
+        severity_text: nil,
+        body: nil,
+        trace_id: nil,
+        span_id: nil,
+        trace_flags: nil,
+        attributes: nil,
+        context: nil
+      )
+        unless @delegate.nil?
+          return @delegate.on_emit(
+            timestamp: nil,
+            observed_timestamp: nil,
+            severity_number: nil,
+            severity_text: nil,
+            body: nil,
+            trace_id: nil,
+            span_id: nil,
+            trace_flags: nil,
+            attributes: nil,
+            context: nil
+          )
+        end
+
+        super
+      end
+    end
+  end
+end

logs_api/lib/opentelemetry/internal/proxy_logger_provider.rb

@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module OpenTelemetry
+  module Internal
+    # @api private
+    #
+    # {ProxyLoggerProvider} is an implementation of {OpenTelemetry::Logs::LoggerProvider}.
+    # It is the default global logger provider returned by OpenTelemetry.logger_provider.
+    # It delegates to a "real" LoggerProvider after the global logger provider is registered.
+    # It returns {ProxyLogger} instances until the delegate is installed.
+    class ProxyLoggerProvider < Logs::LoggerProvider
+      Key = Struct.new(:name, :version)
+      private_constant(:Key)
+      # Returns a new {ProxyLoggerProvider} instance.
+      #
+      # @return [ProxyLoggerProvider]
+      def initialize
+        super
+
+        @mutex = Mutex.new
+        @registry = {}
+        @delegate = nil
+      end
+
+      # Set the delegate logger provider. If this is called more than once, a warning will
+      # be logged and superfluous calls will be ignored.
+      #
+      # @param [LoggerProvider] provider The logger provider to delegate to
+      def delegate=(provider)
+        unless @delegate.nil?
+          OpenTelemetry.logger.warn 'Attempt to reset delegate in ProxyLoggerProvider ignored.'
+          return
+        end
+
+        @mutex.synchronize do
+          @delegate = provider
+          @registry.each { |key, logger| logger.delegate = provider.logger(key.name, key.version) }
+        end
+      end
+
+      # Returns a {Logger} instance.
+      #
+      # @param [optional String] name Instrumentation package name
+      # @param [optional String] version Instrumentation package version
+      #
+      # @return [Logger]
+      def logger(name = nil, version = nil)
+        @mutex.synchronize do
+          return @delegate.logger(name, version) unless @delegate.nil?
+
+          @registry[Key.new(name, version)] ||= ProxyLogger.new
+        end
+      end
+    end
+  end
+end

logs_api/lib/opentelemetry/logs.rb

@@ -4,11 +4,6 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
-require_relative 'logs/log_record'
-require_relative 'logs/logger'
-require_relative 'logs/logger_provider'
-require_relative 'logs/severity_number'
-
 module OpenTelemetry
   # The Logs API records a timestamped record with metadata.
   # In OpenTelemetry, any data that is not part of a distributed trace or a
@@ -20,3 +15,8 @@ module OpenTelemetry
   module Logs
   end
 end
+
+require 'opentelemetry/logs/log_record'
+require 'opentelemetry/logs/logger'
+require 'opentelemetry/logs/logger_provider'
+require 'opentelemetry/logs/severity_number'

logs_api/test/opentelemetry_logs_api_test.rb

@@ -0,0 +1,72 @@
+# frozen_string_literal: true
+
+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+require 'test_helper'
+
+describe OpenTelemetry do
+  class CustomLogRecord < OpenTelemetry::Logs::LogRecord
+  end
+
+  class CustomLogger < OpenTelemetry::Logs::Logger
+    def on_emit(*)
+      CustomLogRecord.new
+    end
+  end
+
+  class CustomLoggerProvider < OpenTelemetry::Logs::LoggerProvider
+    def logger(name = nil, version = nil)
+      CustomLogger.new
+    end
+  end
+
+  describe '.logger_provider' do
+    after do
+      # Ensure we don't leak custom logger factories and loggers to other tests
+      OpenTelemetry.logger_provider = OpenTelemetry::Internal::ProxyLoggerProvider.new
+    end
+
+    it 'returns a Logs::LoggerProvider by default' do
+      logger_provider = OpenTelemetry.logger_provider
+      _(logger_provider).must_be_kind_of(OpenTelemetry::Logs::LoggerProvider)
+    end
+
+    it 'returns the same instance when accessed multiple times' do
+      _(OpenTelemetry.logger_provider).must_equal(OpenTelemetry.logger_provider)
+    end
+
+    it 'returns user-specified logger provider' do
+      custom_logger_provider = CustomLoggerProvider.new
+      OpenTelemetry.logger_provider = custom_logger_provider
+      _(OpenTelemetry.logger_provider).must_equal(custom_logger_provider)
+    end
+  end
+
+  describe '.logger_provider=' do
+    after do
+      # Ensure we don't leak custom logger factories and loggers to other tests
+      OpenTelemetry.logger_provider = OpenTelemetry::Internal::ProxyLoggerProvider.new
+    end
+
+    it 'has a default proxy logger' do
+      refute_nil OpenTelemetry.logger_provider.logger
+    end
+
+    it 'upgrades default loggers to *real* loggers' do
+      # proxy loggers do not emit any log records, nor does the API logger
+      # the on_emit method is empty
+      default_logger = OpenTelemetry.logger_provider.logger
+      _(default_logger.on_emit(body: 'test')).must_be_instance_of(NilClass)
+      OpenTelemetry.logger_provider = CustomLoggerProvider.new
+      _(default_logger.on_emit(body: 'test')).must_be_instance_of(CustomLogRecord)
+    end
+
+    it 'upgrades the default logger provider to a *real* logger provider' do
+      default_logger_provider = OpenTelemetry.logger_provider
+      OpenTelemetry.logger_provider = CustomLoggerProvider.new
+      _(default_logger_provider.logger).must_be_instance_of(CustomLogger)
+    end
+  end
+end

logs_sdk/lib/opentelemetry/sdk/logs.rb

@@ -5,6 +5,7 @@
 # SPDX-License-Identifier: Apache-2.0
 
 require_relative 'logs/version'
+require_relative 'logs/configuration_patch'
 require_relative 'logs/logger'
 require_relative 'logs/logger_provider'
 require_relative 'logs/log_record'

logs_sdk/lib/opentelemetry/sdk/logs/configuration_patch.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+require 'opentelemetry/sdk/configurator'
+
+module OpenTelemetry
+  module SDK
+    module Logs
+      # The ConfiguratorPatch implements a hook to configure the logs portion
+      # of the SDK.
+      module ConfiguratorPatch
+        def add_log_record_processor(log_record_processor)
+          @log_record_processors << log_record_processor
+        end
+
+        private
+
+        def initialize
+          super
+          @log_record_processors = []
+        end
+
+        # The logs_configuration_hook method is where we define the setup
+        # process for logs SDK.
+        def logs_configuration_hook
+          OpenTelemetry.logger_provider = Logs::LoggerProvider.new(resource: @resource)
+          configure_log_record_processors
+        end
+
+        def configure_log_record_processors
+          processors = @log_record_processors.empty? ? wrapped_log_exporters_from_env.compact : @log_record_processors
+          processors.each { |p| OpenTelemetry.logger_provider.add_log_record_processor(p) }
+        end
+
+        def wrapped_log_exporters_from_env
+          # TODO: set default to OTLP to match traces, default is console until other exporters merged
+          exporters = ENV.fetch('OTEL_LOGS_EXPORTER', 'console')
+
+          exporters.split(',').map do |exporter|
+            case exporter.strip
+            when 'none' then nil
+            when 'console' then Logs::Export::SimpleLogRecordProcessor.new(Logs::Export::ConsoleLogRecordExporter.new)
+            when 'otlp'
+              otlp_protocol = ENV['OTEL_EXPORTER_OTLP_LOGS_PROTOCOL'] || ENV['OTEL_EXPORTER_OTLP_PROTOCOL'] || 'http/protobuf'
+
+              if otlp_protocol != 'http/protobuf'
+                OpenTelemetry.logger.warn "The #{otlp_protocol} transport protocol is not supported by the OTLP exporter, log_records will not be exported."
+                nil
+              else
+                begin
+                  Logs::Export::BatchLogRecordProcessor.new(OpenTelemetry::Exporter::OTLP::LogsExporter.new)
+                rescue NameError
+                  OpenTelemetry.logger.warn 'The otlp logs exporter cannot be configured - please add opentelemetry-exporter-otlp-logs to your Gemfile. Logs will not be exported'
+                  nil
+                end
+              end
+            else
+              OpenTelemetry.logger.warn "The #{exporter} exporter is unknown and cannot be configured, log records will not be exported"
+              nil
+            end
+          end
+        end
+      end
+    end
+  end
+end
+
+OpenTelemetry::SDK::Configurator.prepend(OpenTelemetry::SDK::Logs::ConfiguratorPatch)

logs_sdk/lib/opentelemetry/sdk/logs/logger_provider.rb

@@ -9,6 +9,9 @@ module SDK
     module Logs
       # The SDK implementation of OpenTelemetry::Logs::LoggerProvider.
       class LoggerProvider < OpenTelemetry::Logs::LoggerProvider
+        Key = Struct.new(:name, :version)
+        private_constant(:Key)
+
         UNEXPECTED_ERROR_MESSAGE = 'unexpected error in ' \
           'OpenTelemetry::SDK::Logs::LoggerProvider#%s'
 
@@ -28,6 +31,8 @@ def initialize(resource: OpenTelemetry::SDK::Resources::Resource.create, log_rec
           @mutex = Mutex.new
           @resource = resource
           @stopped = false
+          @registry = {}
+          @registry_mutex = Mutex.new
         end
 
         # Returns an {OpenTelemetry::SDK::Logs::Logger} instance.
@@ -44,7 +49,9 @@ def logger(name:, version: nil)
               "invalid name. Name provided: #{name.inspect}")
           end
 
-          Logger.new(name, version, self)
+          @registry_mutex.synchronize do
+            @registry[Key.new(name, version)] ||= Logger.new(name, version, self)
+          end
         end
 
         # Adds a new log record processor to this LoggerProvider's
@@ -135,6 +142,8 @@ def on_emit(timestamp: nil,
                     instrumentation_scope: nil,
                     context: nil)
 
+          return if @stopped
+
           log_record = LogRecord.new(timestamp: timestamp,
                                      observed_timestamp: observed_timestamp,
                                      severity_text: severity_text,