Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Create Logs SDK LoggerProvider #1517

Merged
merged 19 commits into from
Jan 22, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions logs_sdk/Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,8 @@
source 'https://rubygems.org'

gemspec

gem 'opentelemetry-api', path: '../api'
gem 'opentelemetry-logs-api', path: '../logs_api'
gem 'opentelemetry-sdk', path: '../sdk'
gem 'opentelemetry-test-helpers', path: '../test_helpers'
2 changes: 1 addition & 1 deletion logs_sdk/lib/opentelemetry-logs-sdk.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@
#
# SPDX-License-Identifier: Apache-2.0

require 'opentelemetry/sdk'
require 'opentelemetry/sdk/logs'
require 'opentelemetry/sdk/logs/version'
4 changes: 4 additions & 0 deletions logs_sdk/lib/opentelemetry/sdk/logs.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
# SPDX-License-Identifier: Apache-2.0

require_relative 'logs/version'
require_relative 'logs/logger'
require_relative 'logs/logger_provider'
require_relative 'logs/log_record_processor'
require_relative 'logs/export'

module OpenTelemetry
module SDK
Expand Down
23 changes: 23 additions & 0 deletions logs_sdk/lib/opentelemetry/sdk/logs/export.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# frozen_string_literal: true

# Copyright The OpenTelemetry Authors
#
# SPDX-License-Identifier: Apache-2.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches the pattern in OpenTelemetry::SDK::Trace::Export

The ExportError class is not present because it was not needed for the LoggerProvider. It will be added once it's used in subsequent PRs.

module OpenTelemetry
module SDK
module Logs
# The Export module contains result codes for exporters
module Export
# The operation finished successfully.
SUCCESS = 0

# The operation finished with an error.
FAILURE = 1

# The operation timed out.
TIMEOUT = 2
end
end
end
end
47 changes: 47 additions & 0 deletions logs_sdk/lib/opentelemetry/sdk/logs/log_record_processor.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# frozen_string_literal: true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is a boilerplate/placeholder so the LogRecordProcessor methods can be called. They will be implemented in a later PR.

# Copyright The OpenTelemetry Authors
#
# SPDX-License-Identifier: Apache-2.0

module OpenTelemetry
module SDK
module Logs
# LogRecordProcessor describes a duck type and provides synchronous no-op hooks for when a
# {LogRecord} is started or when a {LogRecord} is ended. It is not required to subclass this
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
# class to provide an implementation of LogRecordProcessor, provided the interface is
# satisfied.
class LogRecordProcessor
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
# Called when a {LogRecord} is emitted. Subsequent calls are not
# permitted after shutdown is called.
# @param [LogRecord] log_record The emitted {LogRecord}
# @param [Context] context The resolved Context (the explicitly passed Context or the current Context)
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
def on_emit(log_record, context); end

# Export all log records to the configured `Exporter` that have not yet
# been exported.
#
# This method should only be called in cases where it is absolutely
# necessary, such as when using some FaaS providers that may suspend
# the process after an invocation, but before the `Processor` exports
# the completed spans.
#
# @param [optional Numeric] timeout An optional timeout in seconds.
# @return [Integer] Export::SUCCESS if no error occurred, Export::FAILURE if
# a non-specific failure occurred, Export::TIMEOUT if a timeout occurred.
def force_flush(timeout: nil)
Export::SUCCESS
end

# Called when {LoggerProvider#shutdown} is called.
#
# @param [optional Numeric] timeout An optional timeout in seconds.
# @return [Integer] Export::SUCCESS if no error occurred, Export::FAILURE if
# a non-specific failure occurred, Export::TIMEOUT if a timeout occurred.
def shutdown(timeout: nil)
Export::SUCCESS
end
end
end
end
end
31 changes: 31 additions & 0 deletions logs_sdk/lib/opentelemetry/sdk/logs/logger.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# frozen_string_literal: true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Logger SDK will be fully written in a separate PR. The code here represents what's necessary to fulfill the LoggerProvider SDK spec.

# Copyright The OpenTelemetry Authors
#
# SPDX-License-Identifier: Apache-2.0

module OpenTelemetry
module SDK
module Logs
# The SDK implementation of OpenTelemetry::Logs::Logger
class Logger < OpenTelemetry::Logs::Logger
# @api private
#
# Returns a new {OpenTelemetry::SDK::Logs::Logger} instance. This should
# not be called directly. New loggers should be created using
# {LoggerProvider#logger}.
#
# @param [String] name Instrumentation package name
# @param [String] version Instrumentation package version
# @param [LoggerProvider] logger_provider The {LoggerProvider} that
# initialized the logger
#
# @return [OpenTelemetry::SDK::Logs::Logger]
def initialize(name, version, logger_provider)
@instrumentation_scope = InstrumentationScope.new(name, version)
@logger_provider = logger_provider
end
Comment on lines +12 to +27
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works toward the SDK Logger Provider => Logger Creation spec fulfillment

https://opentelemetry.io/docs/specs/otel/logs/sdk/#logger-creation

I didn't put any restrictions on initializing a new Logger in the code, but they are present in the documentation. I'm open to suggestions on how to limit Logger creation to only the LoggerProvider#logger method.

Name validation logic is handled in LoggerProvider#logger.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The @api private comment is our standard way of marking "internal" methods. There is no practical way to enforce it.

end
end
end
end
126 changes: 126 additions & 0 deletions logs_sdk/lib/opentelemetry/sdk/logs/logger_provider.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# frozen_string_literal: true

# Copyright The OpenTelemetry Authors
#
# SPDX-License-Identifier: Apache-2.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The specs for SDK LoggerProvider and SDK TracerProvider are extremely similar.

Given this, the code is very similar to the already implemented TracerProvider in the SDK gem.

module OpenTelemetry
module SDK
module Logs
# The SDK implementation of OpenTelemetry::Logs::LoggerProvider.
class LoggerProvider < OpenTelemetry::Logs::LoggerProvider
attr_reader :resource

UNEXPECTED_ERROR_MESSAGE = 'unexpected error in ' \
'OpenTelemetry::SDK::Logs::LoggerProvider#%s'
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved

private_constant :UNEXPECTED_ERROR_MESSAGE

# Returns a new LoggerProvider instance.
#
# @param [optional Resource] resource The resource to associate with
# new LogRecords created by {Logger}s created by this LoggerProvider.
#
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might notice that limits are not included in the parameters to initialize a LoggerProvider. This is because the limits are implemented in issue #1516.

# @return [OpenTelemetry::SDK::Logs::LoggerProvider]
def initialize(resource: OpenTelemetry::SDK::Resources::Resource.create)
@log_record_processors = []
@mutex = Mutex.new
@resource = resource
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A LoggerProvider MUST provide a way to allow a Resource to be specified. If a Resource is specified, it SHOULD be associated with all the LogRecords produced by any Logger from the LoggerProvider.
source

The Resource can be passed as an arg. If none is supplied, a new Resource will be created on initialization.

The Log Record Definition includes Resource as an attribute. The LogRecord class and the act of emitting log records from a logger are not in the scope of this PR.

The statement "it SHOULD be associated with all the LogRecords produced by any Logger from the LoggerProvider" will be addressed in a subsequent PR. The groundwork laid here enables a logger to access the logger provider's resource: logger.instance_variable_get(:@logger_provider).instance_variable_get(:@resource).

@stopped = false
end

# Creates an {OpenTelemetry::SDK::Logs::Logger} instance.
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
#
# @param [String] name Instrumentation package name
# @param [optional String] version Instrumentation package version
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
#
# @return [OpenTelemetry::SDK::Logs::Logger]
def logger(name:, version: nil)
version ||= ''

if !name.is_a?(String) || name.empty?
OpenTelemetry.logger.warn('LoggerProvider#logger called with an ' \
"invalid name. Name provided: #{name.inspect}")
end

OpenTelemetry::SDK::Logs::Logger.new(name, version, self)
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
end

# Adds a new log record processor to this LoggerProvider's
# log_record_processors.
#
# @param [LogRecordProcessor] log_record_processor The
# {LogRecordProcessor} to add to this LoggerProvider.
def add_log_record_processor(log_record_processor)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The LoggerProvider MAY provide methods to update the configuration. If configuration is updated (e.g., adding a LogRecordProcessor), the updated configuration MUST also apply to all already returned Loggers (i.e. it MUST NOT matter whether a Logger was obtained from the LoggerProvider before or after the configuration change). Note: Implementation-wise, this could mean that Logger instances have a reference to their LoggerProvider and access configuration only via this reference.

source

Every logger instance has a reference to the logger provider that created it. (see L50) This gives loggers access to the log record processors.

@mutex.synchronize do
if @stopped
OpenTelemetry.logger.warn('calling LoggerProvider#' \
'add_log_record_processor after shutdown.')
return
end
@log_record_processors = @log_record_processors.dup.push(log_record_processor)
end
end

# Attempts to stop all the activity for this LoggerProvider. Calls
# {LogRecordProcessor#shutdown} for all registered {LogRecordProcessor}s.
#
# This operation may block until all log records are processed. Must
# be called before turning off the main application to ensure all data
# are processed and exported.
#
# After this is called all newly created {LogRecord}s will be no-op.
#
# @param [optional Numeric] timeout An optional timeout in seconds.
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
# @return [Integer] Export::SUCCESS if no error occurred, Export::FAILURE if
# a non-specific failure occurred, Export::TIMEOUT if a timeout occurred.
def shutdown(timeout: nil)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See: https://opentelemetry.io/docs/specs/otel/logs/sdk/#shutdown

  • Only called once by setting @stopped to true L92
  • Provides a way to let the caller know whether it succeeded, failed, or timed out by returning OpenTelemetry::SDK::Logs::EXPORT constants (L81, L87, L93)
  • Implements a timeout through calls to OpenTelemetry::Common::Utilities.timeout_timestamp and OpenTelemetry::Common::Utilities.maybe_timeout (L84, L86)
  • Invokes shutdown on all the processors (L85 - 89)

@mutex.synchronize do
if @stopped
OpenTelemetry.logger.warn('LoggerProvider#shutdown called multiple times.')
return OpenTelemetry::SDK::Logs::Export::FAILURE
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
end

start_time = OpenTelemetry::Common::Utilities.timeout_timestamp
results = @log_record_processors.map do |processor|
remaining_timeout = OpenTelemetry::Common::Utilities.maybe_timeout(timeout, start_time)
break [OpenTelemetry::SDK::Logs::Export::TIMEOUT] if remaining_timeout&.zero?
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved

processor.shutdown(timeout: remaining_timeout)
end

@stopped = true
results.max || OpenTelemetry::SDK::Logs::Export::SUCCESS
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
end
end

# Immediately export all {LogRecord}s that have not yet been exported
# for all the registered {LogRecordProcessor}s.
#
# This method should only be called in cases where it is absolutely
# necessary, such as when using some FaaS providers that may suspend
# the process after an invocation, but before the {LogRecordProcessor}
# exports the completed {LogRecord}s.
#
# @param [optional Numeric] timeout An optional timeout in seconds.
kaylareopelle marked this conversation as resolved.
Show resolved Hide resolved
# @return [Integer] Export::SUCCESS if no error occurred, Export::FAILURE if
# a non-specific failure occurred, Export::TIMEOUT if a timeout occurred.
def force_flush(timeout: nil)
@mutex.synchronize do
return Export::SUCCESS if @stopped

start_time = OpenTelemetry::Common::Utilities.timeout_timestamp
results = @log_record_processors.map do |processor|
remaining_timeout = OpenTelemetry::Common::Utilities.maybe_timeout(timeout, start_time)
return Export::TIMEOUT if remaining_timeout&.zero?

processor.force_flush(timeout: remaining_timeout)
end

results.max || Export::SUCCESS
end
end
end
end
end
end
15 changes: 9 additions & 6 deletions logs_sdk/opentelemetry-logs-sdk.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,18 @@ Gem::Specification.new do |spec|
spec.require_paths = ['lib']
spec.required_ruby_version = '>= 3.0'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating dependencies to more recent versions. This came about when I realized the SimpleCov version that was installed didn't include a feature I wanted to use.

spec.add_dependency 'opentelemetry-logs-api', '~> 0.1.0'
spec.add_dependency 'opentelemetry-api', '~> 1.2'
spec.add_dependency 'opentelemetry-logs-api', '~> 0.1'
spec.add_dependency 'opentelemetry-sdk', '~> 1.3'

spec.add_development_dependency 'bundler', '>= 1.17'
spec.add_development_dependency 'minitest', '~> 5.0'
spec.add_development_dependency 'rake', '~> 12.0'
spec.add_development_dependency 'rubocop', '~> 1.51.0'
spec.add_development_dependency 'simplecov', '~> 0.17'
spec.add_development_dependency 'minitest', '~> 5.19'
spec.add_development_dependency 'opentelemetry-test-helpers', '~> 0.4'
spec.add_development_dependency 'rake', '~> 13.0'
spec.add_development_dependency 'rubocop', '~> 1.56'
spec.add_development_dependency 'simplecov', '~> 0.22'
spec.add_development_dependency 'yard', '~> 0.9'
spec.add_development_dependency 'yard-doctest', '~> 0.1.6'
spec.add_development_dependency 'yard-doctest', '~> 0.1.17'

if spec.respond_to?(:metadata)
spec.metadata['changelog_uri'] = "https://open-telemetry.github.io/opentelemetry-ruby/opentelemetry-logs-sdk/v#{OpenTelemetry::SDK::Logs::VERSION}/file.CHANGELOG.html"
Expand Down
22 changes: 3 additions & 19 deletions logs_sdk/test/.rubocop.yml
Original file line number Diff line number Diff line change
@@ -1,24 +1,8 @@
# inherit_from: .rubocop_todo.yml
inherit_from: ../.rubocop.yml

AllCops:
TargetRubyVersion: '3.0'

Lint/UnusedMethodArgument:
Enabled: false
Metrics/AbcSize:
Metrics/BlockLength:
Enabled: false
Metrics/LineLength:
Enabled: false
Metrics/MethodLength:
Max: 50
Metrics/PerceivedComplexity:
Max: 30
Metrics/CyclomaticComplexity:
Max: 20
Metrics/ParameterLists:
Enabled: false
Naming/FileName:
Exclude:
- 'lib/opentelemetry-logs-sdk.rb'
Style/ModuleFunction:
Metrics/AbcSize:
Enabled: false
Loading
Loading