Copyright (C) 2022, TomTom International BV
This class library contains a number of useful tools when coding in Kotlin. The code has been made open-source by TomTom so others can use it and contribute to it.
Currently, the library contains:
-
traceevents- This is a library to log semantic events in an application, where perhaps normally logging and log analysis would be used. -
uid- This is a simple library to deal with more performant and 'typesafe' UUIDs. -
memoization- This modules the ability to cache function results. -
extensions- General-purpose extension functions that can make your code more concise and readable.
Use Maven to run the unit tests as follows:
mvn clean test
To build the library JAR:
mvn clean package
If you wish to contribute to this project, please feel free to do so and send us pull requests. The source code formatting must adhere to the standard Kotlin formatting rules.
If you use IntelliJ IDEA, you can easily import the default Kotlin formatting rules like this:
Preferences -> Editor -> Code Style -> Kotlin -> (Scheme) Set From...
And then choose the pre-defined style Kotlin Style Guide. Voila!
Trace events offer a flexible way of logging semantic events during the run of an application. Semantic events are events that have some "external meaning", like "a route is planned", "the radio was switched on", "a connection to a phone was established", "a user logged in" and so on. (In contrast to debug messages likes "string is too long", "found record in database", etc. which only have an internal meaning to the program.)
The purpose of logging such events can be multiple, such as:
- for logging, or live monitoring of the system, when events are shown on a console or dashboard;
- for post-mortem debugging, to see what has happened in the system, after a bug was found;
- for system testing, where the test framework checks received events;
- for simulation purposes, where the events are forwarded to and processed by another system;
- to gather user behavior data, where user events are sent to a statistics module.
By default, a simple event logger is provided which logs all events directly to println. Although
it is enabled by default, you can turn it off, or replace it with your own. It's easy to modify this
to, for example, use the standard AndroidLog logger.
What makes this event logger different from a normal Log or, for example, Android Log directly, is
that the events are 'type-safe':
they are defined as Kotlin functions with (type-safe) parameters, rather than as handcrafted (and
often refactor-unsafe) strings.
This means that rather than calling something like Log.d(TAG, "User logged in, user=$user"), you
would write a type-safe expression like tracer.userLoggedIn(user).
The code for that looks like this:
(1) | interface MyTraceEvents : TraceEventListener {
(1) | fun userLoggedIn(user: User)
(1) | }
|
| fun someFunctionToLogIn(user: User) {
| // Throw the trace event:
(2) | tracer.userLoggedIn(user)
| }
|
| companion object {
(3) | private val tracer = Tracer.Factory.create<MyTraceEvents>()
| }
The lines marked (1) define the trace events; the line marked (2) throws (or logs) the event. That's it. Almost.
Trace events are defined in a local interface that derives from TraceEventListener. The interface
lists all events as type-safe function calls. In order to get an object to actually be able to call
these functions, you need to call Tracer.Factory.create from your class's companion object. This
explains line (3).
It creates a proxy object tracer (also shown in line (2) above), which implements all trace events
for you. The implementation of the trace event methods sends the serialized functional name and its
parameters to an event queue. The events on this queue are subsequently consumed by trace events
consumers, asynchronously. You can add a 'tagging' object to create. The class name of the
object (typically this) may be added to log if @TraceOptions(includeTaggingClass = true) is
used.
By default, a logging trace event consumer is enabled, which sends the events to a Log method that
send the message to stdout or can be redirected to, for example, Android Log. This consumer is a
special case in that it is actually a synchronous consumer, to make sure the order of events in the
log is consistent with the other log messages from your application.
The logger can be enabled or disabled using Tracer.enableTraceEventLogging. It can also be
switched to asynchronous mode. using Tracer.setTraceEventLoggingMode.
Note that only the logging consumer can be synchronous or asynchronous; custom event consumers are always processed in asynchronous mode.
You can replace the default logger, which logs to println with your own by creating an instance of
the interface Log.Logger like this:
import android.util.Log
import TraceLog.Logger
object MyAndroidLogger : Logger {
override fun log(logLevel: LogLevel, tag: String?, message: String, e: Throwable?) =
when (level) {
LogLevel.DEBUG -> Log.d(tag, message, e)
LogLevel.INFO -> Log.d(tag, message, e)
...
}
}
setLogger(MyAndroidLogger)
Or, to reset it to the default implementation:
setLogger()
Trace events in a TraceEventListener can be annotated with the @TraceLogLevel annotation.
@TraceLoglevel allows you to specify a log level, using parameter logLevel that will be used to
output the trace event to a standard logger using a specific log level, such as INFO, DEBUG
, ERROR, etc.
@TraceOptions offers the option to specify logging a full stack trace of a logged exception, using
the optional parameter includeExceptionStackTrace. This applies to the last argument of an event (
if it is a Throwable object). By default, a stack trace is included.
@TraceOptions also offers the option to specify logging the filename and line number of the caller
of an event. By default, the caller source code location is not provided.
@TraceOptions offers the option to add the class passed to create()
to the log message, or omit that. By default, the caller class is not included.
@TraceOptions offers the option to add the interface that defines the event to the log message, or
omit that. By default, the event interface is not included.
Example:
interface MyEvents : TraceEventListener {
@TraceLogLevel(LogLevel.ERROR, includeExceptionStackTrace = true, includeOwnerClass = true)
fun foundAnException(e: Exception)
}
Trace events are used for production code. Test cases should not use trace events. Test cases should use the logging tools of the test framework, or simply the Android log, to show progress or states.
By default, trace events are logged at LogLevel.DEBUG level to the default logger, that can be
redirected to, for example Android Log. If you prefer to have certain events logged at a different
log level, you can specify the log level with an annotation from TraceLogLevel, like this:
interface MyTraceEvents : TraceEventListener {
fun userLoggedIn() // This events gets logged at the default DEBUG level.
@TraceLogLevel(LogLevel.WARN)
fun cannotAccessDatabase() // This events gets logged at WARN level.
}
This paragraph describes some simple coding conventions to promote consistent declaration and usage of events.
In general, events are coupled to specific classes and as such it is advised to declare them inside the class file that uses the events. Place the interface declaration at the end of the class.
Tip: In IntelliJ or Android Studio you can easily view all trace event interfaces by clicking on
any occurrence of TraceEventListener and pressing Ctrl-Alt-B (or Option-Cmd-B)
The following naming conventions apply to events function names in a TraceEventListener:
Tracing is used for ongoing states and events that have occurred. It's not used for actions. As a result, the "trace event" names specify an ongoing state description, or use past tense. They don't use present tense and are not imperative.
Examples event and state names:
- Correct (events):
userLoggedIn(),connectionEstablished() - Correct (states):
serviceReady(),establishingConnection()
Examples of incorrect names:
- Wrong (imperative):
logUserIn(),establishConnection() - Wrong (wrong state name):
serviceInReadyState(),startingToEstablishConnection(),serviceInReadyState()
Note that there's not always a crystal-clear distinction between what events and states are, but for the naming convention that doesn't matter.
Trace event arguments are type-safe. They don't need to be strings. In fact, it's often better to pass the original object, rather than some form of string representation of it, as the object itself provides more information to trace consumer.
So, rather than defining an error event like
fun connectionLost(message: String)
when you know the message comes from an exception, like connectionLost(exception.message) you
better define the event as
fun connectionLost(reason: IOException)
and invoke it as connectionLost(exception).
Trace events should always avoid throwing unexpected NPE's. This might happen if your trace event has a signature like
fun connectionEstablished(channel: Channel)
and you find that sometimes you need to call the event as connectionEstablished(channel!!), rather
than as connectionEstablished(channel).
If that happens, you should consider changing the event signature to
fun connectionEstablished(channel: Channel?)
(and the invocation to connectionEstablished(channel)), just to make sure the !! operator cannot
throw a NPE.
Especially in cases where the arguments for an event come from Java libraries, where it's not always
clear if the type is @Nonnull (or @NonNull) or @Nullable, care should be take to make the
event interfaces NPE-safe.
There are 2 types of trace events consumers:
-
Generic trace event consumers, derived both from
GenericTraceEventConsumerand from aTraceEventListenerinterface.These consumers receive every event thrown in the system. They receive the event information as part of their
GenericTraceEventConsumer.consumeTraceEventimplementation. This method gets aTraceEventparameter that contains all information about the trace event, including date/time of occurence, class name, context information (per tracer instance, or even per thread the trace was created on), and the values of the parameters of the trace event method call (which can be obtained positionally, or by name fromgetNamedParametersMap, as a convenience).Generic consumers typically forward events to another system, such as the Android
Log, store them in a database, or perhaps even send them across application (or machine) boundaries. -
Specific trace event consumers, that implement a specific
TraceEventListenerinterface.For example, you could implement the
MyTraceEventsinterface (see above) in a class calledMyTraceEventsConsumerand register it as a trace events consumer. From then on, whenever a function from the MyTraceEvents interface is called, the corresponding implementation inMyTraceEventsConsumerwill be called (asynchronously).Specific consumers typically provide specific handler code for specific events. They react on specific events, rather than forward them. For example, switching on a red light on an alarm dashboard, when the event
temperatureTooHigh()is received.
Note that trace event consumer may receive mutable objects in an event, which may have been modified after the actual event was thrown. For more info, see the FAQ below.
The Tracer class defines the standard Log functions v(), d(), i(), w() and e(). These
can be used like this:
tracer.d("Found record $record") // Equivalent of `Log.d(TAG, msg)
tracer.w("Exception found", e) // Equivalent of `Log.w(TAG, msg, e)
By default, tracers are set up to send these log messages synchronously to the default logger.
It is possible to register trace event consumers to process these log messages as well, for example, to send logs to another system for analysis.
If you wish to define a tracer to only log standard messages using d() etc., you don't need to
create (or use) an TraceEventListener interface. You can just use this:
val tracer = Tracer.Factory.createLoggerOnly()
Note: Using the Log.x functions on tracers defeats the advantages of using type-safe
arguments. Always consider using trace event functions, when possible.
By default, trace event arguments are formatted using their default toString function. If you wish
to override them, use Tracer.registerToString<T>{ <toString implementation> } to override
the toString method. For example:
Tracer.registerToString<Coordinate>{ "($lat, $lon" }
By default, the toString for Arrays is replaced with one that provides a list of elements,
rather than an object reference.
Use Tracer.resetToDefaults() to de-register all custom toString handlers.
Sometimes multiple tracers may exist for a single class (if multiple instances of the tracer are initiated).
In those cases, it may be necessary to be able to disambiguate the tracer that the trace events came from.
This is solved by adding a context string to the create method. This context string is passed to
trace event consumers. Alternatively, trace event consumers can specify a regular expression to make sure
they only get the trace events for the specified tracer context(s).
// Declare 2 tracers.
val tracerMain = Tracer.Factory.create<SomeClass>(this::class, "main loop")
val tracerSec = Tracer.Factory.create<SomeClass>(this::class, "secondary")
// Declare a consumer for events from any tracer starting with "main".
val consumerMain = MyEventConsumerForSomeClass()
Tracer.addTraceEventConsumer(consumerMain, Regex("main.*"));
// Declare a consumer for all events (leaving out the regex).
val consumerAll = MyEventConsumerForSomeClass()
Tracer.addTraceEventConsumer(consumerAll);
Note that only GenericTraceEventConsumers are able to retrieve the context string passed by the tracer (as it is part
of the TraceEvent data object. Specific TraceEventConsumers (that implement the original tracer interface), cannot
access the context while processing events.
Sometimes it's useful to be able to retrieve more dynamic information from a trace event that perhaps wasn't even available at the location where the event was generated, but at an earlier time. This effectively allows the developer to create much richer events than are normally possible.
Java logging introduced MDC, mapped diagnostic context, for that. This allowed Java users to add thread-local
data that was added to log lines. The tracevents module allows something similar with the TraceThreadLocalContext
feature.
Here's an example of storing additional context in trace events.
TraceThreadLocalContext.put("currentRequest", myRequest)
This would add a map called traceThreadLocalContext to the event with value {"currentRequest": "myRequest"}
to each trace event generated from the same thread as this call to put was on.
The map traceThreadLocalContext is available for GenericTraceEventConsumers only and it is null when
no thread-local data was stored, or a non-empty map when data was stored.
This is particularly useful for adding information that isn't necessarily available at the location where the trace event is generated. Without this feature, such events would require more parameters, just for the sake of adding context to events, which would clutter the main code.
For example, suppose you've written a media player and wish to generate a skipToNext event when the user
presses the "skip to next" button. You would like to know the current song name when the
user presses the button, but that information is not available in the "skip to next" method.
Let's have a look at what this looks like without the TraceThreadLocalContext.
Let's assume the media player always plays random songs and when you press "skip to next" it selects a new random song. The interface for the events for the media player could look like:
interface MediaPlayerEvents : TraceEventListener {
fun eventPlaySong(songName: String)
fun eventSkipToNext()
}
And the events to log the song name, the playing position, and the skip to next event, would be generated in different methods:
fun playRandomSong() {
val song = chooseRandomSong(allSongs)
eventPlaySong(song.name) // <-- generate an event with the current song name
startPlaying(song)
}
fun userPressedNext() {
eventSkipToNext() // <-- generate the sip to next event
playRandomSong()
}
When this code is run, you'll get events like:
eventPlaySong(songName = "Dancing Queen")
eventPlaySong(songName = "Bohemian Rhapsody")
eventSkipToNext()
eventPlaySong(songName = "Breakfast in America")
Especially when these events are interlaced with other events, it becomes really hard to figure out which song was playing then "skip to next" was pressed. And if the program is multi-threaded, it becomes undoable.
Now, let's have a look at the code when TraceThreadLocalContext is used. Eveything remains the same except
for the implementation of playRandomSong:
fun playRandomSong() {
val song = chooseRandomSong(allSongs)
TraceThreadLocalContext.put("songName", song.name) // <-- store the song name in (thread-local) data
eventPlaySong(song.name) // <-- generate an event with the current song name
startPlaying(song)
}
The events that are generated now, look like this:
eventPlaySong(songName = "Dancing Queen", traceThreadLocalContext = null)
eventPlaySong(songName = "Bohemian Rhapsody", traceThreadLocalContext = {"songName": "Dancing Queen"})
eventSkipToNext(traceThreadLocalContext = {"songName": "Bohemian Rhapsody"})
eventPlaySong(songName = "Breakfast in America", traceThreadLocalContext = {"songName": "Bohemian Rhapsody"})
Now it becomes trivial to see that "Bohemian Rhapsody" was skipped, and that "Breakfast in America" was played before that.
Advanced examples of using this trace event mechanism are:
-
Sending events to a simulation system, which simulates the environment of the system. For example, an event that the cabin temperature has been set, may be processed by a simulator which uses a trace event consumer to receive such messages.
-
Displaying events on a dashboard, to gain more insight in the current status of the system, rather than having only a scrolling log to look at.
-
Collecting or sending system usage data for analytics. Developers can define all sorts of semantic events, and the system may collect them easily in a database, for later processing.
-
Should I use
Traceor (Android)Log?As a rule of thumb: you should always use
Tracerinstead of AndroidLog. If you only want to regular log messages, you can use the log functionsd()etc. -
What is the performance penalty of using
Tracerover AndroidLog?Using the default (synchronous) logging of log messages via
Tracer, over using AndroidLog, introduces a performance overhead of less than 20%. -
Is it safe to trace mutable objects?
Yes and no. The tracer module never modifies objects. But consumers of trace events will receive a mutable object, which may have been modified since the event was thrown. This may not be expected by the event consumer, although often it's not a problem at all. If you have to get around this you need provide a clone of the object in the trace call yourself. Beware that cloning objects to make them immutable is relatively expensive and may generate significant overhead.
For more information, for example on how to define and register trace event consumers, please refer
to the documentation in the Tracer class.
Generic immutable unique ID class. Really just an abstraction of UUIDs. Used to uniquely identify
things. No 2 Uid objects shall be equal unless they are the same instance of the Uid class or
their underlying UUIDs are equal.
The class has a generic type T to allow creating typesafe Uid's, like Uid<Message>. The class
represents UUIDs as Strings internally, to avoid loads of UUID to String conversions all the
time. This makes the class considerably faster in use than the regular Java UUID class.
Examples:
var messageId = Uid<Message>() // Creates a new unique message ID.
var personId = Uid<Person>() // Creates a new unique person ID.
messageId = personId // <-- Does not compile, the IDs are type safe.
val testId = Uid.fromString("1-2-3-4-5") // Allows shorthand notation for UUIDs,
// easy for testing, or input.
val s = messageId.toString() // Serialized to string.
val id = Uid<Message>(s) // Deserialized from string. Faster than `fromString`
// if the format is known to be the serialized format.
val messageId: Uid<Message> // If you need, you can translate IDs from one type to
val personId = messageId as Uid<Person> // another using 'as'. This is useful if the type
// information was lost, for example, in serialization.Provides the memoize extension to Kotlin functions that allows optimizing expensive functions by
caching the results corresponding to some set of specific inputs.
In order for memoization to work properly:
- all input arguments must have proper
equalsandhashCodeimplementations, - given the same input, the function must always return same output (i.e. not dependent on external parameters), and
- the function should not exhibit any side effects, other than the returned result.
To memoize a function, use the extension:
memoize(Int)- Extension creates Least Recently Used (LRU) cached function that will remove least recently used item if number of stored results exceeds provided limit.
Note that currently only functions with a maximum of 4 parameters are supported by memoize.
Example:
val function1: (Int) -> String = { p1: Int -> p1.toString() }.memoize()
function1(10) // First call with 10 - actual function is called and result is cached.
function1(11) // First call with 11 - actual function is called and result is cached.
function1(10) // Second call with 10 - value is returned from cache.General-purpose extension functions that can make your code more concise and readable. These extension functions aim to compliment Kotlin's functional programming style where Kotlin's default syntax and stdlib requires unwanted boilerplate code.
For example, it allows replacing the following snippet:
someComponent.getSomeNullableValue()
.let { it ?: someComponent.getFallbackValue() }
?.let {
if (it) doSomething()
else null
}with:
someComponent.getSomeNullableValue()
.ifNull { someComponent.getFallbackValue() }
.ifTrue { doSomething() }Copyright (C) 2020-2022, TomTom (http://tomtom.com). Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
[http://www.apache.org/licenses/LICENSE-2.0]
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Use Maven to run the unit tests as follows:
mvn clean test
To build the library JAR:
mvn clean package
If you wish to contribute to this project, please feel free to do so and send us pull requests. The source code formatting must adhere to the standard Kotlin formatting rules.
If you use IntelliJ IDEA, you can easily import the default Kotlin formatting rules like this:
Preferences -> Editor -> Code Style -> Kotlin -> (Scheme) Set From...
And then choose the pre-defined style Kotlin Style Guide. Voila!
Author: Rijn Buve
Contributors: Timon Kanters, Jeroen Erik Jensen, Krzysztof Karczewski
- Dependency updates in
pom.xml. No code changes.
- Introduce caching of trace event method annotations in the
Tracer.invoke()function. - Skip creation of a
TraceEventobject if there are no event consumers. This saves us from performing an expensiveLocalDateTime.now()call. - Skip retrieval of trace event method annotations for simple log function invocations, if there are no event consumers.
- Remove string concatenations in
createLogMessage()functions. Only useStringBuilderfunctions. - Change
toStringRegistryto useitem::javaClass.nameas key, and useitem::class.toString()only as fallback. - Skip reading
item.javaClass.getMethod("toString").declaringClassinconvertToStringUsingRegistry(): theitem.toString()function ofAnywill anyway print the name of the class. - Fix execution of unit tests requiring mockk-jvm (
extensionsandtraceeventsmodules): no unit test for these modules was being run anymore, after the 1.8.2 update. - Fix the
IfExtensionsTestunit tests: these tests were broken by mockk 1.13.4 (due to mockk/mockk#1033).
- Dependencies updated.
- Events traces are not queued till the processing starts.
- Added
Tracer.hasQueuedTraceEvents: Booleanto support testing that certain trace events were not sent.
- Fixed documentation on logging sync instead of async by default. No functional change.
- Remove use of reflection on the critical path of creating a
Tracerobject by removing the use ofKClass<*>.isCompanion. Remove the$Companionsuffix from the tagging class instead. - Initialize
Tracerlibrary faster by removing the use of reflection to obtain Tracer library class names. - Replace the use of
LocalDateTimewithInstantto measure elapsed time. This omits loading timezone data during initialization of theTracerlibrary. - Improve documentation of
TracerEvent.parameterNamesProvider.
- Reduce use of reflection on the critical path of trace events by not obtaining the trace event parameter names on the thread that traces the event. Only obtain the trace event parameter names when requested by a trace event consumer.
- Updated dependencies.
- Added Kotlin Contracts to
ifTrueandifNull.
- Removed cast extensions.
- Enables explicit API, requiring stricter type and visibility definitions of our public APIs.
- Added extensions module, providing general-purpose extension functions that can make your code more concise and readable.
- Updated dependencies.
- Added
getNamedParametersMaptoTraceEvent, which allows generic trace event consumers to access parameters and their values through a map that maps parameters names to their values. The other way to access parameter values is by usingargs, which is an array with parameter values, in the order of the method declaration.
- Added ability to store thread-local (MDC-style) context to trace events using
TraceThreadLocalContext. This context can be processed byGenericTraceEventConsumers. Initial idea by Chris Owen.
-
Updated POM dependencies.
-
Replaced deprecated
Channelmethodsofferandpollwith current method variantstrySendandtryReceive. -
Replaced some
internalproperties withprivateto be stricter on visibility. -
Removed redundant
suspendmodifiers from methods.
-
Updated POM dependencies.
-
Removed dependency on
kotlin.bintray.com.
- Added
contextto logging, if context is not empty.
- Added
contexttoTracer.createto allow disambiguation of tracers, if there are multiple for the same class.
- Updated dependencies and copyright.
- Added Kotlin function memoization extensions.
- Updated dependencies (except
mockkas 1.10.2 will fail the test for unclear reasons).
- Added
Uidclass for UUID handling.
- Minor bug fixes.
- Updated dependencies. No functional changes.
- Made
TraceEventConsumerCollectiona public class to be able to use it also for consuming streamed events.
-
Minor bug fixes.
-
Made
TraceEvent.stackTraceHoldernullable.
- Bug fix, renaming non-DEX formattable function name.
-
Renamed annotations:
@TraceOptions(includeExceptionStackTrace, includeTaggingClass, includeFileLocation, includeEventInterface) -
Added ability to add tagging class to tracers.
- Removed restriction that tracers can only be created in a
companion object.
-
Cleaned up annotation
@TraceLogLevelto only include trace level.
Added @TraceOptions(includeExceptionStackTrace, includeCalledFromClass, includeCalledFromFile, includeEventInterface)
- Added
throwableHoldertoTraceEventso event handlers can inspect the stack as well.
- Added
includeFileLocationto@TraceLogleveladd the caller filename and line number to the logger.
-
Added
includeExceptionStackTraceto annotation@TraceLogLevel. -
Added
includeOwnerClassto annotation@TraceLogLevel. -
Removed time stamp from
SYNClogging (already added by most loggers). -
Added unit tests to check message formats.
- Added
Tracer.Factory.createLoggerOnly(this).
-
Rename
LogtoTraceLogandLogLeveltoTraceLogLevel. -
Fixed string representation of arrays.
-
Added unit tests for message formatting.
- Added
Tracer.RegisterToStringto register string handlers for class types.
- Fixed license and copyright messages to Apache License 2.0.
-
Bug fixes to simple loggers
Tracer.d(message, exception). -
Fixed formatting of event parameters for arrays and lists.
-
Fixed unit test helper method.
-
TAGis non-nullable for loggers. -
Renamed directory structure from
src/main/kotlinto/src/main/javafor IntelliJ to understand package names -
Added TravisCI support for Github, including status badges in README.
- Initial release