Flattr tinylog

Benchmark

Method

Every benchmark was executed 120 times for each logging framework. The 10 best and 10 worst runs were discarded to avoid that outliers that could distort the whole result. The final result is the average of the remaining 100 runs.

All logging frameworks were configured to write log entries with the level info or higher to a log file. The output of log entries with the level trace and debug was disabled. After each benchmark run, the created log file is automatically checked for completeness and correctness.

The output format is: Date [Thread] Class.Method(): Message

For all logging framework, the default synchronous and - if available - the asynchronous output were tested. Most logging frameworks offer mechanisms for asynchronous output of log entries. tinylog uses a writing thread, Log4j 1.x and Logback use an asynchronous appender and Log4j 2 the promoted asynchronous loggers. If asynchronous logging is activated, log entries are written buffered to the log file.

Environment

All benchmarks were executed on an Intel Core i7-4790 (3.60 GHz quad-core CPU) with 32 GB memory under Windows 7 (SP1) with JRE 8u25.

The following logging frameworks were tested:

Maximum logging performance

This benchmark measures how fast each logging framework can output log entries. The benchmark just creates log entries with the levels error, warning, info, debug and trace in a loop one million times.

tinylog
with writing thread
 3 s 144 ms
 1 s 693 ms
JUL
7 s 73 ms
Log4j 1.x
with AsyncAppender
9 s 233 ms
 5 s 449 ms
Log4j 2
with AsyncLogger
11 s 966 ms
 5.868 ms
Logback
with AsyncAppender
12 s 371 ms
 7 s 60 ms

Influence on compute-intensive application

In practice, the maximum logging performance is probably less important than the influence of logging on the performance of the actual application. Thus, the latter is tested in a second benchmark that calculates all prime numbers from 2 to 10,000,000. The calculation is done in 16 threads to use all cores at full capacity. All found prime numbers are logged as info log entries and all others are dropped as trace log entries.

tinylog
with writing thread
 2 s 830 ms
 1 s 603 ms
Log4j 2
with AsyncLogger
 5 s 435 ms
 4 s 99 ms
JUL
11 s 623 ms
Log4j 1.x
with AsyncAppender
15 s 250 ms
 9 s 59 ms
Logback
with AsyncAppender
18 s 622 ms
 3 s 637 ms

Conclusion

In comparison to other logging frameworks, tinylog is the fastest in both benchmarks. The logging performance can be further improved by using asynchronous and buffered output of log entries. On the other hand, this has the disadvantage that possibly the last - and thus the most important - log entries are lost after a crash of the JVM.

The benchmark program is available on GitHub as open source for the reproducing the results or adaption of the benchmarks to own requirements. The benchmarks are parameterized, making it possible to configure the output format of log entries or the stack trace depth, for example.