Flattr tinylog

Benchmark

Method

Every benchmark was executed 120 times for each logging framework. The 10 best and 10 worst runs were dropped to avoid that outliers can distort the whole result. The final result is the average of the remaining 100 runs.

All logging frameworks were configured to write log entries with the level info or higher to a log file. The output of log entries with the level trace and debug was disabled. After each benchmark run, the created log file will be automatically checked for completeness and correctness.

The output format is: Date [Thread] Class.Method(): Message

For all logging framework, the default synchronous and - if available - asynchronous output were tested. The most logging frameworks offer mechanisms for asynchronous output of log entries. tinylog uses a writing thread, Log4j 1.x and Logback an asynchronous appender and Log4j 2 the promoted asynchronous loggers. If asynchronous logging is activated, log entries will be written buffered to the log file.

Environment

All benchmarks were executed on an Intel Core i7-4790 (3.60 GHz quad-core CPU) with 32 GB memory under Windows 7 (SP1) with JRE 8u25.

The following logging frameworks were tested:

Maximum logging performance

This benchmark measures how fast each logging framework can output log entries. The benchmark just creates one million times log entries with the levels error, warning, info, debug and trace in a loop.

tinylog
with writing thread
 3 s 144 ms
 1 s 693 ms
JUL
7 s 73 ms
Log4j 1.x
with AsyncAppender
9 s 233 ms
 5 s 449 ms
Log4j 2
with AsyncLogger
11 s 966 ms
 5.868 ms
Logback
with AsyncAppender
12 s 371 ms
 7 s 60 ms

Influence on compute-intensive application

Probably more important than the maximum logging performance is in practice, how logging influences the performance of the actual application. This is tested in a second benchmark that calculates all prime numbers from 2 to 10,000,000. The calculation is done by 16 threads to use all cores to full capacity. All found prime numbers are logged as info log entry and all others are dropped as trace log entry.

tinylog
with writing thread
 2 s 830 ms
 1 s 603 ms
Log4j 2
with AsyncLogger
 5 s 435 ms
 4 s 99 ms
JUL
11 s 623 ms
Log4j 1.x
with AsyncAppender
15 s 250 ms
 9 s 059 ms
Logback
with AsyncAppender
18 s 622 ms
 3 s 637 ms

Conclusion

In comparison to other logging frameworks, tinylog is the fastest in both benchmarks. The logging performance can be further improved by using asynchronous and buffered output of log entries. But on the other hand, this has the disadvantage that possibly the last - and thus the most important - log entries are lost after a crash of the JVM.

The benchmark program is available on GitHub as open source for reproducing the results or adapting benchmarks to own requirements. The benchmarks are parameterized. It is for example possible to configure the output format of log entries or the stack trace deep.