Flattr tinylog GitHub

Benchmark

Method

All logging frameworks are configured to write log entries with the level info or higher to a log file. The output of log entries with the level trace and debug is disabled. After each benchmark run, the created log file is automatically checked for completeness and correctness.

The output format is: Date [Thread] Class.Method(): Message

For all logging framework, the default synchronous and - if available - the asynchronous output have been tested. All logging frameworks but JUL offer mechanisms for asynchronous output of log entries. tinylog uses a writing thread, Log4j 1 and Logback use an asynchronous appender and Log4j 2 the promoted asynchronous loggers. If asynchronous logging is activated, log entries are written buffered to the log file.

Every benchmark has been executed 120 times for each logging framework. For each run and framework, a fresh JVM has been set up. The 10 best and 10 worst runs have been discarded to avoid that outliers that could distort the whole result. The final result is the average of the remaining 100 runs.

Environment

All benchmarks have been executed on an Intel Core i7-4810MQ (2.80 GHz quad-core CPU) with 16 GB memory under Windows 7 (SP1) with JRE 8u60.

The following logging frameworks have been tested:

Maximum logging performance

This benchmark measures how fast each logging framework can output log entries. The benchmark just creates log entries with the levels error, warning, info, debug and trace in a loop one million times.

tinylog
with writing thread
 2 s 613 ms
 1 s 321 ms
JUL
7 s 335 ms
Log4j 1
with AsyncAppender
9 s 705 ms
 5 s 710 ms
Logback
with AsyncAppender
12 s 302 ms
 7 s 317 ms
Log4j 2
with AsyncLogger
14 s 566 ms
 7 s 183 ms

Influence on compute-intensive application

In practice, the maximum logging performance is probably less important than the influence of logging on the performance of the actual application. Thus, the latter is tested in a second benchmark that calculates all prime numbers from 2 to 10,000,000. The calculation is done in 16 threads to use all cores at full capacity. All found prime numbers are logged as info log entries and all others are dropped as trace log entries.

tinylog
with writing thread
 3 s 390 ms
 1 s 363 ms
Log4j 2
with AsyncLogger
 5 s 982 ms
 4 s 30 ms
JUL
11 s 944 ms
Log4j 1
with AsyncAppender
15 s 894 ms
 9 s 239 ms
Logback
with AsyncAppender
19 s 537 ms
 3 s 663 ms

Conclusion

In comparison to other logging frameworks, tinylog is the fastest in both benchmarks. The logging performance can be further improved by using asynchronous and buffered output of log entries. On the other hand, this has the disadvantage that possibly the last - and thus the most important - log entries are lost after a crash of the JVM.

The benchmark program is available on GitHub as open source for the reproducing the results or adaption of the benchmarks to own requirements. The benchmarks are parameterized, making it possible to configure the output format of log entries or the stack trace depth, for example.