Hi users!

We have released v1.14.0. ChangeLog is here.

This release is a new release of v1.14 series. We recommend to upgrade Fluentd because it contains fixes about in_tail bugs because a bug that deleted paths are not removed was fixed.

System configuration about metrics parameters were added

In this release, new system configuration parameters were added.

• enable_input_metrics
• enable_size_metrics

You can enable these parameters in <system> directive.

<system>
  enable_input_metrics
  enable_size_metrics
</system>

This feature might need to pay higher CPU cost, so these event metrics features are disabled by default. These features are also enabled by --enable-input-metrics,--enable-size-metrics command line.

And more, metrics plugin interface was improved. This improvements are introduced to enhance collecting metrics functionality.

In the future, 3rd party plugin will be able to handle these metrics mechanism.

@ROOT label was changed as reserved label

In this release, the @ROOT label is reserved for a builtin label which is used for getting root router by plugin's event_emitter_router API.

This is incompatible change. Do not use @ROOT like <label @ROOT>.

This label is aimed to assign a label back to the default route. For example, timed-out event records are handled by the concat filter which can be sent to the default route.

in_syslog: send_keepalive_packet option was added

TCP keepalive was supported by in_syslog to avoid dead connection, as same as in_forward.

in_http: cors_allow_credentials option was added

This option tells browsers whether to expose the response to frontend when the credentials mode is include.

The default value is false.

Fixed a crash bug on logging peer host name with errors

When a client with invalid TLS certificates is trying to connect, it is hard to investigate which client has the problem without peer information, so it was changed to log peer information since v1.12.4.

But, if it fails to resolve peer name information when logging, it caused a crash bug unexpectedly. In this release, this bug was fixed.

Upgrade attention for TD Agent users

If you use Fluentd v1.12 or later, we recommend to use at least v1.12.4. It is because in_tail contains serious bugs in it.

Note that the previous version of td-agent 4.1.1 ships a buggy Fluentd v1.12.3. We recommend upgrading to latest td-agent 4.2.0. It bundles Fluentd 1.13.3. If you want to upgrade Fluentd further more, upgrade it by yourself.

$ sudo td-agent-gem install fluentd --version=1.14.0

Enjoy logging!

Hi users!

We have released v1.14.1. ChangeLog is here.

This release is a maintenance release of v1.14 series.

We recommend to upgrade Fluentd because it contains fixes about in_tail crash bug.

in_tail: Fixed a crash bug that it raise undefined method of eof? error

In this release, one of in_tail plugin bug was fixed.

This crash bug will happen only when read_bytes_limit_per_second was specified. read_bytes_limit_per_second parameter was introduced since Fluentd v1.13.0, so it affects v1.13.0 or later with read_bytes_limit_per_second was specified.

If you don't use read_bytes_limit_per_second configuration parameter at all, it has nothing to do with it.

Fixed a error when using @include directive

If you use the following example, it causes an unexpected error.

@include "https://raw.githubusercontent.com/fluent/fluentd/master/example/in_forward.conf"

The error occurs when http or https scheme URI is used in @include directive with Ruby 3. On Ruby 2.7, it raise an warning message (using deprecated method), not an error.

out_copy: Fixed to suppress a wrong warning for ignore_if_prev_success

Because of internal inappropriate conditional handling, even though a user set ignore_if_prev_success, it doesn't work as expected.

Instead, it emits the following warning.

2021-09-24 14:10:54 +0900 [warn]: #0 ignore_errors are specified in all <store>, but ignore_if_prev_success is not specified. Is this intended?

This behavior was fixed.

Upgrade attention for TD Agent users

If you use Fluentd v1.12 or later, we recommend to use at least v1.12.4. It is because in_tail contains serious bugs in it.

As latest td-agent 4.2.0 bundles Fluentd 1.13.3. If you want to upgrade Fluentd further more, upgrade it by yourself.

$ sudo td-agent-gem install fluentd --version=1.14.1

Enjoy logging!

Hi users!

We have released v1.14.2. ChangeLog is here.

This release is a maintenance release of v1.14 series.

IMPORTANT: This release contain the fix for CVE-2021-41186 - ReDoS vulnerability in parser_apache2. This vulnerabbility is affected from Fluentd v0.14.14 to v1.14.1. We recommend to upgrade Fluentd to v1.14.2 or use patched version of parser_apache2 plugin.

Fixed ReDoS vulnerability in parser_apache2

parser_apache2 plugin in Fluentd v0.14.14 to v1.14.1 suffers from a regular expression denial of service (ReDoS) vulnerability.

A broken apache log with a certain pattern of string can spend too much time in a regular expression, resulting in the potential for a DoS attack.

There are two workarounds:

• Don't use parser_apache2 for parsing logs which cannot guarantee generated by Apache.
• Put patched version of parser_apache2.rb into /etc/fluent/plugin directory (or any other directories specified by the environment variable FLUENT_PLUGIN or --plugin option of fluentd).

As noted above, it requires a maliciously compromised Apache log to cause ReDoS. Usually, such a log will not be generated by Apache, so the risk of ReDoS is low.

Patched version of parser_apache2 is available from parser_apache2.rb

fluent-cat: Added --event-time option for testing

In this release, --event-time option was added to fluent-cat.

You can send event in specific event time with --event-time option instead of sending time in the previous versions.

echo '{"test": "ok"}' | fluent-cat --event-time "2021-10-29 13:14:15.0+00:00" tag

GitHub: Your Configuration field was automatically quoted

Recently, when creating issue on GitHub, Your Configuration field was automatically quoted.

Most users doesn't quote Your Configuration field manually when they report a bug. So it was changed to be quoted automatically.

Upgrade attention for TD Agent users

If you use Fluentd v1.12 or later, we recommend to use at least v1.12.4. It is because in_tail contains serious bugs in it.

As latest td-agent 4.2.0 bundles Fluentd 1.13.3. If you want to upgrade Fluentd further more, upgrade it by yourself.

$ sudo td-agent-gem install fluentd --version=1.14.2

Enjoy logging!

Hi users!

We have released v1.14.3. ChangeLog is here.

This release is a maintenance release of v1.14 series.

in_tail: Fixed a bug that no new logs are read

This release contains a bug fix that no new logs were read when the following conditions were met.

• enable_stat_watcher is enabled. (default: true)
• enable_watch_timer is disabled. (default: true)

Thus, this bug was happened when you configure it like this:

<source>
  @type tail
  ...
  enable_stat_watcher true
  enable_watch_timer false
  ...
</source>

As enable_stat_watcher is enabled by default, this bug was occurred when enable_watch_timer false was explicitly set.

in_tail: Fixed a bug that the beginning and initial lines are lost

In this release, fixed a bug about in_tail that the beginning and initial lines are lost after startup when read_from_head false and path includes wildcard '*'.

This wrong behavior was caused by the following logic:

• New files which are detected on refresh are not read from the beginning and initial lines are lost
• Same as above for rotated files

Fixed a bug that processing messages were lost

In this release, fixed a bug that processing messages were lost when BufferChunkOverflowError was thrown even though only a specific message size exceeds chunk_limit_size.

Here is the example that it causes the bug:

  message a\n
  message b is longer.....message data exceeds chunk_limit_size\n
  message c\n

In above example, if incoming partial data exceeds chunk_limit_size, it raises BufferChunkOverflowError, then "message a" and "message c" is also lost.

It is not appropriate because the problem exists only a partial message data. Now such a message is changed to skip processing furthermore.

Note that even though with v1.14.3, still https://github.com/fluent/fluentd/issues/1849 is not fixed yet. This is also similar but another issue which cause BufferChunkOverflowError.

Upgrade attention for TD Agent users

If you use Fluentd v1.12 or later, we recommend to use at least v1.12.4. It is because in_tail contains serious bugs in it.

As latest td-agent 4.2.0 bundles Fluentd 1.13.3. If you want to upgrade Fluentd further more, upgrade it by yourself.

$ sudo td-agent-gem install fluentd --version=1.14.3

Enjoy logging!

Hi users!

We have released td-agent v4.3.0. td-agent is a stable distribution package of Fluentd.

This release includes a security fix. Please check the topics below for details.

• CVE-2021-41186 - ReDoS vulnerability in parser_apache2.

Changes from td-agent v4.2.0

• Update bundled Fluentd from v1.13.3 to v1.14.3. Please see each release announcement of Fluentd for more detail:
  • Fluentd v1.14.0 has been released
  • Fluentd v1.14.1 has been released
  • Fluentd v1.14.2 has been released
  • Fluentd v1.14.3 has been released
• Update bundled Ruby from v2.7.4 to v2.7.5
• Support Debian 11 "bullseye"
• deb: Remove needless post-install script
• Windows: Add default log rotation setting

Please see ChangeLog for more detail.

Download

Please see the download page.

End of life announcement

• The package for Ubuntu xenial won't be provided from the next minor update (td-agent 4.4.x) because Ubuntu 16.04.x had already reached to End of Standard Support.
• td-agent 3 series won't be maintained anymore, we'll post another article for it.

Hi users and developers!

We announce the dropping schedule for TD Agent 3 development.

About Treasure Agent (td-agent) 3

Since td-agent 3.0.0 was released in 2017, td-agent 3 had been maintained for a long time.

As new major version of td-agent 4 was released in August 2020, and several updates was shipped, we decided to stop maintaining td-agent 3.

Scheduled end of life - Feb 2022

As you know, already stepping down maintenance activity, new minor update for td-agent 3 will not be shipped anymore.

Thus, we recommend to use td-agent 4, latest fluentd v1 series, for new deployment :)

How to migrate to Treasure Agent (td-agent) 4

There is a good article to do it.

• Upgrade td-agent from v3 to v4

Follow the above instructions.

Hi users!

We have released v1.14.4. ChangeLog is here.

This release is a maintenance release of v1.14 series.

in_tail: Add max_line_size option to skip long lines

Starting from v1.14.4, in_tail supports a new option max_line_size that allows to skip lines above a certain size.

<source>
  @type tail
  path /var/log/nginx/*.log
  ...
  max_line_size 8MB  # skip very long lines
  ...
</source>

This option can be used to ensure that messages coming from in_tail is within chunk_limit_size, and will eliminate costly error handling on very large messages.

Improvement on BufferChunkOverflowError handling

Beginning with this release, Fluentd will no longer raise BufferChunkOverflowError on an oversized event stream if each event in the stream is smaller than chunk_limit_size

Example

es = Fluent::ArrayEventStream.new([[timestamp, {"message" => "a" * 1_000_000}],
                                 [timestamp, {"message" => "b" * 1_000_000}],
                                 [timestamp, {"message" => "c" * 1_000_000}]])

# Previously this could result in BufferChunkOverflowError even
# when chunk_limit_size is large enough (e.g. 1.2MB).
router.emit_stream(@tag, es)

See #3560 for more information.

out_file: Fix inccorect append writes on macOS

Prior to this release, out_file failed to write to files if append option was enabled on macOS with Ruby 2.7/3.0.

This issue was due to a bug in Ruby's IO implementation. This release contains a workaround for that bug.

See #3579 for details.

Enjoy logging!

Hi users!

We have released v1.14.5. ChangeLog is here.

This release is a maintenance release of v1.14 series.

in_http: Add support for "application/x-ndjson"

In this release, in_http supports the new Content-Type "application/x-ndjson". Its format is "ndjson", which is basically a list of JSON objects separated by "\n".

Here is an example of the format.

{"foo": "bar"}
{"buz": "hoge"}

See #3616 for more information.

Add support for the UCRT binary for Windows RubyInstaller 3.1

RubyInstaller 3.1 has switched C-Runtime from MSVCRT to UCRT, and the ruby and gem platform has changed to x64-mingw-ucrt. There was a problem in this environment that Windows-specific dependent gems could not be installed because Fluentd for this architecture had not been released.

Beginning with this release, Fluentd for this architecture is also released and this problem is solved.

See #3613 for more information.

out_forward: Fix hang-up issue during TLS handshake

There was a known issue that TLS handshake takes a long time or hangs when out_forward uses TLS transport, even though connect_timeout is specified.

With this release, connect_timeout is now reliably applied to TLS transport and able to prevent hangs.

See #3601 for more information.

Miscellaneous fixes

• Fix a bug of retrying once when retry_max_times is 0. #3608
• Fix "invalid byte sequence is replaced" warning not to write out invalid characters. #3596
• Bump up required ServerEngine to v2.2.5. #3599

Enjoy logging!

Hi users!

We have released v1.14.6. ChangeLog is here.

This release is a maintenance release of v1.14 series.

Enhancement

Enable server plugins to specify socket-option SO_LINGER

In this release, we add a new option linger_timeout to server plugin-helper so that we can specify SO_LINGER socket-option when using TCP or TLS server function of the helper.

The helper has used 0 for linger_timeout internally before. This made the connection closing immediate, and especially on non-Windows, this made the server send RST rather than FIN on the closing.

The main use of this option is to make the server send FIN on closing on non-Windows, by setting a positive value such as 1.

This new option is in transport section, and we can specify this for plugins using TCP or TLS server function of the helper, such as in_tcp, in_http, in_syslog, and in_forward.

<source>
  @type tcp
  ...
  <parse>
    ...
  </parse>
  <transport tcp>
    linger_timeout 1
  </transport>
</source>

Note: in_forward plugin had linger_timeout option before, but the option is now deprecated and unified to transport section.

See #3644 for more information.

Add --umask to fluentd command line parameter

When we launch Fluentd with --no-supervisor, the umask value of the environment is not applied because Fluentd overwrites the value.

In this release, We can use --umask option as a fluentd command line parameter to specify the umask value in the case of launching Fluentd with --no-supervisor.

$ fluentd -c {config_path} --no-supervisor --umask 002

See #3671, #3679 for more information.

Bug Fixes

Fix wrong calculation of retry count and interval

There were mistakes in the calculation about exponential_backoff retry_type.

In this release, the calculation is fixed, and the count and the interval change as following.

Count

• Before: At most retry_max_times + 1 retries were triggered.
• This Release: At most retry_max_times retries are triggered.

Interval

• Before: By default, n-th retry was triggered after 2^(n-1) seconds elapsed since the first time.
• This Release: By default, n-th retry is triggered after 2^(n-1) seconds elapsed since the last time.

When to switch to secondary

• Before: The total time calculation was wrong. Although retry_max_times + 1 retries were mistakenly triggered, retry_secondary_threshold was applied to the total time to the (retry_max_times)-th retry. So switching to secondary occurred earlier than the correct time.
• This Release: retry_secondary_threshold is correctly applied to the total retry time.

The examples of retry_max_times: 10 are as follows. (Set retry_randomize: false for testing.)

Only primary output

With secondary output

See #3649 for more information.

Apply modifications to the records being passed to @ERROR label

Filter plugins such as filter_record_transformer can modify the contents of records before the output.

If an error occurred in Output plugin such that records were passed to @ERROR label, those modifications were not applied to those records being passed.

In this release, records reflecting those modifications are now passed to @ERROR label.

See #3631 for more information.

Miscellaneous fixes

• Support IPv6 address for rpc_endpoint in system config. #3641.
• Fix some typo of metric names. #3630, #3673.
• Declare null_value_pattern as regexp. #3650.
• CI: Support Ruby 3.1 except Windows. #3619.
• Fix CHANGELOG.md heading styles. #3648.
• Switch to GitHub Discussions. #3654.

Enjoy logging!

Hi users!

We have released td-agent v4.3.1. td-agent is a stable distribution package of Fluentd.

Changes from td-agent v4.3.0

• Update bundled Fluentd from v1.14.3 to v1.14.6. Please see each release announcement of Fluentd for more detail:
  • Fluentd v1.14.4 has been released
  • Fluentd v1.14.5 has been released
  • Fluentd v1.14.6 has been released
• Update bundled Ruby from v2.7.5 to v2.7.6
• Add some new plugins
  • fluent-plugin-opensearch
  • fluent-plugin-metrics-cmetrics
  • fluent-plugin-calyptia-monitoring
  • fluent-plugin-windows-exporter
• rpm: Remove .build-id directories

Please see ChangeLog for more detail.

Download

Please see the download page.

About packages for latest distributions

Although Ubuntu 22.04 LTS (Jammy Jellyfish) is already released, we don't yet provide the official td-agent package for it. An experimental package which bundles Ruby 3.1 is work in progress.

End of life announcement

• The package for Ubuntu xenial won't be provided from the next minor update (td-agent 4.4.0) because Ubuntu 16.04.x had already reached to End of Standard Support.
• td-agent 3 series won't be maintained anymore.

We have released td-agent v4.3.1.1 for Windows. This version is a bug fix release of v4.3.1 only for Windows platform.

Changes from td-agent v4.3.1

• Update OpenSSL to v1.1.1.o-2 to fix memory leak: fluent-package-builder#374
• Update nokogiri to 1.13.6 to fix some vulnerabilities.

Download

Please see the download page.

We have released td-agent v4.3.2. This release is mainly for supporting new distributions released in 2022Q2. We choose Ruby 3.1 to bundle for such new distributions to adopt to recent environment such as OpenSSL 3.0. For other distributions, we still use Ruby 2.7. We'll migrate to Ruby 3 completely after Ruby 3.2 is released.

Changes from td-agent v4.3.1

• Support new distributions
  • Red Hat Enterprise Linux 9
  • Ubuntu 22.04 LTS (Jammy Jellyfish)
• Remove nokogiri from Linux and macOS packages
  • It was not used in actual by default on such environments.
• Update yajl-ruby to v1.4.3 to address vulnerability
• Update OpenSSL to 1.1.1p (for macOS and Windows)

Download

Please see the download page.

Hi users!

We have released v1.15.0. ChangeLog is here.

This release is a new release of v1.15 series.

In this release, we added some new functions and fixed some crash bugs, especially about in_tail.

Enhancement

Support YAML configuration format

Fluentd now supports YAML configuration format as follows.

config:
  - source:
      $type: forward
      port: 24224
  - source:
      $type: http
      port: 9880
  - match:
      $tag: myapp.access
      $type: file
      path: /var/log/fluent/access

This is not 1-by-1 correspondence for the traditional Fluentd config file syntax.

Please see the document for detailed usage instructions or see #3712 for more information.

in_tail: Add log throttling in files based on group rules

In this release, we add a new option directive group to in_tail.

From v1.13, We can use read_bytes_limit_per_second to throttle log flow rate, but we can only set it for each source unit.

By using this new option, we can throttle log flow rate for each file unit.

Please see the document for detailed usage instructions or see #3535 for more information.

Add restart_worker_interval to system configuration

In this release, we add a new option restart_worker_interval.

By default, Fluentd immediately restarts stopped workers, but you may want to keep the worker stopped for a while for some reason. In such cases, you can use this option as follows.

<system>
  restart_worker_interval 1m
</system>

By this setting, a stopped worker will restart after 1 minute.

See #3768 for more information.

fluent-ctl: Add a new command dump

In this release, we can use a new command dump in fluent-ctl.

This command is mainly for Windows.

As for non-Windows, you can manually send SIGCONT to each supervisor/worker process to dump their internal status. (Dump fluentd's Internal Information)

By using this new command, we can get the internal status on Windows too.

On Windows, this makes all Fluentd processes (including all worker processes) dump their internal status to the system temp directory (C:\\Windows\\Temp).

$ fluent-ctl dump [PID_OR_SVCNAME]

You can omit [PID_OR_SVCNAME] if you run the fluentd as a Windows service, whose service name is the default value fluentdwinsvc.

Please see the document for detailed usage instructions or see #3680 for more information.

Bug Fixes

• Output plugins: Fix a possible crash of flush thread. #3755
• in_tail: Fix crash bugs on Ruby 3.1 on Windows. #3766
• in_tail: Fix a possible crash on file rotation when follow_inodes true. #3754
• in_tail: Fix a bug that in_tail cannot open non-ascii path on Windows. #3774

Miscellaneous bug fixes

• out_forward: Fix to update timeout of cached sockets. #3711
• Fix a bug that fluentd doesn't release its own log file even after rotated by external tools. #3782

Enjoy logging!

Hi users!

We have released v1.15.1. ChangeLog is here.

This release is a maintenance release of v1.15 series.

Bug Fixes

out_file: Add support for concurrent append

Using "append" option of out_file in multiple workers could cause file corruption due to write conflicts. This could happen when multiple workers append to the same file at the same time, although it was very rare.

In this release, we add a lock-system to Fluentd to prevents write conflicts between multiple workers.

This prevents write conflicts of out_file.

See #3808 for more information.

Miscellaneous bug fixes

• in_tail: Show more information on skipping update_watcher. #3829

Enjoy logging!

Hi users!

We have released td-agent v4.4.0. td-agent is a stable distribution package of Fluentd.

Changes from td-agent v4.3.2

• Update bundled Fluentd from v1.14.6 to v1.15.1. Please see each release announcement of Fluentd for more detail:
  • Fluentd v1.15.0 has been released
  • Fluentd v1.15.1 has been released
• Update OpenSSL to 1.1.1q (for macOS and Windows)
• Remove TD_AGENT_OPTIONS= from td-agent.service. Instead it's loaded from /etc/sysconfig/td-agent (rpm) or /etc/default/td-agent (deb).
• Drop Ubuntu 16.04 "Xenial Xerus" support as we announced before.
• Update plugins and gems

Please see ChangeLog for more detail.

Download

Please see the download page.

Hi users!

We have released v1.15.2. ChangeLog is here.

In this release, we fixed the problem that out_file's append mode with --daemon flag failed to flush the buffer.

Bug Fixes

out_file: Fix append mode with --daemon flag

In the previous version v1.15.1, we added a new feature to prevent write conflicts between multiple workers.

• #3808

However, this fix didn't work correctly with --daemon flag of Fluentd command.

• Command Line Option

With that flag, out_file in the append mode fails to flush the buffer. This is a critical bug for td-agent, because td-agent uses --daemon flag by default.

In this release, we fixed this problem.

See #3864 for more information.

Miscellaneous fixes

• Add a new system configuration enable_jit. #3857
• child_process: Plug file descriptor leak. #3844
• Drop win32-api gem to support Ruby 3.2. #3849, #3866

Enjoy logging!

Hi users!

We have released td-agent v4.4.1. td-agent is a stable distribution package of Fluentd.

Changes from td-agent v4.4.0

• Update fluentd to 1.15.2
  • out_file: Fix append mode with --daemon flag
  • Drop win32-api gem dependency
  • Please see also the release announcement of v1.15.2
• Update fluent-plugin-kafka to 0.18.1
  • Fix outkafka2 doesn't respect `chunklimitrecordsandchunklimit_size`

Download

Please see the download page.

Hi users!

We have released td-agent v4.4.2. td-agent is a stable distribution package of Fluentd.

Changes from td-agent v4.4.1

• Update fluentd to 1.15.3
  • Fix log initializer to correctly create per-process files on Windows
  • out_file: Fix the multi-worker check with <worker 0-N> directive
  • Remove meaningless oj options
  • Support glob for !include directive in YAML config format
• macOS: Support Apple Silicon
• Update plugins
  • fluent-plugin-elasticsearch 5.2.4
  • fluent-plugin-opensearch 1.0.8
  • fluent-plugin-s3 1.7.2
  • fluent-plugin-windows-eventlog 0.8.2

Download

Please see the download page.

Hi users!

We have released v1.15.3 on 2022-11-02. ChangeLog is here.

In this release, we fixed several bugs.

Bug Fixes

out_file: Fix the multi-worker check with <worker N-M> directive

In the v1.15.1, we added a new feature to prevent write conflicts between multiple workers.

• #3808

However, this fix didn't work correctly with <worker N-M> directive.

• <worker N-M> directive

With this setting, the configuration check before starting the supervisor mistakenly determines that the setting of out_file is invalid and causes Fluentd to terminate.

In this release, we fixed this problem.

See #3942 for more information.

Fix log rotation on Windows

We can configure log rotation setting as follows.

• Log Rotation Setting

On Windows, the log files must be separated by each process.

NOTE: When --log-rotate-age is specified on Windows, log files are separated into log-supervisor-0.log, log-0.log, ..., log-N.log where N is generation - 1 due to the system limitation. Windows does not permit delete and rename files simultaneously owned by another process.

However, that log rotation setting in the system-directive didn't satisfy this and Fluentd outputed the logs of the supervisor process into the log file of the worker-0 process.

This could cause errors during log rotation.

In this release, we fixed this problem.

See #3939 for more information.

Support glob for !include directive in YAML config format

In the v1.15.0, we supported YAML config format.

• Config File Syntax (YAML)
• #3712

However, handling glob of !include directive was leaked.

In this release, we supported it.

See #3917 for more information.

Remove meaningless and unsecure oj options

In this release, we removed the following oj options.

• FLUENT_OJ_OPTION_MODE=object
• FLUENT_OJ_OPTION_MAX_NESTING

FLUENT_OJ_OPTION_MODE=object was unsecure, so we removed it.

• GHSA-fppq-mj76-fpj2

This option is not used unless a user explicitly sets the environmental variable, so almost all users do not need to care about this vulnerability.

FLUENT_OJ_OPTION_MAX_NESTING was not supported, so we removed it.

• https://github.com/fluent/fluentd/issues/3311#issuecomment-923741031

See #3929 for more information.

Enjoy logging!

Hi users!

We have released v1.16.0 on 2023-03-29. ChangeLog is here.

This release is a new release of v1.16 series. In this release, we enhanced the feature for chunk file corruption and fixed some bugs, mainly about logging and race condition errors.

Enhancement

Backup corrupted chunk files at resuming

Some chunk files in buffer_file or buffer_file_single are possibly corrupted after Fluentd stopped abnormally, such as due to a power failure.

Fluentd has a feature to backup unrecoverable chunks as before. However, when Fluentd resumes, remaining corrupted chunk files were just deleted.

Since this release, Fluentd also backups those files at resuming. In addition, Fluentd outputs some useful logs about possibly corrupted chunk files due to an abnormal termination.

Please see the document and the following PRs for more information.

• buffer: backup broken file chunk
• Add warning messages for restoring buffer with flush_at_shutdown true
• Add logs for time period of restored buffer possibly broken

in_tcp: Add send_keepalive_packet option

Since this release, you can use send_keepalive_packet option for in_tcp, as well as in_forward or in_syslog.

Please see the document and #3961 for more information.

Bug Fixes

• out_secondary_file: Fix race condition. #4081
• out_secondary_file: Suppress warning using different secondary for outsecondaryfile. #4087
• out_forward: Fix error of ack handling conflict with require_ack_response. #4030
• Windows: Fix bug that Fluentd sometimes tries to use an unavailable port and fails to start. #4092
• Windows: Fix bug that flushing or graceful reloading cause reopening wrong log file. #4054
• Fix not to log some initial messages without format setting applied. #4091
• Fix bug that some system configs are not reflected. #4064, #4065
  • suppress_repeated_stacktrace
  • ignore_same_log_interval (It was not reflected only to the supervisor process)
• Non-Windows: Fix not to generate sigdump file after receiving SIGTERM. #4043
• Fix bug that system_config.workers value can be wrongly 1 at config check at startup. #4066
• Fix bug that a variable whose type is not Fluent::Config::Element can be passed to Fluent::Plugin::Base::configure(). #4066

Enjoy logging!