Log bulk request response body on error, not just when debug logging is set (#1096)

Prior to this commit, an error response to a bulk request would only log the
response body when debug logging is enabled. This removes an important piece
of information when tracking down issues, which is particularly important in
this case, as an error response to a bulk request may result in requests being
retried indefinitely, causing pipeline stalls.

This commit adds the response body to the `log_hash` regardless of the log
level,so 429 responses will remain unchanged, and will only log when debug is
enabled, but other error responses will include the response body to add this
additional context to help track down issues

Closes: #999

Author: robbavey

CHANGELOG.md

@@ -1,3 +1,6 @@
+## 11.12.1
+ - Log bulk request response body on error, not just when debug logging is enabled [#1096](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1096)
+
 ## 11.12.0
  - Add legacy template API support for Elasticsearch 8 [#1092](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1092)
 

lib/logstash/plugin_mixins/elasticsearch/common.rb

@@ -335,8 +335,8 @@ def safe_bulk(actions)
         retry unless @stopping.true?
       rescue ::LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError => e
         @bulk_request_metrics.increment(:failures)
-        log_hash = {:code => e.response_code, :url => e.url.sanitized.to_s, :content_length => e.request_body.bytesize}
-        log_hash[:body] = e.response_body if @logger.debug? # Generally this is too verbose
+        log_hash = {:code => e.response_code, :url => e.url.sanitized.to_s,
+                    :content_length => e.request_body.bytesize, :body => e.response_body}
         message = "Encountered a retryable error (will retry with exponential backoff)"
 
         # We treat 429s as a special case because these really aren't errors, but

logstash-output-elasticsearch.gemspec

@@ -1,6 +1,6 @@
 Gem::Specification.new do |s|
   s.name            = 'logstash-output-elasticsearch'
-  s.version         = '11.12.0'
+  s.version         = '11.12.1'
   s.licenses        = ['apache-2.0']
   s.summary         = "Stores logs in Elasticsearch"
   s.description     = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"