[WIP] Add guides for collecting data using the nginx OTel input package #4725
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
✅ Vale Linting ResultsNo issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
Contributor
vishaangelova
force-pushed
the
1877-nginx-integration
branch
from
c764290 to
b35fdaf
Compare
vishaangelova
changed the title
vishaangelova
changed the title
vishaangelova
force-pushed
the
1877-nginx-integration
branch
from
fc85b9a to
510c7f4
Compare
Contributor
alexandra5000
left a comment
alexandra5000
left a comment
There was a problem hiding this comment.
Sorry I didn't have time to review the last file yet! Hopefully my comments will be useful to you - some of them may apply to that file as well. I'll go back to it on Monday morning.
| ## Configure OpenTelemetry input packages | ||
| ## Hybrid agents [otel-integrations-hybrid-agents] | ||
|
|
||
| {{agent}} policies can include configurations for both ECS-based integrations and OpenTelemetry input packages, which essentially turns the enrolled {{agents}} into *hybrid agents*. |
Contributor
There was a problem hiding this comment.
Suggested change
| {{agent}} policies can include configurations for both ECS-based integrations and OpenTelemetry input packages, which essentially turns the enrolled {{agents}} into *hybrid agents*. | |
| You can configure {{agent}} policies to use both ECS-based integrations and OpenTelemetry input packages. This effectively turns enrolled {{agents}} into hybrid agents. |
For more direct presentation of information.
|
|
||
| Hybrid agent policies are useful when you want to: | ||
|
|
||
| - Keep an ECS-based integration for one data type (for example, to ingest logs and use built-in dashboards) |
Contributor
There was a problem hiding this comment.
Why not "Use" for both?
| - Keep an ECS-based integration for one data type (for example, to ingest logs and use built-in dashboards) | ||
| - Use an OpenTelemetry input package for another data type (for example, to collect metrics using OpenTelemetry standards) | ||
|
|
||
| This type of hybrid data collection provides flexibility without forcing a single ingestion model, and allows you to: |
Contributor
There was a problem hiding this comment.
Suggested change
| This type of hybrid data collection provides flexibility without forcing a single ingestion model, and allows you to: | |
| This type of hybrid data collection provides flexibility without locking you into a single ingestion model, and allows you to: |
Forcing sounds unnecessarily brutal 😄
|
|
||
| The installation and configuration of OpenTelemetry input packages is similar to that of ECS-based integrations and allow you to specify the namespace, dataset name, data stream type, and more. For more information, refer to [Add an integration to an {{agent}} policy](/reference/fleet/add-integration-to-policy.md). | ||
| - Gradually migrate toward OpenTelemetry | ||
| - Standardize ingestion on OpenTelemetry receivers at scale |
Contributor
There was a problem hiding this comment.
Suggested change
| - Standardize ingestion on OpenTelemetry receivers at scale | |
| - Standardize ingestion using OpenTelemetry receivers at scale |
How about like this?
|
|
||
| ## Configure OpenTelemetry input packages [configure-otel-input-packages] | ||
|
|
||
| The installation and configuration of OpenTelemetry input packages are similar to those of ECS-based integrations, and allow you to specify the namespace, dataset name, data stream type, and more. For more information, refer to [Add an integration to an {{agent}} policy](/reference/fleet/add-integration-to-policy.md). |
Contributor
There was a problem hiding this comment.
Suggested change
| The installation and configuration of OpenTelemetry input packages are similar to those of ECS-based integrations, and allow you to specify the namespace, dataset name, data stream type, and more. For more information, refer to [Add an integration to an {{agent}} policy](/reference/fleet/add-integration-to-policy.md). | |
| The installation and configuration of OpenTelemetry input packages are similar to those of ECS-based integrations, and allow you to specify the namespace, dataset name, data stream type, and more. For step-by-step instructions, refer to [Add an integration to an {{agent}} policy](/reference/fleet/add-integration-to-policy.md). |
| 7. Select **Save and continue**. | ||
| ::::{note} | ||
| The NGINX OpenTelemetry Assets content package, which includes relevant assets for visualizing OTel-based metrics, is automatically installed when data is ingested through the NGINX OpenTelemetry Input Package integration. You can then find the content package in the **Installed integrations** list. |
Contributor
There was a problem hiding this comment.
Suggested change
| The NGINX OpenTelemetry Assets content package, which includes relevant assets for visualizing OTel-based metrics, is automatically installed when data is ingested through the NGINX OpenTelemetry Input Package integration. You can then find the content package in the **Installed integrations** list. | |
| The NGINX OpenTelemetry Assets content package is installed automatically when data is ingested through this integration. You can find it under **Installed integrations** and use it to visualize OTel-based metrics. |
Reduce wordiness
| :::: | ||
| ::::{note} | ||
| OpenTelemetry input packages are distinct from [running {{agent}} as an EDOT Collector](/reference/fleet/otel-agent.md), and cannot be used with {{agent}} running in `otel` mode. |
Contributor
There was a problem hiding this comment.
That's interesting. As far as I know the otel mode pretty soon will be the only one available for the EDOT Collector. Are we actually encouraging the use of contrib OTel Collector? Is this the only supported way of using these packages? If so, this guide could benefit from stating that explicitly somewhere at the beginning. I get you said "OpenTelemetry" at the beginning but it wasn't super clear to me that EDOT isn't supported in this case.
| ## Validate your data | ||
| After you apply the policy changes, validate both the ECS-based logs and the OTel-based metrics. |
Contributor
There was a problem hiding this comment.
Suggested change
| After you apply the policy changes, validate both the ECS-based logs and the OTel-based metrics. | |
| After you apply the policy changes, validate that both the ECS-based logs and the OTel-based metrics are flowing in. |
Just for clarity what to validate exactly 🙂
| data_stream.dataset : "nginx.access" or "nginx.error" | ||
| ``` | ||
| 3. Go to **Dashboards**, then select **[Logs Nginx] Access and error logs** to view the dashboard installed with the Nginx integration. |
Contributor
There was a problem hiding this comment.
Would that be the the ECS-based dashboard?
| Go to **Dashboards**, then select **[Metrics Nginx OTEL] Overview** to view the dashboard for visualizing OTel-based metrics. | ||
| This dashboard becomes available with the NGINX OpenTelemetry Assets content package, which is automatically installed when data is ingested trough the NGINX OpenTelemetry Input Package integration. |
Contributor
There was a problem hiding this comment.
Maybe like this:
Suggested change
| This dashboard becomes available with the NGINX OpenTelemetry Assets content package, which is automatically installed when data is ingested trough the NGINX OpenTelemetry Input Package integration. | |
| This dashboard is provided by the NGINX OpenTelemetry Assets content package, installed automatically when data is ingested through the NGINX OpenTelemetry Input Package. |
?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR updates the Collect OpenTelemetry data with Elastic Agent integrations doc with more information on hybrid agents.
It also adds guides for configuring a hybrid Fleet-managed agent and a hybrid standalone agent to collect:
Resolves elastic/ingest-docs#1877
Generative AI disclosure
Tool(s) and model(s) used: Version: 2.3.41 with gpt-5.2