Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/_docs/code-snippets/xml/configuring-metrics.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
<!-- tag::data-region-metrics[] -->
<property name="defaultDataRegionConfiguration">
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<!-- enable mertrics for the default data region -->
<!-- enable metrics for the default data region -->
<property name="metricsEnabled" value="true"/>
<!-- other properties -->
</bean>
Expand All @@ -40,8 +40,6 @@
<property name="name" value="myDataRegion"/>
<!-- Enable metrics for this data region -->
<property name="metricsEnabled" value="true"/>

<property name="persistenceEnabled" value="true"/>
<!-- other properties -->
</bean>
</list>
Expand Down
2 changes: 1 addition & 1 deletion docs/_docs/code-snippets/xml/persistence-metrics.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="persistenceEnabled" value="true"/>

<!-- enable mertrics for the default data region -->
<!-- enable metrics for the default data region -->
<!--property name="metricsEnabled" value="true"/-->
<!-- other properties -->
</bean>
Expand Down
Binary file modified docs/_docs/images/jconsole.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 10 additions & 7 deletions docs/_docs/monitoring-metrics/configuring-metrics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ For this reason, some metrics are disabled by default.
Cache metrics show statistics on the amount of data stored in caches, the total number and frequency of cache operations, etc. as well as some cache configuration properties for information purposes.

To enable cache metrics, use one of the methods described below for each cache you want to monitor.
Cache metrics can also be enabled or disabled at runtime, without restarting the nodes, via
`IgniteCluster.enableStatistics(...)` or the link:tools/control-script#manage-cache-metrics-collection[control script].

[tabs]
--
Expand All @@ -51,12 +53,10 @@ tab:C++[unsupported]

== Enabling Data Region Metrics
Data region metrics expose information about data regions, including memory and storage size of the region.
Enable data region metrics for every region you want to collect the metrics for.

Data region metrics can be enabled in two ways:

* in the link:memory-configuration/data-regions[configuration of the region]
* via JMX Beans
Enable data region metrics in the link:memory-configuration/data-regions[configuration of the region] for every region
you want to collect the metrics for.
The setting is read at node startup and cannot be changed at runtime.
Some metrics, such as `TotalAllocatedSize`, are collected regardless of it.

The following example illustrates how to enable metrics for the default data region and one custom data region.

Expand All @@ -81,7 +81,10 @@ tab:C++[unsupported]
--

== Enabling Persistence-related Metrics
Persistence-related metrics can be enabled/disabled in the data storage configuration:
The `io.datastorage` register (WAL, checkpoint and storage size metrics) is collected only when persistence metrics are
enabled in the data storage configuration; otherwise its metrics report zero, except `LastArchivedSegment`, which
reports `-1`.
The property is deprecated but still required; it is read at node startup and cannot be changed at runtime.

[tabs]
--
Expand Down
76 changes: 54 additions & 22 deletions docs/_docs/monitoring-metrics/new-metrics-system.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Some metrics represent <<histograms>>.

And then there are different ways to export the metrics — what we call _exporters_.
To put it another way, the exporter are different ways you can access the metrics.
Each exporter always gives access to all available metrics.
Unless an export filter is configured, each exporter gives access to all available metrics.

Ignite includes the following exporters:

Expand All @@ -52,15 +52,17 @@ The list of all registries and the metrics they contain are described link:monit

== Metric Exporters

If you want to enable metrics, configure one or multiple metric exporters in the node configuration.
This is a node-specific configuration, which means it enables metrics only on the node where it is specified.
Metrics are collected regardless of the exporters configured; exporters only define how the metrics are exposed.
By default, a node registers the JMX exporter and, if an SQL engine module is on the classpath, the `SYS.METRICS` view.
To change the set of exporters, configure them in the node configuration.
This is a node-specific configuration, which means it affects only the node where it is specified.

[tabs]
--
tab:XML[]
[source, xml]
----
include::code-snippets/xml/metrics.xml[tags=ignite-config;!discovery, indent=0]
include::code-snippets/xml/metrics.xml[tags=ignite-config;!discovery;!opencensus-exporter, indent=0]
----

tab:Java[]
Expand Down Expand Up @@ -112,19 +114,18 @@ tab:C#/.NET[]
tab:C++[unsupported]
--

==== Enabling JMX for Ignite

By default, the JMX automatic configuration is disabled.
To enable it, configure the following environment variables:

* For `control.sh`, configure the `CONTROL_JVM_OPTS` variable
* For `ignite.sh`, configure the `JVM_OPTS` variable
==== Enabling Remote JMX Access

Ignite registers its MBeans in the platform MBean server of the node process, so a local JMX client, such as JConsole
running on the same host, needs no additional configuration.
To connect from another host, enable remote JMX with the standard JVM options.
For `ignite.sh`, add them to the `JVM_OPTS` environment variable; for a node embedded into an application, pass them to
the JVM of the application.
For example:

[source,shell]
----
JVM_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=${JMX_PORT} \
export JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=<port> \
-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"
----

Expand All @@ -138,23 +139,34 @@ The ObjectName consists of a domain and a list of key properties, and can be rep

domain: key1 = value1 , key2 = value2

All Ignite metrics have the same domain: `org.apache.<classloaderId>` where the classloader ID is optional (omitted if you set `IGNITE_MBEAN_APPEND_CLASS_LOADER_ID=false`). In addition, each metric has two properties: `group` and `name`.
For example:
All Ignite MBeans have the same domain: `org.apache`. The key properties are:

org.apache:group=SPIs,name=TcpDiscoverySpi
* `clsLdr` - the hash code of the class loader that loaded Ignite, as a hex string. Present by default; omitted if you
set the `IGNITE_MBEAN_APPEND_CLASS_LOADER_ID` system property to `false`.
* `jvmId` - the name of the JVM (`<pid>@<host>`). Present only if the `IGNITE_MBEAN_APPEND_JVM_ID` system property is
set to `true`.
* `igniteInstanceName` - the instance name of the node. Present only if the node has one.
* `group` and `name` - for a metric register, `group` is the part of the register name before the first dot and `name`
is the rest. Registers without a dot in the name, such as `tx`, have no `group` property.

This MBean provides various metrics related to node discovery.
For example, the `io.discovery` and `tx` registers are exposed as:

org.apache:clsLdr=5ffd2b27,group=io,name=discovery
org.apache:clsLdr=5ffd2b27,name=tx

The MBean ObjectName can be used to identify the bean in UI tools like JConsole.
For example, JConsole displays MBeans in a tree-like structure where all beans are first grouped by domain and then by the 'group' property:
For example, JConsole displays MBeans in a tree-like structure where beans are grouped by domain and then by the key
properties in the order listed above:

image::images/jconsole.png[]

{sp}+

=== SQL View

`SqlViewMetricExporterSpi` is enabled by default, `SqlViewMetricExporterSpi` exposes metrics via the `SYS.METRICS` view.
The `SYS.METRICS` view is registered automatically on every node that has an SQL engine module (`ignite-indexing` or
`ignite-calcite`) on the classpath.
It does not need to be configured and cannot be filtered.
Each metric is displayed as a single record.
You can use any supported SQL tool to view the metrics:

Expand Down Expand Up @@ -304,9 +316,19 @@ To monitor the size of the persistent storage on disk, use the following metrics

The metrics belong to the `io.datastorage` register, see link:monitoring-metrics/new-metrics#data-storage[Data Storage].

[NOTE]
====
The `io.datastorage` register is collected only when persistence metrics are enabled in the
link:monitoring-metrics/configuring-metrics#enabling-persistence-related-metrics[data storage configuration]; otherwise
its metrics report zero, except `LastArchivedSegment`, which reports `-1`.
====

===== Data Region Size

Metrics collection for data regions is disabled by default. You can link:monitoring-metrics/configuring-metrics#enabling-data-region-metrics[enable it in the data region configuration].
`PagesFillFactor` requires data region metrics to be enabled in the
link:monitoring-metrics/configuring-metrics#enabling-data-region-metrics[data region configuration], and so do
`PhysicalMemoryPages` and `PhysicalMemorySize` for regions with persistence enabled; the other metrics listed below are
always collected.

The size of the data region on a node comprises the size of all partitions (including backup partitions) that this node owns for all caches in that data region.

Expand Down Expand Up @@ -338,7 +360,11 @@ See link:persistence/persistence-tuning#pages-writes-throttling[Pages Writes Thr
* `LastCheckpointDuration` - The time in milliseconds it took to create the last checkpoint.
* `CheckpointBufferSize` - The size of the checkpointing buffer.

The metrics belong to the `io.datastorage` register, see link:monitoring-metrics/new-metrics#data-storage[Data Storage].
The metrics belong to the `io.datastorage` register, see link:monitoring-metrics/new-metrics#data-storage[Data Storage];
they require persistence metrics to be enabled in the
link:monitoring-metrics/configuring-metrics#enabling-persistence-related-metrics[data storage configuration].
`DirtyPages` and `CheckpointBufferSize` are summed over data regions, so they also require
link:monitoring-metrics/configuring-metrics#enabling-data-region-metrics[data region metrics] to be enabled.

=== Monitoring Rebalancing
link:data-rebalancing[Rebalancing] is the process of moving partitions between the cluster nodes so that the data is always distributed in a balanced manner. Rebalancing is triggered when a new node joins, or an existing node leaves the cluster.
Expand All @@ -355,7 +381,11 @@ Topology refers to the set of nodes in a cluster. There are a number of metrics

* `TotalServerNodes` - The number of server nodes in the cluster.
* `TotalClientNodes` - The number of client nodes in the cluster.
* `TotalBaselineNodes` - The number of nodes that are registered in the link:clustering/baseline-topology[baseline topology]. When a node goes down, it remains registered in the baseline topology and you need to remove it manually.
* `TotalBaselineNodes` - The number of nodes that are registered in the
link:clustering/baseline-topology[baseline topology]. When a node goes down, it remains registered in the baseline
topology and you need to remove it manually, unless
link:clustering/baseline-topology#baseline-topology-autoadjustment[baseline topology autoadjustment] is enabled (the
default for in-memory clusters).
* `ActiveBaselineNodes` - The number of nodes that are currently active in the baseline topology.

The metrics belong to the `cluster` register, see link:monitoring-metrics/new-metrics#cluster[Cluster].
Expand Down Expand Up @@ -392,7 +422,9 @@ For a more accurate estimate, use the `IndexRebuildKeyProcessed` cache metric:
* Use `IndexRebuildKeyProcessed` to know the number of keys with rebuilt indexes. If the rebuilding is in progress, it gives a number of keys with indexes being rebuilt at the current moment. Otherwise, it gives a total number of the of keys with rebuilt indexes. The values are reset before the start of each rebuilding.

=== Monitoring Transactions
Note that if a transaction spans multiple nodes (i.e., if the keys that are changed as a result of the transaction execution are located on multiple nodes), the counters will increase on each node. For example, the 'TransactionsCommittedNumber' counter will increase on each node where the keys affected by the transaction are stored.
The commit and rollback counters are updated only on the node that started the transaction, no matter how many nodes
store the affected keys, so the sum of `txCommits` over all nodes, including client nodes, is the number of
transactions committed in the cluster.

* `LockedKeysNumber` - The number of keys locked on the node.
* `TransactionsCommittedNumber` - The number of transactions that have been committed on the node. Exposed as `txCommits` in the `tx` register.
Expand Down
Loading