Integration Server 11.1 | Integration Server Administrator's Guide | Statistics Data Collector Configuration Parameters
 
Statistics Data Collector Configuration Parameters
The Dashboard visible in Integration Server Administrator displays key system and server metrics. The Statistics Data Collector is the Integration Server component that captures these metrics, stores them in the relational database, and then transfers the metrics to an indexed file-based repository. The Dashboard retrieves metrics from the repository. The Statistics Data Collector writes data to the database to which the ISDashboardStats functional alias points using the JDBC connection pool associated with the ISDashboardStats functional alias. Periodically, the Statistics Data Collector polls the database and indexes any new data. The Statistics Data Collector uses Apache Lucene for searching and indexing data.
Note:
Prior to Integration Server 10.15, the Statistics Data Collector used the ISInternal functional alias to write statistics to the associated database. When you start Integration Server 10.15, the Statistics Data Collector begins writing to the database using a connection from the JDBC connection pool associated with the ISDashboardStats functional alias. Data that existed in the database prior to the change of the functional alias is retained.
Statistics Data Collector deletes old data in batches using a scheduled system task that runs every 4 hours. (Note this is a JVM system task, not an Integration Server system task). For every scheduled delete task, Statistics Data Collector deletes the batches of data collected over a 4 hour period starting with the oldest data. The delete task performs batch deletions every 30 seconds until all data older than the retention date is removed the retention date is controlled by the retentionDays parameter, which is described below.
The Statistics Data Collector provides configuration parameters that control behavior such as whether or not the Statistics Data Collector is enabled, thread usage, searching, indexing, and storage. The default configuration may be suitable for most installations, however some environments, such as a production environment or an Integration Server acting as the Enterprise Gateway Server may benefit from different values for these parameters.
Note:
On an Enterprise Gateway Server, statistics data collection is disabled by default.
The key settings for the Statistics Data Collector can be configured via the Dashboard configuration which is available by selecting > Dashboard configuration on the Dashboard. For more information about Dashboard configuration, see Configuring the Dashboard.
Most of the settings for the Statistics Data Collector can be configured through configuration parameters available in the monitor.cnf which is located in Integration Server_directory/instances/instanceName/config/monitor/monitor.cnf. You can edit this file using a text editor.
After you edit the file, you must restart Integration Server for changes take effect.
The following table lists and describes the configuration parameters available in monitor.cnf.
Parameter
Description
charset
Character encoding used for resource files when data fetched by Lucene Monitor is indexed.
Default: UTF-8
enabled
Whether or not the Statistics Data Collector is enabled. When disabled, the Statistics Data Collector does not consume any resource or add any overhead to Integration Server.
When the Statistics Data Collector is disabled, many of the tables and graphs on the Dashboard will be empty.
Default: true
Note:
On an Enterprise Gateway Server, statistics data collection is disabled by default.
fileBlockSize
Number of bytes to use for buffered block reads of data.
Default: 4096
indexerThreadCount
Number of resources that may be indexed concurrently. If there are more index tasks than there are available threads, the tasks wait until a thread becomes available. Indexer threads are from a separate thread pool; that is, indexer threads are not from the server thread pool. The minimum value is 2.
Default: 8
indexInterval
Number of seconds between invocations of indexing tasks for any new data.
Default: 90
jvmThreadStatsCollection
Whether or not the Statistics Data Collector gathers metrics about JVM thread states. This information is displayed in the Thread summary graph on the JVM metrics tab in the Dashboard. When jvmThreadStatsCollection is set to false, the Thread summary graph on the JVM metrics tab does not display any data.
Default: true
Note:
Statistics Data Collector captures JVM thread state information every 60 seconds (this is not configurable). During collection of thread state information, the Statistics Data Collector pauses all running JVM threads for an instant while the state of each thread is captured. Under high loads, this pause may be noticeable.
luceneMemorySize
Number of megabytes of memory in the JVM to dedicate to Lucene search. The Statistics Data Collector uses Apache Lucene for searching and indexing data. The minimum value is 1.
Default: 256
Note:
The default value for luceneMemorySize is 256 MB, which is the recommended minimum value. Specifying less memory may cause performance problems or runtime errors with the Statistics Data Collector.
maxJdbcPoolConnections
Number of concurrent JDBC connections from the JDBC pool used by the ISDashboardStats function that can be used by the Statistics Data Collector. The minimum value is 4.
Default: 4
Note:
The JDBC pool alias used by the ISDashboardStats function must have a Maximum Connections value that supports the number of connections that can be used with the Statistics Data Collector.
maxResults
Maximum number of results permitted by the Statistics Data Collector. The minimum value is 1.
Default: 1500000
maxTextLength
The maximum amount of textual data, measured in bytes, that may be stored in Lucene. The maxTextLength cannot be larger than the maximum CLOB size supported by the installed RDBMS.
Default: 65535
Note:
The maxTextLength value also determines the longest line-delimited string that can be stored in the Statistics Data Collector repository. Statistics Data Collector truncates strings that exceed this limit to this value. The term Statistics Data Collector repository refers to the collection of files located in Integration Server_directory/instances/instanceName/monitorRepo/core.
mergeInterval
Number of seconds between invocations of merge tasks on the Lucene index. A merge task reclaims disk space by removing Lucene documents deleted from the index since the last merge operation.
Default: 21600
mirrored
This parameter is not used.
monitorTimeDelta
Number of milliseconds that sets the threshold for which a Statistics Data Collector event taking longer than this amount of time results in Integration Server writing a warning message to the server log. Only internal monitoring events related to the health and performance of the Statistics Data Collector can trigger the warning message. This value must be greater than 250 ms.
Default: 30000
networkBlockSize
Number of bytes to use for network operations.
Default: 4096
pageSize
The batch size for rows of data for the SQL query results and for inserting data.
Default: 512
Note:
If the ISDashboardStats function writes to MySQL 5.7 Community Edition, set this value to Integer.MIN_VALUE: -2147483648. Otherwise, the Statistics Data Collector caches all the results in memory at once which can lead to OutOfMemoryExceptions with large result set sizes.
pollingInterval
Length of the polling interval, measured in seconds, for repeating system tasks in Statistics Data Collector such as indexing, merging, logging, and batch JDBC inserts. The maximum value is 30 seconds.
Default: 5
restartHours
Frequency with which the Statistics Data Collector restarts. The purpose of the restart is to minimize the disk file consumption produced by the document index from an embedded third-party library (org.apache.lucene). That is, restarting the Statistics Data Collector reduces the size of the Integration Server_directory/instances/instanceName/monitorRepo directory. Restart of the Statistics Data Collector does not affect a running Integration Server.
Default: 6
retentionDays
Maximum number of days to retain data in the database used by the Statistics Data Collector. Periodically, the Statistics Data Collector removes any indexed Lucene documents older than the retentionDays value. Integration Server also removes any captured data that is older than retentionDays and stored in relational tables or flat files. The minimum value is 1.
Default: 2
searcherThreadCount
Number of search requests that can execute concurrently. If the number of search requests exceeds this value, the pending search requests block until a currently executing search request completes. The minimum value is 2.
Default: 2
timeSlice
This parameter is not used.
Note:
Integration Server also includes server configuration parameters that can impact the Statistics Data Collector. For example, the watt.server.statisticsDataCollector.dbInsert.secsTimeout parameter specifies the number of seconds that the Statistics Data Collector waits when attempting to insert metrics into the database. Server configuration parameters that impact the Statistics Data Collector begin with "watt.server.stasticsDataCollector".