Logging Configurations
To troubleshoot any operational issues efficiently, it is crucial to manage the log files. It enables you to track and analyze the activity, usage, problems, and security like, user access and critical configuration changes. It helps you to identify unexpected anomalies in logs. Additionally, it is also important to add the configurations related to log rotation and retention settings.
This section provides recommendations on configuring the log levels of every component of API Gateway to enable automatic log rotation. By default, the log files are stored in the following locations:
API Gateway logs:
SAG_Install_Directory\IntegrationServer\instances\instance_name\logs.
SAG_Install_Directory\profiles\IS_instance_name\logs.
External Elasticsearch logs:
SAG_Install_Directory\logs.
External Elasticsearch logs:
SAG_Install_Directory\profiles\IS_instance_name\apigateway\dashboard\config or Kibana_Install_Directory\config.
Note:
By default, the log files are in INFO mode.
Log File Rotation Settings
API Gateway
To enable automatic log rotation for the API Gateway server, follow these logging guidelines. You must have the Manage user administration functional privilege assigned to configure the watt parameters in API Gateway UI for server log and audit log. You can configure the watt parameters in the Watt keys section under API Gateway UI > Administration > General > Extended settings > Show and hide keys by providing the recommended values.
Server.log
Log level of the API Gateway server.
Server log file server.cnf is located at SAG_Install_Dir\IntegrationServer\instances\instance_name\config.
To configure the total size of the logs as 1 GB, set the following values to the corresponding properties in the Watt keys section under API Gateway UI > Administration > General > Extended settings > Show and hide keys .
watt.server.serverlogFilesToKeep=100
watt.server.serverlogRotateSize=10MB
Audit.log
IBM logs the audit information for different categories of system transactions and events.
Audit log file server.cnf is located at SAG_Install_Dir\IntegrationServer\instances\instance_name\config.
To configure the total size of the logs as 1 GB, set the following values to the corresponding properties in the Watt keys section under API Gateway UI > Administration > General > Extended settings > Show and hide keys .
watt.server.audit.logFilesToKeep=100
watt.server.audit.logRotateSize=10MB
Osgi.log
Log level of the Osgi file type.
Osgi log file log4j2.properties is located at SAG_Install_Dir\profiles\IS_instance_name\configuration\logging.
To configure the total size of the logs as 300 MB, set the following values in log4j2.properties log file and save the file.
appender.rolling.policies.size=10MB
appender.rolling.strategy.max=30
Wrapper.log
Log level of the Wrapper file type.
Wrapper log file custom_wrapper.conf is located at SAG_Install_Dir\profiles\IS_instance_name\configuration.
To configure the total size of the logs as 300 MB, add the following properties and its values as suggested in custom_wrapper.conf log file and save it.
wrapper.logfile.maxfiles=30
wrapper.logfile.maxsize=10MB
External Elasticsearch
To enable automatic log rotation for external Elasticsearch, follow these logging guidelines. For more information about external Elasticsearch, see
Elasticsearch documentation.
elasticsearch.log
Log level of external Elasticsearch.
External Elasticsearch log file log4j2.properties is located at SAG_Install_Dir\config.
It is recommended to set the following properties for the rolling file on log4j2.properties.
#Condition and Action to apply when handling roll overs
appender.rolling.strategy.action.condition.nested_condition.type = IfAny
#Perform the actions only if you have accumulated too many logs
appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
# The size condition on the compressed logs is 512 MB
appender.rolling.strategy.action.condition.nested_condition.exceeds = 512MB
# A nested condition to apply to files matching the glob
appender.rolling.strategy.action.condition.nested_condition.lastMod.type = IfLastModified
# Retains logs for seven days
appender.rolling.strategy.action.condition.nested_condition.lastMod.age = 7D
The properties for the old style pattern appenders is as follows. If the log4j2.properties in your system uses the old style layout of appenders, set the configurations for the following properties. Note that these should be considered as deprecated and can be removed in the future.
appender.rolling_old.strategy.action.condition.nested_condition.type = IfAny
appender.rolling_old.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
appender.rolling_old.strategy.action.condition.nested_condition.exceeds = 512MB
appender.rolling_old.strategy.action.condition.nested_condition.lastMod.type = IfLastModified
appender.rolling_old.strategy.action.condition.nested_condition.lastMod.age = 7D
If the file size is 512MB or last modified date is 7D, log rotation is enabled.
A sample configuration is as follows:
status = error
# log action execution errors for easier debugging
logger.action.name = org.elasticsearch.action
logger.action.level = debug
appender.rolling.type = Console
appender.rolling.name = rolling
appender.rolling.layout.type = ESJsonLayout
appender.rolling.layout.type_name = server
appender.rolling.strategy.action.condition.nested_condition.type = IfAny
appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
appender.rolling.strategy.action.condition.nested_condition.exceeds = 512MB
appender.rolling.strategy.action.condition.nested_condition.lastMod.type = IfLastModified
appender.rolling.strategy.action.condition.nested_condition.lastMod.age = 7D
appender.rolling_old.strategy.action.condition.nested_condition.type = IfAny
appender.rolling_old.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
appender.rolling_old.strategy.action.condition.nested_condition.exceeds = 512MB
appender.rolling_old.strategy.action.condition.nested_condition.lastMod.type = IfLastModified
appender.rolling_old.strategy.action.condition.nested_condition.lastMod.age = 7D
rootLogger.level = info
rootLogger.appenderRef.rolling.ref = rolling
appender.deprecation_rolling.type = Console
appender.deprecation_rolling.name = deprecation_rolling
appender.deprecation_rolling.layout.type = ESJsonLayout
appender.deprecation_rolling.layout.type_name = deprecation
logger.deprecation.name = org.elasticsearch.deprecation
logger.deprecation.level = warn
logger.deprecation.appenderRef.deprecation_rolling.ref = deprecation_rolling
logger.deprecation.additivity = false
appender.index_search_slowlog_rolling.type = Console
appender.index_search_slowlog_rolling.name = index_search_slowlog_rolling
appender.index_search_slowlog_rolling.layout.type = ESJsonLayout
appender.index_search_slowlog_rolling.layout.type_name = index_search_slowlog
logger.index_search_slowlog_rolling.name = index.search.slowlog
logger.index_search_slowlog_rolling.level = trace
logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling.ref = index_search_slowlog_rolling
logger.index_search_slowlog_rolling.additivity = false
appender.index_indexing_slowlog_rolling.type = Console
appender.index_indexing_slowlog_rolling.name = index_indexing_slowlog_rolling
appender.index_indexing_slowlog_rolling.layout.type = ESJsonLayout
appender.index_indexing_slowlog_rolling.layout.type_name = index_indexing_slowlog
logger.index_indexing_slowlog.name = index.indexing.slowlog.index
logger.index_indexing_slowlog.level = trace
logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling.ref = index_indexing_slowlog_rolling
logger.index_indexing_slowlog.additivity = false
Note:
Log4j’s configuration parsing does not recognize extraneous whitespaces. Ensure to trim any leading and trailing whitespace when you copy the configurations.
External Kibana
To enable automatic log rotation for external Kibana, follow these logging guidelines. For more information about external Kibana, see
https://www.elastic.co/guide/en/kibana/current/introduction.html.
kibana.log
Log level of external Kibana.
External Kibana log file kibana.yml is located at SAG_Install_Dir\profiles\IS_instance_name\apigateway\dashboard\config or Kibana_InstallDirectory\config.
Provide values for the following properties based on your logging requirements in kibana.yml log file and save it.
# =================== System: Logging ===================
#Set the value of this setting to off to suppress all logging output, or to debug to log everything. Defaults to 'info'
# ===================
logging.root.level: debug
# Enables you to specify a file where Kibana stores log output.
# ===================
logging.appenders.default:
type: file
fileName: /var/logs/kibana.log
layout:
type: json
For example, to configure the total size of the logs as 300 MB, provide 300mb in the property logging.appenders[].<appender-name>.policy.size:
logging:
appenders:
json-file-appender:
type: rolling-file
fileName: ../logs/startup.log
policy:
type: size-limit
size: 10mb
strategy:
type: numeric
pattern: '-%i'
max: 30
layout:
type: json
root:
appenders: [json-file-appender]
level: info