API Gateway 11.1 | Administering API Gateway | Deployment | Deployment Configurations | Connecting to an External Elasticsearch
 
Connecting to an External Elasticsearch
 
Troubleshooting Tips: External Elasticsearch
This section explains the changes that you must make in the config.properties file to enable API Gateway to communicate with the external Elasticsearch.
You can also connect to an external Elasticsearch through externalized configurations. For more details, see Externalizing Configurations .
Note:
If you use an external Elasticsearch with same version as external Elasticsearch, then you can use the external Kibana or dashboard that is shipped with API Gateway, else they have to be configured separately. If you have configured Elasticsearch externally, then you have to configure external Kibana externally. To know the compatible external Kibana versions for your external Elasticsearch, see https://www.elastic.co.
Important:
When you use an external Elasticsearch, you must use the external Elasticsearch plugin mapper size. Without this plugin, API Gateway does not start. To install the mapper size plugin, use the command sudo bin/elasticsearch-plugin install mapper-size. If you have deployed a clustered environment and have configured external Elasticsearch on multiple nodes, you must install this plugin on all the nodes that use external Elasticsearch.
*To connect to an external Elasticsearch
1. Navigate to Install_Dir/msr/IntegrationServer/packages/WmAPIGateway/config/resources/elasticsearch/config.properties
The config.properties file contains all the properties and external Elasticsearch configurations.
Note:
To exclusively route analytics data to a separate external Elasticsearch instance, you need to configure the required properties in the following file - SAGInstallDir/msr/IntegrationServer/packages/WmAPIGateway/config/resources/elasticsearch/analyticsds.properties
2. Configure the following properties:
Property and Description
pg.gateway.elasticsearch.autostart
This property specifies whether the external Elasticsearch starts automatically. If an external Elasticsearch is configured it has to be manually started. This property needs to be set to false to avoid external Elasticsearch starting automatically.
Default value: true
pg.gateway.elasticsearch.client.http.response.size
This property specifies the response size, in MB, for API Gateway external Elasticsearch client.
Default value: 100
pg.gateway.elasticsearch.config.location
This property specifies the location of the config file if you want to read port details from some other external Elasticsearch config file
pg.gateway.elasticsearch.hosts
Mandatory
This property lists external Elasticsearch hosts and ports. The values are comma separated.
Default value: localhost:9240
Note:
Once a host is added to this property, this is the value that is used to connect to external Elasticsearch and the host configured in gateway-es-store.xml is not considered.
pg.gateway.elasticsearch.http.keepAlive
Mandatory
This property creates the persistent connection between client and server.
Default value: true
pg.gateway.elasticsearch.http.connectionTimeout
Mandatory
This property specifies the time, in milliseconds, after which the connection times out.
Default value: 10000
pg.gateway.elasticsearch.http.socketTimeout
Mandatory
This property specifies the wait time, in milliseconds, for a reply once the connection to external Elasticsearch is established after which it times out.
Default value: 30000
pg.gateway.elasticsearch.http.maxRetryTimeout
Mandatory
This property specifies the wait time, in milliseconds, for retries after which it times out.
Default value: 100000
It is advisable to set max retry time for a request to (number of nodes * socketTimeOut )+connectionTimeout
pg.gateway.elasticsearch.http.keepAlive.maxConnections
Mandatory
This property specifies the maximum number of persistent connections that can be established between an API Gateway and external Elasticsearch cluster.
Default value: 50
pg.gateway.elasticsearch.http.keepAlive.maxConnectionsPerRoute
Mandatory
This property specifies the maximum number of persistent connections that can be established per HTTP route to an external Elasticsearch server.
Default value: 15
pg.gateway.elasticsearch.http.username
This property specifies the user name to connect to external Elasticsearch using basic authentication.
pg.gateway.elasticsearch.http.password
This property specifies the password to connect to external Elasticsearch using basic authentication.
pg.gateway.elasticsearch.https.keystore.filepath
This property specifies the Keystore file path for establishing HTTPS communication with external Elasticsearch.
pg.gateway.elasticsearch.https.truststore.filepath
This property specifies the truststore file path for establishing HTTPS communication with external Elasticsearch.
pg.gateway.elasticsearch.https.keystore.password
This property specifies the Keystore password for establishing HTTPS communication with external Elasticsearch.
pg.gateway.elasticsearch.https.keystore.alias
This property specifies the Keystore alias for establishing HTTPS communication with external Elasticsearch.
pg.gateway.elasticsearch.https.truststore.password
This property specifies the truststore password for establishing HTTPS communication with external Elasticsearch.
pg.gateway.elasticsearch.https.enabled
This property specifies whether you want to enable or disable the HTTPS communication with external Elasticsearch.
Default value: false
If this property is set to false none of the above properties related to HTTPS are respected.
pg.gateway.elasticsearch.outbound.proxy.enabled
This property specifies whether you want to enable or disable outbound proxy communication.
Default value: true
pg.gateway.elasticsearch.outbound.proxy.alias
This property specifies the outbound proxy alias name used to connect to external Elasticsearch.
pg.gateway.elasticsearch.https.enforce.hostname.verification
This property enforces the host name verification for SSL communication.
Default value: false
pg.gateway.elasticsearch.sniff.enable
Mandatory
This property enables sniffers to add the other nodes in an Elasticsearch cluster to the client so that the client can talk to all nodes.
Default value: true
This configuration must be set to false if you are changing the network when API Gateway or external Elasticsearch is running.
pg.gateway.elasticsearch.tenantId
This property allows you to specify a tenant name of your choice. This value must be same across all nodes.
The default value of this property is the Integration Server instance name. So, ensure that you provide same name for all Integration Server nodes in a cluster.
If you modify this value, you must edit the value in all nodes and restart the API Gateway server for the change to take effect.
pg.gateway.elasticsearch.sniff.timeInterval
Mandatory
This property enables adding the newly added external Elasticsearch cluster nodes to existing REST client in a specified time interval in milliseconds.
Default value: 60000
3. Restart API Gateway for the HTTP client to take effect.
Note:
If hosts and ports are changed for external Elasticsearch then you have to update the appropriate external Elasticsearch configuration for external Kibana separately and restart the external Elasticsearch server as well as external Kibana.
You can also externalize the external Elasticsearch tenant ID and configuration by using a master configuration file. For details, see Externalizing Configurations .