disafam
Sailor

ElasticSearch Service Down

ElasticSearch Service is running but in Health Center it shows as "Service Down".

What would be the cause?

0 Kudos
Reply
9 Replies
ranjith_koppu
SailPoint Employee
SailPoint Employee

@disafam 

Scheduled Task Handler service is responsible to report health of Elasticsearch & RabbitMQ.

You need to see the STH service logs to see if it has any errors while pinging for the ES health.

This server hosting the STH service should be able to reach ES on port 9200. You need to have port 9200 open for incoming connections on the ES service.

 

Thanks.

Ranjith

0 Kudos
Reply
disafam
Sailor

I found the following error message in the log files: Does this mean the port is closed?

2022-02-03 16:36:08,068,6,ERROR,WBX.whiteOPS.Elasticsearch.ElasticUtils,getClusterStats,Error getting Elasticsearch cluster stats. Uri: https://abcdefghijkl.bbd.yht.fake:9200/_cluster/stats
System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 1.1.1.1:9200

0 Kudos
Reply
ranjith_koppu
SailPoint Employee
SailPoint Employee

@disafam 

yes this means the STH service cannot connect to the ES service on port 9200

0 Kudos
Reply
matthewfry
Navigator

I want to say I got this same error before, and the reason was a bad password for the elasticsearch service account.

0 Kudos
Reply
shlomil
Wayfarer

this can be several things

1. proxy on the elastic server so there is no connection.

2. if you are running on VM the service is up but unable to bind local address 
(you can see that in the elasticsearch log).

3. one of the shards is broken so the specific month is no good and the elastic is down.

0 Kudos
Reply
disafam
Sailor

port 9200 is open for incoming connections on the ES service.  Still ElasticSearch Service is running in Windows but in Health Center it shows as "Service Down". In the WebUI  getting a Loading Failed. I restart the service manually, it runs for a minute and down again. Please advise.

 

0 Kudos
Reply
Harneet_Singh
Lookout

Seems like one of the shards is broken. Just try to open the browser on Elastic Search server and copy the below link and hit enter.

 https://abcdefghijkl.bbd.yht.fake:9200/_cluster/stats

If the browser prompt you to enter credential, enter the credential and see if the node status is green or not. 

Again, try the above steps from another FAM core server (Event Manager) and see the credential prompt is coming and you are able to see the node status.

 

By doing this steps, you will get answer to 2 questions:

1. Does ES cluster health is good or not?

2. Does ES cluster connection is allowed from Event Manager or any other FAM core server or not?

0 Kudos
Reply

@Harneet_Singh  thanks for sharing the informative details,  we just tried to open the browser on Elastic Search server and copied  the below link and hit enter and it is not moving ahead, it says page not found ? Is there anything we are missing ?

 https://abcdefghijkl.bbd.yht.fake:9200/_cluster/stats

0 Kudos
Reply
barbara_hodgkin
SailPoint Employee
SailPoint Employee

@Akhilesh_Tripathi 

   @Harneet_Singh was sharing an example URL.  You will need to replace your Elasticsearch server computer name into the abcdefg... or use localhost if you are browsing to the URL from the Elasticsearch server. 

   https://localhost:9200/_cluster/stats from Elasticsearch server or

   https://{your elasticsearch server}:9200/_cluster/stats

This should prompt for the elasticsearch credentials you entered in the Server Installer when defining the elasticsearch server.

If you do not remember the credentials, you can run the server installer and reset as desired.  They should take effect immediately.

You can also utilize this article to help troubleshoot: Event Manager and Elasticsearch Troubleshooting

Please note the other optional URL: https://{servername}:9200/_cat/indices

0 Kudos
Reply