sail101
Mate

FAM Elasticsearch Performance Issues

Jump to solution

Hi,

We are facing frequent issues with FAM 8.1 Elasticsearch DB service. We do have significant number of events (1700 per second) coming in but the ElasticSearch host server seems to have good configuration (1 TB HDD, 24 GB Ram - 8 GB JVM Heap). The ElasticSearch service becomes unresponsive and sometimes the current month's index goes Red. Has anyone had similar issues ? Is there any tool that can monitor ES performance (indexing, search etc.). Looks like Metricbeat is not supported by 5.1.1. Tried enabling monitoring on ES cluster but it didn't work. Also, can anyone share their Events Per Seconds & ES Server - Memory/Disk/CPU configuration details.

1 Solution

Accepted Solutions
sail101
Mate

With the help of support we were able to find out the reason . The index files were getting locked by backup solution and ES was not able to access it . Haven't seen the issue after excluding in-use files from Backup tool

View solution in original post

4 Replies
Harneet_Singh
Lookout

Hi @sail101 ,

I will recommend you to increase the RAM on the ES server and give it a shot.

Also, could you please share the CPU and it's utilization on the ES server?

sail101
Mate

@Harneet_Singh we have increased the RAM to 32 GB (12 GB ES). Heap memory consumption doesn't seem to be an issue though. CPU is 2.2 GHZ 8 Cores. The average utilization is less than 40% . The events count per Event manager Logs has never been above 2500 per second. (Average 500/s). Would you be able to share any Events per Second stats and your ES RAM cofing. We recently found out that possibly in our case the issue is some other process locking the index files and not performance.

Harneet_Singh
Lookout

@sail101, we have around 1200 events per second and the resources on the ES server is 96 GB RAM & 16 Core CPU (2.3 GHZ).

My ES server resources (Both RAM & CPU) are always above 70%. Are you seeing the service stops automatically?

Also, check your task schedular task is completed succesfully without any error. Please check the logs as well.

sail101
Mate

With the help of support we were able to find out the reason . The index files were getting locked by backup solution and ES was not able to access it . Haven't seen the issue after excluding in-use files from Backup tool