How to Monitor SearchBlox log Files?

Log files for monitoring

The following log files are recommended for monitoring:

<SEARCHBLOX_INSTALLATION_PATH>/webapps/searchblox/logs/status.log
<SEARCHBLOX_INSTALLATION_PATH>/webapps/searchblox/logs/es.log

Linux path
/opt/searchblox/logs/searchblox.log

Windows path
C:/SearchbloxServer/services/log/wrapper.log

 Error messages to look for within the logs

Terms/errors to be monitored in the application log files

The status.log and es.log files can be monitored for the following errors:

1. gc overhead limit exceeded (memory issue)
2. OutOfMemoryError
3. java.lang.OutOfMemoryError: Java heap space
4. java.lang.OutOfMemoryError: PermGen space
5. shutdown or Shutdown
6. high disk watermark [*] exceeded
7. Too many open files

 Possible Causes

• Errors 1 - 4 are due to insufficient RAM. Please increase memory allocation and restart.
• Error 5 indicates shutdown (reasons could be memory or disk issue or network issue)
• Error 6 indices insufficient disk space issue. Please increase disk space allocated.
• Error 7 occurs in Linux if the ulimit value is not set to maximum.

Troubleshooting

Elasticsearch log in Linux

Error specific to /opt/searchblox/logs/elasticsearch.log for Linux:

• max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Resolution:
Increase map count using the command:
sysctl -w vm.max_map_count=262144

Too Many Open Files Error: 

If this error is observed in index.log or status.log it is required to increase the ulimit.

• Check the ulimit using the command:
  ulimit -n
• Edit the limits.conf file using the command:
  vi /etc/security/limits.conf
• Give the following in the file before the end of file

Text

* soft nofile 100000
* hard nofile 100000

root soft nofile 100000
root hard nofile 100000

• Now you can check the ulimit value using the command:
  ulimit -n
• ulimit value should be 100000 or higher
• Please reboot after this change.

To learn more on Monitoring SearchBlox read: Monitoring SearchBlox