Is it possible to speed up the indexing process?

Absolutely. If you have a large server with 16GB or more memory and 8 cpu cores or higher, you can change the settings within the SearchBlox server to index at a faster rate.

NOTE: Please delete all existing collections before you make the following changes. and restart SearchBlox after the changes are made.

There are 2 files that can be updated to enable faster indexing: 

1.) Increase parallel processing within SearchBlox:

Edit the following lines in C:\SearchBloxServer\webapps\searchblox\WEB-INF\classes\application.conf (Windows) or /opt/searchblox/webapps/searchblox/web-inf/classes/application.conf (Linux)

If you are trying to increase the speed of a Filesystem collection, make sure the nr-of-instances is changed from 5 to 10 or 20, depending on your server size.

scanPathActor-filesystem-dispatcher {
router = round-robin
nr-of-instances = 10
}

indexingActor-filesystem-dispatcher {
router = round-robin
nr-of-instances = 10
}

If you are trying to increase the speed of an HTTP collection, make sure the nr-of-instances is changed from 5 to 10 or 20 depending on your server size.

scanPathActor-http-dispatcher {
router = round-robin
nr-of-instances = 10
}

indexingActor-http-dispatcher {
router = round-robin
nr-of-instances = 10
}

2.) Increase the memory allocation to SearchBlox:

Edit the following lines within C:\SearchBloxServer\services\conf\wrapper.conf (Windows) or /SearchBloxServer/services/conf/wrapper.conf (Linux)

wrapper.java.additional.3a =-Xms4g
wrapper.java.additional.3b =-Xmx4g

Change these values to:

wrapper.java.additional.3a =-Xms8g
wrapper.java.additional.3b =-Xmx8g

To update the memory allocation when running using the startSearchBlox command, edit the start.ini file within the SearchBloxServer folder.

-Xmx4g

-Xms4g

Change this value to

-Xmx8g

-Xms8g

Remember to save and restart the SearchBlox server after these changes are made. Then, create a new collection and test the indexing speed to see the difference.

 

Have more questions? Submit a request

Comments