Is it possible to speed up the indexing process?

Absolutely. If you have a large server with 16GB or more memory and 8 CPU cores or higher, you can change the settings within the SearchBlox server to index at a faster rate.

NOTE: Please delete all existing collections before you make the following changes. and restart SearchBlox after the changes are made.

There are 2 files that can be updated to enable faster indexing: 

1.) Increase parallel processing within SearchBlox:

Edit the following lines in C:\SearchBloxServer\webapps\searchblox\WEB-INF\classes\application.conf (Windows) or /opt/searchblox/webapps/searchblox/web-inf/classes/application.conf (Linux)

If you are trying to increase the speed of a Filesystem collection, make sure the nr-of-instances is changed from 5 to 10 or 20, depending on your server size.

scanPathActor-filesystem-dispatcher {
router = round-robin
nr-of-instances = 10
}

indexingActor-filesystem-dispatcher {
router = round-robin
nr-of-instances = 10
}

If you are trying to increase the speed of an HTTP collection, make sure the nr-of-instances is changed from 5 to 10 or 20 depending on your server size.

scanPathActor-http-dispatcher {
router = round-robin
nr-of-instances = 10
}

indexingActor-http-dispatcher {
router = round-robin
nr-of-instances = 10
}

2.) Increase the memory allocation to SearchBlox:

Edit the start.ini file within the SearchBloxServer folder.

-Xmx8g
-Xms8g

or change to 16g when you have at least a 32GB server.

-Xmx16g
-Xms16g

 

Remember to save and restart the SearchBlox server after these changes are made. Then, create a new collection and test the indexing speed to see the difference.

 



To learn more about SearchBlox please visit our developer document site https://developer.searchblox.com/docs

Have more questions? Submit a request

Comments