Can SearchBlox spider through a proxy server?

Yes, SearchBlox supports spidering through a proxy server. A web spider (aka crawler), is simply a web crawler that systematically browses through indexing. You can set the proxy URL, username, and password under Collections > Settings in the Admin menu. Also, you can set a different web proxy for each collection.


Related Topics:

Can I restrict the crawler from indexing certain folder/URL paths?

How can I see detailed spider/crawler activity?

Can I customize the settings for the SearchBlox crawler?

Can I set a time delay on the crawler/spider between making requests to a website?

Have more questions? Submit a request

Comments