Yes, duplicate documents can be prevented from getting into the search collection.
Under Collections > Settings, enable Remove Duplicates.
Duplicate documents are defined by SearchBlox as having the same content within them. When the feature is enabled, it will allow only one document to be indexed if there are two (or more) documents with the same content.
To learn more on Remove Duplicates read: Using Robots.txt in SearchBlox
Comments
0 comments
Please sign in to leave a comment.