Rebuilding the full text search index

Rebuilding the index means that all documents stored in the library will be re-indexed along with the metadata values associated with them.

You should check with the FileCare team before re-indexing your system. In particular if you have over 500 000 documents you may need the assistance of a technician.

WARNING: Certain changes in FileHold configuration can cause a re-indexing of documents, such as editing or deleting a drop down or drill down metadata field value or deleting a metadata field from a schema. If the user performs one of these actions, a message “You are about to make a change that will cause x documents to be re-indexed. While these documents are being re-indexed, users may notice decreased performance in the system.” This message appears when at least 1000 documents are affected by the re-indexation. This setting can be controlled by the setting "ReindexWarningThreshold" in the web config file in C:\Program Files\FileHold Systems\Application Server\LibraryManager.

Rebuilding the index can take a number of hours to complete. Re-indexing should be planned during a time of low or no user activity. The performance of re-indexing will vary on each system according to the performance of the server CPU, the drive where the index is stored, the size of the library, the average size of the documents, and the number of documents that will be excluded from the search. As a rule of thumb you should anticipate approximately 5000 documents being indexed every hour. You can adjust your prediction as the actual indexing takes place. The total size of the index is subject to the maximum index size limit for dtSearch of 1 TB.

FileHold Cloud customers and customers with hosted mode enabled cannot rebuild the FTS index unless they are a support user.

There are two options to rebuild the index. Read each option carefully.

Option 1: From web client

Go to Administration Panel > System Configuration > Settings > Search.
Click the Advanced button.
In the Initialize Index area, click Initialize Index to start full-text search indexing.

This will wipe out the existing Full Text Search collection and create a queue for all documents in the system to be reindexed in the Microsoft SQL Databases.
On large collections, this may also interfere with documents being added to the system by FileHold users.
This task takes considerable time and is only recommended if there are significant reasons for re-indexing the entire system.
We recommend this be run over the weekend.
Before doing this you should ensure an IT Administrator is available in case server changes to the configuration is needed.
The scheduled task runs this process, and an IT server administrator can disable this scheduled task (Update FTS index) during business hours. This process may take minutes or hours or longer - it depends on whether you have tens of thousands, hundreds of thousands or millions of documents in your collection.
We recommend you run this on a Friday afternoon so that it can work over a weekend without users accessing the system so that it goes as fast as possible.
Contact FileHold Support if you have any questions, but include the following information:
1. A screenshot of the problematic searches using Desktop Client or Web Client, and what a description of what you were expecting. This helps us understand the issue.
2. The HealthChecker Report created by following this procedure as an IT administrator with domain admin and SQL server admin abilities.
3. A screenshot of the Full Text Search Status page from Web Client > Library Admin > Search Status
4. A screenshot or several screenshots that captures all settings in Web Client > Library Admin > Search Settings.

Option 2: From FH Instrumentation Tools

You must be a Windows server administrator or domain administrator to perform this task.

Run the FHInstrumentation Health Checker. If all checkpoints in the Health Check Report are green or amber/yellow then proceed, if you find any red items then please send FileHold support all of the following items:

Health Checker Report
Temporarily enable Full Text Search debugging mode to generate a Full Text Search debug report This will be explained below this procedure and attach this debug report as well.
Please include several screenshots to cover all search settings in Web Client > Library Admin > Search settings. This normally takes two screenshots to capture.
Include a screenshot of the Web Client > Library Admin > Search Engine > Status Page

Run the FHInstrumentation file as an administrator and without restricted permissions.
Under Actions > Search Engine, select Rebuild FTS and click Start.
Click Rebuild and wait for it to complete. If you find any errors, send the contents of the error message to FileHold support. If it completes successfully, then click Finish.

Do one of the following to re-index the document collection:

Choice A: If you have under 10,000 to 20,000 files/documents.

Choice B. If you have over 10,000 to 20,000 files/documents then we recommend that you simply re-enable the scheduled task Update FTS index. The process will start up, and this may take hours or a day or more for the system to index the entire queue.

In either case, the full text search system may not return complete results while this indexing is in progress. As more documents are indexed, then more documents are capable of being included in full text search results.You can find out the status by going to Web Client > Library Admin > Search Settings > Status page. Please note that this page may not appear if the SQL server is busy (as SQL locks tables etc) so please be patient and try it a few times.

After some time the process will complete. Then you should re-enable the Full Text Search Scheduled task so that the scheduled task can keep up with documents being added to the system, by updating the search index.

Enabling search debugging

This feature only works currently in HTTP mode (not HTTPS). Each time you search it will update a technical debug file. The goal of this tool is to share technical debugging files with FileHold support.

If you are experiencing issues with certain searches, please enable this temporarily, then run the search and send us all the files. Once you send the files to FileHold support, then you should immediately disable the debugging mode as it will cause server performance issues when users are using search.

To enable search debugging

Using FHInstrumentation tool, select Search Engine > Turn on Debug and click Start.
Enter the destination file name and location, and click Next.

Click Turn On.
Once it has completed successfully, you should repeat the problematic search query in Desktop Client or Web Client - and then make a copy of the debug file to send to FileHold Support.
Keep repeating this for various problematic search queries, so that each query has its own unique debug file.
Make a search, save a copy of the debug file, and continue.
Send 2-3 search examples at maximum, a long list of things is not useful.
Now that this is complete, disable the debugging mode to avoid performance issues by selecting Turn Off Debug, clicking Start, and then click Turn Off. Debug will be turned off automatically when IIS recycles or is restarted.

Delete orphan documents from the FTS queue

If there are issues with the full text search queue, such as orphan files or duplicate files, and the dtSearch indexation process does not work then this tool will remove any entries that are causing the issues. Use this tool when there is an error in the event log that relates to the indexation queue.

To remove orphan documents from the FTS queue

Using FHInstrumentation tool, select Search Engine > Delete orphan documents from search queue and click Start.
Enter the server name and login credentials.
Click Next.
Click Delete.
A message is displayed that orphan document deletion is completed successfully. Click Finish.