Using findinsite-ms subsets and dynamic database searching

findinsite-ms can search several search databases - the list is specified in the Searching section of the Control Panel. A default search form searches all the selected available search databases.

Alternatively, you can set up a search form so that the user can choose which search databases to search - this is called a subset search.

Another alternative is to use Dynamic Database Searching which lets you specify which search database(s) to search on a per-request basis. This might be less efficient as the databases are loaded on demand.

Example search database list

The examples on this page assume that you have the following three search databases available:

Screenshot of available search database list

Default search

If you look carefully at the search database list above, you will see that the "included" check box for phdcc_online is not checked. This means that a default search will not include this search database.

This is typical HTML code for a default search, giving the user a single search box and a Search button:

<FORM method='get' action="/findinsite/search.aspx" style="display:inline;">
Search for: <INPUT NAME="Text" TYPE="text" SIZE=20 MAXLENGTH=80>
<INPUT TYPE="SUBMIT" VALUE="Search">
</FORM>

This form looks like this:

Subset search

If you want to let your users choose which search databases to search, you can set up a Subset search. You should provide check boxes for each available search database.

A Subset search must incude a PickSubsets form parameter, and form parameters named Subset1, Subset2, etc., one for each available search database. In this example:

Subset1 corresponds with FindDb

Subset2 corresponds with phdcc_online

Subset3 corresponds with webapp1
Specify CHECKED=checked if you want the search database included in the search by default.

<FORM method='get' action="/findinsite/search.aspx" style="display:inline;">
<INPUT NAME=PickSubsets VALUE=true type='hidden'>
<INPUT NAME=Subset1 TYPE=checkbox VALUE="on" CHECKED=checked>findinsite-ms: search engine<BR>
<INPUT NAME=Subset2 TYPE=checkbox VALUE="on" CHECKED=checked>phdcc: Software tools for CDs and the Web<BR>
<INPUT NAME=Subset3 TYPE=checkbox VALUE="on" CHECKED=checked>webapp1: Our application<BR>
Search for: <INPUT NAME=Text TYPE=text SIZE=30 MAXLENGTH=80>
<INPUT TYPE=SUBMIT VALUE="Search">
</FORM>

This form looks like this:

As can be seen from this example, you set the initial state of each "Included" check box to any value.

Dynamic database searching

Supported in version 1.60 onwards.

Dynamic database searching lets you specify which search database(s) to search on a per-request basis. You use the Searching section of the Control Panel to enable dynamic database searching - by specifying a "dynamic database searching regular expression" which determines which off-site databases it is OK to search - consult Microsoft's technical documentation on regular expressions.

Using dynamically loaded databases will run slower than normal, especially when a database is loaded. Dynamic databases from "http:" sources will load more slowly than those from file sources.

The search database(s) are loaded on demand and cached in memory for about 20 minutes after their last use. A background thread checks regularly (every 5 minutes) to see if any databases can be removed from the cache. The cache is also cleared when the "regular expression" is changed in the Control Panel. [Future option: check to see if any databases have been updated and load them if need be.]

If many dynamic databases are loaded simultaneously then the findinsite-ms memory usage will increase. If findinsite-ms runs out of memory then it is stopped without warning by ASP.NET. You can keep an eye on memory usage in the About section of the Control Panel.

Using dynamic database searching

Once dynamic database searching is enabled, use it by adding a db hidden parameter to your search forms. The db value should be a search database filename (or "@" followed by the filename of a text file that has a list of search databases). In this simple example, the dbname search database in the usual findinsite-ms FindData work directory is used:

<FORM method='get' action="/findinsite/search.aspx" style="display:inline;">
<INPUT NAME=db type='hidden' VALUE=dbname>
Search for: <INPUT NAME=Text TYPE=text SIZE=30 MAXLENGTH=80>
<INPUT TYPE=SUBMIT VALUE="Search">
</FORM>

If the db parameter value contains no forward or back slashes then the file is loaded from the findinsite-ms work directory - the value is NOT checked against the "dynamic database searching regular expression". Otherwise, the db value must match the 'regular expression'. For example, if the dynamic database searching 'regular expression' is ^http://www\.phdcc\.com/ then this db value is valid:
<INPUT NAME=db type='hidden' VALUE="http://www.phdcc.com/fiscd/fiscddb">

Database lists

You can specify a list of search databases to search in a text file by prefixing the db value with "@", eg:
<INPUT NAME=db type='hidden' VALUE="@http://www.phdcc.com/findinsite/dbs.txt">

The list-file at http://www.phdcc.com/findinsite/dbs.txt has lines (each ending in CR-LF) that follow the Subsets list file usage spec; in this case the first two lines are ignored. For example, to specify two search databases, use this:
ignored ignored http://www.phdcc.com/fiscd/fiscddb http://www.phdcc.com/fiscd/fields
Both the list-file and the search databases that it references must match the 'regular expression' if they are off-site. If a search database cannot be loaded then it is ignored.

Note that the databases in the cache are identified by the db parameter value. If the contents of a list-file changes then these are ignored if the list-file filename is unchanged (though obviously it will be reloaded if it has been removed from the cache).

%DYNAMIC_DB% output template variable

The value of the db parameter is output by findinsite-ms wherever a template uses the %DYNAMIC_DB% variable - this lets further searches on the same information. The default Heading and Footer templates use this as follows:
<INPUT NAME=db type='hidden' VALUE="%DYNAMIC_DB%">