Federated Searching Revisited

Local index is not federated search
Federated search is not local indexing. While local indexing of metadata for disparate sources is a useful tool it is not by ant stretch of the imagination federated search.

The true value in federated search is the ability to search disparate collections, and truthfully searching all of Ebsco’s dbs at once is not federated search insofar as they are (most probably) similar in architecture. Federation occurs when you search different vendors (or in house & vendors) dbs. There are several methods by which this can be accomplished and the most fruitful is by using workers designed for each specific application. For example Autonomy IDOL (formerly Verity) has a Federated Search application that has wokers for Factiva, EbscoHost, and hundreds of other sources as well as its native workers for the indexes it creates locally. By using the workers tailored to each specific at runtime a local index is not needed.

The use of a local index is problematic. Differences in metatagging, time to refresh, and any number of other factors make this a less desireable option than individually parsing the search to the collection using workers at the time of the query.

An additional feature of these workers is their ability to assist in the ranking of results. The workers are able to pass search limits when needed to each collection. Passing these limits may not be applicable to all collections however the individually configured workers are superior.

Ranking is another concern of federated search. It is nice to have a results set, but relevance ranking is often troublesome. When searching collection A,B & C you may rank the results 1A, 1B, 1C, 2A, 2B…. and so on. This however may not be the best method. Local relevance testing may show that 2B is more relevant that 3A but if you simply take the ranking from the various collection and then order them in a hierarchy based on external relevance rankings you do a disservice to your users.

Federated search is much more than getting results from more than one place and displaying them together. Libraries are way behind the technological curve, however the ROI for libraries is not as evident as it is in the public sector.