Heuristic information retrieval: a competition-based connectionist model

Abstract

In this paper, we adapt a competition-based connectionist model, which has been proposed for diagnostic problem solving, to information retrieval. In our model, documents are treated as and user information needs as manifestations, and a competitive activation mechanism is used which converges to a set of disorders that best explain the given manifestations. By combining the ideas of Bayesian inferencing and diagnostic inferencing using parsimonious covering theory, this model removes many difficulties of direct application of Bayesian inference to information retrieval, such as the unrealistically large number of conditional probabilities required as part of the knowledge base, the computational complexity, and unreasonable independence assumptions. Also, Bayesian inference strengthens the parsimonious covering model by providing a likelihood measure which can be used to rank documents as well as to guide the search to the most likely retrieval. We also incorporate two types of relevance information to improve the model. First, the 1911 Roget's thesaurus is used to provide semantic relevance information among the index terms. Second, the associations between documents and index terms are represented by the node connections in the model specifying the cause-effect relations. Our experimental results using 4 standard document collections demonstrate the efficiency and the retrieval precision of this model, comparable to or better than that of various information retrieval models reported in the literature.