A neural network model for information retrieval using latent semantic indexing

Abstract

In this paper, we incorporate the latent semantic indexing (LSI) technique into a competition-based neural network model for information retrieval. The neural network model was originally developed using a causal inference network that connects the index terms and related documents. The model's retrieval performance was further enhanced by using Roget's thesaurus to relate synonymous index terms. However, in a thesaurus-based information retrieval model, the semantic information embodied is reflected by the terms in its thesauri and the documents stored in its database, and the indexing vocabulary needs to be updated to account for the changes in the domain knowledge it covers. Since the process of merging or updating thesauri is rather expensive, we incorporated the LSI technique into our neural network model, instead of making explicit use of a thesaurus, in an attempt to capture the semantic relationship between the documents and the index terms. Our results show that by incorporating the LSI method, the neural network model generates an appreciable improvement over the thesaurus-based model.