New Caching Techniques for Web Search Engines

MAURICIO MARÍN; VERÓNICA GIL-COSTA; CARLOS GÓMEZ-PANTOJA

Chicago, Illinois USA

Conferencia; High Performance Distributed Computing (HPDC); 2010

This paper proposes a cache hierarchy that enables Web search engines to efficiently process user queries. The different caches in the hierarchy are used to store pieces of datawhich are useful to solve frequent queries. Cached items range from specific data such as query answers to generic data such as segments of index retrieved from secondary memory. The paper also presents a comparative study based on discrete-event simulation and bulk-synchronous parallelism. The studied performance metrics include overall query throughput, single-user query latency and power consumption. In all cases, the results show that the proposed cache hierarchy leads to better performance than a baseline approach built on state of the art caching techniques.