A Fault-Tolerant Cache Service for Web Search Engines

CARLOS GÓMEZ-PANTOJA; GIL COSTA VERÓNICA; DOLORES REXACHS; MARÍN MAURICIO; EMILIO LUQUE

Conferencia; In 10th IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA 2012); 2012

 Large Web search engines are constructed as acollection of services that are deployed on dedicated clustersof distributed-memory processors. In particular, efficient userquery throughput heavily relies on using result cache servicesdevoted to maintaining the answers to most frequent queries.Load balancing and fault tolerance are critical to this service.This paper proposes the design of a result cache servicebased on consistent hashing and a strategy for enabling faulttolerance. Performance evaluation is performed by using actualqueries from a commercial search engine. The results show thatthe proposed cache service outperforms baseline approaches,decreasing the average query response time, increasing querythroughput and efficiently recovering performance after processorfailures.