An Architecture and Platform for Developing Distributed Recommendation Algorithms on Large-Scale Social Networks (Indexed SCI, IF JCR 2015=0.878)

CORBELLINI, A.; MATEOS, C.; GODOY, D.; ZUNINO, A.; SCHIAFFINO, S.

JOURNAL OF INFORMATION SCIENCE

SAGE PUBLICATIONS LTD

Lugar: London; Año: 2015 vol. 41 p. 686 - 704

0165-5515

The creation of new and better recommendation algorithms for social networks is currently receiving much attention due to the increasing need of new tools for assisting users. The volume of available social data as well as experimental datasets forces recommendation algorithms to scale to many computers. Given that social networks can be modelled as graphs, a distributed graph-oriented support able to exploit computer clusters arises as a necessity. In this work, we propose an architecture, called Lightweight-Massive Graph Processing Architecture (LMGPA), which simplifies the design of graph-based recommendation algorithms on clusters of computers, and a Java implementation for this architecture composed of two parts: Graphly, an API offering operations to access graphs, and jLiME, a framework that supports the distribution of algorithm code and graph data. The motivation behind the creation of this architecture is to allow users to define recommendation algorithms through the API and then customize their execution using job distribution strategies, without modifying the original algorithm. Thus, algorithms can be programmed and evaluated without the burden of thinking about distribution and parallel concerns, while still supporting environment-level tuning of the distributed execution. To validate the proposal, the current implementation of the architecture was tested using a followee recommendation algorithm for Twitter as case study. These experiments illustrate the graph API, quantitatively evaluate different job distribution strategies w.r.t. recommendation time and resource usage, and evidence the importance of providing non-invasive tuning for recommendation algorithms.