Extending the Usability of a Dynamic Tuning Environment

PAOLA GUADALUPE CAYMES SCUTARI

TDX- Universitat Autònoma de Barcelona

Lugar: Barcelona; Año: 2007 p. 235

978-84-690-7657-6

In the last years, computing performance demand has been in increase specially in the scientific field that has to solve complex problems. In general, the data set size and the complexity of the operations over them require the use of very powerful systems. Thus, the increasing necessity for high performance systems/computing has been directing the attention of the scientific field towards the parallel/distributed paradigm. The optimization process, so-called tuning process, is the process followed in order to adapt and improve the behaviour of the applications by modifying their critical parameters. It includes several and successive phases. Firstly, during a monitoring phase the information about the behaviour of the application is captured. Next, the information is analyzed, by looking for bottlenecks, deducing their causes and trying to determine the adequate actions to eliminate them. Finally, appropriate changes have to be applied to the code to solve the problems and improve the performance. Fortunately, through the years different approaches and tools have been developed with the aim of helping the user during some of the optimization phases (monitoring, analysis or tuning phases). One of the available tools is MATE (Monitoring, Analysis and Tuning Environment), which is an automatic and dynamic tuning environment for parallel applications. MATE works in three continuous and iterative phases in order to adapt the deployment of the application according to the current state of the execution environment. MATE includes the knowledge to tune performance problems in pieces of software called ``tunlets´. Each tunlet includes the logic to collect behavioural information, analyze it on the fly and decide what the required tuning actions are. The objective of this work is to extend the usability of MATE. Our work covers two different aspects of MATE: – the improvement of the performance reached by the centralized analysis executed by MATE, due to it turns in a bottleneck as the size of the application increases. Thus, we provide an alternative to make MATE scalable. – the increase in the user-friendliness of MATE in order to facilitate the inclusion of new performance knowledge in it. Thus, we make MATE transparent for the users. According to the first aspect, we propose a novel approach to execute the analysis process, called Distributed-Hierarchical Collecting-Preprocessing Approach. This approach is based on the distributed collection of events which alleviates the centralized old-fashion in which collection was done, and in the preprocessing of cumulative or comparative operations as possible. Thus, the Global Analyzer receives just the necessary information condensed in a unique message from each Collector-Preprocessor, which considerably reduces the overload of Global Analyzer. In this way, MATE is provided with scalability properties. According to the second aspect, we provide a methodology, including a designed language and a developed translator to automatically insert tunlets (knowledge) in MATE. When some problem has to be tuned in a parallel application the user has to develop the corresponding tunlet. By using our methodology, the user is exempted from being involved in implementation details of MATE. Thus, by defining a set of abstractions about the application and the performance model, such abstractions can be formalized in a tunlet specification using the provided language. Such specification will be automatically translated in a tunlet ready to be used in MATE. Both scalability and transparency of MATE are qualities necessary to make MATE a more useful and user-friendly tool. The proposals and developments presented attempt to provide MATE with such characteristics. Experiments showed the viability of the proposals.