CONICET | Buscador de Institutos y Recursos Humanos

The interdisciplinary field of neuroscience has made signicant progress in recent decades, providing the scientic community in general with a new level of understanding on how the brain works beyond the store-and-re model found in traditional neural networks. Meanwhile, Machine Learning (ML) based on established models has seen a surge of interest in the High Performance Computing (HPC) community, especially through the use of high-end accelerators, such as Graphical Processing Units (GPUs), including HPC clusters of same. In our work, we are motivated to exploit these high-performance computing developments and understand the scaling challenges for new, biologically inspired, learning models on leadership-class HPC resources. These emerging models feature sparse and random connectivity profiles that map to more loosely-coupled parallel architectures with a large number of cores per node. Contrasted with traditional ML codes, these methods exploit loosely-coupled sparse data structures as opposed to tightly-coupled dense matrix computations, which benet from SIMD-style parallelism found on GPUs. In this paper we introduce a hybridMessage Passing Interface (MPI) and Open Multi-Processing (OpenMP)parallelization scheme to accelerate and scale our computational model based on the dynamics of cortical tissue. We ran computational tests on a leadership class visualization and analysis cluster at Argonne National Laboratory. We include a study of strong and weak scaling, where we obtained parallel eciency measures with a minimum above 87% and a maximum above 97% for simulations of our biologically inspired neural network on up to 64 computing nodes running 8 threads each. This study shows promise of the MPI+OpenMP hybrid approach to support flexible and biologically-inspired computational experimental scenarios.In addition, we present the viability in the application of these strategiesin high-end leadership computers in the future.