Performance Characterization of State-of-the-art Deep Learning Workloads on a Minsky Platform

MAURICIO GUIGNARD; MARCELO SCHILD; CARLOS S. BEDERIÁN; NICOLÁS WOLOVICK; AUGUSTO J. VEGA

Honolulu

Conferencia; 51st Hawaii International Conference on System Sciences; 2018

University of Hawai'i at Manoa

Deep learning algorithms are known to demandsignificant computing horsepower, in particular whenit comes to training these models. The capability ofdeveloping new algorithms and improving the existingones is in part determined by the speed at which thesemodels can be trained and tested. One alternativeto attain significant performance gains is throughhardware acceleration. However, deep learning hasevolved into a large variety of models, including but notlimited to fully-connected, convolutional, recurrent andmemory networks. Therefore, it appears difficult that asingle solution can provide effective acceleration for thisentire deep learning ecosystem.This work presents detailed characterizationresults of a set of archetypal state-of-the-art deeplearning workloads on a last-generation IBM POWER8system with NVIDIA Tesla P100 GPUs and NVLinkinterconnects. The goal is to identify the performancebottlenecks (i.e. the accelerable portions) to provide athorough study that can guide the design of prospectiveacceleration platforms in a more effective manner.In addition, we analyze the role of the GPU (asone particular type of acceleration engine) and itseffectiveness as a function of the size of the problem.