Efficient domain adaptation while minimizing energy and hardware resource consumption

HERNAN MAINA; NICOLÁS WOLOVICK; LUCIANA BENOTTI

Workshop; Workshop Widening Access to Natural Language Processing; 2023

Large language models are expensive to traindue to energy and hardware requirements andexhibit global north positionality (Santy et al.,2023). Domain adaptation has emerged as apromising strategy for mitigating positionalityby tailoring language models to different cultural and value contexts (Hershcovich et al.,2022). However, domain adaptation is expensive for developing countries. In this paper,we study strategies for domain adaptation thatminimize energy and hardware resource consumption. With our optimization strategies, weare able to apply domain adaptation to rathersmall language models using the hardware atour global south university, with no need to renthardware in the north.