Transfer Learning in RNA secondary structure prediction

LUCIANO ZABLOCKI; LEANDRO BUGNON; DIEGO H. MILONE

Santa Fe

Jornada; IA@Litoral; 2023

sinc(i) - UNL - CONICET

RNA (Ribonucleic acid) is a vital molecule in cellular processes and gene regulation. RNA consists of a sequence of nucleotides: Adenine (A), Guanine (G), Cytosine (C), and Uracil (U). Bases can interact to form base-pairs: the secondary structure. The secondary structure can be represented as a binary matrix A, where Aij = 1 indicates pairing between i-th and j-th bases. Discovering the secondary structure of RNA is important for under-standing functions of RNA since the structure essentially affects the interaction and reaction between RNA and other cellular components. Experimental methods are slow, expensive, and technically challenging, which causes a significant volume of RNA data available for use, without their corresponding structure. Thus, predicting the RNA secondary structure becomes an interesting and useful task to solve. Transformers are deep neural models characterized by their self-attention mechanisms, which play a pivotal role inassessing the relative importance of elements within sequences. RNA-FM[1], characterized by a transformer-inspired architecture, leverages large amounts of unlabeled RNA datafor self-supervised training. Employing RNA-FM embeddings in a downstream task, we haveillustrated that learned representations can help to achieve a strong performance in the prediction.These outcomes are readily comparable with the partitions introduced in other works. The recently introduced sincFold is an end-to-end deep learning method primarily employed for RNA secondarystructure prediction, showcasing remarkable performance. The original design of sincFold underwent a modification. The 1D-convolutional feature extraction block was replaced withthe RNA-FM encoder.