Can Yaras recognized for his research aimed at efficient algorithms for LLMs

Can Yaras received a best poster award at the 2024 Midwest Machine Learning Symposium for his research, “Deep LoRA: Simple & Efficient Adaptation of Foundation Models to Data-Deficient Tasks.” Yaras is a doctoral student in ECE working with Prof. Qing Qu and Prof. Laura Balzano.

The goal of this research is to understand how large language models (LLMs) (like ChatGPT) learn, and exploit these insights to come up with more computationally efficient algorithms for training and deploying such models.

“The computational demands of increasingly powerful and prevalent AI models are ever-growing, requiring enormous amounts of hardware and energy to develop,” said Yaras. “Our research can dramatically reduce these costs with minimal loss (or even improvement) in model performance.”

To do this, Yaras says he proposed a novel, theoretically-grounded algorithm for adapting LLMs to new tasks. The algorithm, called “Deep LoRA,” generalizes better to new tasks with limited data compared to existing approaches, while learning more compressible models with less hyperparameter tuning.

“The biggest challenges for this research are knowing “where to look” for low-dimensional structure in the model,” explained Yaras. “We tackled this by drawing insights from more classical problems in signal processing and machine learning such as matrix sensing, which gave us a clue as to how to tackle more modern problems.”

Yaras says some of the greatest issues facing AI today are accessibility and sustainability. 

“Modern machine learning is prohibitively compute and data hungry, monopolized by a select few organizations having the resources to develop these models,” said Yaras. “On the other hand, the exorbitant energy demands of AI are unignorable and being discussed at the national policy level. Tackling these problems aligns with my personal beliefs of democratization and reducing carbon footprint.”