Defesa de Tese de Doutorado: Optimizing the Energy Efficiency and Performance of Deep Learning Algorithms Using Hyperparameter Self-Tuning Techniques: A Step Towards Green AI

Resumo:

The use of Artificial Intelligence (AI) algorithms has grown significantly in the last decade and continues to grow at a fast pace. This expressive growth is mainly due to Machine Learning (ML). However, some ML algorithms, like those based on Deep Learning (DL) models require a high computational power to reach the desired levels of precision, often requiring thousands of machine hours on HPC, which limits access to these algorithms to small institutions or groups of researchers. In addition, the use of these computing resources generate high energy consumpti on, which has become a major problem, not only financially but also ecologically. Recent studies indicate that the current trajectory of AI, with DL models trained on increasingly large data sets, is unsustainable and harmful in several ways, including its massive carbon footprint and water waste. Motivated by the importance of this topic, this work proposes a methodology capable of training and adjusting ML algorithms more efficiently, both with regard to the consumption of computational resources, energy and time, with minimal loss of accuracy in predictions. Thus, in this research our methodology to generate more sustainable algorithms is based on the approach of Automated Machine Learning (AutoML), which automates the development of some parts of the pipeline of an ML model. Although there are already several AutoML frameworks available, most only seek to maximize the predictive performance in a given task without considering aspects related to energy efficiency. So in this work an AutoML approach to improve the efficiency of ML models was developed and evaluated. The term efficiency encompasses the accuracy of the models, the time taken to reach a solution, and its energy consumption. To this we developed multi-objective Genetic Algorithms able Hyperparameter and Architecture Optimization for two ML algorithms (XGBoost and Convolutional Neural Network). The results assessing the proposed AutoML tool using the multi-objective Genetic Algorithm indicates the viability of it, generating more sustainable algorithms in search of a greener AI. In addition, it also makes the training and use of these algorithms more inclusive, i.e., accessible to small groups or institutions so that they can enjoy the benefits offered by these algorithms.