From chatbots on WhatsApp to search engines like Google and facial recognition apps, neural networks are a powerful advance in technology, but one aspect in the development of increasingly powerful AIs is beginning to worry the scientific community: as they evolve, they swallow more and more more gigantic amounts of energy.
The situation has been debated for some time at the academy, but the lay world only became aware of the problem when Google dismissed researcher Timnit Gebru, one of the most respected in the field of ethics in artificial intelligence, after an article by her and other scientists had its publication refused by the company.
In it, Gebru and the other authors (some from Google itself) highlighted and expanded an article published in 2019 by researcher Emma Strubell, from the Language Technologies Institute at Carnegie Mellon University, which pointed out how energy consumption, carbon emissions and the financial costs of great language models have exploded since 2017.
As AI receives more data to process and learn, more information and energy it needs, increasing its carbon footprint. According to Strubell, teaching a neural network to think pollutes as much as producing and driving five cars over their lifetime.
“Training large AI models consumes a lot of computer processing power and therefore a lot of electricity,” said Kate Saenko, a computer scientist at Boston University, in an article for The Conversation. According to her, the training of an AI is far from the efficiency of a human being, and the ultimate goal to be achieved is artificial neural networks that apply mathematical calculations to mimic human brain cells using gigantic databases to learn.
“A recent model, called Bidirectional Encoder Representations from Transformers (BERT), used 3.3 billion words from English books and Wikipedia articles. In addition, during training, BERT read this data set not 1, but 40 times. For comparison, a child who learns to speak can hear 45 million words at age 5, 1 / 3,000 times than BERT, ”she said, referring to Google AI.