Training alone produces excessive emissions.
McClure ’24 – (Paul McClure, Bachelor of Nursing, Graduate Diplomas in Advanced Nursing and Psychological Studies, a Master of Laws (Juris Doctor), and a Bachelor of Arts in journalism and creative writing, previously worked as an intensive care nurse at one of Australia’s busiest hospital and a criminal defense lawyer, “The bad and ugly: AI is harmful, unreliable, and running out of data”, 4/22/24, New Atlas, https://newatlas.com/technology/ai-index-report-negatives/)
The environmental cost of training AI systems varies and is, in the case of some models, hefty, according to the 2024 AI Index report. For example, Meta’s Llama 2 70B model released approximately 291.2 tonnes (321 US tons) of carbon. That’s 291 times more than the emissions produced by a single traveler on a round-trip flight from New York to San Francisco and 16 times higher than the average American’s yearly carbon emissions.
However, that’s nothing compared to the whopping 502 tonnes (553 US tons) reportedly released during GPT-3 training.
Variations in emissions data are due to factors like model size and data center energy efficiency. And, the report writers noted that most prominent model developers – including OpenAI, Google, Anthropic – don’t report carbon emissions produced during training, making it difficult to conduct a thorough evaluation. Independent researchers estimated the GPT-3 figure in the above paragraph, as the developers didn’t disclose the actual figures.