TUM Researchers Develop 100x Faster Method to Cut Energy Consumption in AI Neural Network Training

(IN BRIEF) Researchers at the Technical University of Munich have developed a new method for training neural networks that is 100 times faster and more energy-efficient than traditional iterative approaches. By using probabilities instead of iterative adjustments, the method reduces computational power requirements while maintaining accuracy. This development is crucial as AI applications, such as large language models, are rapidly increasing energy consumption in data centers. The new approach, which focuses on critical data points, could significantly reduce the environmental impact of AI training, with applications in fields like image recognition, language processing, and climate modeling.

(PRESS RELEASE) MUNICH, 7-Mar-2025 — /EuropaWire/ — Researchers at the Technical University of Munich (TUM) have unveiled a groundbreaking method that could drastically reduce the energy consumption required for training artificial intelligence (AI) neural networks. This new approach, which is 100 times faster than conventional methods, promises to significantly cut the power usage of AI applications. Rather than relying on the traditional iterative process, the researchers calculate parameters directly using probabilities, yielding results comparable in quality to those produced by existing methods.

The growing reliance on AI technologies, such as large language models (LLMs), has led to an exponential increase in the energy demands of data centers. In 2020 alone, these data centers consumed approximately 16 billion kWh in Germany, representing about 1% of the nation’s total energy usage. This figure is expected to rise to 22 billion kWh by 2025, underscoring the urgent need for more energy-efficient AI training methods.

The new technique, developed by Felix Dietrich, Professor of Physics-enhanced Machine Learning, and his team, focuses on utilizing probabilities rather than traditional iterative adjustments. By concentrating on critical data points where significant changes occur, this method enhances efficiency, using far less computational power. Dietrich explains, “Our approach allows us to determine the necessary parameters with minimal computational effort, resulting in faster and more energy-efficient neural network training. Moreover, the accuracy of this new technique is on par with that of traditional iterative methods.”

The researchers believe that this new approach could be a game-changer in fields that require energy-intensive AI applications, such as image recognition, language processing, and even climate modeling or financial forecasting. By reducing energy consumption while maintaining the quality of predictions, the method could help mitigate the environmental impact of AI development.

The work was presented at the 38th conference on Neural Information Processing Systems (NeurIPS) in 2024, with further findings published in Advances in Neural Information Processing Systems.

Publications

Rahma, Atamert, Chinmay Datar, and Felix Dietrich, “Training Hamiltonian Neural Networks without Backpropagation”, 2024. Machine Learning and the Physical Sciences Workshop at the 38th conference on Neural Information Processing Systems (NeurIPS) https://neurips.cc/virtual/2024/99994

Bolager, Erik L, Iryna Burak, Chinmay Datar, Qing Sun, and Felix Dietrich. 2023. “Sampling Weights of Deep Neural Networks.” In Advances in Neural Information Processing Systems, 36:63075–116. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2023/hash/c7201deff8d507a8fe2e86d34094e154-Abstract-Conference.html

Further information and links

• The Physics-enhanced Machine Learning professorship is part of the TUM School of Computation, Information and Technology
• Prof. Felix Dietrich is a core member of the Munich Data Science Institute (MDSI) and an associate member of the Munich Center for Machine Learning (MCML)
• AI research at TUM

Media Contacts:

Julia Rinner
Corporate Communications Center
presse@tum.de

Contacts to this article:

Prof. Dr. Felix Dietrich
Physics-enhanced Machine Learning
Technical University of Munich
felix.dietrich@tum.de
www.tum.de

SOURCE: Technical University of Munich