Combining AI and thermal video offers a new window into weightlifting

Researchers have developed a new method that combines video from thermal cameras with AI-based digital processing to enhance weightlifting training. By providing data-driven insights that enable targeted training and recovery strategies, the approach could help to optimize performance and safety in a variety of sport and exercise contexts.

Thermal, or infrared, images can provide valuable information for sports and health by tracking muscle activation and detecting areas of strain or fatigue. This information can be used to prevent injuries, monitor thermal responses and quantify physical exercise, ultimately helping athletes boost their skills. However, most current methods use before-and-after snapshots, offering only a limited view of the complex dynamics happening inside the body.

“We developed an approach that analyzes a complete thermal video recording while automatically obtaining the relevant parameters,” said research team leader Laura Viafora from Universidad de Concepción in Chile. “By analyzing the entire movement continuously, rather than just specific moments, we can track the progression of temperature variation in real-time and understand what happens throughout the entire exercise.”

In the Optica Publishing Group journal Applied Optics, the researchers show that the new method can be used to monitor muscle exertion during weightlifting using images from either an inexpensive thermal camera attached to a smartphone or a high-end thermal device, providing flexibility and accessibility for different training needs.

“Our goal is to continue developing this concept so that it can be used by athletes and coaches,” said Viafora. “We also aim to refine this technology for applications in the healthcare field by providing specialists with thermal data and information body position, which could contribute to more effective rehabilitation after an illness or injury.”

Turning images into insights

For a sport like weightlifting, athletes and coaches can get a lot of useful information from tracking small changes in temperature and body positioning throughout an exercise. To accomplish this, the researchers developed data processing algorithms and then used Google MediaPipe artificial intelligence software to identify individuals and their body parts within the images and extract the required information.

To obtain temperature data, the researchers identified the body part of interest and then converted the pixel color information into temperature values. For positional information, key points in the joints were identified and used to calculate the corresponding angles. Finally, to study the barbell’s movement, they detected the weight plates in the scene and then recorded the locations of their central points in each frame. Repeating this process for all frames allowed the researchers to produce graphs and frame-by-frame images tracking positional and temperature changes throughout the exercise.

To test the new method, they recorded athletes using a low-end thermal camera attached to a smartphone, a high-end stand-alone thermal camera and a conventional camera. This approach allowed them to compare results using various cameras and evaluate the effectiveness of body detection and angle estimation. The videos acquired with the various cameras were all uploaded to a computer for analysis.

These initial tests showed positive results, demonstrating that the new method could be used to generate a color-labeled sequence of thermal images from multiple camera types as well as reports on body part positions, which could be helpful for guiding athletes toward safer and more efficient practices.

Optimizing detection

The different camera types did come with some limitations. For example, both types of thermal cameras performed well at distinguishing the athlete’s body from the background, but this was found to be more complicated with a conventional camera. The quality of the thermal camera also influenced preliminary detection, highlighting the importance of selecting equipment that matches the level of precision needed. Some movements were also difficult to detect such as when the athlete was curled up or a weight obscured the body.

“Using thermal images to analyze the entire movement continuously allows us to obtain not only positional data throughout the exercise but also information about body temperature,” said Viafora. “We can see whether one side of the body is being exerted more than the other, and thermal images make it easier to identify individuals compared to conventional images, which can be influenced by background colors, clothing and skin tone.”

Now that they have demonstrated the feasibility of this approach, the researchers would like to use the method to analyze images of athletes from various disciplines, including conventional and paralympic sports. They are also working to refine their algorithm so that it can provide actionable information to users. For example, if an athlete raises their arm, the system could provide real-time feedback on whether the movement was performed correctly.

Paper: L. A. Viafora, S. N. Torres, G. Machuca, P. Gutierrez, A. Jara, P. Coelho, R. F. Soto, “An infrared imaging technique for weightlifting exercise assessment,” Applied Optics, 63, 28, 7529 (2024).
DOI: https://doi.org/10.1364/AO.532763

Image: Researchers combined video from thermal cameras with AI-based digital processing to enhance weightlifting training. From left to right, the images show pose detection results for a visible camera, a high-end infrared camera, a low-end infrared camera and angle calculations using specialized video annotation software.

View more Credit: Laura Viafora, Universidad de Concepción