A noted Indian dancer traveled to the University of Iowa in fall 2023 to help an engineering professor train and test artificial intelligence (AI) algorithms to recognize and classify changing features through a classical dance called Bharatanatyam, which originated in the South Indian state of Tamil Nadu. 

Ananya Sen Gupta, associate professor of electrical and computer engineering, has been working with Subhajit Khush Das, an artist and choreographer from Kolkata, India, on this STEAM effort. STEAM refers to an approach to learning that integrates art into the traditional STEM fields of science, technology, engineering, and math.  

Sen Gupta, an expert in pattern recognition for environmental sensing, studies salient information across sensor data collected in dynamic oceanic and space environments. A difficult challenge is to train and test popular machine learning techniques to discover and learn feature patterns that change dynamically with time-varying environmental conditions. This is where dance can help.  

Sen Gupta had watched Das’s YouTube videos and believed his ability to create geometrically accurate and intricate patterns using the human form could teach AI to recognize patterns and shapes in other environments, such as the ocean or space.  

For example, AI may struggle to differentiate between a mine buried on the ocean floor, an interesting rock, or marine life that has come to investigate.   

“Features morph,” said Sen Gupta, also a faculty affiliate of the Iowa Technology Institute. “They change their shape and can get entangled like dreadlocks. That’s where algorithms of today are not smart enough to detangle the overlapped morphing features, as it is hard to separate features against the interference, non-linear overlaps, and changing environmental conditions. It’s hard to know what they are looking for using conventional training and testing setups. A lot of classical theory falls.”  

Sen Gupta uses Das's unique hand gestures (mudras), which resemble smoothly changing geometric features, for their research.

Das began his maiden United States tour working to create a unique geometric data set for use in AI research. Das spent several weeks at Iowa’s campus, where he performed and customized dances that could be recorded and used to train and test the AI.

As part of the visit, he also co-presented with Sen Gupta at the Iowa Department of Physics and Astronomy colloquium and to the Department of Electrical and Computer Engineering advisory board. Sen Gupta jointly presented a paper on the research with Das at OCEANS 2023, a conference sponsored by the Institute of Electrical and Electronics Engineers (IEEE) Oceanic Engineering Society.  

To facilitate these research and collaborative activities, where Das was the performing artist and Sen Gupta the scientist, Sen Gupta utilized Iowa Technology Institute departmental funds to cover all trip costs and Das's honorarium. The data analysis portion of Sen Gupta’s research is being sponsored by the Office of Naval Research with expected funding of $300,720 over three years.   

The STEAM collaboration between Das and Sen Gupta also enabled other faculty to meet and collaborate with Das. Das leveraged his presence on campus to teach masterclasses in the Iowa Department of Dance and collaborated with Tyler Bell, assistant professor of electrical and computer engineering, in a virtual reality experiment.