A noted Indian dancer traveled to University of Iowa in fall of 2023 to help an engineering professor train and test artificial intelligence (AI) algorithms to recognize and classify changing features through Bharatanatyam, a classical dance that originated in the South Indian state of Tamil Nadu. 

Ananya Sen Gupta, associate professor of electrical and computer engineering, has been working with Subhajit Khush Das, an artist and choreographer from Kolkata, India, for more than a year on this STEAM effort. STEAM refers to an approach to learning that integrates art into the traditional STEM (Science, Technology, Engineering and Math) fields. 

Sen Gupta, an expert in pattern recognition for environmental sensing, routinely studies salient information across sensor data collected in dynamic oceanic and space environments. A hard challenge is to train and test popular machine learning techniques to discover and learn feature patterns that change dynamically with time-varying environmental conditions. This is where dance can help.

Sen Gupta had watched Das’ YouTube videos and believed his ability to create geometrically accurate and intricate patterns using the human form could teach AI to recognize patterns and shapes in other environments, such as the ocean or space. 

For example, AI may struggle to differentiate a mine buried on the ocean floor, an interesting rock, or marine life that has come to investigate.  

“Features morph,” said Sen Gupta, who is also a faculty affiliate of the Iowa Technology Institute. “They change their shape and can get entangled like dreadlocks. That’s where algorithms of today are not smart enough to detangle the overlapped morphing features as it is hard to separate features against the interference, non-linear overlaps, and changing environmental conditions. It’s hard to know what they are looking for using conventional training and testing setups. A lot of classical theory falls.” 

Sen Gupta uses Das' unique hand gestures (mudras), which resemble smoothly changing geometric features, for her research. 

Sen Gupta formally invited Das to start his maiden U.S. tour working with them to create a unique geometric dataset that they can use for their AI research. Das spent several weeks at the University of Iowa working with Sen Gupta where he performed and customized dances that could be recorded and used to train and test the AI.

As part of the visit, he also co-presented with Sen Gupta at the University of Iowa Department of Physics and Astronomy colloquium and the Department of Electrical and Computer Engineer advisory board. Sen Gupta jointly presented a paper on the research with Das at OCEANS 2023, a conference sponsored by IEEE (Institute of Electrical and Electronics Engineers) Oceanic Engineering Society. 

To facilitate these research and collaborative activities, where Das was the performing artist and Sen Gupta the scientist, Sen Gupta utilized their Iowa Technology Institute departmental funds to cover all trip costs and Das' honorarium. The data analysis portion of Sen Gupta’s research is being sponsored by the Office of Naval Research with expected funding of $300,720 over three years.  

The STEAM collaboration between Das and Sen Gupta also enabled other faculty to meet and collaborate with Das. Das leveraged his presence on campus to teach masterclasses in the Iowa Dance Department and also collaborate with Prof. Tyler Bell in a virtual reality experiment.