Interacting without touching: how AI is shaping a new reality

Ap2021

Amid the Covid-19 pandemic, a Swiss-South Korean research project is attempting to make public spaces accessible again with contact-free interactions helped by artificial intelligence (AI).

Picture yourself sitting at a coffee shop table with a group of friends or with your family, all the while a pandemic wreaking havoc. Meanwhile, you are chatting and laughing while enjoying your drink safely.  An intelligent system managing the interactions between guests tells you when to drink and when to wear the mask.

Swiss-South Korean project

The project “Contactless interactions through machine learning” involves a group of university students from South Korea as well as Swiss and European technology designers and developers in a workshop called “Machine learning for creatives”. It was held by Matteo Loglio, one of the research team members at SUPSI. The group combined their skills and cultural perceptions in order to “humanise”’ artificial intelligence, bringing it closer to people’s everyday needs.

The project was presented during the “Science Club” event in March 2021, organised by the Office of Science and Technology of the Swiss Embassy in South Korea. Swissinfo.ch moderated the event. You can watch it again here. 

The ongoing Covid-19 pandemic and its imposed distancing measures means touch between two people has become taboo. Shaking hands and hugging now seem like behaviour relegated to the past. A project carried out by the University of Applied Sciences and Arts of Southern Switzerland (SUPSI), in collaboration with the Hongik University in Seoul, is trying to imagine a “touch-free” society that maintains a sense of “real” contact while remaining as risk-free as possible. This is made possible via intelligent interfaces based on machine learning.

These interfaces aim not only to encourage compliance with protective measures, such as wearing a mask and keeping your distance, but also to completely reinvent public spaces.

Would we accept the presence of more technology in our life in exchange for a sense of “normal”? The Swiss-South Korean research team is trying to figure out where the possibilities and limitations lie.

“Mask traffic light” is a traffic light system that manages the consumption of drinks between groups of people in bars or cafes. “If the light is red, you have to hold the mask while someone else in your group drinks with their face uncovered. When it turns green, you can pull the mask down and consume your drink,” explains Jae Yeop Kim, professor of industrial design at Seoul’s Hongik University. SUPSI / Hongik University SUPSI / Hongik University

Cultural technology

“The aim is to use AI not as a control technology that is ‘behind the scenes’, but as a tool adapted to everyday life,” says Serena Cangiano, a researcher in charge of FabLab, SUPSI’s Laboratory of Visual Culture. It is coordinating the Swiss-South Korean project together with Hongik University’s Department of Industrial Design. 

In light of the challenges posed by the pandemic, the Swiss-Korean research group wants to redesign public spaces, making our relationship with technology more acceptable and natural in different social contexts. 

Digitalisation is increasingly clashing with a collaborative economy, characterised by the exchange and sharing of physical objects. Examples are shared private transport systems, such as cars, bicycles, and scooters, which are also becoming increasingly common in Switzerland. In the case of these infrastructures, touch remains the main sense people rely on, for example to unlock or drive one of the vehicles.

“In public spaces, we experience shared interfaces all the time,” says Cangiano. “Our research focuses on the design of totally touch-free interfaces and how it is possible to translate and rethink our daily interactions.”

How little touch can we handle?

According to Laura Crucianelli, a researcher in cognitive neuroscience, touch is fundamental to linking our minds and bodies to the social world. “Touch is the first sense by which we encounter the world, and the final one to leave us as we approach death’s edge,” the researcher wrote in an article in the digital magazine aeon. 

Touch is also the only sense that expects to be reciprocated. Several studies have shown the importance of touch for brain development and have linked behavioural and cognitive deficits of orphans to a lack of physical affectivity in the early years of their lives, Crucianelli writes. Some researchers have linked the lack of possibilities for touch during the pandemic to increased psychological problems in the general public.

Would a touch-free society really be desirable? “Covid-19 has changed our lives: the way we work, talk and eat,” says Jae Yeop Kim, professor at the Hongik University. Kim thinks the pandemic is forcing us to change and to use technologies such as machine learning to develop a completely new idea of society. 

Stemming from this concept, the designers and students in the two universities worked on various projects based on contact-free interaction, such as “contact-free karaoke”; the music stops if one of the participants doesn’t wear a mask or hold the microphone at the right distance. 

An example of how touch-free karaoke works. SUPSI / Hongik University

Another project involves using facial recognition software to identify people who are not wearing face masks and promote their use. “Wearing a face mask is not only a social issue, but also a matter of individual freedom and responsibility,” says the Korean professor.

Many of these projects have been developed based on cultural experiences in South Korea, and the Swiss-Korean team frequently discussed and analysed to what extent they would be embraced in other parts of the world. For Western countries, the team focused on making daily experiences more inclusive and “hygienic”. 

A team of European developers created disinfectant dispensers which are responsive to the height of the user. Another project optimised a contactless faucet for hands-free interactions especially in public toilets. The system uses machine learning to recognise individual gestures and regulate water temperature, all without a single touch. A demonstration is available online. 

Universal technology

Serena Cangiano says the collaboration between Switzerland and South Korea has made it possible to reflect on the pandemic in a very innovative and creative way, leveraging the unique assets of both countries in the field of technological research. “South Korea is a very technologically advanced country with a strong design culture, just like Switzerland,” she says. She adds that the cooperation between Switzerland and South Korea works well because both countries have a strong tech “heritage”.

But relationships and interactions are not conceived in the same way everywhere. Cangiano gives the example of protective masks, which are perceived very differently in Europe and Asia. “When we design new devices, and therefore new ways of interaction, we have to be aware that they will be accepted differently in different cultural contexts,” Cangiano points out.

The multicultural team of the project “Untouched interactions through machine learning” is working on the development of “touch-free” solutions based on artificial intelligence. SUPSI / Hongik University SUPSI / Hongik University

While the first phase of the project focused on creating a context of multicultural exchange, the aim now is to further facilitate collaboration between the two universities. The goal is to involve civil society and other organisations and institutions in a final stage.   

The aim is to create “universal” systems that can be adopted in different cultural environments.Cangiano explains: “The objective is not to develop an ideal solution only for Switzerland and South Korea but to leverage a design drive approach and machine learning as enabling technologies for more personalised, inclusive experiences, which can be valid in every cultural context.”