Interactive Body Analyzer
Designed visualization to study
human interaction with data using gestures
Overview
Full Body Interaction beyond Fun: Engaging Museum Visitors in Human-Data Interaction.
Ongoing research on studying human interaction with data using human gestures and body as controls in collaboration of the research conducted by Dr. Francesco Cafaro. This project is NSF EAGER research funded.
Here is the link: https://soic.iupui.edu/news/cafaro-nsf-eager-grant/
Team Contributors: This project is a collaborative effort of Ph.D. Students, Milka Trajkova from HCI and Aaeshah Alhakamy from Computer science, Software engineering students, Rashmi Mallappa, Sreekanth Reddy, Rohit Pawar and HCI master student, Steffi Gogoi.
Duration: August 2018 – May 2019
My Role
UX Research and Analysis, Technology exploration research, Data visualization, Design mock UI
Problem Statement
The purpose of using gestures and body movements to control museum installations is that they create engagement and fun. There is often a hypothesis that the full-body technologies themselves are engaging for people, but this is not always true. In a real scenario, people generally interact with the museum exhibits for less than two minutes and if they are not able to quickly understand how to operate with the full-body interaction, they usually leave thinking that the system is broken and or is not working.
This is particularly problematic when visitors are exposed to interactive installations for data exploration, as in the case of Human-Data Interaction (HDI). For example, Puppet Parade, which is a fun activity that allows children to puppeteer creatures that are displayed on large projected walls which are not true in case of exploration of data as this is not a fun activity for young visitors. To make the HDI interaction to be self- standing, and to attract visitors of different age groups without the help of museum docents, it is important to identify the entry point to the interaction.
In today’s generation, the internet is bombarded with enormous of data, and data exploration is not only limited to research laboratories but could be found by surf the internet, read newspapers, or listen to debates on TV. As the data is available from wide sources so the patterns and correlation are easy to see. Henceforth, in modern science, it is important to understand the Correlation and Causation (and their relationship) because there are certain misconceptions about the concept and is hard to interpret. There are two common misconceptions that are required to be understood, firstly, confounding correlation with causation may lead people to assume causality when there is none, second, correlation does not imply causation although a correlation can offer hints at causality.
As there is a need to educate people on the cause and effect to apply in the modern scientific world, why not make it interesting by having full-body interaction to interact with the data and make it more engaging?
The Challenge
How do we make the museum exhibits more interesting and engaging to the visitors?
The Solution
An immersive experience to the museum visitors which will facilitate people’s understanding of exploring data without being manipulated and make it a fun learning process. The user can use full body interaction in the form of hand gestures, body movements to explore data.
Introducing Interactive Body Analyzer
Move and Click
Selects an item in the menu.
Rotate globe
Move in front and back to rotate the globe.
Hand Swipe
Provides the user the ability to change the data.
Jump
Moves the globe from upside down.
Zoom
Increases the size of the globe with zoom out feature and back to normal from zoom in feature.
Hand Grab (Fist)
Our Goal
The prime motive is to educate the teenagers to reason about the correlation and causation and to eradicate the myth of leading people to assume a correlation and causality concepts in the real world through real-time examples and let them differentiate it. The reason for educating the correlation and causation is that people are unaware of these two highly important terms applied in modern science.
The challenging and creative aspect of the work is to leverage the theory of interactive installation to involve tangible and embodied interaction to facilitate reason about causation and correlation. Eventually, this will serve as the priming effect of the interaction which may have a lasting learning effect on the way in which people understand STEM concepts and the scientific method (causality, the correlation between events, news etc..) and to distinguish scientific facts from fiction in a variety of contexts.
The Concept in detail
The interactive installation provides an educational tool in the science museum of Charlotte, NC to museum visitors especially teenagers by implementing interactive installation by using Kinect for full-body interaction in a fun learning way which will facilitate to learn STEM concepts like Correlation and Causation and to distinguish between them. Also, crafting interactive installations in which the interaction facilitates people’s comprehension of what they see on the screen.
It uses two globes on the screen display to interact with which will consist data of common terms to understand the correlation and causation concept that will help the museum visitors especially teenagers to understand it through the process of full-body interaction.
It is a great learning tool for education using the fun element of full-body interaction which gives the control to the body to interact with the data on the screen for the museum visitors as they get a privilege to experience interaction and immerse in a wholly new experience.
What is Correlation and Causation?
Correlation is usually defined as a measure of the linear relationship between two quantitative variables (e.g., height and weight). Often a slightly looser definition is used, whereby correlation simply means that there is some type of relationship between two variables.
When the values of one variable increase as the values of the other increase, this is known as a positive correlation. An example of positive correlation may be that the more you exercise, the more calories you will burn.
When the values of one variable decrease as the values of another increase to form an inverse relationship, this is known as negative correlation.
Causation, or causality, is the capacity of one variable to influence another. The first variable may bring the second into existence or may cause the incidence of the second variable to fluctuate.
Causation is often confused with correlation, which indicates the extent to which two variables tend to increase or decrease in parallel. However, correlation by itself does not imply causation. There may be a third factor, for example, that is responsible for the fluctuations in both variables.
A simple example is a strong correlation between ice cream sales and the number of drownings – both increase during the summer, but one does not cause the other.
What is Human-Data Interaction?
Human-Data Interaction (HDI) is a term that was initially introduced to interactive museum installations that allow multiple people to collaboratively explore large sets of data using hand gestures and body movements.
Working of the Kinect
The implementation in the Kinect is designed and coded in such a way that using full-body interaction, the data can be explored as displayed on the screen. The Kinect is coded with Unity and inputted the datasets relating to ice cream sales and the number of drownings, for example, which is visualized on the world map and the data representation is by the country color. Also, the datasets will involve certain variables which will be common to understand and can be the reason about correlation and causation. The datasets have been collected through the government websites to have real-time data that could be displayed on the globes. The user interface of the installation is designed for all the types of users but as the teenagers are our major target users, so it is highly designed to keep our users engaged. Also, there are different types of learners in a real scenario so we designed based on auditory, visual and kinesthetic learners that could be benefitted from this interactive feature. There will be an avatar that will have voice input which will be played initially to make them for what the interaction is all about. Visually, people can see the interactive globes on the display and by using full body gestures the kinesthetic learners can learn easily.
Working of the Full body interaction:
There are four strategies for providing entry-points to the interaction which involves:
1) Instrumenting the floor;
2) Forcing collaboration;
3) Implementing multiple body movements to control the same effect; and,
4) Visualizing the visitors’ silhouette beside the data visualization.
Also, there will be different gestures that will be implemented with the Kinect to operate with full body, for example, when you move back the globe will rotate in backward direction and when you move in front, the globe will rotate in forwarding direction, Similarly, we will include a clap feature that can also control the globe.
Target Clients, Consumers, and/or Market Competition (Impact)
This project implemented for Discovery place which is a science museum for children in Charlotte, NC and is open to all the museum visitors whether its children, their parents, or the teenagers and the adults. We are targeting teenagers who will be able to understand and reason about correlation and causation. Other than that, any person who is curious to understand the underlying concepts and have a fun learning experience by interacting with data can visit the museum and experience it.
Our main target users are teenagers ages from 16-21. This project will help the museum directors to showcase another interactive element that will be unique to this museum as there is no implementation of full body interaction with any other museums. It will also help the researchers to see it getting implemented as this is a huge cost attached to this implementation provided the time investment of each resources working in this project.
Product Differentiation with respect to Competition
Interactive museum installation is exclusively designed and developed keeping museum visitors in mind. There are certain installations in museums who have implemented full-body technologies like Metamorphosis allows visitors to go through the stages of a butterfly’s metamorphosis using “engaging ubiquitous technologies”; Word Out! is an interactive game (designed to facilitate the learning of alphabet letters) that uses full-body technologies to “stimulate fun and engagement”.
Our project is unique as it uses full-body interaction to facilitate learning STEM concepts of Correlation and Causation.
Next Steps
I am working on the technical exploration and research on the tangible objects utilized to interact with the data on the screen and operate the globe.