Abstracts Track 2024


Area 1 - Interactive Devices

Nr: 70
Title:

Designing Diversity Computing Spaces: How to Foster Meaningful Interactions for Early Adolescents - Two Hands-on Examples

Authors:

Rafael Vrecar, Anna Blumenkranz, Moritz Kubesch and Christopher Frauenberger

Abstract: Within the so-called “Diversity Computing Spaces project” (DivComp), funded by the FWF (1), we aimed to develop two Diversity Computing Spaces, based on the concept of “Diversity Computing” [3]. As described by principal investigator Christopher Frauenberger, “[...] This project takes a design research approach to investigate how interactive technologies can create smart, physical spaces that scaffold shared, meaningful experiences for diverse groups of people.” (2). Our overall context is a school setting with 10-14 year old students. While we already successfully presented the design process at CHI [2, 1], this abstract and the (hopefully!) accompanying demonstration aim to show the two resulting spaces in detail and action. The first diversity computing space consists of four areas of activity: A) a “bench” consisting of five cushions which act as buttons (true/false, i.e., a person sits on it vs. no one sits on it); B) a giant round glass drawing board (diameter: 60cm), and C) two smaller glass drawing boards. All drawing boards are filmed with webcams from below and streamed to three corresponding screens. To make the space more engaging, the video streams are altered through different effects using Shadertoy (3). The effects chosen are influenced by how many students sit on the bench and which cushions are used. The results are presented on one big and two small screens. The big screen shows the video feed of the large drawing area, and the small screen shows the corresponding smaller drawing areas. However, the big screen via picture-in-picture also shows the two smaller drawing areas. The whole space is connected through a custom-made scaffold, separating it from its surroundings while making it an open and inviting area. The second diversity computing space centers around a custom-made table where people can kneel around on different soft carpets or pillows. It consists of three activity areas, each being an iPad (10th generation). Each iPad offers a different input our output modality. One iPad acts as a keyboard and allows textual input (“storytelling”), and another iPad allows drawing with one’s fingers (“illustrating”). Here, the line thickness and the color of the line can be chosen freely. The third iPad shows the result (“artwork”) composed by a locally running generative artificial intelligence from the prompt and input drawing. The output is an image that takes the two inputs as a basis. Both interactions are very different, yet aim towards the same goal: to create a meaningful interaction for all people engaging with(in) those spaces. In both cases, the adolescents can take on different roles and switch them as they please. People can focus on drawing, figuring out how the seating arrangement influences the effects or just watching. In the second prototype, they can work together or even against each other. For example, they can draw and write towards the “same” goal, i.e., a specific image. However, they can also completely contradict each other, and look what happens when they “confuse” the AI. Regardless of the results and role an individual takes on in these interactions, a “side goal” is to expose the students to the concepts of video effects and, more importantly, the strengths and weaknesses/pitfalls of generative AI. (1) Austrian Science Fund (2) https://divcomp.frauenberger.name/en (3) https://www.shadertoy.com/ [1] DOI 10.1145/3613904.3642240. [2] 10.1145/3544548.3581155. [3] 10.1145/3243461.

Nr: 81
Title:

Designing a Gesture-Based System to Control Software in Augmented Reality

Authors:

Albert Lukasik, Jacek Matulewski, Klaudia Karkowska, Monika Boruta-Zywiczynska, Izabela Grzankowska and Michał Joachimiak

Abstract: Artificial intelligence (AI) and augmented reality (AR) interaction has been developing rapidly. While interacting with this type of technology, not only do we want communication to be user- and creator-friendly, but we also want it be efficient. In recent years, there has been a tendency to use interfaces that do not require "unnatural" intermediaries, such as a keyboard and mouse. Examples are speech-operated assistants such as Alexa by Amazon or Siri by Apple. In our research, we looked at the possibility of using another semiotic system - gesture - as a means of operating a software in AR. We decided to test the idea that gestures are a fairly universal and convenient form of communication, while interacting with an app. In our talk, we present the steps that led to the development of the system of simple, single-hand, static gestures that facilitate human-computer interaction. We discuss the possible implementation of the app described, and we briefly look at psychometry and its potential use when predicting AR interaction style of users. The key part of our presentation is the development of the said control system. The process started with a selection of actions to operate AR applications (such as: forwards, play, mute, or screen share). We chose 50 action-keys to operate the interface and asked a group of experts to propose gestures that would be most intuitive to use for the selection. Then, a set of static gestures (images) was presented to a group of naive participants who were asked to describe the function they would assign to each gesture. The group was also asked to fill in a survey on the ease of gesture production, and to complete a battery of psychological tests, e.g. temperament measure, attention span, or short-term memory test. All of the participants were later invited to reproduce gestures using an AR headset. After the testing phase, we observed a few areas in which we could improve the proposed system further. For instance, participants found it difficult to recall gestures assigned to scrolling the screen. In such situations, they resorted to pointing to launch a function. Such a result prompted us to make changes in the interface, such as adding a slider. Another problem was that participants used recurrent gestures for several options, which imposed a limit on the gestures used to move around the interface. It turned out to be difficult to memorize 50 gestures for 50 different options.