Welcome to the DIS’22 conference program.

We wish to acknowledge the people of the Kulin Nations, on whose land we are standing today, and we pay our respects to their Elders, past and present.

The DIS’22 program responds to the need for designers and design researchers to not only design for wellbeing, but also to look after their own wellbeing and the people around them, through and with design.

We had over 1600 registrations to our free offering, with registration now closed. DIS 2022 is an online conference following an asynchronous format. This means that there are no scheduled sessions and there is no need to attend in-person or at a specific time of the day. You can read every PDF for free, watch a prerecorded teaser video and the presentation at your own time. We encourage all authors to monitor Twitter and Discord (see below) for questions during the conference. So if you have questions, please get in touch with the authors. Of course, interested parties could also ask questions later, however, in line with our theme of digital wellbeing, we encourage to focus interactions on the conference time window.

All program content is available below, but also accessible via the Proceedings page on the ACM digital library. You can also watch all teaser videos within one large playlist, and here is also a playlist of all presentations. The Doctoral Consortium is for invited PhD candidates only (who have received separate instructions). Their submissions, however, are accessible in the ACM Companion Proceedings .

We encourage a positive and supportive environment, so we please ask you to familiarise yourself with the policy against harassment at ACM activities.

Thank you to everyone who helped to make this happen, all the volunteers and all the authors, so please enjoy the great articles!

Table of Contents

• Accessibility & Privacy
• Computational Design
• Connectedness
• Crafting & Materials
• Design Theory & Critical Design
• Game Design
• Immersion
• Interaction Techniques
• Multisensory Design
• Space and Interaction
• Wellbeing and Health

DIS ’22: Designing Interactive Systems Conference

Full Citation in the ACM Digital Library

SESSION: Accessibility & Privacy

Personalized Font Recommendations: Combining ML and Typographic Guidelines to Optimize Readability

• Tianyuan Cai
• Shaun Wallace
• Tina Rezvanian
• Jonathan Dobres
• Bernard Kerr
• Samuel Berlow
• Jeff Huang
• Ben D. Sawyer
• Zoya Bylinskii

The amount of text people need to read and understand grows daily. Software defaults, designers, or publishers often choose the fonts people read in. However, matching individuals with a faster font could help them cope with information overload. We collaborated with typographers to (1) select eight fonts designed for digital reading to systematically compare their effectiveness and to (2) understand how font and reader characteristics affect reading speed. We collected font preferences, reading speeds, and characteristics from 252 crowdsourced participants in a remote readability study. We use font and reader characteristics to train FontMART, a learning to rank model that automatically orders a set of eight fonts per participant by predicted reading speed. FontMART’s fastest font prediction shows an average increase of 14–25 WPM compared to other font defaults, without hindering comprehension. This encouraging evidence provides motivation for adding our personalized font recommendation to future interactive systems.

Addressing Adjacent Actor Privacy: Designing for Bystanders, Co-Users, and Surveilled Subjects of Smart Home Cameras

• James Pierce
• Claire Weizenegger
• Parag Nandi
• Isha Agarwal
• Gwenna Gram
• Jade Hurrle
• Hannah Liao
• Betty Lo
• Aaron Park
• Aivy Phan
• Mark Shumskiy
• Grace Sturlaugson

Many consumer Internet Things (IoT) devices involve spatial sensors such as cameras and microphones. These affect the privacy of nearby people. A prime example is smart home security cameras. We present our work developing scenarios, use cases, and design proposals for addressing smart camera privacy. Preliminary findings from a concept evaluation with 11 participants is presented. The outcomes of this research through design project foreground the importance and challenges of designing to support the privacy of nearby users. We outline actionable design responses while also raising limitations of technology approaches alone to address these issues.

PrivacyToon: Concept-driven Storytelling with Creativity Support for Privacy Concepts

• Sangho Suh
• Sydney Lamorea
• Edith Law
• Leah Zhang-Kennedy

With privacy-related concepts often abstract and difficult to define, comics can be an effective visual storytelling medium for explaining and raising awareness about privacy. However, existing privacy and security educational comics do not support content creation. To address this, we contribute PrivacyToon, a comic-based authoring tool that leverages concept-driven storytelling and ideation cards to help users create customizable privacy-related visual content. Our exploratory user study with 23 students and teachers shows PrivacyToon’s potential as a creative tool for communicating privacy concepts and stories. Our results show that a wide range of creativity preferences and contexts must be considered when designing systems that integrate ideation card-based design processes.

SESSION: Computational Design

Stories from the Frontline: Recuperating Essential Worker Accounts of AI Integration

• Esther Y. Kang
• Sarah E. Fox

This paper aims to elevate essential worker accounts of the introduction of AI technology amid the COVID-19 pandemic. Drawing from a mix of ethnographic observations, interviews, and participatory design encounters with frontline staff, we examine the experiences of workers in a waste management facility in the United States newly tasked with overseeing autonomous floor cleaning robots. To complement and extend managerial and engineering descriptions emphasizing the functionality and performance of these devices, we used recuperative approaches to re-center the socio-material realities of workers on-the-ground. For example, workers reported concerns on the safety of the devices in congested areas and a need for more comprehensive training across all levels of the organization. This research seeks to expand the discourse on ethical AI by situating essential workers as a key source in developing best practices for deploying new technologies and evaluating pilot projects.

Designing Trustworthy User Interfaces for the Voluntary Carbon Market: A Randomized Online Experiment

• Klaudia Guzij
• Michael Froehlich
• Florian Fincke
• Albrecht Schmidt
• Florian Alt

The voluntary carbon market is an important building block in the fight against climate change. However, it is not trivial for consumers to verify whether carbon offset projects deliver what they promise. While technical solutions for measuring their impact are emerging, there is a lack of understanding of how to translate carbon offset data into trustworthy interface designs. With interaction between users and offset projects mainly happening online, it is critical to meet this design challenge. To this end, we designed and evaluated interfaces with varying trust cues for carbon offset projects in a randomized online experiment (n=244). Our results show that content design, particularly financial and forest-related quantitative data presented at the right detail level, increases the perceived trustworthiness, while images have no significant effect. We contribute the first specific guidance for interface designers for carbon offsets and discuss implications for interaction design.

Slanted Speculations:: Material Encounters with Algorithmic Bias

• Gabrielle Benabdallah
• Ashten Alexander
• Sourojit Ghosh
• Chariell Glogovac-Smith
• Lacey Jacoby
• Caitlin Lustig
• Anh Nguyen
• Anna Parkhurst
• Kathryn Reyes
• Neilly H. Tan
• Edward Wolcher
• Afroditi Psarra
• Daniela Rosner

Over the past few years, AI bias has become a central concern within design and computing fields. But as the concept of bias has grown in visibility, its meaning and form have become harder to grasp. To help designers realize bias, we take inspiration from textile bias (the skew of woven material) and examine the topic across its myriad forms: visual, textual, and tactile. By introducing a slanted experience of material and therefore of reality, we explore the translation of fraught machine learning algorithms into personal and probing artifacts. In this pictorial, we present nine pieces that materialize complex relationships with machine learning; ground these relationships in the present and the personal; and point to generative ways of engaging with biased systems around us.

“It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation

• Dilruba Showkat
• Eric P. S. Baumer

The rise of automated text processing systems has led to the development of tools designed for a wide variety of application domains. These technologies are often developed to support non-technical users such as domain experts and are often developed in isolation of the tools primary user. While such developments are exciting, less attention has been paid to domain experts’ expectations about the values embedded in these automated systems. As a step toward addressing that gap, we examined values expectations of journalists and legal experts. Both these domains involve extensive text processing and place high importance on values in professional practice. We engaged participants from two non-profit organizations in two separate co-speculation design workshops centered around several speculative automated text processing systems. This study makes three interrelated contributions. First, we provide a detailed investigation of domain experts’ values expectations around future NLP systems. Second, the speculative design fiction concepts, which we specifically crafted for these investigative journalists and legal experts, illuminated a series of tensions around the technical implementation details of automation. Third, our findings highlight the utility of design fiction in eliciting not-to-design implications, not only about automated NLP but also about technology more broadly. Overall, our study findings provide groundwork for the inclusion of domain experts values whose expertise lies outside of the field of computing into the design of automated NLP systems.

From Explainability to Ineffability?: ML Tarot and the Possibility of Inspiriting Design

• Caitlin Lustig
• Daniela Rosner

Explainability has become a dominant aspect of developing more accountable AI systems. But for AI to be more accountable, we as designers must also reflect on our own positioning—including aspects of ourselves that are difficult or impossible to fully explain, yet still influence our design processes. Just as designers develop explainable AI, we explore how AI can be used to develop the unexplainable designer through documenting our creation and use of a machine learning-generated tarot deck. Alongside design researchers, we consider the promise of such machine learning-generated artifacts for self-reflection on our creative and collaborative roles—and our responsibilities—when designing AI systems, and we discuss how the artifact sparked acts of inspiriting, a process of bringing the ineffable (back) into our engagements with machine learning systems.

A Scoping Review of Ethics Across SIGCHI

• Giovanna Nunes Vilaza
• Kevin Doherty
• Darragh McCashin
• David Coyle
• Jakob Bardram
• Marguerite Barry

Ethical deliberation has proved a consistent feature of Human-Computer Interaction (HCI) since its earliest years, spanning the respectful involvement of research participants to design choices impacting fairness, freedom and welfare. Despite growing discussions, applied knowledge and practical approaches for navigating complex moral dilemmas remain challenging to grasp. Motivated by the need for a structured overview, this paper contributes a scoping review of ethics as discussed across 129 full-length SIGCHI papers containing the search term ‘ethic*’ in their title, abstract or authors’ keywords over the last ten years. Findings show increasing prioritisation of the topic, particularly within Artificial Intelligence. Value-Sensitive and Critical Design appear as the most frequently applied orientations, and participatory approaches are more prevalent than those without end-user input. Engaging with a spectrum from personal to societal concerns, the SIGCHI literature thus echos calls for critical perspectives on user-centred processes and the need to establish more sustainable responsibility structures.

Blockchain and Cryptocurrency in Human Computer Interaction: A Systematic Literature Review and Research Agenda

• Michael Froehlich
• Franz Waltenberger
• Ludwig Trotter
• Florian Alt
• Albrecht Schmidt

We present a systematic literature review of cryptocurrency and blockchain research in Human-Computer Interaction (HCI) published between 2014 and 2021. We aim to provide an overview of the field, consolidate existing knowledge, and chart paths for future research. Our analysis of 99 articles identifies six major themes: (1) the role of trust, (2) understanding motivation, risk, and perception of cryptocurrencies, (3) cryptocurrency wallets, (4) engaging users with blockchain, (5) using blockchain for application-specific use cases, and (6) support tools for blockchain. We discuss the focus of the existing research body and juxtapose it to the changing landscape of emerging blockchain technologies to highlight future research avenues for HCI and interaction design. With this review, we identify key aspects where interaction design is critical for the adoption of blockchain systems. Doing so, we provide a starting point for new scholars and designers and help them position future contributions.

Test-retest Reliability on Movement Times and Error Rates in Target Pointing

• Shota Yamanaka

Target-selection tasks have been frequently conducted to evaluate novel input devices and pointing-facilitation techniques. Recently, Sharif et al. showed that a unified metric to evaluate the pointing performance, called throughput TP, was not stable across two sessions performed by the same participant group, which indicates poor test-retest reliability. Because there are cases in which using TP is inappropriate depending on the research topic, we extend their finding to two other metrics: movement time MT and error rate ER. We demonstrated that, even for the participants who kept their TPs across two sessions stable, they would exhibit unstable MTs and ERs. Thus, if time allows, researchers should design their experiments to run multiple sessions for obtaining the central tendency of user performance, which increases the validity of their user studies.

MoodCubes: Immersive Spaces for Collecting, Discovering and Envisioning Inspiration Materials

• Alexander Ivanov
• David Ledo
• Tovi Grossman
• George Fitzmaurice
• Fraser Anderson

In early stages of creative processes, practitioners externalize and combine inspirational materials, using strategies such as mood board creation to achieve a desired vision and aesthetic. Yet, collecting and combining materials can be difficult: (1) mood boards bias towards 2D images, neglecting audio, video, and 3D models; (2) alternative externalizations such as prototypes are best suited for later stages and can be time-consuming and tedious to create; and (3) online searches lead to disjointed sources between different websites and assets in the file system. To address these challenges, we created MoodCubes, a system for rapid creation and manipulation of multimedia content. When adding content, MoodCubes decomposes objects (e.g., extracting colour palettes), suggests new materials without the need to search (e.g., 3D models, images, lighting effects), and provides filters to change the scene’s aesthetic. We studied eight creative professionals using MoodCubes, which suggested ways the system might advance existing design practices.

Designing Interactive Visuals for Dance from Body Maps: Machine Learning and Composite Animation Approaches

• Nuno N. Correia
• Raul Masu
• William Primett
• Stephan Jürgens
• Jochen Feitsch
• Hugo Plácido da Silva

There is a growing interest in interactive visuals for dance performance. Recent research has identified potential in using interactive visuals to convey to the audience otherwise non-visible elements of performances. Informed by soma design, and with a co-design perspective, we aim to make apparent non-visible bodily aspects of dancers. We propose to design interactive visuals from body maps, following two approaches – Machine Learning and Composite Animation. We conducted a multi-stage study involving 12 dancers. We present and discuss the results of our evaluations, confirming that both prototypes were successful in addressing our aim, with some limitations. We discuss our two approaches, different uses and actors in different stages, tensions between research and dance creation, and potential applications. Our main contributions are the two approaches for designing interactive visuals from body maps and their analysis. These are materialized in two software systems released as open-source and in their design framework descriptions.

SESSION: Connectedness

Flexibility and Social Disconnectedness: Assessing University Students’ Well-Being Using an Experience Sampling Chatbot and Surveys Over Two Years of COVID-19

• Fiona Draxler
• Linda Hirsch
• Jingyi Li
• Carl Oechsner
• Sarah Theres Völkel
• Andreas Butz

COVID-19 caused an abrupt switch from face-to-face to online teaching. This led to unknown challenges and consequences for students and lecturers. In the first semester after its outbreak, we developed a messenger-based chatbot to perform an experience sampling study to evaluate students’ well-being and experiences (n = 31) with the radical changes in higher education. Finding a decrease in students’ perceived motivation but an increase in productivity, we conducted a follow-up survey to compare the development a year later (n = 41). Our results revealed two main student profiles, one feeling severely impacted by the persisting social distance in their study performance and the other appreciating the flexibility and expended free time due to the changes in the teaching formats. Based on our findings, we introduce implications for the overall design of higher education and show the benefits and challenges of combining chatbot-enabled experience sampling with traditional surveys.

Artist Support Networks: Implications for Future Creativity Support Tools

• John Joon Young Chung
• Shiqing He
• Eytan Adar

The artist as a solitary genius does not reflect the reality of art-making. To enable art-making, artists are supported by many other people—subcontractors, collaborators, etc.—who collectively form an Artist’s Support Network. Through an interview of 14 artists, we map the space of relationship types, provided support, interactions, failures, and successes of human support relationships. Moreover, we identified the patterns by which these aspects relate to each other in different support relationships. As technologically-driven Creativity Support Tools (CSTs) emerge to augment and automate portions of the artist’s support network, the detail of these interactions becomes critical. Existing sites of collaboration in support networks invariably shape artists’ expectations. How a CST fits within existing interaction expectations will shape the design, the artist’s understanding, and ultimately, acceptance. With this lens, we reflect on how a CST’s design–and in particular, those support collaboration and AI-driven variants–will mesh with the artist’s support network.

Co-designing Digital Platforms for Volunteer-led Migrant Community Welfare Support

• Joshua Paolo Seguin
• Delvin Varghese
• Misita Anwar
• Tom Bartindale
• Patrick Olivier

Community-based migrant organizations play a vital role in the provision of welfare services to temporary migrant workers, international students, and refugees whose access to government support services are limited. Through a co-design based inquiry, we explored the potential to utilize mainstream digital platforms to support the welfare agenda of an Australia-based Filipino migrant organization. We discuss the advantages and disadvantages of implementing such technology-mediated workflows within the community-based organization and the essential practices that they may undermine. Drawing on this case study, we present a provisional set of reflections for design practitioners working in the space of migrant communities and other marginalized community groups. These include the importance of designing for the community’s long-term and holistic development, leveraging volunteers’ digital literacy, and a call for more malleable platforms that allow community groups to reflect their core values and needs directly onto the platform configurations.

“Slurp” Revisited: Using ‘system re-presencing’ to look back on, encounter, and design with the history of spatial interactivity and locative media

• Shengzhi Wu
• Daragh Byrne
• Ruofei Du
• Molly Wright Steenson

Hand-based gestural interaction in augmented reality (AR) is an increasingly popular mechanism for spatial interactions. However, it presents many challenges. For example, most hand gesture interactions work well for interactions with virtual content and interfaces, but seldom work with physical devices and users’ environment. To explore this, and rather than inventing new paradigms for AR interactions, this paper revisits Zigelbaum, Kumpf, Vazquez, and Ishii’s 2008 project ‘Slurp’ [72] – a physical eyedropper to interact with digital content from IoT devices. We revive this historical work in a new modality of AR through a five step process: re-presecencing, design experimentation, scenario making, expansion through generative engagements with designers, and reflection. For the designers we engaged, looking back and designing with a restored prototype helped increased understanding of interactive strategies, intentions and rationales of original work. By revisiting Slurp, we also found many new potentials of its metaphorical interactions that could be applied in the context of emerging spatial computing platforms (e.g., smart home devices). In doing so, we discuss the value of mining past works in new domains and demonstrate a new way of thinking about designing interactions in emerging platforms.

EmbER: A System for Transfer of Interoceptive Sensations to Improve Social Perception

• Caitlin Morris
• Valdemar Danry
• Pattie Maes

Remote social interactions suffer from a loss of nonverbal cues used to build affiliation and connection. We propose the use of novel sensory channels for sharing social cues from interoceptive data through wearable devices that simulate the breathing and heartbeat patterns of another person, known to be linked to emotional perception and affect. We conducted a study with 16 participants testing the sharing of either heart rate or breathing rate through haptic or audio sensations. Participants experienced these sensations while watching videos of narrators describing personal experiences. We assessed the subjects’ feelings of affiliation and synchrony toward a narrator through surveys, interviews, and correlated physiological data. Our findings show that sensory devices that transfer interoceptive sensations, especially those below the level of conscious perception, can have a positive impact on feelings of connectedness. This has implications for the application of physiological channels in remote interactions to improve social connection.

Slide2Remember: an Interactive Wall Frame Enriching Reminiscence Experiences by Providing Re-encounters of Taken Photos and Heard Music in a Similar Period

• Subin Kim
• Sangsu Jang
• Jin-young Moon
• Minjoo Han
• Young-Woo Park

The abundance of easily captured digital media and storing everything “just in case” has made it difficult for people to revisit them and have a rich reminiscence experience. To reinstate this value, we developed Slide2Remember, a wall photo frame enabling people to re-encounter their histories of digital photos and music. By physically sliding the device’s front cover, users can hear songs that they had listened to in the same period when the photo was taken. From our four-week in-field study with 7 participants, we showed how Slide2Remember supported narrative remembering experiences of past moments and found detailed roles of each medium in the recall of life events. Moreover, the sliding interaction encouraged participants to begin their retrospection with anticipation and curiosity. Our findings imply considerations for providing a new recalling experience by overlaying one digital possession type over another and ways to encounter past data in everyday living spaces.

Covid Connect: Chat-Driven Anonymous Story-Sharing for Peer Support

• Christopher Collins
• Simone Arbour
• Nathan Beals
• Shawn Yama
• Jennifer Laffier
• Zixin Zhao

The mental-health impact of the Covid-19 pandemic and the related restrictions and isolation have been immense. In this paper, we present a system designed to break down loneliness and isolation, and to allow people to share their stories, complaints, emotions, and gratitude anonymously with one another. Using a chatbot interface to collect visitor stories, and a custom visualization to reveal related past comments from others, Covid Connect links people together through shared pandemic experiences. The collected data also serves to reflect the experiences of the community of participants during the third through fifth waves of the pandemic in the local region. We describe the Covid Connect system, and analyze the collected data for themes and patterns arising from stories shared with the chatbot. Finally, we reflect on the experience through an autobiographical lens, as users of our own system, and posit ideas for the application of similar approaches in other mental health domains.

Scaffolding Young People’s Participation in Public Service Evaluation through Designing a Digital Feedback Process

• Andy Dow
• Kyle Montague
• Shauna Concannon
• John Vines

Young people facing marginalisation often rely on publicly funded services for support. Such services must include users in improving their provision, but often lack the processes and tools to facilitate this. The civic turn within HCI means that we are still tackling the complexities of community-based design research required to provide digital tools of relevance to public services. To address this, we worked with groups of young people to explore the design of a service evaluation process, supported by digital resources, intended to support marginalised youths to influence service delivery. Our findings demonstrate how the groups of young people participating in processes of service evaluation using our digital tools embraced the opportunity to express themselves. We also identify tensions from the social values underpinning the youth voluntary sector that impede their participation. We close by discussing challenges for community-based design and implications for digital technologies that facilitate the participation of marginalised young people in civic processes.

The Air Quality Lens: Ambiguity as Opportunity to Reactivate Environmental Data

• Teis De Greve
• Steven Malliet
• Niels Hendriks
• Bieke Zaman

The use of low-cost sensors to collect environmental data can enable citizens to express environmental concerns and foster community building and activism. However, when made public, citizen data is often detached from the subjective experiences integral to citizen sensing. Our work with youngsters from diverse backgrounds explores how existing environmental datasets can be reactivated to engage new stakeholders and discussions. We present a research through design project with air quality data and draw attention to the role of ambiguity in our design process. We synthesize our reflections by discussing three design aspects that can make sense of ambiguity and encourage critical engagements with environmental data. Our goal is to offer a design-oriented account of how citizen-generated environmental data can be reactivated to express matters of concern.

Experiential Value in Group Browsing of Curios on eBay and In-Person: Implications for Future Platform Design

• Effie Le Moignan
• Tom Feltwell
• David Kirk

eBay is the world’s largest auction site, with a variety of antique, vintage and one-off items listed. In-person shopping for these frequently takes place within markets and antiques fairs, underpinned by a rich set of co-browsing practices. However, these collective shopping experiences are poorly supported online, with platforms predominantly designed to accommodate and support an individual user experience. Using collective autoethnography (CAE) over a period of 15 months we engaged in a series of (16) online co-browsing sessions using a configuration of video-mediated communication to co-browse eBay together, augmenting the interface to introduce sociality. This was supplemented by 8hrs of field studies split over two in-person co-browsing sessions within physical retail locations. From these studies we draw together insights on the experiential nature and practices of co-browsing on and offline, presenting recommendations to inform the design of future online retail spaces, with an aim to support better collaboration and sociality.

The Shape of Social Media: Towards Addressing (Aesthetic) Design Power

• Kay Kender
• Christopher Frauenberger

The effects of social media on our society are heavily researched and discussed, but few insights about the role of aesthetic design therein have been gained to this date, despite research in related areas providing precedent: drawing from existing theory on persuasive design, nudge, dark patterns, and advertising, we suggest the term Aesthetic Design Power to conduct a preliminary critical analysis of the design of four influential social media platforms (Facebook, Twitter, Instagram, and TikTok) as a foundation for critical discussions about the aesthetic design of social media and its impact. We present salient design attributes and elements shared by these four platforms, speculate about the possible motivations and effects of these design decisions, and argue for an urgent need for further research into, and discussions about, the visual design and connected Aesthetic Design Power of social media.

Let’s Go to the Mall: Investigating the Role of User Experience in Customers’ Intention to Use Social Robots in a Shopping Mall

• David Golchinfar
• Daryoush Daniel Vaziri
• Gunnar Stevens
• Dirk Schreiber

Aim of this study is to investigate the effects of user experience (UX) on shopping mall customers’ intention to use a social robot. Therefore, we used a Wizard of Oz approach that enabled data collection in situ. Quantitative data was obtained from a questionnaire completed by shopping mall customers who interacted with a social robot. Data was used in a regression analysis, where user experience factors served as predictors for robot use in retail. The regression model explains up to 23.2% of the variance in customers’ intention to use a social robot. In addition, we collected qualitative data on human-robot-interactions and used the data to complement the interpretation of statistical results. Our findings suggest that only hedonic qualities significantly contribute to the prediction of customers’ intention, that shopping mall customers are reluctant to grant pragmatic qualities to social robots, and that UX evaluation in HRI requires additional predictors.

Reinventing the Wheel: The Future Ripples Method for Activating Anticipatory Capacities in Innovation Teams

• Felix Anand Epp
• Tim Moesgen
• Antti Salovaara
• Emmi Pouta
• İdil Gaziulusoy

Global and systemic sustainability challenges increasingly require innovation teams to incorporate holistic, long-term thinking into their ideation. Since a comprehensive foresight process would prove too burdensome, faster methods are needed. The Future Ripples method was devised to meet this need through reflective practice in four consecutive workshops. It builds on the well-known Futures Wheel foresight method, which offers a collaborative process for brainstorming consequences and impacts. Additionally, the new approach encompasses scanning for weak signals and trends while catering to innovation teams. Analysis of the workshop activities and outcomes suggests that the Future Ripples method can nurture the anticipation skills of innovation teams and help them develop diverse, novel, yet plausible futures. The paper also discusses the role of reflection, metaphors, and the balance between critical and creative thinking in developing holistic futures.

The Hidden Language of Vibrators: A Politico-Ontological Reading

• Dianya Mia Hua
• Rhys Jones
• Jeffrey Bardzell
• Shaowen Bardzell

As digital artifacts for women’s pleasurable experience, vibrators are sophistically designed, embedded with cutting-edge technology, and popular among women. Yet they are still facing social, cultural, and legal obstacles; they are disappointingly understudied by academia as a topic of research. In this pictorial, we use vibrators as our vehicles to speculate future aesthetic, social, and political ways of being related to female sexuality. Our speculation is based on a critical politico-ontological reading of the contemporary design of vibrators by situating it in the historical, cultural, and visual contexts through an art historical analysis. We find that the contemporary design of vibrators carries gender norms and sexual normativity into the futures that they construct. We call on Interaction Design and HCI to tackle prejudice against vibrators to explore the potential of vibrators to improve sexual autonomy, sexual self-determination, and sexual independence of women, especially women with dis/abilities and the elderly.

SESSION: Crafting & Materials

Making space for material entanglements: A diffractive analysis of woodwork and the practice of making an interactive system

• Charlotte Nordmoen
• Andrew P. McPherson

A shift in perspective is underway in design research and human-computer interaction (HCI) from humans as the centre of attention to considering complex assemblages of human and non-human stakeholders. While this shift is often approached from a broad ecological level, there is opportunity for a more local shift in understanding our day to day meeting with the material world. Drawing on the posthuman theories of Karen Barad, we explore the creation of a digital interactive system as a material-discursive practice in which matter and culture are inseparably entangled. We seek a fresh look at the process rather than the outcome of interactive system design through a diffractive reading of four traditional woodworking practices and an auto-ethnographic account of the development of a digital sensor and actuator apparatus as a way to find alternative ways of attending to materials in HCI.

Making crafting visible while rendering labor invisible on the Etsy platform

• Lubna Razaq
• Beth Kolko
• Gary Hsieh

Historically, crafts have been associated with women’s small-scale creative production in the home, equated with hobbies or amateur production, and devalued in comparison to both art and industrial production. During its early years, Etsy was seen as a champion of “handmade”, bringing visibility to crafts and providing economic value. This paper presents results of a qualitative study with 18 small online sellers of Etsy platform. Our study shows that Etsy’s sociotechnical design results in a high burden of invisible labor for sellers, including categories of labor not replicated by other online platforms. These new categories include negotiation and articulation work around defining and defending “handmade” products, understanding one’s intellectual property and how (and whether) to defend that IP elsewhere on the platform, understanding working of platform algorithms and adapting to changing platform regulations. Our findings provide new ways to frame the challenges faced by producers/sellers on emerging marketplace platforms.

SESSION: Design Theory & Critical Design

Designing within Capitalism

• Christine T. Wolf
• Mariam Asad
• Lynn S. Dombrowski

Why do social computing projects aimed at alleviating social inequality fail? This paper investigates this question through a qualitative interview study with 25 individuals working to address the problem of wage theft in the United States (US) context. Our analyses uncover failures at three levels or scales of interaction: one, failures at the individual level of technology adoption; two, relational failures (i.e., the anti-labor worker/employer dynamic in the US); and three, institutional or macro-level failures. Taken together, these various failings point to larger, structural forces that negatively fate pro-labor projects’ trajectories – i.e., capitalism. Capitalism’s incarnations in the US play a significant and at times harsh grip in steering the path of social computing design projects. In this paper, we untangle the relationship between capitalism and social computing, providing an analytic framework to tease apart this complex relationship, the lessons learned from our empirical data, as well as ways forward for future, pro-labor, social computing projects.

“Why Do I Care What’s Similar?” Probing Challenges in AI-Assisted Child Welfare Decision-Making through Worker-AI Interface Design Concepts

• Anna Kawakami
• Venkatesh Sivaraman
• Logan Stapleton
• Hao-Fei Cheng
• Adam Perer
• Zhiwei Steven Wu
• Haiyi Zhu
• Kenneth Holstein

Data-driven AI systems are increasingly used to augment human decision-making in complex, social contexts, such as social work or legal practice. Yet, most existing design knowledge regarding how to best support AI-augmented decision-making comes from studies in comparatively well-defined settings. In this paper, we present findings from design interviews with 12 social workers who use an algorithmic decision support tool (ADS) to assist their day-to-day child maltreatment screening decisions. We generated a range of design concepts, each envisioning different ways of redesigning or augmenting the ADS interface. Overall, workers desired ways to understand the risk score and incorporate contextual knowledge, which move beyond existing notions of AI interpretability. Conversations around our design concepts also surfaced more fundamental concerns around the assumptions underlying statistical prediction, such as inference based on similar historical cases and statistical notions of uncertainty. Based on our findings, we discuss how ADS may be better designed to support the roles of human decision-makers in social decision-making contexts.

Healing Justice: A Framework for Collective Healing and Well-Being from Systemic Traumas

• Brooke Bosley
• Christina N. Harrington
• Susana M. Morris
• Christopher A. Le Dantec

The 2020 Black Lives Matter protests in the United States demanded police reform and legislative action. Data-driven policing is just one technological intervention designed with the hope to tackle police brutality. However, these design interventions are often rooted in the continued racial profiling of poor and socially marginalized communities. Designers and researchers need a Healing Justice framework to circumvent this harm. Healing Justice addresses generational trauma and violence in marginalized communities and is not just a framework for policing but can address maternal mortality rates, COVID-19, medical malpractice, and other trauma issues. In this paper, we apply a Healing Justice framework to co-design activities focused on police brutality. We bridge Healing Justice and design by using an Afrofuturist Feminism framework, arguing that Healing Justice and Afrofuturist feminism frameworks lead to collective, grassroots, and pragmatic designs.

Voices of Sexual Assault Survivors: Understanding Survivors’ Experiences of Interactional Breakdowns and Design Ideas for Solutions

• Hyanghee Park
• Jodi Forlizzi
• Joonhwan Lee

From initial case-reporting at the crime scene to finishing legal procedures, survivors of sexual assault navigate numerous human and technological resources provided by various organizations. During the help-seeking process, survivors unavoidably interact with multiple professional stakeholders (e.g., legal authorities) and technologies (e.g., checking their case status on a court website). In the long and complex process, survivors experience interactional breakdowns with technology, and/or humans, but few studies have explored what types of breakdowns systematically occur and how to resolve them. Thus, for this study, we conducted in-depth interviews with survivors and professionals who reside in South Korea to identify what and how breakdowns occur. Moreover, participatory design sessions were conducted with sexual assault survivors and professionals to create designs that could resolve the breakdowns. Consequently, we discovered a total of eleven breakdowns and produced solutions centered on the stakeholders (i.e., survivor and professionals). Specifically, our participants wanted an integrated system that proactively informs survivors of the holistic procedures for help-seeking and legal action, manages their case, and even interacts with legal authorities on the survivor’s behalf. Based on the findings, we provide an agenda of essential designs and features that could mitigate interactional breakdowns. Additionally, we call for the HCI community to approach and solve sexual violence problems through a broad, macroscopic perspective instead of focusing on one specific technology or (social or organizational) system.

Design considerations for a digital service to support prison leavers

• Monika Grierson
• Delvin Varghese
• Mitzi Bolton
• Patrick Olivier

People leaving prison experience significant challenges upon release. Due to limited funding, only a small proportion of prison leavers receive specialist reintegration support to help them return to the community. Those who receive support have better outcomes across a range of domains and are less likely to be reincarcerated. This study sought to understand how digital services, such as smartphone-based applications, can provide a low-cost, scalable option to extend reintegration support to a broader cohort. To ascertain this, interviews and workshops were conducted with people with lived experience of prison, program staff from a leading community service organisation (CSO) that provides reintegration services, and policy and operational staff from an Australian state’s Department of Justice (DOJ). It was found that digital services have the potential to help prison leavers find and access relevant services, coordinate their own support, and establish supportive peer networks. This paper outlines the critical design considerations that such a service should encompass, including that it must recognise the diverse needs of prison leavers and embed self-determination.

Emergence as a Feature of Practice-based Design Research

• William Gaver
• Peter Gall Krogh
• Andy Boucher
• David Chatting

Practice-based design research is often emergent. Methods, tactics, goals and even topics can unfold and change as researchers adapt and learn in the course of their projects. This adaptability is one of the strengths of design as an approach to research, but it seems to contradict assumptions about research as a systematic, inquiry-led investigation. This leads to a tension in practicing and reporting research that we unpack here, before making a series of suggestions for practice-based researchers and reviewers about how to better navigate emergence when pursuing, reporting and evaluating practice-based design research.

Portfolio of Loose Ends

• Bruna Goveia da Rocha
• Kristina Andersen
• Oscar Tomico

Digital Craftsmanship, and other explorative design research practices using digital fabrication, depend on sample making and material exploration. Rather than describing a method of making use of the information present in samples co-inhabiting a timeline, this pictorial reports on the loose ends that may emerge out of the main journey, when taking a traveler’s approach to making. By loose ends, we mean successful samples that are not suitable to the main inquiry of the present design research process, but that under certain circumstances can become starting points for new investigation lines. We describe four examples of such loose ends and introduce the concept of fellow travelers as a metaphor for describing the process of co-inhabiting a timeline and a journey while ultimately having diverging goals and outcomes.

Co-Designing AI Agents to Support Social Connectedness Among Online Learners: Functionalities, Social Characteristics, and Ethical Challenges

• Qiaosi Wang
• Shan Jing
• Ashok K. Goel

Due to the lack of face-to-face interactions, online learners frequently experience social isolation that negatively impacts students’ well-being and learning experiences. Many text-based AI agents have been equipped with different social characteristics and functionalities to support people who are socially isolated. However, the design of agent’s functionalities, social characteristics, and ethical challenges in promoting social connectedness among online learners are underexplored. Taking a co-design approach, we included 23 online learners enrolled in an online for-degree graduate program as active participants in two virtual co-design workshop studies. Through four different co-design activities, we identified online learners’ preferences for AI agent’s functionalities and social characteristics in promoting their social connectedness as well as potential ethical concerns. Based on our findings, we establish the role of AI agent as a facilitator to continuously scaffold online learners’ social connection process. We further discuss the unique ethical challenges regarding agent-mediated social interaction in online learning.

Multi-lifespan Envisioning Cards:: Journeying from Design Theory to Tools for Action

• Daisy Yoo
• Nick Logler
• Stephanie Ballard
• Batya Friedman

Design theory often stands apart from design practice. Tools offer one means to bridge this divide. Yet how to move from abstract concepts of design theory and principles to specific toolkits that can structure and enable action remains elusive. This pictorial presents a way forward, illustrating the movement from theory to toolkit and concrete action for multi-lifespan design. Our key contributions are three-fold: First, we offer an expansion to an existing design toolkit, the Multi-lifespan Envisioning Cards. We demonstrate the toolkit’s use through case studies. Second, we provide insights on the process of constructing a theoretically-informed design toolkit. We pay special attention to the interplay between text, visuals, and embodied actions. Third, we provide a proof-of-concept for how to extend an existing toolkit to account for new theories and perspectives. We reflect on the timeless qualities of the Envisioning Cards and discuss heuristics for designing a robust yet flexible toolkit.

“It Feels Like Being Locked in A Cage”: Understanding Blind or Low Vision Streamers’ Perceptions of Content Curation Algorithms

• Ethan Z. Rong
• Mo Morgana Zhou
• Zhicong Lu
• Mingming Fan

Blind or low vision (BLV) people were recently reported to be live streamers on the online platforms that employed content curation algorithms. Recent research uncovered perceived algorithmic biases suppressing the content created by marginalized populations (e.g., people of color, the LGBT+ community, and content creators of lower socioeconomic status). However, little is known about how BLV streamers, as a marginalized population, perceive the effects of the algorithms adopted by live streaming platforms. We interviewed BLV streamers (N=19) of Douyin — a popular live stream platform in China — to understand their perceptions of algorithms, perceived challenges, and mitigation strategies. Our findings show the perceived factors contributing to disadvantages under algorithmic evaluation of BLV streamers’ content (e.g., issues with filming and timely interaction with viewers) and perceived algorithmic suppression (e.g., content not amplified to sighted users but suppressed within the BLV community). Their mitigation strategies (e.g., not watching other BLV streamers’ shows) tended to be passive. We discuss design considerations to design a more inclusive and fair live streaming platform.

Form Follows Mental Models: Finding Instantiations of Image Schemas using a Design Research Approach

• Cordula Baur
• Carolin Wienrich
• Jörn Hurtienne

Applying image schema theory to the design of user interfaces has been shown to lead to more intuitive, inclusive, and innovative interaction. We propose that this success can be transferred to the design of data physicalisations (dataphys). But currently designers didn’t use image schemas purposefully because the documentation of image schemas is too bulky to apply in practice. Here we propose an image schema toolkit as a more concise instrument to support dataphys design processes. In this work we report on a research-through-design approach aimed at finding appropriate instantiations of image schemas to include in the toolkit. In an iterative process including analysis, synthesis, and evaluation, we created image schema icons, tokens, and interactive objects. Through the design and evaluation of these artefacts we generated knowledge about image schemas, how to instantiate them in visual and physical ways and how to organize them in a toolkit.

Designing for Culturally Sensitive Cultural Change: A case study of designing for the visibility of Saudi women in the digital media

• Taghreed Alshehri
• Reuben Kirkham
• Lynn Dombrowski
• Patrick Olivier

Online self-presentation is recognized as a global phenomenon largely influenced by and sensitive to users’ cultural norms. However, incorporating cultural understanding into the design process can be challenging. Designing for culture creates dilemmas between designing for a pre-existing cultural ‘status quo’ or for cultural change. We argue that culturally sensitive design should not be a tool for (i) perpetuating existing cultural inequalities or (ii) empowering the individual isolated from their wider cultural milieu. We propose “designing for culturally sensitive cultural change”—a process in which we support creating a trajectory departing from the status quo, to bridge the gap between people’s aspirations and practices related to cultural change. We demonstrate this in a case study on designing for Saudi women’s self-presentation in digital media. We conclude with reflections on cultural sensitivity in designing for cultural change and broader implications for HCI.

The Cost of Culture: An Analysis of Cash App and the Financial Inclusion of Black American Communities

• Jay L. Cunningham
• Sydney T. Nguyen
• Julie A. Kientz
• Daniela Rosner

Mobile banking applications have become a primary mode of sharing and managing financial resources. However, prior work has questioned its role in serving Black communities amid legacies of structural racial inequity. To investigate this critical topic, we conducted a 30-day diary study with 21 participants and an interview study with 15 participants, all Black American users of the mobile payment platform Cash App. We observed that addressing hurdles to financial inclusion requires the navigation of institutional distrust, banking barriers, and financial literacy disparities. Our findings suggest that Black Americans persist and navigate contemporary financial systems through legacies of social and cultural capacities. We offer grounded-truth considerations for the continued design of fin-tech that could have equitable impact among historically marginalized communities. Reflecting on the tenor of financial services to lack empathy, we urge designers and scholars of financial technology to reflect on their power to promote racial justice and financial inclusion.

An Introduction to Weave Structure for HCI: A How-to and Reflection on Modes of Exchange

• Laura Devendorf
• Sasha de Koninck
• Etta Sandry

As HCI continues to integrate craft techniques into its repertoire, tensions emerge between what is new and known, knowledge that resides in communities and histories rather than individuals, and how to transfer between the written word and material know-how. We explore these tensions through the process of writing, instructing, and create an introduction to weaving force sensors. The goal of this project is two fold: first, to help HCI understand the potential that a deep understanding of weave structures can hold for advancing our field, and second, to explore how a pictorial might support formats that have long been used for communicating craft knowledge.

Critically Engaging with Embedded Values through Constrained Technology Design

• Dan Richardson
• Bronwyn J. Cumbo
• Tom Bartindale
• Delvin Varghese
• Manika Saha
• Pratyasha Saha
• Syed Ishtiaque Ahmed
• Gillian C. Oliver
• Patrick Olivier

The rise of gig economy platforms has highlighted the impact platform and algorithm design can have upon workers’ experiences. This paper reports on an extended series of collaborative design engagements with a private company and an international non-government organisation during the production of an Interactive Voice Response component for a gig economy platform. We present the findings of a design ethnography undertaken during this process, and discuss how design decisions reflect how each party’s values, motivations and assumptions are embedded within the final technology. There exists a need for simple methods to assist practitioners to surface and critically engage with the disparate values, priorities and assumptions held by the system’s stakeholders. We demonstrate that this can be done during the production of real-world systems through the application of constrained design as a values lever, and discuss how constraint-based values levers can support critical reflection, even in resource-constrained commercial development contexts.

What Is Meaningful Human-Computer Interaction? Understanding Freedom, Responsibility, and Noos in HCI Based on Viktor Frankl’s Existential Philosophy

• Quynh Nguyen
• Julia Himmelsbach
• Diotima Bertel
• Olivia Zechner
• Manfred Tscheligi

Meaningfulness is a profound aspect of our lives. So far, a concrete reflection of what meaning means in HCI is still rare. To understand users as humans, and thus, the humanity in being a user, we adopt Viktor Frankl’s understanding of users as meaning-seeking subjects. To make the concept of meaningfulness more graspable, we refer to what Frankl calls the three existentials of life: freedom, responsibility, and noos. We elaborate four dimensions, namely the existentials as an outcome, as their embeddedness in technology, their role in interaction, and their (non-)usage. This is followed by a discussion on the interwovenness of the three existentials as well as their potential impact on HCI research. By that, we aim to contribute to a profound understanding of meaning for HCI, from the epistemological to the methodological perspective, to enable meaning-centered design.

Plant Radio: Tuning in to plants by combining posthumanism and design

• Margrete Lodahl Rolighed
• Ester Marie Aagaard
• Marcus Due Jensen
• Raune Frankjaer
• Lone Koefoed Hansen

The Plant Radio is an electronic device that makes it possible to perceive how (and that) a plant responds to its environment. The device works by artificially amplifying the EMG signals of a plant. Aesthetically, it builds on the metaphor of the radio broadcast receiver that allows for tuning in to intangible signals sent through the air. By drawing on mediation theory and making tangible the otherwise hidden signals of plants, the Plant Radio exploratively seeks to reconfigure the relationship between plants and humans by allowing new relations to form. In this pictorial, we present the design of the Plant Radio, and we discuss how the Plant Radio serves as a materialization of the posthuman theories that form the backdrop of this project. In doing so, we expand on the design process of the artifact, specifically unfolding the discussions when deciding not to include a power button.

Feminist Care in the Anthropocene: Packing and Unpacking Tensions in Posthumanist HCI

• Cayla Key
• Cally Gatehouse
• Nick Taylor

As posthumanist or post-anthropocentric research in HCI and design proliferates and further commits to working with more-than-humans, design research practitioners are left with many open questions and uncertainties with how to productively engage with more-than-humans in their thinking and working. This paper present results from a workshop with 17 researchers working at the intersection of care ethics and posthumanism to highlight tensions in posthumanist engagement aimed at unpacking some of the challenges, obstacles, and questions encountered by researchers interested in more-than-human centered design. In foregrounding tensions with representation, legitimization, unseen labor, and material narratives we contribute to a design research agenda which seeks to explicate and challenge dominant anthropocentric forces from design. We conclude by discussing epistemological care and urge practitioners to take up new ways of imagining through truly messy methods which contribute to a feminist unsettling of HCI’s methodological commitments, practices, and praxis.

Growing Up in a Complex World: Engaging Children in Socio-Cultural Matters Through Speculative Installations

• Alma Leora Culén
• Katie Coughlin

The interaction design community has a long history of design and research for and with children, including designing installations for public spaces. This paper explores children’s engagement in socio-cultural issues through speculative installations exhibited in a cultural institution. Over the past ten years, we explored the boundaries between technology, play, learning, and mastery through more than forty interaction design student projects collaborating with various cultural institutions. Although only a portion of projects focused on time-relevant socio-cultural issues such as pollution, refugee crises, or climate change, they opened for reflections on possible ways of including children in dialogues concerning contemporary challenges. The paper contributes a framework for designing and analyzing speculative installations for children based on the ‘darkness scale’ (reflecting the seriousness and complexity of the context), scaffolding engagement, age-appropriate speculative pointers, and linking the present with a desirable future. We showcase two installations and use the framework to discuss them.

SESSION: Game Design

“Isn’t this Marvelous”: Supporting Older Adults’ Wellbeing with Smart Home Devices Through Curiosity, Play and Experimentation

• Yolande Strengers
• Melisa Duque
• Michael Mortimer
• Sarah Pink
• Rex Martin
• Larissa Nicholls
• Ben Horan
• Alicia Eugene
• Sue Thomson

HCI research involving older adults has typically focused on improving technology skills, mobility and health outcomes. Technology for positive ageing emphasizing creativity, inquisitiveness and resourcefulness is less commonly explored. This article builds on this research to contribute an understanding of the importance of curiosity, play and experimentation in supporting positive wellbeing outcomes for older adults living with smart home devices. The research was conducted in regional Australia by an aged care provider and two universities. Twenty-three households participated in the interdisciplinary in-home ethnographic and technological smart device trial that included voice assistants, robotic vacuums, smart kettles and smart lights. The article discusses routines and interactions involving these devices, which resulted in wellbeing outcomes that built participants’ digital living skills. These findings inform concluding recommendations about how to design smart home devices, aged care programs and services involving emerging technologies, to support positive ageing.

How Should I Respond to “Good Morning?”: Understanding Choice in Narrative-Rich Games

• Michael Yin
• Robert Xiao

Narrative-rich video games provide opportunities for players to make choices at key points in the game, generating malleability within the game world and its characters. In this study, we explore the types of choices that exist in such games, how choices affect player experience, and how players make decisions when presented with choice. We first conduct interviews with game developers and perform a video observation analysis of existing choices to develop an initial classification system. We then perform a series of semi-structured interviews with video game players to understand how different choices impact player experience. Our findings reveal that choices influence player experience at several levels of meta-gameplay, having impacts on the game itself, the player-game relationship, and the player outside the game. Furthermore, we identify several key factors that affect player decision-making when faced with choice. Finally, we discuss the potential of choice in developing impactful virtual experiences.

Developing Intentional Relationships with Technologies: An Exploratory Study of Players’ Experiences with Built-in Interventions in Games

• Zicheng Zhu
• Alex Mitchell
• Renwen Zhang

There has been growing concern about digital well-being, especially given the emerging adverse impact of technology overuse. While prior studies have developed a variety of stand-alone techniques to combat technology overuse, little work has been done to build interventions directly into technologies to regulate usage. In this study, we designed three interventions that remind players to take a break in a casual mobile game. We explored players’ experiences with these interventions through a 4-day deployment study and follow-up interviews (N=16). Findings suggest that while some players had a positive experience with the game that had built-in interventions, others experienced unintended outcomes such as disrupted immersion or longer play sessions. We also found that interventions seemed to be more likely to succeed when players experienced a sense of accomplishment or negative emotions. We discuss the implications of the study and provide preliminary suggestions for designing built-in interventions.

Spooky Technology: The ethereal and otherworldly as a resource for design

• Daragh Byrne
• Dan Lockton
• Meijie Hu
• Miranda Luong
• Anuprita Ranade
• Karen Escarcha
• Katherine Giesa
• Yiwei Huang
• Catherine Yochum
• Gordon Robertson
• Lisa (Yip Yan) Yeung
• Matthew Cruz
• Christi Danner
• Elizabeth Wang
• Malika Khurana
• Zhenfang Chen
• Alexander Heyison
• Yixiao Fu

Our everyday technologies could have appeared terrifying to our ancestors: instantaneous disembodied communication, access to knowledge, objects with ‘intelligence’ that talk to us (and each other). Black boxes and intangible entities are omnipresent in our homes and lives without our necessarily understanding the hidden flows of data, unknown agendas, imaginary clouds, and mysterious rules that govern them. Have humanity’s ways of relating to the unknown throughout history gone away, or have they perhaps transmuted into new forms? In an ongoing project, we have inventoried examples, encounters and reflections on contemporary technology, framed through the perspective of the haunted, spectral and otherworldly. In this paper, we excerpt this collection to illustrate the value and opportunity of an unfamiliar, disquieting perspective in helping to frame the frictions, beliefs and myths that are emerging around interactions with everyday technologies. We posit and demonstrate ‘spooky technology’ as an accessible framework to reflect and respond to our increasingly entangled relationships with technology.

SESSION: Immersion

“Are You Still Watching?”: Exploring Unintended User Behaviors and Dark Patterns on Video Streaming Platforms

• Akash Chaudhary
• Jaivrat Saroha
• Kyzyl Monteiro
• Angus G. Forbes
• Aman Parnami

Dark patterns in UI promote addictive behaviors. We explore how the effects of dark patterns in video streaming applications can be exacerbated by a range of temporal and contextual factors. Previous work has shown that excessive watching is potentially detrimental to physical and mental health. We conduct a diary study with 22 viewers over 228 sessions to gain insight into users’ states of mind and to identify users’ emotions while interacting with 4 popular streaming platforms. We analyze users during both the selection phase and the completion phase, finding meaningful correlations between user mood and contextual behaviors that highlight how particular individual characteristics and viewing situations can lead to negative behaviors. We discuss the implications of our findings, highlighting important UI design considerations to enhance digital wellbeing. Furthermore, we collect artifacts of problematic UIs, and present a novel taxonomy of dark patterns found in popular video streaming platforms from a user-centric perspective.

Designing public VR installations:

• Stefan Greuter
• Florian ‘Floyd’ Mueller
• Thuong Hoang

Virtual Reality (VR) installations using head-mounted displays (HMD) are becoming increasingly popular in public spaces. However, VR’s immersive nature engages only the HMD wearer and excludes everyone else in the public space, and there is little design knowledge of how to engage those not wearing an HMD. To address this, we draw from our experiences of having designed seven public VR installations to present a design space around the dimensions of “agency” and “interest” with four user engagement frames to articulate twelve different user roles. To guide designers to support all roles and to transition users between those roles, we complement the design space with a set of design tactics for public VR installations. We hope that these combined contributions will help designers engage more people with VR installations so that ultimately more people benefit from what VR has to offer.

COMMUNICATIVE OPENNESS WITHIN ADOPTIVE FAMILIES: THE ROLE OF PHOTOS AND VISUAL COMMUNICATION

• Thomas Dylan
• Daniel Harrison
• Abigail Durrant

HCI has a long history of designing to support family life, despite this little is known about the unique needs of adoptive families. Though work has explored the role of photos, and photo sharing in the family home, this does not take account of the essential role of photos in supporting the well-being of adopted children. We report on a Research through Design (RtD) project investigating how adoptive parents communicate life story, which is essential to a child’s sense of identity. In this pictorial we show the role of photos and visual communication in enabling core aspects of communicative openness between parent and child, as derived from interviews with adopters about their child’s life story book. We conclude by discussing the need for visual guidelines and tools to support a more nuanced use of photos and illustrations.

Movement Guidance using a Mixed Reality Mirror

• Qiushi Zhou
• Andrew Irlitti
• Difeng Yu
• Jorge Goncalves
• Eduardo Velloso

Mirror reflections offer an intuitive and realistic Mixed Reality (MR) experience comparable to other MR interfaces. Their high visual fidelity, and the sensorimotor contingency from the reflected moving body, make the mirror an ideal instrument for MR movement guidance. The translucent two-way mirror display enables users to follow a virtual humanoid instructor’s movement accurately by visually matching it with their reflections. In this work, we conduct the first formal evaluation of movement acquisition performance with simple motor tasks, using visual guidance from an MR mirror and a humanoid virtual instructor. Our results of performance and subjective ratings indicate that, comparing with simulated virtual mirror and with traditional screen-based movement guidance, the real MR mirror yields better acquisition performance and stronger sense of embodiment with the reflection, for upper-body movement. But the benefits diminish with larger-range head movements. We provide design guidelines for future mirror movement guidance interfaces and MR mirror experiences at large.

The Design Space of Livestreaming Equipment Setups: Tradeoffs, Challenges, and Opportunities

• Ian Drosos
• Philip J. Guo

Livestreaming has grown popular in recent years, with millions of people broadcasting themselves making digital art, playing games, programming, and doing other activities on sites like Twitch and YouTube. While many researchers have studied the actions of both streamers and their viewers, to our knowledge there has been no comprehensive analysis of the actual hardware and software equipment used in livestreaming. In this survey paper we present a holistic overview of modern livestreaming equipment in 2022 by analyzing 40 videos where streamers talk about various aspects of their setups. We categorized their equipment choices into a design space with ten dimensions: computer, software, stream control, encoding, cameras, lighting, video accessories, microphones, audio mixers, and audio accessories. We found that each streamer must make tradeoffs between lower- and higher-fidelity options within each dimension. Our design space analysis can inform ideas for future streaming support tools and, more broadly, tools for remote collaboration and learning via live video. As more of us work and learn online, we are in essence becoming amateur livestreamers, so understanding how professional streamers use their equipment to effectively engage their audiences might help us also engage better with our coworkers and classmates.

MeowPlayLive: Enhancing Animal Live Streaming Experience Through Voice Message-Based Real-Time Viewer-Animal Interaction

• Chee Eun Ahn
• Woohun Lee
• Hunmin Park
• Jiwoo Hong

Animal live streaming (i.e., live streams that feature animals) has recently gained popularity. Nevertheless, little is known about the effect of leveraging viewer-animal interaction on the live streaming experience. In this regard, we introduce MeowPlayLive, a live streaming system that allows viewers to interact with the cat in the live stream by sending voice messages. Since messages appear as moving objects on a tablet screen perceivable by the cat, viewers can win a chance to get heard when the cat decides to tap on the objects. By deploying MeowPlayLive in actual live streams, we found that voice message-based viewer-animal interaction motivates viewers to become active participants, renders the live stream more enjoyable, enhances viewers’ attachment to the cat, and positively transforms the streamer-cat relationship. Our results suggest that viewer-animal interaction should be animal-driven, collaborative, and rewarding. We hope to inspire researchers to explore new forms of technology-mediated human-animal interaction.

Lucid Loop: Exploring the Parallels between Immersive Experiences and Lucid Dreaming

• Alexandra Kitson
• Reese Muntean
• Steve DiPaola
• Bernhard E. Riecke

Lucid dreaming is the awareness of being in a dream, allowing dream control and living out fantasies. It also has benefits for growth and well-being. Yet, lucid dreaming is not accessible to most people. So, we created Lucid Loop—a neurofeedback-augmented immersive experience that utilizes AI-enhanced visuals and spatial audio in a virtual reality device for simulating lucid dreaming. We interviewed nine lucid dreamers who tried Lucid Loop and helped us propose design considerations: dreaming allusions, reality checks, focus points with neurofeedback, people in the scene, and immersion. Lucid Loop was like lucid dreaming because of its capacity for emotionality and fluidity between self and environment. Participants also noted several differences where technology might be limited. Lucid Loop appears to accurately simulate lucid dreaming, with implications for enhancing well-being and future applications for lucid dream training. Our research generalizes to technologically-mediated simulations of other emotive or internal experiences.

Helping Helpers: Supporting Volunteers in Remote Sighted Assistance with Augmented Reality Maps

• Jingyi Xie
• Rui Yu
• Sooyeon Lee
• Yao Lyu
• Syed Masum Billah
• John M. Carroll

Remote sighted assistance (RSA) has emerged as a conversational assistive service, where remote sighted workers, i.e., agents, provide real-time assistance to blind users via video-chat-like communication. Prior work identified several challenges for the agents to provide navigational assistance to users and proposed computer vision-mediated RSA service to address those challenges. We present an interactive system implementing a high-fidelity prototype of RSA service using augmented reality (AR) maps with localization and virtual elements placement capabilities. The paper also presents a confederate-based study design to evaluate the effects of AR maps with 13 untrained agents. The study revealed that, compared to baseline RSA, agents were significantly faster in providing indoor navigational assistance to a confederate playing the role of users, and agents’ mental workload was significantly reduced—all indicate the feasibility and scalability of AR maps in RSA services.

CustomizAR: Facilitating Interactive Exploration and Measurement of Adaptive 3D Designs

• Chen Liang
• Anhong Guo
• Jeeeun Kim

Online 3D model repositories such as Thingiverse offer millions of open source designs that are shared for reuse and remix. Many of the designs are customizable to adapt to real-world objects upon personal needs of varying tasks and physical dimensions. However, it is challenging for novices to discover such designs using text-based search queries, comprehend what each parameter means for customization, locate these parameters on the target objects for measurement, and conduct measurements correctly. These challenges may cause the designs to be incorrectly adjusted, thus failing to function as expected and requiring users to start over, which costs additional time and material. We present CustomizAR, a pipeline for facilitating the interactive exploration of adaptive designs and the measurement of real-world constraints to fabricate them correctly. CustomizAR supports the search and discovery of adaptive 3D designs using an object-centric graph-based data structure, and guides users through an interactive measurement process leveraging computer vision techniques. Our technical evaluations and user studies demonstrate that CustomizAR facilitates effective discovery, adjustment, and reuse of adaptive designs that are shared online.

SESSION: Interaction Techniques

Making Crank-Powered Interactions: Methods, Demonstrators, Materials

• Anders Lundström
• Ylva Fernaeus

We here present a series of design explorations focusing on batteryless interactions powered by hand cranking body movements. The explorations concern not only the matter of powering but also the use of human physical engagement for controlling interactive systems, as well as how to support designers in testing, exploring, and making such interactions. This pictorial illustrates contributions concerned with the embodied and electro-physical prototyping of such experiences, in addition to our efforts to facilitate such design work through new technical tools.

Patterns and Opportunities for the Design of Human-Plant Interaction

• Michelle Chang
• Chenyi Shen
• Aditi Maheshwari
• Andreea Danielescu
• Lining Yao

The emergence of living organisms as entities in HCI presents an opportunity to collaborate with other beings through technology, align interspecies motives, and nurture greater empathy for the non-human. Plants are particularly interesting because of their natural ability to sense and respond to environmental stimuli and potential to enable more sustainable interaction design. However, due to the cross-disciplinary and emerging nature of this space, there is a need to identify overarching patterns and discern opportunities for unifying future research. This paper aims to systematically analyze existing Human-Plant Interaction (HPI) works by presenting a survey of projects across HCI, art/design, architecture, and bioengineering. We identify core design paradigms along the dimensions of HPI System Architecture, Plant I/O Coupling, Plant Interfacing and Manipulation Techniques, Application Context, and Scale. From these themes, we assemble a framework for HCI practitioners to approach HPI, and discuss opportunities and open questions for future exploration.

Learning with Stitch Samplers: Exploring Stitch Samplers as Contextual Instructions for E-textile Tutorials

• Lee Jones
• Audrey Girouard

The field of textile fabrication has a strong pattern-making culture that enables individuals to reproduce items at home. Electronic textile (e-textile) researchers within HCI are increasingly exploring how computing can leverage these textile pattern-making practices, accessible fabrication tools, and do-it-yourself (DIY) maker cultures to enable individuals to make technologies for themselves with soft form factors that further blend computing into our everyday environments. In this paper we focus on the pattern-sharing artifact of stitch samplers, which are used for sharing, teaching, and learning stitching techniques, and explore how the design decisions around them should be adapted for practicing e-textile exercises. To do so, we conducted three studies: (1) preliminary interviews with five modern stitch sampler designers to understand what stitch samplers are used for, (2) a think-aloud user study of our initial e-textile sampler with ten beginners, and (3) interviews with five e-textile educators to reflect on applications and to better understand the opportunities and limitations of using samplers for distance learning. This paper contributes a better understanding of how HCI researchers can incorporate craft pattern practices for learning hybrid craft techniques.

Eliciting Gestures for Novel Note-taking Interactions

• Katy Ilonka Gero
• Lydia Chilton
• Chris Melancon
• Mike Cleron

Handwriting recognition is improving in leaps and bounds, and this opens up new opportunities for stylus-based interactions. In particular, note-taking applications can become a more intelligent user interface, incorporating new features like autocomplete and integrated search. In this work we ran a gesture elicitation study, asking 21 participants to imagine how they would interact with an imaginary, intelligent note-taking application. Participants were prompted to produce gestures for common actions such as select and delete, as well as less common actions (for gesture interaction) such as autocomplete accept/reject, ‘hide’, and search. We report agreement on the elicited gestures, finding that while existing interactions are prevalent (like double taps and long presses) a number of more novel interactions (like dragging selected items to hotspots or using annotations) were also well-represented. We discuss the mental models participants drew on when explaining their gestures and what kind of feedback users might need to move to more stylus-centric interactions.

Material Matters: Exploring Materiality in Digital Musical Instruments Design

• Jianing Zheng
• Nick Bryan-Kinns
• Andrew P. McPherson

Research on the design of Digital Musical Instruments (DMIs) has highlighted the importance of musical gestures and embodied interaction in DMI design. However, this research often focuses on technical and sonic factors of design, with less attention on how materials influence the design process and DMI design idea generation. Thus, this paper explores materiality in DMIs design through a material probe approach with deformable materials. This paper reports on a study with fifteen DMI designers investigating the evoked meaning of material properties in a musical context beyond their digital interactivity. Results suggest that material properties inspired participants’ design thinking, and there was a strong connection between tactility and imagined sound production. We also reported the patterns of gestural interaction of deformable materials in DMIs. We reflect on these results to report lessons learned that could inform interactive systems’ material design within and beyond the musical domain.

SensorViz: Visualizing Sensor Data Across Different Stages of Prototyping Interactive Objects

• Yoonji Kim
• Junyi Zhu
• Mihir Trivedi
• Dishita Turakhia
• Ngai Hang Wu
• Donghyeon Ko
• Michael Wessely
• Stefanie Mueller

In this paper, we propose SensorViz, a visualization tool that supports novice makers during different stages of prototyping with sensors. SensorViz provides three modes of visualization: (1) visualizing datasheet specifications before buying sensors, (2) visualizing sensor interaction with the environment via AR before building the physical prototype, and (3) visualizing live/recorded sensor data to test the assembled prototype. SensorViz includes a library of visualization primitives for different types of sensor data and a sensor database builder, which once a new sensor is added automatically creates a matching visualization by composing visualization primitives. Our user study with 12 makers shows that users are more effective in selecting sensors and configuring sensor layouts using SensorViz compared to traditional prototyping utilizing datasheets and manual testing on the prototype. Our post hoc interviews indicate that SensorViz reduces trial and error by allowing makers to explore sensor positions on the prototype early in the design process.

Sparks: Inspiration for Science Writing using Language Models

• Katy Ilonka Gero
• Vivian Liu
• Lydia Chilton

Large-scale language models are rapidly improving, performing well on a wide variety of tasks with little to no customization. In this work we investigate how language models can support science writing, a challenging writing task that is both open-ended and highly constrained. We present a system for generating “sparks”, sentences related to a scientific concept intended to inspire writers. We find that our sparks are more coherent and diverse than a competitive language model baseline, and approach a human-written gold standard. We run a user study with 13 STEM graduate students writing on topics of their own selection and find three main use cases of sparks—inspiration, translation, and perspective—each of which correlates with a unique interaction pattern. We also find that while participants were more likely to select higher quality sparks, the average quality of sparks seen by a given participant did not correlate with their satisfaction with the tool. We end with a discussion about what impacts human satisfaction with AI support tools, considering participant attitudes towards influence, their openness to technology, as well as issues of plagiarism, trustworthiness, and bias in AI.

Designing Electrolysis Ion Display on Everyday Open Wet Surfaces

• Ayaka Ishii
• Kaori Ikematsu
• Itiro Siio

This pictorial presents a novel method to render color patterns using electrolysis applied onto open wet surfaces. By implementing electrodes within a wet object and electrifying them, electrolysis can occur and generate ions. We designed a color-forming display using color indicators reacting with such ions. This method can display patterns according to the shape of the electrode. By reversing the polarity of the electrodes and generating reversed-polarity ions, it is possible to fade the existing pattern and display a contrasting color. The proposed method does not require moving parts; thus, it is low cost and easy to compact. Also, some foods containing pH-reactive substances can be used as a display medium. Therefore, the proposed method can realize an information display using various objects common in everyday life, such as decor or food.

“My Brain Does Not Function That Way”: Comparing Quilters’ Perceptions and Motivations Towards Computing and Quilting

• Victoria Mirecki
• Juliette Spitaels
• Karen Royer
• Jordan Graves
• Anne Sullivan
• Gillian Smith

The systemic, mathematical, and procedural underpinnings of quilting make the domain a useful metaphor for introductory Computer Science (CS) education, although it is currently used primarily in K-16 educational settings. Considering informal CS education for adult women, we examine the potential depth of this metaphor by exploring how skilled craftspeople engage with and understand quilting-as-metaphor in the context of CS education. In this paper we report the findings of our first focus group with quilters to compare their perceptions and experiences related to quilting and CS. We identified six common themes in how quilters relate the two domains: innate versus learned skills, computing skills as an aid to personal expression, avoiding computing, time investment and tangible rewards, community influence on motivation and learning, and systematic prejudice and its effects. We elaborate upon our findings and discuss potential applications to the design of educational technologies that integrate craft and computation.

The Interview Box: Notes on a Prototype System for Video-Recording Remote Interviews

• David Philip Green
• Joseph Lindley

Video-recording remote interviews can sometimes be necessary or desirable, such as in news broadcasting or documentary-making. However, remote interviews are not currently well-supported by digital tools. Unresolved questions about best practices and the kinds of support needed to facilitate remote interviews have become increasingly relevant since the Covid-19 pandemic. To reflect on these questions and explore the design space for systems to support high quality, remote video-recorded interviews, we conducted an exploratory Research through Design study, drawing on professional media-making techniques, novel interviewing methods and a bespoke intervention: The Interview Box. We provide a detailed summary of our design process and, reflecting on both the successes and failures of our interventions, construct two contributions: technical insights relating to the practical challenges of designing and implementing a remote video interview system, and general insights into the broader interaction design challenges of designing for remote video-recorded interviews.

Knitting Access: Exploring Stateful Textiles with People with Disabilities

• Annika Muehlbradt
• Gregory Whiting
• Shaun Kane
• Laura Devendorf

Wearables are often a primary means of collecting data on the body and in-situ. The data collected upon wearables can shape or record interactions in real time, prompting practices like self-care and reflection. In this work, we became intrigued by textile structures that were non-digital but in themselves “stateful”. We explored how these textile interfaces can fit meaningfully into the lives of people with disabilities as sensors and display. Our study revealed interesting practices that emerged for self-tracking that were qualitatively unique in their close relationship to the body and deeply physical modes of engagement. Our findings offer insights into (1) qualities of textile interfaces that are important to people with disabilities, (2) new forms of data that people found to be worthwhile in tracking, and (3) knitted interfaces for sensing and display.

The Crafts+Fabrication Workshop: Engaging Students with Intangible Cultural Heritage-Oriented Creative Design

• Zhicong Lu
• Peng Tan
• Yi Ji
• Xiaojuan Ma

Engaging local communities is essential to safeguarding intangible cultural heritage (ICH). To better encourage the participation of local communities, especially younger generations, e.g., students, ICH experience and education workshops are widely adopted by academia, museums, governments, and non-profit organizations. The expected outcomes of these workshops, such as archives, documents, and even creative design solutions, can benefit the promotion and preservation of ICH. However, because of the steep learning curve of using traditional ICH tools and the lack of interactions between students and ICH practitioners, many ICH workshops currently fail to engage students in learning ICH-related knowledge, developing empathy with ICH, or designing novel artifacts with ICH elements. To bridge this gap, we designed a workshop, which integrated ICH in China, digital fabrication, creative technology, and making, to engage Chinese students with ICH and creative design. We conducted empirical studies to collect feedback from students (N = 30) and ICH professionals (N=6). The application of digital fabrication tools successfully piqued students’ interest in ICH and enabled them to create interactive ICH artifacts through quick prototyping. However, the ICH professionals pointed out several issues of using digital fabrication, especially regarding tacit knowledge, use of traditional tools, and cultural authenticity. We discuss the importance of these factors in students’ acquisition of ICH knowledge and ICH-oriented design, and provide implications for future ICH design workshops.

The Human-Air Interface: Responding To Poor Air Quality Through Lived Experience and Digital Information

• Meghna Gupta
• Grace Eden

The Air Quality Index (AQI) is a set of numerical, color-coded metrics used to communicate air pollution levels. AQI data is available through a variety of mediums such as mobile apps, websites and public displays. However, the information communicated through these may not be easily interpretable to everyone. Especially when scientific data and associated representations are used to convey information to communities whose shared knowledge and practices are significantly different from Western scientific contexts. We discuss findings from a qualitative study conducted in Delhi, India to understand how residents across both low- and high-resourced communities, assess their personal risk around air pollution and the safety measures they take to mitigate that risk. We reveal incompatibilities between the air-quality information displayed in digital platforms and whether that information is interpretable for people with different cultural sensitivities. We conclude with design implications for improving the interpretability and relevance of air quality interfaces.

Woven eTextiles in HCI — a Literature Review

• Emmi Pouta
• Jussi Ville Mikkonen

Advances to functional materials and flexible electronics have yielded new means of integrating electrical properties into textile materials, which invite researchers in various fields to apply woven-textile construction methods in eTextile development. However, common ground for woven eTextiles’ prototyping is still taking shape. This calls for greater understanding of how the knowledge now scattered across diverse research fields can benefit the textiles’ development for HCI. To investigate how eTextile research has employed weaving and extract insight for HCI purposes, the authors reviewed and categorised applications of woven structures and electrical functions, then identified specifically HCI-relevant qualities and means of creating them via weaving. The paper outlines those woven structures useful for HCI and advocates consistent weaving-related terminology, to improve knowledge transfer across disciplines. In addition, the results point to research opportunities involving haptic qualities, the ability to weave multiple layers, functionality integration, and tools and methods.

Going into Depth: Learning from a Survey of Interactive Designs for Aquatic Recreation

• Christal Clashing
• Ian Smith
• Maria F. Montoya
• Rakesh Patibanda
• Swamy Ananthanarayan
• Sarah Jane Pell
• Florian Floyd Mueller

Aquatic recreation encompasses a variety of water-based activities from which participants gain physical, mental, and social benefits. Although interactive technologies for supporting aquatic recreation activities have increased in recent years, the HCI community does not yet have a structured understanding of approaches to interaction design for aquatic recreation. To contribute towards such an understanding, we present the results of a systematic review of 48 papers on the design of interactive technology for aquatic recreation, drawn from the ACM, IEEE, and SPORTDiscus libraries. This review presents an aquatic recreation user experience framework that details problems and opportunities concerning water and HCI. Our framework brings us closer to understanding how technology can interact with users and the aquatic environment to enhance the existing recreational experiences that connect us to aquatic environments. We found that designers can elicit delight, enablement, challenge, and synergy in aquatic recreation experiences.

Computationally augmenting traditional embroidery practices: an autobiographical design process with first-person patient experience for amblyopia follow up treatment activity

• Yidan Cao
• Karen Anne Cochrane
• Lian Loke

Amblyopia is a common neurodevelopmental condition affecting people’s vision and quality of life. Follow up treatment plays an essential role in improving amblyopia, and experts have proposed embroidery as a potential activity many times. The low compliance of amblyopia patients is one of the impediments. However, there are currently no targeted embroidery activities designed for patients. Designing embroidery activities to meet the needs of amblyopia patients in human-computer interaction and increasing patient compliance has become a design challenge in the current research field. In this research, we present an autobiographical design process to explore the augmentation of traditional embroidery activities with computationally generated patterns based on the stitching preferences of the user. We propose two design considerations for future research: Design with technology to assist traditional handcrafting and personalized design for long-term follow-up treatment through lived experience.

p5.fab: Direct Control of Digital Fabrication Machines from a Creative Coding Environment

• Blair Subbaraman
• Nadya Peek

Machine settings and tuning are critical for digital fabrication outcomes. However, exploring these parameters is non-trivial. We seek to enable exploration of the full design space of digital fabrication. To identify where we might intervene, we studied how practitioners approach 3D printing. We found that beyond using CAD/CAM, they create bespoke routines and workflows to explore interdependent material and machine settings. We seek to provide a system that supports this workflow development. We identified design goals around material exploration, fine-tuned control, and iteration. Based on these, we present p5.fab, a system for controlling digital fabrication machines from the creative coding environment p5.js. We demonstrate p5.fab with examples of 3D prints that cannot be made with traditional 3D printing software. We evaluate p5.fab in workshops and find that it encourages novel printing workflows and artifacts. Finally, we discuss implications for future digital fabrication systems.

TEX(alive): A TOOLKIT TO EXPLORE TEMPORAL EXPRESSIONS IN SHAPE-CHANGING TEXTILE INTERFACES

• Jose Francisco Martinez Castro
• Alice Buso
• Jun Wu
• Elvin Karana

Shape-changing textile interfaces have the potential to create unique functions, expressions, and interactions in everyday artifacts. However, the technical expertise required to fabricate and interact with these interfaces limits designers from rapidly iterating through diverse textile expressions. This pictorial presents TEX(alive), a low-cost and open-source physical-digital toolkit to facilitate the creation of temporal expressions in textile interfaces. TEX(alive) comprises pneumatic actuators that can be interactively configured across a 3d printed grid structure on the textile. Creative sessions with seven designers show that TEX(alive) supports the exploration of temporality in textile interfaces, opening up a design space for unforeseen future application scenarios and alive-like expressions in material-driven design. Finally, we suggest coupling TEX(alive) with a computational simulation tool to allow designers to predict spatial shape change when the textile interface increases in size or complexity.

FLEXI: A Robust and Flexible Social Robot Embodiment Kit

• Patrícia Alves-Oliveira
• Matthew Bavier
• Samrudha Malandkar
• Ryan Eldridge
• Julie Sayigh
• Elin A. Björling
• Maya Cakmak

The social robotics market is appealing yet challenging. Though social robots are built few remain on the market for long. Many reasons account for their short lifespan with costs and context-specificity ranking high amount them. In this work, we designed, fabricated, and developed FLEXI, a social robot embodiment kit that enabled unlimited customization, making it applicable for a broad range of use cases. The hardware and software of FLEXI were entirely developed by this research team from scratch. FLEXI includes a rich set of materials and attachment pieces to allow for a diverse range of hardware customizations that ensure the embodiment is appropriate for specific customer/researcher projects. It also includes an open-source end-user programming interface to lower the barrier of robotics access to interdisciplinary teams that populate the field of Human-Robot Interaction. We present an iterative development of this cost-effective kit through the lenses of case studies, conceptual research, and soft deployment of FLEXI in three application scenarios: community-support, mental health, and education. Additionally, we provide in open-access the full list of materials and a tutorial to fabricate FLEXI, making it accessible to any maker space, research lab, or workshop space interested in working with or learning about social robots.

Anywear Academy: A Larp-based Camp to Inspire Computational Interest in Middle School Girls

• James Fey
• Ella Dagan
• Elena Márquez Segura
• Katherine Isbister

This paper presents a case study of designing and running a Larp (live action role play)-based summer camp in which middle school-age girls create social wearables, toward building computational and design skills, interest, and self-efficacy. Our design draws upon prior evidence that edu-larps can address the identity gap for underrepresented groups in STEM. The focus on creation of social wearables built using E-textiles builds on existing larp practices that use costuming as a method for establishing identity as well as for providing a platform campers can use to enhance their dramatic spectacles. Our findings will be of interest to those working in the areas of informal learning of computation through Arduino and another small device programming, as well as those interested in the intersection of larp and technology design practices, and edu-larp.

From Tool to Companion: Storywriters Want AI Writers to Respect Their Personal Values and Writing Strategies

• Oloff C. Biermann
• Ning F. Ma
• Dongwook Yoon

Modern large-scale language models approach the quality of human-level writing. This promises the advent of AI writing companions performing AI-led writing under human control, surpassing traditional writing tools limited to revision and ideation supports. However, human-AI co-writing may endanger writers’ control, autonomy, and ownership by overstepping co-creative boundaries. Our design workbook study with 7 hobbyists and 13 professional writers elicited three sets of primary barriers to the adoption of human-AI co-writing. Storywriters desire retaining control over writing rather than letting AI take the lead when they (1) prioritize emotional values in turning ideas into words over the productivity of AI-generated writing; (2) have high self-confidence and distrust AI in challenging sub-tasks (e.g., creating characters and dialogue); and (3) expect the AI control mechanism to mismatch their writing strategies. We lay the groundwork for AI companions that respect storywriters’ personal values and writing methods.

Feather Hair: Interacting with Sensorized Hair in Public Settings

• Marie Muehlhaus
• Jürgen Steimle
• Marion Koelle

Human hair opens up new opportunities for embodied interactions that build on its unique physical affordances and location on the body. As hair has high socio-cultural significance, the design of hair interfaces is coupled with social and personal needs. Albeit this makes field investigations indispensable, they are missing from prior work. We present a fabrication approach for gesture-controlled hair interfaces that are robust enough to be deployed in the field. Our approach contributes sensorized feather hair extensions that combine capacitive and piezoresistive sensing. Their tactile properties make the interface blend seamlessly with human hair. We furthermore contribute results from a field experiment where participants gained first-hand experience in various social contexts. These show how hair-based interactions have great potential moving beyond planar touch gestures whilst their social appropriateness is context-sensitive. We synthesize the findings into design implications that ground the future design of usable and socially acceptable hair interfaces.

Addressing Hiccups in Conversations with Recommender Systems

• Sruthi Viswanathan
• Fabien Guillot
• Minsuk Chang
• Antonietta Maria Grasso
• Jean-Michel Renders

Conversational Agents (CAs) employing voice as their main interaction mode produce natural language utterances with the aim of mimicking human conversations. To unveil hiccups in conversations with recommender systems, we observed users interacting with CAs. Our findings suggest that those occur as users struggle to start the session, as CAs do not appear exploratory, and as CAs remained silent after offering recommendation(s) or after reporting errors. Users enacted mental models derived from years of experience with Graphical User Interfaces, but also expected human-like characteristics such as explanations and proactivity. Anchoring on these, we designed a dialogue model for a multimodal Conversational Recommender System (CRS) mimicking humans and GUIs. We probed the state of hiccups further with a Wizard-of-Oz prototype implementing this dialogue model. Our findings suggest that participants rapidly adopted GUI mimicries, cooperated for error resolution, appreciated explainable recommendations, and provided insights to improve persisting hiccups in proactivity and navigation. Based on these, we provide implications for design to address hiccups in CRS.

DramatVis Personae: Visual Text Analytics for Identifying Social Biases in Creative Writing

• Md Naimul Hoque
• Bhavya Ghai
• Niklas Elmqvist

Implicit biases and stereotypes are often pervasive in different forms of creative writing such as novels, screenplays, and children’s books. To understand the kind of biases writers are concerned about and how they mitigate those in their writing, we conducted formative interviews with nine writers. The interviews suggested that despite a writer’s best interest, tracking and managing implicit biases such as a lack of agency, supporting or submissive roles, or harmful language for characters representing marginalized groups is challenging as the story becomes longer and complicated. Based on the interviews, we developed DramatVis Personae (DVP), a visual analytics tool that allows writers to assign social identities to characters, and evaluate how characters and different intersectional social identities are represented in the story. To evaluate DVP, we first conducted think-aloud sessions with three writers and found that DVP is easy-to-use, naturally integrates into the writing process, and could potentially help writers in several critical bias identification tasks. We then conducted a follow-up user study with 11 writers and found that participants could answer questions related to bias detection more efficiently using DVP in comparison to a simple text editor.

Integrating Interactive Technology Concepts With Material Expertise in Textile Design Disciplines

• Mei Zhang
• Rebecca Stewart
• Nick Bryan-Kinns

Textile and fashion designers are increasingly interested in integrating interactive technologies into their practice. However, traditional design education typically lacks support for them to develop technical digital and electronics skills alongside their expertise in materials. Reflecting on outputs from an e-textile design workshop and 8-week design projects with four textile design students using an e-textile toolkit, and follow-up data collection with the students one year after the projects, we argue that starting technical explorations with raw materials results in a better understanding and more flexible use of technical knowledge. We also argue that this newly acquired knowledge is then more fully integrated with their pre-existing material knowledge as it is applied to physical interface design. The results contribute to the development of tools and approaches in supporting designers with material expertise to learn tangible interaction design skills.

SESSION: Multisensory Design

Losing Its Touch: Understanding User Perception of Multimodal Interaction and Smart Assistance

• Margarita Esau
• Veronika Krauß
• Dennis Lawo
• Gunnar Stevens

Intelligent Personal Assistants (IPA) are advertised as reliable companions in the everyday life to simplify household tasks. Due to speech-based usability issues, users struggle to deeply engage with current systems. The capabilities of newer generations of standalone devices are even extended by a display, also to address some weaknesses like memorizing auditive information. So far, it is unclear how the potential of a multimodal experience is realized by designers and appropriated by users. Therefore, we observed 20 participants in a controlled setting, planning a dinner with the help of an audio-visual-based IPA, namely Alexa Echo Show. Our study reveals ambiguous mental models of perceived and experienced device capabilities, leading to confusion. Meanwhile, the additional visual output channel could not counterbalance the weaknesses of voice interaction. Finally, we aim to illustrate users’ conceptual understandings of IPAs and provide implications to rethink audiovisual output for voice-first standalone devices.

”Mirror, Mirror, on the Wall” – Promoting Self-Regulated Learning using Affective States Recognition via Facial Movements

• Si Chen
• Yixin Liu
• Risheng Lu
• Yuqian Zhou
• Yi-Chieh Lee
• Yun Huang

Prior research suggests that affective states of self-regulated learning can be used to improve learners’ cognitive processes and their learning outcomes. However, little research explored the effect of using facial movements to detect learners’ affective states on self-regulated learning. In this work, we designed, implemented, and evaluated Mirror: a self-regulated learning tool that applies facial expression recognition to support learners’ reflections in video-based learning. We conducted two studies to identify user needs (with 12 participants) and to evaluate the tool (with 16 participants). The results show that, after watching a video, participants benefited from using Mirror through different reflection processes, e.g., gaining a deeper understanding of their learning experiences through self-observation and attributing causes for their learning affects through self-judgment. Meanwhile, we also identified several ethical concerns, e.g., users’ agency of handling the uncertainty of AI, reactivity towards outcome-based AI, over-reliance on “positive” AI results, and fairness of AI informed decision-making.

Exploring the Effects of Self-Mockery to Improve Task-Oriented Chatbot’s Social Intelligence

• Chengzhong Liu
• Shixu Zhou
• Yuanhao Zhang
• Dingdong Liu
• Zhenhui Peng
• Xiaojuan Ma

An effective task-oriented chatbot should be able to exert a certain level of Social Intelligence (SI), the ability to emulate human social behaviors to reduce user frustration and dissatisfaction. However, few studies explored using humor, a common rhetorical device in human-human interactions, to improve chatbots’ overall SI. To fill this gap, we proposed to apply self-mockery humor to a customer service chatbot in different interaction stages with users. We proposed a pipeline to create situated self-mockery for the chatbot and conducted a within-subject experiment (N=28) to compare it with a chatbot without self-mockery utterance. Results showed that the self-mockery chatbot was perceived as significantly funnier, more satisfactory, and delivering higher performance in two out of the five measured characteristics of SI with comparable performance in the rest. We further discussed how participants’ individual factors might affect the perceived helpfulness of self-mockery on SI and concluded with design considerations.

EmotiTactor: Exploring How Designers Approach Emotional Robotic Touch

• Ran Zhou
• Harpreet Sareen
• Yufei Zhang
• Daniel Leithinger

In this work, we bring designers into the exploration of emotional robotic touch, discuss their design decisions and reflect on their insights. Prior psychology findings show humans can communicate distinct emotions solely through touch. We hypothesize that similar effects might also be applicable to robotic touch. To enable designers to easily generate and modify various types of affective touch for conveying emotions (e.g., anger, happiness, etc.), we developed a platform consisting of a robotic tactor interface and a software design tool. When conducting an elicitation study with eleven interaction designers, we discovered common patterns in their generated tactile sensations for each emotion. We also illustrate the strategies, metaphors, and reactions that the designers deployed in the design process. Our findings uncover that the “otherness” of robotic touch broadens the design possibilities of emotional communication beyond mimicking interpersonal touch.

Storywork & Reciprocity: On the Design of an Audio Documentary that Extends HCI Research back to Participants

• MinYoung Yoo
• Lauren Knight
• William Odom
• Arne Berger

Participatory design means building reciprocal relationships with research participants and opening up conversations with larger communities. We describe and reflect on our design-led research of creating an hour-long audio documentary. Participants’ desires to better understand the experiences of others within our study is our key motivation. We see the creation of the documentary as an important reciprocal step to invite further participation within a longer-term, multi-year project. Rooted in a decolonial perspective on translating academic knowledge to the general public, our work builds on our own prior fieldwork with 9 people with blindness on their reminiscence experiences. The audio documentary aims to deliver insights from research findings and inquiries, inspired by participants’ stories through their voices in the interview recordings. The documentary serves as a gift for participation and a genuine invitation for future research. We conclude with opportunities for future HCI research and practice.

“Short on time and big on ideas”: Perspectives from Lab Members on DIYBio Work in Community Biolabs

• Orlando de Lange
• Kellie Dunn
• Nadya Peek

DIYbio challenges the status quo by positioning laboratory biology work outside of traditional institutions. HCI has increasingly explored the DIYbio movement, but we lack insight into sites of practice such as community biolabs. Therefore, we gathered data on eleven community biolabs by interviewing sixteen lab managers and members. These labs represent half of identified organizations in scope worldwide. Participants detailed their practices and motivations, outlining the constraints and opportunities of their community biolabs. We found that lab members conducted technically challenging project work with access to high-end equipment and professional expertise. We found that the unique nature of biowork exacerbated challenges for cooperative work, partially due to the particular time sensitivities of work with living organisms. Building on our findings, we discuss how community biolab members are creating new approaches to laboratory biology and how this has design implications for systems that support non-traditional settings for scientific practice.

Sonic Technologies of a Queer Breakup

• Brian Kinnee
• Daniela K. Rosner
• Audrey Desjardins

Over the past decade, the growth of voice assistants has presented new challenges within domestic life. Prior research has shown that such technologies affect users unevenly or fail to account for some relationships and domesticities entirely. Our work investigates the under-examined topic of queerness at home. Drawing on the experience of queer breakup, we describe a design inquiry and a first-person research approach exploring two concurrent relationships in separate households both using Alexa. We explore issues of temporality, glitch, and shared accountability. We also ask critical questions with audio experiments, including: How do voice assistants differentiate between queer voices? How should we converse with voice assistants about queerness? And are voice assistants “queer enough”? We contribute a discussion of difference, inclusion, and queer cultures of adversarial use to highlight the limitations of both everyday and professional language for describing and analyzing the particulars of queerness and interaction design.

Characterising Soundscape Research in Human-Computer Interaction

• Stine S. Johansen
• Niels van Berkel
• Jonas Fritsch

‘Soundscapes’ are an increasingly active topic in Human-Computer Interaction (HCI) and interaction design. From mapping acoustic environments through sound recordings to designing compositions as interventions, soundscapes appear as a recurring theme across a wide body of HCI research. Based on this growing interest, now is the time to explore the types of studies in which soundscapes provide a valuable lens to HCI research. In this paper, we review papers from conferences sponsored or co-sponsored by the ACM Special Interest Group on Computer-Human Interaction in which the term ’soundscape’ occurs. We analyse a total of 235 papers to understand the role of soundscapes as a research focus and identify untapped opportunities for soundscape research within HCI. We identify two common soundscape conceptualisations: (1) Acoustic environments and (2) Compositions, and describe what characterises studies into each concept and the hybrid forms that also occur. On the basis of this, we carve out a foundation for future soundscape research in HCI as a methodological anchor to form a common ground and support this growing research interest. Finally, we offer five recommendations for further research into soundscapes within HCI.

Understanding the Negative Aspects of User Experience in Human-likeness of Voice-based Conversational Agents

• Hyeji Kim
• Inchan Jung
• Youn-kyung Lim

With advances in artificial intelligence technology, Voice-based Conversational Agents (VCAs) can now imitate human abilities, sometimes almost indistinguishably from humans. However, concerns have been raised that too much perceived similarity can trigger threats and fears among users. This raises a question: Should VCAs be able to imitate humans perfectly? To address this, we explored what influences the negative aspects of user experience in human-like VCAs. We conducted a qualitative exploratory study to elicit participants’ perceptions and feelings of human-like VCAs through comparable video prototypes of human–agent conversation and human–human conversation. We discovered that the dialogues of the human-likeness outside of the expressed purpose of a VCA and expressions pretending to come from a human identity could lead to negative experiences with VCAs. Based on our findings, we discussed design directions for overcoming potential issues of human imitation.

Mind the Whisper: Enriching Collocated Social Interactions in Public Places through Audio Narratives

• Hüseyin Uğur Genç
• Duru Erdem
• Çağla Yıldırım
• Aykut Coskun

The quality of social interaction has great importance for psychological and physiological health. Previous research indicates that smartphones have adverse effects on collocated social interactions. Most HCI works addressed this issue by restricting smartphone use during social interactions. Diverging from previous work, we designed WHISPER, an audio narrative box that aims to enrich collocated social interactions without restricting mobile technology use. We conducted a user study in a café environment with 21 participants to understand how users react to WHISPER and how it would influence their social interactions. In this paper, we present the result of this study and discuss four implications for technologies designed to enhance collocated social interactions (Respectfulness, Balanced Ambiguity, Adaptability, and Being Targeted) and two implications for research touching upon the HCI work on Design for Behavior Change and Collocated Interactions (Designing responsible interventions for accommodating unintended outcomes and Quantifying the quality of social interactions).

FlavorDesigner App: Capturing Multisensory Experiences and Crafting Personalized Flavors for Cueing their Recall

• Tom Gayler
• Corina Sas
• Vaiva Kalnikaitė

Food experiences integrate multisensory bodily experiences within social interactions infused with rich emotional meaning. This makes food a promising material to be leveraged in the design of novel interactions. However, this richness also raises challenges for food-based design, as current technologies for capturing food experiences have limitedly accounted for their multisensory qualities. We present the design of FlavorDesigner app, a mobile application aimed to support the capture of multisensory food experiences and the crafting of personalized flavor cues to support their later recall. The app interface was evaluated through workshops with 12 participants. Findings outline richer understandings of capturing multisensory experiences, both live and remembered, vocabulary to inform conversations about them, rationale for our app design, and three implications for design and design research including evocative representations for capturing taste and smell; interactive, engaging and valid sensory evaluation scales; and new classes of technologies for food-based multisensory interactions.

Laila is in a Meeting: Design and Evaluation of a Contextual Auto-Response Messaging Agent

• Pranut Jain
• Rosta Farzan
• Adam J. Lee

The ease of smartphone communications has created an expectation of constant connectivity. While the adoption of virtual assistants has improved, their capabilities for handling proactive communication tasks remain underexplored. We present the design, implementation, and evaluation of a Contextual Auto-Response agent to communicate users’ situational awareness. The agent creates auto-responses by modeling availability using smartphone sensors and sharing contextual information on behalf of the user. In a two-week study with 12 participants, we evaluated the perception of this agent and its impact on device usage behavior. Many participants found the agent useful for signaling unavailability, with some caveats. Participants also reported altering device and agent usage based on their understanding of its functions. Our findings indicate the importance of transparency in proactive agent designs and the need for personalization to enable an enhanced and cooperative human-agent interaction.

Exploring Laughter Sound Visualizations for Self Reflection

• Yangyang Yang
• Kimiko Ryokai

We present an exploratory design study on visualizing laughter sounds for personal reflection. We experimented with a variety of graphic design elements to visualize temporal, spatial, and social aspects of laughter sounds. In order to experience their own laughter being visualized, our participants collected audio recordings of everyday conversations with their loved ones. We extracted laughter from the participants’ audio files using a machine learning algorithm, then visualized selected laughter in five different types of visual representations, and shared the result with each participant. Through the journey of collecting, seeing, listening to, and interacting with their personal laughter visualizations, participants explored what laughter means for them in different contexts. The study reveals that interactive laughter visualizations have the potential to evoke memories, support emotional expressions, and promote relationships.

Voice Snapping: Inclusive Speech Interaction Techniques for Creative Object Manipulation

• Farkhandah Aziz
• Chris Creed
• Sayan Sarcar
• Maite Frutos-Pascual
• Ian Williams

Voice input holds significant potential to support people with physical impairments in producing creative visual design outputs, although it is unclear whether well-established interaction methods used for manipulating graphical assets within mainstream creative applications (typically operated via a mouse, keyboard, or touch input) also present benefits for speech interaction. We present three new voice controlled approaches utilizing interface snapping techniques for manipulating a graphical object’s dimensions: NoSnap, UserSnap, and AutoSnap. A user evaluation with people who have physical impairments (N=25) found that each method enabled participants to successfully control a graphical object’s size across a series of design tasks, although the automated snapping approach utilized within AutoSnap was found to be more efficient, accurate, and usable. Subjective feedback from participants also highlighted a strong preference for AutoSnap over the other techniques in terms of efficiency and ease of use.

Do Monkeys Want Audio or Visual Stimuli? Interactive Computers for Choice with White-Faced Sakis in Zoos

• Ilyena Hirskyj-Douglas
• Vilma Kankaanpää

Interactive systems were built to give monkeys a choice over when and where to trigger stimuli as a way to control their environment to improve their welfare indices. Typically, systems only support the triggering of one stimulus – either audio or visual. In this study, a system was developed for monkeys that allowed them to choose between multiple auditory and visual stimuli. Utilising this system over several weeks, we found that monkeys would interact and choose between different stimuli, though not significantly, and that sakis triggered audio stimuli twice as much as visual stimuli. The monkeys interacted with audio and visual stimuli differently over time, spotlighting how we can define and measure the interactivity and user experience for monkey–computer interfaces. Furthermore, the monkeys’ interactions, while initially increasing, declined over the study period, which indicated a novelty effect. This paper builds upon computer systems for primates by uncovering answers to key questions regarding creating and defining interactive systems according to a user’s choices.

Materialising the Immaterial: Provotyping to Explore Voice Assistant Complexities

• Michael Shorter
• Bettina Minder
• Jon Rogers
• Matthias Baldauf
• Aurelio Todisco
• Sabine Junginger
• Aysun Aytaç
• Patricia Wolf

Voice assistants (VAs), typically promoted as omniscient conversational butlers, still remain below users’ expectations. Interaction designers seem to struggle bringing in user perspectives necessary to develop more meaningful VA applications beyond simple use cases such as playing music. One of the reasons might be the immateriality of cloud-based VA technology making it difficult to comprehend such complex and ever-evolving systems. In this paper, we investigate provotyping as a design tool for ‘materialising the immaterial’. In our case study, teams of multidisciplinary experts devised twelve provotypes to explore intangible VA technology. We present and discuss three generalisations in respect to the role of provotypes for the exploration of VAs. Our findings show provotypes can serve as the necessary props by which we can bring in missing perspectives around this technology and generate material which enables designers to speculate, debate, and sketch out ideas for meaningful futures of VA applications.

Exploring Biofoam as a Material for Tangible Interaction

• Eldy S. Lazaro Vasquez
• Netta Ofer
• Shanel Wu
• Mary Etta West
• Mirela Alistar
• Laura Devendorf

Each new material developed opens a broader pallet of aesthetic and functional possibilities for designers. This paper introduces DIS to biofoam, a material that is water-soluble, biodegradable, and can be made conductive. We describe the material in detail: the process of making the material from scratch, the material’s fabrication into forms with hand-craft techniques, and present two HCI specific applications of the biofoam. The biofoam can be cooked, molded, layered, extruded, dissolved or recooked opening up possibilities to consider the entire life cycle of the material in the design process. We contribute design considerations to allow designers to “tune” the biofoam to the desired quality, as well as a characterization of many aspects of the biofoam such as compression, spring back time, water permeability, and electrical conductivity. Finally, we discuss the unique opportunities this material and its life cycle bring to the design and HCI communities.

ChromoFilament: Designing a Thermochromic Filament for Displaying Malleable States

• Donghyeon Ko
• Yeeun Shin
• Junbeom Shin
• Jiwoo Hong
• Woohun Lee

The 3D pen has become a popular crafting tool where hands-on deformations are largely engaged. However, as malleable states are invisible, users might be burnt, or their fabrication might fail. We designed a thermochromic 3D filament, ChromoFilament, that displays the malleable states in three different colors according to the associated temperatures. From color design workshops, we identified proper stages of malleability and design considerations for color combinations, which are applied to ChromoFilament. Next, we depict a way to fabricate ChromoFilament from customizing thermochromic ink to extruding with the coated pellets. Finally, we illustrate the users’ distinctive behaviors with ChromoFilament to imply the effects of visible malleable states. We believe that our material-perspective approach, design process, and a series of findings could not only inspire supporting creativity through thermoforming but also heat-based processing in 3D printing.

On the Design of deformTable: Attending to Temporality and Materiality for Supporting Everyday Interactions with a Shape-Changing Artifact

• Ce Zhong
• Ron Wakkary
• William Odom
• Amy Yo Sue Chen
• MinYoung Yoo
• Doenja Oogjes

HCI research on shape change has highlighted the importance of materiality and temporality in design practice. However, little research has addressed the challenges of tailoring these qualities for everyday interactions with a shape-changing artifact. In this paper, we report from a designer-researcher perspective into the crafting of deformTable: a shape-changing form that responds to the weight of objects placed on its surface. We crafted 5 deformTables over 18 months with special attention given to the physical, temporal, and computational aspects. Specifically, we describe how we focused on speed adjustments and response times in the process to frame the temporal expressions of computing; we also discuss how the material and tools helped shape our decisions in the context of the materiality of computing. This study contributes an RtD case of designing a particular shape-changing artifact through the lens of temporality and materiality to support long-term engagements and everyday interactions.

SESSION: Space and Interaction

“I was Holding a Magic Box”: Investigating the Effects of Private and Projected Displays in Outdoor Heritage Walks

• Mayank Loonker
• Sophia Ppali
• Rocio von Jungenfeld
• Christos Efstratiou
• Alexandra Covaci

Location-aware mobile guides are a popular technology for enhancing the experience of heritage walks in historical outdoor sites. Smartphones, as typical mediums for such systems, have been criticised for limiting users’ opportunities for embodied engagement with the environment. In this work, we investigate how display technologies beyond the traditional personal mobile screen can facilitate embodied experiences during outdoor heritage walks. To do this, we revisit the use of portable projected displays as a medium that allows us to explore the effects of overt and blended displays in this context. We conducted a study with 42 participants on an outdoor heritage walk, using two display modalities: smartphone and projected display. We discovered that besides the display modality, users’ attitude toward technology, their embodied relationship with the device, and incorporation of spatial aspects in interaction play a key role in generating engagement and shaping the experience of heritage walks.

Wearable Choreographer: Designing Soft-Robotics for Dance Practice

• Catarina Allen d’Ávila Silveira
• Ozgun Kilic Afsar
• Sarah Fdili Alaoui

In this pictorial, we describe an auto-biographical design process that led to the fabrication of a soft robotic wearable for lower limb movement guidance that we designated Wearable Choreographer. We first explored the design from a first-person perspective and then shared it with four dancers. Our experiments illustrate how the wearable both constrains and inspires the dancers towards new ways of performing, challenging them to rethink their movements. Our design inquiry contributes with reflections on soft robotics that uncover the challenges and prospects designers and researchers in Human-Computer Interaction face when designing, prototyping and experimenting with such technologies for embodied interactions.

Networked Gardens: Remediating Local Nature Data Through the Internet of Things

• Kellie Vella
• Michael Esteban
• Bernd Ploderer
• Margot Brereton

Urban life is increasingly disconnected from nature. How might we design for greater awareness of the life around us? We present a study of an Internet of Things (IoT) device, the Ambient Birdhouse, and sensors to learn what kinds of interactions and media supported participants’ attention to local nature, and the sharing and exploration of nature data. Eight participants used the system and took part in individual interviews and group discussions. We found that diverse media and its reproductions and transformations (remediation), presented entry points for different kinds of users to engage with nature data. Notably, ambient acoustic media presented a means of both analytic sense-making and sensitizing users to nature. Patterns of sharing illuminated relational networks, as well as the affordances and challenges of different media. We contemplate how remediation impacts engagement with both nature and nature data and what this means for the design of IoT systems.

Ride With Me: Exploring Group Road Cycling Through Contextual Design

• Juliano Franz
• Derek Reilly

Group sports such as cycling promote a healthy lifestyle and contribute to an active social life among individuals with a common interest. HCI research has mainly focused on supporting individual riders like commuters, young children learning to ride safely, and tourists using bikes. In this work, we present the results of a study exploring the work done by road cyclists while training together, following a Contextual Design methodology involving participants from three different cycling groups across different geographic regions. We share their strategies and challenges with group coordination and communication, and uncover a lack of support for sharing individual metrics such as effort in common tools.

Body-Centric NFC: Body-Centric Interaction with NFC Devices Through Near-Field Enabled Clothing

• Huizhong Ye
• Chi-Jung Lee
• Te-Yen Wu
• Xing-Dong Yang
• Bing-Yu Chen
• Rong-Hao Liang

NFC (Near-Field Communication) has been widely applied for human-computer interaction (HCI). However, the short sensing distance of NFC requires the users to initiate the tasks with extra effort mostly using their hands, so it is inconvenient to use NFC in hands-busy scenarios. This paper presents an investigation of body-centric interactions between the NFC device users and their surroundings. The exploration is based on the recent development of near-field enabled clothing, which can passively extend an NFC-enabled device’s reading distance to users’ body landmarks. We present an accessible method for fabricating flexible, extensible, and scalable NFC extenders on clothing pieces, and an easy-to-use toolkit for facilitating designers to realize the interactive experiences. The method and toolkit were tested in technical experiments and in a co-creation workshop. The elicited design outcomes and the further exploratory makings generated knowledge for future research and embodied interaction design opportunities.

Relating to Soil: Chromatography as a Tool for Environmental Engagement

• Anton Poikolainen Rosén

Due to the ongoing environmental crisis, there is an increased interest in technologies that strengthen relations to the environment. This pictorial contributes to a broader discussion in HCI on how technologies could create a different understanding of and relationship to the more-than-human world. It focuses on soil care practices, and how current (limited) capacities of digital sensing could be complemented with soil chromatography – a qualitative chemical test method for visually assessing soil health. The pictorial is based on a series of workshops conducted in an urban community farm. The discussion focuses on how interactive technologies may support the process of conducting and interpreting soil chromatography and what it means to study, care for, and design with the more-than-human.

Exploration on Everyday Objects as an IoT Control Interface

• Chang-Min Kim
• Tek-Jin Nam

As the user needs for IoT applications have grown, solutions for coordinating IoT devices in everyday living contexts are in high demand. This pictorial presents an explorative study and the design of a new IoT coordination approach that utilizes everyday objects as IoT control interfaces. We conducted a workshop study to explore the potential use cases and understand the opportunities and challenges of everyday objects as means for IoT coordination. Using the use cases from the workshop, we created a scenario movie to illustrate the user experience of the smart environment enabled by everyday objects. We also developed a functional prototype of the IoT coordination system with a wearable device that recognizes the usage of everyday objects. Based on the workshop and the design of two research artefacts, we discuss the implications for advancing the IoT ecology through the use of everyday objects and combining digital and non-digital artifacts.

Ecorbis: A Data Sculpture of Environmental Behavior in the Home Context

• Brigitte Stegers
• Kim Sauvé
• Steven Houben

We are in the middle of a climate crisis. Never before has the impact of climate change been this visible, problematic, and timely. While most people have a basic awareness of the enormous potential impact of climate change, the reality is that only a few people have a detailed understanding of how their own (combined) personal activities – including food, transport, energy – impact the climate. In this pictorial, we explore the design process and principles of a data sculpture – Ecorbis – that is designed to help people reflect on how their day-to-day activities translate to climate impact. Ecorbis provides abstract and numerical weekly feedback on the overall environmental behavior of households and allows for in situ reflection and comparison. We conducted an initial 8-day field study with two families that highlighted that Ecorbis raised environmental awareness on climate impact and that the layered design of Ecorbis facilitated reflection.

Understanding Self-Tracked Data from Bounded Situational Contexts

• Ada Ng
• Ashley Marie Walker
• Laurie Wakschlag
• Nabil Alshurafa
• Madhu Reddy

As smartphone and wearable tracking devices have grown in popularity, more individuals have begun collecting their own health data. While these data are often perceived as a persistent record of health and used to inform future behaviors, it is inevitable that some data are captured during a period of disruption or non-routine circumstances. If not appropriately contextualized, visualizations of these data can lead to missed opportunities in self-reflection, or worse, misinterpretation. To better understand how self-tracked data captured during non-routine circumstances are reflected upon after the disruption has ended, we interviewed women about how they might reflect on data from a recent pregnancy. We propose the concept of bounded situational context (BSC) to encapsulate how individuals define the boundaries of disruption within their data based on external and internal contexts. We discuss how self-tracking tools can be designed to align data visualizations with individuals’ perceived boundaries to aid in data interpretation.

AR Exhibitions for Sensitive Narratives: Designing an Immersive Exhibition for the Museum of Memory in Colombia

• Ana María Cárdenas Gasca
• Jennifer Mary Jacobs
• Andrés Monroy-Hernández
• Michael Nebeling

Augmented Reality (AR) in human rights museums and memorialization efforts can empower these initiatives to create stronger connections between audiences and victims; however, there is little research on the risks of depicting sensitive narratives through immersive technologies. We examined the opportunities and challenges of applying AR to memorialization by designing and deploying an AR application with a human rights museum in Colombia. We report lessons from our collaboration about navigating the risk of re-victimizing testimonial authors while creating engaging AR interactions. Furthermore, we report on a user study where participants interacted with our museum exhibition. Based on observations of our co-design process and the user study results, we discuss implications for immersive application design with strategies for selecting immersive content, balancing audience engagement, and identifying technology gaps. Finally, we reflect on the implications for collaborations between HCI researchers, human rights professionals, and organizations to inform designs involving sensitive narratives.

Design and Field Trial of Lumino in Homes: Supporting Reflective Life by Archiving and Showing Daily Moods with Light Colors

• Dohee Kim
• Sangsu Jang
• Beom Kim
• Young-Woo Park

Although archiving daily moods in a diary is a common behavior, reflection is difficult because of the pressure brought about by continuous self-tracking of personal moods. In this paper, we developed Lumino, a standalone device that enables users to log their daily moods with colored lights and switch mode to show the log by physically sliding a circular lighting plate. The results of our three-week in-field study with six participants revealed that Lumino helped users encounter emotional archives as it separated reflective experiences from daily life activities. Moreover, Lumino helped prevent deep reflection on negative emotions and maintained privacy through abstract color expression. We also found user’s various trials to control emotion reflections in their own usage patterns. We propose considerations for augmenting daily mood recording experiences combined with existing diary practices and further implications for designing how to support positively the negative reflections in everyday spaces.

SESSION: Wellbeing and Health

Virtepex: Virtual Remote Tele-Physical Examination System

• Ninad Khargonkar
• Kevin Desai
• Balakrishnan Prabhakaran
• Thiru Annaswamy

Remote strength assessment is critical for providing accessible rehabilitation, especially in the absence of in-person meetings due to the pandemic. In this paper, we introduce ”Virtepex”, an immersive exergame for remote strength assessment developed through participatory design principles. We bring out the design process starting with a needs assessment to highlight the challenges for physicians in telehealth, followed by the expert guidelines for iterative system refinement. Virtepex addresses the challenges for remote strength assessment through a marker-less and an easy-to-setup strength estimation pipeline. It utilizes an RGB-D camera for motion tracking and an inverse dynamics module for force estimation. The force estimates are used for VR object interaction and can be assessed by a physician synchronously or asynchronously for an objective evaluation. Validation by external experts shows that Virtepex produces reliable force estimates for upper body joints, indicating the potential of marker-less force estimation for future remote assessment designs.

“It’s come around way too quickly!” Can technology help parents provide support during menarche?

• Ishkiran Rai
• Dilisha Patel
• Aneesha Singh

Menarche is an important milestone and time of transition, where children and adolescents need information and support. Parents provide significant support, but barriers such as parents’ own lack of confidence and information interfere. Existing technology for menstrual health is not always appropriate or accessible to younger adolescents and children. We ran two studies: Study1, an interview and design study explored how parents support children for menarche, their use of technology for this, and to understand the gaps. Study2 evaluated a design concept based on Study1, to gain further insights. Our findings show that menarche is an emotional time for parents and children; parents provide support and shared sensemaking but there is space for technology in providing scaffolding for parents to provide further support. However, there is a balance between sharing or support and privacy or control that needs to be negotiated between parents and children. We conclude with some reflections.

SKETCHING SOMA BITS

• Charles Windlin
• Kristina Höök
• Jarmo Laaksolahti

To support sketching in soma design processes, we built the Soma Bits. These simple technology pieces enable one bit actuation like vibration, heat or shape-changing behaviors. After using them for three years, we ask how well they support soma design processes and what designs they spur? Through analyzing nine soma design projects, we note how they contributes to forming subtle, intimate, body awareness designs, as well as fast, uncomfortable, sensory misaligned, somaesthetic designs or skill training. More importantly, experiencing actuation in the early sketching phases helps the design team to articulate joint somatic understanding and direction. During past design sessions, we noticed that the Soma Bits lack support for the overall orchestration: looping, tweaking, or arranging sequences of interactions, and coupling sensing to actuation. Hence, we present a novel Soma Bits toolkit concept for sketching, composing, documenting, and sharing somatic experience.

Together alone, Yōkobo, a sensible presence robject for the home of newly retired couples

• Dominique Deuff
• Isabelle Milleville-Pennel
• Ioana Ocnarescu
• Dora Garcin
• Corentin Aznar
• Siméon Capy
• Shohei Hagane
• Pablo Felipe Osorio Marin
• Enrique Coronado Zuniga
• Liz Rincon Ardila
• Gentiane Venture

Feeling together and at the same time feeling free while sharing the same roof is a balance that newly retired couples try to reach. Indeed the beginning of retirement is complex, and sometimes, even when both spouses find themselves at home together, some spouses could experience a feeling of loneliness. To respond to this insight, we introduce the concept of “sensible presence robject” – Yōkobo to fill this loneliness gap through subtle interactions. The pictorial introduces and describes the different steps of the design process of Yōkobo as a non-anthropomorphic and non-vocal robot for the entrance of dwellings. Through its expressiveness, Yōkobo is a presence messenger for newly retired couples. On a larger scale, this research is a manifesto for the slow technology trend in which perceptions and time open a discussion on poetic sensibility.

Understanding the Therapeutic Coaching Needs of Mothers of Children with Cerebral Palsy

• Ebtisam Alabdulqader
• Katy Stockwell
• Kyle Montague
• Dan Jackson
• Andrew Monk
• Lindsay Pennington
• Roisin McNaney
• Stephen Lindsay
• Ling Wu
• Patrick Olivier

Mothers of preschool children with cerebral palsy are often responsible for delivering multiple home therapy programs. Technology could be a way to bridge some of the challenges of home therapy delivery, such as lack of regular contact with professionals and the need for support continuity. We interviewed seven mothers and four speech therapists to explore their challenges, and the types of support they currently receive (or give). Key issues included limitations of existing communication channels between mothers and professionals, the mothers’ social support needs, and the level of commitment required to self-deliver home therapy. Based on findings indicating video sharing as an existing practice among mothers, we conducted three workshops to further investigate how a video-based platform could support home therapy delivery. We conclude with a number of design considerations for such technologies, to improve communication and collaboration between professional therapists, mothers and members of their wider social network.

A Co-Design Approach to Explore Health Data Representation for Older Adults in Chile and Ecuador

• Gabriela Cajamarca
• Valeria Herskovic
• Andrés Lucero
• Angeles Aldunate

Displaying the information collected by mobile health technologies remains a challenge, especially when considering representation of health data for older adults – i.e., where and how to display data captured by health devices. We focus on an underrepresented group in HCI research: older adults in the global south, specifically in Chile and Ecuador. We studied the opinions of a group of 18 older adults on health data representation through interviews and remote co-design practices, encouraging them to imagine representations of health data through a presentation of personal objects. They imagined representations of health data in analog formats, where contextual information would be included. Visualization designs can be integrated into objects that older adults use frequently or are close to them. This study contributes design ideas on representations of health data for older adults in the global south and reflections on how to engage this population in remote co-design activities.

MindPhone: Mindful Reflection at Unlock Can Reduce Absentminded Smartphone Use

• Nađa Terzimehić
• Luke Haliburton
• Philipp Greiner
• Albrecht Schmidt
• Heinrich Hussmann
• Ville Mäkelä

We present MindPhone, a mindfulness-based intervention to tackle absentminded and excessive smartphone use. At unlock, MindPhone prompts one of two questions: what the user intends to do with the smartphone, or what the user intends to do in the real world after using their smartphone. Users may respond actively by writing, or passively by mentally reflecting. We evaluated the effectiveness of the two questions and two response modes in a mixed-method, 2×2 mixed field study with 28 participants over two weeks. Our results show that the real-world prompt significantly reduces absentminded use and encourages a quicker return to the real world, independent of the response mode. Asking about smartphone use intentions raises awareness of reasons for smartphone use. For everyday use of MindPhone, users wish to set the question and response mode based on context. We discuss including awareness of the physical world in future smartphone use interventions.

Control Matters in Elder Care Technology: Evidence and Direction for Designing It In

• Clara Berridge
• Yuanjin Zhou
• Amanda Lazar
• Anupreet Porwal
• Nora Mattek
• Sarah Gothard
• Jeffrey Kaye

Studies find that older adults want control over how technologies are used in their care, but how it can be operationalized through design remains to be clarified. We present findings from a large survey (n=825) of a well-characterized U.S. online cohort that provides actionable evidence of the importance of designing for control over monitoring technologies. This uniquely large, age-diverse sample allows us to compare needs across age and other characteristics with insights about future users and current older adults (n=496 >64), including those concerned about their own memory loss (n=201). All five control options, which are not currently enabled, were very or extremely important to most people across age. Findings indicate that comfort with a range of care technologies is contingent on having privacy- and other control-enabling options. We discuss opportunities for design to meet these user needs that demand course correction through attentive, creative work.

BGM Diary: Supporting Subjective Experience in Blood Glucose Management Training

• Laurens Boer
• Kasper Heiselberg
• Harvey Bewley

Historically, blood glucose monitoring (BGM) device design has focused on the physiological aspects of diabetes, with less consideration for the psychological aspects of relating the measurement to one’s experienced symptoms. In this pictorial, we explore how the subjective experience of blood glucose monitoring can be supported. We developed the BGM diary, a training system for existing BGM devices. The BGM diary involves a practice of guessing and measuring of blood glucose, followed by an annotation of felt symptoms to facilitate learning about experiences and symptoms. We unpack how blood glucose work could involve persons with diabetes in other ways in their care program through an elaboration on the design aspects of the BGM diary and reflections with a diabetes nurse.

SensorBadge: An Exploratory Study of an Ego-centric Wearable Sensor System for Healthy Office Environments

• Hans Brombacher
• Steven Houben
• Steven Vos

Sensing technology is increasingly used to collect data about the workplace to provide insights into building performance and work activities. While such systems provide meaningful insights, they are a ’black box’ in nature considering people as passive data subjects with no input into the sensing and data collection process. We propose a human-data interaction approach where human workers can opt-in using an ego-centric sensor platform – SensorBadge – that provides tools to collect and inspect personal office data. We describe a field exploration of Sensorbadge to understand the wearability, usability, and usefulness of ego-centric data collection. Our results show that office sensing systems should fit seamlessly in the office routine of individuals, without asking for extra effort or creating conflicts with work patterns. Data should be classified and presented against a frame of reference for comparison and visualized to create understandable and actionable representations with personal control of their office environment.

Fabulating Biodata Futures for Living and Knowing Together

• Vasiliki Tsaknaki
• Pedro Sanches
• Tom Jenkins
• Noura Howell
• Laurens Boer
• Afroditi Bitzouni

A growing number of design researchers explore engagement with and through biodata. To help make sense of this growing space, we synthesize three emergent themes: (1) expanding notions of biodata and bodies, (2) attending to a greater diversity of human bodies and experiences with biodata, and (3) biodata collaborations between human and non-human bodies. We illustrate these themes with selected design examples. From this synthesis, we develop three interconnected fabulations reimagining alternative engagements with biodata: Weaving Alongside, Diffracting Selves, and Collective Affect. Our discussion unpacks conceptual work of the fabulations, offering invitations for design research to explore alternative ways of living and knowing together with biodata.

“So What? What’s That to Do With Me?” Expectations of People With Visual Impairments for Image Descriptions in Their Personal Photo Activities

• Ju Yeon Jung
• Tom Steinberger
• Junbeom Kim
• Mark S. Ackerman

People with visual impairments (PVI) access photos through image descriptions. Thus far, research has studied what PVI expect in these descriptions mostly regarding functional purposes (e.g., identifying an object) and when engaging with online, publicly available images. Extending this research, we interviewed 30 PVI to understand their expectations for image descriptions when viewing, taking, searching, and reminiscing with personal photos on their own devices. We show how their expectations varied across photo activities and often went well beyond identifying objects in photos. Based on our findings, we propose design opportunities for generating and providing image descriptions for personal photo use by PVI. The design opportunities for PVI also point to novel support for the sighted for using image descriptions to enrich their experience of photos. 

Exploring self-tracking practices for those with lived experience of bipolar disorder: Learning from combined principles of Patient and Public Involvement and HCI

• Shazmin Majid
• Richard Morriss
• Grazziela Figueredo
• Stuart Reeves

Bipolar Disorder (BD) is a complex, cyclical and chronic mental illness where self-tracking is central to self-management. Mobile technology is often leveraged to support this. Limited research has investigated the everyday practices of self-tracking for BD, and it is unclear how the normative ontology that is seen in existing self-tracking technology discourses (e.g. the Quantified Self movement) is applicable to the domain of mental health. Combining principles of Patient and Public Involvement (PPI)—a staple research design principle in mental healthcare—with design and HCI-oriented research approaches, we conducted interviews and workshops with people with lived experience of BD to explore reasons and methods for self-tracking, and challenges and opportunities for technology. Our results describe recommendations for the design of self-tracking mental health technology. We also reflect upon the complex role of researchers working at the intersection of emerging mental health technologies, the principles of PPI, and HCI research.

An Annotated Soma Design Process of the Pelvic Chair

• Anna Ståhl
• Madeline Balaam
• Marianela Ciolfi Felice
• Irene Kaklopoulou

The Pelvic Chair uses soft robotics to gently touch the outer pelvic floor to create awareness of its anatomy and function. Our pelvic floor is an important muscle group that holds up our inner organs. A soma design approach makes out a novel route to know through our senses what an experience in this context feels like, which can provide alternative ways of being in the world. Through annotated imagery, we explore the affordances of soft latex pockets filled with air to convey touch that is appropriate for this intimate, sensitive area. Iteratively, we investigate how layers of materials, area, shape, volume and temporality, together with the specific placement on the body, convey the kinds of experiential qualities we seek. We highlight that somatic design judgements are crucial in guiding this process, going beyond traditional approaches to design that focus on visual assessments, models, or hand-based touch.

Designing for Caregiving: Integrating Robotic Assistance in Senior Living Communities

• Laura Stegner
• Bilge Mutlu

Robots hold significant promise to assist with providing care to an aging population and to help overcome increasing caregiver demands. Although a large body of research has explored robotic assistance for individuals with disabilities and age-related challenges, this past work focuses primarily on building robotic capabilities for assistance and has not yet fully considered how these capabilities could be used by professional caregivers. To better understand the workflows and practices of caregivers who support aging populations and to determine how robotic assistance can be integrated into their work, we conducted a field study using ethnographic and co-design methods in a senior living community. From our results, we created a set of design opportunities for robotic assistance, which we organized into three different parts: supporting caregiver workflows, adapting to resident abilities, and providing feedback to all stakeholders of the interaction.