DIS2021 Program information

During the COVID-19 pandemic, communication technologies have allowed people to maintain connections with their loved ones over distance. At the same time, we do not yet have a deep understanding of if and how communication needs amongst family and friends change as a result of physical distancing and travel restrictions and how technologies could be better designed to support these needs. For these reasons, we conducted an exploratory study to investigate the use of communication technologies and family communication needs during the first fourth months of the COVID-19 pandemic in Canada. We used contextual interviews with 18 participants and an open-ended survey with 12 respondents. Our results show that people began the pandemic with a period of shifting and trialing new communication practices; this included increased communications with family and friends. People tried to recreate in-person situations with large group video calls beyond the typical two-household connection found pre-pandemic. This created challenges related to control and participation, and saw people explore ways to increase a sense of shared atmosphere over distance with efforts to increase physicality. Yet large amounts of technology use generally did not persist as participants abandoned
many forms of online interaction over time in a form of technology detachment and sometimes cleanse. These results point to design lessons for times of extreme disconnection between family and friends, such as during a pandemic, where control, participation,
and atmosphere receive deep consideration.

The COVID-19 pandemic changed our lives, forcing us to reconsider our built environment, architectural designs, and even behaviours. Multiple stakeholders, including designers,building facility managers, and policy makers, are making decisions to reduce SARS-CoV-2 virus transmission and make our environment safer; however, systems to effectively and interactively evaluate virus transmission in physical spaces are lacking. To help
fill this gap, we propose OccSim, a system that automatically generates occupancy behaviours in a 3D model of a building and helps users analyze the potential effect of virus transmission from a large-scale and longitudinal perspective. Our participatory evaluation with four groups of stakeholders revealed that OccSim could enhance their decision making processes by identifying specific risks of virus transmission in advance, and illuminating how each risk relates to complex human-building interactions. We reflect on our design and discuss OccSim’s potential implications in the domains of ‘design evaluation,’ ‘generative design,’ and ‘digital twins.’

The landscape of everyday fashion has been transformed due to the COVID-19 pandemic. Part of this can be attributed to different types of communities (e.g. fashion, makers, sewing), who have designed and fabricated masks to counter global shortages and negative culture backlashes. In this paper, we present a mix-methods study of individuals and groups within these communities on their motivations and practices in designing and creating face masks during a global crisis. We conducted a survey with 66 mask makers in the Summer of 2020, and we interviewed 23 of them about their attitudes and reflections on their mask making processes, their unique innovations, and the meaning of contributing in an impactful manner in local and global communities. We unpack themes around technology, self-expression and statement making, making and remixing, sustainable practices, as well as the role of design inspirations on methods and practices for mask makers during a crisis.

COVID-19 has heavily impacted our lives. To date, the ongoing pandemic continues to cause dramatic societal changes and raises shared sentiments of uncertainty for our future. As such, however, COVID-19 provides opportunities to explore futures through speculative research. Here, we gamify the story completion method (SCM) to explore futures post-COVID and ask 37 participants to play a day in the life of Sal in a post-COVID
future. The game asks participants to describe what Sal sees, hears, or does throughout a day based on multiple story stems. Our analysis reveals narratives of post-COVID futures as business as usual, back to basics, or everyday chaos. Notably, these narratives raise concerns about privacy loss and increased militarization, but also envision futures post-COVID that reclaim stronger bond with nature and family. We discuss the lessons learned from gamifying the SCM and the temporal implications of performing speculative research during evolving dramatic events.

With the forced reboot of our lives due to the COVID-19 pandemic, our interpersonal relationships are nowhere yet everywhere. However, opportunities for initiating or maintaining friendships and romance in the physical world have dwindled. Within the context of India where multiple realities exist, the question arises – what is the future of these relationships? In this paper, we present the outcomes of a workshop looking at the future of relationships using design fiction. Participants worked in small teams to create scenarios that critically consider the future of love, friendships, and romance within the Indian context. Through the lenses of criticality, empowerment, and value creation, we examine the design scenarios and the design process including criticality of the designs, empowering experiences of the participants, and the perceived value gained from participating in such a workshop. Our findings indicate the potential of design fiction to allow participants to step out of their comfort zone into a critical stance in discussing love and intimacy. Based upon our findings, we discuss implications for design research, practice, and education.

Engaging first-time users of mobile apps is challenging. Onboarding task flows are designed to minimize the drop out of users. To this point, there is little scientific insight into how to design these task flows. We explore this question with a specific focus on financial applications, which pose a particularly high hurdle and require significant trust. We address this question by combining two approaches. We first conducted semi-structured interviews (n=16) exploring users’ meaning-making when engaging with new mobile applications in general. We then prototyped and evaluated onboarding task flows (n=16) for two mobile cryptocurrency apps using the minimalist instruction framework. Our results suggest that well-designed onboarding processes can improve the perceived usability of first-time users for feature-rich mobile apps. We discuss how the expectations users voiced during the interview study can be met by applying instructional design principles and reason that the minimalist instruction framework for mobile onboarding insights presents itself as a useful design method for practitioners to develop onboarding processes and also to identify when not to.

Mechanical hijacking of physical interfaces is a cost-eff method for providing an Internet of Things (IoT) experience. However, existing mechanical hijacking devices (MHD) have limited applicability and usability. This pictorial introduces a research through design project on IoTIZER, which is easy to use and versatile MHD. We report on a design process from identifying a design space, conducting an iterative design and prototyping. We present the final design’s hardware, software, usage scenario and implementation details. We also share lessons from a user study with eight households. The potential value of IoTIZER were appreciated as a versatile MHD. We discuss improvement areas of IoTIZER and implications for creating an IoT environment while preserving the existing conventions.

Although sharing multimedia content on online social networks (OSNs) has many benefits, publishing photos or videos of other people—without obtaining permission—can cause multiparty privacy conflicts (MPCs). Early studies developed technical solutions and dissuasive approaches to address MPCs. However, none of these studies involved, in the design process, the OSN users who have experienced MPCs. Hence, they possibly overlooked the valuable experiences these individuals have accrued. To fill this gap, we recruited participants specifically from this population of users, and we involved them in participatory design sessions to find solutions to reduce the incidence of MPCs. To frame the activities of our participants, we borrowed terminology and concepts from a well-known framework used in the justice systems. Over the course of several design sessions, our participants designed 10 solutions to mitigate MPCs. The designed solutions are based on different mechanisms, including preventing MPCs from occurring, dissuading users from sharing, resolving the conflicts, and educating users about community standards. We discuss the open design and research opportunities suggested by the designed solutions and contribute an ideal workflow that synthesizes the best of each solution. We contribute to the innovation of privacy-enhancing technologies to limit the incidence of MPCs in OSNs.

Improving end-users’ awareness of cybersecurity warnings (e.g., phishing and malware alerts) remains a longstanding problem in usable security. Prior work suggests two key weaknesses with existing warnings: they are primarily communicated via saturated communication channels (e.g., visual, auditory, and vibrotactile); and, they are communicated rationally, not viscerally. We hypothesized that wrist-based affective haptics should address both of these weaknesses in a form-factor that is practically deployable: i.e., as a replaceable wristband compatible with modern smartwatches like the Apple Watch. To that end, we designed and implemented Spidey Sense, a wristband that produces customizable squeezing sensations to alert users to urgent cybersecurity warnings. To evaluate Spidey Sense, we applied a three-phased ‘Gen-Rank-Verify’ study methodology with 48 participants. We found evidence that, relative to vibrotactile alerts, Spidey Sense was considered more appropriate for the task of alerting people to cybersecurity warnings.

Cryptocurrencies have increasingly gained interest in practice and research alike. Current research in the HCI community predominantly focuses on understanding the behavior of existing cryptocurrency users. Little attention has been given to early users and the challenges they encounter. However, understanding how interfaces of cryptocurrency systems support, impede, or even prevent adoption through new users is essential to develop better, more inclusive solutions. To close this gap, we conducted a user study (n=34) exploring challenges first-time cryptocurrency users face. Our analysis reveals that even popular wallets are not designed for novice users’ needs, stopping them when they would be ready to engage with the technology. We identify multiple challenges ranging from general user interface issues to finance and cryptocurrency-specific ones. We argue that these challenges can and should be addressed by the HCI community and present implications for building better cryptocurrency systems for novice users.

When expressing personal data on the displays of personal IoT devices, it is important to be intuitively aware of privacy settings and perform ready-to-hand interactions to respond appropriately to various situations occurring in shared spaces. In this paper, we developed Posit, an interactive calendar in which the disclosure level of schedule content can be changed in three stages according to the object’s placement by the user. The results of our three-week in-field study with six participants revealed that Posit’s interaction was considered to be a simple way of hiding personal schedules quickly, and we could identify the roles of the positional messages in determining the others’ gazes on displays. Additionally, we confirmed that social relationships and trust between colleagues affect the use of Posit. Our findings imply new opportunities in designing interactions for the management of personal privacy by applying physical state-changing interaction and understanding social factors in shared spaces.

Virtual reality (VR) leverages human sight, hearing and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, information conveyed through sound may not be accessible. To help with future design of accessible VR sound representations for DHH users, this paper contributes a consistent language and structure for representing sounds in VR. Using two studies, we report on the design and evaluation of a novel taxonomy for VR sounds. Study 1 included interviews with 10 VR sound designers to develop our taxonomy along two dimensions: sound source and intent. To evaluate this taxonomy, we conducted another study (Study 2) where eight HCI researchers used our taxonomy to document sounds in 33 VR apps. We found that our taxonomy was able to successfully categorize nearly all sounds (265/267) in these apps. We also uncovered additional insights for designing accessible visual and haptic-based sound substitutes for DHH users.

A key aspect of Virtual Reality (VR) applications is the ability to move in the environment, which relies on the illusion of self-motion to create a good user experience. Self-motion has traditionally been studied in psychophysical studies in which a range of wording has been adopted to describe self-motion. However, it is not clear from current research whether the words used in self-motion studies match study participants’ own intuitions about the experience of self-motion. We argue that the terminology used in self-motion studies should be drawn from a participant perspective to improve validity. We undertook an online study involving VR self-motion and card-sorting with 50 participants to examine current self-motion terminology. We found that participants were not familiar with the concept of self-motion and that the virtual scene itself might suggest different terminology. We suggest how studies on motion perception in VR should be designed to better reflect participants’ vernacular.

Serious applications in virtual reality (VR) often imitate real environments, tasks, and interaction techniques. VR however, allows to transcend reality and augment the physical, cognitive, and perceptual capabilities of users through, for example, telekinesis. In this sense, VR offers “superpowers”, which are not always considered as central and unique elements of interaction design. In a workshop with interaction designers, we emphasized the notion of “superpowers” to think less about VR limitations to imitate reality, but more about its possibilities to augment human capabilities. Consequently we developed a design method that explicitly introduced the concept of superpowers (e.g., X-Ray-Vision, super-memory, telekinesis) and a VR application to support exploring each superpower. We evaluated this approach with designers. They responded positively. The notion of superpowers combined with the possibility to easily experience them changed and widened their perspective: away from limitations and reality, towards possibilities and the unique experience of being in VR.

Go-along interviewing is an emerging qualitative research method where researcher and interviewee go together to a location relevant for the research. Usually employed in ethnographic studies, the method is used to provide a contextualized understanding of a participant’s experience. This paper explores performing Go-along interviews in Immersive Virtual Reality (VR). Through an analysis of ten interviews conducted inside our participants’ Virtual Mind Palaces we show how the interlocutors’ shared presence in the virtual environment established a common ground beneficial for communication. Being in VR enabled our participants to demonstrate interactions spontaneously, and, by providing a guided tour, show us relevant objects and locations in their Virtual Mind Palace. Benefits and challenges of adapting this method to VR are discussed and recommendations for researchers who want to conduct VR Go-along interviews are provided. Finally, we argue the method as an effective tool for eliciting contextual, phenomenological accounts of virtual environments.

Free-hand interaction enables users to directly create artistic augmented reality content using a smartphone, but lacks natural spatial depth information due to the small 2D display’s limited visual feedback. Through an autobiographical design process, three authors explored free-hand drawing over a total of 14 weeks. During this process, they expanded the design space from a single-display smartphone format to a dual-display smartphone-wearable format (Portalware). This new configuration extends the virtual content from a smartphone to a wearable display and enables multi-display free-hand interactions. The authors documented experiences where 1) the display extends the smartphone’s canvas perceptually, allowing the authors to work beyond the smartphone screen view; 2) the additional perspective mitigates the difficulties of depth perception and improves the usability of direct free-hand manipulation; 3) the wearable use cases depend on the nature of the drawing, such as: replicating physical objects, “in-situ” mixed reality pieces, and multi-planar drawings.

Debugging printed circuit boards (PCBs) requires frequent context switching and spatial pattern matching between software design files and physical boards. To reduce this overhead, we conduct a series of interviews with electrical engineers to understand their workflows, around which we design a set of AR interaction techniques, we call Augmented Silkscreen, to streamline identification, localization, annotation, and measurement tasks. We then run a set of remote user studies with illustrative video sketches and simulated PCB tasks to compare our interactions with current practices, finding that our techniques reduce completion times. Based on these quantitative results, as well as qualitative feedback from our participants, we offer design recommendations for the implementation of these interactions on a future, deployable AR system.

Augmented reality (AR) is an efficient form of delivering spatial information and has great potential for training workers. However, AR is still not widely used for such scenarios due to the technical skills and expertise required to create interactive AR content. We developed ProcessAR, an AR-based system to develop 2D/3D content that captures subject matter expert’s (SMEs) environment-object interactions in situ. The design space for ProcessAR was identified from formative interviews with AR programming experts and SMEs, alongside a comparative design study with SMEs and novice users. To enable smooth workflows, ProcessAR locates and identifies different tools/objects through computer vision within the workspace when the author looks at them. We explored additional features such as embedding 2D videos with detected objects and user-adaptive triggers. A final user evaluation comparing ProcessAR and a baseline AR authoring environment showed that, according to our qualitative questionnaire, users preferred ProcessAR.

Most participation processes in urban planning offer poor incentives, especially for young citizens, hence important citizen’s needs are excluded. Our work aims at identifying the degree to which Augmented Reality (AR) might motivate young people. We developed an AR-app with Unity3D to create new interaction concepts for use cases in urban planning. Building projects and environment changes are visualized, so citizens can contribute design ideas to the process. Using a human-centered design approach, we invited different stakeholders to participate. We conducted 40 interviews and a survey, then interaction concepts were evolved by citizens in four participatory design workshops. Our findings show that AR can motivate increased participation in urban planning. We also demonstrate a new approach to engaging low-tech users in designing high-tech solutions such as AR systems by using haptic 3D-tools like Lego or clay. Furthermore, we propose ways in which AR could be used collaboratively and embedded in existing participation processes.

To support scientists in maintaining an overview of disciplinary concepts and their interrelations, we investigate whether Augmented Reality can serve as a platform to make automated methods more accessible and integrated into current literature exploration practices. Building on insights from text and immersive analytics, we identify information and design requirements. We embody these in DatAR, a system design and implementation focussed on analysis of co-occurrences in neuroscientific text collections. We conducted a scenario-based video survey with a sample of neuroscientists and other domain experts, focusing on participants’ willingness to adopt such an AR system in their regular literature review practices. The AR-tailored epistemic and representational designs of our system were generally perceived as suitable for performing complex analytics. We also discuss several fundamental issues with our chosen 3D visualisations, making steps towards understanding in which ways AR is a suitable medium for high-level conceptual literature exploration.

When establishing a visual connection between a virtual reality user and an augmented reality user, it is important to consider whether the augmented reality user faces a surplus of information. Augmented reality, compared to virtual reality, involves two – not one – planes of information: the physical and the virtual. We propose SelectVisAR, a selective visualisation system of virtual environments in augmented reality. Our system enables an augmented reality spectator to perceive a co-located virtual reality user in the context of four distinct visualisation conditions: Interactive, Proximity, Everything, and Dollhouse. We explore an additional two conditions, Context and Spotlight, in a follow-up study. Our design uses a human-centric approach to information filtering, selectively visualising only parts of the virtual environment related to the interactive possibilities of a virtual reality user. The research investigates how selective visualisations can be helpful or trivial for the augmented reality user when observing a virtual reality user.

Virtual Reality (VR) headsets can open opportunities for users to accomplish complex tasks on large virtual displays using compact and portable devices. However, interacting with such large virtual displays using existing interaction techniques might cause fatigue, especially for precise manipulation tasks, due to the lack of physical surfaces. To deal with this issue, we explored the design of VXSlate, an interaction technique that uses a large virtual display as an expansion of a tablet. We combined a user’s head movements as tracked by the VR headset, and touch interaction on the tablet. Using VXSlate, a user head movements positions a virtual representation of the tablet together with the user’s hand, on the large virtual display. This allows the user to perform fine-tuned multi-touch content manipulations. In a user study with seventeen participants, we investigated the effects of VXSlate on users in problem-solving tasks involving content manipulations at different levels of difficulty, such as translation, rotation, scaling, and sketching. As a baseline for comparison, off-the-shelf touch-controller interactions were used. Overall, VXSlate allowed participants to complete the task with completion times and accuracy that are comparable to touch-controller interactions. After an interval of use, VXSlate significantly reduced users’ time to perform scaling tasks in content manipulations, as well as reducing perceived effort. We reflected on the advantages and disadvantages of VXSlate in content manipulation on large virtual displays and explored further applications within the VXSlate design space.

In this paper, we describe Vivian, a technical framework for deploying and operating Virtual Prototypes (VPs) in Virtual and Augmented Reality (XR). Vivian is based on a static model of a VP, e.g., a microwave oven. We extend this model with a functionality specification describing the intended functionality of the VP. Based on this, the Vivian Framework is capable of deploying the VP in XR, i.e., on a desktop PC, in mobile AR, and in VR, as well as providing an interaction with it. Through this, it lays the basis for fast and efficient usability evaluations of VPs, which may also take place in a remote fashion. We evaluated the Vivian Framework by modeling and interacting with four VPs. Our study shows that the principles of Vivian cover many aspects to model VPs and their functionality, but that it also has some limitations, e.g., regarding the modeling complexity.

The increasing ubiquity of interactions as a mix between digital content and physical
objects and spaces, brings about new challenges for designers. There is a need to embed digital systems in physical places, whether those are existing physical structures or existing digital platforms. Traditional approaches to product design, interaction design and user experience design do not often take this new context into account. They do not consider how designers produce new digital and physical experiences that work harmoniously to provide new forms of engagement. To address this, we illustrate the constructs of blended experiences and how they can be used in the context of bridging green spaces between different countries. We propose the idea of blended experiences and offer a framework of constructs and techniques that can help designers work in this emerging area of design.

A major challenge for autonomous vehicles (AVs) today is driving in a complex urban environment where various traffic participants, infrastructures, and events are mixed. A growing body of research is studying the interaction between AVs and human road users (HRUs) to mitigate this challenge. Although traffic is complicated, research has focused on limited situations, such as pedestrian crossings. This study aims to explore scenarios that have a high possibility of causing problems in mixed traffic situations. We devised a design workshop method using miniatures and small cameras, thus allowing participants to experience scenarios from the HRU perspective and easily creating new road situations to uncover various problems. By analyzing 133 scenarios found through the workshop, we defined five factors and 51 elements of AV–HRU problem scenarios. Through qualitative analysis of the factors found through experiments and comparison with existing studies, we identified research gaps and discussed future design challenges.

In the near future, mixed traffic consisting of manual and autonomous vehicles (AVs) will be common. Questions surrounding how vulnerable road users such as pedestrians in wheelchairs (PWs) will make crossing decisions in these new situations are underexplored. We conducted a remote co-design study with one of the researchers of this work who has the lived experience as a powered wheelchair user and applied inclusive design practices. This allowed us to identify and reflect on interface design ideas that can help PWs make safe crossing decisions at intersections. Through an iterative five-week study, we implemented interfaces that can be placed on the vehicle, on the wheelchair, and on the street infrastructure and evaluated them during the co-design sessions using a VR simulator testbed. Informed by our findings, we discuss design insights for implementing inclusive interfaces to improve interactions between autonomous vehicles and vulnerable road users.

Matching the autonomous vehicle’s (AV) driving style to its user’s preference is core to a satisfactory user experience. The recent HCI community has undertaken a significant amount of research to understand user-preferred driving styles in AVs. Due to its multifaceted nature, understanding these driving preferences is difficult unless users take roles in an adaptive system and share their needs explicitly. However, there is a lack of a proper channel for users to express their driving-style needs in AVs. To bridge this gap, we suggest a user’s preferred driving-style guidance using voice as a novel input channel for human-centric AV control. We conducted a Wizard-of-Oz driving study on real roads, aiming to explore the guiding experience with the A agent to reflect their driving-style preferences. This paper presents the value of driving-style guidance along with its burden to users, and concludes with its implications in designing a better AV-guiding experience.

Over the past several years, micromobility devices—small-scale, networked vehicles used to travel short distances—have begun to pervade cities, bringing promises of sustainable transportation and decreased congestion. Though proponents herald their role in offering lightweight solutions to disconnected transit, smart scooters and autonomous delivery robots increasingly occupy pedestrian pathways, reanimating tensions around the right to public space. Drawing on interviews with disabled activists, government officials, and commercial representatives, we chart how devices and policies co-evolve to fulfill municipal sustainability goals, while creating obstacles for people with disabilities whose activism has long resisted inaccessible infrastructure. We reflect on efforts to redistribute space, institute tech governance, and offer accountability to those who involuntarily encounter interventions on the ground. In studying micromobility within spatial and political context, we call for the HCI community to consider how innovation transforms as it moves out from centers of development toward peripheries of design consideration.

This study analysed work activity in a hospital basement where humans and robots interacted and cooperated on logistics tasks. The robots were deployed to automate parts of courier processes and improve the work environment for the hospital’s kitchen staff. Human–robot cooperation was studied through ethnographic fieldwork relating to mobile service robots and hospital kitchen staff. The results highlighted problems arising through the assumption that the ‘plug and play’ service robots could effectively automate work tasks. The analysis revealed the complexity of human–robot interaction in dynamic work settings such as hospitals and identified contradictions between the envisioning and realisation of robots at work, as well as the visible and invisible procedures underpinning human–robot cooperation. Consequently, we emphasise the importance of considering robots as agents of change and draw attention to the new work practices that arise when robots assume the roles of workers in dynamic work settings.

Time management planning (TMP) is a practice where people plan what they intend to accomplish and when in a given day. The literature indicates behaviors associated with TMP, but not how people specifically engage in them or how technology is involved. We examined TMP practices of 19 graduate students, noting their methods and how they engage with tools. Students utilized different combinations of TMP behaviors, both in comparison to each other and within their own experiences. We then asked them to plan following specific guidelines over five days. Participants implemented these guidelines in unique ways using unstructured tools (paper, notes applications). Together, these findings suggest that to be useful, TMP software must not impose a specific structure. We demonstrate opportunities to incorporate these findings through the design of a flexible mobile application based on notes applications to facilitate planning while encouraging, but not requiring, the use of TMP behaviors.

This pictorial critically explores the role of visual media representations in the deployment of automated and artificially intelligent (AI) technologies within essential work sectors. We draw on an exhaustive review of local and national newspaper articles about automation in two waste labor industries (cleaning and recycling) over the last five years. We highlight a set of common visual tropes and move to challenge these representations by taking up the lens of countervisuality. Our analysis reveals that press photographs tend to focus on machines and the decision-makers who champion them, overlooking the work that it takes to integrate technology on the ground. Through our countervisuals, we depict the extensive efforts of waste workers to maintain AI technologies, and their potential for surveillance. Through visualizing under-recognized forms of labor that come after the design process ends, we highlight how an outsized emphasis on invention ignores waste workers’ expertise and needs over time.

DHH (d/Deaf and hard of hearing) employees report unique challenges in their workplaces, especially under hearing supervisors. Although a technical intervention might help address these challenges, any intervention first requires a clearer understanding of DHH employees’ reality. To learn more, we conducted an autoethnographic study and an online survey of 19 DHH employees and three hearing supervisors. Through thematic analysis, we identified DHH Status, Timing and Implications of Disclosure, Accommodations, Disability Confidence, and Audism and Hearing Privileges as central factors that influence DHH employee success under hearing supervisors. By identifying these factors, this work provides the contextual knowledge necessary for proposing appropriate future technical and design interventions.

Public services increasingly presume a new and more active role for individuals to play in datafied society. While design efforts increasingly attempt to include stakeholders, such attempts are often limited to professional perspectives. Little is known about how individuals who are subject to these solutions experience the increasing use of data about them. One example of public services is job placement. Taking design fiction as our approach, we invited individuals enrolled in job placement (n=20) to reflect on the ‘work of the unemployed’, a fictive scenario where individuals make themselves eligible for support through sharing data. The fiction addresses power dynamics. The study shows how approaches, such as design fiction, are effective at including marginalized communities through changing the conditions for design. Showcasing the fictional outlook, and how the ‘design experience’ can be disempowering if not qualified through a deeper critique, the paper contributes to agendas on design justice.

This article contributes results of a longitudinal field study of SkillsIdentifier, an employment tool originally designed and assessed in the United States (U.S.), to support “underrepresented” job seekers in identifying and articulating their employment skills. To understand whether the tool could support the needs of job seekers outside the U.S., we assessed it among 16 job seekers with limited education and language resources in Switzerland. While many of our results mirrored those of the U.S., we found that the tool was especially beneficial for non-French speaking immigrants who needed support describing their skills outside of their native language. We also found that listing skills like “active listening” without important context was insufficient and risked hiding key skills and meaning behind those skills to employers. Taking these factors into account, we illustrate the design implications of our findings and directions for practitioners who wish to design employment tools in support of job seekers, especially those who have traditionally been excluded from the labor market. We then provide insight into the potential for unintended consequences as a result of focusing solely on skills in a post-COVID labor market and contribute ways to mitigate them.

Modern office environments foster sitting, a major public health risk, with physical inactivity being the fourth cause of death worldwide. This provocative pictorial presents the design explorations and bodily experimentations culminating in The Office Jungle, a critical and speculative redesign of the office environment that encourages physical activity by embracing wildness. The Office Jungle is a design exemplar of a “wild” office space presented as a suspended geodesic structure. It is built to experience how our office environment and our behaviour at work affect each other. We advocate that bringing wildness into office spaces will create more durable office environments that foster movement. With this pictorial, we aim to spark discussion amongst designers to think in new ways and to consider new opportunities to design for workplaces that integrate physical activity with work.

Gender is encoded in multiple technological infrastructures, most prominently in digital forms across educational, commercial, medical and governmental contexts. To illustrate the pervasiveness of (binary) gender ideologies and the impact this can have on non-binary individuals – like me – encountering them, I conducted an autoethnography. For more than a year, starting with me receiving a legal non-binary status, I documented systems that did not allow me to register my gender correctly. The findings indicate how technological infrastructures predominantly encode gender as a fixed, immutable and static binary variable with limited options for non-binary people to adequately register self-determined choices for gender and/or (gendered) titles. I further analyse the range of reactions that I received when pro-actively asking for workarounds, fixes and updates, indicating how pointing towards those issues can trouble the status quo, identities and power hierarchies in unintended ways. I close on suggestions for the refactoring of existing and design of new technological infrastructures around gender and reflect on the value of lived experience in knowledge production – as well as the cost it comes with for those doing this research.

Voice assistants (VA) collect data about users’ daily life including interaction with other connected devices, musical preferences, and unintended interactions. While users appreciate the convenience of VAs, their understanding and expectations of data collection by vendors are often vague and incomplete. By making the collected data explorable for consumers, our research-through-design approach seeks to unveil design resources for fostering data literacy and help users in making better informed decisions regarding their use of VAs. In this paper, we present the design of an interactive prototype that visualizes the conversations with VAs on a timeline and provides end users with basic means to engage with data, for instance allowing for filtering and categorization. Based on an evaluation with eleven households, our paper provides insights on how users reflect upon their data trails and presents design guidelines for supporting data literacy of consumers in the context of VAs.

Digital educational technologies have been employed in classrooms to collect students’ behavioral data in the hope of supporting teachers in identifying and correcting undesirable behaviors, which raises the concern of heightened surveillance in classrooms. We present a qualitative study of 20 K-8 teachers to understand their experiences and practices of using ClassDojo, a data-driven classroom behavior management intervention. Our analysis reveals a series of unintended socio-technical effects resulting from the use of ClassDojo in practice. In particular, the use of ClassDojo runs the risk of measuring, codifying, and simplifying the nuanced psycho-social factors that may drive children’s behavior and performance, thereby serving as a “Band-Aid” for deeper issues. We discuss how this process could perpetuate existing inequality and bias in education. With the goals of spurring future design and mitigating these unintended effects, we take on the reflexive-interventionist approach and propose three considerations for designing and using future educational technologies: 1) provide context, 2) expose bias, and 3) challenge and reimagine what is normal.

We present I/O Bits, a prototype personal informatics system that explores the potential for user-driven and situated self-tracking. With simple tactile inputs and small e-paper visualizations, I/O Bits are dedicated physical devices that allow individuals to track and visualize different kinds of personal activities in-situ. This is in contrast to most self-tracking systems, which automate data collection, centralize information displays, or integrate into multi-purpose devices like smartwatches or mobile phones. We report findings from an e-paper visualization workshop and a prototype deployment where participants constructed their own I/O Bits and used them to track a range of personal data. Based on these experiences, we contribute insights and opportunities for situated and user-driven personal informatics.

Internet memes are (multi)media pieces, found all across the world-wide-web. Often disposing of a humorous component, they express and reflect on all kinds of local and global phenomena. Within our work, we explore how people can use internet memes to express and reflect on themselves. We built  MEMEory, a mobile meme journaling app. We evaluated the prospect of meme journaling, nicknamed ”memeing”, alongside a written diary in a 2-week field study with 31 participants. Opposed to more neutral chronicle-style text entries, our results suggest that participants used memes to express specific single, rather negative events and emotions throughout the day. When reflecting on daily events, the contained emotional and often humorous connotation of memes helped participants view negative events as more positive in retrospect. Although more difficult, memeing was perceived as significantly more motivating and enjoyable. Qualitative insights show that memeing can present a fun, engaging, expressive and memorable journaling experience.

This study explores generative dimensions of reflection, shifting the focus of inquiry from tracking data to generating insights by visual practice. Two authors conducted autoethnographic research through visual exercises and analyzed structural elements and outcomes of the practice. Five generative reflection stages were developed by thematic analysis of written reflection notes: 1) notice and focus, 2) translate and frame, 3) make with materials, 4) interpret, and 5) continue and connect. Themes and styles of resulting images were interpreted through compositional analysis. Based on the study findings, we discuss some types of agency found by visual practice in generating visual insights: 1) making the invisible visible, 2) de/re-constructing with visual elements, 3) filling with compositional integrity, and 4) re-styling and re-mixing. We conclude with implications of visual inquiry as a means of noticing and investigating agency in design research and creative practice for reflection.

Findings from embodied cognition suggest that our whole body (not just our eyes) plays an important role in how we make sense of data when we interact with data visualizations. In this paper, we present the results of a study that explores how different designs of the ”interaction” (with a data visualization) alter the way in which people report and discuss correlation and causation in data. We conducted a lab study with two experimental conditions: Full body (participants interacted with a 65” display showing geo-referenced data using gestures and body movements); and, Gamepad (people used a joypad to control the system). Participants tended to agree less with statements that portray correlation and causation in data after using the Gamepad system. Additionally, discourse analysis based on Conceptual Metaphor Theory revealed that users made fewer remarks based on FORCE schemata in Gamepad than in Full-Body.

Science faces a reproducibility crisis. There is a need to establish open science practices within the academic reputation economy. Open Science Badges address this issue by promoting and acknowledging research sharing and documentation. The generic design of currently awarded badges enabled their adoption across the sciences. Yet, their general nature makes it difficult to reflect individual practices and needs of distinct scientific fields. In this paper, we explore uses and effects of highly tailored badges in research data management. We implemented six science badges in a particle physics research preservation service. Our exploration showed that scientists were open to encouraging valuable scientific practices through tailored science badges. They described entirely new opportunities for interaction with research repositories. We present design implications for systems that promote reproducibility, related to meaningful criteria, repository navigation, and content discovery. Finally, we discuss the scope and uses of tailored science badges in modern science.

Journaling of consumed foods through digital devices is a popular self-tracking strategy for weight loss and eating mindfulness. Research has explored modalities, like photos and open-ended text and voice descriptions, to make journaling less burdensome and more descriptive than traditional barcode and database searches. However, less is known about how people prefer to journal foods when less constrained by limitations of databases, natural language processing, and image recognition. We deployed a food journal prototype supporting varied devices and input modalities, which 15 participants used to journal 1008 food logs over two weeks. Participants had diverse strategies for indicating what and how much they ate, varying from ambiguous foods to specifying varieties and using different measurements for clarifying amount. Some strategies were interpretable by natural language food identification and image classification services, while others point to open research questions. We finally discuss opportunities for accounting for variance in food journaling.

The factors influencing people’s food decisions, such as one’s mood and eating environment, are important information to foster self-reflection and to develop personalized healthy diet. But, it is difficult to consistently collect them due to the heavy data capture burden. In this work, we examine how speech input supports capturing everyday food practice through a week-long data collection study (N = 11). We deployed FoodScrap, a speech-based food journaling app that allows people to capture food components, preparation methods, and food decisions. Using speech input, participants detailed their meal ingredients and elaborated their food decisions by describing the eating moments, explaining their eating strategy, and assessing their food practice. Participants recognized that speech input facilitated self-reflection, but expressed concerns around re-recording, mental load, social constraints, and privacy. We discuss how speech input can support low-burden and reflective food journaling and opportunities for effectively processing and presenting large amounts of speech data.

Persuasive features are often used in food journaling apps to help people reach various personal goals. Our understanding of persuasive design in food journaling apps has primarily been built from studying and designing apps in Western contexts. However, varied cultural perceptions around journaling goals, such as healthy eating and weight management, suggest that the design of persuasive features may differ across cultures. We therefore investigate how culture influences the use of persuasive techniques in Chinese food journaling apps and consequently people’s journaling experiences. Through reviewing features of Chinese apps and interviewing people who have used them, we find that some Chinese apps heavily emphasize body shape, and people’s motivations for journaling and desires for journaling apps similarly focus more on body shape than diet. We discuss tensions and opportunities for food journaling apps which align with or account for cultural norms while challenging unhealthy or problematic aspects of them.

There has been recent criticism from researchers towards simple replication of traditional role models in the design of virtual agents and robots, and a call for new forms of interaction and communication with technology. By exploring the field of Human-Food interaction (HFI) – a sub-area of Human-Computer Interaction (HCI) which aims to investigate the diversity of ways people interact with food – we therefore specifically examine the design space of edible anthropomorphic virtual agents (EAVAs). To understand human-to-food interactive communication, we conducted an interview study with 19 participants, followed by a co-design workshop on the design of conversational agents for personified food. Based on the results, we implemented a prototype called FoodChattAR that employs augmented reality and chatbots to interact and communicate with food. Our evaluation with 21 participants shows that FoodChattAR turns eating into fun, while at the same time the food conveys relevant societal facts about itself. We contribute to the field of HCI by introducing EAVAs as a novel human-to-food interaction.

Much work has been done on the design of both interactive narratives and food-based interaction systems, however their intersection has received little attention within the design research community. We took the relative dearth of food-based interactive storytelling systems as an opportunity to explore a compelling design space for multisensory interactive narratives, given these few systems often position people only as passive observers. We report the results of a design workshop conducted among the authors who are design researchers in interactive narratives and food-based play. In the workshop, we discussed existing works, asynchronously brainstormed new food-based interactive storytelling systems, and then reconvened to discuss our ideas. We present the portfolio of designs that arose from this workshop, and the design themes that we synthesized from the portfolio. We recommend how to position people not just as observers, but as active participants in the interactive food-based storytelling system.

Designing interactions with food holds potential for rich multisensory experiences but their pervasiveness can challenge our understanding of them. This paper presents the design and evaluation of Sensory probes, a novel, exploratory design research method aimed to sensitize participants towards their food experiences. We report on workshops with 8 participants for co-designing the probes, followed by iterative revision through two-week diary studies with 18 participants. Findings indicate strong engagement with the sensory probes and how they brought forward the bodily and sensory aspects of these experiences, alongside emotional and social ones. We highlight the design rationale for the sensory probes which has been both empirically- and theoretically-grounded, provide reflections on the value of these probes for enabling novel perspectives on food experiences, and on probes’ ability to capture what we called sensory fragments of participants’ experience reflecting distinct sensory aspects form both internal and external senses.

Data have played an extensive role in sustainable HCI research by informing the impacts of our behavior on the environment and helping us make better environmental choices. However, in the area of sustainable food consumption and sustainable HCI, there is little investigation on the roles of food data for the design of technology. This paper presents findings from a qualitative study of sustainable-conscious individuals’ food data seeking experiences. Our results show the way in which the current food data is challenging our understanding of its environmental impacts, which concern data of availability, data representations, and data cultures. Drawing from Loukissas’ six critical data principles, we discuss how “locality” and ”place” could cast a new insight on food and its sustainability. We also offer possible design directions for sustainable HCI technologies utilizing food data.

In this pictorial, we explore the potential of social media to help inspire ideas for future drone design applications that support playful and social experiences. Drawing from a Situated Play Design approach [3], we turn to social media posts to identify recurring playful and social instances of drone use in social settings. We present the results of collecting 143 posts found on Instagram, TikTok, and YouTube, from which we identified a non-exhaustive.list.of.drone-based.play.potentials, i.e. existing ways in which people already appropriate drones to playfully augment social situations. We present these play potentials as potentially.valuable and inherently situated intermediate-level knowledge with generative power. We argue they might inspire the design of future.drone.technologies.and.experiences in Human-Drone Interaction (HDI), in directions that increasingly respond to people’s desires for play and social connection.

As artificial intelligence and smart devices increasingly infiltrate everyday life, cooperative interactions between humans and computers are correspondingly becoming more common. Errors are an inevitability in these interactions and can destabilise long-term working relationships. In this work, three online games of increasing difficulty (N=2037) were designed where participants played a cooperative game with an artificial AI player and encountered an unexpected error. Different training methods were used to establish a rapport between the human and AI players. Overall, despite answering correctly, participants were increasingly more likely to say they had made a mistake and that they were to blame as the difficulty of the game increased. Since participants were also unaware of the extent of their exposure to AI, this study shows that there is a tendency to apply preconceived expectations of AI and misattribute error origination which, if not addressed, could lead to critical breakdowns of trust.

In this paper, we map out the CoCe design space – a design space for co-located collaborative games that use multi-display composition. The design space grew out of the analysis of game instances based on the 4in1 concept. First, we did a horizontal analysis of 16 game instances with 31 corresponding gameplay design patterns (GDP), followed by a vertical analysis of 89 GDPs occurring in the description of the GDP Cooperation. Through inductive analysis, we have identified four perspectives with corresponding dimensions that span the CoCe design space. By applying the CoCe design space with game instances, we illustrate how it can be used both as an analytic tool for analysis of games and also as a generative tool in the design or re-design of cooperative games that use multi-display composition.

Bullying is a problem concerning us all, especially our children. Human Computer Interaction (HCI) has invited children to tackle this problem through design and technology. However, yet there is limited research including critical engagement with the topic, even if the critical agenda is arousing increasing interest in HCI. Design fiction has shown potential in addressing critical concerns. However, albeit design fiction has entered HCI method repertoire already some time ago, it is an ambiguous concept with multiple meanings and usages. We map and articulate some of this variety: fascinating, future-oriented, and provocative studies are identified, while also a lack of critical design fiction, aiming at empowerment of the power-weak. We contribute by exploring such kind of design fiction in collaboration with children to tackle the problem of bullying. We scrutinize our participatory and empowering design fiction process and outcomes and discuss their implications for HCI research and design fiction practice.

Our work centers on aspects of crafting creativity that are often overlooked in digital fabrication: playful- ness, and possibilities for social engagement. We draw from precedents in both crafting (e.g. quilting bees) and gameplay (e.g. “Exquisite Corpse”) to inform the design of a set of turn-based collaborative games which center a computer-controlled embroidery machine as a “player” in games for one or more crafters. We proto- type these games using our own computational input/ output embroidery pipeline and observe how they can guide crafter-players to engage with physical, digital, and social affordances. We summarize our findings on how creative focus can shift over a playful experience of fabrication and how technology can mediate social crafting.

Online services pervasively employ manipulative designs (i.e., dark patterns) to influence users to purchase goods and subscriptions, spend more time on-site, or mindlessly accept the harvesting of their personal data. To protect users from the lure of such designs, we asked: are users aware of the presence of dark patterns? If so, are they able to resist them? By surveying 406 individuals, we found that they are generally aware of the influence that manipulative designs can exert on their online behaviour. However, being aware does not equip users with the ability to oppose such influence. We further find that respondents, especially younger ones, often recognise the ”darkness” of certain designs, but remain unsure of the actual harm they may suffer. Finally, we discuss a set of interventions (e.g., bright patterns, design frictions, training games, applications to expedite legal enforcement) in the light of our findings.

Smartphone apps such as Robinhood and Public that promise to “democratize investing” have risen in popularity over the past few years. These apps allow retail investors, who often possess little prior investing experience, to trade stocks, options, and other securities easily and inexpensively, often commission-free. It seems plausible that the interaction patterns of these new apps may significantly influence trading behaviors of their users. But so far, there is little formal design guidance on how such apps should be designed. This paper introduces a set of design guidelines for encouraging healthy investing behaviors by drawing on three bodies of related work:
1) findings from finance and economics literature on healthy investment practices,
2) the dual process theory from behavioral sciences, and 3) design metaphors used in interfaces with uncertain rewards. Using these guidelines, we qualitatively analyze the user interfaces of some popular investment platforms. Our analysis reveal that, unfortunately, popular trading apps generally do not follow design patterns that encourage healthier trading behaviors. We discuss design implications and opportunities for future design.

Tensions in designing for care are often positioned as conflicts to be resolved. We draw upon queer theories to investigate caring for loved ones as not ”in-line” with normative expectations of care as positive and fulfilling. Through the critique of two autobiographical design projects designed for informal, everyday care of our families, we describe four troubling orientations of care: willful detours, selfish shortcuts, naughty invasions, and unhappy departures. From these, we argue that tensions in care may not always be designed against, but can also be desired and generative. We conclude by discussing a ”wickedness” in caring for loved ones that problematizes in-home technologies as attractively naughty and potentially violent, and the four orientations as resources for interaction designers to spatially navigate tensions of domestic care.

The banking industry has been integrating digital technologies globally. However, accepting new technologies is challenging in particular for older adults. We focus on older adults’ banking experiences in China, where digital transactions have been growing rapidly, to provide a perspective on how they adapt to this trend. We conducted an online survey with 155 older adults who are 60 or above (M = 70, SD = 9) from 18 provinces to explore their banking practices and challenges. Our results show that older adults conduct banking transactions frequently. However, few do so using digital platforms despite long wait times in physical banks. The main concerns reported by them are about security and usability. Nonetheless, they hold a positive attitude towards digital platforms (e.g., apps, virtual banks). Interestingly, age and gender have significant effects on particular banking behaviors. We discuss our findings in the context of prior studies and highlight design opportunities for improving banking accessibility for older adults.

Older adults can experience significant changes to their social networks as they age, triggering changes in their social connection practices. In this paper, we extend research on older adults’ connectedness behaviors using a multimodal connectedness framing—that is, how they engage with others across platforms, devices, and modalities. Using the COVID-19 pandemic as a case study, we investigate how older adults navigate a major change or infrastructural breakdown in their social routines. We conducted a survey with 146 U.S.-based older adults (65+), and follow-up interviews with a subset of 23 survey respondents. Findings revealed the resilience and innovation with which older adults adapted their behaviors across multiple modalities to maintain social relationships and playfully connect with others in person and online. Using these findings, we propose that research on designing for aging extend beyond designing for connection in the smart home; we argue for a design agenda that prioritizes designing for smart relationships with the potential to persist across spaces via multimodal connectedness.

We describe three design works from a research through design project seeking to develop new ways to curate and create digital media to support ongoingness – a form of continuing bonds in bereavement that prioritises the present and an ongoing, positive relationship with the deceased. We use the three pieces in this paper as lenses to view how dialogic potential is at play within each piece – drawing on aspects of dialogism, a Bakhtinian theory, to do so. Our aim is that by sharing the work and blending the particular theoretical perspective of dialogism with experiential and design aspects of the pieces we offer up a rich design space and new ways to conceive of design and digital technologies in the context of bereavement and death.

Increased access to online engagement platforms has created a shift in civic practice, enabling civic leaders to broaden their outreach to collect a larger number of community input, such as comments and ideas. However, sensemaking of such input remains a challenge due to the unstructured nature of text comments and ambiguity of human language. Hence, community input is often left unanalyzed and unutilized in policymaking. To address this problem, we interviewed 14 civic leaders to understand their practices and requirements. We identified challenges around organizing the unstructured community input and surfacing community’s reflections beyond binary sentiments. Based on these insights, we built CommunityPulse, an interactive system that combines text analysis and visualization to scaffold different facets of community input. Our evaluation with another 15 experts suggests CommunityPulse’s efficacy in surfacing multiple facets such as reflections, priorities, and hidden insights while reducing the required time, effort, and expertise for community input analysis.

Vital sign values during medical emergencies can help clinicians recognize and treat patients with life-threatening injuries. Identifying abnormal vital signs, however, is frequently delayed and the values may not be documented at all. In this mixed-methods study, we designed and evaluated a two-phased visual alert approach for a digital checklist in trauma resuscitation that informs users about undocumented vital signs. Using an interrupted time series analysis, we compared documentation in the periods before (two years) and after (four months) the introduction of the alerts. We found that introducing alerts led to an increase in documentation throughout the post-intervention period, with clinicians documenting vital signs earlier. Interviews with users and video review of cases showed that alerts were ineffective when clinicians engaged less with the checklist or set the checklist down to perform another activity. From these findings, we discuss approaches to designing alerts for dynamic team-based settings.

Supervising anesthesiologists oversee junior anesthesiologists in several operating rooms to instruct and provide advice, and hence ensure high patient safety. Supervisors benefit from good situation awareness but accessing information about multiple patients can be challenging. In a user-centered design process, we have developed an interface concept for a head-worn display (HWD) to support multi-patient monitoring. The design includes data about the patients’ physiology, vital status, vital trends, and medications. We conducted an exploratory evaluation of a prototype in a high-fidelity medical simulation with six supervising anesthesiologists. Our results show that the supervisors’ situation awareness of all simulated operations improved because participants noticed and responded to multiple events over the course of the scenario. However, the supervisors reported increased mental workload due to the constant availability of patient data. We discuss insights about the HWD display, the design process, and the experience and views of staff on using the HWD.

The growing HCI interest in wellbeing has led to the emerging area of haptics for affect regulation. In such technologies, distinct haptic patterns are usually designed by researchers; however, current work provides a limited reflection on the rationale for the implemented patterns or the choice of haptic modality. We also know little about how people may benefit from engagement in designing such patterns and what design principles underpin them. We explored vibrotactile and thermal modalities to address these gaps and report on a study with 23 participants. These created haptic patterns for affect regulation during stress elicitation. Findings indicate that subjective and objective measures of anxiety and stress were lower in participants who received haptic patterns than those who did not, and highlighted key experiential qualities of vibrotactile and thermal patterns, and their potential for affect regulation. These open up new design opportunities for affect regulation technologies, including supporting implicit affect regulation through entrainment of slow bodily rhythms, decoupling it from predominant vibrotactile modality, designing thermal biofeedback patterns, and supporting personalized and adaptive patterns.

Personal health informatics are increasingly used in the long-term management of bipolar disorder and other serious mental illnesses. These systems help individuals and members of their support networks track and stabilize important social rhythms. In this study, we presented mental health professionals with nine design concepts for personal informatics systems to support interpersonal and social rhythm therapy. These designs, derived from prior empirical findings about patients’ data practices, utilize social interfaces to support the social ecologies critical to relational recovery in bipolar disorder. Designs included features for sharing data in dynamic social support systems, custom variables, commenting, co-tracking, and data-driven action plans. Qualitative analysis of clinicians’ feedback yielded design recommendations for mental health informatics, such as supporting all cognitive states of the multiplicative patient and the interactions between these states and patient–clinician interfaces. Diversity in clinical practice also necessitates application flexibility and careful integration with existing workflows.

Writing about experiences of trauma and other challenges in life is known to provide measurable health benefits. Though writing for an audience may ensure better benefits, confiding one’s most troubled memories in others risks a social stigma. Conversational agents can provide a virtual audience that ensures privacy and allows social disclosure. To understand the writing experience with an agent, we created Diarybot, a chatbot assistant for expressive writing. We designed two versions, Basic and Responsive, to explore the writing experience with and without bot follow-up interactions compared to a Google doc baseline. Findings from a 4-day user study with 30 participants reveal that social disclosure with Diarybot can encourage narrative writing, with relative ease and emotional expression in Basic chat. Responsive chat can mediate social acceptance of the bot and provide guidance for self-reflection in the process. We discuss design reflections on social disclosure with agents in pursuit of wellbeing.

Gaining an understanding of people’s diverse mental health needs is essential for informing the design of inclusive mental health technologies. However, conversations about mental health experiences can be challenging for both researchers and participants. We present the design of visual cards that illustrate an inclusive mental health concept to support researchers and participants in understanding and sharing mental health experiences. We illustrate the iterative design of the visual cards with our reflections and feedback from ethnically diverse participants. We found that designing the visual cards fostered insightful reflections within the design team regarding the roles of identity, gender, and ethnicity in designing culturally sensitive content and research. Participants from minority ethnic backgrounds valued the illustrative elements of the visual cards and highlighted the importance of supporting different languages and visual cultures. We discuss use cases for the visual cards and implications for designing culturally sensitive mental health technologies.

Promoting healthy lifestyles is an essential endeavor for public health. The design of active urban environments can be an effective medium to nudge people into moving. With technology increasingly integrated into our daily lives, designers have access to more data than ever. In this pictorial, we explore the design space of interActive environments (contraction of ‘interactive’ and ‘active’); places designed to increase the physical activity of users or passers-by through the use of interactive technology. Through sketches, a benchmark of existing concepts and an analysis of designed artefacts, we map the different intervention levels, interaction modalities, behavior change strategies and technological opportunities to design such interActive environments. With this work, we invite the community to consider how digital technology can help understand and shape human behavior in urban environments, and provide inspiration to designers and practitioners.

This paper presents a systematic review of online health communities (OHCs) published between 2009 and 2020 in the ACM Digital Library. Aiming to consolidate the current issues, design knowledge, challenges, and tensions in OHCs, our analysis identified four high-level aspects related to the use and design of OHCs: (1) temporal: OHCs as transition spaces, (2) spatial: bridging experiential knowledge with medical expertise, (3) technological: exchanging and locating peer support, and (4) tension dimensions in OHCs. We further discuss methodological improvements and computing opportunities for OHC research and how to increase OHC members’ agency in such a medically dominated context. These findings have the potential to inform future OHC designs and help researchers and designers position future contributions.

Counsellors use Motivational Interviewing (MI) to boost intrinsic motivation for healthier lifestyles by supporting 1) the identification of personal values and 2) the linking of these values to personal reasons for change. Research is needed to increase the reach of such interventions through the use of online tools. Thus, we explored the use of defamiliarization and poetic gameplay in a serious game, to make habitual health choices difficult and ‘unfamiliar’ in order to promote players’ contemplation about their personal reasons for health. Our evaluation indicates that the players demonstrated evidence of processes of change such as self-reevaluation, environmental reevaluation, and improving decisional clarity. We observed that these methods could promote dialogic reflection with different points of view, but was less effective at promoting transformative reflection that would lead to change. We conclude with a discussion of promising future directions for research.

Online health communities provide a space where people seek out and provide support for weight loss activities, including tracking. Our study examined the experiences of members of an online community (r/loseit on Reddit.com) who posted about using digital tracking tools for weight loss. A targeted search garnered 379 public posts, which were analyzed using Thematic Analysis. Four themes reflected members’ individual and shared experiences: Tracking as gaining insight, Tracking as a vehicle of control, Confronting challenges in sustaining tracking and Teaching and learning the skills of tracking. We highlight complex socio-technical processes that members developed around tracking tools and discuss how knowledge of these appropriations can be applied to designing future user-centered tracking tools to support weight loss. We discuss how the social context of an online health community can shape both the usage of tracking tools and self-regulatory processes for health behaviour change.

The increased popularity of recreational sports, like running, led to the development of numerous technologies supporting people in their training. However, in their current form and interaction, these take a rather standardized approach focusing on quantified data tracking displayed through screens or audio. In this paper, we explore how dynamic data physicalization through a shape-changing interface can open the design space of exercise feedback. Relying on an expert study on the aesthetics of interaction (N=23), we designed Laina, a shape-changing art piece presenting physicalized running data through a slow feedback mechanism. We deployed Laina at 3 participant’s home, during a series of 3-weeks field studies. Results show that Laina allows for deep reflection, anticipation and exploration of running behavior. The aim of our paper is to provide insights on the use of slow feedback mechanisms for exercise-related products, through the design of a dynamic data physicalization artefact.

Recovering from serious illness involves a bodily and psychosocial reorientation in everyday life. Survivors of gynecological cancer often experience bodily changes, fear of cancer recurrence, and changes in sexual health. This paper explores how we can use audiovisual narratives based on experiences of gynecological cancer survival in the design of an online intervention. From a typology of cancer survival, we designed three audiovisual narratives in an experience-centered design process involving gynecological cancer survivors. The narratives were evaluated by 10 participants formerly treated for gynecological cancer. In a thematic analysis, we explore how these narratives set the stage for identification and reflection by being experienced as relatable, provoking, and realistic. Finally, we discuss how the survivors’ experience of the narratives can be construed as meaningful, and how accounts of experiences can be included in a design process to create narrative content for online interventions.

Computer-supported posture guidance is used in sports, dance training, expression of art with movements, and learning gestures for interaction. At present, the influence of display types and visualizations have not been investigated in the literature. These factors are important as they directly impact perception and cognitive load, and hence influence the performance of participants. In this paper, we conducted a controlled experiment with 20 participants to compare the use of five display types with different screen sizes: smartphones, tablets, desktop monitors, TVs, and large displays. On each device, we compared three common visualizations for posture guidance: skeletons, silhouettes, and 3d body models. To conduct our assessment, we developed a mobile and cross-platform system that only requires a single camera. Our results show that compared to a smartphone display, larger displays show a lower error (12%). Regarding the choice of visualization, participants rated 3D body models as significantly more usable in comparison to a skeleton visualization.

Despite an intention to exercise, it remains a challenge for many people to establish a workout routine over a period of time. Amongst identified barriers and enablers to exercise, getting dressed for a workout is considered as one of the tipping points of actually going. Implementing the Aesthetic of Friction in this specific context, could imply the right course of action for the user, while it also allows freedom and encourages meaning-making. In this Research-through-Design project, we designed an interactive shrinking hanger, that implements these key principles, to encourage exercise motivation. We followed an iterative process focusing on the aesthetics of the interaction to find out how a careful consideration of the look and feel of an interactive artefact influences the acceptance of the implemented friction. We document the design process of this aesthetics of friction exemplar, and reflect on how to implement friction in design.

As wearable devices move toward the face (i.e. smart earbuds, glasses), there is an increasing need to facilitate intuitive interactions with these devices. Current sensing techniques can already detect many mouth-based gestures; however, users’ preferences of these gestures are not fully understood. In this paper, we investigate the design space and usability of mouth-based microgestures. We first conducted brainstorming sessions (N=16) and compiled an extensive set of 86 user-defined gestures. Then, with an online survey (N=50), we assessed the physical and mental demand of our gesture set and identified a subset of 14 gestures that can be performed easily and naturally. Finally, we conducted a remote Wizard-of-Oz usability study (N=11) mapping gestures to various daily smartphone operations under a sitting and walking context. From these studies, we develop a taxonomy for mouth gestures, finalize a practical gesture set for common applications, and provide design guidelines for future mouth-based gesture interactions.

Devices for remote communication and sending digital gestures have been widely explored in the field of ubiquitous computing. With the lack of social contact during the COVID-19 pandemic and renewed interest in these devices in mind, we suggest the affordances of a culturally-informed design approach. We present SneezeLove, a device that uses sneezes to communicate that someone is missing you. Users tickle the device’s nose to send an electromechanical ‘sneeze’ to another device. The concept plays on a superstition in Chinese culture that attributes an additional layer of meaning to sneezes, and we found similar beliefs in other cultures that signify that someone out there is thinking of you: hiccups in India, “burning” ears in Ireland. The device, and its design approach, suggests how low-resolution messages can be enlivened and made more personally meaningful by tapping into expressive and specific cultural superstitions.

Past research recognized that paper has many advantages over digital devices, such as affordability, tangibility, and flexibility. Paper, however, also lacks many of the functionalities available in digital technologies, such as access to online resources and the ability to display interactive content. Prior research therefore identified opportunities for fusing the two mediums into a combined interface. This work presents a literature review on this form of innovation – technologies that bridge the paper-digital gap. First, we synthesize an understanding of paper and its relationship with digital devices through the lens of past works. Then, we outline the state-of-the-art for paper-digital interfaces and highlight possible use cases and implementation approaches. Last, we discuss design considerations and future work for developing paper-digital interfaces. Our work may be beneficial for HCI researchers interested in the development of hybrid paper-digital interfaces, and more broadly in embedding digital functionalities in everyday objects.

Compression-based haptic feedback has been used in wearables to issue notifications, provide therapeutic effects, and create immersive storytelling environments. Such worn devices are well studied on the wrists, arms, and head, however, many unconventional yet context-rich areas of the body remain underexplored. Current haptic prototyping techniques have large instrumentation costs, requiring the design of bespoke embedded devices that do not have the flexibility to be applied to other body sites. In this work, we introduce an open-source prototyping toolkit for designing, fabricating, and programming wearable compression-based interfaces, or compressables. Our approach uses a lost-PVA technique for making custom inflatable silicone bladders, an off-the-shelf pneumatics controller, and a mobile app to explore and tune haptic interactions through sketch gestures. We validate the toolkit’s open-endedness through a user study and heuristic evaluation. We use exemplar artifacts to annotate the design space of compressables and discuss opportunities to further expand haptic expression on the body.

This pictorial presents Circuit Assemblies, a design system for beginners to create 3D-printed interactive objects with embedded electronics. Circuit Assemblies are modules used to create objects that light up, move, or spin using basic electronic components like LEDs, batteries, and motors. To support beginners incorporating Circuit Assemblies into 3D designs, a set of virtual components were added to Tinkercad, a popular browser-based 3D CAD application with over 10 million users. In this paper, we begin with a set of design considerations gathered from interviews with three K-12 educators that teach electronics. We then present four different Circuit Assembly modules designed with these considerations in mind, highlighting the unique challenges that arise from combining electronics and 3D design for beginners, both in CAD software and physical assembly. We then present four different Circuit Assembly modules designed with these considerations in mind, highlighting the unique challenges that arise from combining electronics and 3D design for beginners, both in CAD software and physical assembly.

This pictorial presents a design exploration of On-Skin Interfaces for recreational running. By integrating principles of interaction design, art and psychology, we explore the design of unconventional interfaces that facilitate the intuitive understanding of biofeedback and physiological-related information. We explored how principles from agency and bodily ownership can be applied in the design of sport- related wearables. Through our embodied prototyping journey, we gained insights on the implications of using the skin as an interactive design material. We focused on diverse materiality explorations to uncover and highlight the possibilities and challenges of materializing both functional and appealing On-Skin Interfaces. We synthesize and refl on our theoretical and practical explorations and deliver actionable insights for this growing fi of bodily and unconventional interfaces.

On-skin interfaces demonstrate great potential given their direct skin contact; however, conducting field studies of these devices outside of laboratories and in real settings remains a challenge. We conduct a research-through-design investigation using an extended woven practice for fabricating fully-integrated and untethered multi-sensor on-skin systems that are resilient, versatile, and capable of field deployment. We designed, implemented, and deployed a woven on-skin index-finger and thumb-based inertial measurement unit (IMU) sensing system for multi-hour use as a technology probe to understand the social, technical, and design facets towards moving integrated on-skin systems into a wearer’s daily life. Further, we integrate a woven NFC coil into the IMU on-skin system, which is wirelessly powered by a smartwatch substitute, signifying the potential of our woven approach for developing wirelessly powered on-skin systems for longer-term continuous wear. Our investigation and the lessons learned shed light on the opportunities for designing on-skin systems for everyday wear.

This pictorial explores the design space for communicating surface gestures to users of textile interfaces by experimenting with the interfaces’ physical design and affordances. First, we created a collection of functional and non-functional textile samples. Their development was based on three aspects: design, fabrication, and sensing. The design aspect covered different visual (shape, color) and haptic (details, textures) designs, fabrication explored three textile-specific fabrication methods, and electronic sensing offered options for adding touch-sensing capabilities. Second, we reflected on created samples and their characteristics contrasting different designs and speculating on why some work better than others. Our main findings and insights are presented in five clusters: ergonomics, visual affordances, perception of textures, the direction of movement, and the economic usage of design elements. This intermediate-level knowledge can provide a starting point for each professional or novice designer to take inspiration from, when creating their own textile user interfaces.

Abstract: We consider how seams in clothing can be designed for interactive sensing of body movement. Conductive yarn is activated at low voltages which fluctuate as body movement varies tension across the seams. Traditional garment construction describes how fabrics, blocks and seam types are configured according to mechanical performance and historical changes in fashion. Bodice blocks span the human torso, anchoring and reflecting core body movement. We explore the stability and calibration of seam voltage in a bodice block, while performing a yoga-inspired movement sequence. Our exploratory studies test signal orthogonality between seam stitch types and seam placements for supporting reliable sensing. We use our exploratory work to consider design alternatives which balance seam requirements against sensing opportunities, including back seams for torso garments. Our findings contrast optimal seam placement and design in wearable e-textiles with those in traditional clothing.

We present KnitDermis, on-body interfaces that deliver expressive non-vibrating mechanotactile feedback on the wearer’s body. Fabricated through machine knitting, they embed shape-memory alloy micro-springs in knitted channels, which deliver tactile sensations on the skin when activated. KnitDermis interfaces take advantage of machine knitting’s shaping properties which allow it to generate slim, stretchable, and versatile forms that can conform to underexplored body locations, such as protruded joints and convex body locations. We introduce a fabrication approach and a series of case studies to design a wide range of form factors, textures, and tactile patterns, including compression, pinching, brushing, and twisting. We conduct a user study to elicit KnitDermis’ effectiveness and wearability on diverse body locations and engage users to unpack envisioned use cases and perceptions towards the interfaces. We draw insights from our extensive research-through-design investigations on the potential of knitting as a soft approach for close-body and expressive tactile interfaces.

Audio-frequency induction loops (AFILs) are commonly used as an assistive listening technology for hard-of-hearing individuals. They generate an electromagnetic field proportional to a sound source receivable by hearing aids. Our interactive system, the Sound of Space, is based on AFILs that generate a multi-dimensional soundscape in space. Cochlear implant (CI) listeners and hearing-aids wearers can experience the soundscape through bodily movement, whereas hearing individuals can experience it via a corresponding tangible device. While typical AFIL installations transmit a single sound source, in our interactive system we implement overlapping loops and their interference to locate multiple synchronised audio sources (i.e., corresponding electromagnetic fields) in space. The designed system is installed permanently in an integrative school for deaf, hard-of-hearing and hearing students and teachers. In this pictorial, we illustrate our design and implementation process and contribute our learnings of using AFILs as a design material for architectural interactivity.

This paper aims to provide first insights into flash characteristics of bioluminescent microalgae as a potential media for future living light interfaces. A growing number of HCI and interaction design researchers show interest in living material interfaces, which incorporate living organisms for novel responsive behaviour and interaction possibilities in digital and biological hybrids. While much is known about the science
of these organisms, their ‘living aesthetics’, i.e., how humans experience the unique temporal changes in a living media, have hardly been explored. To bridge this gap in designing living light interfaces, this paper presents a study of bioluminescent flash characterisation. A DIY shaking device was designed to interact with the liquid living media, providing a range of stimuli including orbital rotation, pulsation and vibration. The living light aesthetics is presented with rich visuals illustrating the intensity variations over time, textural qualities and spatial distribution.

Living matter is an emerging topic of interest in HCI as researchers are recognizing the unique affordances of biological materials. We explore direct interactions with Dinoflagellates, bioluminescent algae that produce light when exposed to oxygen through physical stimulation. Leveraging Dinoflagellates’ natural feedback mechanism, we propose directly engaging the human user with the organism through physical kinetic interactions. We take an organism-centered design approach, considering the well-being of the organism by focusing first on designing appropriate environments for the organism, then proceeding to exploring the available interactions within these environments. Our framework consists of four components (form, reception, feedback, and control) and can be used to guide designers in their design process with living matter. We demonstrate the framework with environments for Dinoflagellates and three real-life examples: a checkers game, a ball game. and hopscotch. Last, we discuss the constraints and limitations of integrating Dinoflagellates and living matter in interactive systems.

Hydrogels are versatile morphing materials that have recently been adopted for creating shape-changing interfaces. However, most shape-changing interfaces require advanced material synthesis, specialized lab settings for fabrication, and technical knowledge is needed to simulate their morphing behavior. To replicate such structures, these factors become a barrier for makers. Therefore, to democratize the creation of hydrogel-based morphing artifacts and to extend their design space in HCI, we propose a water-triggered morphing mechanism that utilizes the distance between adjacent hydrogel beads adhered on a thin substrate to control their bending angle. This paper describes the bending angle quantification experiments for creating a simulator, the process of developing a computational tool along with its user-friendly workflow and demonstrates kirigami and branch-based artifacts built with the tool. Using our method, anyone can easily design and fabricate custom morphing structures.

In this pictorial, we present critical reflections on the design process of crafting the deformTable. The table is a shape-changing device that can go up and down based on the weight of objects placed on it. This pictorial focuses on key design decisions in the making of a shape-changing artifact like the deformTable that explores how deformability, temporality, and materiality can encourage creative appropriations over time. We believe this pictorial contributes to how to design research products that can investigate shape-changing artifacts in everyday settings over time.

Engaging with new types of programmable and smart materials in interaction design requires that we develop a deeper understanding of the somatic experiences that such materials can evoke. This pictorial presents an autobiographical exploration grounded on soma design methods that led to the Breathing Wings wearable: Covering the area of the back, it offers a dynamic embracing experience created through soft shape- changing materials, inviting the wearer to re-experience neglected body parts, the shoulder blades, where it evokes different qualities of ‘touch’. This work offers a detailed account of opening the space of designing with shape-change actuation through a somaesthetic approach to engaging with the body, materials and felt experiences. This rich design-led inquiry that placed the body in dialogue with materials and wearable, challenged the perception of the boundaries between body and wearable by shifting the agency from the human body to a co-living experience between the two.

We present Electriflow: a new class of soft electrohydraulic actuators as building blocks for prototyping shape-changing interfaces. These actuators are silent and fast in operation and can be fabricated with commodity materials. Electriflow generates an immediate hydraulic force upon electrostatic activation without an external fluid supply source, enabling a simple and compact self-contained design. This paper describes the materials and mechanisms of these shape-changing building blocks, as well as the underlying fabrication process, which includes a software tool that assists in their design, shape visualization and construction. Finally, we explore four classes of application prototypes: tangible animation, actuating origami creases, shape-changing phone, and shape-changing bowl.

Research into shape-changing and deformable artifacts that explore novel interactions has been growing in design and HCI. Yet, there is little discussion on the design processes behind these approaches and in particular, a theoretical understanding of materiality that is central to deformation. Informed by Wiberg’s compositional interaction design, we contribute an investigation into supporting long-term relations with deformation: the transTexture lamp. Specifically, we crafted a dynamic physical form with materials at hand. We instantiated the materiality of interaction being designed as a computational whole. Reflecting on our theory-informed RtD process enriched our understanding of compositional interaction design. These reflections on our design approach may benefit further explorations into the intersections between materiality, longevity, and actuality in the context of design-oriented HCI.

A recent study on motor-skill training showed that adaptive training tools that use shape-change to adapt the training difficulty based on learners’ performance can lead to higher learning gains. However, to date, no support tools exist to help designers create adaptive learning tools. Our formative study shows that developing the adaptive learning algorithm poses a particular challenge. To address this, we built Adapt2Learn, a toolkit that auto-generates the learning algorithm for adaptive tools. Designers choose their tool’s sensors and actuators, Adapt2Learn then configures the learning algorithm and generates a microcontroller script that designers can deploy on the tool. Once uploaded, the script assesses the learner’s performance via the sensors, computes the training difficulty, and actuates the tool to adapt the difficulty. Adapt2Learn’s visualization tool then lets designers visualize their tool’s adaptation and evaluate the learning algorithm. To validate that Adapt2Learn can generate adaptation algorithms for different tools, we built several application examples that demonstrate successful deployment.

The shift towards distance learning brought forth by the pandemic has highlighted the shortcomings of teaching physical skills at a distance. With the emergence of new augmented and connected mediums, new opportunities arise for transferring physical skills that have resisted traditional documentation methods. However, there lacks a framework that allows tutorial authors to capitalize on a new medium’s unique affordances rather than remediating existing tutorial conventions. Our work analyzes a body of tutorials rendered in various mediums for centering clay on a pottery wheel — a foundational skill that exemplifies the difficulties of physical skill transfer. Through the lens of McLuhan’s “The Medium is the Message” we synthesize a taxonomy of medium conventions and themes derived from analyzing a body of centering tutorials and observation of how a tutorial’s medium affects how learners develop physical skills. We leverage our findings to motivate design recommendations to inform how new mediums can support material practices.

We present an embodied conversational assistant (CA) designed specifically for a learning space. We first relate the attributes of our learning space and the needs and opportunities that led to the creation of our CA. We then describe its main qualities, including the mutual assistance designed between the CA and its users. We present a visual narrative of the process of designing and implementing the CA prototype, including scenario building, form making, interaction design, engineering, and textile design. We describe the working prototype and initial trials with students. Finally, we discuss the rich potential of designing embodied CAs for specific learning spaces.

Service robots are entering all kinds of business areas, and the outbreak of COVID-19 speeds up their application. Many studies have shown that robots with matching gender-occupational roles receive larger acceptance. However, this can also enlarge the gender bias in society. In this paper, we identified gender norms embedded in service robots by iteratively coding 67 humanoid robot images collected from the Chinese e-commerce platform Alibaba. We then generated four-step guidance for designers to identify and challenge the gender norms in the robot design. Our research provides both the fundamental grounding and practical guidance for designing catering robots that challenge gender norms and promote social equality.

Most robots are unclothed. However, we believe that robot clothes present an underutilized opportunity for the field of designing interactive systems. Clothes can help robots become better robots––by helping them be useful in a new, wider array of contexts, or better adapt and function in the contexts they are already in. In this paper, we provide a foundation for a research area of robot clothing by speculating on its potential. We systematically present functional requirements of robot clothing, considerations, and parameters for robot clothing designers, as well as key reference cases of robots in clothes. We then discuss what robot clothes can do specifically for the field of designing interactive systems.

This paper is a methodological replication of Barendregt et al. [11], who urged Child-Computer Interaction field to embrace Intermediate Level Knowledge as a meaningful and valid way of generating knowledge. We extend this epistemological gap to the Human-Robot Interaction (HRI). Currently, artefact-centered papers—papers that present the development of an artefact—seem to be one of the primary ways that the HRI field generates knowledge. In this paper, we made an analysis of all papers presented at the HRI Conference from 2006 to 2020. Our results indicate that the 41,2 % of the papers were artefact-centered; and the impact of them, measured in the number of citations, was significantly lower than other kinds of papers. We used 23 artefact-centered papers to formulate two strong concepts and investigate how the foundational design epistemology about intermediate-level knowledge and RtD can contribute to other design-related disciplines to produce useful and valuable knowledge.

The word “robot” frequently conjures unrealistic expectations of utilitarian perfection: tireless, efficient and flawless agents. However, real-world robots are far from perfect—they fail and make mistakes. Thus, roboticists should consider altering their current assumptions and cultivating new perspectives that account for a more complete range of robot roles, behaviors, and interactions. To encourage this, we explore the use of metaphors for generating novel ideas and reframing existing problems, eliciting new perspectives of human-robot interaction. Our work makes two contributions. We (1) surface current assumptions that accompany the term “robots,” and (2) present a collection of alternative perspectives of interaction with robots through metaphors. By identifying assumptions, we provide a comprehensible list of aspects to reconsider regarding robots’ physicality, roles, and behaviors. Through metaphors, we propose new ways of examining how we can use, relate to, and co-exist with the robots that will share our future.

Robots are increasingly being introduced into domains where they assist or collaborate with human counterparts. There is a growing body of literature on how robots might serve as collaborators in creative activities, but little is known about the factors that shape human perceptions of robots as creative collaborators. This paper investigates the effects of a robot’s social behaviors on people’s creative thinking and their perceptions of the robot. We developed an interactive system to facilitate collaboration between a human and a robot in a creative activity. We conducted a user study (n = 12), in which the robot and adult participants took turns to create compositions using tangram pieces projected on a shared workspace. We observed four human behavioral traits related to creativity in the interaction: accepting robot inputs as inspiration, delegating the creative lead to the robot, communicating creative intents, and being playful in the creation. Our findings suggest designs for co-creation in social robots that consider the adversarial effect of giving the robot too much control in creation, as well as the role playfulness plays in the creative process.

Social agents and robots are moving into front-line positions in brick and mortar services, taking on roles where they directly interact with customers. These agents could potentially recognize customers to personalize service. Will customers like this, or might they feel monitored and profiled? Robots could also re-embody (move their “personality” between one body and another) in order to take on multiple roles that are typically performed by different people. Will this make customers feel more taken care of, or will it raise concerns about the robot’s competence and expertise? Our work investigates when robots should and should not recognize customers and re-embody. Our online study used storyboards to present possible future interactions between robots and customers across several different service contexts. Our findings suggest that people generally accept robots identifying customers and taking on vastly different roles. However, in some contexts, these robot behaviors seem creepy and untrustworthy.

People’s physical movement and body language implicitly convey what they think and feel, are doing or are about to do. In contrast, current smart speakers miss out on this richness of body language, primarily relying on voice commands only. We present QUBI, a dynamic smart speaker that leverages expressive physical motion – stretching, nodding, turning, shrugging, wiggling, pointing and leaning forwards/backwards – to
convey cues about its underlying behaviour and activities. We conducted a qualitative Wizard of Oz lab study, in which 12 participants interacted with QUBI in four scripted scenarios. From our study, we distilled six themes: (1) mirroring and mimicking motions; (2) body language to supplement voice instructions; (3) anthropomorphism and personality; (4) audio can trump motion; (5) reaffirming uncertain interpretations to support mutual understanding; and (6) emotional reactions to QUBI’s behaviour. From this, we discuss design implications for future smart speakers.

In this paper, we propose a strong concept for interaction design: Synergistic Social Technology (SST). This concept describes systems in which technology is designed with its own ‘need’ for interaction. As a result of responding to these needs, people who use the system may benefit from social interaction with others who use the system. This concept arose through the design, prototyping, and study of a social wearable design that we called Robo-Shoe-Flies. We articulate the core principles of the SST concept. We also describe the Research-through-Design process that inspired its development and associated design-focused observations. This work may inspire those in the IxD or HCI communities focused on the design and development of technology intended to support social interaction, particularly in the sense of encouraging collective, mutually-beneficial action.

Interactions with voice-based conversational agents (VCAs) in non-humanoid forms are becoming increasingly pervasive, and researches on non-humanoid VCAs engaging diverse human traits have been conducted. However, there have never been studies employing living body’s physiological states to be expressed solely through the voice of such VCAs yet. As physiological expressions of such VCAs can have potential for manifesting health-related issues in a human-like way, we selected healthcare scenarios as a case for exploring novel user experiences that they can induce. We conducted design workshops for identifying design considerations and design opportunities for the physiologically expressible VCAs in the healthcare service domain. Following these findings, we designed the new concept of physiologically expressible healthcare VCAs and conducted a Wizard-of-Oz user study. Finally, we summarize the unique user experiences on physiologically expressible VCA’s healthcare services and user perceptions of its physiological expressions, and discuss design implications for physiologically expressible VCAs.

Annotated datasets of application GUIs contain a wealth of information that can be used for various purposes, from providing inspiration to designers and implementation details to developers to assisting end-users during daily use. However, users often struggle to formulate their needs in a way that computers can understand reliably. To address this, we study how people may interact with such GUI datasets using natural language. We elicit user needs in a survey (N = 120) with three target groups (designers, developers, end-users), providing insights into which capabilities would be useful and how users formulate queries. We contribute a labelled dataset of 1317 user queries, and demonstrate an application of a conversational assistant that interprets these queries and retrieves information from a large-scale GUI dataset. It can (1) suggest GUI screenshots for design ideation, (2) highlight details about particular GUI features for development, and (3) reveal further insights about applications. Our findings can inform design and implementation of intelligent systems to interact with GUI datasets intuitively.

Chatbots are widely employed in various scenarios. However, given the high costs of chatbot development and chatbots’ tremendous social influence, chatbot failures may inevitably lead to a huge economic loss. Previous chatbot evaluation frameworks rely heavily on human evaluation, lending little support for automatic early-stage chatbot examination prior to deployment. To reduce the risk of potential loss, we propose a computational approach to extracting features and training models that make a priori prediction about chatbots’ popularity, which indicates chatbot general performance. The features we extract cover chatbot Intent, Conversation Flow, and Response Design. We studied 1050 customer service chatbots on one of the most popular chatbot service platforms. Our model achieves 77.36% prediction accuracy among very popular and very unpopular chatbots, making the first step towards computational feedback before chatbot deployment. Our evaluation results also reveal the key design features associated with chatbot popularity and offer guidance on chatbot design.

Anthropomorphism in the design of interface agents is implicitly linked to increasing user trust and acceptance. However, the role of perceived anthropomorphism and perceived trustworthiness in trust appropriateness given a system’s capabilities and limitations is unclear. We designed a 2 (reliability: low, high) x 3 (agent appearance: computer, avatar, human) between-subject study to observe how agent appearance influenced user perceptions of and reliance on an automated teammate in a collaborative image classification task. Trust appropriateness was characterized as the degree to which reliance matched an optimal level given the system’s reliability. Although agent appearance did not significantly influence trust appropriateness, it did affect perceptions of trustworthiness, particularly for low reliability agents. Our results suggest that trust and anthropomorphism involve highly related, dynamic perceptions aimed at anticipating system behavior. Based on our findings, recommendations for future research on trust and anthropomorphism are discussed along with some design implications.

Open-domain chatbots that can engage in a conversation on any topic received significant attention in the last several years, which opened opportunities for studying user interaction with them. Drawing from reviews of chatbots posted on Google Play, we explore user experience and expectations of these agents in a mixed-method study. Results of statistical analysis reveal which social qualities of chatbots are the most significant for user satisfaction. Further, we employ natural language processing and qualitative methods to identify how users wish their chatbots to evolve in the future. While currently users mostly value the entertaining component of their experience, their expectations call for more human-like behavior of chatbots. The most prominent expectations include chatbots’ abilities to treat and express emotions and be more attentive to the user. Based on these findings, we conclude with design implications, discussing the directions for developing social skills of open-domain chatbots.

Prior efforts have noted the effect of reliability, risk, and degree of anthropomorphism on trust in the context of human-agent interaction. However, the effects of these factors on resulting emotions while interacting with autonomous agents and their influence on trust are not clear. Towards that, we designed a 2 (partner: automation/human) × 2 (risk: low/high) × 2 (reliability: low/high) between-group study to identify relevant discrete emotions and their (emotions’) influences on users’ trustworthiness perceptions (ability, integrity, and benevolence). The results identified four emotion factors (positive emotions, hostility, anxiety, and loneliness) related to human-agent interaction. Although the reliability condition affected all four emotion factors, the mediating effects of the emotion factors on reliability and trustworthiness perceptions relationships differed for the varying emotion factors. The implications of our findings for trust calibration in the context of designing interactive systems are discussed in the paper.

Online threats to children, in the form of cyberbullying, grooming, and sexting, have reached unprecedented and alarming levels. Support is still lacking despite endless efforts by child safety organisations and online safety educational programmes. This is mainly due to children feeling apprehensive in such situations, ashamed of revealing their distressing encounter to an adult or even for not having anyone to approach with their concerns. This paper investigates how children envision the potential support of a chatbot in such contexts. We captured design requirements for such a chatbot through a participatory design approach involving 110 schoolchildren in the UK. Using LEGO figures, they elaborated and performed stories featuring the interaction of a child under threat with a chatbot. The analysis of the dialogues in their performances and their reflections resulted in a set of expected tasks for the chatbot, a conversation flow, and novel socio-technical requirements addressing potential users’ main concerns and expectations.

We propose a method to integrate more interactivity into automatic hyperparameter optimization systems to leverage the user’s prior knowledge on parameter distribution. In our method, the user continuously observes automatic optimization’s progress and dynamically specifies where to search in the parameter space. We present a prototype implementation of an interactive dashboard for an optimizer to show our method’s feasibility. The interactive dashboard’s main feature is “paintable timeline” where the user can not only observe the past parameter values tested as in standard timeline but also specify the range of future parameters to be tested with simple painting operations. We show three examples where user intervention might improve the performance of automatic optimizations. We run a user study with experts and the results show that, with prior knowledge about parameter distribution of the target problem, interactive optimization can reach better results compared to fully automatic optimization.

Thinking of technology as a design material is appealing. It encourages designers to explore the material’s properties to understand its capabilities and limitations—a prerequisite to generative design thinking. However, as a material, AI resists this approach because its properties only emerge as part of the user experience design. Therefore, designers and AI engineers must collaborate in new ways to create both the material and its application experience. We investigate the co-creation process through a design study with 10 pairs of designers and engineers. We find that design ‘probes’ with user data are a useful tool in defining AI materials. Through data probes, designers construct designerly representations of the envisioned AI experience (AIX) to identify desirable AI characteristics. Data probes facilitate divergent design thinking, material testing, and design validation. Based on our findings, we propose a process model for co-creating AIX and offer design considerations for incorporating data probes in AIX design tools.

Intelligent systems in everyday lives learn about their users to tailor services over time. However, these systems are often designed with little consideration of user agency on their learning processes, hindering users from taking full advantage of the systems. In this paper, we propose Teaching-Learning Interaction (TLI) as a new form of interaction that affords user agency by letting users reflectively shepherd an intelligent system’s manner of learning. Given such agency, users will be able to better personalize services for themselves. We first draw on Schön’s notion of knowing-in-action and reflective practice to theoretically ground our concept. We then present the resulting definition of TLI and three design qualities, which are further concretized with three design examples. We end with discussion on the implications of TLI for HCI design.

With the growing industry applications of Artificial Intelligence (AI) systems, pre-trained models and APIs have emerged and greatly lowered the barrier of building AI-powered products. However, novice AI application designers often struggle to recognize the inherent algorithmic trade-offs and evaluate model fairness before making informed design decisions. In this study, we examined the Objective Revision Evaluation System (ORES), a machine learning (ML) API in Wikipedia used by the community to build anti-vandalism tools. We designed an interactive visualization system to communicate model threshold trade-offs and fairness in ORES. We evaluated our system by conducting 10 in-depth interviews with potential ORES application designers. We found that our system helped application designers who have limited ML backgrounds learn about in-context ML knowledge, recognize inherent value trade-offs, and make design decisions that aligned with their goals. By demonstrating our system in a real-world domain, this paper presents a nove visualization approach to facilitate greater accessibility and human agency in AI application design.

A key task in design work is grasping the client’s implicit tastes. Designers often do this based on a set of examples from the client. However, recognizing a common pattern among many intertwining variables such as color, texture, and layout and synthesizing them into a composite preference can be challenging. In this paper, we leverage the pattern recognition capability of computational models to aid in this task. We offer a set of principles for computationally learning personal style. The principles are manifested in PseudoClient, a deep learning framework that learns a computationa model for personal graphic design style from only a handful of examples. In several experiments, we found that PseudoClient achieves a 79.40% accuracy with only five positive and negative examples, outperforming several alternative methods. Finally, we discuss how PseudoClient can be utilized as a building block to support the development of future design applications.

Recently, methods enabling humans and Artificial Intelligent (AI) agents to collaborate towards improving the efficiency of Reinforcement Learning – also called Collaborative Reinforcement Learning (CRL) – have been receiving increasing attention. In this paper, we provide a long-term, in-depth survey, investigating human-AI collaborative methods based on both interactive reinforcement learning algorithms and human-AI collaborative frameworks, between 2011 and 2020. We elucidate and discuss synergistic analysis methods of both the growth of the field and the state-of-the-art; we suggest novel technical directions and new collaboration design ideas. Specifically, we provide a new CRL classification taxonomy, as a systematic modelling tool for selecting and improving new CRL designs. Furthermore, we propose generic CRL challenges providing the research community with a guide towards effective implementation of human-AI collaboration. The aim is to empower researchers to develop more efficient and natural human-AI collaborative methods that could utilise the different strengths of humans and AI.

The interpretability or explainability of AI systems (XAI) has been a topic gaining renewed attention in recent years across AI and HCI communities. Recent work has drawn attention to the emergent explainability requirements of in situ, applied projects, yet further exploratory work is needed to more fully understand this space. This paper investigates applied AI projects and reports on a qualitative interview study of individuals working on AI projects at a large technology and consulting company. Presenting an empirical understanding of the range of stakeholders in industrial AI projects, this paper also draws out the emergent explainability practices that arise as these projects unfold, highlighting the range of explanation audiences (who), as well as how their explainability needs evolve across the AI project lifecycle (when). We discuss the importance of adopting a sociotechnical lens in designing AI systems, noting how the “AI lifecycle” can serve as a design metaphor to further the XAI design field.

Introspection is the practice of looking inward and examining our ideas, thoughts,and feelings. It involves considering past experiences and asking questions aboutthe future. We report on a design research inquiry that explores Artificial Intelligence(AI), combined with personal data, as a resource for introspection. We investigatehow AI might offer possibilities for generating alternative perspectives on one’slife to support introspection and paradoxes that this might raise. We describe ourdesign-led inquiry, motivate fi approaches to introspective practice as opportunitiesfor potential Introspective AI interventions, and explore them through seven designproposals. Taken together, our proposals provoke questions around how introspectiveAI might be critiqued, imagined, and designed. We conclude with a reflection on ourwork and the opportunities it suggests for future research and practice.

The recent rise in on-device AI computer vision and dialogue systems has facilitateda growing number of AI fitness related instructional apps. However, these technologieshave yet to be explored within the HCI community. To investigate this domain we recruited12 participants and asked them to engage with five recently launched AI fitness instructorapps. We interviewed participants and thematically analysed transcripts to understandtheir experience and expectations of these technologies. Our qualitative analysisoutlines five main themes focusing on; limitations of computer vision, visual feedback,dialogue with the AI, adapting to the user, and working out with the instructor. Basedupon our findings we present five design considerations for designers that relateto three key areas: feedback and motivation, personalising the experience, and buildinga relationship with the AI. We contribute a first look into people’s initial experienceswith on-device AI fitness instructor applications and we provide design considerationsto guide future contextually-aware AI research in this domain.

In this paper we explore what role humans might play in designing tools for reinforcementlearning (RL) agents to interact with the world. Recent work has explored RL methodsthat optimize a robot’s morphology while learning to control it, effectively dividingan RL agent’s environment into the external world and the agent’s interface with theworld. Taking a user-centered design (UCD) approach, we explore the potential of ahuman, instead of an algorithm, redesigning the agent’s tool. Using UCD to designfor a machine learning agent brings up several research questions, including whatit means to understand an RL agent’s experience, beliefs, tendencies, and goals. Afterdiscussing these questions, we then present a system we developed to study humansdesigning a 2D racecar for an RL autonomous driver. We conclude with findings andinsights from exploratory pilots with twelve users using this system.

This pictorial is the result of a long-term research project focused on finding opportunitiesto use Apple Face ID software and infrastructure to build new hybrid identities, exploringits potential as a drag, queer and trans technology design tool. The aim of the projectis to validate new paradigms for identity ambiguity, multiplicity and fluidity, usingbiometric artificial intelligence systems and technologies of control as tools forfree identity expression, definition and personal proclamation. To do that, we presentour motivation, methodology, approach to exploration, workshop results and validatedoutputs, and analyse the physical and digital implications of the research. The prosthesesshown in this paper are functional, and allow users to train Apple Face ID softwareto design their own identities connected to their name, Apple ID Number, bank account,social networks, and any other data or online activity digitally associated to thispersona.

Through a process of robust co-design, we created a bespoke accessible survey platformto explore the role of co-researchers with learning disabilities (LDs) in research design and analysis. A team of co-researchers used this system to create an onlinesurvey to challenge public understanding of LDs [3]. Here, we describe and evaluatethe process of remotely co-analyzing the survey data across 30 meetings in a research team consisting of academics and non-academics with diverse abilities amid new COVID-19lockdown challenges. Based on survey data with >1,500 responses, we first co-analyze ddemographics using graphs and art & design approaches. Next, co-researchers co-analyzed the output of machine learning-based structural topic modelling (STM) applied to open-ended text responses. We derived an efficient five-steps STM co-analysis process for creative, inclusive, and critical engagement of data by co-researchers. Co-researchers observedthat by trying to understand and impact public opinion, their own perspectives also changed.

Information technology is increasingly designed to increase information transparency as a way to increase trust. However, it can be hard to comprehend and anticipate the social implications of information visibility, people’s competing expectations of transparency versus privacy, and the role of enabling technologies. We devised Live Action Role Play (LARP) as a means for people to explore the potential and consequences of transparency, within a social context of daily transactions related to food, fashion, and finance. Through the process of deliberation, we enabled participants to critically assess their co-created LARP experience and articulate their transparency expectations for design considerations. We report on insights into their expectations and perceivedlimitations of information technology in delivering transparency, including social measures required to realise the full potential of technology, as well as transparency, while minimising unintended consequences.

Despite longstanding interests to understand user involvement in participatory, community-based research, we are still learning how including different users can influence the design process. In this paper, we share what we learned examining older adults’ participation in formative studies conducted to redesign a resident-led, internal website at a Life Plan Community in a partnership with residents and staff. We administered questionnaires followed by focus groups and interviews to understand residents’ experiences and concerns with the website; however, we found early in our process that several non-users participated advocating for changes outside the scope of our initial project goals. We reflect on how participation by different users, particularly non-users, helped us uncover design tensions related to underlying socio-technical concerns about the website. Our work builds on work promoting critical reflection in HCI. We extend work on participation, non-use, and advocacy in design and share practical examples of adapting our approach to prepare for design activities.

Digital Commons have been widely studied in HCI and co-design literature as an alternative to technologies produced by capitalistic corporations. In this paper, we present an action-research process that facilitates the use of digital commons among a group of participants. The process core aspect is understanding our participants’ interests and values to present multiple off-the-shelf tools fitting their needs. In doing so, we leveraged on Artifact Ecologies, using them as an action-research tool. The process is articulated in three main phases: 1) we engaged participants in a reflection on their Artifact Ecologies, connecting the artifacts in use to their values; 2) we used these results to selected valuable digital commons; 3) we presented these tools to the group, connecting them to participants’ ecologies. At the end of the process, we performed a collective interview to evaluate the process alongside assessing the adoption of the proposed technologies. Based on our findings, we developed five tactics to foster digital commoning.

People’s data practices and their supporting personal informatics are imbued with a wide range of concepts of value that are central to their data imaginaries. They inform how individuals adopt particular products and services. This elusiveness of value creates a challenge for designers to gauge users’ value preferences, which are personal, contextual and abstract. We propose deliberation as a process to operationalise design for elicitation, reflection and value formation, using Live Action Role Play (LARP). The paper demonstrates how our deliberative LARP precipitates the otherwise ethereal data values to unfold in a social setting. We argue that engaging people in reflecting on and playing out their own value formation offers designers more powerful insights to address complex challenges of data-driven societies. We report on our findings that agency and negotiability are key to transcend the politics of preferences in data practices and thus enhance acceptability.

For readability and disambiguation of the written text, appropriate word segmentation is recommended for documentation, and it also holds for the digitized texts. If the language is agglutinative while far from scriptio continua, for instance in the Korean language, the problem becomes more significant. However, some device users these days find it challenging to communicate via key stroking, not only for handicap but also for being unskilled. In this study, we propose a real-time assistive technology that utilizes an automatic word segmentation, designed for digital minorities who are not familiar with electronic typing. We propose a data-driven system trained upon a spoken Korean language corpus with various non-canonical expressions and dialects, guaranteeing the comprehension of contextual information. Through quantitative and qualitative comparison with other text processing toolkits, we show the reliability of the proposed system and its fit with colloquial and non-normalized texts, which fulfills the aim of supportive technology.

The global education landscape continues to be disrupted by COVID-19. The unique circumstances created by this crisis has led to unexpected opportunities to leverage innovative uses of technologies that can radically innovate education service delivery. We map out the context of Pakistan – a country where challenges pertaining to both access to education and quality learning predate the pandemic. A context largely under explored in design and HCI research. This paper presents insights from conversations with 21 stakeholders across 15 design activities, that were structured around speculative design proposals to collaboratively envision the future of education in Pakistan. The paper offers directions for the design of EdTech through a plural, and culturally rooted speculative discussion, highlighting (1) opportunities and considerations for localising design for education, and (2) reframing the value of education in Pakistan.

Training AI systems requires large datasets. While there are a range of existing methods for collecting such data, such as paid work on crowd sourcing platforms, the strengths and weaknesses of each method leads us to believe that new, complementary methods are needed. The Polyphonic project contributes a novel method for collecting real-world data by piggybacking on game streaming communities such as Twitch, which capture over a trillion minutes of viewer attention a year. By embedding activities within the sociotechnical context of the stream, we can leverage some of this attention for data collection and processing. In this paper, we describe the design and implementation of a proof-of-concept system for collecting home audio data. We conducted a field study in four live streams and found that our proof-of-concept effectively supports data capture. We also contribute further design insights about stream-based data collection systems.

This paper documents the collaborative design project: “Amagugu Ethu / Our Treasures: Understanding Zulu History and Language with Zulu-Speaking Communities and Their Belongings.” With eight Zulu experts, and the company Museum in a Box [32], we collaborated to create an oral history documentation project that focuses on the belongings at the Iziko South African Museum (SAM). Using the Museum in a Box tool, we created a movable, audio-based exhibit that compiled stories and images of objects from the SAM, for use back in the community in KwaZulu-Natal. This paper documents our shared process and considers the development of this exhibit and documentation project as a kind of participatory anarchive or counter archive [20, 41]. We connect practices and theories in participatory design with those in museum studies and archival studies; and show the productive tensions that exist in an experimental, community-engaged oral history documentation project.

Participatory design is positioned as an approach to address community challenges and reimagine potential futures by decentralizing systems of power and centering the marginalized, moving away from designers as experts and framing design as being community-driven. Methods of speculative participatory design are useful in imagining futures among marginalized groups while negotiating existing societal constructs. While research probes have been developed to support the practice of speculation in design, we have yet to see research probes in HCI that consider the lived experiences of Black and brown communities. In this pictorial, we illustrate the process of developing methods cards as design probes to encourage speculative and critical design thinking from an Afrofuturism lens. We highlight how Afrofuturism centers Black communities in design speculation lending to both equity-driven design outcomes, and an accessible resource that can be readily used by designers and non- designers alike in addressing community concerns.

Prototypes and other ‘things’ have had many uses in HCI research—to help understand a problem, as a stepping stone towards a solution, or as a final outcome of a research process. However, within the messy context of a research through design project, many of these roles do not form part of the final research narratives, restricting the ability of other researchers to learn from this practice. In this paper we revisit prototypes used in three different design research projects, conducted over a period when the Internet of Things emerged into everyday life, exploring complex hidden relationships between the internet, people and physical objects. We aim to explore the unreported roles that prototypes played in these projects, including brokering relationships with participants and deconstructing opaque technologies. We reflect on how these roles align with existing understandings of prototypes in HCI, with particular attention to how these roles can contribute to design around IoT.

Creativity Support Tools (CSTs) have become an integral part of artistic creation. The range of CST technologies is broad—from fabricators to generative algorithms to robots. The interaction approaches for CSTs are accordingly broad. CSTs combine specific technologies and interaction types to serve a spectrum of roles and users. In this work, we tackle a comprehensive understanding of how the intersections of users, roles, interactions, and technologies form a design space for CSTs. We accomplish this by reviewing 111 art-creation CSTs from HCI and computing research and analyzing how diverse aspects of CSTs relate to each other. Our findings identify patterns for designing CSTs, which can give guidance to future CST designers. We also highlight under-explored types of CSTs within the HCI community, providing future directions that CST researchers can pursue given the current trajectory of technological advancement. This work contributes an integrating perspective to understand the landscape of art-creation CSTs.

Examples are powerful tools for creativity and can provide inspiration and structure. However, novices often don’t know when to seek examples or how to apply them. Showing real-time examples targeted to the current task (adaptive conceptual guidance) may help novices consider inspiration more often and better implement the ideas that examples illustrate. We explore this in a Wizard-of-Oz system, Shöwn, that presents examples based on the user’s current activity while drawing a comic strip. Shöwn’s design is informed by interviews with novice and expert comic artists (n=18). A between-subjects experiment (n=24) found that adaptive conceptual guidance improved the clarity and uniqueness of drawings and stories compared to non-adaptive examples. Users also found the examples more useful and inspirational than those without adaptive guidance. Our results present an initial exploration of the challenges and benefits of contextually showing examples in interactive creativity tools.

As HCI pedagogy research grows, so too does an emerging set of evidence-based teaching and curricular recommendations. Yet, few studies have implemented and examined these recommendations in the classroom. In this paper, we present a Research Through Design investigation of a studio-based HCI course, which was revised based on HCI education literature. Drawing on reflection surveys, video recordings of student-led user sessions, final project artifacts, and student interviews, we explore how students responded to key educational changes, the strategies that supported and hindered their reflective practices, and how reflection afforded new student insights. Our findings highlight the utility of video-based reflection exercises to support student learning in designing and running user sessions, the importance of multi-faceted reflection prompts, and how students noticed moments of inclusion and exclusion by attending to users’ non-verbal cues. Additionally, we empirically demonstrate the importance of implementing and studying HCI education research recommendations in the classroom.

This article argues that design workbooks can benefit the field of Information Communication Technology and Development (ICTD). To demonstrate this, I present a workbook comprised of 12 speculative design proposals. I then present findings from interviews conducted with 22 participants in Bungoma, Kenya; I used the workbook images as prompts during these sessions. My findings suggest that the design workbook method supports a participant-driven interview process. The workbook images prompted rich responses from participants about the contexts where the ideas would exist. These responses draw attention to the practical problems that might accompany the introduction of the ideas into their communities. Significantly, these responses also included critical feedback. Important information was gleaned from comparing and contrasting the multiple ideas in the workbook; these insights include novel understandings about surveillance and participant/researcher relations. These findings motivate a discussion about how design workbooks support different ways for people to participate in the design process, and encourage different outcomes in ICTD.

In this paper, we investigate how to elicit new perspectives in research-through-design (RtD) studies through annotated portfolios. Situating the usage in human-robot interaction (HRI), we used two robotic artefacts as a case study: we first created our own annotated portfolio and subsequently ran online workshops during which we asked HRI experts to annotate our robotic artefacts. We report on the different aspects revealed about the value, use, and further improvements of the robotic artefacts through using the annotated portfolio technique ourselves versus using it with experts. We suggest that annotated portfolios – when performed by external experts – allow design researchers to obtain a form of creative and generative peer critique. Our paper offers methodological considerations for conducting expert annotation sessions. Further, we discuss the use of annotated portfolios to unveil designerly HRI knowledge in RtD studies.

The purpose of this pictorial is to provide a unique perspective, through an autoethnographic account, of the experiences a woman of colour may have, as she seeks to enter into the world of makerspaces. As an intersectional woman, conducting this autoethnography is a means to assess the cultural and gender diversity of makerspaces in an Australian metro city, and their practices to foster inclusivity. Over a period of 6 months, the lead author explored 4 different types of makerspaces. The autoethnography found cultural and gendered roles and biases in the physical and socio-cultural environments of each makerspace, as well as in terminologies, tools, materials and learning pathways that constrained her involvement. The research identifies important factors that could be amplified to increase the involvement of intersectional women and the foundations for new guidelines and activities that would enable makerspaces to improve their diversity and inclusivity.

Individuals with spinal cord injuries (SCI) can have multiple cognitive and physical disabilities because of their injury. Appropriately-designed technology can be empowering and transformative for this population. Unfortunately, just like most technologies, user-centered research methods do not directly account for differing motor and communication abilities. This paper synthesizes SCI literature and leverages our own experiences in three research projects spanning five years with SCI users to highlight significant challenges that HCI researchers might face while employing user-centered methods with this population; communication disabilities, motor disabilities, and difficult contextual or environmental factors can make it difficult or impossible to use standard HCI methods when working with SCI users. We conclude with a set of guidelines and challenges for the HCI community to consider, which can be used both when evaluating papers that work with this population, and to fuel development of new methods or approaches that better-serve them.

The ongoing transition within tech industry towards holistic service journey orchestration pushes user experience design (UXD) closer to service design (SD). At the same time, new digital channels for traditional services pushes SD closer to UXD. The increasing interplay between UXD and SD can cause confusion about their relationship, boundaries, and responsibilities. Little research has examined perceptions of these aspects in industry and academia. We surveyed 197 UXD and SD practitioners and academics about their understanding of the overlaps and boundaries between the two design practices. Our findings reveal several conflicting views around the scope of UXD and SD work, the subsumptive relation between UXD and SD, and the theoretical roots and methods each draws upon. We suggest constructive ways for the two design practices to complement each other. Our study shows that UX work is changing, and understanding this change is important to maintain the relevance of UX research.

Animated GIFs are often viewed as a nod to early internet culture or as tools for digital communication, but in this pictorial, we highlight a new use of GIFs, as tools for design research. We walk through four case studies from our own research that exemplify GIFs used throughout the design process as empirical probes, prototypes, communication tools, and finalized artifacts. By conducting a collaborative, reflexive analysis of these cases, we present an annotated portfolio of the goals, crafting and aesthetic choices of our GIFs and how creating GIFs added to our research. We conclude by noting that both the aesthetics of movement and the rich, concise, and contextualized nature of gifs added to our depth of thinking and ability to communicate speculative and imaginative concepts. Finally, we also suggest that research dissemination, especially for design research, would be enriched by supporting more diverse knowledge-production artifacts such as GIFs.

The information we access on the Internet appears immediately but usually lives far away. The Desktop Odometer is a device that shows users the distance they travel when browsing the web by tracking the total miles between their current location and the server from which they are requesting information. In this work, we investigated internet infrastructures by designing and producing Desktop Odometers, selling them on Amazon.com, and receiving customer reviews. We present our analysis of customer reviews which reveal how customers describe their understandings of internet infrastructures after using the device. We also recount our RtD approach to making the device; we describe frictions we encountered when navigating other opaque infrastructures in our fabrication process, such as the Google Play store. Finally, we reflect on our use of Amazon.com and customer reviews as a method to engage participants in discussion about internet infrastructure through the sale and review process.

Recent HCI scholarship has critiqued anthropocentrism in design as contributing to the ecological crisis. The current paper contributes theoretically and empirically to this area, adopting the lens of Socio-Ecological Relations, inspired by Environmental History and STS. We argue that a socio-ecological lens can offer practical tools to HCI and Interaction Design to overcome the separation between humanity and nature-as-a-resource. For this reason, our unit of analysis is humanity-in-nature and nature-in-humanity, and we show how this double internality becomes visible using an historical and geographical approach in the design process. Methodologically, using Situational Analysis, we analyse an Action Research project that introduced a digital platform for community radio in rural contexts. We find that Socio-Ecological Relations, performed in specific places, shape the design process and are themselves transformed by becoming part of it. We provide a set of sensitivities for operationalizing this approach for design researchers.

The internet is full of invisible borders—geographic, linguistic, cultural, political—that circumscribe the information each user sees. Search engines shape such “information borders” by tailoring results according to geolocation, language, and other user profiling. We present Search Atlas, a tool paired with visualizations that enable users to see and cross these borders. For instance, how do search results for the same query differ for Brazilian, Turkish, and Indian users? Given a query, the tool displays multiple lists of Google search results, highlighting distinctive words for each set of parameters. Then, we provide visu- alizations that juxtapose and cluster Google results across countries, revealing new information borders and regions that can vary widely depending on the query. By exposing the partial perspective of a search engine, Search Atlas invites users to experience the internet from divergent positions and to reflect on how their online lives are conditioned by technologi- cal infrastructures and geopolitical regimes.

This paper explores how aesthetic violation can be used to influence a user’s behavior when interacting with a product. Aesthetic violation occurs when the elements of a design purposely deviate from the aesthetic expectations of the user in order to encourage a behavioral response. This draws from the theory of processing fluency, which suggests that aesthetic perceptions are a function of a perceiver’s processing dynamics—-the more fluently a perceiver can process an object, the higher the aesthetic response. In this instance, the desire for fluent processing may incite users to escape, minimize, or counteract the violation. A literature review and two workshops were conducted: (1) to generate design strategies and (2) to assess and refine them. The workshops provided insights into the benefits and relevance of aesthetic violation, strategy formation, and integration into the design process.

Bodily transformations that attend breastfeeding include entanglements of more-than-human materials and agencies. These can be seen in exchanges of physical matter, such as bacteria, that blur bodily boundaries. I present three design explorations of my breastfeeding experiences as entangled: knitting bras for lopsided breasts, transforming milk into fiddling necklaces, and site-writing around breastfeeding. Through spatial and conceptual mappings of the explorations, I propose them as alternative narratives in designing for leaky breastfeeding bodies. I also offer two broader reflections on designing with, for, and among more-than-human bodily materials: generous absence and bodily mappings. The accompanying reading instructions to this bodily research open for further encounters and reflections between the three explorations.

In this paper, we outline a diffractive practice of machine learning (ML) in the frame of material-centered interaction design. To this aim, we review related work in ML, HCI, design, new interfaces for musical expression, and computational art, and introduce two practice-based studies of music performance and robotic art based on interactive machine learning tools, with the hope of revealing the computational materiality of ML, and the potential of embodiment to craft prototypes of ML that reconfigure conceptual or technical approaches to ML. We derive five interference conditions for such art-based ML prototypes—situational whole, small data, shallow model, learnable algorithm, and somaesthetic behaviour—and describe their widening of design and engineering practices of ML prototyping. Finally, we sketch how a process of intra-active machine learning could complement that of interactive machine learning to take materiality as an entry point for ML design within HCI.

Work combining live performance and technology often involves incorporating technology directly into the performance as it occurs onstage, including interactive costumes, or performer-controlled sets, lighting or sound. We invert this common approach, developing technology-mediated experiences outside the temporal and spatial confines of a live theatre production. We describe the 4-month co-design process with expert theatre practitioners, and detail how the process 1) shaped our design guidelines and 2) expands the discussion around existing best practices for cross-disciplinary collaboration. In the style of research through design, we present three annotated prototypes: the Augmented Playbill, the Prayer Wheel, and Tarot Cards as well as accompanying AR applications to convey the decisions we made and the philosophy we iteratively developed throughout the project. These artifacts also embody our six design guidelines: resonant affordances, extended narrative, reflective interaction, selective reveal, personalized experience, and privileged access.

Dance making is often a highly idiosyncratic, collaborative endeavour between a choreographer and a group of dancers that constitutes a rich context for designers of creativity-support tools (CSTs). However, long-term, ecologically valid studies of collaboration in dance making are rare, especially when mediated by digital tools. We present a 5-month field study in the frame of a dance course, where a choreographer and six students used a CST originally designed for choreographic writing. We contrast our findings with our initial assumptions about the role of the tool to mediate a diversity of notating styles and hierarchical roles. We highlight the value of and the challenges behind this in-the-wild study in uncovering needs and roles as they emerged over time.

Learning ballet is a complex motor task that can be effectively enhanced by technology. Learning technologies, however, are not typically used for the assessment of ballet technique due to a lack of adequate and non-invasive tools that can be pragmatically adopted. We conducted an interview-based qualitative study with seven expert ballet teachers and six pre-professional dancers to examine their current and future technology use in a ballet technique class. Through inductive and deductive analysis, we identified reasons for technology non-use and derived seven requirements that can inform the design and implementation of ballet assessment technologies including designing for: adaptation to multi-skill/multi-method environments, teacher/dancer skill augmentation, agency, non-invasive design, feedback for gross/fine movements, trust, and proprioception by supporting transformative assessment. We discuss barriers for technology acceptance and unintended consequences that should be considered when designing future technologies for ballet.

In the domain of computing education for children, much work has been done to devise creative and engaging methods of teaching about programming. However, there are many other fundamental aspects of computing that have so far received relatively less attention. This work explores how the topics of number systems and data representation can be taught in a way that piques curiosity and captures learners’ imaginations. Specifically, we present the design of two interactive physical computing artefacts, which we collectively call DataMoves, that enable students, 12-14 years old, to explore number systems and data through embodied movement and dance. Our evaluation of DataMoves, used in tandem with other pedagogical methods, demonstrates that the form of embodied, exploration-based learning adopted has much potential for deepening students’ understandings of computing topics, as well as for shaping positive perceptions of topics that are traditionally considered boring and dull.