DIS2020

DIS 2020 Best Demo Honourable Mention

In this research, we investigate how computer vision and paper can be used to make inexpensive and functional tangible interfaces. Paper is an attractive material as it is a democratic and versatile medium for making. We printed ArUco fiducial markers on paper, and explored how different ways of manipulating the sheet of paper can construct and deconstruct the printed marker. For example, we can fold a strip of paper into an origami push button. Pressing down on this button brings together two parts of a marker, and releasing the button separates the marker halves. In this case, we can use the detection of a marker to know when a button has been pressed. Other types of paper manipulations include printing markers over stretchable kirigami patterns, or using the properties of paper fibers to reveal and conceal markers through wetting or lighting. We translated the insights from our exploration into a series of applications, including a series of interactive data physicalizations. For instance, we made a 3D bar chart, where pressing on a bar will trigger an audio recording with more information about the selected data point.

Who is the target audience and why design for them? We hope to support industrial designers and STEM educators through this work. Prototyping functional tangible interfaces is a complex task that typically involves complicated electromechanical assemblies. Through this research, we propose an economic alternative to rapidly build functional tangible interfaces with paper and computer vision. We imagine that industrial designers can quickly explore the layout of inputs while designing a physical interface. Also, since this technique uses paper, designers can also experiment with different colors, “materials”, and finishings by printing graphics and textures. For these same benefits, we also think that Printed Paper Markers can facilitate STEM learning. For instance, interactive pop-up books or data physicalizations that support tangible interactions.

What were the challenges or limitations encountered in this project? The computer vision (CV) system for Printed Paper Markers is sensitive to its environment. For instance, we had to vary the ArUco marker detection settings to cater to the lighting conditions of a specific context. To mitigate the sensitivity of the CV system, we used infrared (IR) cameras with built-in IR LEDs for illumination. This made CV much more stable; albeit at a higher hardware cost.

What are the opportunities and next steps for this project? We are currently working on a version of Printed Paper Markers that works with a mobile device (e.g. smartphones and tablets). We believe the improved portability with mobile devices will expedite deploying Printed Paper Markers in classrooms and design studios.

To the Demo Visitors: We welcome all feedback! In addition, we are actively seeking people to collaborate on this project. The fabrication files of our explorations are embedded in the pictorial itself, for anyone who wants to try it out for themselves.