Interaction Design Assistant Professor Audrey Desjardins has been busy with two research projects since 2019. She recently received a National Science Foundation grant that allows her to begin a third project. Below is her summary of what she and her collaborators have achieved so far and what is yet to come.
(In)Visible Data: How Home Dwellers Engage with Domestic IoT Data
Funded by Mozilla Corporation.
We looked at the data associated with Internet of Things (IoT) devices in the home. We found that everyday relationships between home dwellers and domestic IoT data often remain secondary interactions, preventing deeper understanding and awareness of data tracked in the home. By conducting design ethnography and design inquiry with ten individuals, we looked at the various ways data and humans are entangled in the home. As a conclusion, we argue that data does not have to be laborious, tidy, or the byproduct of a service but rather can be lively and affecting. We further suggest new modes of engagement with data that expand or step away from self-improvement and reflection: through diverse acts of noticing, by allowing data to remain invisible, and by embracing imaginative practices.
I am currently working on a follow-up project where we are designing and fabricating three artifacts that will allow people to engage differently with their data in the home. The artifacts are building on the idea of (1) data being central to everyday performances, (2) data having its own life in and outside the home, and (3) finding diffuse ways of noticing, or noticing data more subtly. Artifact fabrication has slowed down due to COVID-19, but once the artifacts are made, we will deploy them with participants who will live with them for a few weeks or months. We will then analyze the participants' experience of living with the artifacts.
Publications
- Desjardins, A., and Key, C. (2020). "Parallels, Tangents, and Loops: Reflections on the 'Through' Part of RtD," In Proc. DIS'20 (pictorial), New York, ACM Press.
- Desjardins, A., Biggs, H., Key, C., Viny, J.E. (2020). "IoT Data in the Home: Observing Entanglements and Drawing New Encounters." In Proc. CHI'20, New York, ACM Press.
Collaborators
- Heidi Biggs (MDes 2019)
- Miki Bin (MHCID 2020)
- Cayla Key (IxD BDes 2019)
- Netty Lim (IxD BDes 2019)
- Jena McWirther (IxD BDes 2021)
- Ruby Peven (VCD BDes 2021)
- Jack Sinclair (IxD BDes 2020)
- Jeremy Viny (MDes 2020)
- Philbert Widjaja (VCD BDes 2021)
Voices and Voids: Reclaiming and Transcoding Voice Interaction Data as Performance
Funded by a UW Mellon Faculty Fellowship in the Arts.
In this work, we examine the invisible ubiquity of voice assistants — the Alexas and Siris of the world — and of the data logs they produce in the form of long lists of voice commands and questions. Our minds and bodies have become entangled in a complex assemblage of information, technology, and devices. We allow artificial intelligence agents to listen to our most private conversations in our most intimate spaces, forgetting or ignoring the fact that much of this personal data is stored indefinitely. We reclaim, examine, and ultimately transcode this data through an interdisciplinary performance project. We have created performative works using a combination of design, data-driven art, digital crafting, percussion, and performance to defamiliarize encounters with virtual assistants.
By combining a series of performative vignettes that utilize various media and form, we are able to create a complex, disparate, chaotic, playfully dystopian experience for the viewer. The vignettes include a sound poetry duet with Alexa, movement scores responding to voice assistant requests, typographic compositions exploring the archives of data, neural net poetry, and voice and sound explorations of training data for a voice assistant. The non-linearity of the vignettes allows us and the spectator to think differently about the data, both in content (commands, questions, thoughts, requests, etc.) and in the form (voice, background noises, music, etc.).
At the moment, we are working with a web developer to create a website that will host an adaptation of our vignettes on the web as a net art project. This website will include a series of vignettes and pages that can be visited to see the many facets of the project, painting a diverse, yet interconnected, picture of the many ways we might reclaim, transcode, and engage with data logs from voice assistants. In the next few months, we are tentatively planning an online interactive performance using a platform like Zoom. In late autumn 2020, we are carefully planning for an in-person performance, dependent on COVID-19 restrictions.
Publications + Workshops
- Desjardins, A., Psarra, A., and Whiting, B. (2020). "Everyday Voices and Voids: Reclaiming and Transcoding Voice Interaction Data as Performance." In Proc. ISEA'20.
- ZKM and Solitude Web Residency on "Engineering Care." Our work was shortlisted for this web residency organized by ZKM and curated by Daphne Dragona.
- Adversarial Hacking in the Age of AI, at Transmediale 2020: Voices and Voids was accepted as a contribution to this workshop at the 2020 Transmediale Art and Digital Culture festival in Berlin, where we presented a short talk and live performance.
Collaborators
- Afroditi Psarra (Assistant Professor, DXARTS)
- Bonnie Whiting (Assistant Professor, Music)
- Esteban Yosef Agosin (DXARTS)
- Gabrielle Benabdallah (HCDE)
- Darcy Copeland (Music)
- Jonathan Rodriguez (Music)
- James Wenlock (DXARTS)
Artists in Residence
Funded through the Mellon Fellowship.
- Danny Clay
- Laura Devendorf (UC Boulder)
- Kate Sicchio (Virginia Commonwealth University)
- Jordan Wirfs Brock (UC Boulder)
- Yvonne Wu (Beloit College)
WEB DEVELOPMENT
- Amanda Yeh
Exploring IoT Data Transparency in the Home through Creative Data Representations
Funded by the National Science Foundation.
This project is about using a radically different approach to address data transparency for connected devices (Internet of Things, IoT) in the home. The data these artifacts capture (how many times a door is opened, the questions asked to an Alexa or Google voice assistant, etc.) is often left hidden away from the people generating them at home. By not engaging with the large archives of data they create, these home dwellers are also not making informed decisions about the ways large corporations benefit from their data. In this work, I will develop new techniques for engagement, privacy, and transparency centered in creative representations of people’s domestic IoT data. To go beyond current data visualization practices, I will collaborate with artists to create representations translating domestic data logs into (a) 3D data materializations printed in ceramic and (b) data narratives in the form of short stories. Printed ceramics will build on the fact that touching and manipulating data can be a generative sensemaking activity. Short stories will capitalize on the ways people understand complex and abstract concepts through data narratives.
As this project is funded for September 2020 through roughly August 2022, there are not yet publications or collaborators.