Reflection-stimulating data collection instruments
Engaging design for utilisation in participatory research
engaging design, participatory science
Abstract
The scientific study WaterCounts explores reflexive data collection using specially developed ambiguous counting tools. The study involves citizens on the island of Orø in measuring their water consumption as part of a larger project on sustainability and pluralistic understandings of knowledge. Instead of conventional measuring tools that can create bias (known as Maslow’s hammer), WaterCounts focuses on developing tools that actively involve users and invite qualitative reflection. The design process has aimed to create something playful rather than a game, where the user is seen in the roles of operator, performer, and spectator. Three experimental design drafts have been developed: a phone app, a counting wand, and a pocketmod, with a focus on being tangible and open to interpretation. The counting wand has generated limited interest, while the pocketmod has sparked interesting conversations, but raises challenges regarding data digitisation. There is assumed to be potential for intentional ambiguity in the measurements across all three designs, but validating this requires further investigation.
ACM Classification Keywords
Human-centered computing—Human computer interaction (HCI)—Empirical studies in HCI;500
Hardware—Emerging technologies—Analysis and design of emerging devices and systems—Emerging tools and methodologies;100
Hardware—Communication hardware, interfaces and storage—Tactile and hand-based interfaces;300
Introduction
For the systematic collection of data of a known type, known, well-tested tools can advantageously be used. If not only data, but also the type of data, is investigated, tools targeted at specific types of data can, however, create bias. This issue is known as Maslow’s hammer: [I]f the only tool you have is a hammer, [it is tempting] to treat everything as if it were a nail
[6:15].
For the scientific study WaterCounts, which at the time of writing is under development, the choice of measurement tools is central. The study involves citizens on the island of Orø in measuring their private water consumption as part of a larger research project, AquaSavvy, on ecological and social sustainability, agency and parallel understandings of knowledge (epistemic pluralism) [1].
If the purpose of WaterCounts were merely to collect quantitative data (nails for Maslow’s hammer), the citizens could be equipped with click counters, or the measurement could be further automated electronically (Maslow’s hammer). As part of the AquaSavvy project, however, not only the factual figures are relevant: Also essential is the participants’ ownership of and autonomy over the created knowledge (agency) and their subjective understanding of values in water and water consumption (epistemological pluralism). The WaterCount study therefore requires that the measurements make room for new insights – meaning that the measurement methods do not dominate the data collection process, but actively involve the users and invite qualitative reflection on the process as well.
When designing WaterCounts existing conventional water measurement techniques were deemed too goal-oriented, because there was a need here for a more exploratory, playful approach: A design of counting tools, which to a lesser extent have embedded prior knowledge of what is to be measured, and to a greater extent involve the users of the measuring tools in how it is measured, and what the measurement can be used for. For example, the measurement could include how important it was to have a bath on that particular day, or a tenant might assess that water used for personal hygiene weighs (or takes up space, in subjective volume) differently than a garden watering routine dictated by the landlord.
Here follows a documentation of the design process done in WaterCount for experimental design drafts of tools for the quantitative measurement of water consumption, which are also user-involving and -reflective.
Theoretical foundation
Design processes involve iterative processes, where in early iterations primarily sketches are created and gradually increasingly prototypes [2:138–140]. Sketches are not limited to static constellations, but can also be used to express interactive functionalities [2:135–136].
Within experience-oriented interaction design in particular, the user’s role can be understood as three simultaneous roles – operator, performer and spectator [3:9–10].
Bill Gaver et al. point out that ambiguous information can encourage people to participate in the work of making sense of a system and its context [5:237]. Information ambiguity can be used as a deliberate method: By deliberately obscuring selected information, it opens up possible alternative interpretations of the chosen area [5:236].
In a later work, Gaver distinguishes between play and sport, where play is characterised by being exploratory, and unlike sport, not being goal- and rule-oriented [4:166–168]. Such playful designs require, according to Gaver, both one’s own personal reflections and user involvement in the design process [4:173].
Approach
An overall objective in the design process has been to create something that is playful and not a game. That is, to avoid Maslow’s hammer: Counting methods which, regardless of their effectiveness, limit opportunities for alternative understandings of what is being counted.
The user – or the operator-performer-spectator – is here the participant in the participatory research study WaterCount, and the method (design strategy) primarily used is Gaver’s encouragement to actively goof around[4:173] and the involvement of users to help with this out-of-the-box thinking/playing, to break free from conventional thinking and establish a playful space.
The tripartite roles of the participants
The operator-performer-spectator model helps to see the possibilities for supporting reflective participation, when translated into a research context: Participants in engaging participatory research can be viewed as having the three simultaneous roles of data collector, knowledge creator and knowledge user. Or, in the same structure but formulated from an ownership and agency perspective, with the roles of author, communicator and reader.
A participant, as a data collector/author, can be motivated if the task offers the possibility for creativity, for example, the opportunity to record during which baths songs were sung, and may have an interest in how data is used or misused.
The role of communicator and knowledge creator can trigger reflections on the materiality of the knowledge that is established.
The role of knowledge user/reader can become more reflective and critical in the dialogue with the other roles.
Ambiguity and play
Convention dictates that water is a chemical substance, and its consumption is measured in litres. By equipping the data collector not with a precise litre measure but with a more diffuse counting tool, and perhaps also with the possibility to annotate measurements with subjective notes, the aim is to establish the more complex roleplay collector-creator-user/author-communicator-reader with its invitation to deeper reflection on the roles and what they deal with, which here is the understanding of what knowledge consuming water entails.
Validation
The intended effect of ambiguity in the measurements will be tested through semi-structured interviews among participants and the subsequent analysis of these interviews.
Developed designs
A number of requirements were set for the counting tool, where the most significant were that it must not do anything hidden (for example, not exchange data without explicit consent), must be tangible, and must be able to count four frequent types of water consumption: toilet flushes, showers, laundry and washing up. These consumption types are only meant as inspiration for the design process, not limiting for the subsequent study, as one of the objectives of the study is precisely to keep the tool open to interpretation of its use. For the same reason, it has not been determined either, how, for example, two toilet flushes
and “a medium-length shower translate into water volumes.
After brainstorming ideas for realisable tools within the requirements three ideas were selected to develop further: phone app, counting wand and pocketmod.
The idea was to refine the development of at least two tools, and continuously involve Orø residents in critical reflections on the usability of the different designs.
Phone app
The idea with the phone app is that many already have a mobile phone and are comfortable using it, so it will be relatively easy to add an app with the counting routines that are needed.
A short, semi-structured interview with a neighbour indicated the phone app as the favourite, based on some of the same arguments found in the earlier idea phase.
Pocketmod
A pocketmod is a small 8-page book, made by folding and slitting a single A4 sheet.
The idea behind this counting technique is that it is completely analogue, which in principle is simple, but raises a central question: The collected data must be digitised for further use, so when the collection process is analogue, there is an additional need for the design of a post-processing routine.
Counting wand
The idea behind the counting wand is to challenge the understanding of serious science, by physically designing the data collection tool as and to some extent using it like a magic wand: A microprocessor, a battery and some sensors mounted in a plastic tube or, as in the documented prototype photographed below, rolled into a sheet of paper or a plastic sleeve and secured with gaffer tape.
By making the counting wand a DIY kit it provokes further by demystifying both technology and magic: Is a magic wand magical if you have rolled it yourself? Is a microcomputer an alienating gadget, if you have snapped it together yourself?
Reflections
The motivation behind the requirement for no hidden operations was that the tool should not be perceived as having autonomy. Central to mobile phones is, obviously, that they are mobile, which implies that in their normal state they are constantly exchanging data over networks. A mobile phone will therefore typically be perceived as an autonomous system, regardless of the fact that the individual app can technically be programmed to only perform activities directly triggered by the user themselves.
In the early design phases, many ideas emerged for the counting wand, with an idea that families with children in particular might be interested in this counting tool. Conversations with Orø residents, partly in the form of semi-structured interviews and partly as informal meetings at the local café and during ferry crossings, have however not shown much interest in the wand, and the technical work involved in getting it to practically function may prove too high in relation to the limited interest in it.
The analogue pocketmod has sparked interesting conversations with Orø residents, unfortunately only at informal meetings. There is hope of capturing some of these reflections during future semi-structured interviews, and there is anticipation for further iterations and the development of a technique for digitisation.
The pocketmod, with its analogue format, is naturally ambiguous about what is to be measured, whereas this is possible but technically more extensive to implement – for example through a dictaphone function – with a counting wand or mobile app. Ambiguity is not unequivocally positive either: Water consumption from a washing-up session recorded as hand-drawn jellyfish can be difficult to post-process, so that it can be meaningfully and robustly used quantitatively as well.
At the time of writing, only one has been conducted of the planned semi-structured interviews, which is not a sufficient basis to be able to validate whether the intention behind the developed designs – an experience of ambiguity regarding data collection and the established knowledge – has been achieved.
Conclusion
This study has documented the design process for three experimental, reflection-motivating data collection tools for the WaterCounts project, with the overall aim of avoiding Maslow’s hammer and instead establishing a playful, exploratory approach to citizen-based water consumption measurement.
The three developed designs – phone app, counting wand and pocketmod – meet the established requirements of tangibility, explicit data sharing and openness to interpretation, but with fundamentally different results during the initial user-involved reflections. The counting wand has a potential to demystify technology through its build-it-yourself format, but has shown limited interest among Orø residents. The pocketmod, on the other hand, has sparked a number of qualitative conversations and reflections, but introduces a significant challenge: it lacks a final translation into digital form.
The operator-performer-spectator model has proven useful for understanding the participants’ potential roles as data collectors, knowledge creators and knowledge users, but it requires further investigation to validate whether the intentional ambiguity embedded in these designs actually has an effect.
Credits
Thanks to neighbours, other Orø residents and fellow students at RUC, who have all contributed with constructive reflections throughout the design process.
References
Footnotes
Attribution: Wesha, licensed under CC BY-SA 3.0, via Wikimedia Commons↩︎
