Instrumented Spaces: Measuring Engagement, 2014

Measuring Engagement was a 2014 study by the Iota team, a research group at the agency SapientNitro in Chicago. It was a team effort, and an internal one: to find, in the words of the team’s paper, “more reliable means of measuring the nebulous concept of engagement”. TJ: “We were often thought of as web analytics for the real world.”

What an engaging moment is

In On Engagement, an engaging moment is time, effort or attention that a person gives to an interaction with another person, a place or a thing. Engagement is the impact of those moments on the person’s life over time.

The hypothesis

The team’s documents state it in three parts.

1. Engagement can be studied more effectively as an outcome once something measurable has been found in moments people agree are engaging. With those measures, the conditions and outcomes around the moments can be looked at closely, without self-report or brain scanners.
2. A moment can be recognised. It starts with an abrupt shift in one or more indicators of attention, focused interaction most likely follows, and something has to change. If everything returns to how it was, it was probably not engaging.
3. Engagement itself is a lasting relationship, built from many such moments over time. The team meant to look at this more closely.

The plan

The team’s plan of April 2014, “Instrumenting Consumer Engagement, part 1”, took everyday errands as its scenarios: breakfast at Specialty’s, an afternoon coffee at Intelligentsia, shopping for a belt, and finding a carburetor repair online. For each it set out instruments (Glass video, Fitbit, Basis Band, Moves, SmartThings proximity, RescueTime), an observer watching and recording, an interview afterwards, and possible signs of engagement, such as looking at a menu or talking with the person behind the counter. A sheet coded the breakfast trip by hand, moment by moment, as engaged or not, from two videos side by side.

TJ, on the method: “In exploring hypotheses, METIS and other techniques were used to bridge and map evidence of activities to sensing.” METIS was the team’s way of breaking a high-level concept down into things that can be measured.

The proof of concept

One member, TJ McLeish, walked from the office to a coffee shop and back, wearing Google Glass, a computer worn like spectacles, while a colleague followed with a second camera for richer context. Software written for the trip logged Glass’s sensors. The team drew the trip as one figure. Combining its streams and the frames, it picked out three possible moments: locating a co-worker, noticing a food truck and ordering coffee. Its slides call this a short proof of concept that is not definitive. TJ: “This particular study is more illustration than scientific; it is a provocation.”

This page

The files hold two trips. The video and its sound are from 3 April 2014. The sensor readings are from a repeat trip on 15 May, and none from 3 April have been found. This page rebuilds the team’s figure from those files, and opens on the whole walk as the figure did. The video gives the path and the head, and the readings are fitted onto that path. That makes it an illustration, and how the walk is drawn gives the detail.

Using the page

Space plays and pauses. Click a graph to set the time, scroll on it to zoom and drag to pan. Drag to turn the view, and scroll to zoom. Zoom in on the Monadnock, or use the magnifier, and its revised model fades in. The eye button shows the model from Glass, to set beside the clip, and the chase camera follows from behind. Path switches to the team’s 2014 curve.

The documents

• April 2014. “Instrumenting Consumer Engagement, part 1”, the team’s plan, a workbook.
• July 2014. “Measuring Engagement”, a draft, and the earliest document found that writes out the hypothesis.
• 2015. Rick E. Robinson and Donald Chesnut, On Engagement: Toward a More Reliable Measurement of a Nebulous Concept, SapientNitro.
• 2015. Kai Hansen and Thomas McLeish, “Designing Real Time Sense and Response Environments through UX Research”, eCAADe 33, Vienna, volume 2. Its section “Measuring Engagement” is the source of the study’s name.
• 2016. Kai Hansen, “Designing responsive environments through User Experience research”, International Journal of Architectural Computing 14(4), 372–385, doi:10.1177/1478077116670745.

Reports

How the walk is drawn, with four appendices: the model of the office, the model of the lobby, the model of the Monadnock Building and placing the Flamingo sculpture.

The study, the walk and its instrumentation were a team effort by Iota, at SapientNitro. This rebuild was made in 2026 at TJ’s direction, with Claude Code. Alexander Calder’s Flamingo (1974) is drawn from “The Flamingo” by BoldlyBuilding, licensed under CC BY 4.0.