🦅 eagle eyes

blink and you will miss it

Nov 16, 2024

Paramount recently released the first 7 minutes of SMILE 2 online, but you can only watch it if you smile into your webcam for the ENTIRE time. On Twitter, Puneet Singh observed that it’s “an insane new level of data collecting.” It's a cute and au courant gimmick - 420 seconds of an exaggerated expression in exchange for a sneak peek at a gruesome horror film.

It seems unlikely that this obvious marketing tactic was actually designed to steal the likeness of fans, but only time (and the terms and conditions) can tell how innately trusting fans will be treated by Paramount and its service providers. And while there is speculation on Reddit that the images may be used further for the film’s marketing, the reality is that we are used to passively trading information about ourselves - like our geography, purchase history, and demographics - with companies.

This particular gimmick, however, deserves closer consideration as it reminds us that our expressions are a rich source of data. For starters, we are increasingly able to control technology with our facial expressions - mobile games like Rainbrow and Nose Zone use facial movements to control game play. These activities are benign, but they add to a corpus of research and program development that allows computers to get progressively better at anticipating and interpreting our expressions. Announced this past spring, Apple’s Eye Tracking feature allows people to control their iPad or iPhone with their eyes. Remember when Zoom announced they were developing a system to scan users’ faces and listen to their speech in order to determine their emotions? Cameras allow companies to capture imperceptible impressions from us about our mood and how we respond to ads and design decisions without us even realising.

Some of this facial monitoring can be legitimately helpful. For instance, the technology is used in advanced driver-assistance systems (ADAS) to monitor driver attention, detect drowsiness, and improve safety. It can also be used to diagnose neurological and psychiatric conditions, and making the world more accessible to more people is crucial. Eye-tracking tech is sometimes deployed during the remote proctoring of tests in an effort to detect and deter cheating, and in gaming, augmented reality headsets rely on eye tracking to allow us to engage in the metaverse. It’s here, it’s. happening.

But not all firms aren’t analysing how we feel and where we gaze just to be polite or help us have fun. They want to proxy our emotions and reactions in order to better understand physiological responses to advertising. Recall that when Google acquired Fitbit, the European Commission made the merger conditional on a ‘data wall' that would separate health and fitness data from advertising purposes. Without this kind of separation, the digital giant might be able to detect a change in pulse after viewing ads. But that measure hasn’t stopped firms from trying to understand our reactions to content and ads through other mechanisms.

Our passive expressions are being watched when we shop, and sometimes when we veg at home on the couch.

Our ‘smart’ televisions already watch us. A recent report from the Center for Digital Democracy detailed how our television sets have been transformed into a sophisticated monitoring, tracking, and targeting device; urging the FTC and FCC to investigate connected TV industry data harvesting. A system from Caltech and Disney Research uses a facial expression tracking neural network to learn and predict how an audience will react during testing. Facial biometrics are already becoming a part of just participating in an audience.

This application of surveillance systems allows advertisers to ‘optimise’ ad placements. Companies like Sticky and Realeyes use eye-tracking to measure emotional responses to advertisements. Similar companies like EyeSee and ShopperMX use eye tracking and emotion recognition to derive insights on shopper behaviour. To some, this may seem benign, but for others, it comes as a surprise.

It shouldn’t. Longos sponsored a research lab at the University of Guelph where researchers constructed a simulated store environment that used eye-tracking technology to study shoppers’ visual focus in an effort to gain insights into consumer decision-making in 2018. And eye tracking studies are increasingly commonplace, but in focus groups with consent. Often, companies review camera footage from self-checkout kiosks to study interface UX effectiveness and ads. Grocers like Loblaw and Pattison Food Group work with companies like Eagle Eye, which recently announced an AI-powered platform for retailers that tracks responses to promotions. Earlier this year, American grocer Kroger denied that they would be using facial recognition to influence pricing decisions and last year, Australians balked at front-facing cameras at Coles and Woolworths.

We are used to surveillance in commercial contexts being explicit - bulky CCTV cameras and warnings that ‘you are on camera!’ But we haven’t deeply considered whether the passive and near-constant participation in data-surveillant marketing practices is a ‘fair term’ of the exchange. A camera can determine whether we scanned or purchased an item, but that security-based bargain is fundamentally different from attempting to capture our innermost feelings in the moment.

Walmart recently added body cameras for staff to wear. This commercialization of defence technology is not a good sign(!), and regulators are catching on. Quebec’s biometric bill requires that organisations must notify the government of the use of biometric processes at least 60 days prior to the creation of a biometric database. As a result, firms are grappling with how to comply or whether they should simply leave Quebec’s market out of their eye-tracking plans. That said, how different is tracking someone’s gaze from monitoring their mouse strokes?

If surrendering the many nuances conveyed by our faces is a new part of participating in an audience or engaging in the economy, it’s time to push back against that greedy violation. We can still turn that frown upside down (*for more than seven minutes).

🎙️ The Globe and Mail podcast that I host, Lately, will be back for a third season in January. In the meantime, you can catch up on the conversations we had this fall:

🎙️ The CIGI podcast that I co-host, Policy Prompt - all about long-form interviews on transformative technology and policy stuff - also has some great content to catch up on.

How every computer is a Chinese computer (Thomas Mullaney)
Siri, tell us about human rights and robot wrongs (Susie Alegre)

*Rizzle Kicks has a new album coming out called Competition is for Losers. Here’s one of their singles.

regs to riches

Discussion about this post