Hito Steyerl: Actual Reality OS

Using data about wealth, social housing and austerity collected with local research partners and community groups the app shows the Serpentine Galleries shaped and abstracted by the actual forces of its social environment, charting real-life inequality through virtual means. This warped simulacrum of the physical building is designed to restore a social vision to an often unseen stark class reality, demanding the visitor leave their comfort zone and question their surroundings.

Actual RealityOS is an open source digital tool for data visualisation that brings together augmented reality, immersive audio and strategies of data collection and mapping for mobile devices. The app considers the symbiotic relationship between technology and power. Nowhere is this relationship more clear than in the decision making power imbued within technologies like artificial intelligence and predictive modelling now used to ascertain access to services like insurance, housing and social benefits.

Core to these technologies is their use of data and machine learning as a, perceived, neutral means of assessment when data, in its mediation and representation, is political and often reflects the inherent biases of societal power structures. Actual RealityOS uses the potential of augmented reality to create a tool for visualising otherwise unseen information: the nascent immersive technology allows users to see a real-time composite of virtual imagery against their physical environment through the camera of a phone or tablet. Focusing on the UK and, more specifically, the Serpentine’s location in the Royal Borough of Kensington and Chelsea, the app exposes inequality data – modelled, sonified and as testimonial – concerning wealth, income, social housing, domestic and hospitality workers’ rights, and the impact of austerity measures.

The architecture of the Serpentine Sackler Gallery becomes the base metric onto which the data is mapped to the external facade of the building by overlaying, in augmented reality, a warped virtual simulacra that charts the stark reality of inequality at 1:1 scale. The building is surrounded by a digital perimeter of personal testimony in contrast to the abstraction of statistical representation while a data-driven soundtrack guides you. The audio is produced through a process of data sonification in which statistics are represented as sound.

Created by Jules Laplace and Bethany Barrett using an automated tool that maps data points to harmonic scales which are then played through a Behringer DeepMind synthesiser, the inclusion of sound augments reality in another, non-visual, form that presents another perspective of both data analysis and social experience

Featured on:
The Guardian
Art Net
Plinth

Graphic Design, Typography, AR Design and Production:
Ayham Ghraowi with Matt Wolff, Hrefna Sigurðardóttir, and Ben Ganz

AR development and modelling:
Ivaylo Getov (Luxloop) with Caco Peguero (United Futures) and Werkflow

Production and data set collection:
Serpentine Galleries

Music
Jules Laplace, Bethany Barrett

Production coordinator:
Hanna Mattes

Commissioned by the Serpentine Galleries
Kay Watson, Digital Curator

Research Partners
Architects for Social Housing
Disabled People Against Cuts
The Voice of Domestic Workers
Constantine Gras.

Hito Steyerl project is supported by the Luma Foundation. Additional support by Yanghyun Foundation, In Between Art Film, The Reuben Foundation, and Goethe Institut.

Digital engagement supported by Bloomberg Philanthropies. The Serpentine Galleries is supported using public funding by Arts Council England.