Screen Motion

Concept
Screen Motion is an exploration of how a narrative story can be controlled by a viewer in a physical space. We wanted to explore semi-passive interactions by investigating ways that a viewers movement could trigger changes in the narrative elements of a story, while keeping the familiarity of a more traditional film watching experience.

From a storytelling perspective, we wanted to explore how to make these kinds of interactions make sense, by thinking about how story elements would intensify or cool-off based on the viewer’s physical relationship to the screen. By adding this kind of interaction, each user has a unique path through the story. When thinking about narrative content for this kind of interaction, a story tree is developed that covers the different ways that the story branches, similar to the way that a narrative for a video game would be developed.

Story Design
For the prototype, we chose to film a conversation between two people because it would make sense that the emotions would change rapidly as each person responds to the previous statement. Each shot plays out for its full duration, regardless of the viewers movement, then at the point of the cut or edit, the next segment’s emotion is chosen as a reaction to where the viewer is standing at the time of the cut. For the simplicity in prototyping, we decided to work with a story that has 4 paths of identical action that happen simultaneously, but with a different emotional context. In our prototype, the four paths included anger, flirtation, sadness, and happiness. Each emotion corresponds with one of 4 interaction zones defined by their proximity to the screen.

Interaction Tech
The interaction space is broken up into multiple zones. Various kinds of sensors can be used to trigger the interaction. For the prototype, we used a depth camera to sense the viewer in the space. As a viewer moves between zones, it triggers the software to make a change in the content that will be shown. There can be a multiple number of zones in different configurations, that can be altered based on the viewers needs. For our prototype we broke the interaction space into 4 zones that were assigned a particular emotion in the content.

Luxloop won Best Hack in Film/TV/Video at MIT Media Lab’s Hacking Arts 2014