Description:

LightSense introduces a new generation of responsive architecture. It invites the audience to engage in intimate and meaningful interaction with a living architectural body.

The kinetic construction combines a lightweight structure with 3D holographic animations of its digital twin. At the core of the installation sits a neural network, that facilitates verbal communication with the audience. Trained with 60,000 poems that provide the vocabulary, syntax, and context, the AI is programmed to actively engage, lead and sustain conversations. LightSense inspires the audience to engage in direct communication about topics we care about. Its responses are truly associative, unpredictable, meaningful, magical and deeply emotional. LightSense listens with interest and curiosity before responding from a personal viewpoint. It offers opinion and an emotional interpretation of the dialogue which in turn triggers emotional and perceptive responses in the audience. Analysing the emotional tenor of this conversation, the system can transform into a series of architectural volumes and immerse the visitors in Pavilions of Love, Anger, Curiosity, and Joy.

The 12 meter long kinetic construction is assembled out of two aluminum space frames. Each frame is used to tension a transparent ‘holo-mesh’, creating an invisible projection screen. Joined with a rotational axis and suspended from the ceiling the structure is held in a delicate horizontal balance. A bird-like motion of the structure can be initiated through physical touch by the visitor or by a computer-controlled motor, moving two 5kg balancing weights along a rail on the underside of each wing.

A further novel component is the rendering principle that allows the projection of stereoscopic images on a moving and folding target screen. The 3D holographic images are precisely calibrated to superimpose both, the kinetic aluminum structure and the surrounding spatial environment.

With a multi-audience setting in mind, the installation takes advantage of an anaglyph projection principle, which allows effortless stereoscopic viewing by using simple 3D cardboard glasses with a red/cyan filter.

Visitors can communicate with LightSense’s AI system through an onsite microphone. A Generative Language Model produces a response in the form of a poem and maps the interaction with the user against emotional proximity values. These values are used to create dynamic audio-visual animations.

The physical structure and the digital duplicate (twin) are interlinked through a position sensor. Both the physical and digital behaviour of the structure are interdependent and controlled by the gaming engine Unity and the AI system.

It is equally possible to link LightSense’s digital twin to an online communication platform such as Zoom. Effectively, this allows the installation to become independent of its location and enables visitors from around the world to interact with the physical construction.

LightSense takes a new human-centric approach to the current notion of AI systems which are typically used for optimization or automation processes to increase efficiency or substitute human action. Instead, it explores how Creative AI can be used as a new design parameter to generate responsive architecture that can actively collaborate with its inhabitants and improvise alongside their actions.