Zubr and Uninvited Guests are excited to introduce Future Places Toolkit, a MyWorld Challenge Call project led by Digital Catapult. This will offer a novel approach to architectural planning consultations, making use of real-time generated content through a hybrid blend of augmented reality (AR) and virtual reality (VR). Our system will enable audiences to participate in geospatial AR drawing sessions, allowing discussions with communities and stakeholders to take place in-situ/where a development is proposed. Using AR, guided conversations, and live 3D sketching, people see their ideas and improvements to the plans visualised immediately around them.
Who are we?
Our project is led by Zubr.co, which is a specialist AR and VR development company based in Bristol. We’re proud to have completed over 300 immersive projects since we formed in 2015, with a strong focus on the arts, heritage and policymaking sectors. Within the last couple of years, we’ve made a beautiful AR app with Soap Soup Theatre that allows children to bring their own imaginary characters to life, an ambitious AR experience that restores the Acropolis of Athens to its former glory, and created a vast VR ‘Museum of the Future’ with the MOD’s Defence Science and Technology Laboratory. Future Places Toolkit is an arts and science venture which sits perfectly at the intersection of the sectors we’re most active in, so it presents a fantastic opportunity for us to collaborate and share with our many industry partners.
Uninvited Guests is a theatre company formed in Bristol in 1998 and have created 25+ works that have toured internationally and to major UK institutions. Their recent performance work uses creative technologies to involve audiences, reflects on the present through the lens of the future, and develops new approaches to immersive technologies in performance, such as the award-winning interactive visitor experience for Historic Royal Palaces, The Lost Palace. As residents at Watershed’s Pervasive Media Studio, they design interactive experiences that integrate emerging technologies, apps and locative devices. For this project they are collaborating with award winning artist Duncan Speakman (winner of Best Immersive XR at BFI London Film Festival and Special Jury Award for Immersive non-fiction at IDFA).
The Future Places Toolkit Demonstrator
Our work on this project is built on top of our existing Demonstrator of the Future Places Toolkit concept. Uninvited Guests first developed a version of this many years ago, called Billennium, before collaborating with Zubr.
Billennium has a long history of development, beginning as a site specific theatre performance called ‘Give Me Back My Broken Night’ (commissioned by Watershed/Theatre Sandbox). In that work, audiences were taken on a guided tour of the future of their city, and then offered a chance to design their imagined or preferred future. Using Pico video projectors hung around their necks, participants would see real-time drawings of their imaginations projected onto a blank paper map held in front of them. In 2018, ‘Billennium’ was developed as a new approach to this project, using Augmented Reality rather than projectors. Created in collaboration with Fenyce, this version allowed participants to see the recounted fictions and their own imaginings overlaid directly on to the physical environment. Limitations of the technology at this point meant that the artist was working primarily in 2D, their images only fully aligned with the environment from certain viewing points.
As this project toured it became apparent through user and stakeholder feedback that it had potential not only as a performative exploration of speculative futures, but as something that could become an innovative tool in actual planning processes, opening them up to more collaborative, transparent and playful techniques. It is from this inspiration that the Future Places Toolkit concept has developed.
How does it work?
The Future Places Toolkit will consist of an intuitive AR application, coupled with a Unity Editor plugin. A session will involve multiple mobile devices running the AR app, connected to a remote PC running the Unity plugin. A professional artist will be using a VR headset with the PC to create 3D visuals in real-time, based on the ideas and feedback of the participants. Participants witness the transformation of the space in front of their eyes, with plans both proposed by the architects and ideated by themselves. Existing buildings can be replaced or enhanced; green spaces created and expanded, and roads transformed into cycle paths – whilst a full digital record of every action is saved for future reference. A selection of powerful tools will be available to the artist, thanks to the native Unity Editor interface – coupled with our custom VR painting tools, which allows anyone to create models without any experience of 3D modelling software. We are using VR because it is the ideal format for a 3D sculpting process, in a way which simply can’t be achieved on a 2D screen.
All 3D objects available to the artist must be created with the VR sculpting tool, which allows us to maintain control over the aesthetic and technical quality of the assets. In short, this ensures that all 3D visualisations adhere to a ‘sketcherly’ appearance, a world away from the harsh technical appearance of inaccessible CAD files; they are also stored as vertex points which requires minimal memory.
We’re designing these tools to be implemented optionally alongside the Google Geospatial API, which allows the live artist in VR to create AR layers on top of practically any real urban environment in the world!
The goal is to create a toolkit which can deliver a powerful and engaging experience to audiences within consultation processes. The toolkit will be easy to use for third parties, extremely flexible for different use cases, and has the potential to facilitate a level of stakeholder engagement which far surpasses what is currently achievable in the architectural planning sector.
Our recent progress
We have almost completed building the artist’s tools and interface for our Unity plugin, and have been inviting artists to visit the Zubr studio for testing. We have overcome some interesting novel technical challenges, including the creation of an algorithm for dynamically filling in hand-drawn 3D shapes using mesh generation by calculating vertex points, and a highly optimised data structure for transmitting complex 3D objects across a network. Currently we are creating an asset library of building parts and street furniture, so that artists can quickly populate a scene with common features without having to create them from scratch every time. These are being drawn up from our wishlist, and include items such as shop fronts, lamp posts, play equipment and vegetation.
During our two Artist Review Sessions in August at the Zubr VR studio, we had the pleasure of hosting artists Jasmine Thompson, Sam Steer and Andy Council as well as Robert Sargent, Callum Swift and Robert Delius of Stride Treglown Architects.
Involving testers and prospective stakeholders with such varied backgrounds enables us to better understand the artists’ workflow preferences, and also provides us with valuable feedback on the overall user experience, helping to steer our decision-making process.
“The concept for drawing in 3D space, especially directly into the site context via Google Earth, was truly eye-opening and exciting.”
Our next steps are to finish developing the AR app, which is designed to receive the 3D models over a network and populate an augmented layer on location. Once this has been achieved, we can begin testing the full system with our steering group and external partners. We can’t wait to get out and about with the Future Places Toolkit, and start exploring how our neighbourhoods can be better designed for citizens with augmented reality.
We’re very thankful for the funding and support from the MyWorld programme and Digital Catapult, and we wouldn’t have been able to create this prototype without them.
By Jack Norris, Director of Zubr.co, Challenge Call project