Media art spectacle in virtual space: Schlosslichtspiele 2020 - Digital Edition in the Unesco City of Media Arts
iXtenda and AV Active bring the Schlosslichtspiele Karlsruhe to living rooms around the globe
Most people should have gotten used to the fact that in the corona year 2020 many things will be different from what they are used to and have grown fond of. All major events: canceled. This was also the case for the Schlosslichtspiele Karlsruhe, which were attended by tens of thousands of people from Karlsruhe and the surrounding area every year. The multimedia spectacle with format-filling projection mappings on the facade of the Karlsruhe castle was to be held for the sixth time in 2020.
While no one was seriously arguing about the reasonableness of canceling the six-week event due to the Corona Protection Ordinance, the desire for a feasible alternative soon arose. Much had already shifted to the Internet - to live streams, hybrid events, or virtual spaces. But would a highly complex event like the Schlosslichtspiele be able to be carried out digitally, for six weeks?
If you are interested in the entire "Making Of" here is a great video, how we brought this digital spectacle to life.
With this question, the organizers of the Schlosslichtspiele, the ZKM, and the Karlsruhe Marketing und Event GmbH, approached Benjamin Brostian, the consultant of many years for innovative technical concepts, in mid-May. The idea: as enormous as it is tricky. Brostian quickly realized that it would need both sound media technology and software expertise to make the Schlosslichtspiele 2020 - Digital Edition possible under the motto "Aborted in reality - departure into virtuality". The planning and implementation of the project were finally carried out by the media technology experts of AV Active GmbH and the software specialists of iXtenda GmbH with the help of photogrammetry, 3D real-time software, and cloud solutions.
From more than 76,000 individual shots taken by drone and SLR cameras (a total of about 10.5 hours of 4K footage), authentic 3D models were created, not only of the castle, but also of the surrounding park and its objects, such as fountains, statues, but also lamps and even litter garbage cans. Unlike clean 3D-models or materials and textures that seem "used", the results of photogrammetry are not perfect and therefore authentic. Damages, dirt, and patina were captured at once - and gave the objects the charm of the real thing.
"Cloud solutions, especially Microsoft Azure, have become very interesting for us in the last few months because they allow us to scale freely and thus reach our goal much faster. Cloud networks are still a big complicated Pandora's box, but they offer us an almost endlessly large platform in terms of development, simulation and services", adds Sandro Forster, developer at iXtenda GmbH.
"When you put the whole thing together for the first time, you quickly realize: there is still something missing", adds Kai Vermeegen, project manager and developer at iXtenda. "The castle, the square, the individual objects, that was all there. What was still missing was some gardening and the wow-effect".
Algorithms were commissioned with the "gardening", whose task was to create about 2,000 unique items of the flora - trees, bushes, shrubs. These, in turn, were then automatically placed on the castle grounds using the 3D survey and photogrammetry data. The same procedure was used with stones, gravel, cobblestones, and the like.
Complex environmental variables in real-time
In order to make the Schlosslichtspiele 2020 - Digital Edition a true-to-life experience, countless environmental variables were taken into account. What does the scenery look like at sunset at 8:58 pm and how does it look four weeks later at the same time? In sunshine and on cloudy days? How does the light source on the facade affect the Schlossplatz? How does all this affect the viewer?
"We work with the real-time solutions of Unreal and Unity, but in this case, we chose Unity and never regretted it for a second. The whole environment reacts in real-time. If it is 20:45 at the moment, it is also 20:45 in the engine," explains Vermeegen.
In general, the topic of real environment variables should be treated with caution. "Not everything that is real is also optimal," laughs Brostian. "In the past years, we have occasionally wished that it would be a little darker or that it would not rain on the day of the premiere... The appearance of the clouds at the Castle Light Festival 2020 was also not linked to the actual weather conditions. All this would be possible, but in fact, it contributes nothing to the effective added value.
From August 5 to September 13, 2020, the Schlosslichtspiele Karlsruhe - Digital Edition 2020 could be seen for one and a half to two hours every evening from 8:15 pm on the interactive live stream on the website (www.schlosslichtspiele.info) as well as on public channels such as YouTube and Twitch. A total of around 100,000 end devices connected to the premiere evening. A success that should not be a flash in the pan. Thanks in part to Karlsruhe's status as "Unesco City of Media Arts 2020", the digital concept developed into a media art event that attracted worldwide attention and continued to enjoy great popularity throughout its entire duration. The evening shows were accessed by an estimated 250,000 people in 120 countries.
In addition, viewers could choose their own perspective on the Schlosslichtspiele website or watch the "Director's Cut" running parallel on YouTube and Twitch.
In addition to popular pieces from past years, such as "300 Fragments" (Maxin10sity - 2015), "Defilee for the 100th Birthday of the Avantgarde" (DSG Animation & VFX - 2017), "Reverb" (Bordo's Artwork - 2015), "The Object of the Mind" (Global Illumination - 2018), three new media artworks were also on the program in 2020: "Attitude Indicator" by the Bielefeld group The Nightlab (TNL) and "Changes" by the group Rüstungsschmie.de. Due to the great success with the audience, the ZKM and Karlsruhe Marketing und Event GmbH decided to also show the new show "Our only blue one" by Maxin10sity, which was only planned for next year.
"I would like to thank all of the project participants. First and foremost the ZKM, the Karlsruhe Marketing und Event GmbH, and the Lord Mayor of Karlsruhe, Dr. Frank Mentrup. Once again, Karlsruhe was the first to venture into new territory with courage and chose a digital format that has never before been implemented in this form for a projection-mapping event. The success proves them right. Thank you for the trust you have placed in us and we look forward to 2021," concludes Benjamin Brostian.