How Harman’s FLUX:: is revolutionising live immersive audio

Hugo Larin, senior business development manager FLUX::, a member of the Harman Professional Solutions Group, delves into Harman’s most recent acquisition, which includes FLUX::’s immersive, processing and analysis solutions across live production, installation sound, content creation and post-production. 

France-based FLUX:: was founded in 2006 for the purpose of creating intuitive and innovative audio software tools. Now, the portfolio of nearly 20 products is used globally by talented professionals across many applications. Larin explains why the sky’s the limit for immersive audio production.

How would you define immersive audio, and what unique experiences does it bring to audiences compared to traditional audio formats?

For me, it's all about the ability to connect to the message and the experience. It’s to envelope the audience and to transport the audience with that connection. The connection is the real highlight about immersive audio overall, from content production, reproduction or from a technology perspective.

Can you provide a brief overview of FLUX:: and its core competencies, particularly in immersive audio and audio analysis?

Our history comes from plugin processing as we have over 20 years of software development plugins for processing tools. When you look at the main competencies, processing is our history and our legacy. 

The second one is audio analysis, and third, but not least, is immersive audio authoring and rendering. You could split FLUX:: in these three categories of competencies,

How does FLUX:: embody a "software first" approach within Harman, and why is immersive audio considered a pivotal element of this strategy?

In terms of a “software first” approach, this is what our users are dealing with on a daily basis. The hardware is important, but the actual software is what our system engineers and integrators interact with day to day, whether they’re actually using the design assistance tools and the deployment tools. 

Within Harman we can really look at all these innovations from a software perspective, like JBL’s Venue Synthesis in the design side and deploying actual audio systems, and then FLUX:: comes to the equation and adds these other layers like SPAT Revolution, the immersive authoring and rendering tool, and audio analysis, which is another important category. 

Hardware is important, but the software is what makes the interaction great with the technology. Software is the glue to making hardware work.

FLUX:: has a rich history with multichannel audio processing. Can you walk us through its evolution, from early plugin support to the development of the SPAT Revolution software engine?

FLUX:: has multi-channel at heart, and that's what it's been for the longest time. Even when 5.1 started to emerge and these multi-channel conversation formats started to emerge, FLUX:: was one of the first companies to actually do plugin processing tools that were supporting not only mono and stereo, but multi-channel. 

Very early on, we addressed audio processing from a 5.1 and a 7.1 perspective, and that has just exploded. Now in our overall journey of multi-channel we continue this in audio analysis. We did it in processing, and in audio analysis we are able to look at audio from a multi-channel perspective.

Today we have SPAT Revolution and all our work on immersive audio, and we can't ignore the relationship that we have had with the IRCAM (Institute for Research and Coordination in Acoustics and Music) – the research centre in Paris, which has done 20-30 years of research on how people perceive sound, acoustic spaces, and rendering technologies. Some of the first research that led to SPAT came to life back in 1995. 

We joined forces with IRCAM over 10 years ago and became a business partner, and we brought these technologies to the market.

That was the beginning of the SPAT Revolution in a lot of ways. We moved from beyond 5.1, 7.1, binaural audio, ambionics and ambisonic transcoding to dealing with reproduction techniques for larger systems. Multi-channel is deep inside FLUX::.

What challenges do audio engineers face when transitioning to immersive audio, and how does FLUX:: address those challenges with its products?

Immersive audio is not new – we can call it immersive audio, immersive, sound, 3D audio, spatial audio, multi-channel audio – but we're in some ways talking about the same thing overall. When we use the word ‘immersive’, we're referring to the actual experience: we're immersing our audience. We're transporting them. We're connecting them. 

Something that all manufacturers seem to agree on is that we are all referring to the quality of the experience. But 3D audio / spatial audio is not new. Audio engineers, creators and artists have been creating for the longest time, but with a huge challenge, which was the workflow: there was no real, defined workflow.

So, we have looked at: how do you create this content? How can you scale it? How can you collaborate with other engineers? How can you deliver the experience to multiple venues? As we're transitioning, I think that the biggest challenge is those workflows: being able to have workflow from content creation to reproduction. 

When creating content, how do we deliver that media and reproduce this audio? To me, that is the biggest challenge, and FLUX:: has been focusing on addressing this, specifically with the integration into the digital audio workstation.

SPAT Revolution is a standalone software engine, but it's actually a suite of products that includes plugins that do a tight integration to your audio workstation of choice. 

We're expanding the multi-channel capabilities of the digital audio workstation with SPAT Revolution from a content creation perspective. A lot of focus has been put into easing the integration and simplifying how to craft content and deliver it.

FLUX:: doesn't just do audio processing and rendering, it’s audio analysis as well; what can you reveal about this at the moment?

Audio analysis is another pillar of the company. Historically, the Pure Analyser System – PAS – does multi-channel support from an analysis perspective. Being loudness, RTA and doing transfer functions for system calibration. It encompasses 13 analysis scopes. 

From an analysis perspective, what we've been doing is beyond strictly looking at calibration per se,. We've built a tool that’s very visually pleasing that allows a mixing engineer to have this companion that provides a good visual interface, and not just in mono and stereo, but in multi channel. 

We have some interesting technology, for example, Nebula. This is a spatial spectrograph with the ability to visualise an audio spectrum over space.

We have just released our fresh new family for audio analysis, MiRA. Those capabilities are increasing and the channel count extends to 24 channels for multi-channel systems. In that particular case, from an analysis perspective, we need to think about integration. How do I route my audio? That is the question. 

We have plugins that take the audio and push it to the analyser on the computer, on the same computer or across a network. We call this SamplePush technology. We are simplifying the integration from the DAW or the mixing console to provide instant analysis. This extends as well to SPAT Revolution integration.

Has this been requested from users?

There's no secret that our community has been asking for this. Workflows are a challenge, so in trying to solve this we've been simplifying the integration to our beloved analyser. The analyser is very important because, yes, there is analysis when calibrating a system, but there's also analysis when producing or performing. 

The monitored data serves to perfect the production mix or take that information back to the calibration. Analysis plays an important pre, post and live production role. We’ve been full on in making this new product, MiRA. Our users have been wanting it. We've added custom layouts and workspaces to it too. It's been extremely well received…

For you, there are three fundamentals of immersive audio: localisation, envelopment and sonic artistry. Can you elaborate on each and their significance in creating an immersive experience?

We have technical, perceptive and artistic capabilities that come from using immersive audio technology. I split those in those three categories. The first one is the localisation: that's the simplest one to understand. It's more technical in the sense that you're actually deploying these technologies to provide a better sense of localisation. 

This is something that a traditional system can't do, or can do, but only for a limited part of the audience. These technologies allow us to actually have a very stable localisation image , or spatial imagery for a wider audience.

For instance?

Localisation is ‘what you see is what you hear’, and that brings an important conversation. It's not just to connect to the artist or performer. 

It includes connecting with the speaker at a conference or corporate event, the spiritual leader and extends to the visual scenery around you. It's more technical. It's just better technology to render and reproduce the audio with localisation for the widest possible audience.

With envelopment, we could call this transporting the audience. This is getting in the perceptual and creative intent. 

With envelopment, what we mean is that an immersive audio deployment allows us to transport our audience into a space that they're not. Using spatial audio technology and advanced systems, we can sonically transport you. Envelopment is that anchor.

The third anchor is what I call sonic artistry. We're talking about creative potential at its best as an open canvas. 

This canvas allows artists and creators to free their mind and do whatever they want with the space to engage their audience. The sky's the limit at this point; you've got a completely new canvas to explore. You can paint with sound.

SPAT Revolution has been described as a flexible and compatible solution for audio content creation. What sets it apart from other immersive audio software on the market?

Some technological solutions out there are pretty close ecosystems. They're confined to specific targeted systems and specific deliveries. They are targeting very specific applications, or targeting traditional, live inputs from a console going to a rendering engine.

In terms of flexibility and compatibility from a content creation perspective or from a mixing perspective, SPAT, first of all, is not strictly dealing with loudspeaker setups. We've got multiple reproduction approaches. 

Some of them are more fitting for some applications. It can be part of your content creation monitoring journey all the way to the final delivery reproduction. That flexibility comes from the variety of reproduction. We call them panning or reproduction techniques – the access to render not just a channel-based system, but for binaural audio and ambisonics. 

When dealing with ambisonic capture, for instance, maybe you're crafting some content, and there's a desire to have a river sound. Well, you don't need to go and record the river; these libraries of sound already exist. 

We have the ability to take an ambisonic capture and bring it into our immersive tool and transcode it to the desired output. You don't have to go back to another tool to do that transposition. That flexibility and that maximum compatibility goes beyond that.

From an integration perspective, using our plugin integration tool or our advanced open sound control (OSC) engine, we bring a lot of capabilities. 

At the base these are elements that set us apart. Whatever you have to deal with, SPAT Revolution can potentially be all that is needed to deal with all these stream formats and reproduction with strong integration capabilities. 

It provides maximum flexibility because it runs on generic hardware as well. So that means my laptop on the plane and a pair of headphones, and away I go!

How does SPAT Revolution create new opportunities for real-time reproduction of immersive audio?

When we're talking about live, the key element is we're dealing with an audience. They are not necessarily in the sweet spot in that audience. One of the key things is better reproduction techniques for immersive audio and with SPAT Revolution, that’s one of those reproduction techniques that we're extremely proud of. 

Whenever we have the appropriate loudspeaker set up to do it, we bring Wave Field Synthesis (WFS) to the table, which is very unique in order to bring image stability with a loudspeaker collaboration. We're moving away from the traditional illusion that there is a sound somewhere – we call it the phantom source. 

We're moving away from these reproduction approaches to an approach that's actually about virtualising the physical characteristics and about how a sound normally emanates in acoustical environments. We have a technique that basically reproduces this. We're extremely proud.

Can you share some real-world examples of projects or applications where FLUX:: immersive audio solutions have made a significant impact?

One recent highlight are these flying theatres that we've been involved with. There's one in Reykjavik Iceland, in Vegas, a freshly redesigned one in Vancouver and most recently a new venue in Chicago. They've been advancing the actual sound reproduction systems. 

We've been using Wave Field Synthesis in these deployments as well. But the key there is how, in the studio on a smaller traditional monitoring system or over a pair of headphones, the content creators and the mixers were able to start, and then minimise the deployment on site to be able to deliver this to 40+ channels of loudspeakers. 

This is a good example of how we help streamline the pre and postproduction work with SPAT Revolution.

How do you see the role of immersive audio evolving in different industries like corporate events, installations, gaming or live events?

In current gaming workflows, there's some processing capability limitations. What I see here is this ability to use immersive audio technology for fan engagement events or watch parties. In the corporate world, someone could be presenting a new product and they could produce an engaging video and story with multichannel audio. 

Or a president could walk on stage, and as they walk, they are localised in terms of audio and tracked, so you always hear them from where exactly they are. You’re engaged and connected. It's not just the mere pair of speakers on each side of the stage for basic hearing.

In terms of evolving into typical movie theatre installations, they're there to present a program of movies, but there's definitely more that can be done in these spaces. Especially theatres with advanced systems and in terms of offering a more diverse program, maybe with some live performance, digital art or esports events. 

It’s about transforming a space to gather a more diverse customer base and increase business. Immersive audio installations are becoming increasingly popular across various industries, including entertainment, museums, and digital arts. These installations use advanced audio technologies like spatial audio, to create a more engaging and interactive experience for audiences and set themselves apart.

What feedback have you received from users of SPAT Revolution, and how has it influenced ongoing development and updates?

User feedback has been critical to our development in terms of really understanding what can be done and how we could do it better. The overall conversation right now is, how can we help them streamline that actual workflow? How do I design a system? 

Do I have a design tool that helps me choose loudspeakers from a prediction, modeling or a design perspective, and simplify design, to deployment, to analysis, to control? We’ve been thinking about this workflow and extending it to how we can simplify the digital audio workstation in content creation – all the way to the delivery.

As immersive audio continues to gain traction, what advice would you give to aspiring audio engineers looking to specialise in this field?

Education about understanding the technologies, the audio workstations, their limitations and their capabilities. There's reproduction technique, understanding the formats, binaural, ambisonic and then in the context of loudspeakers, the different reproduction and panning techniques and how they are different from an experience delivery perspective.

Diving deeper in crafting content, understanding perceptual factors and acoustic spaces, and how they play a role in creating immersive soundscapes, and improving the overall listening experience. Is a sound closer or further away? 

Understanding concepts of distance, attenuation, air frequency absorption, and the use of real or artificial reflections, and how to use these to “trick” an audience to actually elevate the listening experience.

Looking to the future, what trends do you see in immersive audio technology, and how do you envision FLUX:: contributing to that evolution?

It's about streamlining the process: more tools, simplifying the design, simplifying the deployment, simplifying the interactivity with the audio objects, and simplifying the actual delivery. We want to streamline to empower creators and professionals, from content creation to delivery. This is what the SPAT Revolution is about. 

It's content creation on a pair of headphones, all the way to the largest reproduction system possible. There will be a lot of challenges ahead and interesting trends from a manufacturing perspective on streamlining the tools. It's been a fun journey, and it's not over…