Blender 3x Audio Research

Work in Progress Last Updated: 25-05-2022 by Uriel Deveaud (AKTIV25)

🔘 Introduction

This current documentation (in progress) is intended to demonstrate the requested audio features in the context of editing audio contents/data within Blender¹ itself, the well known Free and Open Source 3D application, and within Audaspace², the Audio C++ Library built in Blender to process digital audio. It is mainly used in the Video Sequencer Editor (VSE) for playing Audio strips and 3D Space as 3D Speaker Objects.

Here, i want to present specific features which will make it possible to work on the audio data, including routing, processing and visualizations. For all users with basic needs in terms of Audio Mixing, this proposal is focused on:

• Media Productivity³, including audio and Video editing, Animations, Games Post-production...
• VR/AR Art and Experiments, including Multiverse Production, Installations and Real-time Performances
• Surround Contents Creation, including music clip edition and digital mixing console, up to 8 outputs channels

The main goal is to build a new patch (or maybe a Git Branch) to present to the Blender Foundation⁴ and GSoC 2023⁵. The development of this patch is encouraged by the Blender community members and it will be first tested by other Blender developpers, then it will be sent to the BF for final approval. I invite you to post in the "Pull Requests" section any comments or suggestions that you think it is important to consider. Thank you.

---

🔘 Ongoing works

• Main Development Research References: Concepts, Strategy, GUI,
• Additional Research: Spatial Audio and Matrix Routing
• Personal documentation: Blender and Audaspace related specifications.
• Features Development: Visualization Tools and Operators and Properties
• GSoc 2023 Presentation (Video, 3D Avatar, Surround Audio)

---

🔵 Development Strategy and GSoc Proposal

• 1️⃣ New General Audio Features & System Preferences
  • Channels Options, Custom Channels, Channels Outputs Configuration, Gui Implementation
  • Classes, Properties, Operators, Device, Device_channels, Device_output_Name
  • Sources, Notes, List

• 2️⃣ New Sequencer Sound Strip Properties
  • Mixer Properties, Sound_Strip Output Bus Assign, Device Main Mixer, Sound Modifiers, Preview
  • Classes, Properties, Operators, Device, Device_channels, Device_output_Name
  • Sources

• 3️⃣ New Sound Mixer Space Features
  • Sound Mixer Space, Audio Signals, Audio Nodes, Animate, FX Processing
  • Sources

Opt

• 4️⃣ New Sound Clip Editor Features (Optional)
  • Clip Editor, Markers, Fades, Export, FX Processing
  • Sources

This document is not a definitive version yet! I do add, remove and update many parts, "without announcement", to re-format the complete documentation. Stay tuned!

---

🔘 Audio Data Specifications

Specifications	Details
Main OS Plateforms	Windows, MacOS, Linux (32/64bits operating systems)
Main Application	Blender 3D
Sound Engine	Audaspace
Codebases	C++, Python
Audio Devices	WASAPI (Pulseaudio, Coreaudio, ASIO), OpenAL, OpenAL Soft, SDL
Audio Output Channels	0...8 (None, Mono to Surround 7.1)
Audio Input Channels	None
Audio Mixing Buffer Sizes	256, 512, 1024, 2048, 4096, 8192, 16384, 32768
Sample Rates	44.1kHz 48kHz, 96kHz, 192kHz
Sample Formats	8-bit Unsigned, 16/24/32-bit Signed, 32/64-bit Float
Sound Files	AC3, FLAC, MKV, MP2, MP3, OGG, WAV
Audio Level Unit	Decibel (dB), -60 <> 0dB range, linear/Exp.
Scene Audio Main Volume	Value: 0...10000?
Scene Audio Distance Models	None, Inverse, Linear, Exponent (Clamped)
Sequencer Channels	128
Sound Sequence Strips	Direct to "Master" Audio Outputs (user preferences)
Sound Sequence Volume	0.00...100.00? (1.00 is 100%)
Sound Sequence Pitch Adjust	0.10...10.00? (1.00 is 0% +/-)

---

🔘 External Links and References

Blender builds

• Blender Source Code: Repository / Code Structure / Developpers Documentation / Advices
• Building Blender Windows OS / Coding Style / Patching / Diferencial / Arcanist

Audaspace Library and Python

• Audaspace Library 1.3.0 Official documentation, and neXyon´s repository
• Audaspace in Blender Python API 3.1x / Sound Operators / Sound Sequences (Strip)
• Audio Nodes by neXyon / by noemlif

BPY Blender Python

• VSE Addons: Audio Strip Fade / Text to Speech / VSE customizations / Sequencer Audio Recording / Dolby Atmos Exporter / Channels Export / Cut on Peak
• Talks: VSE Sound Clip (RightClicSelect) / Proposal VSE Nodes
• External Tools Addons: Blender Audacity

---

🔘 History

2022 May - Basic Audio Preferences rewriting, Vu-meter Class, Tasks list

2022 Abril - First Dev Planning and ideas, Files and contents structure

2017 Abril - First Vst Plugin Development, Samples Players, Filters in C++

2016 March - First Sequencer Script/Addon Development, Strips Management

2014 February - First Midi Script/Addon Development, Learning RTmidi C++ Lib

2012 March - First Movie Making workshop,teaching using the Blender VSE (82 students)

2011 January - First Professional Blender Training, Design and Architecture (113 students)

2010 January - First Audio Script/Addon Development, Learning Audaspace C++ Lib

2009 October - First 3D Animation School Program, (185 students)

2008 October - First Professional Video Editing using Blender VSE

2007 September - First Blender Python Script Development

2005 June - First Professional Animation Production using Blender

2002 December - First 3D Model Production using Blender Free Software

2001 September - First Audio/Midi Hardware Development⁶

🔘 Infos

• Author: Uriel Deveaud⁷ - Kore Teknology

Footnotes

1. Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation. Please, visit the Blender Official Website. ↩

2. Audaspace (pronounced "outer space") is a high level audio library written in C++ with language bindings for Python for example. It started out as the audio engine of the 3D modelling application Blender and is now released as a standalone library. Audaspace Website... TODO ↩

3. In This case, Blender is used as a Audio/Video Sequencer Editor ↩

4. The Blender Foundation (2002) is an independent public benefit organization with the purpose to provide a complete, free and open source 3D creation pipeline, managed by public projects on blender.org. ↩

5. GSoc or Google Summer of Code is a global, online program focused on bringing new contributors into open source software development. GSoC Contributors work with an open source organization on a 12+ week programming project under the guidance of mentors. ↩

6. Google Patents: Google Patents: Real-time Surround Digital Mixer by uriel Deveaud @2001 ↩

7. This work is dedicated to all Blender users ;) and the Free Software Foundation Members. I do this work in my free time, without any profit, enjoy! ↩