SDK Updates And More

3DceptionLogo

We’ve been up to lots here at Two Big Ears with feature improvements across our product range, new platform support, new mediums and continuing performance optimisations. Nothing better than a quick blog post to summarise!

Unity 5.2

With the need for spatial audio gaining prominence, Unity’s audio team have addressed the performance issues with 3rd party spatialisation plugins — like 3Dception — with a new Spatialisation Plugin SDK. This new feature is a part of Unity 5.2, releasing tomorrow. We have been trialling and providing feedback on this new SDK over the past few months and Unity’s audio team were kind enough to implement some of our feature requests.

What does this new SDK mean for 3Dception? First off, a massive leap in performance (up to 6x!) which so far was only possible with other middleware or native implementations. Additionally, the new SDK creates a more streamlined workflow when implementing 3D audio. 3Dception’s feature set remains the same but will be translated to this easier workflow.

3Dception 1.2 supports the new SDK, in addition to a whole set of performance optimisations and spatialisation improvements. Unity 5.2 will be released on September 8 and 3Dception 1.2 will be released shortly after was released on 17 September. Existing components such as 3Dception Source and 3Dception Filter have now been simplified to work directly with a Unity audio source. The release includes an automatic update wizard to help upgrade existing projects.

3Dception 1.2 is a Unity 5.2+ only release, but will be back-ported to support Unity 4.x-5.1 (with the older Unity audio APIs). We will be eventually deprecating support for these earlier versions, although 3Dception 1.1 and prior will continue to work.

Wwise

3Dception Wwise 1.1 was released recently and includes many performance optimisations, spatialisation improvements and support for Wwise 2015.x. We have been pleased to see our user base quickly upgrade to Wwise 2015 and make use of all the new features and improvements from Audiokinetic.

Geometry

The geometry system has been in closed beta for the past few months. We’ve been collecting lots of feedback and making many many many improvements to the system. It isn’t ready for prime-time yet, but will be soon. Get in touch with us if you’d like to try it out!

New Platforms & Mediums

We’ve also been working on some new platforms, product integrations and new media of interaction.  There’s been lots of exciting stuff that we’ve been busy with and you should hear more about it all very very soon 🙂

Great Content

We’ve had loads of games that have been in development since we released 3Dception 0.1 (!). Many of these are finally getting closer to release and we can’t wait to talk about them. Watch your social media!

 

Pitch And Perception

George Vlad is a sound design intern at Two Big Ears. Starting with this post he will be documenting his experiments with sound design parameters and audio spatialization on this blog.

Following up on the previous blog post I have been investigating the role of pitch in the perception of movement and spatialization. The first thing I need to mention is that this is my no means an exhaustive process. I would rather say that I’m scratching the surface and providing food for thought for whoever finds this as fascinating as I do.

I designed a set of 5 similar sounding files resembling an internal combustion engine hum. The frequency content in these files lies mostly below 1 kHz as you can see in the picture. The Wwise engine is programmed to pick one of the 5 files at random creating an indefinite loop. I first played the Intro scene with the original sound palette so as to examine its perception with regards to azimuth, elevation, movement and spatialization. My conclusions with regards to this particular file should be taken with a grain of salt when applied to other files. Pitch is only one of many parameters that can alter and skew the perception of movement, elevation, spatialization and so on.

A few words about the Wwise and 3Dception settings I used. The default doppler option in 3Dception was disabled to avoid pitch changes. 3Dception’s room modeling was enabled with the default settings and the attenuation mode was set to 3Dception’s default as well. You will notice an audible gap in the video at certain points. This seems to be caused by the Wwise engine that needs to catch up with the fact that the files are shorter once they are pitched up. I then proceeded to raising the pitch on the sound palette by 2 semitone increments. This helped me observe the changes in perception and enabled me write the following notes.

Azimuth

Although the unpitched sound is diffuse, there’s no doubt that initially it is coming from the left side. Once the Robot object reaches the camera and goes off screen again it is similarly easy to pinpoint it as coming from the right side. It is safe to say that increasing pitch does not affect the azimuth perception by a great deal.

Continue…

3Dception Unity 0.6 — What’s New?

We’ve just pushed out v0.6.0b of 3Dception Unity as we get closer to exiting beta. This update includes over 30 new features, fixes and improvements. Here’s a summary of what’s new:

More Sources!

The active source limit for non-commercial projects has been increased to 10! Yay!

Pricing

The Basic option has been dropped to £9/month from £14/month. You can also optionally pay upfront for a year of free upgrades, if monthly payments isn’t your thing.

Room Model With Variable Surfaces

Room models now include separate reflection properties per surface in the room (walls, ceiling, floor), making it possible to design dynamic environments with a lot more character. The UI gizmos have also been improved, so that the walls change in opacity depending on their reflection property. In one look you’d be able tell what your room sounds like. Cool! Here’s a GIF, just because:

rooms

Continue…

3Dception for Fabric

UserInterface

Our aim with 3Dception is to make great sounding algorithms available across different devices, systems and workflows. This doesn’t just mean good quality and efficient algorithms, but also supporting the workflows that make good design possible. Fabric for Unity is a great middleware tool if you are developing games and apps for Unity. We’re happy to announce that we’ve partnered with Tazman-Audio to make 3Dception compatible with Fabric. Integration scripts for Fabric will be available for free in the upcoming update of 3Dception — which includes a whole bunch of improvements, optimisations and great new features.

3Dception is currently available for Unity (natively and Fabric) and Wwise across OSX, Windows, Linux, iOS and Android. We’ve got a long list of announcements to make in the coming months and we can’t wait to share it!

Full press release below! Continue…

Unity and libpd

pd-unity-screen

We are big fans of Pure Data at Two Big Ears. It makes it quick and easy to prototype ideas, or even treat our prototypes as ‘shippable’ products. Our most recent project (which you should hear about in the next week or so) involved a lot of work in Unity. There have been many attempts to get Pure Data (libpd) and Unity working. Patrick Sébastien had done amazing work in getting libpd to work with Unity on Windows (libpd-4-unity). In my spare time I took it on myself to extend support to OSX, Android and iOS.

Success? Yes (almost).

You will find some differences in the way it is all setup, if you have used libpd-4-unity in the past. The project structure needed streamlining to maintain cross-platform compatibility. Thankfully there is a whole library of helper APIs in Unity to make this possible. Let me drill into the specifics:

General:

libpd works *within* Unity. Unlike Kalimba (which is a great alternative for iOS/Android), it is a plugin for Unity and runs on the audio thread. This does mean that libpd works only with Unity Pro (although, there are workarounds to this that I haven’t tested or tried).

The ‘heart’ of the libpd setup in Unity is a single script: LibPdFilterRead.cs. It includes basic methods for initialising libpd and opening and closing patches. It automatically queries the sample rate and buffer size and sets things up depending on the platform/system.

Asset Hierarchy:

All Pd assets (patches, audio files, text files, externals) must be placed in Assets > StreamingAssets > PdAssets. libpd will not be able to find any of the resources in a build if the assets are placed in another folder.

asset-hierarchy

Setup:

It is recommended that you add LibPdFilterRead.cs to the camera or an empty game object in the scene. It is not recommended to have multiple active instances of LibPdFilterRead.cs in a scene (this could be enforced programmatically, but hasn’t been done).  The scene 01_LibPd_Basic.unity included in the project demonstrates this.

LibPdFilterRead.cs uses the onAudioFilterRead callback in Unity. You can think of it as a plugin insert on the DSP chain. This means that you can also send audio data into Pd from Unity (audio playback within Unity or microphone input). The scene 02_LibPd_ADC.unity included in the project demonstrates this.

The project includes two other scripts: GUIToggleScript.cs and GUITextScript.cs that demonstrate sending and receiving messages from Pd.

Creating your own project:

To use libpd-4-unity in your own project, copy the ‘Plugins’ folder , the ‘LibPD’ folder (this includes the C# bindings) and create the ‘PdAssets’ folder under StreamingAssets (create this folder too if you don’t have it already). Add LibPdFilterRead.cs to a game object, specify the patch name and you should be good to go!

filterread

OSX:

There doesn’t seem to be any major problems on OSX — I’ve tried building multiple projects and they all work fine. Externals work fine too! The full libpd API hasn’t been tested, but this will happen at some point in the near future.

The Unity Editor (previewing your game within Unity) at times fails to open a patch (SIGILL for those interested). I haven’t had the chance to track down this issue and it can be temporarily fixed by restarting Unity. This doesn’t happen in a build though.

This said, unity-4-libpd hasn’t been extensively tested so if you come across any problems do let us know!

Android:

Oh the joys of Android development!

The good news: it runs fine. No issues if you have a patch with no dependencies (audio files, text files, externals). The problem is that any data within an Android APK isn’t easily accessible by Unity/libpd. The workaround is to run a few lines of code and copy the data from the APK to the SD card (this is what a lot of apps do). Currently it only copies the Pd patch you’d like to open in Unity and nothing else. Ideally, it should be copying across the whole PdAssets folder to the SD card (or could borrow a trick or two from how libpd-android is setup). This is high on my priority list and will be fixed as soon as I get the time. If you have ideas for workarounds let me know!

iOS:

This is pending and is on the to-do list. In time!

Windows:

I haven’t tested the current build on Windows yet, but Patrick’s previous work seemed to have run fine. Feel free to let us know. I’ll test it as soon as I have my windows development pipeline working again.

Documentation:

The documentation/readme/Wiki needs work. I will add information on building the plugins for OSX and Android soon.

To conclude:

There is work pending, but the project in its current form should allow you to get started on creating cool procedural audio/music within your game! If you have ideas for further development, please fork and create a pull-request.

Thanks again to Patrick Sébastien and everyone else involved in the libpd/Pd project for all their hard work.

Go get it here.

Give us a shout if this helped or made life tougher for you. Follow us on Twitter or on Facebook.