The FMOD & Unity Essentials Course Module 6 - Lesson 3 FMOD Callback Events Resources: Examples: Witcher 3: Blood and Wine Implementation Talk: https://www.youtube.com/watch?v=aLq0NKs3H-k Event Callbacks FMOD Scripting Example: https://www.fmod.com/resources/documentation-unity?version=2.1&page=examples-timeline-ca llbacks.html System.Runtime.InteropServices: https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices?view=net-5.0 GCHandle: https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.gchandle?view=net- 5.0 StructLayout(LayoutKind.Sequential): https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.structlayoutattribute ?view=net-5.0 Marshal: https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.marshal?view=net-5 .0 Concepts: Managed vs Unmanaged Memory: - https://docs.microsoft.com/en-us/dotnet/standard/automatic-memory-management - https://engineering.hexacta.com/memory-a-forgotten-topic-355088e2f2c1 - https://www.red-gate.com/products/dotnet-development/ants-memory-profiler/solving-me mory-problems/understanding-and-troubleshooting-unmanaged-memory Garbage Collection and Automatic Garbage Collection: https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/fundamentals What is a Runtime Framework (.NET): https://dotnet.microsoft.com/learn/dotnet/what-is-dotnet-framework FMOD threads and asynchronicity: https://fmod.com/resources/documentation-api?version=2.1&page=white-papers-threads.html https://fmod.com/resources/documentation-api?version=2.1&page=white-papers-studio-threads. html Singleton Pattern: http://wiki.unity3d.com/index.php/Singleton Unity Physics: https://docs.unity3d.com/Manual/PhysicsSection.html Triggers and Colliders: https://learn.unity.com/tutorial/physics-interactions-colliders-and-triggers-2019-3# Layers: https://docs.unity3d.com/Manual/Layers.html LayerMasks: https://docs.unity3d.com/ScriptReference/LayerMask.html C# Comparison Operations: Logical Operators: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/boolean-logical-o perators Bitwise Operators: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/bitwise-and-shift- operators Terms: "this" keyword: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/this Event delegates: https://learn.unity.com/tutorial/events-uh#5c894782edbc2a1410355442 Blittable types: https://docs.microsoft.com/en-us/dotnet/framework/interop/blittable-and-non-blittable-types Pointers (in C): https://www.educative.io/edpresso/what-are-pointers-in-c IntPtr: https://docs.microsoft.com/en-us/dotnet/api/system.intptr?view=net-5.0 OnGUI: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnGUI.html System.String.Format(): https://docs.microsoft.com/en-us/dotnet/api/system.string.format?view=net-5.0 Unity Rendering: https://unity3d.com/real-time-rendering-3d Post Processing: https://docs.unity3d.com/Manual/PostProcessingOverview.html Cinemachine: https://docs.unity3d.com/Packages/[email protected]/manual/index.html ____________________________________________________________________________ Intro - 0:00 Henry: Okay now that we’ve looked at both Delegates and Return Functions, it’s time to look at using those for something more FMOD specific. We’re going to start by looking at FMOD Callbacks, however I’m not going to be the one teaching you this lesson. A good friend of mine, Colin, is going to be the instructor for you and as you’re about to see, he really knows his stuff as you’re about to see! He really understands the sort of “backend” and these “deeper systems” that FMOD uses for both Callbacks and for Programmer Instruments which is the topic of the lesson after this. I’ve just watched his lesson that he has created for Callbacks, it’s very impressive. The one thing I wanted to say is that I don’t want you guys to worry too much about learning every single specific system, tool and thing that Colin points out in this lesson. The main thing is we just want you to get confident with using these tools, so Colin is going to run you through them all, and show you how you can create a really cool system where your game responds to your music, and we can change the color of our camera, just by reading our FMOD event. So I’m going to leave you in the capable hands of Colin. As always, if you have any issues our Discord is always available to you, or you can email me of course. And I hope you enjoy! Colin: Hi, I’m Colin Vandervort, also known as ColinVAudio, and I’ll be helping Henry out for these next two lessons. If you’re interested, I have a youtube channel, twitter and instagram page, where I post about technical game audio implementation concepts and strategies. -- In this lesson, we’ll be unpacking FMOD Event Callbacks. I’ll warn you ahead of time, the implementation for this topic gets pretty complicated and ‘heady,’ but the example I’ll be providing, as well as the one provided in the FMOD documentation is pretty easy to copy and paste, and get working at a basic level. I highly recommend doing the previous lessons before this one if you haven’t, as we’ll be building on a lot of the previously covered material. -- In most game design scenarios, audio is expected to respond and react to the events that are happening in the game. If you click a button, a UI sound is played. If a creature takes a step, a footstep sound is played. If you enter combat in an action game, battle music starts playing. These are very common situations, and are pretty easy implementation problems to reason about. However, sometimes, we might want parts of the game to respond to something that’s happening in our audio instead. It’s the reverse of what we’re used to. Examples of this can be found in any rhythm game, where certain events are synchronized with the tempo or instrumentation in the music, but you’ll also find cases such as in The Witcher 3: Blood and Wine, where enemy combat animations are being predictively assigned to line up with with events in the music. -- Event Callbacks are the means FMOD uses to allow us to build these types of audio-reactive systems. More specifically, Event Callbacks give us access to any tempo, bar and beat information on a track, in real time, as well as our current position along that track, and any custom markers or time signature markers we have assigned to the track. Each time a change in any of these is detected, FMOD will call a function to send that new information to us in the game engine. We can then distribute this information to any systems that will be relying upon it. -- Today, I’ll be walking through the method that’s used in Unity, for setting up Event Callbacks on a music track, and then a common method for distributing that information to other systems. We’ll be using one of these systems in our 3D Adventure Game to trigger some screen shake on each beat of the music in our final Level 2 boss fight, and to set some post-processing color changes on our camera to match the current mood of our music. -- For reference, I’ll be using FMOD Version 2.01 and Unity Version 2020.3. -- To start out, let’s hop over into our FMOD project, and create or select a music track. FMOD Music Setup - 3:54 This will be easier to visualize if you’re using a music track with a pronounced beat, and for that reason, I’m using the battle music from the end of our second level in the 3D game kit. Make sure to add a Timeline tab to our event if you don’t have one already, and place your music on it. Now, right click the Logic Tracks section in our timeline and add a tempo marker for each tempo or time signature section in your track. In our case, this is 112 BPM at 4/4 time. We’ll be using this tempo and time signature data to control our beat event callbacks, and trigger our screen shake event. Additionally, make sure the track loops. From here, create a few destination makers, and scatter them around your track. We won’t be using these for any logical purpose within FMOD, but we will be using their “string” names to apply specific behaviors when they’re reached. Place them in any section of your track where you’d like to highlight a tonal shift, and then give them a name. In my case, I’ll be naming them directly after the post processing colors I’ll be applying to our camera when they’re reached, but they can be realistically named anything. Chorus, Verse and Bridge are some example names that could work. Intro, low-intensity, and high-intensity might also be good names. The only caveat is there cannot be two markers that share the same name, back to back, or the second one won’t be detected as a new marker by our system. -- Now, because I’m mimicking the functionality that the Unity Battle music already had, I’ll be adding a global parameter entitled “MusicSection.” This parameter will be used to fade our battle music in and out when we enter and exit our boss battle arena. The parameter will be continuous, and mapped from 0 to 1. We’ll assign our volume curve such that 0 is silent and 1 is full volume. If you already have general level music set up in your scene, you’ll want to create the opposing automation curve on that music so that it fades out as the battle music fades in.