MFAudioFilter Walkthrough: C++ Capturing Audio Streams with a Media Foundation Audio Filter About This Walkthrough In the Kinect™ for Windows® Software Development Kit (SDK), the MFAudioFilter sample shows how to capture an audio stream from the microphone array of the Kinect for Xbox 360® sensor by using the MSRKinectAudio Microsoft DirectX® media object (DMO) in filter mode in a Windows Media® Foundation topology. This document is a walkthrough of the MFAudioFilter sample. Resources For a complete list of documentation for the Kinect for Windows SDK Beta, plus related reference and links to the online forums, see the beta SDK website at: http://kinectforwindows.org Contents Introduction .......................................................................................................................................................................... 2 Program Basics..................................................................................................................................................................... 2 Create and Configure the MSRKinectAudio DMO .......................................................................................................... 3 Configure System Mode ................................................................................................................................................. 4 Configure Source Mode .................................................................................................................................................. 5 Configure the Array Descriptor ...................................................................................................................................... 5 Configure Feature Mode................................................................................................................................................. 6 Configure Noise Suppression......................................................................................................................................... 6 Configure Automatic Gain Control ................................................................................................................................ 6 Configure Input and Output Types ............................................................................................................................... 7 Incorporate the MSRKinectAudio DMO into a Media Foundation Topology ............................................................. 8 Create the Encoder and Sink Objects.......................................................................................................................... 10 Create the Source Object .............................................................................................................................................. 12 Create the Topology ...................................................................................................................................................... 15 Capture the Audio Stream ................................................................................................................................................ 18 Resources ............................................................................................................................................................................ 21 License: The Kinect for Windows SDK Beta is licensed for non-commercial use only. By installing, copying, or otherwise using the beta SDK, you agree to be bound by the terms of its license. Read the license. Disclaimer: This document is provided ―as-is‖. Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. © 2011 Microsoft Corporation. All rights reserved. Microsoft, DirectShow, DirectX, Kinect, MSDN, Windows, and Windows Media are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners. MFAudioFilter Walkthrough: C++ – 2 Introduction The audio component of the Kinect™ for Xbox 360® sensor is a four-element microphone array. An array provides some significant advantages over a single microphone, including more sophisticated acoustic echo cancellation and noise suppression, and the ability to determine the direction of a sound source. The primary way for C++ applications to access the Kinect sensor‘s microphone array is through the MSRKinectAudio Microsoft® DirectX® media object (DMO). A DMO is a standard COM object that can be incorporated into a Microsoft DirectShow® graph or a Windows Media® Foundation topology. The Kinect for Windows® Software Development Kit (SDK) Beta includes an extended version of the Windows microphone array DMO—referred to here as the MSRKinectAudio DMO—to support the Kinect microphone array. The MFAudioFilter sample shows how to capture an audio stream from the Kinect sensor‘s microphone array by using the MSRKinectAudio DMO in filter mode in a Windows Media Foundation topology. This document is a walkthrough of the MFAudioFilter sample. To prepare for and understand this walkthrough, read ―MicArrayEchoCancellation Walkthrough,‖ which describes how to use the MSRKinectAudio DMO in source mode. Note Media Foundation is COM-based, so this document assumes that you are familiar with the basics of how to use COM objects and interfaces. You do not need to know how to implement COM objects. For the basics of how to use COM objects, see ―Programming DirectX with COM‖ on the Microsoft Developer Network (MSDN®) website. That MSDN topic is written for DirectX programmers, but the basic principles apply to all COM-based applications. Program Basics MFAudioFilter is installed with the Kinect for Windows Software Development Kit (SDK) Beta samples in %KINECTSDK_DIR%\Samples\KinectSDKSamples.zip. MFAudioFilter is a C++ console application that is implemented in MFAudioFilter.cpp. The basic program flow is as follows: 1. Initialize and configure the MSRKinectAudio DMO. 2. Incorporate the DMO into a Media Foundation topology. 3. Record an audio stream from the microphone array and write the data to a Windows Media Audio (.wma) file. The following is a lightly edited version of the MFAudioFilter output: Recording using Media Foundation Sound output will be written to file: C:\Code_Projects\NUI\samples\unmanaged\MFAudioFilter\MFAudioFilter.wma MESessionTopologySet Media session event: 112 MESessionTopologyStatus: MF_TOPOSTATUS_READY MESessionStarted Recording. Press 's' to stop. MESessionStopped. MESessionClosed MFAudioFilter Walkthrough: C++ – 3 Beginning with ―MESessionTopologySet‖ in this output, most of the output represents various Media Foundation events. The exception is the line beginning with ―Recording,‖ which informs the user how to stop the session. The remainder of this document walks you through the application. Note This document includes code examples, most of which have been edited for brevity and readability. In particular, most routine error correction code has been removed. For the complete code, see the MFAudioFilter sample. Hyperlinks in this walkthrough refer to content on the MSDN website. Create and Configure the MSRKinectAudio DMO The application‘s entry point is _tmain, which manages the overall program execution with most of the details handled by private functions. The first step is to create an instance of the MSRKinectAudio DMO, as follows: int __cdecl _tmain(int argc, const TCHAR ** argv) { HRESULT hr = S_OK; CoInitialize(NULL); IMediaObject* pDMO = NULL; IPropertyStore* pPS = NULL; IMFTransform* pMFT = NULL; SetPriorityClass (GetCurrentProcess(), HIGH_PRIORITY_CLASS); CoCreateInstance(CLSID_CMSRKinectAudio, NULL, CLSCTX_INPROC_SERVER, IID_IMediaObject, (void**)&pDMO); pDMO->QueryInterface(IID_IPropertyStore, (void**)&pPS); pDMO->QueryInterface(IID_IMFTransform, (void**)&pMFT); ... } MFAudioFilter first calls the SetPriorityClass function and sets the process‘s priority to HIGH_PRIORITY_CLASS. This helps ensure that the microphone is not preempted during the capture process. MFAudioFilter calls the CoCreateInstance function to create an instance of the MSRKinectAudio DMO and obtain its IMediaObject interface, which supports the methods that control the DMO. MFAudioFilter then calls the DMO‘s QueryInterface method to obtain the following two additional interface pointers: The IPropertyStore interface provides access to the DMO‘s property store, which contains a set of key-value pairs. You configure the DMO by setting the appropriate keys. The IMFTransform interface is used to incorporate the DMO into a Media Foundation topology, as discussed later in this document. MFAudioFilter Walkthrough: C++ – 4 The next section of _tmain is a series of code blocks that configure the DMO by assigning values—as PROPVARIANT structures—to the appropriate property keys. The general procedure for setting a key is as follows: 1. Declare a PROPVARIANT structure and initialize it by calling the PropVariantInit function. 2. Specify the key‘s data type by assigning the appropriate VARENUM value to PROPVARIANT.vt. For example, VT_I4 specifies a 4-byte signed int. 3. Assign a value to the structure‘s value member. The member name depends on PROPVARIANT.vt. For VT_I4, the corresponding value member is PROPVARIANT.lVal. 4. Call the DMO‘s IPropertyStore::SetValue