Adapting Chronotron Speed Changer to Windows Phone 8.1 Meet the App Application Architecture Outlook After a Quick Code Review Universal Apps with Visual Studio Implementation Approach ◦ User Interface and Navigation ◦ Custom Controls ◦ ViewModel ◦ Native Components Project Ne10 Other Considerations Conclusion Chronotron Speed Changer is an audio/video speed changer app. It allows you to manipulate tempo and pitch independently during playback, i.e. change the tempo of audio and video files without affecting pitch (time stretching) and/or change pitch without affecting tempo (key change).
http://www.chronotron.com The implementation is based on the MVVM pattern. The audio/video processing algorithms are implemented using native Media Foundation components.
User Interface ViewModel
Native MF Components The easy ◦ Simple UI: one XAML page, one Settings flyout The challenging ◦ Three custom controls ◦ Native components Each project in the solution is split in three: ◦ A Windows Store Project containing code specific to the Windows Store app ◦ A Windows Phone Project containing code specific to the Windows Phone app ◦ A Shared Project containing code shared across the above two projects Specific or Shared code? ◦ User Interface and Navigation ◦ Custom Controls ◦ ViewModel ◦ Native Components ◦ Other Implementation Approach: Specific Code ◦ Limited screen real state ◦ Major differences in navigation approach and UI behavior (e.g. FileOpenPicker) ◦ Some XAML code could be copied over from the Windows Store project, though Implementation Approach: Shared Code ◦ Minor differences in XAML templates - not all theme resources are available Change the control templates to use other resources and styles ◦ Minor differences in call sequence – e.g. on Windows Phone property binding may happen before OnApplyTemplate Program defensively to accommodate for both XAML implementations No #ifdef were required Implementation Approach: Shared Code ◦ Written in C# ◦ Only one “#ifdef” construct was needed StorageFile.GetThumbnailAsync always fails on Windows Phone 8.1 ◦ Part of the application settings viewmodel is specific in order to allow different default settings on WP Implementation Approach: Shared Code ◦ Compiling the native code for Windows Phone was relatively straightforward The actual processing algorithms were already developed in portable C++; x86-specific optimizations had been implemented as an extension Media Foundation is available on WP 8.1 A few memory management APIs aren't available on WP (e.g. CreateFileMappingFromApp), so a few #ifdefs were required Real-time performance was the main area of work ◦ The good VC++ does a good job at vectorizing simple loops ◦ The not-so-good Memory constraints Solution: Limit memory-hungry functionality, like reverse playback Trigonometric functions in the MSVC Runtime Library are slow on ARM Solution: Use lookup tables and other techniques Video processing on the CPU is prohibitively slow Solution: Use DXGI for video processing (also for the benefit of the Windows Store app!) The Windows Store version relies heavily on x86 SIMD optimization, in particular for the FFT Solution: Rely on Project Ne10. What is it? A library of well-known algorithms optimized for the ARM architecture Developed primarily for Android, but can be compiled with VC++ with minimal changes Free for non-commercial and commercial usage Available at http://projectne10.org
Example: 32-bit floating point real FFT/IFFT runtime on Qualcomm Snapdragon 400, average over 10K calls:
Number of Samples My own FFT written in C Ne10 FFT using NEON intrinsics 2048 0.176ms 0.159ms
4096 0.393ms 0.349ms
The 10-12% improvement is very welcome considering the app performs up to 3 FFT per channel per frame! The Windows Store does not support specifying a trial period for Windows Phone apps ◦ A trial mode offering limited functionality had to be implemented Testing ◦ Tested on a Nokia Lumia 630 The Windows Store app could be effectively used as starting point to develop the Windows Phone app ◦ Most of the code could be shared (with a few caveats), even native code! Meet the new app