Amazon Alexa Voice Services based on i.MX Applications Processors
Paulo Knirsch Product Marketing Manager
May 2018 | Session AMF-SMC-T3021
Company Public – NXP, the NXP logo, and NXP secure connections for a smarter world are trademarks of NXP B.V. All other product or service names are the property of their respective owners. © 2018 NXP B.V. AGENDA
• Voice Market Overview • NXP Alexa Voice Services
− AVS Development Kits
− AVS Solution Integrators • Other Voice Solutions
COMPANY PUBLIC 1 Voice Market Trends
COMPANY PUBLIC 2 COMPANY PUBLIC 3 Market Sizing: Intelligent Home Speaker
Source: Strategy Analytics
COMPANY PUBLIC 4 Voice becoming the key interface in the smart home market
• Smart speakers are driving voice assistants in the home • Google will eventually take smart speaker lead • Google Assistant can leverage the wider Google device & service ecosystem in the home
COMPANY PUBLIC 6 Echo/Assistant to 3rd Party Brands
• In 2022 more than half of Intelligent Speaker shipments will be from third party hardware manufacturers who license the virtual assistant technology.
• Connected audio brands such as Sonos, Bose, Harman Kardon and Denon will be key players in this market as they develop partnerships with voice platform providers.
• 1st party = Amazon Alexa (Echo, Dot) & Google Assistant
COMPANY PUBLIC 7 The Smart Home Revolution: Step 4 Smart Home
Voice Enabled Lamp Light Switch Dimmers Alexa Voice Alexa Service (AVS) Skill Alexa, turn Bedroom on the light
Alexa, turn on the light Alexa Home Hub Skill
Voice Enabled Kitchen Coffee Machine
COMPANY PUBLIC 8 The Smart Home Revolution: Step 5 Smart Home
SMART HOME SMART CITY
COMPANY PUBLIC 9 Alexa Voice Services
COMPANY PUBLIC 10 1 2
3
https://developer.amazon.com/alexa-voice-service/buildCOMPANY PUBLIC 11 Alexa Dev Kits from NXP
COMPANY PUBLIC 12 AVS Certified Development Kits
• AVS launched the Qualcomm, Amlogic and Allwinner one chip solutions at CES 2018
• We are currently in the process of getting the Pico 7D/Audio HAT + DSPConcepts solution certified
COMPANY PUBLIC 13 Arrow kit for Amazon AVS with Synaptics FE and NXP i.MX
Hardware Beamforming “Alexa” Trigger Phrase/ Audio Voice Dereverberation Hot Word Detection Codec/ Assistant AEC ASR ADC 2 or 4 Information Microphone and/or array Action
Front End (FE) (for “far-field”) Back End Processing In the Cloud Conexant Codec and DSP (“Ecosystem Play”) NXP i.MX7D / i.MX 8M
AEC = Acoustic Echo Cancellation; Important distinction: Far field (across the room) vs Near field (ie. on a headset)
COMPANY PUBLIC 14 Arrow Kit for AVS with Synaptics 2Mic and NXP PICO-PI-IMX7
• Kit includes: − PICO-PI-IMX7 i.MX7D development board . Pre-flashed with Yocto Linux BSP . AVS Device SDK . Sensory wake word
− CX20921 evaluation board, pre-flashed with firmware − Microphone module with two omnidirectional mics − Microphone holder board
This kit can be ordered through Arrow https://www.arrow.com/en/products/conexantnxp2micforavs/arrow-development-tools
COMPANY PUBLIC 15 Synaptics AudioSmart DSP • Superior End User Experience – 360 degree far-field pick-up Result • Cost Effective – Only 2 or 4 microphones • Fast Time-to-market – Production-ready solution and systems expertise
Synaptics SSP solves the limitations of current noise suppression technologies, delivering superior price/performance
COMPANY PUBLIC 16 NXP Yocto Linux Image & Source meta-avs-demos (Yocto meta layer) https://source.codeaurora.org/external/imxsupport/meta-avs-demos
• Complementary to the NXP BSP release for i.MX
• Allows to build the AVS SDK or install it at runtime.
Additional package to support AVS_SDK - Curl with http2 support - gstremaer 1.10.4 - portaudio
- Includes set of scripts to install Sensory WakeWord at runtime for easier credentials setup.
- Currently for PicoPi but easily customizable for use with other boards
- Binary image posted to NXP.com here
COMPANY PUBLIC 17 Sensory TrulyHandsfree with Alexa
• TrulyHandsfree − Wakeword engine recognizes “Alexa” trigger − Seamless trigger to commands: . Alexa, what time is it? (no beep or pause needed!)
− Small footprint high accuracy engine capable of much more . Speaker Verification & ID . Noise Robust Phrasespotted Command Sets . Custom Wake Words and End User Defined Wakewords
• Alexa by Amazon − NXP (platform) with TrulyHandsfree sends audio after trigger to AVS − Alexa responds verbally − Wide domain coverage for questions and music playing − Household controls and >10,000 “skills” supported by leading manufacturers
COMPANY PUBLIC 18 Alexa demo with i.MX8M – Hardware
Evaluation Kit for the i.MX 8M Applications Processor
− CX20921 evaluation board − Microphone module with two omnidirectional mics
COMPANY PUBLIC 19 AVS Certified Audio Front End Dev Kits
• The Alexa Premium dev kit was launched at CES and will have 2 SoC’s Raspberry Pi and the Pico i.MX7D
• The Premium dev kit replaces the 7 Mic Far Field that also has a Pico i.MX7D variant
• This uses an Intel DSP that runs the Alexa wake word
• Amazon distributes this to select tier one customers only
COMPANY PUBLIC 20 Voice Front End Solutions- i.MX Current Engagement
Partner Front End Trigger Phrase Amazon Demo Mass Market Device Part (Alphabetical Microphones Interfaces Availability Order) Processing Detection Approved Support No.
2 Arrow CX20921 Launched USB Synaptics Hardware Sensory Yes I2S Now 4 SPI Arrow CX20924
1 Microsemi Hardware Sensory Yes I2S In planning Future Potentially Q2/3 3
DSP 2 to 7 software Sensory Planned API Now TBD Software Front Concepts End Sugr 1-2 Software Sensory Yes API Now TBD
COMPANY PUBLIC 21 Optimized Voice Control
Beamforming Trigger Phrase/ Embedded Voice Cloud Voice Dereverberation Hot Word Control (ASR) Assistant AEC Detection Optional
Digital Information Microphone and/or array, one or Action more Front End Back End Processing e.g. Sensory In the Cloud (“Ecosystem Play”) i.MX7Dual with software
NXP is working with DSP Concepts on an i.MX software Front End solution
COMPANY PUBLIC 22 23 DSP Concepts (DSPC) Introduction
• Based in Santa Clara CA • Established provider of audio processing software • Developed a software solution for far field voice front end for i.MX7D, and will be ported to i.MX 8M: − Supports 2 to 7 microphone arrays − 6 microphone algorithm is already Amazon approved for AVS − Outperforms or matches hardware front end alternatives − Designed 8 microphone far field voice input board, configurable for 2, 4, 6, 7 microphone arrays − Integrated with NXP’s Alexa/AVS i.MX7D based solution
COMPANY PUBLIC 23 DSP Concepts – Soft DSP Trigger Word Performance
Wake Word Barge In Integration
Acoustic Echo AVS integration Canceler
Loudspeaker Ambient Noise Processing Reduction
Beamforming & Far-Field
• Complete / Turnkey • Enables non-skilled customers • Allows differentiation • Reconfigurable / Tunable • Exceptional far field performance
COMPANY PUBLIC 24 Amazon Alexa & Google Assistant Reference Platform
COMPANY PUBLIC 26 Stream810 Module
Stream810 Module S810B 2.2 W S810B 4.4 W MPU MCIMX7D2DVK12SC RAM: DDR3 SDRAM 256 MB 512 MB Flash: NAND 256 MB 512 MB Wi-Fi 802.11 AC dual antenna (2x2): Marvell 88W8897 RMII and 100 Mbit Phy Ethernet RMII SMSC LAN8720A PMIC AXP152
COMPANY PUBLIC 27 Alexa Voice Services Certified System Integrators
COMPANY PUBLIC 29 AVS Ecosystem Components
1 2 3 4 5
Microphone Audio Codec i.MX 7D Array with DSP Yocto Linux AVS SDK Wake Word
1 to 8 Mics Hardware DSP Wake Word Royalty Acoustic design AVS Certification Production
Microphone All Far Field Audio i.MX 7D Array runs on i.Mx Yocto Linux AVS SDK Audio Algorithms Wake Word Wake Word & Audio Algorithms Royalty COMPANY PUBLIC 30 AVS Certified System Integrators
COMPANY PUBLIC 31 AVS Certified System Integrators – NXP Engagement
Company Location Current DSP Key Value NXP engagement Soc Partners SuGR Shenzhen Intel, Ingenic Soft Sugr Sense SW solution w soft DSP > 10 Ported SW stack to Pico products certified by AVS, low power, low 7D cost - Linux & Android supported Creoir Finland NXP Synaptics Audio & acoustics design house offer Developed 7D based Far complete solution, voice tuning/ Field solution certification
COMPANY PUBLIC 32 Creoir Far Field Voice HAT
• Daughterboard plugs directly into PicoPi baseboard • AVS for i.MX6UL, i.MX7D, i.MX8M • Utilizes same Yocto image • Short design cycle 40-pin Header • Creoir are a certified AVS System Integrator with experience in audio design/tuning and I2C Power I2S working with customer in the AVS product Mgmt Header certification process • Creating companion set up app CX2029 1
Mic A Mic B
COMPANY PUBLIC 33 Creoir Far-Field Voice Solution
Fast and flexible solution supporting major AI platforms e.g. Alexa Voice Services (AVS) • Synaptics 2mic/4mic AudioSmart™ Voice Processor for far-field speech recognition • NXP i.MX7D applications processor with versatile interfaces • Yocto Linux OS and customizable companion app
For more information, see announcement Specification in Appendix
COMPANY PUBLIC 34 Creior Voice Integration Services
Creoir offers far field testing in a dedicated test room • Fully automated test environment • Noise suppression • Acoustic Echo Cancellation • Barge-in, reverberation • Wake –word performance
COMPANY PUBLIC 35 Sugr
COMPANY PUBLIC 36 Sugr
COMPANY PUBLIC 37 Google Voice Assistant
COMPANY PUBLIC 38 NXP + Goertek: Android Things Smart Speaker Reference Design
2Mic Array Touch Button Google-managed secure, scalable full-feature platform, offering full integration with Google Assistant and Cast. Offers convenient OTA updates for easy GVA Indicator software platform management LEDs
Google-certified chipset with high technical Speaker performance at low power, enabling superior connectivity and unparalleled integration with
Key Technologies Key i.MX Processor Google Assistant and Cast
Logo Item Description
Built with premium Mic Array 2-Microphone array for optimum voice recognition materials USB Touch Button Core speaker functionality (Volume, Mute, Play/Pause) • Polycarbonate frame • Fabric speaker upper GA LED 4 LEDs indicating Google Assistant activation • Genuine leather strap Charger Speaker Full frequency speaker, designed by Goertek Customizations 12V, 3A Adapter. available on materials, Power Basic Specifications color, etc. USB USB2.0: used for firmware updates
COMPANY PUBLIC 40 Local Voice
COMPANY PUBLIC 42 Local Voice Enablement – Snips
COMPANY PUBLIC 43 www.nxp.com NXP, the NXP logo, and NXP secure connections for a smarter world are trademarks of NXP B.V. All other product or service names are the property of their respective owners. © 2018 NXP B.V.