@RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Quality of Service for…. mobile, fixed, roaming, billing 5D\PRQG:X $VFRP 7HO )D[ UD\PRQGZX#LQIUDV\VDVFRPFK @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Speech evaluation : - where did we start from - the product / applications today - the speech quality evaluation method - ITU - T activities - calibrations @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV In 1993, Ascom delivered the ….. first system in the world to use real speech for speech quality evaluation @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Motivation : RxQual = speech quality 5 4.5 4 3.5 3 2.5 speech quality 2 1.5 1 01234567 AvgQualIF @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Ascom speech evaluation round the world Mobilkom P.T. Satelindo Telefonica Moviles Max.mobil Telkomsel Airtel Mobistar Esat Digifone Telia Radiomobil Omnitel MOPT Lucent Tele Danmark Mobile ONPT Marocco Sonofon Swisscom Mobilix Netcom Etisalat France Telecom Mobile Isla Communic. Nortel Matra Cellular Cellnet era GSM T-Mobil Airtouch Intl. Lucent TMN Mannesmann Telecel Nokia Main Road Hutchison Pannafon MTS Westel KB Impuls Chungwa @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV End to end measurements : Speech quality (excellent, good, medium, poor, bad ) Ping Pong Robotic voice Echo Signal level Silence Speech samples in use : English, French, German, Mandarin, Cantonese @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Products on the market : mobile network measurements --- QVoice fixed network measurements ------ QNet @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QVoice Principles of operation Q VM Indoor QVS QVM OutdoorISD N / PSTN QVoice 1 ...4Calls, Speech Sam ples QVP QVS QVoice Cellular QVM N e tw o rk 1 ...4 GPS Antenna M easurem ent Data @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Purposely designed QVoice hardware for : Indoor measurements Drive tests @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QVoice Results analysis : @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QNet end-to-end network performance measurement for PSTN/ISDN/GSM switching quality transmission quality charging quality from a customer’s point of view @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QNet: different applications simultaneously Q-Statistics Q-Monitoring Q-Analysis @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QNet system architecture QNet CU QNet PU RS232 S , a/b 0 Client 1 Server TSU ISDN / PSTN S0 LAN Client n RS232S0, a/b RS232 S0, a/b QNet PU QNet PU @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QNet supports complex test scenarois @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Switching parameter Parameter ISDN PSTN teleservice, bearer capability Y Y time, date Y Y duration Y Y call progress indicator Y signaling tone measurement Y d-channel protocol analysis Y cause/location value Y unsuccessfull call attempt Y Y connection setup delay Y Y connection release delay Y unsuccessfull release attempt Y Y premature release Y Y @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Transmission parameter Parameter ISDN PSTN loop transfer delay Y BERT (G.821: ES, SES, DM) Y unweighted noise (O.41) Y Y weighted noise (O.41) Y Y impulsive noise (O.71) Y Y notch noise Y Y envelope delay distortion Y Y interruptions (O.61, O.62) Y Y echo: level/delay (G.122, sine-burst) Y Y attenuation (on 4 frequencies) Y Y speech quality measurement (Ascom PACE) Y Y @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Charging parameter Parameter ISDN PSTN charging out of rating Y Y initial charge out of rating Y Y no charging Y Y CDR verification Y Y @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV PACE : a robust objective speech quality assessment testing wireless & wired networks in service @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV 3 types of objective methods : Comparison of transmitted signal and a reference Absolute Estimation without known reference e.g. INMD Transmission Rating from knowledge of network e.g. E model PACE is a comparison method @RBNL Sound Pressure Signal Source Produced by Talker Loss Between Microphone and Earphone 4XDOLW\RI6HUYLFH Room Noise Speech Source Coding Coder Attenuation Transmission Parameters Delay / Echo IRUZLUHG ZLUHOHVVQHWZRUNV Distortions Transmission Channel Time-Variation Noise Room Noise Decoder Loss Between Earphone and Microphone Coded Signal Transmission Path Between Earphone and Eardrum @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV PACE Principle Source Speech Signal Quality Time align. Pre- Post- Assessment Level equal. processing processing Coded Signal @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV PACE Preprocessing Coded Signal Hamming DFT Source Window | | Signal ToTo Critical Band Time Frequency AssessmentAssessment Filtering Spreading Spreading @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV PACE Assessment Similarity FromFrom calculation PreprocessingPreprocessing for Similarity fourth calculation energy for Similarity Similarity level To Post third Weighting calculation processing energy dependent for //// Similarity level on second calculation energy energy for level level first energy level @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV PACE Postprocessing Maping Auditory distance to the MOS x MOSpMOSp MOS X MOS = ax3 + bx2 + cx + d @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Codecs and Systems used to calibrate PACE G.711 64 kbit/s PCM G.726 32 kbit/s ADPCM IS-54 8 kbit/s American Cellular GSM-FR GSM Full Rate G.729 8 kbit/s CS-ACELP G.728 16 kbit/s Low-Delay Code Excited Linear Prediction JDC 3.45 kbit/s Japanese Digital Cellular (Half Rate) DCME Digital Circuit Multiplication Equipment: Quad-Coder, DTX-240 (24 kbit/s variable) IS-9x CDMA 8 kbit/s Code Division Multiple Access Multiple Transcodings, Random -Burst frame erasure, Random bit errors Circuit noise, Comfort noise, etc. @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Calibration results presented to …. ITU-T SG 12 Q 13 Speech Quality Expert Group (SQEG) 2nd international workshop on Advances in Objective Speech Quality Assesment September 98 , UK @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Results of ITU-T Supplem. 23 test data, Exp.1 5 ρ = 0.955 4.5 4 3.5 3 MOS 2.5 2 1.5 1 1 1.5 2 2.5 3 3.5 4 4.5 5 PACE Experiment 1: Interworking with fixed and mobile codecs (G.726, G.728 …). Multiple encodings by tandeming with other coding standards. @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV Correlations between different Algorithms and MOS Experime LanguageCand. 1 Cand. 2 Cand. 3 Cand. 4 Cand. 5 PACE Exp1 French 0.94 0.92 0.82 0.93 0.96 0.94 Exp1 Japanese 0.94 0.86 0.87 0.94 0.93 0.94 Exp1 English 0.96 0.83 0.89 0.92 0.94 0.97 Exp2 French 0.95 0.37 0.94 0.97 Exp2 Japanese 0.93 0.96 0.41 0.91 0.95 Exp2 English 0.94 0.96 0.47 0.9 0.97 Exp3 French 0.74 0.74 0.74 0.75 0.94 Exp3 Italian 0.95 0.71 0.86 0.46 0.94 Exp3 Japanese 0.91 0.85 0.88 0.55 0.92 Exp3 English 0.82 0.85 0.82 0.65 0.93 @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QVoice (mobile) QNet (fixed) PACE speech evaluation testing of Interconnected networks @RBNL4XDOLW\RI6HUYLFHIRUZLUHG ZLUHOHVVQHWZRUNV QoS in an Interconnected Environment trtrtzrtzrtttttttttttttt Server Client 1 Client 2 Client n Applications QVoice QNet QNet QVoice LAN BRA PSTN / ISDN GSM BRA, PRA, a / b MSC BSC BTS PSTN / ISDN Cu st o m e r GSM BRA, a / b BRA, a / b GSM PU PU PU.