2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems
6DOVD$VVW %HDW &RXQWLQJ 6\VWHP (PSRZHUHG E\ 0RELOH 'HYLFHV WR $VVLVW 6DOVD 'DQFHUV
Abstract²'DQFLQJ LV DOZD\V FKDOOHQJLQJ HVSHFLDOO\ IRU EHJLQ QHUV ZKR PD\ ODFN VHQVH RI UK\WKP 6DOVD DV D SRSXODU VW\OH RI GDQFLQJ LV HYHQ KDUGHU WR OHDUQ GXH WR LWV XQLTXH RYHUODSSHG UK\WKPLF SDWWHUQV PDGH E\ GLIIHUHQW /DWLQ LQVWUXPHQWV HJ &ODYH VWLFNV &RQJD GUXPV 7LPEDOH GUXPV WRJHWKHU ,Q RUGHU WR GDQFH LQ V\QFKURQL]DWLRQ ZLWK WKH 6DOVD EHDWV WKH EHJLQQHUV DOZD\V QHHG SURPSWV HJ EHDW FRXQWLQJ YRLFH WR UHPLQG WKHP RI WKH EHDW WLPLQJ 7KH WUDGLWLRQDO ZD\ WR JHQHUDWH WKH 6DOVD PXVLF ZLWK EHDW FRXQWLQJ YRLFH SURPSWV UHTXLUHV SURIHVVLRQDO GDQFHUV RU PXVLFLDQV WR FRXQW 6DOVD EHDWV PDQXDOO\ ZKLFK LV *HQHUDWLQJUHDOWLPHDVVLVWHGFRXQWLQJYRLFHHQHUDWLQJUHDOWLPHDVVLVWHGFRXQWLQJYRLF RQO\ SRVVLEOH LQ GDQFH VWXGLRV $GGLWLRQDOO\ WKH H[LVWLQJ PXVLF EHDW WUDFNLQJ VROXWLRQV FDQQRW ZHOO FDSWXUH WKH 6DOVD EHDWV GXH WR +HDGSKRQH LWV LQWULFDF\ RI UK\WKPV ,Q WKLV ZRUN ZH SURSRVH D PRELOH GHYLFH HQDEOHG EHDW FRXQWLQJ V\VWHP SalsaAsst ZKLFK FDQ SHUIRUP UK\WKP GHFLSKHULQJ DQG ¿QHJUDLQHG 6DOVD EHDW WUDFNLQJ WR DVVLVW 6DOVD GDQFHUV ZLWK EHDW FRXQWLQJ YRLFHYLEUDWLRQ SURPSWV 7KH SURSRVHG V\VWHP FDQ EH XVHG FRQYHQLHQWO\ LQ PDQ\ VFHQDULRV ZKLFK FDQ QRW RQO\ KHOS 6DOVD EHJLQQHUV PDNH DFFHOHUDWHG OHDUQ LQJ SURJUHVV GXULQJ SUDFWLFH DW KRPH EXW DOVR VLJQL¿FDQWO\ UHGXFH SURIHVVLRQDO GDQFHUV¶ HUURUV GXULQJ WKHLU OLYH SHUIRUPDQFH 7KH GHYHORSHG 6DOVD EHDW FRXQWLQJ DOJRULWKP KDV WKH FDSDELOLW\ WR WUDFN EHDWV DFFXUDWHO\ LQ ERWK UHDOWLPH DQG RIÀLQH PDQQHUV 2XU H[WHQVLYH WHVWV XVLQJ 40 6DOVD VRQJV XQGHU 8 HYDOXDWLRQ PHWULFV )LJ ,OOXVWUDWLRQ RQ KRZ 6DOVD$VVW DVVLVWV 6DOVD GDQFHUV GHPRQVWUDWH WKDW SalsaAsst FDQ DFFXUDWHO\ WUDFN WKH EHDWV RI 6DOVD PXVLF DQG DFKLHYH PXFK EHWWHU SHUIRUPDQFH FRPSDULQJ WR H[LVWLQJ EHDW WUDFNLQJ DSSURDFKHV RXU HDUV ,Q RUGHU WR GDQFH LQ V\QFKURQLVDWLRQ ZLWK WKH 6DOVD EHDWV 6DOVD GDQFHUV HVSHFLDOO\ IRU EHJLQQHUV DOZD\V QHHG WKH , ,1752'8&7,21 EHDW FRXQWLQJ YRLFH WR UHPLQG WKHP RI WKH WLPLQJV RI EHDWV 'DQFLQJ LV QRW RQO\ D SHUIRUPDQFH DUW VKRZHG RQ WKH VWDJH +RZHYHU WKH WUDGLWLRQDO ZD\ WR JHQHUDWH WKH 6DOVD PXVLF ZLWK E\ SURIHVVLRQDO GDQFHUV EXW D QRUPDO SK\VLFDO H[HUFLVHWKHUDS\ EHDW FRXQWLQJ YRLFH UHTXLUHV SURIHVVLRQDO GDQFHUV RU PXVLFLDQV DQG VRFLDO DFWLYLW\ IRU WKH JHQHUDO SXEOLF ZKLFK SOD\V DQ LP WR FRXQW WKH EHDWV PDQXDOO\ DQG V\QWKHVL]H WKH EHDW FRXQWLQJ SRUWDQW VRFLDO UROH QRZDGD\V ,Q HYHU\ KXPDQ FXOWXUH SHRSOH YRLFH ZLWK WKH 6DOVD PXVLF ZKLFK LV LQHI¿FLHQW DQG QRW DEOH XVXDOO\ KDYH VRPH NLQGV RI PXVLF ZLWK D UHJXODU EHDW WKDW WR EH DSSOLHG LQ ODUJHVFDOH 6DOVD VRQJV 7KHUHIRUH D V\VWHP HOLFLWV V\QFKURQLVHG ERG\ PRYHPHQW LH GDQFLQJ +RZHYHU WKDW FDQ DXWRPDWLFDOO\ FRXQW 6DOVD EHDWV ZLWKRXW UHTXLULQJ GDQFLQJ LV DOZD\V GLI¿FXOW HVSHFLDOO\ IRU EHJLQQHUV GXH WR WKHLU SURIHVVLRQDO GDQFHUV RU PXVLFLDQV LV KLJKO\ GHVLUDEOH VHOIFRQVFLRXVQHVV RU ODFNLQJ VHQVH RI UK\WKP ,PSRUWDQWO\ 6RPH SUHYLRXV VWXGLHV >@ >@ >@ >@ >@ KDYH DOUHDG\ EHDW SHUFHSWLRQ DQG V\QFKURQLVDWLRQ DUH PHQWDOO\ GHPDQGLQJ EHHQ FRQGXFWHG RQ WUDFNLQJ EHDWV DXWRPDWLFDOO\ E\ DQDO\]LQJ IRU SHRSOH ZKR XQOLNH PXVLFLDQV KDYH OLWWOH SULRU H[SHULHQFH PXVLF DFRXVWLF VLJQDOV +RZHYHU EHFDXVH RI WKH LUUHJXODU LQ EHDW RU UK\WKP GHWHFWLRQ >@ 6DOVD UK\WKPLF SDWWHUQV PDGH E\ GLIIHUHQW /DWLQ LQVWUXPHQWV 6DOVD LV D SRSXODU IRUP RI VRFLDO GDQFHV ZLWK RULJLQV LQ VLPXOWDQHRXVO\ WKH EHDWV WKHVH DSSURDFKHV GHWHFWHG DUH XVXDOO\ &XED D PDMRU FURVVURDG RI 6SDQLVK (XURSHDQ DQG $IULFDQ QRW DW WKH FRUUHFW PHWULFDO OHYHO RI 6DOVD EHDWV LH FXOWXUHV 'LIIHUHQW IURP RWKHU W\SHV RI GDQFHV ¿QGLQJ EHDWV )RU LQVWDQFH WKH PHWULFDO OHYHOV RI WKHVH DSSURDFKHV LQ 6DOVD PXVLF LV HYHQ WRXJKHU GXH WR WKH LQWULFDFLHV RI 6DOVD GHWHFWHG PD\ EH DW KDOI PHWULFDO OHYHO RI 6DOVD EHDWV LH PXVLF 8QOLNH RWKHU W\SHV RI GDQFH PXVLF ZKLFK XVXDOO\ KDYH RU RU DW WZLFH PHWULFDO OHYHOV RI 6DOVD EHDWV LH RQH GRPLQDQW UK\WKP HJ EDVV RU GUXP 6DOVD PXVLF KDV 7KDW LQGLFDWHV WKDW WKHVH DSSURDFKHV PDQ\ RYHUODSSHG UK\WKPLF SDWWHUQV SURYLGHG E\ GLIIHUHQW /DWLQ ZRXOG OHDG WR D PDVV RI RYHUXQGHU GHWHFWLRQV IRU WUDFNLQJ LQVWUXPHQWV HJ &ODYH VWLFNV &RQJD GUXPV 7LPEDOH GUXPV 6DOVD EHDWV ,Q DGGLWLRQ DQRWKHU GLUHFWLRQ LV XVLQJ WKH PRWLRQ SOD\LQJ DW WKH VDPH WLPH +HQFH LW LV FKDOOHQJLQJ WR ¿QG VHQVLQJ GHYLFH HJ 0RWLRQ $QDO\VLV )DOFRQ >@ RU .LQHFW >@ 6DOVD EHDWV IURP WKHVH RYHUODSSHG UK\WKPLF SDWWHUQV MXVW E\ WR REWDLQ WKH ' PRWLRQ RI D GDQFH PDVWHU DQG DQDO\]H WKH 2155-6814/17 $31.00 © 2017 IEEE 81 DOI 10.1109/MASS.2017.25 PRWLRQ GDWD WR JHW WKH WLPLQJV RI PRYLQJ KLVKHU VWHSV DQG • ([WHQVLYH WHVWV XVLQJ 40 6DOVD GDQFH VRQJV XQGHU 8 WKHQ UHPLQG XVHUV RI WKHVH WLPLQJV XVLQJ YLEUDWLRQ RU FRXQWLQJ HYDOXDWLRQ PHWULFV VKRZ WKDW RXU V\VWHP FDQ DFKLHYH YRLFH +RZHYHU WKHVH DSSURDFKHV KDYH WR LQYROYH SURIHVVLRQDO PXFK EHWWHU EHDW WUDFNLQJ SHUIRUPDQFH FRPSDULQJ WR WKH GDQFHUV DQG QHHG GHGLFDWHG PRWLRQ VHQVRUV ZKLFK UHVWULFWV LWV H[LVWLQJ EHDW WUDFNLQJ VROXWLRQV XVDJH VFHQDULRV DQG UHTXLUHV DGGLWLRQDO FRVW 7KH UHVW RI WKLV SDSHU LV RUJDQL]HG DV IROORZV 6HFWLRQ ,, 7R DGGUHVV WKHVH LVVXHV ZH LPSOHPHQW D PRELOH GHYLFH GLVFXVVHV UHODWHG ZRUN 6HFWLRQ ,,, LQWURGXFHV VRPH EDVLF EDVHG EHDW FRXQWLQJ V\VWHP SalsaAsst WR DVVLVW 6DOVD GDQFHUV NQRZOHGJH DERXW 6DOVD DQG WKH RYHUYLHZ RI RXU SURSRVHG ZLWK DVVLVWHGFRXQWLQJ YRLFHYLEUDWLRQ SURPSWV YLD PRELOH GH V\VWHP 6HFWLRQ ,9 DQG 6HFWLRQ 9 GHVFULEH OfÀine Beat YLFHV HJ VPDUWSKRQHV 7KH SURSRVHG SalsaAsst FDQ SHUIRUP Counting Mode DQG Online Beat Counting Mode WKDW DUH FRUH UK\WKP GHFLSKHULQJ DQG ¿QHJUDLQHG EHDW WUDFNLQJ QRW RQO\ FRPSRQHQWV RI RXU V\VWHP UHVSHFWLYHO\ ,Q 6HFWLRQ 9, ZH KHOSLQJ 6DOVD EHJLQQHUV JHW DFFHOHUDWHG OHDUQLQJ SURFHVV GXULQJ HYDOXDWH WKH SHUIRUPDQFH RI WKH V\VWHP $QG LQ 6HFWLRQ 9,, SUDFWLFH DW KRPH DQG EH PRUH FRQ¿GHQW RQ GDQFH ÀRRU EXW ZH SHUIRUP GLVFXVVLRQ RQ WKH SURSRVHG SalsaAsst V\VWHP DOVR HOLPLQDWLQJ SURIHVVLRQDO GDQFHUV¶ HUURUV GXULQJ WKHLU OLYH )LQDOO\ ZH FRQFOXGH WKH SDSHU LQ 6HFWLRQ 9,,, SHUIRUPDQFH )LJXUH LV DQ LOOXVWUDWLRQ RQ KRZ SalsaAsst DVVLVWV GDQFHUV GXULQJ WKHLU SUDFWLFH VHVVLRQV RU RQ WKH VWDJH ,, 5(/$7(' :25. 6SHFL¿FDOO\ SalsaAsst LV DEOH WR SURFHVV 6DOVD PXVLF ¿OHV RIÀLQH DQG JHQHUDWH D QHZ PXVLF ¿OH ZKLFK KDV ERWK WKH 7KH FUX[ IRU DVVLVWLQJ LQ GDQFLQJ 6DOVD LV WR KHOS SHRSOH RULJLQDO 6DOVD PXVLF DQG LWV V\QFKURQRXV EHDW FRXQWLQJ YRLFH WR UHFRJQL]H WKH EHDWV RU WKH WLPLQJV RI PRYLQJ GDQFH VWHSV :LWK WKH KHOS RI WKH QHZ PXVLF ¿OH ZKLFK FDQ EH SOD\HG ,Q JHQHUDO WKH H[LVWLQJ VROXWLRQV XVHG WR DVVLVW GDQFHUV WR WKURXJK WKH %OXHWRRWK VSHDNHU SHRSOH PD\ IHHO PXFK HDVLHU WR UHFRJQL]H WKH EHDW WLPLQJV FDQ EH FDWHJRUL]HG LQWR WZR JURXSV OHDUQ 6DOVD GDQFH 0HDQZKLOH SalsaAsst LV DOVR DEOH WR VHQVH GDQFH PDVWHU UHOLHG DQG GDQFH PXVLF EDVHG WKH 6DOVD PXVLF YLD WKH EXLOWLQ PLFURSKRQH DQG JHQHUDWH UHDO 7KH WUDGLWLRQDO ZD\ RI OHDUQLQJ 6DOVD LV WR DWWHQG D GDQFH WLPH DVVLVWHGFRXQWLQJ YRLFH WKURXJK KHDGSKRQHV RU YLEUDWLRQ OHVVRQ LQ ZKLFK D GDQFH PDVWHU FDQ FRXQW DORQJ ZLWK WKH EHDWV SURPSWV WR UHPLQG WKH GDQFHUV RI 6DOVD EHDW WLPLQJV RI GDQFLQJ PXVLF IRU WKH EHJLQQHUV ZKR KDYH GLI¿FXOW\ ZLWK 0RUH VSHFL¿FDOO\ RXU V\VWHP RQO\ XVHV 6DOVD PXVLF DXGLR UHFRJQL]LQJ WKH EHDW WLPLQJV ,Q DGGLWLRQ VRPH DSSURDFKHV XVH VLJQDO IRU GHWHFWLQJ WKH 6DOVD EHDWV 7KH RQVHW GHWHFWLRQ GHGLFDWHG VHQVRUV WR FDSWXUH WKH PRYHPHQWV RI D SURIHVVLRQDO DOJRULWKP ¿VW DQDO\]HV WKH DXGLR VLJQDO LQ IUHTXHQF\ GRPDLQ GDQFHU WR JHW WKH FRUUHFW WLPLQJV RI PRYLQJ VWHSV )RU H[DPSOH XVLQJ 6KRUWWLPH )RXULHU 7UDQVIRUP 67)7 WR REWDLQ WKH 1DNDPXUD et al. >@ SURSRVH D GDQFH WUDLQLQJ V\VWHP ZKLFK VSHFWURJUDP DQG WKHQ FRPSXWHV 2QVHW 6WUHQJWK &XUYH 26& FDSWXUHV ' PRWLRQ RI D GDQFH PDVWHU DQG DQDO\]HV WKH %\ ORFDWLQJ WKH ORFDO PD[LPD RI VPRRWKHG 26& ZH FDQ PRWLRQ WR JHW WKH WLPLQJV RI PRYLQJ KLVKHU VWHSV ,Q RUGHU WR REWDLQ WKH RQVHW SRVLWLRQV 7R GHWHFW 6DOVD EHDWV RXU DOJRULWKP IROORZ WKH SURSHU GDQFH VWHSV ZKLOH GDQFLQJ WKH XVHUV QHHG ¿UVW HVWLPDWHV WKH 6DOVD EHDW LQWHUYDO XVLQJ RQVHW SRVLWLRQV WR ZHDU DFWLYH YLEURGHYLFHV ZKLFK FDQ JHQHUDWH YLEUDWLRQ WR DQG WKHQ WUDFN WKH EHDWV IURP RQVHWV EDVHG RQ WKH HVWLPDWHG UHPLQG XVHUV RI WKH VWHS WLPLQJV ,Q DGGLWLRQ 0LVDWR et al. >@ 6DOVD EHDW LQWHUYDO :H GHYHORS WZR DOWHUQDWLYH PRGHV OfÀine SUHVHQW D V\VWHP WKDW XVHV ERWK DFRXVWLF IHDWXUHV RI GDQFH Beat Counting Mode DQG Online Beat Counting ModeLQ PXVLF DQG VNHOHWRQ IHDWXUHV IURP PRYHPHQWV RI D GDQFH PDVWHU RXU V\VWHP IRU ÀH[LEOH XVDJH 7KH RIÀLQH PRGH FDQ SURFHVV WR WUDFN WKH EHDWV +RZHYHU WKHVH DSSURDFKHV UHTXLUH PRWLRQ DQ\ 6DOVD GDQFH VRQJ ¿OHV WR JHQHUDWH WKH EHDW FRXQWLQJ VHQVLQJ GHYLFH HJ 0RWLRQ $QDO\VLV )DOFRQ >@ .LQHFW >@ YRLFHYLEUDWLRQ SURPSWV DORQJ ZLWK WKH RULJLQDO 6DOVD PXVLF IRU FDSWXULQJ WKH PRYHPHQWV RI SURIHVVLRQDO GDQFHUV ZKLFK LV 7KH RQOLQH PRGH OHYHUDJHV WKH EXLOWLQ PLFURSKRQH RQ PRELOH FRVWO\ DQG QRW DOZD\V DYDLODEOH GHYLFHV WR UHFHLYH WKH DFRXVWLF VLJQDO RI 6DOVD PXVLF DQG )XUWKHUPRUH WKHUH DUH D IHZ H[LVWLQJ VROXWLRQV DXWRPDWL JHQHUDWH WKH UHDOWLPH SURPSWV LH EHDW FRXQWLQJ YRLFH DQG FDOO\ WUDFNLQJ GDQFH EHDWV WKURXJK DFRXVWLF VLJQDO SURFHVVLQJ YLEUDWLRQ RI GDQFLQJ PXVLF )RU LQVWDQFH WKH\ LQFOXGH EHDWWHPSODWH PHWKRG >@ %D\HVLDQ IUDPHZRUN >@ G\QDPLF SURJUDP 7KH PDLQ FRQWULEXWLRQV RI RXU ZRUN DUH VXPPDUL]HG DV PLQJ >@ >@ DQG WZRVWDWH PRGHO >@ ,Q SDUWLFXODU * IROORZV 3HHWHUV >@ XVHV EHDWWHPSODWH WUDLQLQJ WR HVWLPDWHG PXVL • 2XU SURSRVHG SalsaAsst FDQ SHUIRUP UK\WKP GHFLSKHULQJ FDO WHPSR $7 &HPJLO >@ IRUPXODWHV WHPSR WUDFNLQJ LQ D DQG ¿QHJUDLQHG EHDW WUDFNLQJ IRU 6DOVD PXVLF VLJQDO DQG %D\HVLDQ IUDPHZRUN 7KH G\QDPLF SURJUDPPLQJ DSSURDFK LV DXWRPDWLFDOO\ FRXQW 6DOVD EHDWV WR UHPLQG WKH GDQFHUV RI XWLOL]HG LQ >@ >@ WR KDQGOH EHDW WUDFNLQJ LQ WKH DXGLR VLJQDO WKH WLPLQJV RI EHDWV ZLWKRXW DQ\ DGGLWLRQDO KXPDQ WDVN 0 'DYLHV DQG 0 3OXPEOH\ >@ SURSRVH D WZRVWDWH PRGHO RU GHGLFDWHG VHQVRUV GHSOR\PHQW ZKLFK LQFOXGHV JHQHUDO VWDWH DQG FRQWH[WGHSHQGHQW VWDWH WR • :H VKRZ WKDW LW LV IHDVLEOH WR HQDEOH WKH PRELOH GHYLFH WR ¿QG WKH EHDW SRVLWLRQV IURP PXVLF VLJQDO ,Q DGGLWLRQ WKH ZRUN EHFRPH D YLUWXDO DVVLVWDQW IRU 6DOVD GDQFH RQO\ XVLQJ WKH LQ > @ LV VLPLODU WR RXUV ZKLFK DOVR FDQ UHFRJQL]H DQG FRXQW EXLOWLQ PLFURSKRQH VSHDNHUV DQG YLEUDWRU RI WKH GHYLFH EHDWV E\ SURFHVVLQJ DXGLR VLJQDO RI GDQFLQJ PXVLF +RZHYHU • :H GHYHORS D QRYHO EHDW FRXQWLQJ DOJRULWKP IRU 6DOVD WKHVH DSSURDFKHV DUH QRW DSSOLFDEOH IRU 6DOVD EHDW WUDFNLQJ PXVLF ZKLFK KDV WKH FDSDELOLW\ WR WUDFN 6DOVD EHDWV 'LIIHUHQW IURP RWKHU W\SHV RI PXVLF 6DOVD¶V UK\WKPLF PDGH DFFXUDWHO\ LQ ERWK UHDOWLPH DQG RIÀLQH PDQQHUV E\ /DWLQ LQVWUXPHQWV PD\ DSSHDU DW HYHU\ WZR 6DOVD EHDWV RU 82 %HDW &ODYH 7LPEDOH D 0XVLFDO PHDVXUH &RQJD &RZEHOO 0RQWXQR *XLU| E 'DQFLQJ PHDVXUH RI 6DOVD 3OD\ WRJHWKHU ZŝŐŚƚ ^ŚŝĨƚ ZŝŐŚƚ 6WUHVVHG 8QVWUHVVHG &ŽŽƚ tĞŝŐŚƚ &ŽŽƚ WĂƵƐĞ ĂĐŬ ƚŽ&ƌŽŶƚ &ŽƌǁĂƌĚ &ŽŽƚ )LJ 5K\WKPLF SDWWHUQV RI GLIIHUHQW /DWLQ LQVWUXPHQWV LQ 6DOVD PXVLF ^ŚŝĨƚ >ĞĨƚ tĞŝŐŚƚ >ĞĨƚ &ŽŽƚ ƚŽĂĐŬ &ŽŽƚ WĂƵƐĞ &ŽƌǁĂƌĚ &ŽŽƚ ĂĐŬ NLQGV RI PXVLF WKH EHDWV RI 6DOVD FRPH IURP YDULRXV /DWLQ F 6DOVD EHDWV FRUUHVSRQGLQJ WR 6DOVD VWHSV LQVWUXPHQWV LQFOXGLQJ &ODYH VWLFNV &RQJD GUXPV 7LPEDOH GUXPV 0RQWXQR &RZ EHOO *XLU HWF (YHU\ LQVWUXPHQW LV )LJ ,OOXVWUDWLRQ RI WKH UHODWLRQVKLS EHWZHHQ 6DOVD PXVLFDO PHDVXUH GDQFLQJ PHDVXUH DQG LWV FRUUHVSRQGLQJ 6DOVD VWHSV SOD\HG ZLWK XQLTXH EHDWV 6RPH RI WKHP DUH HYHQ SOD\HG DW VSHFL¿F KDOI EHDWV )RU LQVWDQFH &ODYHV LV SOD\HG DW DQG HYHU\ WZR EDUV LQ 6DOVD VRQJV )LJXUH KDOI 6DOVD EHDW 7KHUHIRUH WKHVH DSSURDFKHV ZRXOG OHDG WR D VKRZV DQ H[DPSOH WKDW DOO WKHVH LQVWUXPHQWV SOD\LQJ WRJHWKHU PDVV RI PLVWDNHQO\ GHWHFWHG EHDWV IRU 6DOVD PXVLF PDNHV LW GLI¿FXOW WR GHFLSKHU EHDWV IURP WKH 6DOVD PXVLF 'LIIHUHQW IURP WKH SUHYLRXV ZRUN SalsaAsst FDQ DFFXUDWHO\ 6LQFH 6DOVD PXVLF KDV D EXQFK RI GLIIHUHQW FRPELQDWLRQV RI WUDFN WKH EHDWV DW WKH FRUUHFW PHWULFDO OHYHO HPEHGGHG LQ 6DOVD WKHVH /DWLQ LQVWUXPHQWV WKH UK\WKP RI 6DOVD LV K\EULG DQG PXVLF E\ H[SORLWLQJ WKH SURSRVHG OfÀine Beat Counting DQG FRPSOH[ PDNLQJ LW KDUG WR SHUFHLYH RU SLQSRLQW E\ HDUV (YHQ Online Beat Counting DOJRULWKP 0RUHRYHU SalsaAsst OHYHU IRU WKH SURIHVVLRQDO GDQFHUV LW LV DOVR SRVVLEOH WR KDYH HUURU DJHV PRELOH GHYLFHV HJ VPDUWSKRQH WR UDLVH EHDW FRXQWLQJ MXGJPHQWV IRU 6DOVD EHDWV GXULQJ OLYH SHUIRUPDQFH ,Q RUGHU WR YRLFHYLEUDWLRQ SURPSWV WR UHPLQG XVHUV RI WKH WLPLQJV RI KHOS 6DOVD EHJLQQHUV WR DFFHOHUDWH WKHLU OHDUQLQJ SURFHVV DQG EH 6DOVD EHDWV :LWKRXW DQ\ DGGLWLRQDO KXPDQ WDVN RU VHQVRU PRUH FRQ¿GHQW RQ WKH VWDJH ZH SURSRVH SalsaAsst ZKLFK FDQ GHSOR\PHQW SalsaAsst FDQ EH XVHG FRQYHQLHQWO\ DQG ÀH[LEO\ DFFXUDWHO\ LGHQWLI\ WKH EHDWV RI DQ\ 6DOVD PXVLF DQG JHQHUDWH LQ DQ\ VFHQDULRV HJ OLYH SHUIRUPDQFH SUDFWLFLQJ DW KRPH RU UHDOWLPH EHDW FRXQWLQJ YRLFHYLEUDWLRQ SURPSWV LH EHDWV GDQFLQJ VWXGLR WR DVVLVW 6DOVD GDQFHUV WR UHPLQG RI ULJKW WLPLQJV RI PRYLQJ KLVKHU IHHW ZKLOH GDQFLQJ ,,, 35(/,0,1$5,(6 $1' 6<67(0 29(59,(: ,Q WKLV VHFWLRQ ZH ZLOO LQWURGXFH WKH SUHOLPLQDULHV RI 6DOVD B. System Overview UHODWHG PHDVXUHV DQG FKDOOHQJHV RI UHFRJQL]LQJ 6DOVD EHDWV 7KH EDVLF LGHD RI RXU V\VWHP LV WR WUDFN 6DOVD EHDWV IURP WKHQ ZH ZLOO GHVFULEH WKH V\VWHP RYHUYLHZ RI SalsaAsst 6DOVD PXVLF VLJQDO DQG JHQHUDWH EHDW FRXQWLQJ YRLFHYLEUDWLRQ A. Preliminaries SURPSWV YLD PRELOH GHYLFHV WR DVVLVW 6DOVD GDQFHUV $V LOOXV WUDWHG LQ )LJXUH RXU V\VWHP SURYLGHV WZR DOWHUQDWLYH PRGHV 2ULJLQDWLQJ LQ &XED 6DOVD GDQFLQJ LV D UHSUHVHQWDWLRQ OfÀine Beat Counting Mode DQG Online Beat Counting Mode RI D WUDGLWLRQDO /DWLQ FXOWXUH 6DOVD PXVLF DQG GDQFH KDYH IRU ÀH[LEOH XVDJH 7KH OfÀine Beat Counting Mode RI RXU GHYHORSHG VLQFH WKH ¶V IURP WKH PLJUDWLRQV RI /DWLQ SHRSOH V\VWHP FDQ DXWRPDWLFDOO\ DSSHQG EHDW FRXQWLQJ YRLFHYLEUDWLRQ LQLWLDOO\ WR 1HZ 83 Offline Mode Online Mode Salsa Music Real-time File Salsa Music Short Period % % % % % Listening 2QVHW3RVWLRQ (QHUJ\3URILOH Onset Detection )LJ 2QVHW SRVLWLRQV RI SOD\LQJ &ODYH LQ WZR EDUV Salsa Beat Interval (SBI) Estimation 1) Spectrogram: 6SHFWURJUDP GHVFULEHV WKH DXGLR VLJQDO Offline Beat Online Beat LQ WHUPV RI HQHUJ\ VSUHDG RYHU LWV IUHTXHQF\ FRPSRQHQWV Counting Counting DW GLIIHUHQW WLPHV ,Q SalsaAsst WKH VSHFWURJUDP LV REWDLQHG E\ FRPSXWLQJ WKH ORJPDJQLWXGH RI WKH 6KRUWWLPH )RXULHU WUDQVIRUP 67)7 RI DXGLR VLJQDO &RQVLGHU D OHQJWKN PXVLF Beat Counting Voice/Vibration VLJQDO x[n] RQ 0 n N − 1 7KH 6KRUWWLPH )RXULHU Prompts WUDQVIRUP 67)7 RI x[n] LV GH¿QHG DV N −1 )LJ 6\VWHP RYHUYLHZ RI 6DOVD$VVW −j 2π kN STFT {x [n]} (m, k)= x [n] ω [n − mR] e N , n=0 FRUUHVSRQGLQJ WR WKH 6DOVD EHDWV ,Q WKH Online Beat Counting ZKHUH m LV WKH LQGH[ RI ZLQGRZ IUDPH R GHQRWHV WKH KRS VL]H Mode RXU V\VWHP OHYHUDJHV WKH PLFURSKRQH RQ PRELOH GHYLFHV RI 67)7 ZKLFK LV HTXDO WR 32 VDPSOHV DQG k UHSUHVHQWV WKH WR UHFHLYH WKH DFRXVWLF VLJQDO RI 6DOVD PXVLF DQG JHQHUDWH WKH kWK IUHTXHQF\ ELQ ω[n] GHQRWHV WKH ZLQGRZ IXQFWLRQ +HUH UHDOWLPH SURPSWV LH EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ ZH FRPSXWH WKH 67)7 RI 6DOVD PXVLF VLJQDO ZLWK +DPPLQJ 0RUH VSHFL¿FDOO\ RXU V\VWHP OLVWHQV IRU D VKRUW SHULRG RI ZLQGRZ IXQFWLRQ ZKLFK LV GH¿QHG DV WLPH LH 10 VHFRQGV DW WKH EHJLQQLQJ IRU FRPSXWLQJ WKH 0 51 − 0 46 2πn 0 ≤ ≤ HVWLPDWHG YDOXH RI 6DOVD %HDW ,QWHUYDO 6%, %DVHG RQ WKH . . cos M , n M ω [n]= , HVWLPDWHG 6%, RXU V\VWHP LV DEOH WR JHQHUDWH WKH UHDOWLPH 0 , 2WKHUZLVH EHDW FRXQWLQJ YRLFHYLEUDWLRQ SURPSWV DORQJ ZLWK WKH GDQFH ZKHUH LV WKH OHQJWK RI ZLQGRZ IUDPH ZKLFK LV HTXDO WR 256 PXVLF WR LQGLFDWH WKH WLPLQJV RI EHDWV :H OHDYH WKH GHWDLOV RI M VDPSOHV DQG GHQRWHV WKH VDPSOHV LQ ZLQGRZ %\ WKH ZD\ OfÀine Beat Counting Mode DQG Online Beat Counting Mode n ZH XVH 44 1 DV WKH VDPSOLQJ UDWH LQ 67)7 WR 6HFWLRQ ,9 DQG 6HFWLRQ 9 UHVSHFWLYHO\ . kHz $IWHU FRPSXWLQJ WKH 67)7 WKH VSHFWURJUDP RI WKH PXVLF [ ] ,9 2))/,1( %($7 &2817,1* 02'( VLJQDO x n FDQ EH REWDLQHG E\ FDOFXODWLQJ WKH ORJPDJQLWXGH RI WKH 67)7 ZKLFK GH¿QHG DV ,Q WKLV VHFWLRQ ZH ¿UVW GHVFULEH Onset DetectionDQGWKHQ SUHVHQW Salsa Beat Interval (SBI) Estimation EDVHG RQ WKH RQVHW Specg {x [n]} (m, k)=log10(|STFT {x [n]} (m, k)|). SRVLWLRQV :H ¿QDOO\ GHVFULEH OfÀine Beat Counting DOJRULWKP $V VKRZQ LQ )LJXUH D WKH UHG UHJLRQV RI VSHFWURJUDP DUH WR VKRZ KRZ WR DFFXUDWHO\ GHWHFW WKH 6DOVD EHDWV IURP RQVHWV WKH IUHTXHQF\ UHJLRQV ZLWK D KLJK GHJUHH RI HQHUJ\ 2Q WKH A. Onset Detection RWKHU FRQWUDU\ WKH WLQW UHJLRQV DUH WKH IUHTXHQF\ UHJLRQV ZLWK D ORZ GHJUHH RI HQHUJ\ ,Q WKLV VWHS ZH GHWHFW DOO WKH RQVHWV LQ WKH 6DOVD PXVLF 2) Onset Strength Curve: :KLOH VSHFWURJUDP UHSUHVHQWV WKH VLJQDO $Q RQVHW UHIHUV WR WKH EHJLQQLQJ RI D VXGGHQ EXUVW HQHUJ\ GLVWULEXWLRQ RI GLIIHUHQW IUHTXHQF\ WKH 2QVHW 6WUHQJWK RI HQHUJ\ FDXVHG E\ SOD\LQJ LQVWUXPHQWV >@ )RU H[DPSOH &XUYH 26& LV WKH PHDVXUHPHQW RI WKH HQHUJ\ FKDQJHV LQ )LJXUH VKRZV WKH WLPH GRPDLQ ZDYHIRUP RI SOD\LQJ WKH VSHFWURJUDP %\ VXPPLQJ WKH SRVLWLYH FKDQJHV DFURVV DOO &ODYH LQ WZR EDUV DQG WKH FRUUHVSRQGLQJ RQVHW SRVLWLRQV ZKLFK IUHTXHQF\ ELQV LQ VSHFWURJUDP ZH DUH DEOH WR REWDLQ WKH UDZ LOOXVWUDWHV WKH UHODWLRQVKLSV EHWZHHQ EHDWV DQG RQVHWV :H FDQ 2QVHW 6WUHQJWK &XUYH 26& DV LOOXVWUDWHG LQ )LJXUH E 7KH ¿QG WKDW WKH RQVHW LV WKH SRVLWLRQ ZKHUH WKH EHDW PD\ DSSHDU 26& LV GH¿QHG DV IROORZV>@ 7KHUHIRUH RQVHW GHWHFWLRQ LV WKH EDVLF VWHS IRU WKH IXUWKHU EHDW N WUDFNLQJ OSC(m)= H(Specg(m, k) − Specg(m − 1,k)), 7KH SURFHGXUH RI RQVHW GHWHFWLRQ LV WR FRPSXWH WKH VSHF k=1 WURJUDP IURP WKH DXGLR VLJQDO WKHQ REWDLQ WKH 2QVHW 6WUHQJWK &XUYH 26& DQG ¿QG WKH ORFDO PD[LPD IURP VPRRWKHG 26& ZKHUH Specg(m, k) UHSUHVHQWV WKH VSHFWURJUDP RI WKH kWK DV RQVHW SRVLWLRQ 7KH IROORZLQJ DUH WKH GHWDLOV IUHTXHQF\ ELQ RI WKH mWK IUDPH N GHQRWHV WKH QXPEHU 84 2QVHW,QWHUYDO VHF )UHTXHQF\ N+] 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 0XVLF,QGH[ 7LPH VHF D 6SHFWURJUDP )LJ 'LVSHUVLRQV RI 2,V LQ QLQH 6DOVD VRQJV 7$%/( , )5(48(1&< 7$%/( 2) 2,6,16$/6$ 2, VHF )UHTXHQF\ 3URSRUWLRQ $PSOLWXGH 7LPH VHF E 5DZ 2QVHW 6WUHQJWK &XUYH 26& 6PRRWKHG26& 7UHQG $PSOLWXGH B. Salsa Beat Interval (SBI) Estimation %DVHG RQ WKH RQVHW SRVLWLRQV ZH FDQ HVWLPDWH 6DOVD %HDW 7LPH VHF ,QWHUYDO 6%, 6%, GHQRWHV WKH WLPH LQWHUYDO EHWZHHQ WKH F 6PRRWKHG 26& DQG LWV WUHQG DGMDFHQW 6DOVD EHDWV )RU H[DPSOH WKH WLPH EHWZHHQ EHDW 1 DQG 2 LV RQH 6%, ZKLOH WKH WLPH EHWZHHQ 1 DQG 1.5 LV D KDOI RI 2QVHW6WUHQJWK&XUYH 6%, 0RUHRYHU WKH 6%, LV D ¿[HG YDOXH IRU D 6DOVD VRQJ >@ 2QVHWSRVLWLRQ 2XU PHWKRG IRU 6DOVD %HDW ,QWHUYDO 6%, HVWLPDWLRQ LV EDVHG RQ WKH REVHUYHG UHODWLRQVKLS EHWZHHQ RQVHW DQG 6DOVD EHDW LQ 6DOVD PXVLF $V VKRZQ LQ )LJXUH GLIIHUHQW 6DOVD LQVWUXPHQWV $PSOLWXGH DUH SOD\HG ZLWK XQLTXH UK\WKPV IRU H[DPSOH WKH &ODYH LV SOD\HG DW ZKHUH WKH RQVHWV FDQ EH REVHUYHG WKHUH +RZHYHU ZKLOH SOD\LQJ DOO WKHVH LQVWUXPHQWV LQ 6DOVD 7LPH VHF G 'HWUHQGHG 26& DQG 2QVHW SRVLWLRQV PXVLF ZH FDQ ¿QG WKDW WKH RQVHWV DSSHDU HYHU\ KDOI 6DOVD EHDW 7KDW LV WR VD\ WKH RQVHW LQWHUYDO 2, LV VXSSRVHG WR EH )LJ ,OOXVWUDWLRQ RI WKH SURFHGXUH RI 2QVHW 'HWHFWLRQ HTXDO WR KDOI 6%, LQ 6DOVD PXVLF :H GHPRQVWUDWH WKDW WKLV RE VHUYHG UHODWLRQVKLS EHWZHHQ RQVHW DQG 6DOVD EHDW LV UHDVRQDEOH WKURXJK DQDO\]LQJ WKH GLVWULEXWLRQ RI RQVHW LQWHUYDOV 2,V LQ x+|x| 6DOVD PXVLF $V VKRZQ LQ )LJXUH ZH OLVW WKH GLVWULEXWLRQV RI RI IUHTXHQF\ ELQV H(x)= LV WKH KDOIZDYH UHFWL¿HU 2 RQVHW LQWHUYDOV 2,V LQ QLQH 6DOVD VRQJV :H FDQ ¿QG WKDW PRVW IXQFWLRQ RI 2,V LQ D 6DOVD VRQJ DUH FRQFHQWUDWHG DURXQG D YDOXH 7KDW 3) Onset Positions: $IWHU REWDLQLQJ WKH UDZ 2QVHW 6WUHQJWK LV PRVW RI 2,V DUH DSSUR[LPDWHO\ HTXLYDOHQW 7KHUHIRUH WR &XUYH WKH VPRRWKHG 26& DQG LWV WUHQG FDQ EH REWDLQHG E\ HVWLPDWH KDOI 6%, RI D 6DOVD VRQJ HJ 6DOVD ZH GR VWDWLVWLFV OHYHUDJLQJ *DXVVLDQ ORZSDVV ¿OWHU ZLWK GLIIHUHQW SDUDPH RI WKH IUHTXHQF\ RI 2,V DV VKRZQ LQ 7DEOH , DQG FRPSXWH WHUV >@ DV VKRZQ LQ )LJXUH F ,Q SDUWLFXODU WKH UDZ 26& WKH DULWKPHWLF DYHUDJH RI 2,V ZKRVH IUHTXHQF\ SURSRUWLRQ LV LV ¿OWHUHG E\ D *DXVVLDQ ¿OWHU ZLWK D FXWRII IUHTXHQF\ HTXDO KLJK LH > 5% 7KLV PHDQ YDOXH LV DEOH EH UHJDUGHG DV WKH WR WKH VDPSOLQJ UDWH GLYLGHG E\ 20 WR REWDLQ WKH VPRRWKHG DSSUR[LPDWHG YDOXH RI 2, LH KDOI 6%, 7KH HVWLPDWHG 6%, 26& ,WV WUHQG LV REWDLQHG E\ ¿OWHULQJ WKH VPRRWKHG 26& WKHQ FDQ EH REWDLQHG E\ GRXEOLQJ WKH DSSUR[LPDWHG KDOI 6%, XVLQJ DQRWKHU *DXVVLDQ ¿OWHU ZLWK D FXWRII IUHTXHQF\ HTXDO C. OfÀine Beat Counting WR WKH VDPSOLQJ UDWH GLYLGHG E\ 150 )XUWKHUPRUH ZH FDQ FRPSXWH WKH GHWUHQGHG 26& E\ VXEVWUDFWLQJ WKH WUHQG IURP $FFRUGLQJ WR WKH GHWHFWHG RQVHW SRVLWLRQV DQG HVWLPDWHG WKH VPRRWKHG 26& DQG ¿QG LWV FRUUHVSRQGLQJ SHDNV DV WKH 6%, 6DOVD EHDWV ZRXOG EH VHDUFKHG DFFXUDWHO\ +HUH ZH XVH RQVHW SRVLWLRQV DV LOOXVWUDWHG LQ )LJXUH G DQ H[DPSOH DV VKRZQ LQ )LJXUH WR H[SODLQ RXU PHWKRG IRU 85 &DQGLGDWH $OJRULWKP 2QOLQH %HDW &RXQWLQJ %HDW %HDW%HDW 'DWD 7KH UHFHLYHG UHDOWLPH DXGLR VLJQDO DW VDPSOLQJ UDWH V V fs =44.1kHz X =[X(1),X(2),X(3)...] 7KH HVWLPDWHG YDOXH RI 6DOVD %HDW ,QWHUYDO EB =( +01) ∗ (VWLPDWHG6%, 7KH OHQJWK RI ZLQGRZ IUDPH Win EB . fs D 6HDUFKLQJ UHJLRQ KDV RQVHW V 5HVXOW =[ (1) (2) (3) ( ) ] &DQGLGDWH %HDW 7LPH RI %HDWV B B ,B ,B ..., B i , ... 5HDOWLPH %HDW &RXQWLQJ 9RLFH DQG 9LEUDWLRQ 3URPSWV %HDW EHJLQ (1) = 0 (1) V V B s. ,QVHUW 3URPSWV DW B B(2) = B(1) + EB. ,QVHUW 3URPSWV DW B(2) LI i 64 WKHQ i−1 1 B(i+1)−B(i) (VWLPDWHG6%, EB = i−1 E 6HDUFKLQJ UHJLRQ KDV QR RQVHW // Replace EB with the mean value )LJ ,OOXVWUDWLRQ RI KRZ WR WUDFN 6DOVD EHDW IURP RQVHWV of previous SBIs HQG ZKLOH i 3 GR EHDW VHDUFKLQJ )LUVW RI DOO ZH ¿QG WKH ¿UVW RQVHW SRVLWLRQ DV 3HUIRUP 2QVHW 'HWHFWLRQ RQ WKH ¿UVW EHDW SRVLWLRQ DQG ODEHO LW DV %HDW 6WDUWLQJ IURP X[B(i − 1) ∗ fs):B(i − 1) ∗ fs)+Win)] %HDW ZH DGG WKH HVWLPDWHG 6%, WR JHW WKH FDQGLGDWH RI WKH )LQG WKH QHDUHVW RQVHW SRVLWLRQ N DURXQG VHFRQG EHDW SRVLWLRQ LH %HDW :H WKHQ VHDUFK DURXQG WKH B(i − 1) + EB FDQGLGDWH WR ORFDWH WKH QHDUHVW RQVHW 6LQFH WKH KXPDQ HDU LI |N − B(i − 1) − EB| 0.08 WKHQ FDQQRW GLVWLQJXLVK WKH GHOD\DGYDQFH RI WKH VRXQG OHVV WKDQ B(i)=N. ,QVHUW 3URPSWV DW B(i) 0.1s >@ WKH WROHUDWH GHOD\ DQG DGYDQFH IRU WKH GHWHFWHG EHDW HOVH DUH 0.1s 7KHUHIRUH WKH VHDUFKLQJ UHJLRQ LV FRQ¿QHG WR ±0.1s B(i)=B(i − 1) + EB. ,QVHUW 3URPSWV DW DURXQG WKH FDQGLGDWH ,I LW KDV RQVHWV LQ WKH VHDUFKLQJ UHJLRQ B(i) DV VKRZQ LQ )LJXUH D ZH UHJDUG WKH QHDUHVW RQVHW SRVLWLRQ HQG DV WKH EHDW SRVLWLRQ DQG ODEHO LW DV %HDW ,I QRW DV LOOXVWUDWHG HQG LQ )LJXUH E WKH FDQGLGDWH ZRXOG EH UHFRJQL]HG DV WKH EHDW // The units of EB, B, N, are all SRVLWLRQ 7KHQ ZH UHSHDW WKH VWHSV DERYH WR VHDUFK WKH QH[W second(s) EHDW SRVLWLRQ 7KLV PHWKRG LV DEOH WR HOLPLQDWH WKH LPSDFWV HQG RI WKH IDOVH DQG PLVVHG RQVHW GHWHFWLRQV DQG WKHQ ORFDOL]H WKH 6DOVD EHDWV DFFXUDWHO\ 0RUHRYHU WKH EHDW ODEHOV DUH F\FOHG HYHU\ 8 EHDWV 2XU V\VWHP WKHQ JHQHUDWHV EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ SURPSWV FRUUHVSRQGLQJ WR WKH EHDW ODEHOV WKDW GDQFHUV QHHG WR VWHS LH 9 2 1/,1( %($7 &2817,1* 02'( Counting LV SURYLGHG LQ $OJRULWKP 6SHFL¿FDOO\ ZH ¿UVW UHJDUG WKH EHJLQQLQJ RI WKH SOD\LQJ 6DOVD PXVLF DV WKH WLPH ,Q WKLV VHFWLRQ ZH LQWURGXFH KRZ WR WUDFN EHDWV IURP RI ¿UVW EHDW (1) DQG WKHQ JHW WKH WLPH RI (2) E\ DGGLQJ UHDOWLPH 6DOVD PXVLF DQG JHQHUDWH UHDOWLPH EHDW FRXQWLQJ B B HVWLPDWHG 6%, )RU VHDUFKLQJ WKH QH[W EHDWV ZH ¿UVW JHW WKH YRLFH DQG YLEUDWLRQ SURPSWV RQ PRELOH GHYLFHV 'LIIHUHQW IURP FDQGLGDWH EHDW WLPH ( − 1) + WKURXJK DGGLQJ HVWLPDWHG OfÀine Beat Counting ZH GRQ¶W KDYH D 6DOVD VRQJ ¿OH WR B i EB 6%, WR WKH SULRU EHDW WLPH 7KHQ ZH SHUIRUP RQVHW GHWHFWLRQ HVWLPDWH 6DOVD %HDW ,QWHUYDO 6%, ,QVWHDG LQ Online Beat RQ WKH VLJQDO EHWZHHQ WKH WLPH ( −1) DQG ( −1)+ +0 1 Counting RXU V\VWHP ¿UVW OLVWHQV IRU D VKRUW SHULRG RI WLPH B i B i EB . WR REWDLQ WKH RQVHW SRVLWLRQV +HUH 0 1 LV WLPH RI WKH WROHUDWH LH 10 VHFRQGV RI 6DOVD PXVLF WR HVWLPDWH WKH 6%, YLD Onset . s GHOD\ IRU GHWHFWHG EHDWV )URP WKHVH RQVHWV ZH ¿QG WKH QHDUHVW Detection DQG Salsa Beat Interval (SBI) EstimationZKLFK RQVHW WR WKH FDQGLGDWH EHDW WLPH ,I WKH WLPH GLIIHUHQFH EHWZHHQ KDYH EHHQ LQWURGXFHG VHFWLRQ ,9 %DVHG RQ WKH HVWLPDWHG 6%, WKH QHDUHVW RQVHW DQG FDQGLGDWH LV OHVV WKHQ 0 1 LH WLPH RI IURP WKH VHFRQGV KLVWRULFDO GDWD RXU V\VWHP WKHQ XVHV WKH . s WKH WROHUDWH GHOD\DGYDQFH WKH QHDUHVW RQVHW LV UHJDUGHG DV EXLOWLQ PLFURSKRQH RI PRELOH GHYLFH WR VHQVH WKH UHDOWLPH WKH 6DOVD EHDW 2WKHUZLVH WKH FDQGLGDWH VKRXOG EH LGHQWL¿HG 6DOVD PXVLF VLJQDO DQG VDYH WKH GDWD LQ WKH EXIIHU :H XVH WZR DV WKH EHDW $GGLWLRQDOO\ FRQVLGHULQJ WKH GHULYHG IURP WKUHDGV IRU UHDOWLPH EHDW FRXQWLQJ 7KH PDLQ WKUHDG LV XVHG WR EB WKH VKRUW SHULRG KLVWRULFDO GDWD PD\ QRW EH TXLWH DFFXUDWH WKH SOD\ WKH EHDW FRXQWLQJ YRLFH SURPSWV DQG JHQHUDWH YLEUDWLRQ YDOXH RI ZRXOG EH FRQWLQXDOO\ XSGDWHG WR WKH PHDQ YDOXH SURPSWV $QRWKHU WKUHDG LV IRU VHDUFKLQJ 6DOVD EHDWV RI WKH EB RI WKH WLPH LQWHUYDOV RI GHWHFWHG EHDWV DIWHU GHWHFWLQJ 64 EHDWV UHFHLYHG VLJQDO LQ WKH EXIIHU 7KH DOJRULWKP RI Online Beat ZKLFK FDQ PDNH EB PRUH DFFXUDWH 86 9, 3(5)250$1&( (9$ / 8$7 , 2 1 WLPLQJV JLYH WKH JUHDWHVW VFRUH 7KH EHDW DFFXUDF\ WKHQ LV ,Q WKLV VHFWLRQ ZH HYDOXDWH WKH EHDW WUDFNLQJ DFFXUDF\ PHDVXUHG E\ VXPPLQJ XS WKHVH VFRUHV RI ERWK OfÀine Beat Counting DOJRULWKP DQG Online Beat • 36FRUH ZKLFK PHDVXUHV EHDW WUDFNLQJ DFFXUDF\ E\ VXP Counting DOJRULWKP LQ RXU V\VWHP PLQJ WKH FURVVFRUUHODWLRQ EHWZHHQ LPSXOVH WUDLQV RI WKH GHWHFWHG EHDWV DQG WKH JURXQG WUXWK DQQRWDWLRQV FRQVLGHU A. Experimental Methodology LQJ GHYLDWLRQV ZLWKLQ 20% RI WKH DQQRWDWHG EHDW LQWHUYDO :H JHQHUDWH D WHVW GDWDVHW ZKLFK FRQVLVWV RI 40 6DOVD PXVLF DV FRUUHFW DXGLR ¿OHV ZLWK WKH JURXQG WUXWK DQQRWDWLRQV RI 6DOVD EHDW 2) Continuity-based Metrics: 5 WLPLQJV (DFK ¿OH LV DQ HQWLUH 6DOVD VRQJ ZKLFK LV DURXQG • &0/W ZKLFK ¿QGV WKH FRUUHFW GHWHFWHG EHDWV DW WKH 44 1 PLQXWHV LQ OHQJWK VWHUHR DQG . kHz VDPSOLQJ UDWH $OVR PHWULFDOOHYHORI6DOVDEHDWV LH WKH ¿OHV DUH DQQRWDWHG E\ WKH SURIHVVLRQDO 6DOVD GDQFHUV E\ DQG FDOFXODWHV UDWLR RI WKH QXPEHU RI FRUUHFW EHDWV WR WKH UHFRUGLQJ EHDW FRXQWLQJ YRLFH LH EHDWV OHQJWK RI DQQRWDWLRQ VHTXHQFH WR WKH DXGLR ¿OHV :H PDQXDOO\ QRWH GRZQ WKH JURXQG • &0/F ZKLFK ¿QGV WKH ORQJHVW VHJPHQW RI FRQWLQXRXVO\ WUXWK DQQRWDWLRQV DFFRUGLQJ WR WKH EHDW FRXQWLQJ YRLFH LQ HYHU\ FRUUHFW GHWHFWHG EHDWV DW WKH FRUUHFW PHWULFDO OHYHO DQG PXVLF ¿OH FDOFXODWHV WKH UDWLR RI WKH OHQJWK RI WKH ORQJHVW VHJPHQW (YHU\ PXVLF ¿OH LQ RXU GDWDVHW LV IHG LQWR OfÀine Beat WR WKH OHQJWK RI DQQRWDWLRQ VHTXHQFH Counting DOJRULWKP ZKLFK FDQ DXWRPDWLFDOO\ UHFRUG WKH WLP • $0/W ZKLFK ¿QGV WKH FRUUHFW GHWHFWHG EHDWV DW WZLFH LQJV RI WKH GHWHFWHG EHDWV 7R WHVW WKH Online Beat Counting RU KDOI WKH FRUUHFW PHWULFDO OHYHO DQG FDOFXODWHV UDWLR RI DOJRULWKP WKH 6DOVD PXVLF ¿OH LV SOD\HG E\ WKH H[WHUQDO WKH QXPEHU RI FRUUHFW EHDWV WR WKH OHQJWK RI DQQRWDWLRQ VSHDNHU 2XU V\VWHP ¿UVW OLVWHQV IRU VHFRQGV RI WKH SOD\LQJ VHTXHQFH PXVLF WR FRPSXWH WKH HVWLPDWHG 6%, $IWHU VHFRQGV SOD\LQJ • $0/F ZKLFK ¿QGV WKH ORQJHVW VHJPHQW RI FRQWLQXRXVO\ WKH OfÀine Beat Counting DOJRULWKP ZRXOG UHFRUG WKH EHDW FRUUHFW GHWHFWHG EHDWV DW WZLFH RU KDOI WKH FRUUHFW PHWULFDO WLPLQJV LQ UHDOWLPH ,Q DGGLWLRQ ZH DOVR FRPSDUH WKH EHDW OHYHO DQG FDOFXODWHV WKH UDWLR RI WKH OHQJWK RI WKH ORQJHVW WUDFNLQJ DFFXUDF\ RI RXU DOJRULWKPV ZLWK RWKHU WKUHH EHDW WUDFN VHJPHQW WR WKH OHQJWK RI DQQRWDWLRQ VHTXHQFH LQJ DSSURDFKHV WKH FRQWH[WGHSHQGHQW EHDW WUDFNLQJ V\VWHP 3) Beat Error Histogram: :H FRPSXWH WKH EHDW HUURU DV WKH LH >@ WKH WZRIROG G\QDPLF SURJUDPPLQJ EHDW WUDFNHU GLIIHUHQFH EHWZHHQ HDFK DQQRWDWLRQ DQG WKH QHDUHVW GHWHFWHG LH >@ DQG WKH EHDW WUDFNHU XVLQJ G\QDPLF SURJUDPPLQJ EHDW 7KH EHDW HUURU RI HYHU\ GHWHFWHG EHDW LV WKHQ GLYLGHG LH >@ E\ WKH DQQRWDWLRQ LQWHUYDO ZKLFK QRUPDOL]HV WKH EHDW HUURUV B. Evaluation Metrics EHWZHHQ −0.5 DQG 0.5 ,Q DGGLWLRQ LI DQ\ EHDW HUURUV DUH [−0 505] )RU HYDOXDWLQJ WKH EHDW WUDFNLQJ DFFXUDF\ DJDLQVW PDQXDO EH\RQG . . WKH\ FDQ EH FRQ¿QHG EDFN LQWR WKH UDQJH [−0 505] DQQRWDWLRQV ZH XVH HLJKW HYDOXDWLRQ PHWULFV IURP WKH EHDW . . XVLQJ PRGXOR DULWKPHWLF )LQDOO\ ZH XWLOL]H WKHVH Φ WUDFNLQJ HYDOXDWLRQ WRROER[ >@ ZKLFK DUH JHQHUDOO\ DGRSWHG EHDW HUURU GDWD WR SRSXODWH D KLVWRJUDP 7KH EHDW HUURU LV GH¿QHG DV > @ LQ EHDW WUDFNLQJ HYDOXDWLRQ )LUVW $FFXUDF\EDVHG 0HWULFV ⎧ ZKLFK FDQ DVVHVV WKH WRWDO QXPEHU RI FRUUHFW GHWHFWHG EHDWV minj (|a−bi|) =1 ⎨⎪ a −a ,j QRW FRQVLGHULQJ WKH PHWULFDO OHYHO LQFOXGH )PHDVXUH &HPJL j+1 j Φ(i)= , O$FF >@ 36FRUH >@ 6HFRQG &RQWLQXLW\EDVHG 0HWULFV >@ ⎩⎪ minj (|a−bi|) 1 LQFOXGHV &0/F &RUUHFW 0HWULFDO /HYHO ZLWK FRQWLQXLW\ UH aj −aj−1 ,j > TXLUHG &0/W &RUUHFW 0HWULFDO /HYHO ZLWK QR FRQWLQXLW\ ZKHUH b DQG a UHIHU WR WKH VHTXHQFH RI GHWHFWHG EHDWV DQG UHTXLUHG $0/F $OORZHG 0HWULFDO /HYHO ZLWK FRQWLQXLW\ th DQQRWDWLRQV UHVSHFWLYHO\ bi GHQRWHV WKH WLPLQJV RI WKH i EHDW UHTXLUHG $0/W $OORZHG 0HWULFDO /HYHO ZLWK QR FRQWLQXLW\ th aj GHQRWHV WKH WLPLQJV RI WKH j DQQRWDWLRQ UHTXLUHG ZKLFK FDQ PDQLIHVW ZKHWKHU WKH GHWHFWHG EHDWV DUH DW WKH PHWULFDO OHYHO RI 6DOVD EHDWV DQG WKH FRQWLQXLW\ RI WKH C. Results FRUUHFW GHWHFWHG EHDWV 0RUHRYHU WR YLVXDOL]H WKH EHDW WUDFNLQJ )LJXUH D DQG )LJXUH E SUHVHQW WKH SHUIRUPDQFH RI RXU SHUIRUPDQFH ZH XWLOL]H %HDW (UURU +LVWRJUDP > @ WR VKRZ WKH OfÀine Beat Counting DQG Online Beat Counting DOJRULWKPV GLVWULEXWLRQ RI WKH EHDW HUURU SUREDELOLW\ 'HYLHV >@ -DQJ >@ DQG (OOLV >@ XQGHU WKH $FFXUDF\EDVHG 1) Accuracy-based Metrics: 0HWULFV DQG &RQWLQXLW\EDVHG 0HWULFV UHVSHFWLYHO\ $V VKRZQ • )PHDVXUH ZKLFK FRXQWV WKH QXPEHU RI WUXH SRVLWLYHV LQ )LJXUH D ERWK OfÀine Beat Counting DOJRULWKP DQG TP FRUUHFW GHWHFWLRQ ZLWKLQ D WROHUDQFH ZLQGRZ RI Online Beat Counting DOJRULWKP RXWSHUIRUP DOO WKH RWKHU EHDW ±70ms IDOVH QHJDWLYHV FN PLVVHG GHWHFWLRQ DQG IDOVH WUDFNLQJ V\VWHPV EDVHG RQ WKH SHUIRUPDQFH LQGLFHV RI ) SRVLWLYHV FP H[WUD GHWHFWLRQ DQG GH¿QHV WKH EHDW DFFX PHDVXUH &HPLO$FF DQG 3VFRUH 7KDW LV RXU V\VWHP KDYH 2∗TP UDF\ DV 2∗TP+FN+FP ∗ 100% FDSDFLW\ WR ¿QG PRUH FRUUHFW EHDWV WKDQ RWKHU V\VWHPV • &HPJLO$FF ZKLFK XWLOL]HV D *DXVVLDQ HUURU IXQFWLRQ ZLWK 0RUHRYHU RWKHU V\VWHPV DOVR VHHP WR SHUIRUP ZHOO ZKHUH D VWDQGDUG GHYLDWLRQ RI 40ms WR HYDOXDWH WKH WLPH HUURU WKH )PHDVXUH VFRUHV RI 'HYLHV >@ DQG -DQJ >@ DUH 69.03% EHWZHHQ WKH GHWHFWHG EHDW DQG WKH FORVHVW DQQRWDWLRQ VR DQG 83.24% UHVSHFWLYHO\ +RZHYHU DV VKRZQ LQ )LJXUH E WKDW WKH GHWHFWHG EHDWV O\LQJ H[DFWO\ RQ WKH DQQRWDWHG EHDW RQFH WKH FRQVWUDLQWV RYHU WKH PHWULFDO OHYHO RI 6DOVD EHDWV DUH 87 100 )PHDVXUH &HPJLO$FF 80 36FRUH 60 3URSRUWLRQ 40 0HDQ6FRUH 20 %HDW(UURU Φ D 6DOVD$VVW 2IÀLQH 0 SalsaAsst SalsaAsst Davies [6] Jang [5] Ellis [4] (Offline) (Online) D $FFXUDF\EDVHG 0HWULFV 100 &0/W &0/F 80 $0/W $0/F 3URSRUWLRQ 60 40 %HDW(UURU Φ 0HDQ6FRUH E 6DOVD$VVW 2QOLQH 20 0 SalsaAsst SalsaAsst Davies [6] Jang [5] Ellis [4] (Offline) (Online) E &RQWLQXLW\EDVHG 0HWULFV )LJ &RPSDULVRQ RI EHDW WUDFNLQJ DOJRULWKPV XQGHU $FFXUDF\EDVHG 0HWULFV 3URSRUWLRQ DQG &RQWLQXLW\EDVHG 0HWULFV %HDW(UURU Φ F 'DYLHV >@ UHTXLUHG WKHVH V\VWHPV DUH QR ORQJHU DFFXUDWH WKDW WKH &0/W 1 49% RI -DQJ >@ LV RQO\ . 7KH &0/W VFRUHV RI 'HYLHV >@ DQG (OOLV >@ DUH HYHQ HTXDO WR 0% ZKLFK LQGLFDWHV WKDW WKH GHWHFWHG EHDWV RI WKHVH V\VWHPV DUH DOO QRW DW WKH PHWULFDO OHYHO RI 6DOVD EHDWV 2XU SURSRVHG DOJRULWKPV KRZHYHU DUH 3URSRUWLRQ VWLOO DFFXUDWH ZKHUH &0/W RI OfÀine Beat Counting DOJRULWKP LV 82.28% DQG &0/W RI Online Beat Counting DOJRULWKP LV %HDW(UURU Φ G -DQJ >@ 65.96% ZKLFK GHPRQVWUDWHV WKDW PRVW EHDWV GHWHFWHG E\ RXU DOJRULWKPV DUH H[DFWO\ DW WKH PHWULFDO OHYHO RI 6DOVD EHDWV QDPHO\ RXU DOJRULWKPV DSSO\ WR 6DOVD PXVLF PXFK EHWWHU WKDQ RWKHUV 2Q WKH RWKHU KDQG &0/F RI RXU DOJRULWKPV UHÀHFWLQJ WKH 3URSRUWLRQ FRQWLQXLW\ RI WKH FRUUHFW GHWHFWHG EHDWV DW WKH 6DOVD EHDWV PHW 37 04% ULFDO OHYHO LV RQO\ HTXDO WR . IRU OfÀine Beat Counting %HDW(UURU Φ DOJRULWKP DQG 22.41% IRU Online Beat Counting DOJRULWKP H (OOLV >@ %XW WKH ORZ &0/F PHUHO\ LQGLFDWHV WKDW WKH ZURQJ GHWHFWHG )LJ %HDW (UURU +LVWRJUDPV RI RXU DOJRULWKPV DQG RWKHU H[LVWLQJ EHDW EHDWV RI RXU V\VWHP DUH VFDWWHUHG UDWKHU WKDQ FHQWUDOL]HG ZKLFK WUDFNLQJ DSSURDFKHV FDQQRW SURYH WKH 6DOVD EHDW GHWHFWLRQ RI RXU DOJRULWKPV DUH LQDFFXUDWH EHFDXVH RXU DOJRULWKPV REWDLQ D KLJK VFRUH XQGHU &0/W $GGLWLRQDOO\ ZH QRWLFH WKDW WKH $0/W DQG $0/F RI LQ )LJXUH D PHUHO\ H[KLELWV D VWURQJ SHDN FORVH WR WKH 'HYLHV >@ DUH ERWK JRRG DQG WKH $0/F RI 'HYLHV >@ LV HUURU RI 0 LQGLFDWLQJ PRVW EHDWV ZH GHWHFWHG DUH FORVH WR WKH HYHQ EHWWHU WKDQ RXU DOJRULWKPV +LJK $0/W DQG $0/F VFRUHV JURXQG WUXWK DQQRWDWLRQV )RU Online Beat Counting DOJRULWKP GHPRQVWUDWH WKDW 'HYLHV >@ FDQ DFFXUDWHO\ ¿QG WKH EHDWV DW WKH DV VKRZQ LQ )LJXUH E DOWKRXJK LWV %HDW (UURU +LVWRJUDP WZLFH RU KDOI PHWULFDO OHYHO RI 6DOVD %HDWV 1HYHUWKHOHVV WKH VKRZV VRPH HUURUV FRQFHQWUDWH RQ −0.4 PRVW EHDW HUURUV &0/W DQG &0/F VFRUHV RI 'HYLHV >@ DUH HTXDO WR 0%WKDWLV VWLOO IRFXV DURXQG 0 8QOLNH RXU DOJRULWKPV WKH %HDW (UURU WR VD\ 'HYLHV >@ FDQ PHUHO\ GHWHFW WKH EHDWV DW WKH WZLFH RU +LVWRJUDP RI 'HYLHV >@ VKRZQ LQ )LJXUH F KDV WZR VWURQJ KDOI PHWULFDO OHYHO RI 6DOVD %HDWV EXW LV QRW DEOH WR ¿QG EHDWV SHDNV DURXQG ±0.2 DQG RWKHU WZR VWURQJ SHDNV DURXQG ±0.5 DW WKH 6DOVD %HDWV PHWULFDO OHYHO ZKLFK LQGLFDWHV WKDW D VXEVWDQWLDO SDUW RI GHWHFWHG EHDWV DUH )XUWKHUPRUH DV LOOXVWUDWHG LQ )LJXUH ZH XWLOL]H %HDW QRW PDWFKHG ZLWK WKH JURXQG WUXWK /LNHZLVH )LJXUH G (UURU +LVWRJUDP WR YLVXDOL]H WKH SHUIRUPDQFH RI RXU DOJR DQG )LJXUH H GHPRQVWUDWH WKDW -DQJ >@ DQG (OOLV >@ DOVR ULWKPV DQG RWKHU WKUHH H[LVWLQJ EHDW WUDFNLQJ DSSURDFKHV 7KH SHUIRUP EDGO\ GXH WR WKH GLVSHUVLYH GLVWULEXWLRQV RI WKHLU EHDW %HDW (UURU +LVWRJUDP LV DEOH WR VKRZ WKH GLVWULEXWLRQ RI DOO HUURUV EHDW HUURUV LQ RXU GDWDVHW )RU RXU SXUSRVHG OfÀine Beat 7KHUHIRUH WKH UHVXOWV XQGHU 8 HYDOXDWLRQ PHWULFV FOHDUO\ Counting DOJRULWKP LWV %HDW (UURU +LVWRJUDP DV LOOXVWUDWHG GHPRQVWUDWH WKH IHDVLELOLW\ RI RXU SURSRVHG V\VWHP IRU DFFX 88 UDWHO\ WUDFNLQJ 6DOVD EHDWV IURP 6DOVD PXVLF VLJQDO FRPSDULQJ DORQJ ZLWK WKH GDQFH PXVLF 7KH SURSRVHG V\VWHP FDQ EH XVHG WR RWKHU H[LVWLQJ EHDW WUDFNLQJ V\VWHPV LQ PDQ\ VFHQDULRV HJ KRPH SUDFWLFH GDQFH VWXGLR DQG OLYH SHUIRUPDQFH ZKLFK FDQ QRW RQO\ KHOS 6DOVD EHJLQQHUV JHW 9,, ',6&866,21 DFFHOHUDWHG OHDUQLQJ SURFHVV EXW DOVR HOLPLQDWH SURIHVVLRQDO ,Q WKLV VHFWLRQ ZH PDLQO\ GLVFXVV WKH SRWHQWLDO LPSURYHPHQW GDQFHUV¶ HUURUV GXULQJ WKHLU OLYH SHUIRUPDQFH ([WHQVLYH WHVWV RI WKH SURSRVHG SalsaAsst V\VWHP ZKLFK ZLOO EH SUHVHQWHG LQ XVLQJ 40 6DOVD GDQFH VRQJV XQGHU 8 HYDOXDWLRQ PHWULFV VKRZ RXU IXWXUH ZRUN WKDW RXU V\VWHP FDQ DFKLHYH PXFK EHWWHU 6DOVD EHDW WUDFNLQJ SHUIRUPDQFH FRPSDULQJ WR WKH H[LVWLQJ EHDW WUDFNLQJ VROXWLRQV A. Additional Assistance from Inertial Sensors of Mobile Devices ,; $&.12:/('*0(17 SalsaAsst KDV WXUQHG RXW WR EH D SURPLVLQJ VWHS WRZDUGV 7KLV ZRUN ZDV VXSSRUWHG LQ SDUW E\ WKH 1DWLRQDO 6FLHQFH HQDEOLQJ WKH PRELOH GHYLFH WR EH D YLUWXDO DVVLVWDQW IRU GDQFLQJ )RXQGDWLRQ XQGHU JUDQW QXPEHUV &16 DQG &16 6DOVD ZKLFK FDQ KHOS GDQFHUV WR UHFRJQL]H WKH FRUUHFW WLPLQJV RI 6DOVD EHDWV %\ H[SORLWLQJ PRUH VHQVRUV VXFK DV LQHUWLDO 5()(5(1&(6 VHQVRUV LH DFFHOHURPHWHU DQG J\URVFRSH LQ PRELOH GHYLFHV LW LV SRVVLEOH WR SURYLGH DGGLWLRQDO DVVLVWDQFH WR 6DOVD GDQFHUV >@ ' 'UREQ\ et al. ³6DOWDWH D VHQVRUEDVHG V\VWHP WR VXSSRUW GDQFH EH JLQQHUV´ LQ CHI’09 Extended Abstracts on Human Factors in Computing )RU LQVWDQFH ZH FDQ XWLOL]H WKH EXLOWLQ LQHUWLDO VHQVRUV RI WKH Systems $&0 SS ± VPDUWSKRQHV WKDW DUH ZRUQ ZLWK GDQFHUV WR GHULYH WKHLU GDQFLQJ >@ * 3HHWHUV ³7HPSODWHEDVHG HVWLPDWLRQ RI WLPHYDU\LQJ WHPSR´ PRYHPHQWV WR VHH ZKHWKHU WKH PRYHPHQWV DUH V\QFKURQL]HG WR EURASIP Journal on Applied Signal Processing YRO QR SS ± WKH GHWHFWHG 6DOVD EHDWV ,W FDQ EH XVHG WR HYDOXDWH GDQFHUV¶ >@ $ 7 &HPJLO et al. ³2Q WHPSR WUDFNLQJ 7HPSRJUDP UHSUHVHQWDWLRQ DQG GDQFLQJ V\QFKURQL]DWLRQ GXULQJ SUDFWLFH DQG KHOS WKHP LP NDOPDQ ¿OWHULQJ´ Journal of New Music Research YRO QR SS SURYH LW E\ SURYLGLQJ DSSURSULDWH IHHGEDFN LI GDQFHUV KDYH DQ\ ± >@ ' 3 (OOLV ³%HDW WUDFNLQJ E\ G\QDPLF SURJUDPPLQJ´ Journal of New HUURU PRYHPHQWV ,Q WKLV FLUFXPVWDQFH SalsaAsst ZLOO QRW RQO\ Music Research YRO QR SS ± DVVLVW LQ LGHQWLI\LQJ 6DOVD EHDWV EXW DOVR HYDOXDWH WKH DFWXDO >@ )+ ) :X et al. ³$ WZRIROG G\QDPLF SURJUDPPLQJ DSSURDFK WR EHDW GDQFLQJ VWHSV DQG FRUUHFW WKHLU PLVWDNHV LI DQ\ WUDFNLQJ IRU DXGLR PXVLF ZLWK WLPHYDU\LQJ WHPSR´ LQ ISMIR SS ± >@ 0('DYLHVet al. ³&RQWH[WGHSHQGHQW EHDW WUDFNLQJ RI PXVLFDO DXGLR´ B. Remove the Requirement of Pre-listening in Online Beat IEEE Transactions on Audio, Speech, and Language Processing YRO Counting Mode QR SS ± >@ $ 1DNDPXUD et al. ³0XOWLPRGDO SUHVHQWDWLRQ PHWKRG IRU D GDQFH 'LIIHUHQW IURP RWKHU UHDOWLPH EHDW WUDFNLQJ V\VWHPV WKH WUDLQLQJ V\VWHP´ LQ CHI’05 extended abstracts on Human factors in Online Beat Counting Mode RI RXU V\VWHP QHHG WR OLVWHQ IRU computing systems $&0 SS ± D VKRUW WLPH SHULRG HJ 10 VHFRQGV RI PXVLF EHIRUH UHDO >@ 0 2KNLWD et al. ³$XGLRYLVXDO EHDW WUDFNLQJ EDVHG RQ D VWDWHVSDFH PRGHO IRU D PXVLF URERW GDQFLQJ ZLWK KXPDQV´ LQ 2015 IEEE/RSJ WLPH 6DOVD EHDWV FRXQWLQJ ZKLFK LV XVHG IRU HVWLPDWLQJ 6DOVD International Conference on Intelligent Robots and Systems ,((( %HDW ,QWHUYDO 6%, ZKLFK GHWHUPLQHV WKH PHWULFDO OHYHO RI SS ± 6DOVD PXVLF +RZHYHU VXFK VKRUW SHULRG OLVWHQLQJ LV MXVW D > @ 7 0L]XPRWR et al. ³$ URERW OLVWHQV WR PXVLF DQG FRXQWV LWV EHDWV DORXG E\ VHSDUDWLQJ PXVLF IURP FRXQWLQJ YRLFH´ LQ 2008 IEEE/RSJ VPDOO VHFWLRQ RI DQ HQWLUH 6DOVD PXVLF ZKLFK ZRXOG QRW EH International Conference on Intelligent Robots and Systems ,((( LQFRQYHQLHQW IRU GDQFHUV WR XVH RXU V\VWHP LQ SUDFWLFH ,Q WKH SS ± IXWXUH ZH ZLOO UHPRYH VXFK D UHTXLUHPHQW RI SUHOLVWHQLQJ E\ >@ 6 3LHWUREUXQR ³(PERG\LQJ FDQDGLDQ PXOWLFXOWXUDOLVP WKH FDVH RI VDOVD GDQFLQJ LQ PRQWUHDO´ Revista mexicana de estudios canadienses GHVLJQLQJ D TXLFNHU 6%, HVWLPDWLRQXSGDWH PHWKRG ZKLFK FDQ >@ 3 5HQWD ³6DOVD GDQFH /DWLQRD KLVWRU\ LQ PRWLRQ´ Centro Journal TXLFNO\ FDSWXUH WKH 6%, ZLWKLQ MXVW 23 PXVLF EHDWV LH OHVV YRO QR WKDQ 1 VHFRQG DQG XSGDWH LW DFFRUGLQJO\ >@ %DU PXVLF >2QOLQH@ $YDLODEOH KWWSV HQZLNLSHGLDRUJZLNL%DU PXVLF >@ ( ' 6FKHLUHU ³7HPSR DQG EHDW DQDO\VLV RI DFRXVWLF PXVLFDO VLJQDOV´ 9,,, &21&/86,21 The Journal of the Acoustical Society of America YRO QR SS 7KLV SDSHU SURSRVHV D EHDW FRXQWLQJ V\VWHP SalsaAsst ± >@ 6 'L[RQ ³2QVHW GHWHFWLRQ UHYLVLWHG´ LQ Proceedings of the 9th Inter- EDVHG RQ PRELOH GHYLFHV HJ VPDUWSKRQHV WR DVVLVW XVHUV WR national Conference on Digital Audio Effects YRO &LWHVHHU GDQFH 6DOVD SalsaAsst FDQ SHUIRUP UK\WKP GHFLSKHULQJ DQG SS ± ¿QHJUDLQHG EHDW WUDFNLQJ IRU 6DOVD PXVLF DQG JHQHUDWH EHDW >@ ' 'HXWVFK Psychology of music (OVHYLHU >@ & ( 5LFH ³+XPDQ HFKR SHUFHSWLRQ´ Science YRO QR SS FRXQWLQJ YRLFH RU YLEUDWLRQ SURPSWV WR UHPLQG WKH GDQFHUV ± RI 6DOVD EHDW WLPLQJV 7ZR DOWHUQDWLYH PRGHV DUH GHYHORSHG WR >@ 0 ( 'DYLHV et al. ³(YDOXDWLRQ PHWKRGV IRU PXVLFDO DXGLR EHDW WUDFNLQJ SURYLGH SURPSWV ZLWK GDQFHUV LQ RIÀLQH DQG UHDOWLPH PDQQHUV DOJRULWKPV´ Queen Mary University of London, Centre for Digital Music, Tech. Rep. C4DM-TR-09-06 UHVSHFWLYHO\ 6SHFL¿FDOO\ LQ WKH RIÀLQH PRGH SalsaAsst LV DEOH >@ 0 ) 0F.LQQH\ et al. ³(YDOXDWLRQ RI DXGLR EHDW WUDFNLQJ DQG PXVLF WR SURFHVV DQ\ 6DOVD PXVLF ¿OH WR JHQHUDWH D QHZ PXVLF ¿OH WHPSR H[WUDFWLRQ DOJRULWKPV´ Journal of New Music Research YRO ZKLFK SRVVHVVHV ERWK RULJLQDO 6DOVD PXVLF DQG LWV FRUUHVSRQG QR SS ± > @ 0 ( 'DYLHV et al. ³2Q WKH XVH RI HQWURS\ IRU EHDW WUDFNLQJ HYDOXDWLRQ´ LQJ EHDW FRXQWLQJ YRLFH SURPSWV ,Q WKH UHDOWLPH PRGH RXU LQ 2007 IEEE International Conference on Acoustics, Speech and Signal V\VWHP OHYHUDJHV WKH HPEHGGHG PLFURSKRQH RQ WKH PRELOH Processing YRO ,((( SS ,9± GHYLFH WR UHFHLYH WKH DFRXVWLF VLJQDO RI 6DOVD PXVLF DQG UDLVH WKH UHDOWLPH SURPSWV LH EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ 89