6RUWLQJ<RXU/LQJXLVWLF'DWD LQVLGHWKH2UDFOHL'DWDEDVH $Q2UDFOH7HFKQLFDO:KLWH3DSHU 6HSWHPEHU 6RUWLQJ<RXU/LQJXLVWLF'DWD LQVLGHWKH2UDFOHL'DWDEDVH ,QWURGXFWLRQ 7KH:RUOGRI6RUWLQJ5XOHV :HVWHUQ(XURSHDQ/DQJXDJHV $VLDQ,GHRJUDSKV ,62,(&²,QWHUQDWLRQDO6WULQJ2UGHULQJ 6RUWLQJLQVLGHWKH2UDFOH'DWDEDVH %LQDU\6RUW 0RQROLQJXDO/LQJXLVWLF6RUW 0XOWLOLQJXDO/LQJXLVWLF6RUW /LQJXLVWLF6RUW3DUDPHWHUV 8VLQJD/LQJXLVWLF,QGH[ 5HTXLUHPHQWVIRU8VLQJ/LQJXLVWLF,QGH[HV &DVH,QVHQVLWLYH6HDUFK *(1(5,&B%$6(/(77(56RUW &XVWRPL]DWLRQRI/LQJXLVWLF6RUWV 6XPPDU\ Sorting Your Linguistic Data Inside the Oracle9i Database Page 2 6RUWLQJ<RXU/LQJXLVWLF'DWD LQVLGHWKH2UDFOHL'DWDEDVH INTRODUCTION 6RUWLQJFKDUDFWHUVWULQJVFDQEHDQH[WUHPHO\LQWULFDWHRSHUDWLRQWKHFRPSOH[LW\RI ZKLFKPD\QRWEHDSSDUHQWWRPRVWXVHUV'LIIHUHQWODQJXDJHVKDYHWKHLURZQ VRUWLQJUXOHVVRPHODQJXDJHVDUHFROODWHGDFFRUGLQJWRWKHOHWWHUVHTXHQFHLQWKH DOSKDEHWVRPHDFFRUGLQJWRWKHQXPEHURIVWURNHFRXQWVLQWKHOHWWHUDQGWKHUH DUHHYHQODQJXDJHVZKLFKDUHRUGHUHGE\WKHSURQXQFLDWLRQRIWKHZRUGV 7UHDWPHQWRIOHWWHUDFFHQWVGLIIHUVDPRQJODQJXDJHVDVZHOO 6RUWLQJLVQRWVWUDLJKWIRUZDUGIRUGD\WRGD\(QJOLVKXVDJHHLWKHU,I\RXORRNXS ZRUGVLQDQ(QJOLVKGLFWLRQDU\\RXZLOOSUREDEO\ILQGWKDWXSSHUDQGORZHUFDVH FKDUDFWHUVDUHPL[HGWRJHWKHULQWHUPVRIRUGHULQJ/RRNLQJIRUQDPHVLQWKH WHOHSKRQHGLUHFWRU\\RXPD\ILQGWKDWFHUWDLQZRUGVPD\EHWUHDWHGWKHVDPHIRU H[DPSOHWKHSUHIL[0DFDQG0FDUHJURXSHGWRJHWKHU 6RUWLQJFDQEHIXUWKHUFRPSOLFDWHGZKHQ\RXQHHGWRVRUWGDWDIURPPRUHWKDQ RQHODQJXDJHKRZGR\RXKDQGOHWKHFDVHZKHQRQHODQJXDJHGHPDQGVWKDWD JLYHQOHWWHULVVRUWHGDIWHUWKHOHWWHU]ZKLOHWKHVDPHOHWWHULQDQRWKHUODQJXDJH UHTXLUHVLWWREHFROODWHGEHIRUHWKHOHWWHUD" ,QWKHUHVWRIWKLVSDSHUZHZLOOXVHWKHWHUP¶OLQJXLVWLFVRUW·RUFROODWLRQWRGHVFULEH WKHFXOWXUDOO\H[SHFWHGRUGHULQJRIHOHPHQWVRIDWH[WOLVWLIWKLVOLVWLVFRQVLGHUHG VRUWHGLQDJLYHQODQJXDJH$QDWLYHVSHDNHURIWKHODQJXDJHH[SHFWVWRORFDWHDQ HOHPHQWRIWKHOLVWUHODWLYHWRRWKHUHOHPHQWVEDVHGRQWKLVRUGHULQJ)RUH[DPSOH (QJOLVKVSHDNLQJXVHUVZRXOGH[SHFWWRILQGDZRUGEHJLQQLQJZLWK%WRFRPH DIWHUDOOZRUGVVWDUWLQJZLWK$DQGEHIRUHDOOZRUGVEHJLQQLQJZLWK&LQDQRUGHUHG OLVWRI(QJOLVKZRUGV 3UHVHQWLQJGDWDQRWVRUWHGLQWKHOLQJXLVWLFVHTXHQFHWKDW\RXUXVHUVDUH DFFXVWRPHGWRFDQPDNHVHDUFKLQJIRULQIRUPDWLRQGLIILFXOWDQGWLPHFRQVXPLQJ :LWKWKH2UDFOHLGDWDEDVHZHKDYHH[SDQGHGRXUFRYHUDJHRIELQDU\VRUWV OLQJXLVWLFVRUWVDQGDGGHGWKHQHZPXOWLOLQJXDOOLQJXLVWLFVRUWVWRPHHWWKHGHPDQGV RIFXVWRPHUVZKRQHHGWRVHDUFKDQGVRUWGDWDLQPXOWLSOHODQJXDJHV7KLVSDSHU RXWOLQHVWKHEDVLFFRQFHSWRIOLQJXLVWLFVRUWSURYLGHVH[DPSOHVRIKRZVRUW SURSHUWLHVLQIOXHQFHWKHFROODWLRQRUGHURIGLIIHUHQWODQJXDJHVDQGH[SODLQVKRZ Sorting Your Linguistic Data Inside the Oracle9i Database Page 3 \RXFDQFXVWRPL]HWKHZD\\RXUGDWDLVVRUWHGLQVLGHDQ2UDFOHGDWDEDVH THE WORLD OF SORTING RULES 7KHVHFWLRQDERYHPHQWLRQHGVRPHRIWKHLQWULFDFLHVLQYROYHGZKHQVRUWLQJGDWDLQ DFXOWXUDOO\H[SHFWHGRUGHU7KLVVHFWLRQZLOOJRRYHUWKHPLQPRUHGHWDLOVKRZLQJ WKDWHYHU\ZULWLQJVFULSW HJ/DWLQ*UHHN&\ULOOLFHWF KDVDQRUGHULQJRILWVRZQ DQGVRPHZULWLQJVFULSWVKDYHPDQ\FRQIOLFWLQJRUGHULQJVIRUWKHGLIIHUHQW ODQJXDJHVWKDWWKH\VXSSRUW Western European Languages ,Q*HUPDQLVRQHFKDUDFWHUEXWLWLVWUHDWHGDVGRXEOHV¶VV·IRURUGHULQJ SXUSRVHV*HUPDQVSHDNHUVDUHDOVRDFFXVWRPHGWRWUHDWLQJlDQG|DVHTXLYDOHQW WRXHDHDQGRHUHVSHFWLYHO\7KHVHWKUHHFKDUDFWHUVDUHVRUWHGLQWKHVDPHRUGHUDV WKHYRZHOSDLUV 6SDLQWUDGLWLRQDOO\WUHDWVFKOODVZHOODVxDVOHWWHUVRIWKHLURZQRUGHUHGDIWHUFO DQGQUHVSHFWLYHO\)RUH[DPSOHWKHIROORZLQJ6SDQLVKZRUGVZRXOGEHVRUWHGDV OLVWHG cabalmente, caballa, cantina, caña, clamar, curador, chácara 5HFHQWO\WKLVWUDGLWLRQDO6SDQLVKVRUWLQJSUDFWLFHKDVEHHQUHSODFHGZLWKWKH PRGHUQ6SDQLVKVRUWZKLFKUHPRYHVWKHVSHFLDOVWDWXVRIFKDQGOO ,QWKH'DQLVKDOSKDEHWWKUHHDGGLWLRQDOOHWWHUVQDPHO\ ¡DQGnVRUWDIWHU],W DOVRWUHDWVWKHOHWWHUFRPELQDWLRQDDDVDVHSDUDWHOHWWHURUGHUHGODVWLQWKHDOSKDEHW DIWHUnDQGDOVRFRQVLGHUVOHWWHUVDQG\DVYDULDQWVRIWKHVDPHOHWWHU'XHWRWKHVH UXOHVWKHIROORZLQJFLWLHV=ULFK$DUILWDQGcUEXVZLOODOODSSHDUDIWHU=\UDUGRZLQ DOLVWRIFLWLHV ,Q)UHQFKDFFHQWHGYRZHOVDUHILUVWWUHDWHGOLNHXQDFFHQWHGYRZHOVIURPWKHSRLQW RIYLHZRIWKHLUEDVHVRUWLQJRUGHU$IWHUWKHWH[WLVVRUWHGZLWKRXWFRQVLGHULQJ DFFHQWVLWLVIXUWKHUVRUWHGVRWKDWZLWKLQHDFKYRZHOVHWWKHRUGHULVQRDFFHQW DFXWHDFFHQWJUDYHDFFHQWFLUFXPIOH[DFFHQWIROORZHGE\XPODXW7RPDNHWKLV HYHQPRUHFRPSOH[DFFHQWHGYRZHOVDUHHYDOXDWHGIURPULJKWWROHIWZKLOHQRQ DFFHQWHGEDVHFKDUDFWHUVDUHZHLJKWHGIURPOHIWWRULJKW7KXVËGLWFRPHVEHIRUH (GtWZKHQFROODWLQJWKHWZRVWULQJVXVLQJD)UHQFKOLQJXLVWLFVRUW $FFHQWHGOHWWHUVLQWURGXFHRWKHUFRPSOH[LW\DVWKH\FDQKDYHGLIIHUHQWPHDQLQJV DFURVVODQJXDJHVDQGVRPHWLPHVHYHQZLWKLQWKHVDPHODQJXDJH&RPSOLFDWLRQ DULVHVZKHQGHDOLQJZLWKPXOWLQDWLRQDOFRGLQJVFKHPHVVXFKDV8QLFRGHWKDW VXSSRUWVHYHUDOODQJXDJHVVLQFHWKHVHODQJXDJHVPD\FRQWDLQFRQIOLFWLQJDOSKDEHWLF Sorting Your Linguistic Data Inside the Oracle9i Database Page 4 UXOHVIRUWKHVDPHOHWWHU)RUH[DPSOHWKHOHWWHUl DZLWKDQXPODXW LVVRUWHG EHIRUHELQ*HUPDQEXWLWLVVRUWHGDIWHU]LQ6ZHGLVK7KXVLIWKHVRUWHGWH[W EHORQJVWRPXOWLSOHODQJXDJHVLWXVXDOO\FDQQRWEHVRUWHGFRUUHFWO\IURPWKHSRLQW RIYLHZRIDOORIWKHP Asian Ideographs $V\RXFDQVHHFXOWXUDOH[SHFWDWLRQVYDU\DJUHDWGHDOEHWZHHQODQJXDJHVDQGWKH SUHYLRXVH[DPSOHVDUHMXVWIRU:HVWHUQ(XURSHDQODQJXDJHVXVLQJWKHVDPH/DWLQ ZULWLQJVFULSW,IGHDOLQJZLWKWROHWWHUVLQDQDOSKDEHWLVTXLWHDFKDOOHQJH WKHQZRUNLQJZLWK$VLDQLGHRJUDSKVFDQEHDQH[WUHPHO\GDXQWLQJWDVN)RUWKH &KLQHVHODQJXDJHDORQHWKHUHDUHRYHUFKDUDFWHUV7KHVRUWLQJUXOHVDOVR YDU\EHWZHHQ6LPSOLILHG XVHGLQ&KLQDDQG6LQJDSRUH DQG7UDGLWLRQDO XVHGLQ 7DLZDQ+RQJ.RQJ 0DFDX &KLQHVHZULWLQJVFULSWV +HUHDUHVRPHH[DPSOHVRIWKHPRVWFRPPRQFROODWLRQPHWKRGVXVHGIRUVRUWLQJ &KLQHVHFKDUDFWHUV 6WURNHFRXQW7KLVLVRQHRIWKHPRVWFRPPRQZD\VWRVRUW&KLQHVHFKDUDFWHUV &KLQHVHFKDUDFWHUVDUHFRPSRVHGRIDUDGLFDO EDVHHOHPHQW DQG]HURRUPRUH FRPSRQHQWV%RWKWKHUDGLFDODQGWKHFRPSRQHQWVDUHFRPSRVHGRIVWURNHV7KH QXPEHURIVWURNHVYDULHVEHWZHHQFKDUDFWHUV&KDUDFWHUVZLWKLGHQWLFDOVWURNH FRXQWVZLOOEHIXUWKHUVRUWHGE\WKHUDGLFDO 7KLVH[DPSOHLOOXVWUDWHVKRZWKUHH&KLQHVHFKDUDFWHUVDUHRUGHUHGEDVHGRQWKHLU VWURNHFRXQWV +RZHYHUWKHQXPEHURIVWURNHFRXQWVIRUDJLYHQ&KLQHVHFKDUDFWHUPD\GLIIHU EHWZHHQLWVVLPSOLILHGDQGWUDGLWLRQDOIRUP+HQFHLWLVFRPPRQWRKDYHWZR GLVWLQFWVRUWLQJDOJRULWKPVWRKDQGOHERWKZULWLQJVW\OHV 7KHH[DPSOHEHORZVKRZVKRZWKHWZRIRUPVRIWKH&KLQHVHFKDUDFWHU¶JUDVV·DUH WUHDWHGGLIIHUHQWO\LQWHUPVRIVWURNHFRXQWV 6LPSOLILHG&KLQHVH7UDGLWLRQDO&KLQHVH RI6WURNHV Sorting Your Linguistic Data Inside the Oracle9i Database Page 5 5DGLFDO7KLVRUGHULQJLVIRXQGLQPRVW&KLQHVHGLFWLRQDULHV,WLVVLPLODUWRWKH VWURNHFRXQWRUGHULQJEXWKHUHWKHUDGLFDORUGHULQJWDNHVSULRULW\RYHUWKHVWURNH FRXQW$OWKRXJKLWLVDPRUHWUDGLWLRQDOPHWKRGRIVRUWLQJ&KLQHVHFKDUDFWHUV VRPHWLPHVLWFDQEHTXLWHFXPEHUVRPHHYHQIRUQDWLYH&KLQHVHVSHDNHUVVLQFHLW PD\QRWEHLQWXLWLYHWRORFDWHWKHFRUUHFWUDGLFDOIRUDJLYHQ&KLQHVHFKDUDFWHU7KH RUGHURIWKHUDGLFDOLVE\WKHUDGLFDOVWURNHFRXQW)RUFKDUDFWHUVZLWKLGHQWLFDO UDGLFDORUGHUWKHQXPEHURIVWURNHFRXQWVRIWKHFKDUDFWHUZLOOEHXVHG 7KLVH[DPSOHLOOXVWUDWHVKRZWKUHH&KLQHVHFKDUDFWHUVDUHRUGHUHGEDVHGRQWKHLU UDGLFDOVWURNHFRXQWV +RZHYHUVRPHUDGLFDOVZLOOFKDQJHVKDSHZKHQWKH\DUHELQGZLWKRWKHUFKDUDFWHU FRPSRQHQWVWRIRUPWKHILQDOFKDUDFWHU7KHRUGHURIDUDGLFDOLVDOZD\VEDVHGRQ LWVRULJLQDOIRUP 7KHH[DPSOHEHORZVKRZWKHRUGHURIIRXUFKDUDFWHUVEDVHGRQWKHRULJLQDOVWURNH FRXQWRIWKHLUUDGLFDOV1DPHVLQSDUHQWKHVHVVKRZWKH8QLFRGHFRGHVRIWKHVH FKDUDFWHUV 7RDGGHYHQPRUHFRPSOH[LW\WRVRUWLQJUDGLFDOVMXVWOLNHLQWKHSUHYLRXVVWURNH FRXQWVRUWVRPH&KLQHVHFKDUDFWHUVPD\KDYHDGLIIHUHQWYDULDWLRQRIWKHUDGLFDO EHWZHHQWKHVLPSOLILHGDQGWUDGLWLRQDOIRUPRI&KLQHVH+HQFHLWLVFRPPRQWR KDYHWZRGLVWLQFWVRUWLQJDOJRULWKPVWRKDQGOHWKHGLIIHUHQFHVEHWZHHQWKHWZR ZULWLQJVW\OHV 7KHH[DPSOHEHORZGHPRQVWUDWHVWKHUDGLFDOGLIIHUHQFHVEHWZHHQ6LPSOLILHG &KLQHVH 6& DQG7UDGLWLRQDO&KLQHVH 7& IRUWKHVDPHVHWRIFKDUDFWHUV Sorting Your Linguistic Data Inside the Oracle9i Database Page 6 3URQXQFLDWLRQ 3LQ\LQ ²3LQ<LQLVWKHSURQXQFLDWLRQEDVHGVRUWIRU6LPSOLILHG &KLQHVH%RSRPRIRLVLW·VFRXQWHUSDUWIRU7UDGLWLRQDO&KLQHVH3URQXQFLDWLRQVRUW VRXQGVVWUDLJKWIRUZDUGXQWLO\RXHQFRXQWHUZRUGVWKDWFDQSURQRXQFHGLIIHUHQWO\ GHSHQGLQJRQWKHFRQWH[W7KHUHDUHDOVRFDVHVZKHUHFKDUDFWHUVZLWKWKHVDPH 3LQ<LQFDQKDYHILYHGLIIHUHQWWRQHVWKHUHDUHIRXUWRQHVLQVSRNHQ&KLQHVHSOXVD VSHFLDOOLJKWWRQH7RQHVDUHPDUNHGIURPWRZLWKWKHOLJKWWRQHEHLQJWKHILIWK WRQHDQGWKHODVWLQWKHRUGHURIWRQHV&KDUDFWHUVZLWKWKHVDPH3LQ\LQDQGWRQH DUHIXUWKHUVRUWHGE\WKHVWURNHFRXQW 7KHH[DPSOHEHORZLOOXVWUDWHVKRZIRXU&KLQHVHFKDUDFWHUVDUHRUGHUHGEDVHGRQ WKHLU3LQ<LQVHTXHQFH1DPHVLQSDUHQWKHVHVVKRZWKHSLQ\LQDQGWKHWRQHYDOXHV RIWKHVHFKDUDFWHUV ISO/IEC 14651 – INTERNATIONAL STRING ORDERING ,QWKHPLGVD4XpEHFFHU$ODLQ/D%RQWpZDVVXUSULVHGDQGSX]]OHGZLWKWKH LQFRQVLVWHQWEHKDYLRUZLWKZKLFKWKHGLIIHUHQWFRPPHUFLDOVRUWSURJUDPVGHDOW ZLWK)UHQFKKRPRJUDSKZRUGVHYHQWKRXJKWKHVHZHUHLQWXLWLYHWR)UHQFKQDWLYH VSHDNHUV+HUHDOL]HGWKHQHHGIRUDQLQWHUQDWLRQDOVWDQGDUGWKDWZRXOGDOORZ GHILQLWLRQRIDXQLYHUVDOPHWKRGRORJ\IRUPXOWLVFULSWRUGHULQJ ,QWKHSURMHFWIRUDQLQWHUQDWLRQDOVWULQJRUGHULQJVWDQGDUG,62ZDV FUHDWHG7KLVLQWHUQDWLRQDOVWDQGDUGSURYLGHVDPHWKRGIRURUGHULQJWH[WGDWD ZRUOGZLGHDQGSURYLGHVD&RPPRQ7HPSODWH7DEOHZKRVHWDLORULQJPHHWVWKH UHTXLUHPHQWVRIDJLYHQODQJXDJHDQGFXOWXUHZKLOHUHWDLQLQJXQLYHUVDOSURSHUWLHV IRURWKHUVFULSWV Sorting Your Linguistic Data Inside the Oracle9i Database Page 7 7KH&RPPRQ7HPSODWH7DEOHUHTXLUHVVRPHWDLORULQJLQGLIIHUHQWORFDO HQYLURQPHQWV+RZHYHUFRQIRUPDQFHWRWKLV,QWHUQDWLRQDO6WDQGDUGUHTXLUHVWKDW DOOGHYLDWLRQVIURPWKH7HPSODWHFDOOHGGHOWDVEHGHFODUHGWRGRFXPHQWUHVXOW GLVFUHSDQFLHV7KLV6WDQGDUGGHVFULEHVDPHWKRGWRRUGHUWH[WGDWDLQGHSHQGHQWO\ RIFRQWH[W SORTING INSIDE THE ORACLE DATABASE 7H[WLVFRQYHQWLRQDOO\VRUWHGLQVLGHDGDWDEDVHDFFRUGLQJWRWKHELQDU\FRGHVXVHG WRHQFRGHWKHFKDUDFWHUV7\SLFDOO\WKLVGRHVQRWSURGXFHDVRUWRUGHUWKDWLV OLQJXLVWLFDOO\FRUUHFW,QVRPHFDVHVLWFDQEHFRUUHFWLIWKHJLYHQHQFRGLQJVFKHPH VSHFLILHVDOOWKHFKDUDFWHUVLQDVFHQGLQJELQDU\YDOXHDFFRUGLQJWRWKHDSSURSULDWH DOSKDEHWLFFRQYHQWLRQ8QIRUWXQDWHO\PRVWHQFRGLQJVFKHPHVGRQRWIROORZDQ\ VXFKFRQYHQWLRQDQGHYHQLIWKH\GRLWLVQRWSRVVLEOHIRUWKHPWRFRYHUWKHPRUH