Clumps, Hoops, and Bubbles How Akamai Maps the Net Compressed
Total Page:16
File Type:pdf, Size:1020Kb
The National Security Agency’s Review of Emerging Technologies 6 £n Î U Óä£ä Clumps, Hoops, and Bubbles How Akamai Maps the Net Compressed Sensing and Network Monitoring Revealing Social Networks of Spammers Challenges in Internet Geolocation NSA’s Review of Emerging Technologies The Letter from the Editor Next Wave 7KLV LVVXH RI 7KH 1H[W :DYH LV ODUJHO\ GHULYHG IURP WDONV JLYHQ DW WKH DQG 1HWZRUN 0DSSLQJ DQG 0HDVXUHPHQW &RQIHUHQFHV 100&V KHOG DW WKH /DERUDWRU\ IRU 7HOHFRPPXQLFDWLRQV 6FLHQFHV /76 LQ &ROOHJH 3DUN 0DU\ODQG 7KHVH FRQIHUHQFHV HYROYHG IURP WKH 1HW7RPR ZRUNVKRSV ,,9 ZKLFK JUHZ RXW RI UHVHDUFK RQ QHWZRUN WRPRJUDSK\ VSRQVRUHG E\ WKH ,QIRUPDWLRQ 7HFKQRORJ\ ,QGXVWU\ &RXQFLO ,7,& %\ LW EHFDPH REYLRXV WKDW D EURDGHU VFRSH ZDV QHHGHG WKDQ VWULFWO\ QHWZRUN WRPRJUDSK\ DQG WKH QDPH FKDQJH ZDV LQVWLWXWHG LQ 1HWZRUN WRPRJUDSK\ DQG PDSSLQJ DUH FORVHO\ UHODWHG ÀHOGV 7R H[SODLQ WKH GLIIHUHQFH EHWZHHQ WRPRJUDSK\ DQG PDSSLQJ KHUH DUH WZR VLPSOH GHÀQLWLRQV 1HWZRUN WRPRJUDSK\ LV WKH VWXG\ RI D QHWZRUN·V LQWHUQDO FKDUDFWHULVWLFV XVLQJ LQIRUPDWLRQ GHULYHG IURP HQGSRLQW GDWD 1HWZRUN PDSSLQJ LV WKH VWXG\ RI WKH SK\VLFDO FRQQHFWLYLW\ RI WKH ,QWHUQHW GHWHUPLQLQJ ZKDW VHUYHUV DQG RSHUDWLQJ V\VWHPV DUH UXQQLQJ DQG ZKHUH $ GHHSHU H[SODQDWLRQ RI WRPRJUDSK\ IROORZV )RU D ORQJHU GLVFXVVLRQ RI PDSSLQJ SOHDVH VHH WKH DUWLFOH ´0DSSLQJ 2XW )DVWHU 6DIHU 1HWZRUNVµ 1HWZRUN WRPRJUDSK\ LV JHQHUDOO\ RI WZR W\SHV³ERWK RI WKHP PDVVLYH LQYHUVH SUREOHPV 7KH ÀUVW W\SH XVHV HQGWRHQG GDWD WR HVWLPDWH OLQNOHYHO FKDUDFWHULVWLFV 7KLV IRUP RI WRPRJUDSK\ RIWHQ LV DFWLYH LQ QDWXUH XVLQJ PDQ\ SLQJV WUDFHURXWHV DQG RWKHU PDSSLQJ WRROV WR REWDLQ WKH QHFHVVDU\ GDWD 'XH WR WKH ODUJH DPRXQW RI XQGHVLUDEOH WUDIÀF H[SHULHQFHG E\ PDQ\ QHWZRUNV URXWHUV RU RWKHU QHWZRUN HTXLSPHQW PD\ QRW UHVSRQG WR SLQJ RU WUDFHURXWH UHTXHVWV 7KLV GHÀFLHQF\ KDV OHG WR D VHFRQG IRUP RI QHWZRUN WRPRJUDSK\ WKDW LV VRPHWLPHV FDOOHG LQIHUHQWLDO QHWZRUN WRPRJUDSK\ 7KLV IRUP RI QHWZRUN WRPRJUDSK\ XVHV LQGLYLGXDO URXWHU RU QRGHOHYHO PHDVXUHPHQWV WR UHFRYHU SDWKOHYHO LQIRUPDWLRQ 7KLV GDWD FDQ EH REWDLQHG SDVVLYHO\ DQG LW GRHV QRW FUHDWH D WUDIÀF EXUGHQ WKDW KDV WKH SRWHQWLDO WR FKDQJH WKH ORJLFDO QHWZRUN VWUXFWXUH 7KH VWXG\ RI QHWZRUN WRPRJUDSK\ LQFOXGHV QHWZRUN WRSRORJ\ ERWK ORJLFDO DQG SK\VLFDO WKH RULJLQGHVWLQDWLRQ WUDIÀF PDWUL[ DQG TXDOLW\ RI VHUYLFH SDUDPHWHUV VXFK DV ORVV UDWHV RU GHOD\ FKDUDFWHULVWLFV $FFXUDWH DQG WLPHO\ LQIRUPDWLRQ DERXW WUDIÀF ÁRZV DUH QHFHVVDU\ IRU JRRG QHWZRUN PDQDJHPHQW 1HWZRUN WRPRJUDSK\ UHVHDUFK OHDGV WR RWKHU WRSLFV RI LQWHUHVW +RZ GR \RX PHDVXUH WKH QHWZRUN" :KDW NLQG RI QHWZRUNV GR WKHVH WHFKQLTXHV DSSO\ WR" 'RHV LW PDWWHU LI \RX WHVW SDUWV RI WKH QHWZRUN LQGLYLGXDOO\ DQG WKHQ SXW WKHP DOO WRJHWKHU RU GRHV WKH HQWLUH QHWZRUN QHHG WR EH LQ WKH WHVW" LQWHJUDWLRQ WHVWLQJ :KDW VHQVLQJ WHFKQLTXHV DUH EHVW WR XVH" ([DFWO\ ZKDW NLQG RI GDWD GR \RX QHHG WR JDWKHU" :KDW DERXW WHFKQLTXHV IURP RWKHU GLVFLSOLQHV VXFK DV VRFLDO QHWZRUNLQJ" :LOO WKH\ DSSO\ WR WKH QHWZRUNV \RX DUH LQWHUHVWHG LQ" +RZ GRHV LQGXVWU\ GR WKHLU QHWZRUN PDSSLQJ" :KDW DERXW DWWULEXWLRQ" 6RPH RI WKHVH TXHVWLRQV ZHUH DGGUHVVHG DW WKH 100& VHVVLRQV DQG WKHUHIRUH DUH DGGUHVVHG LQ WKH IROORZLQJ DUWLFOHV 6HH ´&RPSUHVVHG 6HQVLQJ DQG 1HWZRUN 0RQLWRULQJµ IRU H[DPSOH UHJDUGLQJ TXHVWLRQ QXPEHU IRXU DERYH 0DQ\ PRUH TXHVWLRQV DULVH LQ WKH VWXG\ RI QHWZRUN PDSSLQJ DQG PHDVXUHPHQW 7KH 100& VHULHV KDV EHHQ D KXJH VXFFHVV ZLWK SDUWLFLSDQWV IURP GLIIHUHQW FRXQWULHV IHGHUDO DJHQFLHV XQLYHUVLWLHV DQG LQGXVWU\ 100& ZLOO EH KHOG $XJXVW DW 0F*LOO 8QLYHUVLW\ LQ 0RQWUHDO &DQDGD The graph that appears on the cover of this issue of The Next Wave shows the router level connectivity of the Internet as measured by The Next Wave is published to disseminate significant technical the Internet Mapping Project. advancements in telecommunications and information technologies. The work is being commercially Mentions of company names or commercial products do not imply developed by Lumeta Corporation. endorsement by the US government. Articles present views of the authors and not necessarily those of NSA or the TNW staff. &UHGLW For more information, please contact us at [email protected] 3DWHQW V SHQGLQJ DQG FRS\ULJKW /XPHWD &RUSRUDWLRQ $OO ULJKWV UHVHUYHG CONTENTS FEATURES 0DSSLQJ 2XW )DVWHU 6DIHU 1HWZRUNV +RZ $NDPDL 0DSV WKH 1HW $Q ,QGXVWU\ 3HUVSHFWLYH &RPSUHVVHG 6HQVLQJ DQG 1HWZRUN 0RQLWRULQJ 5HYHDOLQJ 6RFLDO 1HWZRUNV RI 6SDPPHUV &KDOOHQJHV LQ ,QWHUQHW *HRORFDWLRQ RU :KHUH·V :DOGR 2QOLQH" &OXPSV +RRSV DQG %XEEOHV³0RYLQJ %H\RQG &OXVWHULQJ LQ WKH $QDO\VLV RI 'DWD Mapping Out Faster, Safer Networks Maps. We use them every day. Your GPS guides you to that new restaurant you’ve wanted to try. The information map in the mall points out where HERE is. Online gamers pull up battle maps to navigate virtual worlds. The social network of your friends and your friends’ friends weaves a cat’s cradle of intertwined relationships. Your computer files are stored in folders that are displayed hierarchically. Site maps lay out how web pages link up. And think how much easier life would be if you had a map of the labyrinth of telephone options you need to navigate—“Press 1 for hours and locations”…”Press 2 to report a problem”…”Press 3 for account information”— when you try to pay your electric bill over the phone. Maps don’t just show how things are connected. They can also identify trouble spots and weak points you need to be aware of. GPS maps are able to alert you to traffic tie ups due to accidents or lane closures so you can adjust your route. Your security system might display a floor plan that shows which windows and doors are unlocked so you can protect your property. Network mapping does the same things for the Internet, helping to direct traffic and expose vulnerabilities. Network mapping can happen at different layers of the Internet, including applications, routing, or physical infrastructure, or in different parts of the Internet. Because the Internet changes constantly, any map of any variety—there are many Internet maps and no two agree—addresses a moving target. 4 Mapping Out Faster, Safer Networks FEATURE Tracing network routes DGRSWHG WUDFHURXWH DV DQ HDV\ ZD\ WR H[SORLW QHWZRUN 1HWZRUN PDSV WUDFN WKH URXWHV LQIRUPDWLRQ YXOQHUDELOLWLHV ,W GLGQ·W WDNH F\EHUFULPLQDOV ORQJ SDFNHWV WDNH DFURVV DQ ,3 ,QWHUQHW SURWRFRO WR GLVFRYHU WKDW QRW RQO\ FDQ WKH XWLOLW\ EH XVHG WR QHWZRUN WR UHDFK D UHPRWH KRVW 1HWZRUN URXWLQJ LV ORFDWH D QHWZRUN·V ZHDN SRLQWV LQLWLDWLQJ WUDFHURXWH RSSRUWXQLVWLF DVVLJQLQJ SDFNHWV WR WKH ÀUVW DYDLODEOH IURP PXOWLSOH V\VWHPV FDQ ÁRRG D QHWZRUN WR ODXQFK URXWHU 7KLV DSSURDFK PHDQV WUDIÀF FDQ EH GLUHFWHG D GHQLDORIVHUYLFH DWWDFN DORQJ GLIIHUHQW SDWKV WR UHDFK D GHVWLQDWLRQ DQG The Internet Mapping Project WKH QXPEHU RI KRSV QHHGHG WR JHW WKHUH FDQ YDU\ 7UDFHURXWHV ZHUH LQLWLDOO\ XVHG E\ QHWZRUN 1HWZRUN PDSSLQJ PDNHV LW HDV\ WR YLVXDOL]H ZKDW DGPLQLVWUDWRUV WR WURXEOHVKRRW DQG WXQH ORFDO URXWHV DUH EHLQJ WDNHQ QHWZRUNV EXW WKH XWLOLW\ ZRXOG HYHQWXDOO\ EH 7KH WUDFHURXWH QHWZRUN XWLOLW\ ZDV LQWURGXFHG DSSOLHG RQ D PXFK ODUJHU VFDOH $V WKH :RUOG :LGH RQ 8QL[ RSHUDWLQJ V\VWHPV LQ WR PDS QHWZRUN :HE UDSLGO\ JUHZ LQ SRSXODULW\ GXULQJ WKH V WUDIÀF 9DULDQWV RI WKH WUDFHURXWH SURJUDP DUH XVHG WKH QHHG IRU D ZRUOGZLGH PDS ZDV UHDOL]HG (IIRUWV RQ RWKHU RSHUDWLQJ V\VWHPV³WUDFHFHUW DQG SLQJ WR PDS QHWZRUN WUDIÀF JOREDOO\ EHJDQ LQ HDUQHVW XWLOLWLHV DUH XVHG RQ :LQGRZV RSHUDWLQJ V\VWHPV ZLWK WKH ,QWHUQHW 0DSSLQJ 3URMHFW VWDUWHG E\ %LOO DQG WUDFHSDWK LV WKH QHWZRUN WRRO XVHG RQ FXUUHQW &KHVZLFN DQG +DO %XUFK DW %HOO /DEV LQ /LQX[ LQVWDOODWLRQV (YHU\ GD\ IRU HLJKW \HDUV WKH SURMHFW UHFRUGHG 1HWZRUN WHFKQLFLDQV XVH WKH WUDFHURXWH XWLOLW\ WUDFHURXWHV IRU WULOOLRQV RI SDFNHWV WUDYHOLQJ DFURVV WR WURXEOHVKRRW QHWZRUN SUREOHPV .QRZLQJ D KXQGUHGV RI WKRXVDQGV RI ,3 QHWZRUNV 7KH QHWZRUN SDFNHW·V WUDFHURXWH FDQ KHOS LGHQWLI\ IDLOHG URXWHUV PDS WKDW HPHUJHG SDLQWHG D SLFWXUH UHVHPEOLQJ D RU ÀUHZDOOV WKDW DUH REVWUXFWLQJ WUDIÀF 7UDFHURXWH VN\ ÀOOHG ZLWK ÀUHZRUNV RQ WKH )RXUWK RI -XO\ 6HH FDQ DOVR EH XVHG IRU SHQHWUDWLRQ WHVWLQJ WR KXQW IRU WKH FRYHU LPDJH IRU DQ H[DPSOH QHWZRUN HQWU\ SRLQWV WKDW FRXOG SRVH D VHFXULW\ ULVN 1RZ PDQDJHG E\ WKH /XPHWD &RUSRUDWLRQ +DFNHUV DUH HVSHFLDOO\ LQWHUHVWHG LQ ÀQGLQJ ZKLFK VSXQ RII IURP %HOO /DEV LQ WKH ,QWHUQHW EDFN GRRUV LQWR QHWZRUNV DQG WKH\ KDYH UHDGLO\ 0DSSLQJ 3URMHFW FRQWLQXHV WR FKDUW WKH EDFN URDGV The Next Wave Vol 18 No 3 2010 5 DQG WKRURXJKIDUHV RI ,QWHUQHW WUDIÀF 7KH JRDO RI WKH Address space SURMHFW KDV EHHQ WR SURYLGH JOREDO QHWZRUN YLVLELOLW\ $V HQWHUSULVHV DQG JRYHUQPHQW DJHQFLHV WU\ WKURXJK WKH DFFXUDWH PHDVXUHPHQW RI IRXU IDFWRUV WR EDODQFH WKH IRUFHV IRU QHWZRUN FKDQJH ZLWK WKH QHWZRUN WRSRORJ\ DGGUHVV VSDFH OHDNV UHTXLUHPHQWV IRU ULVN PDQDJHPHQW DQG FRPSOLDQFH DQG GHYLFH ÀQJHUSULQWV ,QGHSHQGHQW GLVFRYHU\ LQLWLDWLYHV ,7 VHFXULW\ PDQDJHUV DUH IDFHG ZLWK WKH SURFHVVHV DUH XVHG WR UHYHDO WKHVH IRXU FRPSRQHQWV IRUPLGDEOH WDVN RI VHFXULQJ ZKDW WKH\ DUHQ·W HYHQ WKDW GHÀQH D QHWZRUN DZDUH RI 7KH VROXWLRQ OLHV SDUWO\ LQ GLVFRYHULQJ DOO RI D QHWZRUN·V HQWLWLHV³WKRVH WKDW DUH DXWKRUL]HG DV Network topology ZHOO DV WKRVH WKDW DUH XQDXWKRUL]HG 1HWZRUN KRVW 1HWZRUN WRSRORJ\ GHVFULEHV WKH ÁRZ RI GLVFRYHU\ LV XVHG WR FRQGXFW D FHQVXV RI ,3 DGGUHVVHV QHWZRUN WUDIÀF DQG WKH ERWWOHQHFNV WKDW VORZ LW DFURVV SURWRFROV DQG UHYHDO NQRZQ DQG SUHYLRXVO\ GRZQ $ FRPSXWHU·V QHWZRUN GLVFRYHU\ VHWWLQJ XQGHWHFWHG QHWZRUN HQWLWLHV +RVW GLVFRYHU\ LV RQH DIIHFWV ZKHWKHU LW FDQ VHH RWKHU FRPSXWHUV RQ WKH RI WKH HDUOLHVW SKDVHV RI QHWZRUN UHFRQQDLVVDQFH QHWZRUN RU EH VHHQ E\ WKHP $ FRPSXWHU FDQ $GGUHVV