Usenet、Netnews
News
Usenet、NetNews InterNet News InterNet – Network News Protocol Transfer – Reader: tin Reader: INN Server: History of netnews History of newsgroupsHistory News operations NNTP • • • • • • News softwares Transfer protocol Introduction News outline News
Computer Center, CS, NCTU 2 Introduction to News mail mail forums and web - mail messages. mail - FAQ, spam. FAQ, The initial community Internet The initial board systems (BBS) bulletin Like e Hybrid between server a Absence of central dedicated administrator and • • • • • Usenet has significant cultural importance in the networkedworld in the importance cultural significant hasUsenet The format and transmission of Usenet articles is similar to that of that to is similar articles of Usenet transmission and The format e Internet Usenet is a worldwide distributed Internet discussion system. Internet distributed is a worldwideUsenet Introduction(1) news to
Computer Center, CS, NCTU 4 492 7.52 2011 2003 257 5.42 2002 2010 181 4.65 2001 2009 82 3.07 2000 2008 26 2.95 1999 2007 12 2.27 1998 2006 9 1.52 1997 2005 have” resource? have” - 1 4.5 1996 2004 (TB) (GB) http://groups.google.com Year Year • allows newsgroupallows posting. is a “must Usenet blogs, and mailing lists. mailing and blogs, to of Usenet to attempt newsgroupsbuy archive efforts to Google made and alsoone. Googlea web interface has searcha complete create Spamming: In 1994, advertisers began broadcasting messags to all messags to began broadcasting advertisers1994, In Spamming: newsgroupsat once. forums, Internet with respect to in importance diminished hasUsenet Introduction(2) news to Daily Vol. Daily Daily Vol. Daily
Computer Center, CS, NCTU 5 Introduction(3) news to
Computer Center, CS, NCTU 6 Tom Truscott, Steve Daniel Truscott, Tom Better performance performance Better for the grownnews traffic Rewritten in C by Rewritten Written in 1981 by Mark Horton and Matt Glickman Development in 1983 ~ 1989 was passed on to Rick Adams a message format compatible with Internet mail With Tom Truscott and Jim Ellis conceived conceived the idea in 1979 and Jim Ellis Tom Truscott Bourne shell scripts assisted with writing Steve Bellovin program The first “netnews” transport protocol as its UUCP “netnews” The public release of in 1980 • • • • • • • • • • B News A News Origins History(1)
Computer Center, CS, NCTU 7 up connections - forward batch - Unix Copy Protocol Unix Copy - and to - - Duplicate Duplicate articles Unix used over dial Mostly commonly for mail NNTP for news SMTPand protocols Replaced the TCP/IP by Bang path are still use within the newsto record network the nodes through passed which a message has Store • • • • • • UUCP History(2)
Computer Center, CS, NCTU 8 Network News Transfer Network NewsProtocol netnews articles in 1985 to distribute Introduced TCP/IP network over and is created Specification in 1986 Lapsley Brian Kantor by Phil port 563 for NNTPS for NNTP and TCP TCP port 119 Protocol) NNRP(Network Reader News articles Sending required • • • • • • NNTP History(3)
Computer Center, CS, NCTU 9 Released by Rich Salz in 1991 Salz in by Rich Released NNTP functionality The first news serverto fully integrate package NNTP and UUCPsupport used news server package The most commonly now actively by ISC(Internet Systems Consortium) developed Still Developed by Henry Spencer Developed Geoff Collyer in 1987 and code of B NewsRewrite the program in a cleaner, faster result to of a function The creation in 1992 news index command NNTP XOVER header and Allow news readers to information rapidly retrieve threading • • • • • • • • • • INN (InterNet News) INN (InterNet The CNEWS The CNEWS version History(4)
Computer Center, CS, NCTU 10 Like the public message boards on BBS. on boards message public the Like Over 100000 Usenet newsgroups Types: text or binary. • • • Newsgroups A usenet newsgroup is a repository, a discussion group. discussiona a repository, newsgroupA is usenet The articles are organized into topical categories called newsgroups. called topicalcategories into organized are The articles it making theoretically hierarchies, intoNewsgroups are organized groups. related to find simpler newsgroups (1) newsgroups
Computer Center, CS, NCTU 11 European networks refuse networksfor to payEuropeansome of the highcontent volumelow and as groupssuchreligion racism. and suggestedcategorytalk.* The was fr.*from for groupsgatewayedARPANET mod.*for moderated discussions net.* for unmoderatedgroups Primary reason: the difficulty of maintaining ofthe existing groups. all Primary reason: the difficulty a list of maintaining Alternative explanation: July 1986 ~ MarchJuly 1986 ~ 1987 Before this • • • • Great Renaming newsgroups (2) newsgroups
Computer Center, CS, NCTU 12 rec.* for discussions of recreational activities sci.* for science related discussions for soc.*socializing discussionsand of social issues talk.* for discussions about various controversial topics misc.* for miscellaneous discussions humanities.* for discussions of the humanities comp.* for comp.*computer related discussions for discussionsannouncementsnews.*about andnetnews Big 8 in 1995 Big 8 in Initiated Rick Adams by Initiated organized Gene Spafford Cabal” the “Backbone 1987 Big 7 in • • • • Great Renaming newsgroups (3) newsgroups
Computer Center, CS, NCTU 13 rec.food.recipies 8 management 8 management to create board will vote on whether the group. - Drugs, and sexnot allowed. were Dissatisfied with mod.gourmand in MayCreated 1987. Post an RFD (Request for into news.announce.newgroups Discussion) (Request Post an RFD news.groups.proposalsDiscuss in is formalized The proposal charter. a name, description, with Big • • • • • • • Brain Reid suggested the alt.* hierarchy as an alternative to talk.* to alternative as an hierarchy alt.*the Reid suggestedBrain There was a great controversy over what newsgroups should be allowed. should newsgroups over what controversy There was a great After the Great Renaming, the creation and organization of new andof organization creationthe Great Renaming,After the process. voting an automatednewsgroupsbecame newsgroups (4) newsgroups
Computer Center, CS, NCTU 14 7. - jokingly and referred to ALT jokingly Lunatics standing for “Anarchists, Terrorists” fearing newsalt.sex.*: the major distributed. may be prevented from widely often of extremely alt.binaries.*: large size. It is ‘alternative’ Big mainstream It is ‘alternative’ to the alt.gourmand, newsgroupswere The first alt.* and alt.drugs alt.config, a public procedure involving follow an informal Most newsgroups before created. in alt.config discussion subject or type. of any specific Not confined to newsgroups Because of the up, some groups sprung anarchistic the nature with which • • • • • • • Two major sections of alt.*of sectionsTwo major alt.* hierarchy alt.* newsgroups (5) newsgroups
Computer Center, CS, NCTU 15 8 hierarchies, 8 - de.*, fr.*, … de.*, Messages posted in there “private” there “private” Messages postednewsgroups in are generally not passed to the Internet the other providers or in general. Other language Other language later been have hierarchies existing created to the in parallel ones. English tw.*, tw.bbs.* (innbbsd) • • • Some users prefer to use “Usenet” to refer only to the Big only to refer to use “Usenet” prefer Some users medium. entire “netnews” the use to incorporate and to Some access providers also create their own versions, prefixing own the versions,their create providers also Some access way. a similar own name inwith their newsgroupsnames The language of preference in the “original” Usenet hierarchies, in the “original” The languageof preference English. is including alt.*, newsgroups (6) newsgroups
Computer Center, CS, NCTU 16 Printable Printable or yEnc. MIME, - A moderated newsgroup has one or more individuals as moderators. individuals more one or has newsgroup A moderated readers by to Usenet, but are emailed Messages submitted are not distributed moderators for approval, instead. to the articles, Moderators receive review them, and inject submitted approved must header “Approved:” bear the articles. line. Such articles Binaries alt.binaries.* postedBinaries are in encoded in distribute text content to ASCII. originally Usenet was created by uuencode, Base64,Encoded binaries Quoted and PAR.by means of RAR might be split into pieces The binaries • • • • • • • Moderated newsgroupsModerated Binary newsgroups newsgroups (7) newsgroups
Computer Center, CS, NCTU 17 reply article reply is called a thread. - Articles can be traced to onewhich single non exchanges articles exchanges articles with them. The overview is a list of the most frequently of the most frequently The overview is a list used headers, and additional information such as article sizes. available user’s news only article, on that When a user posts an it is initially server. Each news serverto one or more other servers(peers) talks and Binary content: split among multiple articles, reassembled into a single unit reassembled single unit articles, among multiple into a split Binary content: reader by the news are responsesof the articles majority other article. some to Plain text: • • • • newsfeeds Headers overviews and Articles and posts Articles News operations News (1)
Computer Center, CS, NCTU 18 news/tin) ends have become more ends have become more 、 - based (chinese/tin news reader: tin - However, these integratedHowever, clients arequality: of low incorrectly implementing protocols,Usenet standardsconventions. and A user subscribes to a newsgroup, the news reader software keeps track of track keeps software news reader the a newsgroup, to A user subscribes user has read. which articles that the rise of the World Wide Web, With front web common. text Common Read and post articles Read and post reader server and news implementations: news In early news were a single program suite. or an integrated have news commonly “communication suites” Mail clients reader. • • • • • • News Reader News operations News (2)
Computer Center, CS, NCTU 19 carry load the heaviest CPU Transit server: the handlebasic articles exchange. server:Numbering insert the “Xref:” headerline into each article server:end Back data storage server:Front end actually a user wouldstore connectto. Only andoverviews, Split into Split Continually storageincreasing Continually Network capacity requirements of a the ability to successfully receive server Completion: traffic all available articles the amount of time are made Retention: to readers Overall system performance • • • • • • News Server News operations News (3)
Computer Center, CS, NCTU 20 suited to analyze files the - Since 1990s Since Articles seriallyare appended to large indexedcontainer files the of Whenendthe file is articles reached,new are written at the beginningof the file, overwriting the oldest entries. Traditional text manipulation tools are less well Oldest storage scheme Performance relates to OS’s ability to create, removelocate files and within a dictionary – – – – – – One file Oneper article Cyclical storage Flexibility and space used. to retention Flexibility A disk storage for the body of an article for the body of an A disk storage Types: • • • Spools News operations News (4)
Computer Center, CS, NCTU 21 The maximumThe article size varies from site to site long server the keepsarticles How binaryandText How quickly a serverHow quickly can deliver an article to the user Article sizesArticle Retention • • • Speed Storage management Storage News operations News (5)
Computer Center, CS, NCTU 22 ID - Delete a article specifiedMessage by In casesunsuitable whereUsenet contentposted,hasfor has support been automated removalof a postingcreating from bynetwork a the whole cancel message. Create a new newsgroupCreate anew a newsgroupRemove Lists all groupsof a hierarchy cancel cancel messages newgroup messages rmgroup messages checkgroup messages • • • • Control messages (RFC5537) Control News operations News (6)
Computer Center, CS, NCTU 23 MMF 、 Challenge - F/s/l(From/subject/lines) 、 Cancel 、 ID) - 20% useful information20% (mime) html post host/lines) - 、 、 Ph/l(Posting 、 40% cancel40% messages 、 Key = hash(password, Messagehash(password,= Key - Specific strings as binarysuch post see ‘em No Cancel All notices must besignedaccepted bepublic key PGP to using byinnbbsdpatch leeym 40% spams40% There is no authentication in cancel messages Written by perl md5(body)by Count NoCeM Cancel message Cancel Cleanfeed • • • Spam filter News operations News (7)
Computer Center, CS, NCTU 24 nonsense – Group.NCTU.edu.tw tw.bbs.campus.nctu ↔ nctu.talk tw.bbs.campus.nctu tw.bbs.comp.linux ↔ ptt tw.bbs.comp.linux NCTU_Talk Take a break a Take
Computer Center, CS, NCTU 25 NNTP
RFC 3977 Interact with INN news server standardextensions) - Reserved for Reservedprivate use (non Connection, setup, andmiscellaneous messagessetup, Connection, selection Newsgroup Article selection Distribution functions Posting for Reservedauthentication privacy andextensions Informative message completedCommand OK so far; the OK sendrest Command of it syntacticallywas Command correct failedbut for some reason unavailable,unsupported, or syntaxerrorunknown, Command ------x9x x0x x1x x2x x3x x4x x8x 1xx 2xx 3xx 4xx 5xx Response codes • Command and responseandCommand NNTP (1) NNTP
Computer Center, CS, NCTU 27 ). posting ok posting INN INN 2.4.6 ready ( INN INN 2.4.6 ready NNRP NNRP server server telnet csnews2 433 telnet InterNetNews $ telnet csnews2 119csnews2 telnet InterNetNews telnet csnews2 433csnews2 telnet For your readers For your peers liuyh@bsd5$ Trying 140.113.235.121... csnews2. Connected to Escape character'^]'. is 200 csnews2 502 deniedPermission liuyh@nsd5$ Trying 140.113.235.121... csnews2. Connected to Escape character'^]'. is • • Trying 140.113.235.121... csnews2.cs.nctu.edu.tw. Connected to Escape character'^]'. is 200 csnews2 liuyh@csnews NNTP NNTP vs. NNRP NNTP (2) NNTP
Computer Center, CS, NCTU 28 班 A 物件導向程式設計 ] ". ". 課程 [ 0000000284 0000000108 y . ] (Moderated) ] 計算機網路管理 ] programming.a programming.a group description group flagshigh low - - 系計中公告 課程 [ 0000000505 0000000131 y oriented oriented - - center [ center center 0000000509 0000000071 m - - AVTIVE NEWSGROUPS cs.computer cs.course.netadmin cs.course.object list newsgroups cs.* list 215 form Descriptions in " 215 inNewsgroups form " cs.computer cs.course.netadmin cs.course.object list active cs.* active list • • LIST (find the newsgroups) LIST (find the NNTP (3) NNTP
Computer Center, CS, NCTU 29 ID: - OVERVIEW.FMT . Message References: Bytes: Lines: Xref:full 215 Order of fields in overview in overview database. 215 fields Order of Subject: From: Date: list overview.fmt list • LIST format) (overview NNTP (4) NNTP
Computer Center, CS, NCTU 30 cs.course.netadmin cs.course.netadmin cs.course.netadmin cs.course.netadmin 359 131 505 359 131 505 … . 141 504 505 211 131 132 134 138 listgroup group 211 LISTGROUP a newsgroup) (select GROUP newsgroup) a (select NNTP (5) NNTP
Computer Center, CS, NCTU 31 \ \ [email protected]> status [email protected]> retrieved; Article [email protected]> Article retrieved; - - - 131
Computer Center, CS, NCTU 32 \ \ ID: - Subject: From: Date: Message References: Bytes: Lines: Xref:full ) art )
Computer Center, CS, NCTU 33 mail - for - \ Tung University Chiao ") … Scotasay 20070201 (" release - [email protected]> - Host: bsd5.cs.nctu.edu.tw Date: Thu, 8 Apr 2010 10:49:51 +0000 (UTC) To: [email protected] - - cs.course.netadmin - : College of Computer of NationalScience, : College 上課講解手 Computer Center, CS, NCTU 34 mail - for - Tung University Chiao release cs.course.netadmin 上課講解手 Hsiang Liu Computer Center, CS, NCTU 35 Computer Center, CS, NCTU 36 ihave cs.test Distribution functions - : Computer Center, CS, NCTU 37 25 1 y 0 1 y 100222 000000 47 1 y 9 1 y 2 2 y 100401 000000 cs.course.netadmin [email protected]> [email protected]> [email protected]> [email protected]> New news follows New newsgroups follow. - - - - … NEWNEWS NEWNEWS newsgroups [YY]YYMMDD hhmmss NEWGROUPS [YY]YYMMDD hhmmss Computer Center, CS, NCTU 38 tin A Usenet news reader 8 - 8 - ASCII or UTF - ASCII [ANSI1986] to UTF ASCII [ANSI1986] - 8 SHOULD NOT be used 8 - ASCII) - bit encodings other UTF bit encodings other than - The character set of article bodies SHOULD be indicated in the article in the article be indicated SHOULD bodies article set of character The headers be in US The newsgroupSHOULD description 8 But… hope? IMHO,what is the news/tin (default: US news/tin (default: (default: Big5) chinese/tin This specification from US extends NNTP [RFC3629] NOT use any other Implementations encoding. MUST • • • • • • • • • In RFC In RFC 3977 Two versions in ports Two versions in Install (1) Computer Center, CS, NCTU 40 r’ - NNTPSERVER=csnews2.cs.nctu.edu.tw ‘make install’ is ok ‘make install’ TIN_EDITOR VISUAL EDITOR TINRC=‘ • • • • • • Environment Variables Environment Back to install tin install Back to Install (2) Computer Center, CS, NCTU 41 Type) headerpostings. in - Big5 = 8 environment - used for MIME (Contentfor MIME used Screen:utf8 encodingbig5 tin (This is for post) LC_ALL=zh_TW.Big5 env If you use UTF Generate ~/.tin/ and ~/.newsrc Generate ~/.tin/ Charset # mm_network_charset • • Second execution Edit ~/.tin/tinrc First execution Connect Start Computer Center, CS, NCTU 42 u: unsubscribe from chosen group chosen from u: unsubscribe y (yank): list all newsgroupsall y (yank): list name for a group forwards/backwardssearch /?: chosengroup subscribe to s: Show some default subscribed newsgroupsShow default some Search and subscribeand Search newsgroups Computer Center, CS, NCTU 43 mode (including all headers) all mode (including - r: reply through mail author r: reply through mail to ^H: in raw show article w: post an article group to current an article w: post search forwards/backwards/?: within this article article f: post followup to current D: cancel or supersede (delete) current (overwrite) article h: help forwards/backwards by Subject line search for/?: articles for articles aA: search forwards/backwards by author display of all/unread articles r: toggle article next unread Computer Center, CS, NCTU 44 - - 669,687 - - - 1758,1766 - 664,668 - 914,923,930 - 1145,1164,1186 - 658,661 - 4441 - 4466 675699 1 - - - ! 1 1 ! ! 646 - 505 644,648,652,656 - - 1 newsrc 1 1 ! : : 230198 - 1 ! cat ~/. 546806,546808,546823,546885,546908 $ - 1 ! newsgroups you subscribed/unsubscribedyou newsgroups articles readhaveyou ~/.newsrc ~/.newsrc will keep • 1187,1190,1192,1252,1323,1337,1433,1444,1451,1475,1489,1512,152 0,1523,1536,1559,1572,1614,1619,1647,1687,1757 1768,1772,1781,1795,1818 1819,1906,1918,1988,1997,2012,2028,2057,2059,2142 cs.course.netadmin cs.course.sysadmin 688,694,698,703,707,799,831,868,903,913 932,968,983,1023,1054,1115,1131,1142 news.newusers.questions misc.test misc.test.moderated news.announce.newgroups news.answers liuyh@NASA news.announce.newusers After you quit tin or save configuration or save quitAfter you tin newsrc Computer Center, CS, NCTU 45 - mail - for - center:486 - example cs.talk Computer Center, CS, NCTU 46