ISSUE 2 SUMMER 1998 K This publication may be freely reproduced in whole or in part B ■■■■■■■■■■■■■■■■■ or the typical modern ISP,the It was a nightmare made even worse by the memory since the PPro motherboards could clock probably started running a fact that so many resources were accommodate up to 512MB of RAM where we bit over two years ago. Sure, there concentrated in the minis that a crash, expected to need only 192MB in the absolute are plenty of older ISPs, but few of software problem, internet attack, or user worst case. Most shell boxes run with 128MB them have had to contend with mistake could effectively take down our just fine, supporting up to 2000 user F100% per month growth and many of the entire service. accounts on each. larger ones also started more as BBS systems then ISPs. If you were like us, however, you This situation convinced the entire What we worried about was that the started right as the Internet really started to BEST Engineering department maintenance required to support a become popular and that meant having to that minis, the so- larger number of smaller boxes subsequently deal with the strain of very called would increase. In fact, the time rapid growth. ss required to maintain 24 FreeBSD boxes has occupied less than half For us, two years ago was indeed 11 fforor ISPISP the time we spent dealing with the the beginning.We had made ## two minis and plethora of smaller an educated guess as to SDSD vendor boxes, something which was our best market, hit it BEST Internet Communications, Inc. o CONTINUED... running, and turned rreeBeeB out to be right.With FF that out of the way, the by Mathew Dillon “enterprise name of the game for servers’’, are a huge BEST Internet was to simply mistake.To make a mini cost accomplish the engineering and effective would require decking each programming feats required to support one out with half a million dollars In This Issue: a rapidly exploding user base. worth of hardware, and then actually to use it would require putting a lot of FreeBSD No.1 for ISPs In the very beginning, we used BSDI and marbles in one bag, so to speak. by Mathew Dillon, BEST Internet Communications FreeBSD and, if the times had been less trying, we wouldn’t have made the almost In moving back to running FreeBSD on What is FreeBSD? fatal mistake of shifting to larger multi- rack-mounted Pentium Pro 200’s, there by Jordan K. Hubbard, Editor processor minis from a brand-name UNIX were some issues that concerned us vendor who shall remain nameless.We and some that did not.We knew that Coda on FreeBSD somehow convinced ourselves that moving the reliability was there, especially by M. Satyanarayanan, Carnegie Mellon University to larger commercially-supported boxes with the PPro 200s and their would both help sell our service and provide integrated ECC and secondary Another FreeBSD Success Story: adequate margins for growth. caches.We had also learned that the In ISP Revolutionized governing limitation to all ISP support In ISP Revolutionized by Dan Benjamin, Network Analyst That mistake almost drove us out of business. boxes is “disk I/O”,“disk I/O”, and The scalability never materialized and the “disk I/O.”(Short of putting ten SCSI work required to maintain the minis went interfaces in each of the two mini Getting Kids on the Web with TCJA Technisch Creatief Jeugdcentrum Arnhem from difficult to near impossible. Every computers, the ten FreeBSD boxes that by Wilko Bulte, hardware addition required a ridiculous replaced them would win hands amount of work and money to install.We down).We also knew that CPU power FreeBSD and the Palm Pilot couldn’t get bug fixes for weeks or months. would not be a factor and, indeed, it by Oliver Fromme There were all sorts of odd licensing fees that hasn’t been – the new machines made it impossible to ““genericize”” machine usually hit physical disk I/O limits long How FreeBSD Became installations. I will spare you the rest of the before the CPU saturates. Finally, we thethe FavoriteFavorite OSOS onon mymy DesktopDesktop gory details – it wasn’t a pretty scene. knew that we could balance out and by Martin Cracauer, BIK – Aschpurwis + Behrens tune “CPU vs.disk I/O” by adding PAGE 2 okXR13 ISSUE 2 • SUMMER 1998 ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ▼▼ ▲▲ ■ made possible by the fact that every single FreeBSD box is configured and updated If you’re looking for a stable, easy-to-install operating system with a long and proven ◗ from a template machine.That’s also something we just couldn’t do with the vendor ◗ boxes due to the massive customization each one required and odd licenses which track record, FreeBSD is the OS for you! ■ required us to be frugal about installations. Who is the FreeBSD Project? ◗ FreeBSD has also done an excellent job with service-specific boxes such as our ◗ ■ “newsfeeds box,” our “newsreader boxes,” our “sendmail back-end boxes,” and other The FreeBSD Project is a group of over 100 various administra-tive-duty machines. Since the single largest cost for an ISP-capable developers led by a 17 member “core team” rack-mount PC is usually the disk drives, most of the non-news boxes are incredibly which sets overall direction and ensures a ■ ◗ cheap because they require no real disk storage.A shell machine will have a 4G and ◗ consistent project focus. Most of the two 9G disks to start out with, whereas a proxy box generally requires only a single 4G developers come from commercial software disk.The ability to flexibly tailor PC hardware configurations to specific tasks can lead development and BSD UNIX backgrounds, ■ to some incredible cost savings without impacting maintainability. providing a unique perspective on what can ◗ ◗ and should be done with this technology in Ultimately, the KISS principle applies to administration. No (major) NFS mounts, no the marketplace. Our long history of ■ attempt to allow users to log into just any shell machine – we assign them one and that software development experience has also is where their account resides. No attempt to slave news is made since each newsreader paid dividends in the way that the project is ◗ box is self contained – users pick one and don’t switch.We were worried that the load ◗ organized, with multi-track development ■ would not balance well. In fact, it balances extremely well. Sure, some shell machines (stable and experimental), strong use of are loaded a bit more than others, but if it isn’t noticeable it isn’t a problem. It is, source code control (CVS) and a highly perhaps, a somewhat different approach to the concept of a “distributed system’’, but ■ centralized development model (not just the ◗ it works, nonetheless, and works very well. ◗ kernel, but the entire system) being just some of the results of our experience. Where we used to get a cascade of complaints when some part of the system would ■ wedge, now we get only a few.Where we used to have a difficult time working out the Where do I get more information? ◗ logistics of when to purchase a new machine or when to add a disk, now it becomes ◗ trivial because the user base on each machine builds, matures, and that’s it – we go onto ■ The project runs a web server at the next machine.We stay one or two boxes ahead of the growth curve and everything http://www. freebsd.org which contains runs smooth as glass. a lot of useful information (also mirrored in ◗ ◗ ■ over 25 countries—please see I think it would be fair to say that FreeBSD not only saved us from past mistakes, but http://www.<countrycode>.freebsd.org now empowers us to continue building our business at whatever rate we choose. 4 for the nearest local server) and you’re highly encouraged to visit it. ◗ ◗ Hi, I’m Dot, one of To reach the FreeBSD development team, Jordan’s 15 cats. please send email to [email protected]. ◗ ◗ If you have a matter which requires the WhatWhat isis specific attentions of the core team (a commercial request or some other proposed cooperative venture), please ◗ FFrreeBeeBSD?SD? ◗ ◗ ◗ ◗ send email to [email protected]. by Jordan K. Hubbard Editor So, in short... ◗ ◗ reeBSD is a UNIX®-compatible operating barrier to some, we’ve made sure that the F system based on 4.4 BSD Lite2 from the system is always available free of charge on the The marketing hype may tell you that NT University of California, Berkeley and its Internet (ftp://ftp.freebsd.org/pub/FreeBSD) can do everything a well-configured UNIX contributors. BSD has been in more or less and we welcome both free and commercial machine can do, but we know differently ◗ ◗ constant development for over 15 years.The redistribution of the technology. from practical experience and if we didn’t same technologies used in FreeBSD have also feel this was the best possible technology for been the basis for many commercial operating FreeBSD can be used for everything from the needs of Internet Service Providers and other commercial/private interests, we’d be ◗ ◗ systems, from Digital’s Ultrix to Sun’s SunOS, creating web servers, providing enterprise and still remains a staple of commercial OS mail services, file and printer sharing (for using something else ourselves! As it is, development today. other UNIX systems as well as the Windows thousands of commercial users, including and Macintosh platforms) and serious such notables as Yahoo! and Walnut Creek ◗ ◗ Like its commercial counterparts, FreeBSD software development.