Mirrors Primary (US) Issues April 2002
April 2002 Get BSD Contact Us Search BSD FAQ New to BSD? DN Print Magazine BSD News BSD Mall BSD Support Source Wars Join Us
T H I S M O N T H ' S F E A T U R E S From the Editor Review: FreeBSD Services Limited's 4.5-RELEASE DVD Editorial by Sam Smith by Chris Coleman Building a self-funded If this was just a review of FreeBSD 4.5, and I just company is difficult in any mentioned that it comes on DVD, then the review would economy, and more have missed the point. There is a significant difference, especially so in the current when it comes to using the discs, between the CD and DVD one. Its a backhanded distributions... Read More blessing, however. One the one hand, you are protected from the "dot com" syndrome, where you burn A Tour Through The NetBSD Source Tree: Part II - though money faster than Libraries investors can throw it at by Hubert Feyrer you. But on the other hand, you can't get money if you In Unix(-like operating systems), commonly used routines wanted to, investors, loans that can be accessed from many programs are grouped or otherwise. within so-called libraries and that can be used from application programs. The src/lib directory contains the Get BSD Stuff libraries that come with NetBSD, and there's quite a number of them. Let's have a look! Read More
DOSSIER and the Meta Project (Part 3) by Rich Morin
Meta is an exploration into integrating system metadata and documentation. It can be described as a distributed mechanism to facilitate access to information on Free and Search Open Source software. Meta servers provide access to documentation, metadata, rules, and even source code, by Monthly Ezine extracting information from a variety of system file Search formats, making it available in a consistent, easily accessible manner. Read More Daily Daemon News R E G U L A R C O L U M N S KDE 3.0 out for FreeBSD FreeBSD/sparc64 boots SMP Trawling the Ports Collection Need A Light? by Greg Lehey Geek Dating "It has Excel and Word" Our newest column is a slow saunter through the ports tree. NetBSD on Microsoft Only when moving slowly can the wild beauty of X-Box individual ports truly be appreciated. Herein we present you Microsoft/Unisys with information gathered while walking through and Anti-Unix Site Runs Unix playing with the ports tree. This month: bash, bash prompts, Review of The Complete and xtset. Read More FreeBSD
BSD Support Forum The Answer Man by Gary Kline, Dirk Myers, and David Leonard Problem with Daemonnews FreeBSD 4.5 Set This month we bring you a column on tuning your BSD Can I use 4.4 Release CD system to best suit your needs and wants. Obviously, to install package on 4.5 because everyone has different requirements, there may Stable? well be myriad different ways to tweak your system. If you Star Office installation run more than one machine you may wish to tune each from CD differently. Read More CVSup Finished Successfully? Digital Cameras ? Where in rc.i386? The Daemon's Advocate Cron problem by Greg Lehey Can't get PCMCIA card to work How have project management practices influenced BSD in the past? How will they be needed in the future, as the BSD Source Wars projects grow larger and more internally diverse? Here are some history, predictions, and questions about what is to Week 22 come. Read More
BSDMall FreeBSD 4.5 Now Shipping FreeBSD Training in New York City! Internet and Network Security Training in St. Louis Sangoma T1 controller with Integrated CSU/DSU Daemon Xing Clothes etc. FreeBSD in a Box Just $79! Darwin -The Core of the MAC OS X - $30 Borderware Mail Gateway $960 Need Reseller Pricing - Go to Cylogistics!
Miscellaneous Credits The hard-working crew Tarball Download a tar.gz version of this issue PDF Download a PDF version of this issue
Copyright © 1998-2002 DæmonNews. All Rights Reserved. Building a BSD Company
Chris Coleman
Building a self-funded company is difficult in any economy, and especially so in the current one. Its a backhanded blessing, however. One the one hand, you are protected from the "dot com" syndrome, where you burn though money faster than investors can throw it at you. But on the other hand, you can't get money if you want to, investors, loans or otherwise.
A few months back, Wes Peters and I attended a "Money Connections" seminar with the intention of looking for a bit of seed capital to expand the BSDmall side of Daemon News. (Most of Daemon News is volunteer, and the rest is subsidized by BSDmall, especially the print magazine.) However, the group of local investors basically told us that we didn't want investor money and they wouldn't give it to us if we asked. Of course they mixed that in with a few good business tips like "buy low, sell high" to make us think we were still getting our moneys worth.
From that experience, we learned that we didn't want just anyone's money. If we did accept money, it would have to come from the BSD community, from people with a vested interest in seeing a BSD company thrive so it could promote BSD. (Nice little circle. ;-) We also learned that there was more we could do to buckle down and grow at the rate the BSD community was expanding.
We are partnered with Cylogistics, a BSD distributor that helps resellers get access to BSD merchandise. They helped fund us during the times when we were running things on a shoe string budget, so much so, we were actually sharing an office with them. Soon however, the Cylogistics staff began working more hours for BSDmall than for Cylogistics. This was mostly because we were growing, but also because the office wasn't optimized for the kind of business we were doing. Retail has lots of small transactions and wholesale does a few large ones; we weren't prepared for the kind of volume it takes to make any money, so we weren't making much.
It soon became clear that we had outgrown that situation. Cylogistics was growing and needed its staff back to continue promoting BSD in the reseller channel and we needed more staff than we were borrowing from them. The worst part is that our customer service started to slide. We had always had a great customer service policy, and any amount of customer neglect was unacceptable. Yet, suddenly, it seemed we didn't have enough manpower to live up to our expectations. Orders that were easily filled, went out quickly and the customers were very happy, but anything out of the ordinary and the customers occasionally got lost in the system. We needed a new system. As a customer I hate having my order lost, and worse not being able to get ahold of someone to fix the problem.
We quickly realized things needed to change and started taking corrective action. We needed to grow, but that first step is always the hardest and takes lots of planning. We also had to admit that open source couldn't do everything, at least not within our other restraints. We switched the BSDmall shopping cart system from an open source system to a Yahoo! Store. Even though the Y! store was more expensive, we didn't have the time or resources to continue developing on the open source project. However, the open source shopping cart worked fine to start with and we wouldn't have an online store now if it wasn't for it. It just didn't scale fast enough to meet our needs.
The next thing we did was route all of our customer e-mail into a trouble-ticket management system so customers who emailed us wouldn't get lost or ignored.
The new store went online in November and the trouble-ticket system online in December. However, this didn't solve all our problems. I was still trying to remotely manage the understaffed office we shared with Cylogistics in California while holding down a day job here in Utah. I also happened to be handling most of the customer support e-mail, without being able to see what was going on in the shipping office. Something had to change, which was the reason Wes and I were at the money seminar a few months back.
Having learned that we didn't want their money and knowing that we couldn't get it if we wanted it, it was very difficult to figure out how to resolve things. With much help from my wife, we decided to move the Daemon News/BSDmall operations here to Utah and I would work full time in the office. Betting your livelihood on a small company can be quite unnerving!
The biggest benefit of the move has been to our customers. I was able to re-arrange some of our operating costs and hire a customer service rep. Now, if you have a problem, it will get handled and in a reasonable amount of time. Our phones are staffed, so you can always reach someone during our business hours. Our product shipping is a much more efficient process as well. We have always had a policy to stand behind what we sell, but we now have the manpower to make that possible. Cylogistics is also thriving. They have their manpower back and are putting it to good use promoting our products in the BSD channel. BSD is growing and so are we.
Our goal is to promote BSD. With our partnership with Cylogistics, we are making it easy for BSD vendors and manufacturers to get their products to the BSD market. We have worked very hard to get to this point, and feel we have made a positive impact on the commercial growth of BSD. If you have a BSD based product, project, or idea, give me a call. If we can't help you, we probably know someone who will.
Help us promote BSD. If you have time, volunteer. If you know stuff, write! If you are on the pulse of BSD news, share it. If you have money, we accept Visa, MasterCard, American Express, Discover, and Diners Club and will send you cool BSD stuff. If you have lots of money, well, we have a toll-free number you can call. :-) April 2002 Search Submit Article Contact Us Join Us Merchandise
Review: FreeBSD Services Limited's 4.5-RELEASE DVD
Sam Smith
If this was just a review of FreeBSD 4.5, and I just mentioned that it comes on DVD, then the review would have missed the point. There is a significant difference, when it comes to using the discs, between the CD and DVD distributions.
Since 4.5 is a release from the stable branch of FreeBSD, there's nothing radically different from 4.4. There's support for IDE drives bigger than 137Gb, various improvements in different filesystems, the linux emulation is better, and IPF now supports both IPv6 and storing NAT state across a reboot. It's better, more functional, but nothing big and new has been added.
However, while the install kernel on the DVD has more devices included than the floppy install kernel, FreeBSD itself is identical. The installer is the same, and once installed it's the same quality people have come to expect from previous FreeBSD releases, there's just more of it available while you're installing from the DVD. But that said, while the DVD is good at what it does (giving you access to lots of data if you want it), it generally stays out of your way and lets you get on with the install, with the media being largely irrelevant.
The package itself contains 2 discs, a single sided dual layered (7.6Gb in size) DVD, and a double sided disc with a 4.3Gb DVD image on one side and a standard CD image on the other. The first DVD image is a superset of the CD image, with the additions mainly being additional packages. This DVD is everything you need to install (including all 5,683 packages) and is also a emergency "fixit" disc. The shear number of packages included on the DVD you boot from allows almost anything to be installed very quickly. The second DVD image contains a complete ports tree, including all distributable distfiles. This is intended to be mounted as /usr/ports; and makes the installation of ports a lot faster once you have installed any packages you want. You don't have to swap DVD's when installing packages, they're either on that DVD, or they're not available. Similarly with the ports tree, once mounted, everything that could be there, is.
Even more BSD
On 14th January 2002, Caldera, who own the copyright to the original AT&T UNIX source code, released it under a BSD-style license. These old versions, are once again unencumbered and are freely available both from The Unix Heritage Society Archive, and are included on the first DVD. While their practical utility may be limited, they are of interest to all those who have interest in BSD and BSD history. Additionally, on the DVD, but not widely available is the FreeBSD repository and code from version 1.1.5 of FreeBSD -- the final release before FreeBSD moved over to the 4.4BSD codebase, on which all of Free, Open and Net BSDs are based.
As bonus extras, the DVD contains the complete FreeBSD, OpenBSD and NetBSD CVS repositories detailing all changes since the repositories were set up, along with archives of FreeBSD mailing lists going back to 1993, which is a useful resource to have access to. The CVS repositories can be unpacked and updated using the usual methods of anonymous cvs and cvsup to get a fully populated and up to date repository. Making the set not only extremely comprehensive for FreeBSD, but containing significant interest for those of other BSDs as well.
Overall, while it is unlikely that the majority of people will use all of the content of the disc, there is a lot of useful content there for everyone, not just FreeBSD users, to get something out of the set. The CVS and list archives are useful to keep on the shelf until the next time they’re needed.
Resources
FreeBSD Services Limited
http://www.freebsd-services.com
FreeBSD
http://www.FreeBSD.org
The Unix Heritage Society (for versions 3 and 4 of BSD)
http://www.tuhs.org
Author maintains all copyrights on this article. Images and layout Copyright © 1998-2002 Dæmon News. All Rights Reserved. April 2002 Search Submit Article Contact Us Join Us Merchandise
A Tour through the NetBSD Source Tree: Part II - Libraries
Hubert Feyrer
In Unix(-like operating systems), commonly used routines that can be accessed from many programs are grouped within so-called libraries and that can be used from application programs. The src/lib directory contains the libraries that come with NetBSD, and there's quite a number of them. Let's have a look! csu: Sources for crt0.o and it's C++ cousin c++rt0.o. These are not libraries, but equally important nevertheless, as they do the startup of every program - pull in shared libs, for C++ call any constructors, then call main(). When main() returns, several cleanup functions are called that call e.g. C++ destructors. After that, the return code from main() is returned to the calling program and the program terminates. Sources in this directory are partly coded in machine language for efficiency, and there are special functions for each architecture and object file format (a.out, ELF). libarch: This library contains some architecture-specific functions that might be useful to access from userland programs. For i386, functions exist to manipulate the per-process I/O permission bitmap, I/O privilege levels and set virtual 8086 processor registers and mode. Other archs have functions for handling CPU caches from userland here. libbz2: The bz2 algorithm compresses files using the Burrows-Wheeler block-sorting text compression algorithm, and Huffman coding. This library is used in several programs to allow reading and writing of bz2 compressed archives and files. libc: This is probably the most important library in NetBSD, as it contains the "trampoline" stub code to do kernel calls from C programs. The stubs are tiny pieces of C/assembler code that take any arguments for system calls (check your section 2 of the NetBSD manual pages, starting with intro(2)!), then bring them into shape to do a system call using whatever way the underlying hardware uses to do so, usally a trap or software interupt. The system call is executed in kernel context, and any return value is taken by the stub and fed back too the calling routine. Besides the stub code for system calls, libc also contains code for a lot of convenience functions that can be called from programs without linking in any special library, as libc is always pulled in automatically. These convenience functions include code for
database access using the Berkeley DB code support functions for shared libraries, which do load shared objects (dlopen(3), ...) assisting profiling various hash routines (sha1(3), md5(3), ...) locale and NLS handling processing network related information, like the name service switch routines and the DNS resolver code regular expression handling RPC handling including service creating/accessing for both clients and servers and XDR handling routines for 64bit type handling, for 32bit CPUs software floating point operation routines for machines that don't have a floating point unit (FPU) the "stdio" set of functions that handles buffered input/output and many other things. the "stlib" functions string manipulation functions the terminal I/O subsystem that allows hardware independent programming of terminal driven software, by using a translation table for terminal functions, termcap. timezone-handling functions network information service (NIS) handling
Also important, this directory contains all the manpages for the system and library calls (manual pages section 2 and 3). libasn1, libcom_err, libhdb, libkadm, libkadm5clnt, libkadm5srv, libkafs, libkdb, libkrb, libkrb5, libkstream, libroken, libsl, libss, libvers: These directories contain various auxilliary libraries for both Heimdal and MIT Kerberos. Unfortunately, there is little documentation about the functions and interfaces they provide, and they are currently used by Kerberos only. libcompat: The routines collected here are for backward compatibility with old software interfaces, and must be linked in explicitly. Included here are the regexp(3) routines which were obsoleted by regex(3), and various functions for terminal handling, queue management and remote program execution. libcrypt: This library contains (only) the DES routine used for password encryption. It was put into it's own library so it can be omited easily from systems that are exported to countries which fall under the US export restrictions. Fortunately these are much less strict nowadays than when libcrypt was invented. libcrypto: This library contains various routines for data encryption, decryption, hashing and signing. The functions are part of the OpenSSL distribution, and the source is accessed using the reachover mechanism, the actual sources are in src/crypto. libcrypto_idea, libcrypto_rc5: While export restriction and movingg restricted functions into their own libraries, taking care of existing patents is another issue. The IDEA and RSA RC5 algorithms that come with OpenSSL are patented, and may not be used under certain conditions. To allow disabling them easily, they were moved into their own libraries again. Using shared library techniques, libcrypto will pull them in if available, and fail with an abort(3) call if they cannot be found (when called). libcurses: This directory contains sources to the curses(3) terminal handling library, which can be used to do simple, terminal/screen-based applications in a hardware/terminal independent way. Note that NetBSD does not use the ncurses implementation, but one that is derived from 4.4BSD and that was extended to follow the latest standards. libdes: The libdes that existed on 4.4BSD systems and that provided routines for the Data Encryption Standard for encrypting and decrypting data are contained in the OpenSSL-based libcrypto these days, and libdes is just a dummy directory that keeps information about the libdes shared library. The library itself is a link to libcrypto and is generated when libcrypto is built/installed. libedit: This library contains a set of functions for command line editing and history processing. It originated in 4.4BSD and was extended to provide source-level compatibility with GNU's readline library by the NetBSD project. libgssapi: This library implements the Generic Security Services, which provides security services to callers in a generic, source-level compatible way, and which usually sits above the cryptographic libraries. See RFCs 1508 and 1509 for more information, there's (unfortunately) not much documentation contained in this library, which is mostly used by Kerberos. libipsec: These routines convert IPsec policy specification structures from and to (human-readable) string, and allow accessing the pfkey API. libkvm: The kvm(3) library provides a uniform interface for accessing kernel virtual memory images, including live systems and crashdumps. Available functions include retrieving the current load average, a list of open files as well as routines to access arbitrary symbols for both reading and writing. This library is used by many programs to retrieve status information from various kernel subsystems. libl: The (f)lex lexical analyzer has some common routines that are stored in this library. The sources are taken directly from src/usr.bin/lex by reachover Makefiles. The resulting library is installed as both libl and libfl, as the "lex" that's shipped with NetBSD is really the "flex" implementation. libm: The math library contains many functions for single and double precision floating point arithmetics, trigonometric functions and many more. This library must be linked in explicitly when using simple math functions like sin(3) and cos(3).
Many of the functions in this library exist in specially optimized versions written in machine language for various CPU/FPU architectures, for maximum performance. There is also the distinction between machines using IEEE floating point format internally and these that use a different format, which is reflected in the source. libmenu: The menu(3) provides a terminal independent menu system using the curses(3) library. libossaudio: This library provides compatibility to the Open Sound System API. It can be linked against sources that (usually) originate on Linux, and the OSS sound calls will be mapped to use the native NetBSD sound system. Beware that for using this library, the needed header file
As you can see, there is quite a number of libraries available to be used by applications and programmers. In addition with the source layout for programs described in the first part of this series, this is all that makes up the NetBSD operating system’s userland. In the next part, we will have a look at the part that’s not user-visible, diving into the kernel sources.
Author maintains all copyrights on this article. Images and layout Copyright © 1998-2002 Dæmon News. All Rights Reserved. April 2002 Search Submit Article Contact Us Join Us Merchandise
DOSSIER and the Meta Project (Part 3)
Rich Morin
Meta can be described as a distributed mechanism to facilitate access to information on Free and Open Source software. Although Meta can support many forms of use, most users will access it by means of a command-line or GUI-based "operating system browser", running on a local host machine. Meta servers will provide support for these browsers, supplying access to documentation, metadata, rules, and even source code.
Browsing is not the only possible form of use, however, and even browser-based access may depend on some higher-order analytic activities. This article speculates on some of these activities, concluding this three-part series. If you're just coming into the series, you may want to look at Part 1 and/or Part 2, acquainting yourself with the problem description, proposed solutions, etc.
Augmented Browsing
An operating system browser should be able to report, in an integrated manner, on the state of the local system and/or network. Assuming the ability to use background processing time and a bit of storage, it should be able to expedite its responses and expand on its coverage.
Ideally, the browser would have instantaneous access to a complete, integrated, and timely "snapshot" of the system. Unfortunately, a running operating system is a moving target. We don't have time or space to record everything that happens; even getting consistent snapshots is tricky.
Time elapses in the process of taking a snapshot, so no practical snapshot of a running system can be perfect. Fortunately, most files and directories change very slowly and the system provides ways (e.g., time stamps) to show whether a file has been modified. In short, we can generate a consistent snapshot, with some effort.
In addition, file system technology is improving. Log-structured file systems aren't for everyone, but they do appear to solve the problem. Kirk McKusick's recent work on soft updates and file system snapshots for BSD, while solving the problem of snapshot consistency for fsck(8), largely solves it for our purposes, as well.
By analyzing snapshots of the file system, a Meta daemon can maintain an up-to-date analysis and summary of the local system and/or network configuration. Armed with this information, a browser can report, quickly and in considerable detail, on the local configuration.
Adding the Meta server's wealth of generalized knowledge, however, a browser can do far more. Using facts and rules about common and best practice, it can warn the user about possible configuration problems (e.g., missing files, unusual permissions and/or ownerships). It can also explain the format and function of given files, even if they do not currently exist on the local system. Some control and log files, for example, must be created by the system administrator before the associated daemon(s) will make use of them.
In fact, because Meta collects and archives knowledge about assorted Unix variants, it can tell the user about things that are only documented in other systems' manuals. For instance, a Linux manual page might well contain a reference that FreeBSD misses. If the referenced item exists on FreeBSD, Meta could use the reference to assist a FreeBSD user.
Alternatively, let's say that a Linux user is trying to port a shell script that uses the fstat(1) command. This command doesn't exist on Linux, so "man fstat" won't help much. In fact, it will tell the user about fstat(2), which isn't even particularly related. Meta, in contrast, can tell the user that FreeBSD has an fstat(1) command, display its man page, etc. The user can then decide whether to port the command, look for a Linux equivalent, etc.
Even if the "same" command exists on the user's system, there may be differences in options or behavior. For example, most BSD-style commands do not support GNU-style "--" options. Some GNU options, such as "ls --all", are simply verbose renditions of traditional flags. Others, like "ls --indicator-style", provide totally new functionality.
By allowing the user to view documentation for multiple operating systems, Meta can help the user to sort things out. DOSSIER's document collections take a similar approach, by the way, often including multiple man pages for a given command.
Automated Analysis
Analyzing system files and relationships is a daunting task. Each file type has its own format and the relationships between files are often murky. Consequently, most current analysis tools (e.g., cscope, tripwire) have very limited scope.
Meta's global approach to data collection, however, promises to resolve most of these issues. Meta extracts information from a variety of system file formats, making it available in a consistent, easily accessible manner. It also tracks many file relationships, mapping them into its data structures.
With these problems under control, many forms of analysis become trivial. A programmer, for example, might ask for a list of programs that use system calls this(2) and that(2), but not the_other(2). A developer might ask which OS variants support a given system call.
Maintainers of documentation and software may also benefit. In loading the supporting data structure for the FreeBSD Browser, I found and reported large numbers of inconsistencies in "Files" and "See Also" entries. A full Meta server could generate specialized reports, allowing OS developers to locate and clean up specific classes of problems.
Meta's ability to examine cross-OS relationships could be quite useful in attempts to standardize OS features. For example, it should be possible to leverage Meta's knowledge base in evaluating compliance to standards such as POSIX or LSB.
Given that background processing time is essentially free, it makes sense to have Meta compare different OS versions. The resulting information can be retained to expedite interactive inquiries; summary reports can be generated for examination by interested parties. The Grand Experiment
The evolution of Unix into a family of related operating systems has, more or less by accident, created a "grand experiment" in software engineering. Counting only Free and Open Source variants, we have:
the GNU/Linux family Unix V[1-7] and 32V (courtesy of Caldera) 4.4BSD (courtesy of Caldera and UC Berkeley) modern BSDs (Darwin, FreeBSD, NetBSD, OpenBSD) assorted, lesser-known Unix variants a slew of add-on software
By comparing and contrasting these variants, both as distributed releases and running systems, we should be able to find some interesting patterns. In particular, we may find clues about the "common and best practice" alluded to, above.
As a simple example, suppose we ask Meta to compare file ownerships and permissions on each of the OS variants. A knowledgeable system administrator could then examine Meta’s results, trying to determine the reasons for the differences.
Differences which appear to be ill-considered or dangerous can be reported, as bugs, to the OS variant’s developers. Differences which seem benign, if peculiar, can be published as part of a public "standardization report". The developers’ rationale for the differences can, if desired, be included in the report.
Similarly, running systems can be examined for unusual settings. If a significant number of administrators are changing the permissions on a particular file or directory, there may be an issue that the OS developers should examine. Again, Meta’s role is merely to examine and report; the parties involved must decide what action to take.
In the process of examining these differences, it is almost inevitable that some general rules will emerge. These can be folded back into Meta’s analysis software, allowing it to spot additional problems without human assistance. Although I don’t see Meta becoming a true "Expert System", some aspects of expert system technology will surely play a part in its operation.
In summary, I see Meta as both a community resource and an evolving research project. Although it can never be completely finished as long as Unix-derived systems continue to evolve, this does not detract from its utility and interest.
Author maintains all copyrights on this article. Images and layout Copyright © 1998-2002 Dæmon News. All Rights Reserved. April 2002 Search Submit Article Contact Us Join Us Merchandise
Trawling the Ports Collection
Greg Lehey
One of the most important parts of the BSD distributions is the Ports Collection, called the Packages Collection in NetBSD. It neatly solves the problem of how to get third party software running on your machine, and there's something in there for everybody. The idea came from FreeBSD, which still has the largest collection of ports--nearly 7,000 of them, in fact, including just about anything you can think of. A large number of programs that are part of Linux base distributions are in the BSD Ports Collections.
Of course, you can have problems with the Ports Collection:
How do you know what's there?
We continually hear complaints from converts from Linux: ``I hate this csh shell. Where's bash?''. It's there of course--in fact, there are two different versions--but where?
How do you know how to use the ports? Often the documentation is less than perfect, or you don't know where to find it.
Let's consider finding bash. If you're running FreeBSD, you could start with link to the index page on the FreeBSD home page, where you'll find a list of the ports categories and links to more details. But what category is bash in? There are 64 categories, and you may not know where to look for bash. By the time you get to Shells, somewhere near the bottom, you've compared so many others that may no longer be sure. And that's just for a program you know and you're looking for. How can you find things you don't know? And once you've found it, how do you use it?
This is the first of a series of articles which addresses these issues. In particular, every month I'll dig out an interesting port or two, but first, this month I'll give a brief introduction to the Ports Collection and the environment in which I do my work. Hopefully you'll find it interesting; one way or the other, I'd be interested in feedback. Installing ports
I'm assuming here that you know the basics of the Ports collection. If not, read James Howard's article in the June 2000 edition of Daemon News. That article mentions the standard way of installing a port, using make. In fact, you have three alternatives:
1. You can build the package from source, as described in the article. This is the standard way, and it is the most flexible, but it is also the slowest. 2. You can get a pre-compiled package and install it using pkg_add. A package is a tar archive which contains the installable results of building a package. You just need to install it, so it is a lot faster. On the down side, there may be version dependencies, such as on libraries, which mean that the installed result may just plain not run, or it may create spurious error messages. For example, while writing this article, I was given an OpenBSD package for xtset, which we'll look at below. It was built on a -CURRENT OpenBSD machine, and I'm running 3.0. This is what I got:
$ cd /usr /usr/libexec/ld.so: warning: libc.so.28.0: minor version >= 3 expected, using it anyway /usr/libexec/ld.so: warning: libc.so.28.0: minor version >= 3 expected, using it anyway $
The error messages come from xtset. In this case, they're benign, but similar problems could stop the package working altogether.
3. If you don't have the package locally, you may still be able to install a remote package. The way you do this depends on the system you're using: NetBSD and OpenBSD require an explicit URL, while FreeBSD can work out the URL given the package name. For example, on FreeBSD you could install zsh with the command:
# pkg_add -r zsh Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-stable/Latest/zsh.tgz... Done. updating /etc/shells
With NetBSD and OpenBSD, you need to find out the location first. Then you write something like:
# pkg_add ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.5/i386/shells/zsh-3.0.8.tgz ftp: connect to address 3ffe:8050:201:1860:2a0:c9ff:feed:b7ea: No route to host setenv PKG_PATH='ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.5/i386/shells' missing: ./libexec/cgi-bin (created) missing: ./share/mk (created) missing: ./share/nls/uk_UA.ISO_8859-1 (created)
That ``No route to host'' message is benign: ftp.NetBSD.org has an IPv6 address, and ftp tries it first. My system isn't connected to an IPv6 network, so there's not much hope of it working. The real issue here, though, is knowing that the zsh you're looking for is at ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.5/i386/shells/zsh-3.0.8.tgz. I normally find these things out by searching the ftp site. Once you're at the site, of course, it's easier just to fetch the file locally. What's in a port?
But what is zsh? Why should you want to install it? Those are the questions that I'm going to be addressing in these articles. If you look carefully at the example above, it looks as if it should be a shell. But what kind of shell? Well, there's the documentation, of course, but it would take a lot of time to read the documentation for that many ports.
The Ports Collection acknowledges this problem and includes summary documentation. Again, the location depends on the version of BSD you're running:
1. FreeBSD supplies two files: a one-line description in pkg-comment and a multi-line version in pkg-descr. These files are in the base directory of the port. For the zsh we're looking at, they have the following contents:
pkg-comment
The Z shell
pkg-descr
Zsh is a UNIX command interpreter (shell) which of the standard shells most resembles the Korn shell (ksh), although it is not completely compatible. It includes enhancements of many types, notably in the command-line editor, options for customising its behaviour, filename globbing, features to make C-shell (csh) users feel more at home and extra features drawn from tcsh (another `custom' shell).
If you want to use zsh completion system, you should type the following commands:
$ autoload -U compinstall $ compinstall
See also zshcompsys(1) manpage. :)
WWW: http://www.zsh.org/
2. In NetBSD and OpenBSD, there's only one file, pkg/DESCR, which in general correspond to FreeBSD's pkg-descr. In this case, they contain the first paragraph of the pkg-descr above.
So, are you happy with that description? Neither am I. There's more documentation with the package, of course, but nothing telling you ``use zsh because of its really neat command line completion'', or maybe ``I don't use zsh because it's such a pain to set up the command line completion to do exactly what I want to do''. Which is correct? That's a matter of opinion. These articles are my opinion. Getting started
So, what shall we start with? I have a few favourites which I install on every system I use for any length of time. We'll look at the shell environment this month.
A shell: bash
I've already mentioned bash, the Bourne Again Shell from the Free Software Foundation. It's one of a plethora of shells available, and it's the one I use. Why do I use this one? I have a number of reasons:
1. I've been using it for about 13 years, more or less since it first became available. Before changing, I need a reason.
2. It has nice command line editing, better than most. tcsh has very similar editing, but either it can't do some of the functions that bash can, or I haven't found out how. See #1.
3. It supports Bourne Shell syntax. This is very important: you need to know Bourne Shell syntax for any serious shell script. Hardly anybody writes scripts for csh or derivatives, so why learn two shell syntaxes? Alternatives
There are a number of good alternatives to bash, of course:
I've already mentioned tcsh. The command line editing is quite good, but it doesn't seem to understand the Meta (Alt) key. It also has csh syntax, which invalidates it for me.
I've also mentioned zsh. I tried it for a while, but I never got the command line completion configured the way I want it. Maybe that's been improved in the meantime, but I have better things to do with my time.
There are a number of versions of the Korn Shell, ksh. OpenBSD ships with one which seems quite good. But again it doesn't seem to have as good command line editing as bash.
Installing bash
We've seen how to install ports. Simply do:
# cd /usr/ports/shells/bash2 (FreeBSD, OpenBSD) # cd /usr/pkgsrc/shells/bash2 (NetBSD) # make install
That's straightforward enough. Of course, after installation you're still stuck with the same old shell you had before. You need to run chsh. This will give you an editor with details of your account, which looks something like this:
#Changing user database information for grog. Shell: /bin/sh Full Name: Greg 'groggy' Lehey Office Location: Echunga SA Office Phone: +61-8-8388-8286 Home Phone: +61-8-8388-8250 Other information:
Just change the /bin/sh on the second line to /usr/local/bin/bash, save and exit, and you're in business. Well, sort of. Next time you log in, you should have a bash shell. But how do you use it? Where's the documentation?
The documentation bash comes with a lot of documentation, including a print manual and two man pages (bash(1) and bashbug(1)). The same content is also available as info pages. info is the GNU documentation system; we might look at it in a later article. At the moment you can live without it. bash(1) currently runs to 5144 lines, nearly 100 pages. It's probably not the sort of thing you'd want to read on line, but it's handy for searching. By contrast, bashbug(1) is very short and just tells you how to report a bug. The real documentation is in the 138 page reference manual. But where does it go? It's in /usr/local/share/doc/bash. It's available in PostScript and HTML formats:
$ ls -l share/doc/bash/ total 2 -r--r--r-- 1 root bin 86238 Mar 27 14:28 article.ps -r--r--r-- 1 root bin 48004 Mar 27 14:28 article.txt -r--r--r-- 1 root bin 243837 Mar 27 14:28 bash.html -r--r--r-- 1 root bin 408887 Mar 27 14:28 bash.ps -r--r--r-- 1 root bin 8043 Mar 27 14:28 bashbug.ps -r--r--r-- 1 root bin 385523 Mar 27 14:28 bashref.html -r--r--r-- 1 root bin 654109 Mar 27 14:28 bashref.ps -r--r--r-- 1 root bin 125347 Mar 27 14:28 builtins.ps
You still need to print it out, of course. That's straightforward enough if you have a PostScript printer, but otherwise you'll need ghostscript, which we'll look at some other time.
Some interesting customizations
Shells are infinitely customizable, and since they're one of the two main tools you use to interface with the computer (the other is the editor), it's worth making things comfortable.
Everybody has his own views on this, of course. Here are some of mine. I put this in my .bashrc file: case `tty` in /dev/ttyp*) TERM=xterm-color ;; /dev/pts/*) TERM=xterm-color ;; /dev/ttyq*) TERM=xterm-color ;; /dev/ttyv*) TERM=cons25 ;; /dev/ttyE*) TERM=cons25 ;; /dev/ttyC*) TERM=vt220 ;; *) TERM="" ;; esac export TERM
These definitions set the TERM environment variable depending on the device name. if [ "$TERM" = "xterm" -o "$TERM" = "xterm-color" ] ; then PS1="\[^[[34m\]=== \u@\h (`tty`) \[^[[31m\]\w\[^[[34m\] \# ->\[^[[30m\]\[^[[47m\] " else PS1="=== \u@\h (`tty`) \w \# -> " fi PS2="\u@\h \w \! ++ "
The sequence ^[ is in fact an escape character. These definitions first check what kind of terminal I'm running on. If it's an xterm, it sets a prompt in colour, which makes it easy to distinguish on a large screen:
=== root@kimchi (/dev/ttyp1) /etc 20 ->
If it's not an xterm, the prompt is monochrome, but otherwise the same. This information includes:
The first part, ===, is just to make it easier to find in a large list on an X display, in the same way that the colour helps. An xterm window (a terminal emulator which runs under X11) on a high resolution X display can contain over 100 lines, and searching for command prompts can be non-trivial.
Next, root@kimchi is the user ID and the name of system on which I am working, in the RFC 822 format used for mail IDs. Multiple systems and multiple users can all be present on a single X display. This way, I can recognize which user I am and what system I am running on.
/dev/ttyp1 is the name of the current controlling terminal. This can sometimes be useful.
/etc is the name of the current working directory. When you're in your home directory, it is represented as ~(tilde). Most shells, but not all of them, support this symbolism.
20 is the prompt number. Each time you enter a command, it is associated with this number, and the prompt number is incremented. One way to re-execute the command is to enter !!20 (two exclamation marks and the number of the command), but bash offers better solutions.
Next, a function definition:
Make () { if [ "${MAKE}" = "" ]; then MAKE=make fi xtset `uname -n|sed 's:\..*$::'`:`pwd`: Make "$* "`date` (echo '=====' `date`: Make $*; /usr/bin/time -l ${MAKE} 2>&1 $*) | tee -a Make.log cd . }
This defines a function called Make. As the name suggests, it's like make. It runs make and copies the output to a file called Make.log. If something goes wrong in a make, you can then go back and look at Make.log to see what happened. It also uses xtset to set the title of the xterm to information about the build. For example, when building a FreeBSD kernel, it is set to: zaphod:/usr/src/sys/i386/compile/ZAPHOD: Make all -j2 -k Wed Mar 27 18:52:47 CST 2002
Finally, it times the build, so I can have more of an overview over what's going on.
What, you don't find xtset on your system? No, that's not surprising. It's in the Ports Collection. Well, in the FreeBSD Ports Collection, anyway. xtset
By comparison, xtset is tiny. It takes an argument which sets the title of an xterm. If you're not running on an xterm, it does nothing. There's really not much else to say about it, though of course it comes with a man page (xtsert(1)). Here's another shell function which takes advantage of it: cd () { command cd $* if [ $? = 0 ]; then xtset -t "%u (%T) %h:%D" xtset -i "%u@%h:%D" true else false fi }
This command redefines the cd builtin command. First it executes the builtin, then it uses xtset to set the titles separately for the xterm window and the icon. The format is similar to that of the prompt. When in my home directory, the title is
grog (ttyp1) wantadilla:~
This can get rather long for an icon, so the icon format is different:
grog@wantadilla:~
xtset is in /usr/ports/x11/xtset on FreeBSD. If you’re using NetBSD or OpenBSD, don’t despair: pick up the tarball from ftp://ftp.lemis.com/pub/xtset-1.0.tar.gz and install it manually. It’s not quite as easy as installing it from the Ports Collection, but it’s not that difficult either.
But why should you bother? There are good reasons: as I write this article, I have 77 windows open, spread across 5 screens. More than half of those are xterms, spread over about 10 machines. Just finding which is which becomes a challenge; xtset helps. More information about the Ports and Packages collections
These articles aren’t intended to be a complete guide to the Ports Collections. If you don’t have any background in installing ports, you may find it heavy going. You can get more information from these links:
FreeBSD Ports Collection NetBSD Packages Collection OpenBSD Ports Collection
Author maintains all copyrights on this article. Images and layout Copyright © 1998-2002 Dæmon News. All Rights Reserved. April 2002 Search Submit Article Contact Us Join Us Merchandise
Help, I've Fallen - How To Tune Your BSD System
Gary Kline, David Leonard, and Dirk Myers
This month we bring you a column on tuning your BSD system to best suit your needs and wants. Obviously, because everyone has different requirements, there may well be myriad different ways to tweak your system. If you run more than one machine you may wish to tune each differently.
Among the different kinds of BSD administrators might be, oh:
Joe College: experiments with BSD on an older (75-200) MHz box running over a modem from his dorm room, flat, or parents' house. He's upgraded to a 10GB IDE and wants to see just how much BSD can do on this antique hardware.
Gary Geek: has multiple computers running BSD. Maybe over a dial-up; maybe ISDN or DSL. One of his platforms may be doing DNS, another serving text-only (or nearly) web pages.
Wanda Website: has at least 2 fast servers. She is doing her own DNS and trying to get her own web business going. She has either a fast DSL or most of a T1 line. Concerned mostly that her cgi scripts and web pages are served rapidly.
Clark Consultant: runs his own small (few-person) consulting business. Web admin, DNS admin, network consulting hired gun. Wants speed and performance to match his 25 years in this industry. Works 20-hour days so he can pay his staff and himself; wants to tune his entire 7-platform network of various kinds of BSD architecture machines to play nicely with a few DOS PC's and squeeze ever last Hz of performance from his Free/Net/OpenBSD hardware.
Because of the scope of this topic -- in general and in specific -- we plan another column. Perhaps this October; maybe not until next year. We request input on this column: specific questions, questions with answers and comments. In a word, whatever you think would be helpful in sharing back with the BSD or open-source Unix community.
Meanwhile, among the topics for our second system-tuning column are:
Improving your interactive performance under X11.
nice'ing down low-priority programs.
Improving your system performance if you have heavy file-access use: (squid, file server, news server, web server) Program optimization, assuming that you are a competent hacker.
More, including whatever helpful tips we receive to share back!
As a first cut on this topic, let's assume that you are running no more than three BSD systems that you have networked with NIC's and a hub; your pipeline to the outside world is either a dial-up line, perhaps dedicated, or a cable or DSL link. Your main system is a faster machine with lots of RAM; if you have other systems, they may be considerably less powerful.
You may be a programmer at some level; you may be a tinkerer; you certainly are a user of a lot of the BSD ports that are available... so, naturally, you want the best performance from your hardware.
So, where should you start?
Before you begin, make some measurements before you change things so that you know whether your tuning has helped. To gather the measurements, use these commands: iostat(8), vmstat(8), pstat(8), netstat(1), nfsstat(1), sysstat(8), and time(1). To decide what to measure, guess at which parts of your system are overloaded, and which parts that are underloaded could take on some of that load. Continue to measure the results after each change. top(1) will give you a starting point to see where your system resources are being used.
iostat displays kernel I/O statistics on terminal, device and CPU operations. vmstat reports statistics that the kernel keeps on process, virtual memory, disk, trap, and CPU activity. pstat displays system data structures. pstat displays open file entry, swap space utilization, terminal state, and vnode data structures. netstat shows the network status. This command shows what services have open sockets on your machine. nfsstat displays statistics on NFS client and server activity. systat displays various system statistics in a screen oriented fashion using the curses screen display library, ncurses(3).
Since you're going to invest at least several days in tuning, it only makes sense to spend an time reading the manual pages and becoming familiar which what these statistical utilities do. Still, we give the following sample uses.
You'll notice that the output goes both to stdout and is saved in a temporary root directory named systune. Thus you can compare the results as you tune and with your final results.
# iostat -odICTw 2 -c 60 | tee /root/systune/iostat.out
# vmstat -i | tee /root/systune/vm.interruptstat.out # vmstat -w 5 -c 60 | tee /root/systune/vmstat.out
# pstat -t -T | tee /root/systune/pstat.out
# nfsstat -c | tee /root/systune/nfsstat.out # nfsstat -s | tee -a /root/systune/nfsstat.out # nfsstat -W | tee -a /root/systune/nfsstat.out # netstat -an | tee /root/systune/netstat.out
Some specific thoughts on the actual tuning include:
Where your system allows more RAM, adding RAM will certainly boost performance. With more RAM, you can set up a memory file system (MFS) and boost your CPU-intensive performance significantly. Michael Lucas gives the how-to's in BSD Tricks: MFS
If you need a huge amount of disk storage, it might pay to buy, say, two 160GB IDE drives. You may want to join then with vinum http://www.vinumvm.org/. (Currently, this "volume manager" is avilable only on FreeBSD; as soon as volunteers step forth and port it to NetBSD and OpenBSD it will be available across the spectrum.)
You might try juggling disks around, or trade disk for memory or network bandwidth.
Select kernel features carefully at compile time. Leave out drivers and features that your system doesn't use, so the kernel image takes up less RAM. (The physical memory occupied by the kernel is not available to processes, as the kernel image and some of its data structures are not paged to disk.)
Some tuning parameters involve how much hardwired ram is used for some features.
# kldstat -v will give you some clues regarding what kernel modules you have added. Check the man page for kldstat(1) for more details.
Here are a few additional disk- and memory-type recommendations:
BUFCACHEPERCENT is percent of RAM to use as a file system buffer (defaults to 5). If your system uses the disk heavily and has free memory, increasing this value may improve performance. (Check to see that paging decreases)
Figure out the bandwidth (or expected bandwidth) to your disks and then spread/reorganise them evenly about the disk controllers.
Put especially large and active files or partitions (including swap) on their own disk. Or use ccd(4)/raid(4) (or vinum on FreeBSD) and spread the data over multiple disks.
Use newfs(8) on such disks to change the ratio of inode space to data space; or the block and fragment sizes, etc.
Use iostat(8) to check your progress.
Reduce disk/io by using a memory filesystem (MFS) for /tmp. --Make sure to read Micheal Lucas' article, though! BSD Tricks: MFS
Turn off unneeded logging, or send logs to a network loghost or nfs.
Understand your entire system that might comprise multiple hosts, and the people and jobs that actually use them. Think holistically.
About the Authors
Gary Kline has been porting code since the late 1970’s. When he isn’t hacking code, he’s hacking prose or pretend poetry, or listening to jazz radio and slurping down espresso.
For four years he has been writing the software equivalent of a mind-machine, dubbed Muuz, and has already released some alpha code for FreeBSD. Check the FreeBSD ports tree if you are interested. A new release in due in the first quarter of the new century...with luck!
Recent adventures include an ISDL link to the net, including the thrills of learning about the Domain Name System and network and mail administration, and souping up his Public Service Unix website.
[home|mail]
David Leonard is a PhD student in the Department of Computer Science and Electrical Engineering at the University of Queensland, Brisbane, Australia.
His area of research is QoS-adaptive component software architectures, and in his spare time is a developer for the OpenBSD project. That said, David enjoys living the quiet life with his wife, Kylie and cat, Mu. He especially enjoys frequenting Moreton Bay’s many fabulous places to eat. Mmmmm!
[home|mail]
Dirk Myers normally writes about C++. In his so-called spare time, he does unskilled lab work related to his wife’s thesis project.
[mail]
Author maintains all copyrights on this article. Images and layout Copyright © 1998-2002 Dæmon News. All Rights Reserved. April 2002 Search Submit Article Contact Us Join Us Merchandise
BSD project management
Greg Lehey
My last article aroused more interest than I've had for a while. Gratifyingly, almost all people who commented agreed that we need to be open to the GNU and Linux people.
The BSDCon
In February we had the third US BSDCon, this time in San Francisco. For the first time it was run by USENIX.
I found that the event was not as successful as previous BSDCons. It would be unfair to blame USENIX for this: things have changed a lot in the last year, and we've seen much less activity at other conferences as well. In addition, of course, the novelty is beginning to wear off. This is the fourth BSDCon I've been to, and people are getting used to them, so for example there were fewer photos taken and published on the web than on previous occasions.
From my point of view, I found the subsequent FreeBSD kernel summit to be the most interesting part of the conference: increasingly, we're getting a lot of FreeBSD hackers together to discuss the direction of the project. This time we had about 30 people crammed into a room which had been intended for far fewer. It was well organized, and we got through our material on time. It did, however, highlight a problem that is becoming increasingly important: how do we organize the project?
Free software project management
Last month marked the tenth anniversary of the first public release of 386BSD. That release was not even what we would call alpha quality today: to give a feeling, the FAQ which Karl Lehenbauer published on 29 March 1992 contained the following warning:
WARNING * WARNING * WARNING ======
Do not even test-boot 386BSD from floppy with a hard disk present on the system if you do not want to risk trashing your hard disk!
If 386BSD crashes, it may write a core dump on your hard disk!
At that time, there was plenty of work for all volunteers to do just to make it into a system reliable enough to be an alternative to commercial software. There are a lot of stories to tell about how that happened, and how the NetBSD and FreeBSD projects arose as a result, but I would guess that the systems passed the Microsoft ``Windows'' reliability threshold in 1994 or 1995. NetBSD, FreeBSD or BSD/386
I was in a different position. At almost exactly the same time, I bought my first copy of BSDI's BSD/386, beta version 0.3.2 on tape. By comparison with 386BSD, it was rock solid. By comparison with Microsoft, it was rock solid. In fact, by comparison with the Interactive UNIX/386 that I was running on another machine, it was rock solid. I had originally bought BSD/386 because I had just left a company where I had access to the UNIX (System V) source code, and I felt lost without it. In comparison with the cost of a System V license, $1000 seemed to be a small price to pay, and I hadn't expected to get very much for it. After a couple of weeks, though, I was hooked. This was a system that really worked, and which didn't hang or panic the way the Interactive system did. It also ``felt'' better than System V: things worked without having to be worked around.
At that time, I didn't have very much to do with the free versions of BSD. I had paid my money, I had a product which I thought was very good, and which definitely didn't overwrite my disks on the rare occasions when it panicked. I came to FreeBSD by another channel: I was porting software to Novell's UnixWare, and I needed a US publisher. I finally came to Walnut Creek CDROM, who had supported FreeBSD right from the beginning. That's how I became involved with the FreeBSD project. I installed FreeBSD and played around with it, and I installed it on some customer sites where they didn't want the expense of BSD/386, but I didn't really use it much myself.
That had to change in 1995, when Walnut Creek asked me to write a book about FreeBSD. To my surprise, by that time it was no longer clear that FreeBSD was any worse than BSD/OS, as BSD/386 was now called. In the course of time, I stopped using BSD/OS altogether--I didn't even install the last update I received. FreeBSD had made it.
What next?
That was six years ago, more than half the life of the project. Once the system was stable and showed reasonable performance, development targets changed. There was still plenty to do:
Device drivers were sadly lacking. As new hardware came out, new drivers were needed. This is still an ongoing issue. The system needed to be adapted to new buses. In 1992, about the only buses a computer were the ISA bus and the Microchannel (MCA). EISA was also there, but it was pretty compatible with ISA. MCA wasn't important enough to worry about, and FreeBSD has never supported it. In the years that followed, though, a number of new buses appeared, notably the PCI bus and the laptop PCMCIA standard. In FreeBSD, the first support for both was pretty rudimentary, and to get things right required significant restructuring of the kernel, some of which is still going on. Similar issues apply to interfaces like USB, FireWire and infrared (IrDA) and system management features like APM and APCI. Multiple processor machines appeared and became relatively commonplace. FreeBSD had to be adapted to handle multiple processors. The first attempt was simple but not very effective: UNIX was never designed for multiple processors, and the compromise chosen performed very badly under some circumstances, notably those chosen by the Mindcraft Benchmark. As a result of this, we decided to reimplement the SMP code in a more intelligent way, about which I reported in August 2000. That work is still ongoing, and there's no evidence that it's nearing completion.
And the problem? There is a difference in approach between these three categories. If you decide to write a driver, say, for a new sound card or Ethernet adaptor, you can go and do it. Depending mainly on how much support you get from the manufacturer, it may be easy, difficult or impossible. But when you're finished (if you finish), the rest is simple: you commit it to the source tree and wait for people to tell you about bugs, then you fix them. The majority of users who don't have that specific hardware hardly even know you've committed the software, and they are certainly not inconvenienced.
Now look at the new bus code. That looks pretty simple, really: if you only have an ISA bus, what do you care if somebody commits software for PCMCIA?
To a certain extent, you don't. But rewriting the bus code to support multiple buses also affects the ISA code. There could be a number of cases where code introduced to support PCMCIA will also affect the ISA code. Even if you, as a user, don't have any problems, the device driver writers will. The interfaces change, and typically people like writing code more than documenting it, so it can be a frustrating experience just to find out how to modify your existing driver, or how to write a new driver.
Rewriting the SMP code is the most invasive of all. It affects everything in the system. You can't write kernel code without being affected. In August 2000 I wrote:
For some time, I have had a theory that the open source model works well for small projects, but it is not optimal for really big undertakings. Even before the Mindcraft incident I had decided that getting good SMP support for BSD would be a proof of this theory.
As I went on to describe, help came from an unexpected quarter: BSDi gave us alpha-quality SMP code, and we got it working pretty quickly. That didn't solve the problem, of course, it just gave us the framework in which we could develop the new software.
And the problem, again?
The real problem we're seeing here is the issue of project management. In the old days, people could go off and do their own thing. Nobody worried, because most things didn't affect anybody else, and it was pretty much left to the individual to decide what to do and how to do it.
All that changes radically with projects like the SMPng project. Just about anything anybody does will have an effect on the other developers. Under these circumstances, the ``go it alone'' approach doesn't work well. We've seen the results already in the project: the initial set of locking primitives (which the project tenaciously insists on calling mutexes) were not adequate for the desires of some of the developers, so they introduced their own favourite constructs. It didn't take at all long before rules (not just documentation) had to be introduced to describe how synchronization should be done.
And the solution?
So we've identified the problem. But how do we solve it? Commercial software development is quite clear: there's a project manager, and his word is law. But that means more planning than the average free software hacker wants to do, and more compromise than many are prepared to accept. We still have people in the project who take the attitude ``I want to work on this particular issue, and it's my God-given right to do so''. That worked once upon a time, but it really doesn't work any more. BSD may be free, but even volunteer projects need some management.
I don’t know the answer to this one. As I said over two years ago, this issue may be the one which changes the shape of our projects for ever. My best bet is that some commercial entity developing BSD takes over the project and supplies the management; but that would probably involve an independent development which would be put back into the project on completion.
I’d be interested in feedback on what other people think about these issues. In a certain sense, we’re treading new ground. As always, the time ahead will be interesting.
Author maintains all copyrights on this article. Images and layout Copyright © 1998-2002 Dæmon News. All Rights Reserved.