Common Knowledge Pittsburgh (CKP) maintains a number of servers (running several flavors of Unix) at different sites (often connected by low-speed lines). Maintaining the local software on each server became a large problem which was solved by the use of two different software programs, depot and sup. sup allows a host to update its software from a master server via the network, and depot places the software into the /usr/local hierarchy.
This document is meant for people who have dealt with maintenance of software installations on Unix machines. I am including the following definitions for people who may be interested in the issues CKP has faced but who are not very familiar with the daunting task of Unix maintenance.
The problems associated with software maintenance on Unix servers can be broken down into several distinct problems:
Unix does not provide an easy answer for any of these problems. (To be fair, I haven't yet seen an OS that solves these problems well on its own.) You will need some extra program or method to do it. I looked at several systems, none of which satisfied my requirements. Here are a few simple solutions with an description of their problems.
I never did find a system that would solve all three problems. I did, however, find a system to solve (1) well, and another to solve (2) well. I created my own system which combines the two systems I found and adds a few touches to solve (3).
sup (software upgrade protocol) was a system developed at CMU to distribute the source code to the Mach operating system. You need a master server running the sup file serving program (supfileserv). The master server has "collections" of software which it can serve. When a client wants to update its software, it contacts the master server and specifies which collection(s) it wants. The server sends a list of the files in the collection along with their last change date. The client decides what files it needs (based on the timestamps) and requests them from the server.
sup uses a single TCP connection for all of its data transfer, so it is very light on resources. The only security it allows is a plain-text password, though I have heard of a version that will do kerberos encryption. CKP doesn't need any security restrictions on our sup servers as all of our Unix software is free.
The standard Unix way of installing software is to put everything under /usr/local. depot allows you to put each package in a separate directory. When you run depot, it will create links from each of the files in each package to the appropriate place under /usr/local. depot keeps track of what has been installed, so it is easy to remove a package, upgrade to a newer version of a package, or back out of a new version.
One price of this convenience is speed. When using depot, every file in /usr/local is a soft link. These require at least two extra disk accesses to resolve. (Modern systems may cache soft links and/or store the link path in the inode; this will remove some of the speed penalties.) Another problem is that depot likes to completely control /usr/local; you should treat /usr/local as if it were read-only. This means that log and (changeable) data files should go elsewhere (I recommend /var).
CKP maintains one set of packages for each platform. There is also a set of "standard" packages (shell scripts or Perl) which will run on any platform.
Both depot and sup need to be run with very specific options. To make this easy on the administrators of the machines, I wrote a script called localupdate. localupdate will run sup to get the latest software, run depot to install the software, and mail the results back to the admin. localupdate was written in Perl.
CKP maintains a collection of packages for each supported platform. The master sup server keeps the files in /depot. Each collection has two items in /depot: a directory (named after the collection) and a file (named after the collection with .list on the end). The directory contains all of the packages in the collection; the file contains the names of those packages which should be installed.
There is also a directory named /depot/sup which contains information used by the sup server. Each collection has a directory in /depot/sup containing two files. scan is a file generated by the supscan program; it is a list of all files in the collection along with their timestamps. It is used to make the server more efficient. list is a file which states which files belong in a collection. Here are the contents of /depot/sup/solaris/list:
upgrade solaris.list upgrade solaris rsymlink solaris
This means that the file solaris.list is in the collection, as are all files in the solaris hierarchy (/depot/solaris, that is). rsymlink solaris means that symbolic links in /depot/solaris should be transfered as links.
To recap: the files in a collection include the following. <os> is the collection name, <package#> are arbitrary packages.
/depot/<os>.list /depot/<os>/<package1>/bin/* /depot/<os>/<package1>/lib/* /depot/<os>/<package1>/man/* /depot/<os>/<package2>/bin/* /depot/<os>/<package2>/lib/* /depot/<os>/<package2>/man/* /depot/sup/<os>/list /depot/sup/<os>/scan
Note that when the files are transfered to the client workstations they are put in /usr/local/depot rather than /depot.
It was stated above that /depot/<os>.list is a list of all of the packages in the collection. To be more accurate, it is a list of all of the packages that should be installed by default. If you have two versions of a package (pine-3.89 and pine-3.91) you would only list the version that most machines should use. You can also add new packages for testing on a few machines without making them available everywhere.
Each software collection is made up of a number of packages. Ordinarily software in installed in the /usr/local directory structure, like so (using tin 1.22 as an example):
/usr/local/bin/tin /usr/local/bin/rtin /usr/local/man/man1/tin.1
To build a package, you would install the files using the same directory structure (bin, man, lib, etc.) in the package directory. Thus, for NetBSD:
/depot/netbsd/tin-1.22/bin/tin /depot/netbsd/tin-1.22/bin/rtin /depot/netbsd/tin-1.22/man/man1/tin.1
(Remember that on the client workstation, files are installed in /usr/local/depot rather than /depot). Symbolic links are fine in a package, as long as they are relative (thus, a symbolic link to ../lib/less.hlp is fine; a link to /usr/local/lib/less.hlp is not. Hard links are also allowed, though sup will transfer the files as distinct files.
When depot runs on the client, it makes symbolic links in /usr/local that point to the actual files. Thus, if you have tin-1.22 and pine-3.91 installed, you would have:
Link Actual file (which link points to) /usr/local/bin/pine /usr/local/depot/netbsd/pine-3.91/bin/pine /usr/local/bin/pico /usr/local/depot/netbsd/pine-3.91/bin/pico /usr/local/bin/tin /usr/local/depot/netbsd/tin-1.22/bin/tin /usr/local/bin/rtin /usr/local/depot/netbsd/tin-1.22/bin/rtin /usr/local/man/man1/pine.1 /usr/local/depot/netbsd/pine-3.91/man/man1/pine.1 /usr/local/man/man1/pico.1 /usr/local/depot/netbsd/pine-3.91/man/man1/pico.1
depot creates a file in /usr/local/depot called struct.depot. In this file it keeps a list of all of the links it has created in /usr/local. You should never edit this file by hand! When depot is run it makes a backup copy of this file as struct.depot.OLD.
Thus far I have described how to manage files that are identical on every machine (or at least every class of machines). There are two additional problems; how to manage files that are different on each machine and how to deal with software that is not on the master sup server.
Every CKP machine has at least two collections which are created and maintained by sup: /usr/local/std and /usr/local/<os>. I create a third, /usr/local/site, for site-specific modifications. This collection is maintained by the site admin, not by sup.
The site collection always has a package named localfiles. Whenever I am compiling a package and I notice a file which must be customized by each site (like /usr/local/plexus/local.conf in plexus-30), I append .samp (for sample) to its name. Then when a machine is being configured, I make a copy of the .samp file (without .samp) in the localfiles package and modify it as needed. Taking the example above, I install the sample plexus configuration file on the master sup server as
/depot/std/plexus-30/plexus/local.conf.samp
On a client, this becomes:
/usr/local/depot/std/plexus-30/plexus/local.conf.samp
depot installs it in:
/usr/local/plexus/local.conf.samp
I copy the file to:
/usr/local/depot/site/localfiles/plexus/local.conf
and customize it. depot installs the (customized) file as:
/usr/local/plexus/local.conf
And I am done.
Thus, the package localfiles in the collection site ends up containing all of the locally customized files. Of course, there must be a file /usr/local/depot/site.list which contains all of the packages in site, including localfiles.
If you wish to install a new package which will exist only on one machine, you must create your own package in the collection site. Just create a directory, install the files, add the package name to site.list, and run localupdate. For example, if I wish to install TeX, a page layout program, I install it in:
/usr/local/depot/site/tex3.14
I also put its name, "tex3.14" in site.list, and when I run localupdate I have a TeX installed.
Note: Compiling and installing a program using this model is not as easy as it seems. You must tell the program to look for its data files in /usr/local, but to actually install files in /usr/local/depot/whatever. Better programs will have two sets of paths in their Makefile, one for where the program will be installand and one for where the program will end up living. GNU programs are exeptionally nice in this regard; you can say "make", then "make prefix=/usr/local/depot/netbsd/gcc-2.7 install". Most programs, though, will take a bit of poking through the Makefile to determine what should be installed where. Be assured that this effort is far less than the effort of trying up update software on many different machines by hand!
The previous section told how to add a package to a specific machine. The flip side of that is removing a package from a machine. For example, when I was testing Perl version 5 I needed to remove Perl version 4 from my machine. Simply put the name of the package to be removed in site.list prefixed by a dash. For example, here is what my site.list looked like:
localfiles -perl4.036 perl5.001m
The order of lines in the file is not important; localupdate will read all of the files before removing any lines. Of course, once I made Perl 5 the default I had to remove these lines. The current version of localupdate will not complain if you try to remove a nonexistant package or include packages in two different *.list files. This may be a bug, and will probably change, but in the meantime you should sanity-check the files on occasion.
Currently CKP maintains a single master sup server, sup.pps.pgh.pa.us. This server contains the software collections for all platforms. An alternate setup might have the collection for each architecture on a machine of that architecture (thus, the ultrix collection on an Ultrix machine, probably the machine used to build the Ultrix software). There are advantages and disadvantages for each setup, of course. I believe the current setup is best; this way you can easily compare packages in different collections. When you make a package on the appropriate machine, it is simple to tar it up, FTP it, and untar it on the master server.
This page is still a work in progress. We hope to have this finished by May 10, 1996.