OpenCSW

The goal of this project is to ease the selection of different tools with the similar functionality under a specific name which will be stable on updates. It was discussed to facilitate the "alternatives" system from Linux. There are three major implementations, one from Debian, one independent (written in shell and text utils¹) and one from Red Hat, a part of chkconfig. The Debian implementation is written in Perl whereas the Red Hat implementation is a clone of Debians written in C. We initially chose to use the Red Hat variant². However, this was found to have issues blocking clean sharing (via NFS, or otherwise) of /opt/csw. Phil is now close to releasing an OpenCSW-specifically targetted clean-room implementation, that resolves this issue.

What does it do?

Using the "alternatives" framework, allows that more than one package can provide a back-end binary that implements the features of a user-facing binary name(or possibly even a library).

A trivial oft-used example that we dont actually implement:
Lets say we wanted to offer /opt/csw/bin/editor.

We have multiple packages that could provide that functionality. Each of those packages could register itself as providing "/opt/csw/bin/editor", short name of "editor".
Then, when one of these packages is installed, or removed, the "alternatives" framework sets a symlink for /opt/csw/bin/editor -> (one of the implementations), based on either a "priority" registered by the application itself, or by site admin preference.

The main project administrative overhead, is that maintainers must agree on the "short name" involved, and also that they cant squabble over whose package should be "highest" priority.
(which is obviously why we dont attempt to do this for "editor" ;-)

A selection of packages that benefit from using "alternatives"

Where marked means it has been done.

Alternatives between different versions of programs

• automake (for selection of preferred version of automake to use by default)
• mutt (for selection of slang vs ncurses based screen formatting)
• tcpwrappers (for selection of standard vs extended syntax in config files)
• sudo (for selection of standard and ldap version)
• pigz (preferred tool over gzip)
• pbzip2 (preferred tool over bzip2)
• apache2 (preferred mpm)

Select between 32 / 64 bit versions

The packages listed here behave differently between 32 and 64 bit because of different on-disk formats of the data files. In general the 64 bit version should be preferred if the system supports it to allow smooth data growth without the need to export/import when going ftom 32 to 64 bits. However, manual selection of 32 bit version is possible for certain usage scenarios which are known to stay within 32 bit data space and need the speed. The automatic selection could be done by an additional mode in the CAS matching up the results of isalist with available binaries.

• MySQL
• Postgres
• OpenLDAP
• Kerberos

Selection of basic and fancy library versions

Some libraries can be compiled with a lot of dependencies. For the sake of completeness all possible dependencies should be compiled in. This is contrary to minimalism, where you only want necessary functionality. Both goals can be achieved by making different flavors of a package, where each one is compiled against a different set of libraries. Starting with a basic set there could be different levels up to full dependencies. Direct candidates are:

• curl (for basic access HTTP/HTTPS would be sufficient)
• neon GAR Makefile (minimal dependencies for SVN, fancy ones like libssh2 pull in CSW X11)
• giflib GAR Makefile (minimal dependencies without X11, with OpenCSW X11) [OpenCSW X11 canceled]

Things to keep in mind:

• checkpkg: There are two alternative libs, one libxxx-minimal.so and one libxxx-full.so. The app xxx tries to link to libxxx.so which is linked during pkgadd time via alternatives. No chance for checkpkg to verify the symbols unless GAR tells checkpkg the alternatives-pathes. -> This could be leveraged by not renaming the libs during merge, but do different self-contained pkgroots instead.
• Using <pkg>-config and pkgconfig: There is only one file with minimal dependencies in it created from the minimal library. It is therefore important at least for the full library to be self-contained by linking to all needed libs directly.
• When libraries are switched between minimal/fancy the original name of the library being a link now was registered in /var/sadm/install/contents without alternatives.
  • The registration is necessary for checkpkg to find out the package for the library bindings.
  • The registration should be done for the package with the minimal library as the dependencies from the packages using the library should also go the minimal library package.
  • The registration is only relevant when libraries are switched. For all other usecases there is no need to register the link as it is removed by the CAS on package removal automatically.
  • The registration is only relevant when the switched library is the one with the SONAME bound against

Direct (non-"alternatives") handling for shared libraries

After testing it has been shown that there is a more elegant solution to alternatives for libraries using the AUXILIARY filter for linkage instead of the alternatives application. Basically it goes like this:

cc -G -c empty.c
ld -G -h libneon-minimal.so.26 -o stub-libneon.so.26 empty.o
ld -R /opt/csw/lib -G -h libneon.so.26 -o libneon.so.26 -f libneon-full.so.26 -f libneon-max.so.26 stub-libneon.so.26

• The idea is now to take the static library and make a shared library with AUX vector in it with something like

ld -G -o ~/libtidy-minimal.so -z allextract -h libtidy.so.26 -F libtidy-full.so.26 /opt/csw/lib/libtidy.a

• Solaris 10 Linkers and Libraries Guide: http://dlc.sun.com/pdf/817-1984/817-1984.pdf

Replacement of existing Sun packages

• sendmail (for selection of mta)
• postfix (for selection of mta)
• qmail (for selection of mta)
• cups (for selection of printing system)
• ssh (for selection of Sun SSH and OpenSSH)
• pigz (for selection of Sun gzip and pigz)
• pbzip (for selection of Sun bzip2 and pbzip2)

Issues on replacing Sun functionality by CSW packages

• SUNW packages must be removed. Otherwise patches installed later will undo the changes. Especially, removef is not sufficient as the packages stays installed and the patch would pkgrm/pkgadd the SUNW package completely.
• Reinstalling SUNW packages is not straight-forward as they must be from the same maintenance update and then all patches need to be reapplied³
• The replacement functionality could be provided by a package named CSWsunw<pkg1>, which would depend on CSW<pkg1> and marked incompatible against SUNW<pkg2>and install symlinks or alternatives on the previous tool locations.
• For these CSWsunw<pkg> packages there may be an option in pkgutil like '-R' ("replacement") which automatically deinstalls the conflicting packages first.

GAR integration

Alternatives have been integrated into GAR for easy addition to packages. You can see the GAR docs on Alternatives⁴ for details.

Alternatives and NFS-Shared /opt/csw

Unfortunately the links of the alternatives are kept under /etc/opt/csw/alternatives. When using an NFS-shared /opt/csw the libraries using alternatives can not be found on the machines using NFS. This means that any package currently usage CSWalternatives, will BREAK, on NFS-shared /opt/csw systems.

As a workaround the necessary links inside /opt/csw/lib can be done on the NFS master manually, Alternatives will then not touch these manually generated links.

For the future, we are working on a new implementation of an "alternatives" system.

Next steps for OpenCSW

1. Build "alternatives" package as back end⁵
   1. Done: http://mirror.opencsw.org/experimental.html#alternatives
   2. Done: Installed on build8st and build8xt
2. Done: Integrate alternatives-support in some packages for testing
   1. Done: mutt
   2. Done: automake
   3. Done: tcpwrappers
3. Done:Release alternatives
4. File upgrade bugs against the packages that should use alternatives
5. Change backend implementation, so that it is transparently friendly towards NFS. Dagobert has said he does not object to this, but also "does not have time" to change the code.

Keeping manual selections on package upgrade

There are two different cases⁶:

1. Use alternatives to select different binaries inside a package. This would be used for CSWautomake, where there are multiple versions of the same software inside the package. In this case the manual selection should be preserved iff there is an update going on.
2. Use alternatives to select different flavors of an application in different packages. This would be used for CSWmutt-ncurses and CSWmutt-slang. As you either upgrade a package or simply deinstall it you cannot decide from an in-package perspective if the current selection should be kept or discarded.

For case 1. the pre- and post-operations look useful to copy the link from /etc/opt/csw/alternatives/… to the preservation area similar to cswpreserveconf and replace the newly made link in /etc/opt/csw/alternatives, but only iff the group is operating in manual mode. The replacement should be done with set to readlink() (set is btw. only available in the RedHat implementaion, so we probaly choose the right one :-). Otherwise an addition of a new version would not automatically lead to updated links even if the group was in automatic mode.

For case 2. the situation is slightly different: When multiple packages containing binaries of the same linkgroup are upgraded and the same procedure as 1. would be used the link would be preserved and recreated multiple times, but I guess it must be this way as on —install in the case of the package the link would be adjusted and manual-mode will not be preserved on complete deinstallation of the linkgroup.

Current implementation: A path-group can be in manual mode, then no automatic updating on priority is done. If the package containing the manual selection is removed the pathgroup switches to automatic mode and selects the path with the highest priority. On package install it is checked if the contained pathgroup was previously selected in manual mode. If this was the case, the previous selection is reinstantiated. The only drawback is when someone selects a manual path, removes the package and then reinstalls it month later - then the previous selection is also redone. But without the notion of "update" on pkgrm/pkgadd this can not be solved.

Usage example

Here⁷ is the original document.

We start with no symlink.

netra ~ # ls -l /opt/csw/bin/sudo*
-rwsr-xr-x   1 root     bin       221736 Oct  5 19:26 /opt/csw/bin/sudoedit
-rwxr-xr-x   1 root     bin       245232 Oct  5 19:30 /opt/csw/bin/sudo.ldap
-rwsr-xr-x   1 root     bin       221736 Oct  5 19:26 /opt/csw/bin/sudo.minimal

Let's tell the alternatives system that we've got something called 'sudo' which is provided by sudo.minimal.

netra ~ # update-alternatives --install /opt/csw/bin/sudo sudo
/opt/csw/bin/sudo.minimal 40
update-alternatives: using /opt/csw/bin/sudo.minimal to provide
/opt/csw/bin/sudo (sudo) in auto mode.
netra ~ # ls -l /opt/csw/bin/sudo*
lrwxrwxrwx   1 root     root          30 Dec  6 14:41 /opt/csw/bin/sudo -> /etc/opt/csw/alternatives/sudo
-rwsr-xr-x   1 root     bin       221736 Oct  5 19:26 /opt/csw/bin/sudoedit
-rwxr-xr-x   1 root     bin       245232 Oct  5 19:30 /opt/csw/bin/sudo.ldap
-rwsr-xr-x   1 root     bin       221736 Oct  5 19:26 /opt/csw/bin/sudo.minimal

The alternatives system has chosen the best alternative, the highest priority number. (A bit counterintuitive direction.) Let's see what /etc/opt/csw/alternatives/sudo points at.

netra ~ # ls -l /etc/opt/csw/alternatives/sudo
lrwxrwxrwx   1 root     root          25 Dec  6 14:41
/etc/opt/csw/alternatives/sudo -> /opt/csw/bin/sudo.minimal

We can now add the ldap-enabled version of sudo:

netra ~ # update-alternatives --install /opt/csw/bin/sudo sudo
/opt/csw/bin/sudo.ldap 20

Let's see what the alternatives system says about sudo now.

netra ~ # update-alternatives --display sudo
sudo - auto mode
link currently points to /opt/csw/bin/sudo.minimal
/opt/csw/bin/sudo.ldap - priority 20
/opt/csw/bin/sudo.minimal - priority 40
Current `best' version is /opt/csw/bin/sudo.minimal.

It has chosen the implementation based on priority. Let's try to
modify it by hand.

netra ~ # update-alternatives --config sudo
There are 2 choices for the alternative sudo (providing /opt/csw/bin/sudo).

 Selection    Path                       Priority   Status
------------------------------------------------------------
* 0            /opt/csw/bin/sudo.minimal   40        auto mode
  1            /opt/csw/bin/sudo.ldap      20        manual mode
  2            /opt/csw/bin/sudo.minimal   40        manual mode

Press enter to keep the current choice[*], or type selection number: 1
update-alternatives: using /opt/csw/bin/sudo.ldap to provide
/opt/csw/bin/sudo (sudo) in manual mode.

…and see where the symlink points to now.

netra ~ # ls -l /opt/csw/bin/sudo*
lrwxrwxrwx   1 root     root          30 Dec  6 14:44
/opt/csw/bin/sudo -> /etc/opt/csw/alternatives/sudo
-rwsr-xr-x   1 root     bin       221736 Oct  5 19:26 /opt/csw/bin/sudoedit
-rwxr-xr-x   1 root     bin       245232 Oct  5 19:30 /opt/csw/bin/sudo.ldap
-rwsr-xr-x   1 root     bin       221736 Oct  5 19:26 /opt/csw/bin/sudo.minimal

This symlink hasn't changed, but…

netra ~ # ls -l /etc/opt/csw/alternatives/sudo
lrwxrwxrwx   1 root     root          22 Dec  6 14:44 /etc/opt/csw/alternatives/sudo -> /opt/csw/bin/sudo.ldap

This one now points at sudo.