CSW class action utilities

Incentive for cswclassutils

There are several common tasks which need to be done by packages on installation and deinstallation. Examples of such common tasks are

  • preserve existing configuration files
  • register daemons with SMF
  • setup cronjobs
  • add service users/groups

To not re-implement this in every package we leverage the class action script (CAS) facililty1 within pkgadd/pkgrm which provides a native ability to use installation and de-installation hooks for packages. We ship one single package with such CSW-specific class action scripts (cswclassutils) which are then used by all the other packages on demand.

Technical background

Class action scripts (CAS) live in /usr/sadm/install/scripts and are triggered on package installation / de-installation. This not only happens once during a pkgadd/pkgrm invocation, but on a file-by-file basis (that's the way Sun has designed it). To this end, the pkgmap2 file contains a "class" field for each file that comes with a package. This field defaults to none, but can carry any other class name (only one though). An example from dovecot:

...
1 f cswinitsmf /etc/opt/csw/init.d/cswdovecot 0755 root bin 1219 29065 1238276763
...

As can be seen from the third field, the cswdovecot file is associated with the class "cswinitsmf", hence two related scripts will be run on installation / deinstallation of this file. On installation it's /usr/sadm/install/scripts/i.cswinitsmf, on removal it's /usr/sadm/install/scripts/r.cswinitsmf (notice the "i" and "r"). Both files are not shipped with the package, but with cswclassutils and take care of registering and unregistering a SMF service. On installation you will see a line similar to…

[ verifying class <cswinitsmf> ]

…which means that the package file(s) belonging to the class cswinitsmf are being installed. Please note, that for this to happen, the pkginfo file3 of the package also needs to list the relevant class in CLASSES.

CLASSES=none cswusergroup cswcpsampleconf cswinitsmf

Last but not least, you need to add a dependency on CSWcswclassutils, in your depend file:

P    CSWcswclassutils

Known issues/caveats

cswclassutils and sparse zones

cswclassutils lives in /usr/sadm/install/scripts, so you have to install the cswclassutils package in the global zone (we are restricted to /usr/sadm/install/script because that is the only (permanently installed) location for class action scripts used by pkgadd/pkgrm). If cswclassutils in the global zone is downrev and you try to install packages with a dependency on cswclassutils in a sparse zones, pkg-get/pkgutil will try to install an updated cswclassutils package in the sparse zone (it has to do so, because the installed packages might rely on new cswclassutils features which were introduced in an updated cswclassutils version).

Thoughts:
To avoid the necessary installation to /usr/ and thus be friendlier to sparse zones, we could ship any required class action scripts directly inside the packages (in install/, just like postinstall or other installation related scripts), but this has two downsides

  1. Class action scripts shipped directly within a package trigger a warning on pkgadd and require user confirmation. This can be suppressed via an admin file (e.g. the default one used by pkgutil), but from a security standpoint it is questionable whether we should do something like this.
  2. Once a bug in a class action script were discovered, all packages shipping with this class action script would need to be rebuilt.

For the bug aspect, we could install the real class action scripts somewhere beneath /opt. Just like it is done right now with cswclassutils in /usr. Then we could ship stub class action scripts with each package, which would simply exec the actual class action script beneath /opt. This would still leave the installation warning though (see #1)

cswclassutils and package installation from the global zone

This is relevant to package maintainers mostly. Packages which are installed from the global zone without the -G flag will be installed into local zones also. Solaris-wise this is not implemented as pkgadd -R. Instead, the undocumented syscall zone_enter will be executed for a local zone installation to switch into the execution context of that zone. Subsequent commands are then executed from within the zone. Our cswclassutils CAS will thus also be called from within the zone, they will see all paths as from within the zone and not see any of the PKG_INSTALL_ROOT variables that would be set during an alternate root installation.

cswclassutils and alternate root installation

Alternate root installations via -R don't execute CAS from the alternate root, but from the fixed location /usr/sadm/install/scripts. Thus, a jumpstart installation where / is read-only and doesn't contain our cswclassutils CAS is doomed to break in some way because required CAS will not be executed. Depending on the CAS this has different effects (required users will not be created, SMF services will not be registered, etc.). OpenCSW package installations (for packages using cswclassutils) via pkgadd -R are strongly dis-encouraged and not supported.

Thoughts: For Jumpstart installations it might be possible to install a cswclassutils package into the installation source location so that -R will be supported. OTOH, this adds considerable burden for us to consider within the CAS (e.g. in i.cswusergroup we can't just call getent to get a list of configured users). Furthermore, Jumpstart supports calling scripts after the installation has finished and the system is up and running. Thus, people can use this stage to install CSW packages as they would do usually (without -R). This is not only easier to debug and develop, but also a similar usage pattern to what one would see in environments with puppet or other configuration management solutions where packages are installed after the base OS is up and running.

List of scripts

Currently this package contains these class action scripts:

  • cswinitsmf - provides SMF support (Peter Bonivart)
  • cswcpsampleconf - handles configuration files (Phil Brown)
  • cswpreserveconf - handles configuration files and preserves altered ones upon removal (Phil Brown)
  • cswmigrateconf - handles migration of configuration files, especially /opt/csw/etc to /etc/opt/csw/ (Maciej Blizinski)
  • cswusergroup - adding users/groups (Peter Bonivart)
  • cswpycompile - creates/removes python's normal and optimized bytecode files (Ben Walton)
  • cswinetd - installs services to run under inetd, supporting solaris 8/9 with the traditional inetd.conf or solaris 10+ with the smf managed services (Ben Walton)
  • cswetcservices - supports adding entries to /etc/services (Ben Walton)
  • cswtexinfo - handles addition/removal of texinfo pages to the texinfo dir (Dagobert Michelsen)
  • cswcrontab - adds/removes lines to crontabs of arbitrary users (Dagobert Michelsen)
  • cswpostmsg - display messages at the end of installs (Peter Bonivart)
  • cswetcshells - register a shell in /etc/shells, de-register upon removal (Ben Walton)
  • cswsslcert - generate an ssl certificate and private key (optionally a CSR instead of a cert) (Ben Walton)

Future additions to the package may be scripts for (please go ahead and help write the scripts if you see the need and have the knowledge):

  • Improved cswpreserveconf or varient, that handles copying from [any template dir], to /etc/opt/csw, rather than always just the same dir.

This wiki page gives more details about the class action scripts and how to use them. It should be noted that they are also documented in the package itself, under /opt/csw/share/doc/cswclassutils/README.CSW.

Also note that class action scripts are installed in /usr/sadm/install/scripts so if you run a sparse zone you need to install cswclassutils from the global zone. (or do some very custom things, such as provide a special /usr/sadm/install overlay mount for the zone)

cswinitsmf

This script makes it real easy to have "init scripts" be run, either in traditional init script style, or via SMF. The great thing is that you, the maintainer, need not worry about all the ugly details; it gives you an easy wrapper to handle BOTH ways, with one shot! You only need to provide the init script itself, and cswinitsmf will let the end-user decide on their preferred deployment method.

Add this line to the Makefile. It enables cswinitsmf support, for an init script you provide in that location. Note the path, and use the same directory for yours:

INITSMF = /opt/csw/etc/init.d/cswfoo

That's it! On a system with SMF your init script will be used as method and a manifest will be autogenerated during package installation.

Note that the SMF service name will be the same as the init script file name. For the example, it would become
svc:/network/cswfoo, unless overridden by the "In-Line tweaks" mentioned below.

If using this class manually, it's important that it be last in the list of CLASSES.

Requirements for init script

Your init script must support start and stop. Restart and refresh will be supported if present.

In-Line tweaks (optional)

In your init script you can have "magic" comments that configure the service creation or symlink creation during package installation.

These are all optional. If you do not set them, then reasonable defaults (shown in the examples below) will be chosen.

The line must start with one hash mark to be immediately followed by the variable, then whitespace to the value. Comments like in the examples below are OK but there must be whitespace after the value.

Examples of optional tweaks:

#RC_KNUM 20         # Number used for kill script symlink, e.g. K20cswfoo
#RC_SNUM 80         # Number used for start script symlink, e.g. S80cswfoo
#RC_KLEV 0,1,2,S    # Run levels that should have a kill script symlink
#RC_SLEV 3          # Run levels that should have a start script symlink
#FMRI network       # FMRI path for service (S10+), default is /network.
#                     Changing the value here, yields a generated FMRI of
#                     "svc:/somethingelse/cswfoo:default"
#AUTOENABLE yes     # If set to no will not enable service regardless of
#                     local csw.conf, use when a package needs setup before
#                     being useful, would otherwise leave service in
#                     maintenance mode
#MANIFEST /absolute/path/to/manifest   # If set, use this manifest instead
#                     of autogenerating one (default)

Note that AUTOENABLE is an override of the local csw.conf so packages that always need some setup before being useful can not be started at install. In the case of a non-SMF install rc-links will be created however, to make it easy for the user after setup. This means that, e.g., if the system is rebooted before setup has taken place the service will be started. Therefor it's recommended that service scripts make use of something similar to the example below to stop the startup in a graceful way if proper configuration is missing:

if [ ! -f "$CONFFILE" ]; then
  echo "$CONFFILE missing. Exiting."
  exit 1
fi

Enable/Disable SMF in csw.conf

The script will respect use_smf in our global csw.conf. If the user has a Solaris 10 machine where no SMF is to be used, "use_smf=no" can be added to /etc/opt/csw/csw.conf (or /opt/csw/etc/csw.conf). Otherwise it will default to use SMF on Solaris 10.

The user can also control if services will be started and enabled during install or not. Default is to start. This is how it's decided if the service is to be started or not:

1. default => start
2. if autoenable_daemons is no => don't start
3. if autoenable_yourservicename is no => don't start
4. if autoenable_yourservicename is yes => start

All cases are matched so the last true one wins.

cswcptemplates

(CSWcas-cptemplates)

This class is designed to make it easy to copy files from a standard "template" directory, to "local" filesystems such as /var/opt/csw or /etc/opt/csw.

Summary

Files placed under /opt/csw/etc/templates/YOURPKG/$SOMEPATH/file
will get copied to $(ROOT)/$SOMEPATH/file in the final stages of pkgadd, if that file does not already exist.
That same file will get removed on pkgrm, if the file has not changed.

Short cut

Files placed directly in the top level, /opt/csw/etc/templates/YOURPKG/file, will get treated as if they were
YOURPKG/csw/opt/etc/file

GAR use

Placing any file under the (….)YOURPKG directory will automatically invoke use of the class action.

Internals

Files designated in the prototype file in the pkg, similar to

f cswcptemplates /opt/csw/etc/templates/$YOURPKG/$SOMEPATH/file 0644 root bin

will get copied to

/$SOMEPATH/file

by the class action script, if and only if the file does not already exist.

On pkgrm, if the file has not been changed, it will be removed.
If it has been changed, it will be preserved under /etc/opt/csw/preserve/$YOURPKG, and will be restored on next pkgadd

Since it is presumed most templated files will be destined for /etc/opt/csw, there is a shortcut: If you skip including a directory tree, and ship the template as

f cswcptemplates /opt/csw/etc/templates/$YOURPKG/file 0644 root bin

it will get copied to /etc/opt/csw

If you do NOT use the shortcut, please note that you will also be obliged to at minimum add a directory line in your prototype,
d none /opt/csw/etc/templates/$YOURPKG/$SOMEPATH
otherwise the system wil complain about missing directories at pkgadd time.


cswcpsampleconf & cswpreserveconf

(Note: these should be used for copying files from places like /opt/csw/etc to /opt/csw/etc. For files installed on "local" filesystems such as /etc or /var, use cswcptemplates, above)

These two classes that help to install default, original versions of configuration files. It's done in such a manner that the original file gets installed with the .CSW extension, and you're free to modify the configuration file. Upon package upgrades, your changes are safe; the configuration file won't be removed and/or replaced with the default contents upon package upgrade.

/etc/opt/csw/yourprog.cnf.CSW # the original
/etc/opt/csw/yourprog.cnf     # your potentially modified version

Add this line to the GAR Makefile, note the path:

PRESERVECONF = /etc/opt/csw/yourprog.cnf

or:

SAMPLECONF = /etc/opt/csw/yourprog.cnf

cswcpsampleconf

cswcpsampleconf is a very simplified method, compared to cswpresesrveconf. On pkgadd of your package, it will copy yourprog.cnf.CSW to yourprog.cnf, if a .cnf file does not already exist.

Additionally, it will automatically remove yourprog.cnf on pkgrm, if and only if it has not been changed from the shipped one.

cswpreserveconf

The fancier version, cswpreserveconf, acts similarly to cswcpsampleconf. However, if an altered version of the configuration file is present on pkgrm, it will "preserve" it, in a special "preserve" directory. The advantage of this mechanism, is that it allows for a tool such as pkg-get, to potentially support a "purge" option, that will truly remove a package, without leaving modified config files behind.

In theory, a package maintainer should always use the cswpreserveconf class. However, the simpler one is also provided. If nothing else, it serves as the simplest example to base other class action scripts on.

You don't have to do anything else to use these scripts, unless you explicitly call out a CLASSES definition in your pkginfo file (for example, if you are going to use the cswinitsmf class as mentioned above). If you do, then you should add the class name, to the CLASSES line, somewhere other than the end.

GAR Gotchas

There's a gotcha: A file marked in GAR as handled by csw{sample,preserve}conf will be renamed to ${file}.CSW, and the updated name, not the original one, will appear in the prototype. As a consequence, if you want to list your files in PKGFILES_CSWfoo, you need to use the .CSW name.

PRESERVECONF = /etc/opt/csw/yourprog.cnf
PKGFILES_CSWfoo = /etc/opt/csw/yourprog.cnf.CSW

cswmigrateconf

Migrates configuration files and directories, by default from /opt/csw/etc to /etc/opt/csw.

In your GAR Makefile, write:

MIGRATE_FILES = odbc.ini odbcinst.ini ODBCDataSources
MIGRATE_SOURCE_DIR = /etc/opt/csw/mysubdir            (sets default)
MIGRATE_SOURCE_DIR_odbc_ini = /opt/csw/etc/mysubdir   (overrides the default on a per-file-basis)
MIGRATE_DEST_DIR_odbc_ini = /etc/opt/csw/anotherplace

When you generate multiple packages from one GAR Makefile, there may be one cswmigrateconf file per package. As the config-files will be distributed among the packages the cswmigrateconf needs to be specific for all packages - if you use the default the file will ne the same in all packages which is considered an error.

MIGRATE_FILES_CSWmypkg1 = odbc.ini
MIGRATE_FILES_CSWmypkg2 = odbcinst.ini ODBCDataSources
MIGRATE_SOURCE_DIR = /etc/opt/csw/mysubdir            (sets default for all packages, can be overriden per-package
MIGRATE_SOURCE_DIR_CSWmypkg1 = /opt/csw/etc/mysubdir  (sets package-specific sourcedir)
MIGRATE_DEST_DIR_odbc_ini = /etc/opt/csw/anotherplace  (allows overriding per-file automatically distributed in the package-specific file)

cswusergroup

This script creates users and groups for you. For many the only need for pre/post scripts are to create a user to run a daemon. With this script there's no need to hack your own scripts, instead it will be done in a consistent way.

You provide a configuration file to instruct the script which users and/or groups to create. Multiple files are allowed (though probably not necessary and not recommended) and the files can have multiple lines in them in the following format (passwd style):

user:group:gcos:dir:shell:create_dir:remove_dir:no_login

Examples (note that the comments are not allowed):

clamav:clamav:Clam Antivirus:/export/home/clamav:/bin/false:m:r:NP   # all eight fields specified
bind:bind::::::                    # only user and group specified
:yourgroup::::::                   # if you just want a group created
youruser:::::::                    # if you just want a user created

The last three fields are equivalent to "useradd -m", "userdel -r" and "passwd -N" respectively. A blank field means that option will not be used, any non-blank value triggers it.

The file(s) can be placed anywhere and the name doesn't matter as long as its class in the prototype is cswusergroup but the recommended path and name is:

/etc/opt/csw/pkg/<pkgname>/cswusergroup

The script also supports removing the users and groups when the package is removed but since this can cause problems it's not the default. One such case is if you install MySQL 4 and it creates a mysql user, then you install MySQL 5 and it also uses the mysql user. You now remove MySQL 4 and if it would remove the mysql user you would be in trouble.

To enable removing users and groups you need to set usergroup_remove to yes in csw.conf. In most cases it would be ok to do so but it can not be the default.

usergroup_remove=yes

Add this line to the Makefile, it's the configuration file, note the path:

USERGROUP = /etc/opt/csw/pkg/CSWfoo/cswusergroup

Files/dirs owned by created users/groups

Note that if you want to install files/directories with these users/groups as owners you need to make sure that the cswusergroup is installed before the class containing these files/directories, otherwise the user/group will not exist yet. Assign a new class for these files/directories and put it after cswusergroup, see below:

CLASSES=none cswusergroup ugfiles

This creates the user/groups in the cswusergroup class before adding the files owned by these user/groups in the ugfiles class.

You cannot leave them in the "none" class since pkgadd always installs that class first (and then your users/groups don't exist).

Example prototype entry for files/dirs owned by created user:

f ugfiles /var/opt/csw/foo/bar 0644 clamav clamav

Recent test-wise addition: Configurable UID/GID ranges

This is still in testing and hasn't been yet released: To make cswusergroup account/group creations more determinable for sysadmins, recent additions to i.cswusergroup support configurable UID/GID ranges for created accounts/group via csw.conf and the following set of variables (chosen to have the same prefix as the already existing usergroup_remove variable for consistency purposed).

usergroup_uid_low
usergroup_uid_high
usergroup_gid_low
usergroup_gid_high

Thereby, sysadmins can control in which ID range users and groups are created by CSW packages (default for useradd/groupadd is "highest UID/GID + 1").

  • csw.conf is read from /opt/csw/etc/csw.conf and /etc/opt/csw/csw.conf (the one from /etc takes precedence)
  • If not set, i.cswusergroup defaults to a range of 100-999 for users and groups.
  • To determine already existing accounts within a configured range, i.cswusergroup relies on information retrieved via getent instead of simply parsing /etc/passwd and /etc/group. This way, other account backends configured via nsswitch will also be considered.

These additions do not support a way to specify fixed UID/GIDs for specific users, but only a range from which UID/GIDs are taken.

cswpycompile

This class takes care of creating/removing python's normal and optimized bytecode files. To use this class action script in GAR, simply set the variable PYCOMPILE to a non-null value. Eg:

PYCOMPILE = 1

For non-GAR use, ensure that all files you wish to see byte-compiled by python at installation time have the class 'cswpycompile' set.

cswinetd

This class allows you to supply a file that contains a line that is valid for addition to the traditional inetd.conf file. At installation time, cswinetdconf will either add it to /etc/inetd.conf (solaris 8/9) or install it as a smf managed inetd service (solaris 10+).

Example file:

git     stream  tcp     nowait  gitosis /opt/csw/bin/git        git daemon --base-path=/var/opt/csw/gitosis/repositories --syslog --inetd --verbose

To use this class from GAR, set the variable INETDCONF to a list of files, each of which contains a valid inetd.conf entry. Example:

INETDCONF = /etc/opt/csw/pkg/CSWgitosis/inetd.conf

This class will honour the same autoenable_* variables as cswinitsmf.

For non-GAR use, set the class for the desired files to cswinetd. If using this class manually, it's important that it be last in the list of SPKG_CLASSES (or right before cswinitsmf). This is to ensure that any services started have the binaries in place.

cswetcservices

This class allows for entries to be added to /etc/services. It is (currently) a one-way operation. At package removal time, the entries remain in /etc/services.

To use this class from GAR, set ETCSERVICES to (one or more) files, each of which can contain multiple lines of the form:

<servicename><space><service port>/<service protocol>

Example from /etc/opt/csw/pkg/CSWgit/services:

git 9418/tcp
git 9418/udp

Example GAR use:

ETCSERVICES = /etc/opt/csw/pkg/CSWgit/services

cswtexinfo

This class can be used to automatically insert and remove entries for texinfo documents to the texinfo directory

/opt/csw/share/info/dir

The class action script has been designed to work with the CSWtexinfo package. If that package is not installed a note is printed that a texinfo document was in the package and that it would be helpful to install CSWtexinfo. If CSWtexinfo is installed later all texinfo pages are registered automatically. If you are using GAR, files in /opt/csw/share/info with ".info" in their name will be automatically assigned to the cswtexinfo class and a dependency to CSWcswclassutils will be added. Please note that a dependency to CSWtexinfo is not added automatically, nor does one need to be.

cswcrontab

This class can be used to add and remove entries for crontabs of specific users. The files corresponding to this class containing the lines should be put in

/etc/opt/csw/pkg/<pkgname>/crontabs/<user>

Each line from the crontabs are taken and appended unconditionally to the crontab for the respective user. Each line is suffixed with '% Added by CSW<pkg>'. This suffix string triggers removal. Entries will not be preserved during package updates.

Example GAR use:

CRONTABS = /etc/opt/csw/pkg/CSWfoo/crontabs/root

cswpostmsg

This class can be used to display a message at the end of a package install. With all the other classes now doing common tasks one of the few remaining common things to use a postinstall script for is to display a message and there's no need to get the superuser prompt for that with this class. Just point it at the file containing your message and it will be displayed.

Example GAR use:

POSTMSG = /opt/csw/share/doc/foo/README.CSW

Please note that POSTMSG contains real filenames and not regular expressions. Additionally, it can contain multiple filenames which results in all of the files be printed. The order in which the files are printed is not necessarily the order in which they were specified to POSTMSG.

cswetcshells

Files in this class are added to /etc/shells at install time. It will create /etc/shells with a default set of shells (see man shells) if it doesn't exist. At package removal time, the shell is removed from /etc/shells. This is intended for packages like CSWbash, CSWzsh, etc.

ETCSHELLS = /opt/csw/bin/zsh /opt/csw/bin/someothershell

cswsslcert

Files in this class will trigger the generation of an ssl certificate and private key. At the discretion of the admin, a CSR may be generated instead of a self signed certificate. A file with cswsslcert as the class and delivered to /etc/opt/csw/foo/server will see the creation of /etc/opt/csw/foo/server.crt and /etc/opt/csw/foo/server.key. After initial generation they will never be altered again. The intent of this class is to allow services to enable the encrypted service by default, even if only with a self-signed certificate (encryption without authentication). To designate these files in your GAR recipe, use the SSLCERT variable.

SSLCERT = /etc/opt/csw/foo/server

This CAS uses csw.conf to control the certificate generation. (csw.conf is sourced by the CAS script so any valid /bin/sh syntax is allowed.) This script can set any (or all) of the following variables for use in the certificate generation (defaults will generate certificates for "The Hobbit" :) ):

SSL_COUNTRY
SSL_STATE
SSL_CITY
SSL_ORG
SSL_OU
SSL_HOST
SSL_EMAIL

The values of these variables are provided to openssl to answer the questions asked during certificate creation. In addition to controlling the certificate generation values, an admin may set any of the following in csw.conf to control additional aspects of certificate generation:

cas_ssl_bits #the number of bits used in the certificate (default: 2048)
cas_ssl_days # the number of days the certificate should be valid for (default: 365)
cas_ssl_csr # any value for this variable will see a CSR generated instead of a self-signed certificate...this may impact service startup!

cswtexhash

This class takes care of keeping the Tex packages index in sync with all TeX packages installed. It adds index entries on package installation and removes them on package removal.

cswdictconf

This class takes care of keeping the dictionaries used by the DICT server in sync with all the dictionary packages installed. It add entries in the servers configuration on package installation and removes them on package removal.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License