Configuration Scripts and Processes

Introduction

When creating a package, the project maintainer is given the option to add scripts to the package. Those scripts are run by wpkg in five different situations as presented below.

Tools that are not packaged with wpkg are also given a way to be notified of installations, configurations, removals, and deconfigurations by installing global hooks that are run when those events occur giving a chance to your external tools to react accordingly.

All the script are placed in the WPKG directory along the control file. The --build process automatically picks up those files when your package is created.

Idempotency of Scripts

As with dpkg, the scripts need to be idempotent. This means running the script once or 100 times gives you the exact same result. In other words, if a script creates a directory as a pre installation step, then it cannot fail because that directory exists.

set -e
mkdir /var/lib/wpkg

Controlling Terminal

It should be assumed that the user won't run wpkg in a terminal or some other environment that gives wpkg a way to interact with the user. Even under MS-Windows, opening a window is likely to work in all case, only if you open a window on a remote platform, it is not likely to do anything good to the adminstrator. In other words, all scripts should run without having to ever interact with the user.

Although this constrain will not be removed with time, we will amoliorate the use of the --interactive option to better allow such interactions cross computers.

1. Validation (Validate)

As the final step of the installation or removal validation processes we run the validate script of each package. If any one script generates an error, it is considered that the validation process failed and the process ends there. It is VERY IMPORTANT that the validation processes do NOT change the target in any way other than temporary files as required. These scripts must simply test the validity of the target for the package it was written for and then exit with 1 if not considered valid for that package.

IMPORTANT NOTE, LIMITATION

It is important to understand that wpkg has one limitation with this test. If you attempt to install a web server, say Apache, and your target is already running lighthttpd, then you can detect that because port 80 is already in use and Apache wants that port to boot up by default (you could still run both packages together, only one of them should not make use of port 80). The problem is when you install both packages together. If they do not know about each others, then no Conflicts or Breaks will be defined. This means both are getting to this validation process which will fail for both processes since port 80 is still available (assuming you did not install any other web server yet, of course.) This means the installation process continues and attempts to install and start both servers. The process is expected to fail when the second server postinst runs since port 80 will be locked by the first server.

Note that like with dpkg, you may move such a test in your preinst script. There you will catch the fact that port 80 is not available when attempting to unpack the second package. That would happen before the package installation actually starts and stops then instead of breaking when attempting to start the actual server.

Names:

• Unix -- validate or validate.sh
• MS-Windows -- validate.bat

2. Before Installation (Preparation)

The preinst script is run before the installation occurs. This can be viewed as the final validation step of a package, although the order in which things happens, the preinst script of one package is run right before that package gets unpacked and then configured. Then the following package is dealt with. Validations, on the other hand, are all processed at once and nothing gets installed if any validation fails.

This preparation step is still quite useful to make sure that a package installed just before this one, in the same batch, would not prevent the installation of this package.

Note that it is not expected that this script would install anything on the target system. If it doesn, make sure that the postrm removes such when run.

Names:

• Unix -- preinst or preinst.sh
• MS-Windows -- preinst.bat

3. After Installation (Configuration)

Once wpkg validated all packages, successfully ran the preinst script, and could unpacked all the files of a package successfully, the system runs the configuration command. This means renaming any configuration file to the official name (i.e. remove the .wpkg-new or .wpkg-user extension) and then run the postinst script.

The post installation script is also called the configuration script as it is used to dynamically finalize the configuration step of a package. When dealing with a server, this step also starts the server if it is considered safe to do so immediately. This is one of the most important feature for servers on a Unix system. This script will generally run:

set -e
update-rc.d <new-service-name> enable
service <new-service-name> start

This is quite important if you install mutliple servers at the same time and server A expects server B to already be installed and running to itself get configured and thus started too. Although the Pre-Depends field can be used to prevent the installation of server B at the same time as server A and you to check in server B preinst script whether A is running or not.

Although any package can make use of a configuration script, most of the time it is primarily used by servers.

Note that initialization required by a tool should most certainly be taken cared of by the tool itself so it can run as a standalone tool without the need to run a script while installing said tool with wpkg. For example, PostgreSQL is capable to install a database cluster by running a PostgreSQL administration tool. However, the postinst could run the tool to create a default environment for the tool to function immediately. For example, the tripwire tool can be run once to initialize its database at the time it gets installed:

tripwire -m i -s

This allows the system to run tripwire once a day using cron and get the expected effect. When removing this package, it is expected that its postrm script will delete the database and any other such file.

Names:

• Unix -- postinst or postinst.sh
• MS-Windows -- postinst.bat

4. Before Removal (Deconfiguration)

When wpkg is asked to remove a package, it must deconfigure it first. This is an important step because you cannot hope to safely delete a server from your system without first bringing it down. The prerm script is expected to be used for that purpose.

set -e
service <service-name> stop
update-rc.d <service-name> disable

Some other types of packages may also need a deconfiguration script, although it is less likely to be necessary. If your project creates a lot of data that should also be deleted, the right place is probably the post removal script (postrm). On the other hand, if a lot of that data is considered temporary data, then it could be removed by this step to save disk space (i.e disk caches). There is an example, which as you can notice exits with zero (no error) to make sure the script does not prevent the removal process:

rm -rf /tmp/<package-name>-cache
exit 0

The removal process first calls the deconfigure command, this is where the prerm is called. You can actually just deconfigure a package with the use of the --deconfigure command. In this case the package files remain unpacked.

Names

• Unix -- prerm or prerm.sh
• MS-Windows -- prerm.bat

5. After Removal (Cleanup)

Once wpkg deconfigured a package and removed all the files that were unpacked, it is ready to run your postrm script. This script is expected to do final cleanups. For example, if your project creates a database, then this script is expected to delete all those files. Whether the script returns an error is not important in the removal process (i.e. the package will still be considered removed) therefore, I suggest you do not use the 'set -e' or equivalent on your system so most functions in your scripts have a chance to run before the script ends.

rm -rf /var/lib/<package-name>

Note that you should not remove files that the user created and saved for archival (i.e. the user installed The Gimp, did some work on images, then wants to remove The Gimp package. That should not delete the photos the user worked on with The Gimp!) You may, however, offer a user to delete such files by requesting the user to chose whether to keep those files or not, especially if these are very large or if only that package tools are able to handle such files.

Names:

• Unix -- postrm or postrm.sh
• MS-Windows -- postrm.bat

Additional Hooks

Since version 0.9.0 wpkg offers a way to add hooks to a target. These hooks can be added using wpkg and are used by packages that wan to be notified about changes to a target (at least in term of wpkg installing, upgrading, removing, configuring, and deconfiguring packages.)

There are two levels: hooks that run once per wpkg run (system hooks); and hooks that run once per package being handled (global hooks).

The same set of five scripts apply: validate, preinst, postinst, prerm, and postrm. The validation hooks are useful to prevent the installation of certain packages no matter what. The pre- and post- hooks are particularly useful to create signals and thus inform other applications that something is happening or just happened.

The system validation hooks are run before all the other scripts, but after the internal validation processes ran. The system pre-hooks run before all other pre-scripts. If a pre-hook fails, then the installation or removal process ends immediately.

The global validation hooks are run after the packages validation hook (for technical reasons as the list of packages is generated with that first step). Since the validation process does not modify the target system, there should be no side effects from this order.

The global pre-hooks are run before the package hooks, if they return an error, then the installation/removal do not happen.

The global post-hooks are run after the package hooks. Errors returned by post-hooks are ignored (although the fact will be logged.)

To manage hooks, you can define them in your packages (preferable) or use the wpkg --add-hooks, --remove-hooks, and --list-hooks commands.

Running Scripts and Recovery Process

When running one of the pre- or post- scripts (i.e. not the validation scripts) the packager checks the return value of the script and on an error (i.e. exit value other than 0) may decided to run additional scripts to correct the issue. Most of the processes have a recovery process run in case of error. The following UML diagrams show each one of those processes.

Legend

Instead of writting where each script comes from, we used colors. This makes it easier (unless you are color blind...) The most complicated case is the upgrade which deals with scripts from the new version of the package (Orange), the old version of the package (Blue), and global hooks (Cyan).

• Black circles show the start and end point of each process. Some processes may have multiple start point.
• Green boxes include the title of the algorithm.
• Orange boxes show scripts run from the new package being installed.
• Blue boxes show scripts run from the old package being upgraded or removed.
• Cyan boxes show global scripts run from the package being worked on and the list of installed hooks.
• White rectangle with text represent a step run by wpkg, for example: install.
• White diamons represent a condition where wpkg makes a choice based on either the current status of a package or the result of the last action. Which branch to take is defined by a few words.

1. Installation Validation

The installation process includes a very large number of validations that are run before the installation steps proceed. This validation is run for all installation commands (--install, --unpack, --configure, --reconfigure) and it may vary slightly for each command.

If any one of these validation steps fail, then the installation is aborted at that time and no additional scripts are called.

The validation scripts are expected to not cause any side effects so there is no need to run anything to cancel running them.

2. Unpack, New Installation, Re-Installation, Configuration

The New Installation scripts shown here are used by the commands shown below. The commands expects a new package or the same package (same name, version, architecture.)

a) wpkg --unpack

Start Unpack, then skip the configuration scripts. This is called an Unpack. Note that if the package was previously unpacked, then it is called a Re-Installation instead. Note that using --unpack on a package that was installed does not have the same effect as the --deconfigure: it does not rename the configuration file with the .wpkg-user extension. We can see this here since no [ deconfigure ] step is shown (see removal scripts below.)

Note that if the version of the package is different, then we are doing an upgrade instead.

b) wpkg --install

Start Install, then configuration scripts. If this is the first time the package is installed this is called a New Installation. If the same version of the package was already installed and --skip-same-version was not used, this is called a Re-Installation and the scripts are called with <old-version>.

Note that if the version is different, then it is an upgrade so these installation scripts are not called the same way (see below).

c) wpkg --configure

Start Configuration up to the end. This is called the Configuration step. Note that you can only configure packages that were previously unpakced or deconfigured. This step is automatically run when the user does an installation directly.

3. Upgrade, Downgrade, Deconfigure a Package

The Upgrade Installation algorithm is used whenever a package is unpacked or installed and that package was already unpacked or installed before, but the one you are unpacking or installing now has a different version (when the newly installed package has a smaller version, it is called a Downgrade Installation, but the algorithm remains the same and the scripts are still called with upgrade as a parameter.)

Most of the script are called in the same way as for new installation, only the first command line option is upgrade instead of install.

Note that in case of an upgrade it is expected that the prerm upgrade <new-version> script will stop a server if this package sets one up. The final postinst configure <new-version> script is expected to restart the new version of the server. In contrast, the preinst upgrade <old-version> and postrm upgrade <new-version> scripts are validation and clean up steps that do not directly deal with services.

The commands that use this algorithm are:

a) wpkg --install

The install command, when re-installaling the same version does not make use of the upgrade algorithm. Instead, it uses the New Installation algorithm. This algorithm is used only if the version of the package changes (upward or downward).

b) wpkg --unpack

The unpack function exists early, skipping the configuration steps. However, it will have run the deconfiguration scripts first. So this is similar to a --deconfigure command, except that the configuration files do not get renamed.

c) wpkg --reconfigure

Note that the algorithm does not properly show that the reconfigure command does not run the preinst upgrade <old-version> script, mainly because of space issues. Also, the unpack step does not unpack any files other than the configuration files. In other words, it merely stops a server, extract the original configuration files, and restart the server. For that reason it only works against already fully installed packages.

4. Removal Validation

The different remove commands (--remove, --purge, --deconfigure) require a set of validations which end by running validation scripts as defined by the projects being removed and by global hooks.

These scripts are run as presented here. First we run the validation script of each one of the packages getting removed, then we run the hooks with the list of all the package names getting removed with remove as the first parameter of the scripts.

Note that at this time all the validations are called with remove as the first parameter of the script. Later we may change that and support the name of the command being performed: remove, purge, or deconfigure.

If any one scripts returns with an error, the whole removal is canceled.

5. Removal, Purge, Deconfiguration

The Remove / Purge algorithm runs the scripts that deconfigure the package, remove is files, and in case of a purge, deletes all its configuration files.

The script can also delete temporary and cached files.

The commands that support this algorithms are:

a) wpkg --remove

Deconfigure and then remove a list of packages.

The configuration files are kept back so if the user later re-installs the package, the project settings will still be in place.

b) wpkg --purge

The purge command first calls the prerm script which is expected to stop services if that project has any, it then removes all the configuration files.

Note that the prerm purge does not usually mean much. In most cases you want to write a postrm purge script to finish up the cleaning up of the project.

c) wpkg --deconfigure

The deconfigure command is a special case in wpkg. It does not exist in dpkg.

The function renames the configuration files so the project cannot see them. It can then decide that by default, without a configuration, it cannot run, which is the expected behavior for many servers.

Note that the prerm deconfigure script is expected to stop the server if still running.