wpkg --build | -b

Introduction

The --build command is used to create new source or binary packages that can later be installed on a target system. There are many different types of packages that you can create. This page does not describe those in detail because that's enough for a different topic! Here we talk about the three ways you can use the --build command to create packages: Method 1. One Package, Method 2. Multi-Packages, and Method 3. Source Package & Generate.

Method 1. One Package (WPKG Folder)

The default way to build a package with wpkg is similar to the Debian packaging mechanism which builds one package at a time. This is generally the safest as it ensures that you don't mix unwanted files improperly.

Command Line

The command line for this method is very simple and comes down to:

wpkg --build <path to directory to package>

This command creates a binary .deb package in the current directory (see also the --output-dir option) using the package data defined under <path to directory to package>. The directory is expected to include sub-directories with the different files to be installed on the target (i.e. usr/bin/my-app.exe and usr/share/doc/my-app/copyright) ordered within those directories exactly as they will appear in the root of the target machine.

Directory Structure

In the directory to package, there should be a WPKG directory (wpkg also checks for DEBIAN and ignores the case since MS-Windows has a disk system that is case insensitive.) The WPKG directory includes the user defined control files. These are files that are inserted in the control tarball saved inside the package. The most important file is called "control". Assuming you created a directory named my-app, you could create the following directory tree and files:

my-app/
my-app/WPKG
my-app/WPKG/control
my-app/usr
my-app/usr/bin
my-app/usr/bin/my-app
my-app/usr/share
my-app/usr/share/doc
my-app/usr/share/doc/my-app
my-app/usr/share/doc/my-app/copyright
my-app/usr/share/doc/my-app/readme.txt

This tree shows you all the files that will be installed (in this example, all are under /usr) and the one control file named "control"1.

Control File

The control file is a list of fields and values. The order doesn't matter. To the minimum the following fields must be defined:

Package — the name of the package (i.e. Package: wpkg)
Version — the version of the project in this package
Architecture — for binary packages that require a specific architecture, the name of the required target architecture to install this package
Maintainer — the name and email address of the project maintainers
Description — at least a brief description of the project

For more information about all the fields supported in control files see the Control File page.

The name of the output package is generally built from this information as:

Package <underscore> Version <underscore> Architecture <dot> deb

For example, for a package with these values:

Package: foo
Version: 2:03.7b-4
Architecture: win64

The output package name is:

foo_2:3.7b-4_mswindows-amd64

The packager may canonicalize the version and the architecture values to ensure better forward and backward compatibility.

Method 2. Multi-Packages (control.info file)

wpkg supports the control.info syntax as introduced by Mac Port. This allows you to define all the binary packages of a project in a single file. This method is also the one used when building a source package and then generating the binary packages from the source package.

TO BE NOTED

The Debian approach to this problem was to actually allow someone to write a control file with empty lines. Each empty line separate a package definition from the other. The first definition defines the main package and all those fields are defaults for all the other packages. The other packages may define additional fields and replace fields that were defined in the first list of fields. Each entry must have a Package field with the name of that package (i.e. wpkg, wpkg-dev, wpkg-doc.) We may implement support for this scheme, however, there is a drawback with it: you cannot remove a field that was defined in the main package from any sub-packages. The .info file format, however, allows you to define special fields on a per sub-package basis.

Command Line

The command line is very similar when creating multiple packages at once:

wpkg --build <path to control.info file>/control.info <path to data directories>

wpkg detects this mode if the first filename is a regular file (opposed to a directory as in Method 1.) Now wpkg loads the information from the control.info file and generates the necessary WPKG/control files under each folder as defined in the <path to data directories> directory. These control files are trimmed down as defined in the Control File documentation page.

Directory Structure

Very similar to the previous example except that the structure includes one extra sub-directory level and the WPKG directory is created under those sub-directories. Note that it is customary to place the control.info file in the same directory, although it could be somewhere else (in case of a source package, place it under wpkg/control.info instead.) There is an example with runtime files and documentation files:

my-app/
my-app/control.info
my-app/runtime/usr
my-app/runtime/usr/bin
my-app/runtime/usr/bin/my-app
my-app/runtime/usr/share
my-app/runtime/usr/share/doc
my-app/runtime/usr/share/doc/my-app
my-app/runtime/usr/share/doc/my-app/copyright
my-app/runtime/usr/share/doc/my-app/readme.txt
my-app/doc/usr
my-app/doc/usr/share
my-app/doc/usr/share/doc
my-app/doc/usr/share/doc/my-app
my-app/doc/usr/share/doc/my-app/html
my-app/doc/usr/share/doc/my-app/html/index.html

The WPKG directories are not shown because in most cases they are created by wpkg as it builds the packages. Their content is a copy of the control.info file with only the fields that match that specific sub-directory (the syntax is: field-name/<sub-directory>: value). You'd create them only if other files are necessary in the control archive (i.e. a postinst script, perhaps. See more details below.)

Control Info File

The control.info is very similar to a control file, only it must have a Sub-Packages field that lists the different components. For example, to match the directory structure we have just shown you, we would want the following:

Sub-Packages: runtime*, doc

Here the control.info file asks to build two packages: runtime and doc. In this case, the runtime parameter is followed by an asterisk, which means: do not append that name to the package name. So the runtime package is the package name, version, and architecture. Whereas the documentation package is package name dash doc, version, and architecture; note, however, that the documentation is defined with an architecture of "all" so that part is removed from the final filanem:

foo_2:3.7b-4_mswidows-amd64
foo-doc_2:3.7b-4

Note that if you have a Sub-Packages field in a control file, then it is considered to be a .info file.

Like any control files, the minimum fields have to all be defined (Package, Version, Architecture, Maintainer, Description.) Different fields, such as the Package field, can be defined with a sub-package definition in which case the default is ignored:

Package/doc: foo-documentation

This definition would force the documentation package to be named using foo-documentation instead of the default foo-doc.

In most cases this feature is used to define the different architectures of the different packages as quite often you use this capability to create a runtime package and a documentation package:

Architecture/runtime: mswindows-i386
Architecture/doc: all

The Control File page defines all the fields that accept a sub-package definition. Some fields do not accept such because they must be equal throughout all the packages. For example the Version field cannot change between the different packages of a project.

Other Control Files

In many cases projects do not require any additional control files, only the main control is enough. In that case they are generated from the control.info file so you are set.

However, some projects require the use of installation scripts (preinst, postinst) or have configuration files. In that case, these files have to be added in the WPKG folder in the sub-package directory. For example our foo package could have a foo.conf file defined like this:

my-app/runtime/WPKG/conffiles
my-app/runtime/etc
my-app/runtime/etc/foo
my-app/runtime/etc/foo/foo.conf

The conffiles includes one line: /etc/foo/foo.conf

wpkg accepts such files as they are defined. The WPKG directory and WPKG/control file are created automatically, anything else is not (outside of the md5sums, but that's a bit different since that file doesn't get saved in your WPKG directory.)

Method 3. Source Package & Generate

To make full use of the build capability of wpkg, one wants to create source packages and then generate binary packages from those source packages. The idea is pretty simple: if you have a source package for each project, then the build system can very simply install source packages and run the necessary scripts to build your entire environment. As a result, whether you have 1 package or 10,000, they all can be rebuilt with one command repeated for each source package, even if certain packages are in conflict with each others.

When creating source packages, wpkg looks for information to fill in your control file from several different files. For example, it makes use of the ChangeLog file to define the package name, version, urgency, maintainer, what bugs were closed, and the actual log of changes. This process and the building of binary packages is explained in detailed in A Build System with wpkg.

Output Filename

By default wpkg knows exactly what the output filename has to be and it generates is exactly as it will be expected by the other features available in wpkg. In other words, it should be left alone.

There are cases, however, when it may be practical to follow your own naming convention and thus have a way to indicate the name of the output package (i.e. "a.deb"). This is done with the --output-filename option.

Output Directory

When not specified, the resulting Debian package is saved in your current directory. To change the default, you may use the --output-dir option. This option is used to define the name of the directory where the .deb file is saved. Note that you cannot specify the output filename of the package with this option. Instead you want to use the --output-filename option for that purpose.

Note that if you are making use of a repository, then you want to use the --output-repository-dir instead. It has priority over the --output-dir option and is used when the Distribution field of that package is also defined. In that case your build system can directly save the resulting packages in your repository.

Limitation to the Directory Tree

When installing your files to put in a binary package, none of them can appear in the administration area of the target. Unfortunately, the build command cannot know in advance where the administration directory is going to be. Although by default it is set to /var/lib/wpkg, it is possible to change the default with the --admindir option. The administration directory could even be specified as being outside of the installation target directory. So the build does not generate any errors at this point.

When installing a package, though, the path of each file is checked against the administration directory. If a file would end up in the administration directory when unpacking a package, then the installation process stops with an unrecoverable error. There is no options to circumvent the situation. The only way to always avoid this problem is to defined the administration directory outside of the installation directory (which is not the default.)

Other Details

The control files are NOT handled the same way in wpkg than they are in dpkg. When specialized fields are found in the control file, dpkg keeps them if it corresponds to the type of package being created (i.e. binary, development, source can be distinguised with a letter: B, D, S.) All other such variables get removed. Instead we keep all the fields at this time (obviously  in case of the .info file, fields with a specific sub-package name will only appear in that package.) There are exceptions though. For example the Extra-Size field is used but removed from the final control file.

The memory file system we use to handle package files imposes a maximum size. In 0.7.0 the limit was set to 128Mb which is relatively small. The current limit is 1Gb instead. We may look into offering a command line option that can detect a package that's becoming too large and as a consequence should fail its build.

The memory files are broken up in 64Kb chunks. The smaller allocation is therefore 64Kb. Also the system does not deallocate that memory until wpkg exists. If necessary, I will look into implementing a way to cache really large files which the block implementation will ease greatly.

• 1. I know that's a bit confusing because there are in effect two control files: the file named control with field / value pairs and the control.tar.gz file that includes all the control files found under the WPKG directory.