<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="chap-projects">
<title>Creating and Managing Projects</title>
<para>
Once Hydra is installed and running, the next step is to add
projects to the build farm. We follow the example of the <link
xlink:href="http://nixos.org/patchelf.html">Patchelf
project</link>, a software tool written in C and using the GNU
Build System (GNU Autoconf and GNU Automake).
</para>
<para>
Log in to the web interface of your Hydra installation using the
user name and password you inserted in the database (by default,
Hydra's web server listens on <link
xlink:href="http://localhost:3000/"><literal>localhost:3000</literal></link>).
Then follow the "Create Project" link to create a new project.
</para>
<section>
<title>Project Information</title>
<para>
A project definition consists of some general information and a
set of job sets. The general information identifies a project,
its owner, and current state of activity.
Here's what we fill in for the patchelf project:
<screen>
Identifier: patchelf
</screen>
The <emphasis>identifier</emphasis> is the identity of the
project. It is used in URLs and in the names of build results.
</para>
<para>
The identifier should be a unique name (it is the primary
database key for the project table in the database). If you try
to create a project with an already existing identifier you'd
get an error message such as:
<screen>
I'm very sorry, but an error occurred:
DBIx::Class::ResultSet::create(): DBI Exception: DBD::SQLite::st execute failed: column name is not unique(19) at dbdimp.c line 402
</screen>
So try to create the project after entering just the general
information to figure out if you have chosen a unique name.
Job sets can be added once the project has been created.
<screen>
Display name: Patchelf
</screen>
The <emphasis>display name</emphasis> is used in menus.
<screen>
Description: A tool for modifying ELF binaries
</screen>
The <emphasis>description</emphasis> is used as short
documentation of the nature of the project.
<screen>
Owner: eelco
</screen>
The <emphasis>owner</emphasis> of a project can create and edit
job sets.
<screen>
Enabled: Yes
</screen>
Only if the project is <emphasis>enabled</emphasis> are builds
performed.
</para>
<para>
Once created there should be an entry for the project in the
sidebar. Go to the project page for the <link
xlink:href="http://localhost:3000/project/patchelf">Patchelf</link>
project.
</para>
</section>
<section>
<title>Job Sets</title>
<para>
A project can consist of multiple <emphasis>job sets</emphasis>
(hereafter <emphasis>jobsets</emphasis>), separate tasks that
can be built separately, but may depend on each other (without
cyclic dependencies, of course). Go to the <link
xlink:href="http://localhost:3000/project/patchelf/edit">Edit</link>
page of the Patchelf project and "Add a new jobset" by providing
the following "Information":
<screen>
Identifier: trunk
Description: Trunk
Nix expression: release.nix in input patchelfSrc
</screen>
This states that in order to build the <literal>trunk</literal>
jobset, the Nix expression in the file
<filename>release.nix</filename>, which can be obtained from
input <literal>patchelfSrc</literal>, should be
evaluated. (We'll have a look at
<filename>release.nix</filename> later.)
</para>
<para>
To realize a job we probably need a number of inputs, which can
be declared in the table below. As many inputs as required can
be added. For patchelf we declare the following inputs.
<screen>
patchelfSrc
'Subversion checkout' https://svn.nixos.org/repos/nix/patchelf/trunk
nixpkgs 'Subversion checkout' https://svn.nixos.org/repos/nix/nixpkgs/trunk
officialRelease Boolean false
system String value "i686-linux"
</screen>
</para>
</section>
<section>
<title>Release Set</title>
<!-- TODO -->
there must be one primary job
check the radio button of exactly one job
https://svn.nixos.org/repos/nix/nixpkgs/trunk
</section>
<section>
<title>Building Jobs</title>
</section>
<section>
<title>Build Recipes</title>
<para>
Build jobs and <emphasis>build recipes</emphasis> for a jobset are
specified in a text file written in the <link
xlink:href="http://nixos.org/nix/">Nix language</link>. The
recipe is actually called a <emphasis>Nix expression</emphasis> in
Nix parlance. By convention this file is often called
<filename>release.nix</filename>.
</para>
<para>
The <filename>release.nix</filename> file is typically kept under
version control, and the repository that contains it one of the
build inputs of the corresponding–often called
<literal>hydraConfig</literal> by convention. The repository for
that file and the actual file name are specified on the web
interface of Hydra under the <literal>Setup</literal> tab of the
jobset's overview page, under the <literal>Nix
expression</literal> heading. See, for example, the <link
xlink:href="http://hydra.nixos.org/jobset/patchelf/trunk">jobset
overview page</link> of the PatchELF project, and <link
xlink:href="https://svn.nixos.org/repos/nix/patchelf/trunk/release.nix">
the corresponding Nix file</link>.
</para>
<para>
Knowledge of the Nix language is recommended, but the example
below should already give a good idea of how it works:
</para>
<example xml:id='ex-hello'>
<title><filename>release.nix</filename> file for GNU Hello</title>
<programlisting>
{ nixpkgs }: <co xml:id='ex-hello-co-nixpkgs' />
let
pkgs = import nixpkgs {}; <co xml:id='ex-hello-co-import-nixpkgs' />
jobs = rec { <co xml:id='ex-hello-co-jobs' />
tarball = <co xml:id='ex-hello-co-tarball' />
{ helloSrc }: <co xml:id='ex-hello-co-tarball-arg' />
pkgs.releaseTools.sourceTarball { <co xml:id='ex-hello-co-source-tarball' />
name = "hello-tarball";
src = helloSrc;
buildInputs = (with pkgs; [ gettext texLive texinfo ]);
};
build = <co xml:id='ex-hello-co-build' />
{ tarball ? jobs.tarball {} <co xml:id='ex-hello-co-build-args' />
, system ? builtins.currentSystem
}:
let pkgs = import nixpkgs { inherit system; }; in
pkgs.releaseTools.nixBuild { <co xml:id='ex-hello-co-nix-build' />
name = "hello" ;
src = tarball;
configureFlags = [ "--disable-silent-rules" ];
};
};
in
jobs <co xml:id='ex-hello-body' />
</programlisting>
</example>
<para>
<xref linkend='ex-hello' /> shows what a
<filename>release.nix</filename> file for <link
xlink:href="http://www.gnu.org/software/hello/">GNU Hello</link>
would you like. GNU Hello is representative of many GNU
and non-GNU free software projects:
<itemizedlist>
<listitem>it uses the GNU Build System, namely GNU Autoconf,
and GNU Automake; for users, it means it can be installed
using the <link
xlink:href="http://www.gnu.org/prep/standards/html_node/Managing-Releases.html">usual
<literal>./configure && make install</literal>
procedure</link>;
</listitem>
<listitem>it uses Gettext for internationalization;</listitem>
<listitem>it has a Texinfo manual, which can be rendered as PDF
with TeX.</listitem>
</itemizedlist>
The file defines a jobset consisting of two jobs:
<literal>tarball</literal>, and <literal>build</literal>. It
contains the following elements (referenced from the figure by
numbers):
<calloutlist>
<callout arearefs='ex-hello-co-nixpkgs'>
<para>
This specifies a function of one named arguments,
<varname>nixpkgs</varname>. This function and those
defined below is called by Hydra. Here the
<varname>nixpkgs</varname> argument is meant to be a
checkout of the <link
xlink:href="http://nixos.org/nixpkgs/">Nixpkgs</link>
software distribution.
</para>
<para>
Hydra inspects the formal argument list of the function
(here, the <varname>nixpkgs</varname> argument) and passes
it the corresponding parameter specified as a build input
on Hydra's web interface. In this case, the web interface
should show a <varname>nixpkgs</varname> build input,
which is a checkout of the Nixpkgs source code repository.
</para>
</callout>
<callout arearefs='ex-hello-co-import-nixpkgs'>
<para>
This defines a variable <varname>pkgs</varname> holding
the set of packages provided by Nixpkgs.
</para>
</callout>
<callout arearefs='ex-hello-co-jobs'>
<para>
This defines a variable holding the two Hydra
jobs–an <emphasis>attribute set</emphasis> in Nix.
</para>
</callout>
<callout arearefs='ex-hello-co-tarball'>
<para>
This is the definition of the first job, named
<varname>tarball</varname>. The purpose of this job is to
produce a usable source code tarball.
</para>
</callout>
<callout arearefs='ex-hello-co-tarball-args'>
<para>
The <varname>tarball</varname> takes an additional
argument called <varname>helloSrc</varname>. Again, this
argument is passed by Hydra and is meant to be a checkout
of GNU Hello's source code repository.
</para>
</callout>
<callout arearefs='ex-hello-co-source-tarball'>
<para>
The <varname>tarball</varname> job calls the
<varname>sourceTarball</varname> function, which (roughly)
runs <command>autoreconf && ./configure &&
make dist</command> on the checkout. The
<varname>buildInputs</varname> attribute specifies
additional software dependencies for the job.
</para>
</callout>
<callout arearefs='ex-hello-co-build'>
<para>
This is the definition of the <varname>build</varname>
job, whose purpose is to build Hello from the tarball
produced above.
</para>
</callout>
<callout arearefs='ex-hello-co-build-args'>
<para>
The <varname>build</varname> function takes two additional
parameter: <varname>tarball</varname>, which is meant to
be the result of the <varname>tarball</varname> job, and
<varname>system</varname>, which should be a string
defining the Nix system type–e.g.,
<literal>"x86_64-linux"</literal>.
</para>
<para>
Again, these parameters are passed by Hydra when it calls
<varname>build</varname>. Thus, they must be defined as
build inputs in Hydra: <varname>tarball</varname> should
have type <literal>Build Output</literal>, its value being
the latest output of the <varname>tarball</varname> job,
and <varname>system</varname> should be a string.
</para>
<para>
The question mark after <literal>tarball</literal> and
<literal>system</literal> defines default values for these
arguments, and is only useful for debugging.
</para>
</callout>
<callout arearefs='ex-hello-co-nix-build'>
<para>
The <varname>build</varname> job calls the
<varname>nixBuild</varname> function, which unpacks the
tarball, then runs <command>./configure && make
&& make check && make install</command>.
</para>
</callout>
<callout arearefs='ex-hello-co-body'>
<para>
Finally, the set of jobs is returned to Hydra, as a Nix
attribute set.
</para>
</callout>
</calloutlist>
</para>
</section>
<section>
<title>Building from the Command Line</title>
<para>
It is often useful to test a build recipe, for instance before
it is actually used by Hydra, when testing changes, or when
debugging a build issue. Since build recipes for Hydra jobsets
are just plain Nix expressions, they can be evaluated using the
standard Nix tools.
</para>
<para>
To evaluate the <varname>tarball</varname> jobset of <xref
linkend='ex-hello' />, just run:
<screen>
$ nix-build release.nix -A tarball
</screen>
However, doing this with <xref linkend='ex-hello' /> as is will
probably yield an error like this:
<screen>
error: cannot auto-call a function that has an argument without a default value (`nixpkgs')
</screen>
This is because no value was specified for the
<varname>nixpkgs</varname> argument of the Nix expression.
</para>
<para>
This is fixed by providing a default value for that argument in
the Nix expression, which will allow <command>nix-build</command>
to auto-call the function: instfead of writing <literal>{ nixpkgs
}:</literal>, we now write <literal>{ nixpkgs ? <nixpkgs>
}:</literal>. What it means is that, by default, the
<varname>nixpkgs</varname> variable will be bound to the absolute
path of any <filename>nixpkgs</filename> file found in the Nix
search path. Similarly, a default value for
<varname>helloSrc</varname> needs to be provided.
</para>
<para>
Thus, assuming a checkout of Nixpkgs is available under
<filename>$HOME/src/nixpkgs</filename>, the
<varname>tarball</varname> jobset can now be evaluated by running:
<screen>
$ nix-build -I ~/src release.nix -A tarball
</screen>
Similarly, the <varname>build</varname> jobset can be evaluated:
<screen>
$ nix-build -I ~/src release.nix -A build
</screen>
The <varname>build</varname> job reuses the result of the
<varname>tarball</varname> job, rebuilding it only if it needs to.
</para>
</section>
</chapter>