| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | <html> | 
					
						
							|  |  |  | <head> | 
					
						
							|  |  |  | <link rel="stylesheet" href="style.css" type="text/css"> | 
					
						
							|  |  |  | </head> | 
					
						
							|  |  |  | <body> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <hr> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h1>The Hydra Buildfarm User Manual</h1> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h2>Draft (Version 0.1)</h2> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>Eelco Dolstra and Eelco Visser</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | Delft University of Technology | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | Department of Software Technology | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | Copyright 2008 Eelco Dolstra | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <hr> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h2>Chapter 1. Introduction</h2> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>1.1. About Hydra</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | Hydra is a tool for continuous integration testing and software | 
					
						
							|  |  |  | release that uses a purely functional language to describe build jobs | 
					
						
							|  |  |  | and their dependencies.  Continuous integration is a simple technique | 
					
						
							|  |  |  | to improve the quality of the software development process.  An | 
					
						
							|  |  |  | automated system continuously or periodically checks out the source | 
					
						
							|  |  |  | code of a project, builds it, runs tests, and produces reports for the | 
					
						
							|  |  |  | developers.  Thus, various errors that might accidentally be committed | 
					
						
							|  |  |  | into the code base are automatically caught.  Such a system allows | 
					
						
							|  |  |  | more in-depth testing than what developers could feasibly do manually: | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <ol> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> <em>Portability testing</em>: The software may need to be built | 
					
						
							|  |  |  |   and tested on many different platforms.  It is infeasible for each | 
					
						
							|  |  |  |   developer to do this before every commit.  | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> Likewise, many projects have very large test sets (e.g., | 
					
						
							|  |  |  |   regression tests in a compiler, or stress tests in a DBMS) that can | 
					
						
							|  |  |  |   take hours or days to run to completion.  | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> Many kinds of static and dynamic analyses can be performed as | 
					
						
							|  |  |  |   part of the tests, such as code coverage runs and static analyses. | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> It may also be necessary to build many different <em>variants</em> | 
					
						
							|  |  |  |   of the software.  For instance, it may be necessary to verify that | 
					
						
							|  |  |  |   the component builds with various versions of a compiler. | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> Developers typically use incremental building to test their | 
					
						
							|  |  |  |   changes (since a full build may take too long), but this is | 
					
						
							|  |  |  |   unreliable with many build management tools (such as Make), i.e., | 
					
						
							|  |  |  |   the result of the incremental build might differ from a full build. | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> It ensures that the software can be built from the sources under | 
					
						
							|  |  |  |   revision control.  Users of version management systems such as CVS | 
					
						
							|  |  |  |   and Subversion often forget to place source files under revision | 
					
						
							|  |  |  |   control. | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> The machines on which the continuous integration system runs | 
					
						
							|  |  |  |   ideally provides a clean, well-defined build environment.  If this | 
					
						
							|  |  |  |   environment is administered through proper SCM techniques, then | 
					
						
							|  |  |  |   builds produced by the system can be reproduced.  In contrast, | 
					
						
							|  |  |  |   developer work environments are typically not under any kind of SCM | 
					
						
							|  |  |  |   control. | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> In large projects, developers often work on a particular | 
					
						
							|  |  |  |   component of the project, and do not build and test the composition | 
					
						
							|  |  |  |   of those components (again since this is likely to take too long). | 
					
						
							|  |  |  |   To prevent the phenomenon of ``big bang integration'', where | 
					
						
							|  |  |  |   components are only tested together near the end of the development | 
					
						
							|  |  |  |   process, it is important to test components together as soon as | 
					
						
							|  |  |  |   possible (hence <em>continuous integration</em>). | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> It allows software to be <em>released</em> by automatically | 
					
						
							|  |  |  |   creating packages that users can download and install.  To do this | 
					
						
							|  |  |  |   manually represents an often prohibitive amount of work, as one may | 
					
						
							|  |  |  |   want to produce releases for many different platforms: e.g., | 
					
						
							|  |  |  |   installers for Windows and Mac OS X, RPM or Debian packages for | 
					
						
							|  |  |  |   certain Linux distributions, and so on. | 
					
						
							|  |  |  |   <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | </ol> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | In its simplest form, a continuous integration tool sits in a loop | 
					
						
							|  |  |  | building and releasing software components from a version management | 
					
						
							|  |  |  | system.  For each component, it performs the following tasks: | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <ol> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li>It obtains the latest version of the component's source code | 
					
						
							|  |  |  |   from the version management system. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> It runs the component's build process (which presumably includes | 
					
						
							|  |  |  |   the execution of the component's test set). | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> It presents the results of the build (such as error logs and | 
					
						
							|  |  |  |   releases) to the developers, e.g., by producing a web page. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | </ol> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | Examples of continuous integration tools include CruiseControl | 
					
						
							|  |  |  | Tinderbox, Sisyphus, Anthill and BuildBot. These tools have various | 
					
						
							|  |  |  | limitations. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <ol> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> They do not manage the <em>build environment</em>.  The build | 
					
						
							|  |  |  | environment consists of the dependencies necessary to perform a build | 
					
						
							|  |  |  | action, e.g., compilers, libraries, etc.  Setting up the environment | 
					
						
							|  |  |  | is typically done manually, and without proper SCM control (so it may | 
					
						
							|  |  |  | be hard to reproduce a build at a later time).  Manual management of | 
					
						
							|  |  |  | the environment scales poorly in the number of configurations that | 
					
						
							|  |  |  | must be supported.  For instance, suppose that we want to build a | 
					
						
							|  |  |  | component that requires a certain compiler X.  We then have to go to | 
					
						
							|  |  |  | each machine and install X.  If we later need a newer version of X, | 
					
						
							|  |  |  | the process must be repeated all over again.  An ever worse problem | 
					
						
							|  |  |  | occurs if there are conflicting, mutually exclusive versions of the | 
					
						
							|  |  |  | dependencies.  Thus, simply installing the latest version is not an | 
					
						
							|  |  |  | option.  Of course, we can install these components in different | 
					
						
							|  |  |  | directories and manually pass the appropriate paths to the build | 
					
						
							|  |  |  | processes of the various components.  But this is a rather tiresome | 
					
						
							|  |  |  | and error-prone process.  <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <li> They do not easily support <em>variability in software | 
					
						
							|  |  |  | systems</em>.  A system may have a great deal of build-time | 
					
						
							|  |  |  | variability: optional functionality, whether to build a debug or | 
					
						
							|  |  |  | production version, different versions of dependencies, and so on. | 
					
						
							|  |  |  | (For instance, the Linux kernel now has over 2,600 build-time | 
					
						
							|  |  |  | configuration switches.)  It is therefore important that a continuous | 
					
						
							|  |  |  | integration tool can easily select and test different instances from | 
					
						
							|  |  |  | the configuration space of the system to reveal problems, such as | 
					
						
							|  |  |  | erroneous interactions between features.  In a continuous integration | 
					
						
							|  |  |  | setting, it is also useful to test different combinations of versions | 
					
						
							|  |  |  | of subsystems, e.g., the head revision of a component against stable | 
					
						
							|  |  |  | releases of its dependencies, and vice versa, as this can reveal | 
					
						
							|  |  |  | various integration problems. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | </ol> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <em>Hydra</em>, is a continuous integration tool that solves these | 
					
						
							|  |  |  | problems.  It is built on top of the <a href="http://nixos.org">Nix | 
					
						
							|  |  |  | package manager</a>, which has a purely functional language for | 
					
						
							|  |  |  | describing package build actions and their dependencies.  This allows | 
					
						
							|  |  |  | the build environment for projects to be produced automatically and | 
					
						
							|  |  |  | deterministically, and variability in components to be expressed | 
					
						
							|  |  |  | naturally using functions; and as such is an ideal fit for a | 
					
						
							|  |  |  | continuous build system. | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:20:39 +00:00
										 |  |  | <h3>1.2. About Us</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Hydra is the successor of the Nix Buildfarm, which was developed in | 
					
						
							|  |  |  | tandem with the Nix software deployment system. Nix was originally | 
					
						
							|  |  |  | developed at the Department of Information and Computing Sciences, | 
					
						
							|  |  |  | Utrecht University by the TraCE project (2003-2008). The project was | 
					
						
							|  |  |  | funded by the Software Engineering Research Program Jacquard to | 
					
						
							|  |  |  | improve the support for variability in software systems.  Funding for | 
					
						
							|  |  |  | the development of Nix and Hydra is now provided by the NIRICT LaQuSo | 
					
						
							|  |  |  | Build Farm project. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>1.3. About this Manual</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | This manual tells you how to install the Hydra buildfarm software on | 
					
						
							|  |  |  | your own server and how to operate that server using its web | 
					
						
							|  |  |  | interface. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>1.4. License</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Hydra is free software; you can redistribute it and/or modify it under | 
					
						
							|  |  |  | the terms of the GNU Lesser General Public License as published by the | 
					
						
							|  |  |  | Free Software Foundation; either version 2.1 of the License, or (at | 
					
						
							|  |  |  | your option) any later version. Hydra is distributed in the hope that | 
					
						
							|  |  |  | it will be useful, but WITHOUT ANY WARRANTY; without even the implied | 
					
						
							|  |  |  | warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See | 
					
						
							|  |  |  | the GNU Lesser General Public License for more details. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>1.5. Hydra at nixos.org</h3> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | The nixos.org installation of Hydra runs at  | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <blockquote> | 
					
						
							|  |  |  |   <a href="http://hydra.nixos.org/">http://hydra.nixos.org</a> | 
					
						
							|  |  |  | </blockquote> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | That installation is used to build software components from the  | 
					
						
							|  |  |  | <a href="http://nixos.org">Nix</a>,  | 
					
						
							|  |  |  | <a href="http://nixos.org/nixos">NixOS</a>,  | 
					
						
							|  |  |  | <a href="http://strategoxt.org">Stratego/XT</a>,  | 
					
						
							|  |  |  | and related projects.  | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | If you are one of the developers on those projects, it is likely that | 
					
						
							|  |  |  | you will be using the NixOS Hydra server in some way. If you need to | 
					
						
							|  |  |  | administer automatic builds for your project, you should pull the | 
					
						
							|  |  |  | right strings to get an account on the server. This manual will tell | 
					
						
							|  |  |  | you how to set up new projects and build jobs within those projects | 
					
						
							|  |  |  | and write a release.nix file to describe the build process of your | 
					
						
							|  |  |  | project to Hydra. You can skip Chapter 2. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | If your project does not yet have automatic builds within the NixOS | 
					
						
							|  |  |  | Hydra server, it may actually be eligible. We are in the process of | 
					
						
							|  |  |  | setting up a large buildfarm that should be able to support open | 
					
						
							|  |  |  | source and academic software projects. Get in touch. | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:20:39 +00:00
										 |  |  | <h3>1.6. Hydra on your own buildfarm</h3> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | If you need to run your own Hydra installation, Chapter 2 explains | 
					
						
							|  |  |  | how to download and install the system on your own server. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h2>Chapter 2. Installation and Configuration</h2> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | This chapter explains how to install Hydra on your own buildfarm server. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>2.1. Platform Requirements</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:20:39 +00:00
										 |  |  | To run Hydra you need a Linux server.  For small projects, Hydra can | 
					
						
							|  |  |  | be run on any reasonably modern machine. For individual projects you | 
					
						
							|  |  |  | can even run Hydra on a laptop. However, the charm of a buildfarm | 
					
						
							|  |  |  | server is usually that it operates without disturbing the developer's | 
					
						
							|  |  |  | working environment and can serve releases over the internet. In | 
					
						
							|  |  |  | conjunction you should typically have your source code administered in | 
					
						
							|  |  |  | a version management system, such as subversion. Therefore, you will | 
					
						
							|  |  |  | probably want to install a server that is connected to the | 
					
						
							|  |  |  | internet. To scale up to large and/or many projects, you will need at | 
					
						
							|  |  |  | least a considerable amount of diskspace to store builds. Since Hydra | 
					
						
							|  |  |  | can schedule multiple simultaneous build jobs, it can be useful to | 
					
						
							|  |  |  | have a multi-core machine, and/or attach multiple build machines in a | 
					
						
							|  |  |  | network to the central Hydra server. | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Of course we think it is a good idea to use the <a | 
					
						
							|  |  |  | href="http://nixos.org/nixos">NixOS</a> Linux distribution for your | 
					
						
							|  |  |  | buildfarm server. But this is not a requirement. The Nix software | 
					
						
							|  |  |  | deployment system can be installed on any Linux distribution in | 
					
						
							|  |  |  | parallel to the regular package management system. Thus, you can use | 
					
						
							|  |  |  | Hydra on a Suse, Fedora, or Ubuntu system. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>2.2. Getting Nix</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | If your server runs NixOS you are all set to continue with | 
					
						
							|  |  |  | installation of Hydra. Otherwise you first need to install Nix. | 
					
						
							|  |  |  | The <a href="http://nixos.org/releases/nix/nix-0.12">latest stable release</a> is Nix 0.12. | 
					
						
							|  |  |  | Installation instructions can be found in the  | 
					
						
							|  |  |  | <a href="http://nixos.org/releases/nix/nix-0.12/manual/">Nix User's Guide</a>. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>2.3. Installation</h2> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | To install Hydra, get the most recent 'closure' available from | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <blockquote> | 
					
						
							|  |  |  |   <a href="http://hydra.nixos.org/releases/hydra/unstable">http://hydra.nixos.org/releases/hydra/unstable</a> | 
					
						
							|  |  |  | </blockquote> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | And follow the instructions that are revealed by clicking [help]. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ gunzip < hydra-build.closure.gz | nix-store --import | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | This unpacks the closure and imports its components into the Nix store. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ nix-env -i /nix/store/...-hydra-build | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | This makes the tools in the Hydra package available in your Nix user | 
					
						
							|  |  |  | environment. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Command completion should then reveal a number of tools related to | 
					
						
							|  |  |  | hydra installed: | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ hydra_<tab> | 
					
						
							|  |  |  | hydra_build.pl         hydra_fastcgi.pl       hydra_scheduler.pl | 
					
						
							|  |  |  | hydra_cgi.pl           hydra_init.pl          hydra_server.pl | 
					
						
							|  |  |  | hydra_create.pl        hydra_queue_runner.pl  hydra_test.pl | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>2.4. Configuration</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | The Hydra software is installed in the Nix store, but to run it needs | 
					
						
							|  |  |  | a directory for storing the database, logs, and session data. In your | 
					
						
							|  |  |  | <code>.bashrc</code> or similar configuration file define: | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  |  export HYDRA_DATA=/usr/local/hydra | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | and make sure that you actually create that directory. (Of course, you | 
					
						
							|  |  |  | can use another directory, but then remember to also substitute that | 
					
						
							|  |  |  | name in the commands below.) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Run <code>hydra_init.pl</code> to initialize the database | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ hydra_init.pl | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Run <code>hydra_server.pl</code> to start the webserver at <a href="http://localhost:3000">http://localhost:3000</a> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ hydra_server.pl | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Also start the scheduler, which monitors the source repositories and | 
					
						
							|  |  |  | adds builds to the queue, and the runner, which executes jobs in the | 
					
						
							|  |  |  | queue. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ hydra_scheduler.pl | 
					
						
							|  |  |  | $ hydra_queue_runner.pl | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Now your Hydra server should be up and running and the web interface | 
					
						
							|  |  |  | operational. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>2.5. User Administration</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | To be able to add jobs and create projects you need to register users | 
					
						
							|  |  |  | in the Hydra database. In the current version, the web interface does | 
					
						
							|  |  |  | not yet support user administration. Use the following command to add | 
					
						
							|  |  |  | a new user to the database. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ sqlite3 /usr/local/hydra/hydra.sqlite "insert into Users(userName, emailAddress, password) values('eelco', 'blablah@example.org', '$(echo -n foobar | sha1sum | cut -c1-40)');" | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | where <code>eelco</code> is the username, and <code>foobar</code> the | 
					
						
							|  |  |  | password. (Make sure to use other values!) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | To give this user administrator privileges, follow this up by: | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  | $ sqlite3 /usr/local/hydra/hydra.sqlite "insert into UserRoles(userName, role) values('eelco', 'admin');" | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Now you should be able to create a project using the Hydra web interface. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h2>Chapter 3. Creating Projects</h2> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | The next step is to add projects to the buildfarm. We follow the | 
					
						
							|  |  |  | example of the patchelf project at hydra.nixos.org. Note that the | 
					
						
							|  |  |  | error messages provided as feedback by the webinterface can be | 
					
						
							|  |  |  | somewhat unfriendly in the current version. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <a href="http://localhost:3000/login">Login</a>  | 
					
						
							|  |  |  | to the webinterface of your Hydra installation using  | 
					
						
							|  |  |  | the username and password you inserted in the database.  | 
					
						
							|  |  |  | Then follow the  | 
					
						
							|  |  |  | '<a href="http://localhost:3000/createproject">Create Project</a>'  | 
					
						
							|  |  |  | link to create a new project. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <h3>3.1. General information</h3> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:20:39 +00:00
										 |  |  | A project definition consists of some general information and a set of | 
					
						
							| 
									
										
										
										
											2008-12-01 20:43:09 +00:00
										 |  |  | jobsets.  The general information identifies a project, its owner, and | 
					
						
							|  |  |  | current state of activity. | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:43:09 +00:00
										 |  |  | Here's what we fill in for the patchelf project: | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  |   Identifier: patchelf | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | The <strong>identifier</strong> is the identity of the project. It is | 
					
						
							|  |  |  | used in URLs and in the names of build results. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | The identifier should be a unique name (it is the | 
					
						
							|  |  |  | primary database key for the project table in the database). If you | 
					
						
							|  |  |  | try to create a project with an already existing identifier you'd get | 
					
						
							|  |  |  | an error message such as: | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | <pre> | 
					
						
							|  |  |  | I'm very sorry, but an error occurred:  | 
					
						
							|  |  |  | DBIx::Class::ResultSet::create(): DBI Exception: DBD::SQLite::st execute failed: column name is not unique(19) at dbdimp.c line 402 | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | So try to create the project after entering just the general | 
					
						
							|  |  |  | information to figure out if you have chosen a unique name.  | 
					
						
							|  |  |  | Jobsets can be added once the project has been created. | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  |   Display name: Patchelf | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | The <strong>display name</strong> is used in menus. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  |   Description: A tool for modifying ELF binaries | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | The <strong>description</strong> is used as short documentation of the | 
					
						
							|  |  |  | nature of the project. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  |   Owner: eelco | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | The <strong>owner</strong> of a project can create and edit jobsets. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  |   Enabled: Yes | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | Only if the project is <strong>enabled</strong> are builds performed. | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:43:09 +00:00
										 |  |  | Once created there should be an entry for the project in the | 
					
						
							|  |  |  | sidebar. Go to the project page for the <a | 
					
						
							|  |  |  | href="http://localhost:3000/project/patchelf">Patchelf</a> project. | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <h3>3.2. Jobsets</h3> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | A project can consist of multiple `jobsets', separate tasks that can | 
					
						
							|  |  |  | be built separately, but may depend on each other (without cyclic | 
					
						
							|  |  |  | dependencies, of course). Go to the  | 
					
						
							|  |  |  | <a href="http://localhost:3000/project/patchelf/edit">Edit</a>  | 
					
						
							|  |  |  | page of the Patchelf project and 'Add a new jobset'  | 
					
						
							|  |  |  | by providing the following 'Information': | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | <pre class="programlisting"> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  |   Identifier:     trunk | 
					
						
							|  |  |  |   Description:    Trunk | 
					
						
							|  |  |  |   Nix expression: release.nix in input patchelfSrc | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | This states that in order to build the 'Trunk' jobset, the Nix | 
					
						
							|  |  |  | expression in the file 'release.nix', which can be obtained from input | 
					
						
							|  |  |  | 'patchelfSrc', should be evaluated. (We'll have a look at release.nix | 
					
						
							|  |  |  | later.) | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <p/> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | To realize a job we probably need a number of inputs, which can be | 
					
						
							|  |  |  | declared in the table below. As many inputs as required can be added.  | 
					
						
							|  |  |  | For patchelf we declare the following inputs. | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | <pre class="programlisting"> | 
					
						
							|  |  |  |   patchelfSrc  | 
					
						
							|  |  |  |     'Subversion checkout' https://svn.nixos.org/repos/nix/patchelf/trunk | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | patchelfSrc | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  |   nixpkgs 'CVS checkout' https://svn.nixos.org/repos/nix/nixpkgs/trunk | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | nixpkgs | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | <pre class="programlisting"> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  |   officialRelease   Boolean false | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | officialRelease | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 20:50:50 +00:00
										 |  |  | <pre class="programlisting"> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  |   system String value "i686-linux"  | 
					
						
							|  |  |  | </pre> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | system  | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <h3>3.2. Release Set</h3> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | there must be one primary job | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | check the radio button of exactly one job | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | https://svn.nixos.org/repos/nix/nixpkgs/trunk | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <h3>3.3. Building Jobs</h3> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <h3>3.4. release.nix</h3> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | - Voorbeelden van Nix expressies voor Hydra: | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | https://svn.nixos.org/repos/nix/patchelf/trunk/release.nix | 
					
						
							|  |  |  | https://svn.nixos.org/repos/nix/nix/trunk/release.nix | 
					
						
							|  |  |  | https://svn.nixos.org/repos/nix/hydra/trunk/release.nix | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2008-12-01 23:20:42 +00:00
										 |  |  | <h3>3.5. Building on the command line</h3> | 
					
						
							| 
									
										
										
										
											2008-12-01 20:03:18 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | Overigens zijn die helemaal niet Hydra-specifiek, je kunt ze gewoon vanaf de | 
					
						
							|  |  |  | command line bouwen, bijv. als je een patchelf checkout hebt (met een nixpkgs | 
					
						
							|  |  |  | checkout in ../nixpkgs): | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | $ nix-build release.nix -A rpm_fedora10i386 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | </body> | 
					
						
							|  |  |  | </html> |