Commit Graph

20 Commits

Author SHA1 Message Date
e0d3a1c1a5 Make nix-build args copy-pastable via set -x
A reproduce script includes a logline that may resemble:

> using these flags: --arg nixpkgs { outPath = /tmp/build-137689173/nixpkgs/source; rev = "fdc872fa200a32456f12cc849d33b1fdbd6a933c"; shortRev = "fdc872f"; revCount = 273100; } -I nixpkgs=/tmp/build-137689173/nixpkgs/source --arg officialRelease false --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux /tmp/build-137689173/nixpkgs/source/pkgs/top-level/release.nix -A 

These are passed along to nix-build and that's fine and dandy, but you can't just copy-paste this as is, as the `{}` introduces a syntax error and the value accompanying `-A` is `''`.

A very naive approach is to just `printf "%q"` the individual args, which makes them safe to copy-paste. Unfortunately, this looks awful due to the liberal usage of slashes:

```
$ printf "%q" '{ outPath = /tmp/build-137689173/nixpkgs/source; rev = "fdc872fa200a32456f12cc849d33b1fdbd6a933c"; shortRev = "fdc872f"; revCount = 273100; }'
\{\ outPath\ =\ /tmp/build-137689173/nixpkgs/source\;\ rev\ =\ \"fdc872fa200a32456f12cc849d33b1fdbd6a933c\"\;\ shortRev\ =\ \"fdc872f\"\;\ revCount\ =\ 273100\;\ \}
```

Alternatively, if we just use `set -x` before we execute nix-build, we'll get the whole invocation in a friendly, copy-pastable format that nicely displays `{}`-enclosed content and preserves the empty arg following `-A`:

```
running nix-build...
using this invocation: 
+ nix-build --arg nixpkgs '{ outPath = /tmp/build-138165173/nixpkgs/source; rev = "e0e4484f2c028d2269f5ebad0660a51bbe46caa4"; shortRev = "e0e4484"; revCount = 274008; }' -I nixpkgs=/tmp/build-138165173/nixpkgs/source --arg officialRelease false --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux /tmp/build-138165173/nixpkgs/source/pkgs/top-level/release.nix -A ''
```
2021-03-06 23:25:26 -05:00
9516b256f1 Normalize nixexpr{input,path} from builds to jobsetevals.
Duplicating this data on every record of the builds table cost
approximately 4G of duplication.

Note that the database migration included took about 4h45m on an
untuned server which uses very slow rotational disks in a RAID5 setup,
with not a lot of RAM. I imagine in production it might take an hour
or two, but not 4. If this should become a chunked migration, I can do
that.

Note: Because of the question about chunked migrations, I have NOT
YET tested this migration thoroughly enough for merge.
2021-01-22 09:10:18 -05:00
1d11b29e73 reproduce bash script must supply revCount as int
The reproduce script would fail for me, because it passed revCount as
a string to nix-build, which would then fail to subtract from it.
2020-09-12 14:04:43 +02:00
215b724e44 Fix passing arguments to the reproduction script 2018-05-10 15:11:08 +02:00
8913c682cf Doh 2017-12-25 15:02:17 +01:00
ff8cb93fe7 reproduce.tt: Wrap script to prevent problems piping into bash 2015-05-26 17:11:29 +02:00
c32a244bd7 reproduce.tt: Add --print-flags option
Useful when scripting stuff (e.g. if you want to do a nix-instantiate
instead of a nix-build).
2015-05-26 16:36:08 +02:00
f5ecb7a81a reproduce.tt: Use -I instead of $NIX_PATH 2015-05-26 16:07:00 +02:00
b8a0956443 reproduce.tt: Add option to fetch but not build 2015-05-26 16:01:37 +02:00
91ecee0e5d Fix reproduce script 2015-05-26 15:54:38 +02:00
f50477141d Add an input type "nix" for passing arbitrary Nix expressions 2013-09-30 12:03:25 +02:00
8e246a835a Don't do camel case in filenames 2013-06-02 23:30:46 +02:00
591acda6f9 reproduce.tt: Save the revCount when reproducing more than once
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-05-30 14:06:32 -04:00
8e716ff062 reproduce.tt: Create the checkout dir before an hg clone
For some reason, hg clone from a local (path-based) repo will fail if
the parent directory of the destination directory doesn't exist (though
it succeeds when cloning from an http repo).

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-05-30 13:31:53 -04:00
4b1a8384db reproduce.tt: Set revCount for hg repos
The revision counting changes depending on which revision is cloned
initially, so clone the default branch first and then checkout the
required revision to match hydra's revCount.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-05-16 11:24:17 -04:00
a84db1c7cc Use extra-binary-caches 2013-05-07 15:57:20 +02:00
edaeb892e7 reproduce.tt: Use color 2013-05-02 17:55:47 +02:00
a468a28840 reproduce.tt: Check for the presence of commands like nix-build 2013-05-02 17:55:47 +02:00
659898bd67 reproduce.tt: Add --help flag
And a --dir flag to override the directory where inputs etc. are
stored.
2013-05-02 17:55:47 +02:00
9f027b22b1 Allow users to reproduce builds on their own systems
You can now do:

  bash <(curl http://hydra-server/build/1238757/reproduce)

to download and execute a script that reproduces a Hydra build
locally.  This script fetches all inputs (e.g. Git repositories) and
then invokes nix-build.

The downloaded sources are stored in /tmp/build-<buildid> and reused
between invocations of the script.

Any additional command line options are passed to nix-build.  So

  bash <(curl http://hydra-server/build/1238757/reproduce) --run-env

will drop you in a shell where you can interactively hack on the
build, e.g.

  $ source $stdenv/setup
  $ set +e
  $ unpackPhase
  $ cd $sourceRoot
  $ configurePhase
  $ emacs foo.c &
  $ make

and so on.
2013-04-04 17:30:07 +02:00