ahuston-0/hydra
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

webhooks: implement authentication for GitHub and Gitea

Browse Source 
- Add HMAC-SHA256 signature verification for webhooks
- Support multiple secrets for rotation
- Add security logging for authentication events
- Maintain backward compatibility (auth optional during migration)
- Add comprehensive test coverage

Without authentication, anyone could trigger job evaluations by sending
POST requests to webhook endpoints. This could lead to resource exhaustion
through repeated requests or manipulation of build scheduling. While not
a data breach risk, it allows unauthorized control over CI/CD operations.
This commit is contained in:
Jörg Thalheim
2025-08-03 07:31:13 +02:00
committed by ahuston-0
parent 4d2d0f9722
commit f2cbf14f7e
7 changed files with 596 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
doc/manual/src/SUMMARY.md
View File

@@ -10,6 +10,7 @@
- [RunCommand](./plugins/RunCommand.md)
- [Using the external API](api.md)
- [Webhooks](webhooks.md)
- [Webhook Authentication Migration Guide](webhook-migration-guide.md)
- [Monitoring Hydra](./monitoring/README.md)
## Developer's Guide

34
doc/manual/src/configuration.md
View File

@@ -266,6 +266,40 @@ default role mapping:
Note that configuring both the LDAP parameters in the hydra.conf and via
the environment variable is a fatal error.
Webhook Authentication
---------------------
Hydra supports authenticating webhook requests from GitHub and Gitea to prevent unauthorized job evaluations.
Webhook secrets should be stored in separate files outside the Nix store for security using Config::General's include mechanism.
In your main `hydra.conf`:
```apache
<webhooks>
Include /var/lib/hydra/secrets/webhook-secrets.conf
</webhooks>
```
Then create `/var/lib/hydra/secrets/webhook-secrets.conf` with your actual secrets:
```apache
<github>
secret = your-github-webhook-secret
</github>
<gitea>
secret = your-gitea-webhook-secret
</gitea>
```
For multiple secrets (useful for rotation or multiple environments), use an array:
```apache
<github>
secret = your-github-webhook-secret-prod
secret = your-github-webhook-secret-staging
</github>
```
**Important**: The secrets file should have restricted permissions (e.g., 0600) to prevent unauthorized access.
See the [Webhooks documentation](webhooks.md) for detailed setup instructions.
Embedding Extra HTML
--------------------

168
doc/manual/src/webhook-migration-guide.md Normal file
View File

@@ -0,0 +1,168 @@
# Webhook Authentication Migration Guide
This guide helps Hydra administrators migrate from unauthenticated webhooks to authenticated webhooks to secure their Hydra instances against unauthorized job evaluations.
## Why Migrate?
Currently, Hydra's webhook endpoints (`/api/push-github` and `/api/push-gitea`) accept any POST request without authentication. This vulnerability allows:
- Anyone to trigger expensive job evaluations
- Potential denial of service through repeated requests
- Manipulation of build timing and scheduling
## Step-by-Step Migration for NixOS
### 1. Create Webhook Configuration
Create a webhook secrets configuration file with the generated secrets:
```bash
# Create the secrets configuration file with inline secret generation
cat > /var/lib/hydra/secrets/webhook-secrets.conf <<EOF
<github>
secret = $(openssl rand -hex 32)
</github>
<gitea>
secret = $(openssl rand -hex 32)
</gitea>
EOF
# Set secure permissions
chmod 0600 /var/lib/hydra/secrets/webhook-secrets.conf
chown hydra:hydra /var/lib/hydra/secrets/webhook-secrets.conf
```
**Important**: Save the generated secrets to configure them in GitHub/Gitea later. You can view them with:
```bash
cat /var/lib/hydra/secrets/webhook-secrets.conf
```
Then update your NixOS configuration to include the webhook configuration:
```nix
{
services.hydra-dev = {
enable = true;
hydraURL = "https://hydra.example.com";
notificationSender = "hydra@example.com";
extraConfig = ''
<webhooks>
Include /var/lib/hydra/secrets/webhook-secrets.conf
</webhooks>
'';
};
}
```
For multiple secrets (useful for rotation or multiple environments), update your webhook-secrets.conf:
```apache
<github>
secret = your-github-webhook-secret-prod
secret = your-github-webhook-secret-staging
</github>
<gitea>
secret = your-gitea-webhook-secret
</gitea>
```
### 2. Deploy Configuration
Apply the NixOS configuration:
```bash
nixos-rebuild switch
```
This will automatically restart Hydra services with the new configuration.
### 3. Verify Configuration
Check Hydra's logs to ensure secrets were loaded successfully:
```bash
journalctl -u hydra-server | grep -i webhook
```
You should not see warnings about webhook authentication not being configured.
### 4. Update Your Webhooks
#### GitHub
1. Navigate to your repository settings: `https://github.com/<owner>/<repo>/settings/hooks`
2. Edit your existing Hydra webhook
3. In the "Secret" field, paste the content of `/var/lib/hydra/secrets/github-webhook-secret`
4. Click "Update webhook"
5. GitHub will send a ping event to verify the configuration
#### Gitea
1. Navigate to your repository webhook settings
2. Edit your existing Hydra webhook
3. In the "Secret" field, paste the content of `/var/lib/hydra/secrets/gitea-webhook-secret`
4. Click "Update Webhook"
5. Use the "Test Delivery" button to verify the configuration
### 5. Test the Configuration
After updating each webhook:
1. Make a test commit to trigger the webhook
2. Check Hydra's logs for successful authentication
3. Verify the evaluation was triggered in Hydra's web interface
## Troubleshooting
### 401 Unauthorized Errors
If webhooks start failing with 401 errors:
- Verify the secret in the Git forge matches the file content exactly
- Check file permissions: `ls -la /var/lib/hydra/secrets/`
- Ensure no extra whitespace in secret files
- Check Hydra logs for specific error messages
### Webhook Still Unauthenticated
If you see warnings about unauthenticated webhooks after configuration:
- Verify the configuration syntax in your NixOS module
- Ensure the NixOS configuration was successfully applied
- Check that the webhook-secrets.conf file exists and is readable by the Hydra user
- Verify the Include path is correct in your hydra.conf
- Check the syntax of your webhook-secrets.conf file
### Testing Without Git Forge
You can test webhook authentication using curl:
```bash
# Read the secret
SECRET=$(cat /var/lib/hydra/secrets/github-webhook-secret)
# Create test payload
PAYLOAD='{"ref":"refs/heads/main","repository":{"clone_url":"https://github.com/test/repo.git"}}'
# Calculate signature
SIGNATURE="sha256=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2)"
# Send authenticated request
curl -X POST https://your-hydra/api/push-github \
-H "Content-Type: application/json" \
-H "X-Hub-Signature-256: $SIGNATURE" \
-d "$PAYLOAD"
```
For Gitea (no prefix in signature):
```bash
# Read the secret
SECRET=$(cat /var/lib/hydra/secrets/gitea-webhook-secret)
# Create test payload
PAYLOAD='{"ref":"refs/heads/main","repository":{"clone_url":"https://gitea.example.com/test/repo.git"}}'
# Calculate signature
SIGNATURE=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2)
# Send authenticated request
curl -X POST https://your-hydra/api/push-gitea \
-H "Content-Type: application/json" \
-H "X-Gitea-Signature: $SIGNATURE" \
-d "$PAYLOAD"
```

76
doc/manual/src/webhooks.md
View File

@@ -3,6 +3,58 @@
Hydra can be notified by github or gitea with webhooks to trigger a new evaluation when a
jobset has a github repo in its input.
## Webhook Authentication
Hydra supports webhook signature verification for both GitHub and Gitea using HMAC-SHA256. This ensures that webhook
requests are coming from your configured Git forge and haven't been tampered with.
### Configuring Webhook Authentication
1. **Create webhook configuration**: Generate and store webhook secrets securely:
```bash
# Create directory and generate secrets in one step
mkdir -p /var/lib/hydra/secrets
cat > /var/lib/hydra/secrets/webhook-secrets.conf <<EOF
<github>
secret = $(openssl rand -hex 32)
</github>
<gitea>
secret = $(openssl rand -hex 32)
</gitea>
EOF
# Set secure permissions
chmod 0600 /var/lib/hydra/secrets/webhook-secrets.conf
chown hydra:hydra /var/lib/hydra/secrets/webhook-secrets.conf
```
2. **Configure Hydra**: Add the following to your `hydra.conf`:
```apache
<webhooks>
Include /var/lib/hydra/secrets/webhook-secrets.conf
</webhooks>
```
3. **Configure your Git forge**: View the generated secrets and configure them in GitHub/Gitea:
```bash
grep "secret =" /var/lib/hydra/secrets/webhook-secrets.conf
```
### Multiple Secrets Support
Hydra supports configuring multiple secrets for each platform, which is useful for:
- Zero-downtime secret rotation
- Supporting multiple environments (production/staging)
- Gradual migration of webhooks
To configure multiple secrets, use array syntax:
```apache
<github>
secret = current-webhook-secret
secret = previous-webhook-secret
</github>
```
## GitHub
To set up a webhook for a GitHub repository go to `https://github.com/<yourhandle>/<yourrepo>/settings`
@@ -10,11 +62,16 @@ and in the `Webhooks` tab click on `Add webhook`.
- In `Payload URL` fill in `https://<your-hydra-domain>/api/push-github`.
- In `Content type` switch to `application/json`.
- The `Secret` field can stay empty.
- In the `Secret` field, enter the content of your GitHub webhook secret file (if authentication is configured).
- For `Which events would you like to trigger this webhook?` keep the default option for events on `Just the push event.`.
Then add the hook with `Add webhook`.
### Verifying GitHub Webhook Security
After configuration, GitHub will send webhook requests with an `X-Hub-Signature-256` header containing the HMAC-SHA256
signature of the request body. Hydra will verify this signature matches the configured secret.
## Gitea
To set up a webhook for a Gitea repository go to the settings of the repository in your Gitea instance
@@ -22,6 +79,23 @@ and in the `Webhooks` tab click on `Add Webhook` and choose `Gitea` in the drop
- In `Target URL` fill in `https://<your-hydra-domain>/api/push-gitea`.
- Keep HTTP method `POST`, POST Content Type `application/json` and Trigger On `Push Events`.
- In the `Secret` field, enter the content of your Gitea webhook secret file (if authentication is configured).
- Change the branch filter to match the git branch hydra builds.
Then add the hook with `Add webhook`.
### Verifying Gitea Webhook Security
After configuration, Gitea will send webhook requests with an `X-Gitea-Signature` header containing the HMAC-SHA256
signature of the request body. Hydra will verify this signature matches the configured secret.
## Troubleshooting
If you receive 401 Unauthorized errors:
- Verify the webhook secret in your Git forge matches the content of the secret file exactly
- Check that the secret file has proper permissions (should be 0600)
- Look at Hydra's logs for specific error messages
- Ensure the correct signature header is being sent by your Git forge
If you see warnings about webhook authentication not being configured:
- Configure webhook authentication as described above to secure your endpoints

78
src/lib/Hydra/Controller/API.pm
View File

@@ -12,6 +12,8 @@ use DateTime;
use Digest::SHA qw(sha256_hex);
use Text::Diff;
use IPC::Run qw(run);
use Digest::SHA qw(hmac_sha256_hex);
use String::Compare::ConstantTime qw(equals);
sub api : Chained('/') PathPart('api') CaptureArgs(0) {
@@ -274,13 +276,84 @@ sub push : Chained('api') PathPart('push') Args(0) {
);
}
sub verifyWebhookSignature {
my ($c, $platform, $header_name, $signature_prefix) = @_;
# Get secrets from config
my $webhook_config = $c->config->{webhooks} // {};
my $platform_config = $webhook_config->{$platform} // {};
my $secrets = $platform_config->{secret};
# Normalize to array
$secrets = [] unless defined $secrets;
$secrets = [$secrets] unless ref($secrets) eq 'ARRAY';
# Trim whitespace from secrets
my @secrets = grep { defined && length } map { s/^\s+|\s+$//gr } @$secrets;
if (@secrets) {
my $signature = $c->request->header($header_name);
if (!$signature) {
$c->log->warn("Webhook authentication failed for $platform: Missing signature from IP " . $c->request->address);
$c->response->status(401);
$c->stash->{json} = { error => "Missing webhook signature" };
$c->forward('View::JSON');
return 0;
}
# Get the raw body content from the buffered PSGI input
# For JSON requests, Catalyst will have already read and buffered the body
my $input = $c->request->env->{'psgi.input'};
$input->seek(0, 0);
local $/;
my $payload = <$input>;
$input->seek(0, 0); # Reset for any other consumers
unless (defined $payload && length $payload) {
$c->log->warn("Webhook authentication failed for $platform: Empty request body from IP " . $c->request->address);
$c->response->status(400);
$c->stash->{json} = { error => "Empty request body" };
$c->forward('View::JSON');
return 0;
}
my $valid = 0;
for my $secret (@secrets) {
my $expected = $signature_prefix . hmac_sha256_hex($payload, $secret);
if (equals($signature, $expected)) {
$valid = 1;
last;
}
}
if (!$valid) {
$c->log->warn("Webhook authentication failed for $platform: Invalid signature from IP " . $c->request->address);
$c->response->status(401);
$c->stash->{json} = { error => "Invalid webhook signature" };
$c->forward('View::JSON');
return 0;
}
return 1;
} else {
$c->log->warn("Webhook authentication failed for $platform: Unable to validate signature from IP " . $c->request->address . " because no secrets are configured");
$c->response->status(401);
$c->stash->{json} = { error => "Invalid webhook signature" };
$c->forward('View::JSON');
return 0;
}
}
sub push_github : Chained('api') PathPart('push-github') Args(0) {
my ($self, $c) = @_;
$c->{stash}->{json}->{jobsetsTriggered} = [];
return unless verifyWebhookSignature($c, 'github', 'X-Hub-Signature-256', 'sha256=');
my $in = $c->request->{data};
my $owner = $in->{repository}->{owner}->{name} or die;
my $owner = ($in->{repository}->{owner}->{name} // $in->{repository}->{owner}->{login}) or die;
my $repo = $in->{repository}->{name} or die;
print STDERR "got push from GitHub repository $owner/$repo\n";
@@ -297,6 +370,9 @@ sub push_gitea : Chained('api') PathPart('push-gitea') Args(0) {
$c->{stash}->{json}->{jobsetsTriggered} = [];
# Note: Gitea doesn't use sha256= prefix
return unless verifyWebhookSignature($c, 'gitea', 'X-Gitea-Signature', '');
my $in = $c->request->{data};
my $url = $in->{repository}->{clone_url} or die;
$url =~ s/.git$//;

49
t/Hydra/Controller/API/checks.t
View File

@@ -6,6 +6,7 @@ use Catalyst::Test ();
use HTTP::Request;
use HTTP::Request::Common;
use JSON::MaybeXS qw(decode_json encode_json);
use Digest::SHA qw(hmac_sha256_hex);
sub is_json {
my ($response, $message) = @_;
@@ -21,7 +22,13 @@ sub is_json {
return $data;
}
my $ctx = test_context();
my $ctx = test_context(hydra_config => qq|
<webhooks>
<github>
secret = test
</github>
</webhooks>
|);
Catalyst::Test->import('Hydra');
# Create a user to log in to
@@ -188,16 +195,20 @@ subtest "/api/push-github" => sub {
my $jobsetinput = $jobset->jobsetinputs->create({name => "src", type => "git"});
$jobsetinput->jobsetinputalts->create({altnr => 0, value => "https://github.com/OWNER/LEGACY-REPO.git"});
my $payload = encode_json({
repository => {
owner => {
name => "OWNER",
},
name => "LEGACY-REPO",
}
});
my $signature = "sha256=" . hmac_sha256_hex($payload, 'test');
my $req = POST '/api/push-github',
"Content-Type" => "application/json",
"Content" => encode_json({
repository => {
owner => {
name => "OWNER",
},
name => "LEGACY-REPO",
}
});
"X-Hub-Signature-256" => $signature,
"Content" => $payload;
my $response = request($req);
ok($response->is_success, "The API enpdoint for triggering jobsets returns 200.");
@@ -214,16 +225,20 @@ subtest "/api/push-github" => sub {
emailoverride => ""
});
my $payload = encode_json({
repository => {
owner => {
name => "OWNER",
},
name => "FLAKE-REPO",
}
});
my $signature = "sha256=" . hmac_sha256_hex($payload, 'test');
my $req = POST '/api/push-github',
"Content-Type" => "application/json",
"Content" => encode_json({
repository => {
owner => {
name => "OWNER",
},
name => "FLAKE-REPO",
}
});
"X-Hub-Signature-256" => $signature,
"Content" => $payload;
my $response = request($req);
ok($response->is_success, "The API enpdoint for triggering jobsets returns 200.");

209
t/Hydra/Controller/API/webhooks.t Normal file
View File

@@ -0,0 +1,209 @@
use strict;
use warnings;
use Setup;
use Test2::V0;
use Test2::Tools::Subtest qw(subtest_streamed);
use HTTP::Request;
use HTTP::Request::Common;
use JSON::MaybeXS qw(decode_json encode_json);
use Digest::SHA qw(hmac_sha256_hex);
# Create webhook configuration
my $github_secret = "github-test-secret-12345";
my $github_secret_alt = "github-alternative-secret";
my $gitea_secret = "gitea-test-secret-abcdef";
# Create a temporary directory first to get the path
use File::Temp;
my $tmpdir = File::Temp->newdir(CLEANUP => 0);
my $tmpdir_path = $tmpdir->dirname;
# Write webhook secrets configuration before creating test context
mkdir "$tmpdir_path/hydra-data";
# Create webhook secrets configuration file
my $webhook_config = qq|
<github>
secret = $github_secret
secret = $github_secret_alt
</github>
<gitea>
secret = $gitea_secret
</gitea>
|;
write_file("$tmpdir_path/hydra-data/webhook-secrets.conf", $webhook_config);
chmod 0600, "$tmpdir_path/hydra-data/webhook-secrets.conf";
# Create test context with webhook configuration using include
my $ctx = test_context(
tmpdir => $tmpdir,
hydra_config => qq|
<webhooks>
Include $tmpdir_path/hydra-data/webhook-secrets.conf
</webhooks>
|
);
# Import Catalyst::Test after test context is set up
require Catalyst::Test;
Catalyst::Test->import('Hydra');
# Create a project and jobset for testing
my $user = $ctx->db()->resultset('Users')->create({
username => "webhook-test",
emailaddress => 'webhook-test@example.org',
password => ''
});
my $project = $ctx->db()->resultset('Projects')->create({
name => "webhook-test",
displayname => "webhook-test",
owner => $user->username
});
my $jobset = $project->jobsets->create({
name => "test-jobset",
nixexprinput => "src",
nixexprpath => "default.nix",
emailoverride => ""
});
my $jobsetinput = $jobset->jobsetinputs->create({name => "src", type => "git"});
$jobsetinput->jobsetinputalts->create({altnr => 0, value => "https://github.com/owner/repo.git"});
# Create another jobset for Gitea
my $jobset_gitea = $project->jobsets->create({
name => "test-jobset-gitea",
nixexprinput => "src",
nixexprpath => "default.nix",
emailoverride => ""
});
my $jobsetinput_gitea = $jobset_gitea->jobsetinputs->create({name => "src", type => "git"});
$jobsetinput_gitea->jobsetinputalts->create({altnr => 0, value => "https://gitea.example.com/owner/repo.git"});
subtest "GitHub webhook authentication" => sub {
my $payload = encode_json({
repository => {
owner => { name => "owner" },
name => "repo"
}
});
subtest "without authentication - properly rejects" => sub {
my $req = POST '/api/push-github',
"Content-Type" => "application/json",
"Content" => $payload;
my $response = request($req);
is($response->code, 401, "Unauthenticated request is rejected");
my $data = decode_json($response->content);
is($data->{error}, "Missing webhook signature", "Proper error message for missing signature");
};
subtest "with valid signature" => sub {
my $signature = "sha256=" . hmac_sha256_hex($payload, $github_secret);
my $req = POST '/api/push-github',
"Content-Type" => "application/json",
"X-Hub-Signature-256" => $signature,
"Content" => $payload;
my $response = request($req);
is($response->code, 200, "Valid signature is accepted");
if ($response->code != 200) {
diag("Error response: " . $response->content);
}
my $data = decode_json($response->content);
is($data->{jobsetsTriggered}, ["webhook-test:test-jobset"], "Jobset was triggered with valid authentication");
};
subtest "with invalid signature" => sub {
my $signature = "sha256=" . hmac_sha256_hex($payload, "wrong-secret");
my $req = POST '/api/push-github',
"Content-Type" => "application/json",
"X-Hub-Signature-256" => $signature,
"Content" => $payload;
my $response = request($req);
is($response->code, 401, "Invalid signature is rejected");
my $data = decode_json($response->content);
is($data->{error}, "Invalid webhook signature", "Proper error message for invalid signature");
};
subtest "with second valid secret (multiple secrets configured)" => sub {
my $signature = "sha256=" . hmac_sha256_hex($payload, $github_secret_alt);
my $req = POST '/api/push-github',
"Content-Type" => "application/json",
"X-Hub-Signature-256" => $signature,
"Content" => $payload;
my $response = request($req);
is($response->code, 200, "Second valid secret is accepted");
};
};
subtest "Gitea webhook authentication" => sub {
my $payload = encode_json({
repository => {
owner => { username => "owner" },
name => "repo",
clone_url => "https://gitea.example.com/owner/repo.git"
}
});
subtest "without authentication - properly rejects" => sub {
my $req = POST '/api/push-gitea',
"Content-Type" => "application/json",
"Content" => $payload;
my $response = request($req);
is($response->code, 401, "Unauthenticated request is rejected");
my $data = decode_json($response->content);
is($data->{error}, "Missing webhook signature", "Proper error message for missing signature");
};
subtest "with valid signature" => sub {
# Note: Gitea doesn't use sha256= prefix
my $signature = hmac_sha256_hex($payload, $gitea_secret);
my $req = POST '/api/push-gitea',
"Content-Type" => "application/json",
"X-Gitea-Signature" => $signature,
"Content" => $payload;
my $response = request($req);
is($response->code, 200, "Valid signature is accepted");
if ($response->code != 200) {
diag("Error response: " . $response->content);
}
my $data = decode_json($response->content);
is($data->{jobsetsTriggered}, ["webhook-test:test-jobset-gitea"], "Jobset was triggered with valid authentication");
};
subtest "with invalid signature" => sub {
my $signature = hmac_sha256_hex($payload, "wrong-secret");
my $req = POST '/api/push-gitea',
"Content-Type" => "application/json",
"X-Gitea-Signature" => $signature,
"Content" => $payload;
my $response = request($req);
is($response->code, 401, "Invalid signature is rejected");
my $data = decode_json($response->content);
is($data->{error}, "Invalid webhook signature", "Proper error message for invalid signature");
};
};
done_testing;