Building an AI Code Review Bot with Claude and GitHub Webhooks

Pull request review is the highest-leverage place to add AI to a software team. Every PR already triggers a webhook. Every PR already has a diff. The only thing missing is a reviewer that never gets tired, never skips files, and never forgets to check the obvious things null checks, missed edge cases, security smells, and tests for the new branches.

A well-tuned AI reviewer does not replace your senior engineers; it gives them back the hour they spent reading boilerplate so they can focus on the architectural calls that actually matter.

This guide builds a self-hosted AI code review bot using Laravel, GitHub webhooks, and Claude. It posts inline review comments on real PRs, scoped to the lines that actually changed. The whole thing is a few hundred lines of PHP plus a tight prompt no managed service, no SaaS subscription, no data leaving your infrastructure.

Why build it instead of buying it

The build-vs-buy answer flips around team size 20: smaller teams should buy and move on; larger teams have enough custom workflow (monorepos, deployment policies, security reviews) that a hosted product feels generic. This guide is for the second camp.

Architecture

GitHub PR opened/synced
        ↓
Webhook → Laravel → Verify signature → Queue ReviewPullRequestJob
        ↓
Fetch diff via GitHub API
        ↓
Filter to reviewable files → Send to Claude with structured output
        ↓
Post inline comments via GitHub Reviews API

Step 1 : Webhook endpoint with signature check

Never skip the signature check. A leaked webhook URL with no verification is a free RCE-shaped problem.

// app/Http/Controllers/GithubWebhookController.php
namespace App\Http\Controllers;

use App\Jobs\ReviewPullRequestJob;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Log;

class GithubWebhookController extends Controller
{
    public function __invoke(Request $request)
    {
        if (! $this->verifySignature($request)) {
            return response('invalid signature', 401);
        }

        $event = $request->header('X-GitHub-Event');
        $payload = $request->all();

        if ($event === 'pull_request' && in_array($payload['action'], ['opened', 'synchronize'])) {
            ReviewPullRequestJob::dispatch(
                repo: $payload['repository']['full_name'],
                prNumber: $payload['pull_request']['number'],
                headSha: $payload['pull_request']['head']['sha'],
            );
        }

        return response()->noContent();
    }

    private function verifySignature(Request $request): bool
    {
        $secret = config('services.github.webhook_secret');
        $signature = $request->header('X-Hub-Signature-256', '');
        $expected  = 'sha256=' . hash_hmac('sha256', $request->getContent(), $secret);

        return hash_equals($expected, $signature);
    }
}

Step 2 : Fetch the diff

GitHub returns diffs in unified format when you request the right Accept header. We will also pull file metadata so we can filter out vendor and lock files.

// app/Services/Github/PullRequestClient.php
namespace App\Services\Github;

use Illuminate\Support\Facades\Http;

class PullRequestClient
{
    public function files(string $repo, int $pr): array
    {
        return Http::withToken(config('services.github.token'))
            ->acceptJson()
            ->get("https://api.github.com/repos/{$repo}/pulls/{$pr}/files", ['per_page' => 300])
            ->throw()
            ->json();
    }

    public function postReview(string $repo, int $pr, string $sha, array $comments, string $summary): void
    {
        Http::withToken(config('services.github.token'))
            ->acceptJson()
            ->post("https://api.github.com/repos/{$repo}/pulls/{$pr}/reviews", [
                'commit_id' => $sha,
                'event'     => 'COMMENT',
                'body'      => $summary,
                'comments'  => $comments,
            ])
            ->throw();
    }
}

Step 3 : Filter what is worth reviewing

Most PRs include lockfiles, snapshots, and generated migrations. Reviewing them wastes tokens and produces noise.

// app/Services/Review/DiffFilter.php
namespace App\Services\Review;

class DiffFilter
{
    private const SKIP_PATTERNS = [
        '#^vendor/#', '#^node_modules/#', '#composer\.lock$#', '#package-lock\.json$#',
        '#\.svg$#', '#\.snap$#', '#dist/#', '#public/build/#',
    ];

    public function reviewable(array $files): array
    {
        return collect($files)
            ->filter(fn ($f) => in_array($f['status'], ['added', 'modified']))
            ->filter(fn ($f) => ! $this->skipped($f['filename']))
            ->filter(fn ($f) => ($f['changes'] ?? 0) > 0 && ($f['changes'] ?? 0) < 800)
            ->values()
            ->all();
    }

    private function skipped(string $path): bool
    {
        foreach (self::SKIP_PATTERNS as $pattern) {
            if (preg_match($pattern, $path)) return true;
        }
        return false;
    }
}

Step 4 : Strict structured-output prompt

The model must return inline comments anchored to specific lines. We force JSON output with a clear schema, then validate before posting anything to GitHub.

// app/Services/Review/Reviewer.php
namespace App\Services\Review;

use Anthropic\Anthropic;

class Reviewer
{
    public function __construct(private Anthropic $client) {}

    public function reviewFile(array $file): array
    {
        $prompt = <<<TXT
        Review this diff. Return ONLY JSON, no prose.

        File: {$file['filename']}

        Patch:
        {$file['patch']}

        Schema:
        {
          "summary": "1-2 sentence overall verdict",
          "comments": [
            {
              "line": <line number from the new file>,
              "severity": "info|warn|block",
              "category": "bug|security|performance|style|tests",
              "body": "<= 280 chars, focus on the fix"
            }
          ]
        }

        Rules:
        - Only comment on lines that appear with a leading + in the patch.
        - Skip cosmetic feedback unless it hides a bug.
        - Prefer one strong comment over five weak ones.
        - If the change looks fine, return an empty comments array.
        TXT;

        $response = $this->client->messages->create([
            'model'      => 'claude-sonnet-4-6',
            'max_tokens' => 1500,
            'system'     => 'You are a senior engineer doing focused PR review.',
            'messages'   => [['role' => 'user', 'content' => $prompt]],
        ]);

        $text = $response->content[0]['text'] ?? '{}';
        $data = json_decode($text, true) ?: ['summary' => '', 'comments' => []];

        return $this->validate($data, $file);
    }

    private function validate(array $data, array $file): array
    {
        $valid = collect($data['comments'] ?? [])
            ->filter(fn ($c) => isset($c['line'], $c['body'])
                && is_int($c['line'])
                && mb_strlen($c['body']) <= 280)
            ->map(fn ($c) => [
                'path' => $file['filename'],
                'line' => $c['line'],
                'side' => 'RIGHT',
                'body' => "**[{$c['severity']} · {$c['category']}]** {$c['body']}",
            ])
            ->values()
            ->all();

        return ['summary' => $data['summary'] ?? '', 'comments' => $valid];
    }
}

Step 5 : The review job

// app/Jobs/ReviewPullRequestJob.php
namespace App\Jobs;

use App\Services\Github\PullRequestClient;
use App\Services\Review\DiffFilter;
use App\Services\Review\Reviewer;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class ReviewPullRequestJob implements ShouldQueue
{
    use Queueable;

    public int $timeout = 600;
    public int $tries = 2;

    public function __construct(
        public string $repo,
        public int $prNumber,
        public string $headSha,
    ) {}

    public function handle(PullRequestClient $gh, DiffFilter $filter, Reviewer $reviewer): void
    {
        $files = $filter->reviewable($gh->files($this->repo, $this->prNumber));

        $allComments = [];
        $summaries = [];

        foreach ($files as $file) {
            $result = $reviewer->reviewFile($file);
            $allComments = array_merge($allComments, $result['comments']);
            if ($result['summary']) {
                $summaries[] = "**{$file['filename']}** — {$result['summary']}";
            }
        }

        $body = "## AI review\n\n" . (empty($summaries)
            ? "_No blocking issues found._"
            : implode("\n", $summaries));

        $gh->postReview($this->repo, $this->prNumber, $this->headSha, $allComments, $body);
    }
}

Step 6 : Skip and trust controls

Two flags every team will ask for within a week:

• [skip ai-review] in the PR body or commit message bypass the bot.
• Allow-list by repository or label roll out gradually instead of flooding every PR with comments on day one.

if (str_contains($payload['pull_request']['body'] ?? '', '[skip ai-review]')) {
    return response()->noContent();
}

if (! in_array($payload['repository']['full_name'], config('review.enabled_repos'))) {
    return response()->noContent();
}

Severity rubric

Without a shared rubric, the bot's "warn" feels the same as its "block" and the team starts ignoring everything. Encode the rubric in the prompt and surface it in every comment:

Cost per PR : back-of-envelope

Most PRs touch 4 to 8 files with under 200 changed lines. A typical per-file review prompt fits in 2 to 4K input tokens and produces 300–600 output tokens. That works out to a few cents per PR orders of magnitude cheaper than the engineer-minutes saved.

• Batch trivial files. If five files have under 20 changed lines combined, send them in one prompt with file headers.
• Cache the system prompt. Your reviewer rubric and instructions are static Anthropic's prompt cache cuts that cost by ~10x on repeat calls.
• Skip dependency bumps. A 30,000-line lockfile change is not what the AI is good at; defer those to Renovate or Dependabot's own checks.

Metrics worth tracking

Lessons from running this in production

• One file at a time beats one giant prompt. Per-file calls are cheaper, parallelizable, and produce sharper feedback. Long single-prompt reviews dilute attention across files.
• Limit comments per file to 3. Long PRs already overwhelm humans. The bot should be pointed, not noisy.
• Always tag severity. Engineers triage [block] first, ignore [info] last. No severity = read by nobody.
• Track resolution rate. If >60% of comments are dismissed without changes, the prompt is too aggressive tune it down.
• Cache by file SHA. If the same file blob has been reviewed before, reuse the result instead of paying for it twice. PRs often re-push the same files after rebasing.
• Run as a separate GitHub App. A dedicated bot identity makes its comments distinguishable from human reviewers and easy to filter in notifications.
• Add a "ask the bot" command. Let authors reply with /ai explain to get more context on a comment same backend, different prompt.

The best AI reviewer is the one your team forgets is AI. It catches things, it suggests fixes, and it stays quiet when the change is fine.

This bot is a few hundred lines of Laravel and one well-tuned prompt. It is also the AI feature that produces the most visible engineering ROI in the shortest time.