Implemented in ci-toolkit

Automatic Codex reviews on new PR commits

It is easy to push a fix and forget to request another review. ci-toolkit turns a pull request label into an ongoing request for review: each new head gets its own request when there is no existing request or current Codex verdict. The review gate then evaluates the resulting evidence.

Supported in the private ci-toolkit GitHub App. Runnerless itself is in early development; signing up does not enable this feature.

How it works

01 /

Opt in on the pull request

Apply codex: auto-review to an open PR. The App creates and normalizes this label and requests @codex review when the current head has neither a recorded request nor a current verdict. The label is a workflow control, not a claim that the review has completed.

02 /

Follow later heads independently

Push another commit and the new head receives an independent review request. Repeated webhook delivery does not mean that the same head should get an endless stream of duplicate requests. Existing current-head requests and verdicts are considered before another request is made.

03 /

Stop requesting future reviews

Remove the label to stop requests for later heads. Removing it does not cancel an already submitted request. Teams can also configure the deployment to hold newly opened PRs in progress for one review window.

A concrete example

Workflow example

Apply label: codex: auto-review

Head A → request review once
Head B → request review once
Remove label → no requests for later heads

Know the boundary

Current limits

This capability requests reviews through the existing Codex integration; it does not run an AI model in a one-second local-check budget. Availability, latency and model behavior belong to that integration. The current feature is in ci-toolkit, while general Runnerless AI hooks remain planned.

Common questions

Will removing the label cancel a review?

No. It disables requests for future heads while leaving an in-flight request alone.

Is requesting a review the same as passing the gate?

No. The gate separately checks trusted current-head evidence, unresolved threads and newer unanswered requests.

Runnerless / early access

Want this in
your workflow?

Get early access ↗

Register interest and tell us what slows your CI down. The optional follow-up questions come after your email is saved.