Ultrareview from the terminal

The ultrareview CLI is a window onto the reviewer your organization already configured. It uses the same profiles, the same policy locks, the same taught notes and the same muted rules as a review triggered from GitHub, because it is the same reviewer.

Two ways to run it, and the difference is where the work happens.

The pr commands ask our servers. Fetching the diff, every model call and posting back to GitHub all happen on our infrastructure. Your machine sends three facts and gets a run id. You can close the laptop, the CI container can exit, and the review still lands on the pull request.

ultrareview review runs the reviewer here, on the branch in front of you, before there is a pull request to review. Your code does not leave the machine except to the model provider you configured, and your organization's settings are read over the API so the review still says what your pull request review will say.

Install

npm i -g ultrareview

That puts one command on your path, ultrareview. It needs Node.js 22 or newer, and nothing else: it is a single bundled file with no dependencies to resolve.

ultrareview --version

Set it up

export ULTRAREVIEW_TOKEN=urk_...
ultrareview doctor

Mint the token in the dashboard, under Configuration and then API tokens. Give it reviews:run if it will start reviews and reviews:read if it will read them. Neither implies the other, on purpose: reading a review that already happened costs nothing, and asking for a new one bills your model budget.

What doctor tells you

doctor is the first thing to run and the thing to run again whenever something stops working. It reports five checks at once rather than stopping at the first problem, because a tool that reports one fault, gets fixed, and then reports the next is a tool you run four times.

Everything checks out.
  ok   settings: base URL https://api.mobsession.ai (default), token urk_abcdefgh... (environment)
  ok   service: reachable, and this token is "ci" in acme
  ok   scopes: holds reviews:read, reviews:run
  ok   expiry: 83 days left
  ok   repositories: every repository in the organization

Three of those are worth knowing about before they surprise you.

Scopes checks for both. It wants reviews:read and reviews:run, because those are the two the CLI's own commands need. A token deliberately limited to reading will therefore fail this check and exit 7, even though every read it will ever do works perfectly. Read the named scope rather than the exit code when that is the setup you meant to have.

Expiry warns while the token still works. Inside seven days of expiry the check says so every time, because an expiry discovered as a failed call halfway through a job is the thing this exists to prevent. It is a warning rather than a failure: the token authenticates and every command runs, so the exit code stays zero. Once the token has actually expired the check fails and exits 4.

Repositories is never a failure. A token limited to some repositories is a good configuration. It is reported because a repository outside the limit answers exactly like one that does not exist, and this is the only place that difference is visible.

doctor also runs without a token, which is the state you are most likely in when you need it. It says the settings check failed and exits 4, rather than refusing to run at all.

Only a failure changes the exit code. That is what lets you put doctor in a pipeline at all, since a token minted with a short life would otherwise fail its own preflight from the day it was created.

Review what you are working on

ultrareview review

This is the one command that runs the reviewer on your machine. It reviews your branch against the merge base with your default branch, and it includes work you have not committed and files you have not added, which is the whole point: a pull request review cannot see any of that, and by the time it can, you have already asked people to look.

Before it spends anything on a model it prints what it is about to do, on standard error:

ultrareview review
  repository   acme/storefront      branch  feat/checkout-retry -> main
  changes      14 files, +412 -88 (9 committed, 5 uncommitted, 1 untracked)
  compared to  the merge base with main, a4f19c2. A review here is always a full look at the branch, never an update to the last one
  carries      no findings from any earlier review, so everything below is raised fresh
  settings     your team's: 1 profile, 12 notes, 47 muted rules, 2 policy locks
  reading      your team's notes, and this repository's own guidance files
  provider     policy not set, so any model your settings name may run here
  model        chosen by your team's settings, and run on this machine against your ANTHROPIC_API_KEY

Every line is a decision made on your behalf, printed so you can correct it. Two of them are there because a local review is genuinely not the same object as a pull request review, and pretending otherwise would be convenient and dishonest. There is no pull request, so there is no "last reviewed commit" to compare against and no earlier findings to carry: it is always a full look at the branch and always a first one.

What it needs

A model key on this machine, because the model call leaves from here on your own credential:

export ANTHROPIC_API_KEY=sk-...

OPENAI_API_KEY, GOOGLE_API_KEY, TOGETHER_API_KEY, GROQ_API_KEY and OPENROUTER_API_KEY work too, and setting several lets a configuration that mixes providers run here exactly as it runs hosted. If you have the claude CLI installed, ULTRAREVIEW_LLM=claude-cli runs the review through it instead. That is not free; it spends your Claude subscription's limits rather than an API key.

Without any of those, the command exits 6 and names every variable it accepts. Exit 6 means this machine, and it is never fixed by rotating a credential or by asking an administrator.

Signed out, it still runs

It does not refuse. It runs, and it says what it did not have:

  settings     your team's settings are not loaded, so this review will raise things your pull request review would not

That is the honest cost. Without your organization's settings, every rule your team has already muted fires again, its policy locks do not apply, and its notes are not read. Set ULTRAREVIEW_TOKEN and it becomes the review your pull request actually gets. The token needs the context:read scope, which is its own scope on purpose: a token that reads your repository's settings was never granted its notes and its muted rules.

If a token IS set and the settings cannot be read, the command refuses rather than quietly running a different review. --no-context runs it anyway, deliberately.

Options

ultrareview review --base develop          # compare against something else
ultrareview review --fail-on never         # report, never fail
ultrareview review --format json           # one object, for a hook or a script
ultrareview review --allow-incomplete      # accept a review that covered less than it should

--fail-on defaults to your repository's own merge policy: never when reviews are advisory, major when they block. That is deliberate, so the gate on your terminal and the check on your pull request never disagree about the same finding. Naming a severity yourself is allowed and is documented as being stricter than GitHub.

The command exits 3 when it covered less than it should: a reviewer that did not finish, a file it could not read, files a routing rule excluded, or findings that were never double-checked. Each one is named. A green check on a review that never read the code is worse than a red one, so that fails by default.

Ask for a review

ultrareview pr review --repo acme/storefront --pr 412

It returns as soon as the review is queued and prints the run id. A review takes minutes, and a command that blocks for minutes is a command nobody puts in a script, so waiting is opt-in:

ultrareview pr review --repo acme/storefront --pr 412 --wait

The repository is spelled the way GitHub spells it, owner/name, and matched without regard to case.

Asking twice about an unchanged commit serves the review that already happened, rather than paying for the same work again. Pass --force when you want it done again anyway.

Review through a different lens

ultrareview pr review --repo acme/storefront --pr 412 --profile strict-security

--profile picks the review lens by name, for this one review. Leave it out and the repository's own routing rules decide, which is what you want almost always: routing exists so that the security profile reads auth/ and the default profile reads everything else, without anybody choosing per run.

Run one review on a different model

ultrareview pr review --repo acme/storefront --pr 412 --model openrouter/stealth/ox-alpha

This changes the model and nothing else. The routing, the rule packs, the severity floors and the instructions stay exactly as configured, so the model is the only variable and "would the other model have caught this?" has an honest answer. Nothing is stored: the next review is back on your own configuration without anybody having to put it back.

Two limits worth knowing. The provider has to be one your organization holds a key for, and the command refuses immediately, with exit 7, rather than failing minutes into a run. And a profile your administrators have locked keeps the model they pinned, because a lock exists precisely so a path's review cannot be weakened from outside, and swapping the model is the easiest way to weaken one.

Follow one while it runs

ultrareview pr status --run <id>
ultrareview pr status --run <id> --wait

Without --wait it answers once and exits. With it, it keeps asking until the run ends, printing each state change (and only each change, not every poll) to standard error, so a build log gets a story rather than a heartbeat. --timeout bounds the wait and defaults to 15 minutes; --poll sets how often to ask and defaults to 5 seconds. Both take a duration like 20m, 90s or 500ms, and a bare number means seconds.

Running out of the timeout is a fact about the wait, never about the review. The command says so and exits 3; the review carries on, and pr status --run <id> later will tell you how it ended.

If the run was superseded, by a force from a colleague or a fresh push, it names the run that took over and a wait follows the pointer for you, up to three hops. That is the difference between "your answer moved" and "your answer is gone".

pr status reports what became of a request. It does not gate: a run that is still queued or running is exit 0 here, because "not finished" is a perfectly good answer to "what happened to my review". The command that gates is the next one.

Read what it found

ultrareview pr findings --run <id>
ultrareview pr findings --run <id> --fail-on critical

--fail-on sets the lowest severity that fails a build, defaulting to major. never reports without ever failing, which is what an advisory repository means.

This command has no --wait. It reads a verdict; it does not produce one. Wait with pr status --wait, or with pr review --wait, and then read.

Four ways it refuses, all of them deliberate:

A review of a different commit. If the review this run points at describes some other commit, it exits 3 and names both commits rather than reporting findings that are about other code. --allow-stale reads it anyway, and still says so. A build passing on a review of something else is the worst failure this tool could produce, so it is the one it works hardest to avoid.

A review that is still going. A review that has not finished has certified nothing, so exit 3, even when it is already carrying findings. A partial list is worth reading and is not worth trusting.

A review that lost coverage. Exit 3 again, even when nothing it found crosses your threshold. A review that certified less than it should cannot certify the pull request.

An empty list that is not good news. A skipped review, a failed run and a review still going all have zero findings for different reasons, so the run's state decides the exit code rather than the length of the list. A skip in particular is not a clean review: nothing was examined.

Find a review you did not start

Most reviews are triggered by a push, so nobody is holding a run id when the check goes red:

ultrareview pr history --repo acme/storefront --pr 412

Every review of that pull request, newest first, with what became of each one. It always exits zero: it lists history, it does not judge it, and the past is exactly where failures live. pr findings is the command that gates.

See what the reviewer has been doing

ultrareview reviews list
ultrareview reviews list --state failed
ultrareview reviews list --state running --limit 50

Recent reviews across the organization, newest first. --state takes any one of queued, running, complete, cached, failed, skipped, superseded or expired, and the two worth filtering on are the two questions worth asking of a list like this: what failed, and what is still going. --limit raises how far back a page reaches, up to 100, and the command says so when more exist than it showed.

One honest edge: an empty answer under a filter can mean the service read as far back as it reads in one page and everything in it was filtered out. It says that when it happens, rather than reporting "no failures" for a window that simply did not reach any.

How it prints

ultrareview reviews list --json
ultrareview pr findings --run <id> --format json

--json and --format json are the same thing. Every command prints one object, never a stream, so a parser never has to skip lines, and ok is derived from the exit code rather than set by hand, so a result cannot claim success while the process exits non-zero. The progress lines during a --wait go to standard error for exactly that reason.

GitHub code scanning

--format sarif emits a SARIF 2.1.0 log, so findings become native code-scanning alerts:

ultrareview pr findings --run "$RUN" --format sarif > ultrareview.sarif

then hand that file to github/codeql-action/upload-sarif. Severity maps the way code scanning expects: critical and major become errors, minor becomes a warning, and a nit becomes a note.

Upload it even when the review is clean. An empty run is what CLOSES alerts that are now fixed; uploading nothing leaves every previous alert standing as though the code never changed, so a finished clean review emits an empty log rather than no output.

For exactly the same reason, nothing else does. A review still going, a skipped one, a failed one and an unreachable service all have no findings for reasons that are not "the code is clean", and turning any of them into an empty run would close every open alert in the repository because the network was down. Those cases write a document that is deliberately not SARIF, carrying a line that says so, so an upload step fails loudly instead of succeeding destructively.

Only exit 0 and exit 1 produce SARIF, because those are the only two codes that mean a review of this commit was read end to end. Everything else is a reason there is no verdict, not a verdict of nothing. That is the whole guard, and running this in CI shows what it looks like in a workflow.

SARIF only makes sense from pr findings. Asking any other command for it gets you the same deliberately-not-SARIF document, since no other command produces a list of findings.

Exit codes

A build gate reads exit codes, not prose, so these are the load-bearing part of the tool.

Three separations are deliberate, and each one exists because collapsing it causes a specific mistake.

1 is not 3. "Your code has problems" and "the reviewer had problems" call for opposite responses, and a tool that returns the same code for both makes that decision impossible.

4, 6 and 7 are three different things. Rotating a credential never adds a permission it never had, and installing a model key on your machine does nothing about an organization that has not configured one. A pipeline retries a 4 after rotating a secret, and never retries a 6 or a 7.

3 fails by default, against the usual convention. A green check on a review that never read the code is worse than a red one. --allow-stale takes the other posture where it applies, deliberately, one case at a time.

Two smaller things a build will meet. A run id that does not exist is exit 2, not 3, and a run belonging to another organization reads identically to one that never existed, so a limited token cannot learn which runs exist by probing. And a state this build of the CLI does not recognise is exit 3, not 0: a newer server can name a state an older client has never heard of, and we cannot certify what we cannot read.

Every flag

An unknown flag is refused rather than ignored, and the refusal offers the nearest real flag when there is one. That is deliberate: a typo in --fail-on that silently fell back to the default would review with the wrong threshold and report success.

A command's own arguments are checked before the token is looked at, so a mistyped severity or a repository that is not owner/name is reported as what it is, rather than as a missing credential you would go and fix for nothing before meeting the real error on the next run.

Settings, and where they come from

Every setting can come from a flag, an environment variable, or a built-in default, in that order. When it is not doing what you expect:

ultrareview config explain

It names each setting and which layer supplied it. With no token at all it still tells you where the host came from, and exits 4 to say the token is the thing that is missing, because "why is my token not being picked up" is exactly when somebody runs this. It never prints the token itself, only the first twelve characters, which is the same prefix the dashboard shows so you can match one against the other.

The host only matters for a self-hosted installation. Everyone else can ignore it. There is no flag that takes a token or a key, and there never will be: a command line lands in shell history and in the process list.

In a git hook, or an agent's

A hook and a build script want different numbers from the same run, and they disagree about the meaning of 2.

The hook contract is 0 proceeds, 2 blocks, anything else is a non-blocking error. This CLI's table uses 1 for findings and 2 for "you used the command wrongly". Pass those straight into a hook and it does the opposite of what you meant, in both directions at once: a push carrying a critical bug exits 1 and sails through, while a typo in the command exits 2 and stops work that was fine.

So say which language you want:

ultrareview pr findings --run "$RUN" --fail-on major --hook-exit

The third row is the one worth reading twice. A tool that could not answer neither blocks nor passes: it reports. Blocking would stop your work because our service was down, and passing would let a broken tool read as a clean review, which is the failure that makes a gate worth nothing.

Git's pre-push hook needs one more step

The table above is the agent hook contract, where 2 blocks and 1 reports. Git is not that: a pre-push hook blocks on any nonzero status, so handing it 1 would stop your push because our service was unreachable, which is the exact behaviour this flag exists to prevent.

So translate once more, and only 2 may be nonzero:

# .git/hooks/pre-push
#!/usr/bin/env bash
ultrareview review --fail-on major --hook-exit
case $? in
  2)   exit 1 ;;    # the review says do not push this
  130) exit 130 ;;  # interrupted keeps the shell's own convention
  *)   exit 0 ;;    # clean, or the tool could not answer: it has already said so on your terminal
esac

The last line is the one worth reading twice. A tool that could not answer still printed why — you saw it — and it does not stop your push. Blocking there would make an outage of ours into a problem of yours, and passing silently would let a broken tool read as a clean review.

Pre-push, not pre-commit

A commit takes seconds and a review takes minutes, so a pre-commit hook turns every commit into a wait. A push is already the slow, deliberate step, and it is the last moment the code is still only yours.

Reading the notes a review used

A local review prints a line like this before it spends anything:

  settings     your team's: 1 profile, 12 notes, 47 muted rules, 2 policy locks

Twelve notes shaped that review. Until now there was no way to read one from here, and a claim nobody can inspect is a claim nobody can correct: a note that says the wrong thing keeps saying it on every review until somebody happens to open the dashboard.

ultrareview notes list --repo acme/storefront

Every note, with what it applies to. A note can be scoped to one profile, to a set of paths, or to neither, and the list says which:

Auth rules  the security profile, auth/*
Money       every profile, every path

"Every path" is spelled out on purpose. An absent path scope means the note applies everywhere, which is the opposite of how an ignore list reads, and a blank column there would be read as "nothing" by anybody who knows the other one.

Then read one in full:

ultrareview notes show --repo acme/storefront "Auth rules"

Capitalisation does not matter. If the heading is nearly right the command offers what you probably meant, and if two notes share a name it tells them apart by what each applies to rather than picking one.

Both need a token with the context:read scope, the same one a local review uses to load your team's settings. Notes are written in the dashboard; these commands read them.

Checking your repository's config file

A repository can carry an .ultrareview file that says which paths get reviewed, through which lens, and what the review is allowed to block. It is JSON, written by hand, and until now the first thing that told you a key was misspelled was a comment on the next pull request. That is the slowest possible place to learn it, and the most expensive: the review that told you already ran with the wrong settings.

ultrareview config validate

It reads .ultrareview in the current directory, or a path if you give it one, and answers the way the review itself would. No network, no login. It needs neither, because it is reading a file you already have.

Two kinds of answer, and the difference matters:

• Errors mean the review would refuse the file and run on built-in defaults instead. Exit code 2.
• Warnings mean the file is used, but part of it will not do what it appears to say. A routing rule naming a profile that does not exist quietly falls back to the default. A profile naming a rule pack that does not exist quietly gets no instructions. A key spelled blockingPolicyy is simply dropped. Each of those is a setting somebody wrote on purpose that never takes effect, and none of them stop the review, so the exit code stays 0 and the message says what will happen.

To start one:

ultrareview config init

It writes a small starter file and refuses to replace an existing one unless you pass --force, because the file it would overwrite is one somebody wrote by hand.

And to see the whole shape, including every setting and the values each one accepts:

ultrareview config schema

That prints a JSON Schema. Point your editor at it and it will complete the key names and flag the invalid values as you type.

What leaves your machine

The CLI never sends your source code to us. ultrareview review sends it to the model provider you configured, from your machine, on your credential. The settings call sends nothing but the repository in the URL. The pr commands send no code at all: our servers already have it through the GitHub App.

Seeing your local reviews in the dashboard

A local review can tell your organization that it happened, so the Local tab under Reviews shows it. This is off until an administrator turns it on, and while it is off nothing about a local review leaves your machine. The CLI reads that setting before it sends anything, rather than sending and letting the service decide, because those are not the same promise.

When it is on, what is sent is a count and an outcome: how many changed files were reviewed, how many findings at each severity, whether the review covered less than it should, how long it took, and what it was compared against. What is deliberately not sent is anything that could describe the work: no file paths, no finding text, no branch name, no commit, and no code. A local review runs on something you have not pushed, and a record naming the files you are editing would hand unreleased work to everyone in the organization.

A run is attributed to the person who minted the token it used, which is the honest claim available: possession of a token proves possession of a token, and the dashboard says so rather than implying more.

Who sees what is decided by the service, not by the page. An organization administrator sees everyone's local reviews; everybody else sees only their own. A list of when each colleague was reviewing which repository is a productivity monitor, and this is not one.

Related

• Ultrareview in CI: the GitHub Actions recipe, and the SARIF upload guard.
• API tokens: scopes, repository limits, and the API this CLI talks to.
• API reference: every endpoint, generated from the specification the API serves.