Agent reviewers skim. On a 40-file change, a general agent reads part of the diff, cites moved lines, and answers differently when the prompt changes. Open Code Review, Alibaba’s Apache-2.0 CLI, keeps the counting in code and leaves the model the reading. It claims higher precision than Claude Code on the same model, about one ninth of the tokens, lower recall on purpose.
Wire a hybrid reviewer in three files
copy-readyThe workflow, the rule file and the local commands, copied from the project’s own action.yml, rule.json and quick start. Each tab names the file it belongs in.
name: ai-code-review
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
review:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: alibaba/open-code-review@main
with:
llm_url: ${{ secrets.OCR_LLM_URL }}
llm_auth_token: ${{ secrets.OCR_LLM_AUTH_TOKEN }}
llm_model: ${{ vars.OCR_LLM_MODEL }}
llm_use_anthropic: ${{ vars.OCR_LLM_USE_ANTHROPIC }}
rule: .opencodereview/rule.json
effort: medium
max_tokens_budget: "400000"
route_severity_below: low
sticky_summary: "true"
incremental: "true"{
"rules": [
{
"path": "src/api/handlers.ts",
"rule": "Every changed handler must validate the request body with the schema in src/schemas before it reaches the database. Flag any handler that reads req.body directly. Flag string-built SQL as a security finding.",
"merge_system_rule": true
},
{
"path": "db/migrations",
"rule": "A migration that drops or renames a column needs a follow-up migration that keeps the old column readable for one release. Every migration needs a working down path.",
"merge_system_rule": true
},
{
"path": "src/billing/invoice.ts",
"rule": "Money is integer minor units. Flag any float, any rounding call and any currency string that reaches a database column.",
"merge_system_rule": true
}
]
}npm install -g @alibaba-group/open-code-review
ocr config provider # pick a provider, paste the key, test the connection
ocr config model
# working tree, a branch range, or one commit
ocr review
ocr review --from main --to HEAD
ocr review --commit 4f2c9ab
# whole files, where there is no usable diff
ocr scan --path internal/billing
# hand the review to your own coding agent: no OCR model key needed
ocr delegate previewpull_request_target is used because secrets are not exposed to pull_request runs started from a fork. The action reads git objects for the base-to-head diff and does not execute code from the pull request. Paths in the rule file follow the shape of the project’s own rule.json: an exact file or a directory. max_tokens_budget, effort and route_severity_below are action inputs, not new CLI flags.
What the repository actually ships
The repository was two years old inside Alibaba before it was published, and the public shape is odd for an AI tool. Most of what decides a review is not the model. The CLI is a single binary called ocr, installed from npm, and it drives git itself.
| Item | What it is (read 15 November 2026) |
|---|---|
| Licence | Apache-2.0, free forever. The model bill stays yours. |
| Install | npm install -g @alibaba-group/open-code-review |
| Command | ocr — review, scan, session, config, delegate |
| Git requirement | 2.41 or newer, because diff, search and repository reads go through git |
| Model endpoint | OpenAI-compatible or Anthropic protocol; one provider setup, or none in delegation mode |
| CI recipes in the repo | GitHub Actions, GitLab CI, Bitbucket Pipelines, Gerrit, Codeup, GitFlic |
| Agent plugins | Claude Code, Codex, Cursor, Kimi Code, OpenCode, QCA Forward |
| Benchmark set | AACR-Bench: 50 repositories, 200 pull requests, 10 languages, 1,505 annotated issues |
| Action timeouts | 15 minutes per review task, 300 seconds per model request, both inputs you can change |
| Version floor | The action requires v1.9.6; effort presets need v1.10.0 |
action.yml and README, read on 15 November 2026. Star and issue counts move daily.Three ways a prompt-only reviewer loses the plot
Give a general agent a diff and a skill file and every important decision becomes a language decision: which files deserve reading, how much of each file fits, where a finding sits. That holds on a three-file change. It frays on a forty-file one.
Coverage goes first. A large change does not fit one context, so the agent selects. Selection driven by the model is invisible: nothing in the output says which files were never opened. You get a confident review of the half it read.
Position drifts second. Line numbers are produced after the model has already reasoned, from tokens that describe a position rather than point at one. Comments land three lines low, or on the wrong hunk, and the reviewer spends the first minute of every comment looking for the code.
Stability fails third. Natural-language rules cannot be debugged. Change one sentence in the skill file and the set of findings changes, which makes a review impossible to tune and impossible to trust between runs.
| Failure | What the reviewer sees | Why a better prompt does not fix it | What the hybrid layer does instead |
|---|---|---|---|
| Partial coverage | Confident comments on 12 of 40 changed files | The model decides what to read, and the decision is silent | File selection runs in code and the run can report what it skipped |
| Comment drift | Line three low, or on a neighbouring hunk | Position is generated text, not a lookup | A positioning module relocates each finding before it is posted |
| Unstable output | A different set of findings after a prompt edit | Prompt text is the configuration, and it is not testable | Path-matched rules from a template, so the input to the model is fixed |
| Runaway context | A long review that ends half-read | Nothing stops it, so it truncates quietly | Bundles bound each unit, and a token budget can cap the run |
The split, step by step
action.yml. Six of the eight stages run in code; only the sub-agents are the model.Read the pipeline as a list of decisions, and the design argues itself. A decision that must be right goes into code. A decision that needs judgement goes to the model. That is the whole idea, and it is the part worth copying even if you never install this tool.
File selection is a function of the diff, not of attention. Bundling groups files that change together — an English properties file and its Chinese pair become one unit — so each unit fits comfortably and the agent sees both sides of a translation change.
Rule matching pulls review rules per path from a template. The agent then reads the rules that apply to this bundle, not a repository-wide prompt listing forty concerns, which is where attention spreads thin.
Only then does the model run, once per bundle, in isolated context, concurrently. Each sub-agent can open whole files, search the repository and read sibling diffs, which is what separates a deep review from a comment on a changed line.
Two more code stages close the loop. Positioning rewrites each finding to a file and line. Reflection decides whether the comment survives. What surprised me is how far down the list the model sits: by the time it reads anything, the questions of scope and rules are already answered.
The rule file is the real configuration surface
Every review tool eventually needs your house rules: money is integer minor units, migrations must be reversible, handlers must validate input. In a general agent those rules live in one long instruction file. Here they live in a JSON file keyed by path.
The format is small. Each rule has a path (an exact file or a directory), the rule text, and merge_system_rule, which keeps the built-in ruleset alongside yours. The built-in set covers the defects that survive translation between languages: null dereferences, thread-safety, cross-site scripting, SQL injection.
The consequence is boring in a good way. A change under db/migrations/ is reviewed against migration rules, and a change under src/api/ against handler rules, without either set diluting the other. Tuning a rule is editing a string in version control, so a bad rule shows up in a diff.
I noticed one caveat: a path rule fails quietly when a repository reorganises, and a rule that matches nothing looks identical to a rule that matches everything and finds nothing. Check the coverage report after any directory move.
What the benchmark says, and what it does not
“Compared to general-purpose agents (Claude Code), Open Code Review achieves significantly higher Precision and F1 with the same underlying model, while consuming only ~1/9 of the tokens and completing reviews faster. Note that its Recall is lower than general-purpose agents — a deliberate trade-off favoring precision over noise.”
Open Code Review README, Alibaba, read 15 November 2026 — repository
That paragraph is the argument and the caveat in one place, which is rare enough to quote in full. Precision against tokens is the trade the project chose. Recall, the share of real defects found, is the price.
| Claim | Where it comes from | How much weight to give it |
|---|---|---|
| Higher precision and F1 than Claude Code on the same model | Project README | Vendor-run comparison. Not reproduced on this site, and the test rig is theirs. |
| About one ninth of the tokens | Project README | Same caveat. Directionally plausible for a pipeline that reads fewer, smaller units. |
| Lower recall, by design | Project README | Expect to miss defects a chatty general agent would have flagged. |
| 50 repositories, 200 pull requests, 10 languages, 1,505 issues | AACR-Bench, on Hugging Face | A real-world set, cross-checked by 80+ senior engineers, still assembled by the vendor. |
| Two years inside Alibaba, millions of defects found | Project README | Not verifiable from outside. Read it as provenance, not evidence. |
A benchmark on 200 pull requests across 10 languages says little about your repository in your language. The honest use of it is comparative: the same model, two runners, one of them cheaper and more precise. That is a reason to test, not a reason to believe.
What 200 pull requests a month costs in tokens
Token cost is where most teams expect the fight, so put a number on it. At 200 reviews a month, a general agent assumed at 45,000 tokens per review burns 9 million tokens. The hybrid at 5,000 tokens per review burns 1 million, which is the vendor's nine-to-one ratio applied as an illustration rather than a measurement.
| Configuration (modelling assumption) | Tokens / month | At $0.60 per M | At $1.50 per M | At $4.00 per M |
|---|---|---|---|---|
| General agent, 45,000 tokens per review | 9.0 M | $5.40 | $13.50 | $36.00 |
| Hybrid pipeline, 5,000 tokens per review | 1.0 M | $0.60 | $1.50 | $4.00 |
| Difference per month | 8.0 M | $4.80 | $12.00 | $32.00 |
At list prices, tokens never decide this purchase. Thirty-six dollars a month does not justify a migration. Reviewer attention does. The cost difference between the two configurations is $32 a month at the top price in Table 4, so the ROI case rests on reviewer minutes: if one false positive costs three minutes, forty a week is two hours of someone's week, every week.
Wire it in before the next review round
Three files get you a working reviewer: the workflow, the rule file, and the secret that holds the model key. The workflow above needs four values — endpoint URL, token, model name and protocol — and nothing else to produce inline comments on the next pull request.
Then choose the review scope deliberately. The action defaults to reviewing from the merge-base on every push. Cross-push checkpointing exists as an option and is fail-closed: if anything about the previous checkpoint is doubtful, it reviews the full range instead of a shortcut. Turn it on after the first week, when you trust the run.
For local work the CLI is simpler than the CI path. ocr review covers the working tree, a branch range covers the pull request, a commit covers one bad merge, and ocr scan covers a directory with no usable diff, which is what an inherited codebase needs.
Is your reviewer actually reading the diff?
self-checkEight checks taken from the three failure modes the hybrid design targets: partial coverage, comment drift and results that change when the prompt changes. Tick what your setup already does.
Solid: the deterministic layer does the counting and the model does the reading. Measure precision next, and fix recall only where a miss is expensive.
Weights follow blast radius, not frequency: coverage and rule selection decide whether a defect is found at all. Checks 1, 6 and 8 are the ones a general-purpose agent setup usually fails.
Who should buy it, and who should skip it
Run it if you review more than a handful of pull requests a week with a small team, and if a missed defect is expensive. The deterministic layer pays for itself the first time a large change gets a complete review instead of a partial one, and the path rules give a solopreneur the same house standard a larger team writes into a checklist.
Skip it if your changes are small and few. A three-file pull request is where a general agent already does well, and a second tool adds a model key to rotate and a rule file to maintain. Skip it too if you cannot send code to a third-party model endpoint, unless you point ocr at a self-hosted model.
The decisive reason is not speed. It is that coverage becomes a number you can read in a log, and a review you cannot measure is a review you cannot improve.
The Honest Catch
Recall is the catch, and the project says so before you find out. A precision-first reviewer leaves real defects for the humans, which is fine if humans still read the diff and dangerous if the bot becomes the only reviewer.
The benchmark is theirs. No independent replication exists for this comparison, and a vendor-curated set of 200 pull requests cannot promise the same ratio on your languages and your code style.
The project is young in public: 341 open issues at the time of reading, an action that requires a recent version, and inputs like effort that only arrived in v1.10.0. Pin a tag rather than @main in production, and read the release notes before you move.
Your code leaves the machine. In the default mode it goes to whichever provider you configured, so a repository with secrets in the diff has a new place to leak from. Delegation mode changes who pays for the model, not who sees the code.
Frequently asked questions
What is Open Code Review?
An Apache-2.0 command line tool from Alibaba that reviews git diffs with a language model, published after two years as an internal assistant. It installs from npm as @alibaba-group/open-code-review and runs as ocr.
How is it different from reviewing with Claude Code or Cursor?
The model does less. File selection, bundling, rule matching, positioning and reflection run in code, and the agent only reads each bundle. The project reports higher precision at about one ninth of the tokens, with lower recall.
Do I need an API key?
In the default mode, yes: one provider configuration for an OpenAI-compatible or Anthropic endpoint. In delegation mode the host coding agent uses its own model, so the CLI needs no key of its own.
What does it cost?
The tool is free. You pay your model provider. At 200 reviews a month the token bill in Table 4 runs from $0.60 to $36 depending on the token count and the price, which is small next to the reviewer hours it saves.
Does it work with GitLab, Gerrit and Bitbucket?
Yes. The repository ships CI recipes for GitHub Actions, GitLab CI, Bitbucket Pipelines, Gerrit, Codeup and GitFlic, plus plugins for Claude Code, Codex, Cursor, Kimi Code and OpenCode.
Can it review a codebase with no useful diff?
Yes, with a different command. ocr scan --path internal/billing reviews whole files instead of changed lines, which is what an audit of unfamiliar code needs.
Why is recall lower?
By design. The pipeline prefers comments that are right over comments that are numerous, and a wide-net reviewer that flags more real defects also floods the diff with noise. If a miss is expensive in your codebase, tune rules per path and keep a human pass on security-sensitive directories.
Review tooling is only half of the bill an agent stack creates. The seat-versus-agent-message teardown covers what a chat platform charges once agents join the room, the n8n queue-mode failure write-up shows the runtime side of the same automation, and the Vercel versus Cloudflare comparison prices the hosting underneath both.
Methodology & Disclosure: Repository metadata, the licence, the CLI commands, the action inputs and the rules-file format were read on 15 November 2026 from the alibaba/open-code-review repository, its action.yml, the GitHub Actions example, the project's own rule file and the AACR-Bench dataset card. Token counts per review, the nine-to-one ratio applied to them, the blended prices in Table 4 and the three minutes per false positive are modelling assumptions, printed beside their inputs so you can replace them. Both figures are generated by scripts/make_ocr_review_figures_20261115.py from data/ocr_review_20261115.json, so no number in a table was typed by hand.
No first-person production testing is claimed: the claims here come from published documentation, not from a review run on this site's repositories. No vendor paid for placement, and any partner link on this publication carries rel="sponsored nofollow". Review rules live in the Editorial Methodology and verified corrections are logged in the Corrections Log.
Leave a Reply