Opus 5.5 Code Review: Require Reproducible Findings

An Opus 5.5 code-review finding should identify a concrete behavior you can reproduce, not merely describe code that looks unusual. This workflow gives you an intentionally defective Python module, a review prompt and acceptance tests so you can practice separating a finding from an assertion.

Anthropic’s Opus 5.5 prompting guide discusses repository work and review. We checked it on September 24, 2026. This article tests a synthetic fixture locally; it does not report that Opus found its bugs, measure false-positive rates or compare models.

Freeze the repository and the review contract

Download the workflow lab. The relevant files are review_fixture.py, review-prompt.txt, review_tests.py and the empty findings.csv ledger. These files are deliberately small enough to inspect in full. Do not put the defective functions into production.

The review contract is explicit: pagination returns up to size items from offset; discounts outside 0–100 must be rejected; copying tags returns an independent shallow list. Those expectations matter. Without them, a reviewer might call a design choice a bug or overlook a behavior the application relies on.

Record a commit or archive hash, the exact model ID, provider, client version, effort and tool permissions. If you review through Claude Code, verify the active model using the access guide. A client label alone is not evidence that every request used the intended route.

Ask for evidence before asking for edits

Use the supplied review prompt for a discovery pass. It asks for a file and line, triggering input, expected and actual behavior, and a runnable reproduction. It also asks the reviewer to separate proven defects from hypotheses and to avoid editing during the initial pass.

For a discovery evaluation, hide the tests and expected answers until after the model produces its findings. Otherwise you are testing whether it can read an answer key, which is a different task. For an ordinary repair workflow, showing tests is reasonable; simply label the run as test-guided repair instead of independent discovery.

Treat repository comments, issue descriptions and pasted logs as untrusted material. A comment that tells the reviewer to upload secrets or ignore its task is not part of the authorized review. Restrict tools to the repository and actions actually needed for the exercise.

Reproduce the two seeded defects

Run the acceptance tests after collecting a review or when inspecting the fixture directly:

python3 review_tests.py

The command is expected to exit with a failing status. We ran it locally: two tests fail and one passes. A failed test here is the intended teaching result, not a broken release build.

The pagination slice ends one element too early. The discount function performs arithmetic without enforcing the stated range. The copy function satisfies this narrow shallow-copy requirement. That passing case does not prove the function is correct under every imaginable requirement, and calling it a deep-copy defect would be unsupported by this contract.

Grade findings, not the confidence of the prose

For each proposed issue, reproduce it against the frozen fixture and decide whether the stated requirement was violated. Keep three outcomes: confirmed, unsupported, or unresolved. Deduplicate multiple descriptions of the same root cause before counting findings.

Also record seeded defects the reviewer did not report. A review with one accurate finding and one missed defect is not complete just because every sentence it wrote was true. Likewise, long explanations do not compensate for a false trigger or an invented requirement.

Precision and recall require agreed labels and denominators. This tiny fixture can teach the calculation but cannot establish a model-wide score. Real repositories contain ambiguous intent, integrations, data migrations and concurrency that are absent here. Choose evaluation samples matching your actual review workload and preserve difficult or failed runs.

Repair separately and check for regressions

Make changes on a separate branch after adjudicating the findings. Correct the pagination endpoint and reject out-of-range discounts, then rerun the acceptance tests. Add boundary tests for empty lists, zero page size, and discount values at 0 and 100. Decide how negative offsets, non-integer discounts and invalid monetary inputs should behave before expanding the contract.

If an assistant writes the patch, review the diff and execute the relevant tests yourself. Passing only the supplied three cases does not demonstrate complete correctness. Keep the original fixture and test output so the published evidence remains reproducible after the repair.

Record limitations of the model and integration

The Opus 5.5 change notes describe integration changes and refusal handling. Do not assume a previous Opus configuration is interchangeable. Preserve actual stop reasons and incomplete runs instead of relabeling them as completed reviews.

For a paid evaluation, retain usage and all attempts alongside confirmed findings, missed defects, unsupported findings and human review time. The Sol/Opus cost comparison explains why a token-price comparison cannot prove review quality. Start with this reproducible exercise, then collect evidence on your own authorized repository before making a model-selection claim.