How an Agent Reviews a Repository Before Adoption

The problem: useful code, missing proof

A team finds a library that appears to remove a substantial amount of implementation work. Its README shows the exact feature they need, the source is available, and the package can be downloaded. What is missing is evidence that it works inside the team’s actual system.

The existing application has a fixed runtime version, a mature dependency tree, asynchronous request handling, strict static checks, and a controlled deployment image. The candidate library documents its happy path but says little about concurrency, optional dependencies, version boundaries, or failure behavior.

The task for the agent is therefore not “add this library.” It is “determine whether this external repository can be introduced without creating an unmeasured compatibility risk.” The expected output is a review card, not an enthusiastic summary.

Define compatibility before inspecting code

Compatibility is contextual. A project may be healthy in isolation and still be unsuitable for a particular application. Before reviewing the candidate, the agent records the target environment:

• runtime and operating-system versions;
• package manager and installation mode;
• direct and transitive dependencies;
• required public API behavior;
• synchronous, asynchronous, or multi-process execution model;
• network, filesystem, and environment-variable restrictions;
• linting, type-checking, testing, and security gates;
• deployment architecture and rollback requirements.

This list becomes the acceptance boundary. Without it, repository review tends to drift toward superficial signals such as popularity, recent commits, or attractive documentation.

The concrete case

Assume the application needs a parsing component. A candidate package exposes a concise interface and could replace several internal modules. The application, however, requires a specific runtime range, processes untrusted input, runs concurrent jobs, and installs from a pinned lockfile.

The repository does not contain a compatibility table for this environment. No benchmark or test result is assumed. The agent must inspect what is present, create a minimal integration probe, and clearly distinguish observed facts from unresolved questions.

The review is performed in a temporary branch or isolated working copy. The production dependency files are not modified during initial inspection, and repository scripts are not executed before they are read.

Step 1: capture provenance and project shape

The first pass establishes exactly what is being reviewed. A branch name such as main can move; a commit identifier cannot. The agent records the repository URL, selected revision, package version if applicable, license file, and the date of review.

git clone --filter=blob:none --no-checkout "$CANDIDATE_URL" candidate
cd candidate
git checkout --detach "$CANDIDATE_COMMIT"
git rev-parse HEAD
git status --short
find . -maxdepth 2 -type f | sort | sed -n '1,200p'

The shallow structural scan looks for manifests, lockfiles, build definitions, test directories, release notes, generated files, examples, contribution instructions, and automation configuration. It also reveals whether the published package is likely to differ from the repository checkout.

At this stage, the agent does not classify missing files as defects. For example, the absence of a lockfile can be reasonable for a library. It is recorded because it affects how reproducible dependency resolution will be during evaluation.

Step 2: inspect declared compatibility

Next, the agent reads package manifests and build metadata rather than relying on the README alone. Important fields include runtime constraints, required and optional packages, platform markers, build backends, native extensions, feature flags, entry points, and excluded files.

# Examples; select commands appropriate to the project.
sed -n '1,240p' pyproject.toml
sed -n '1,240p' package.json
sed -n '1,240p' Cargo.toml
find .github -maxdepth 3 -type f -print 2>/dev/null

The agent compares these declarations with the target application and builds a small constraint table:

A continuous-integration matrix is useful evidence of project intent, but it is not automatically proof for the application. The agent checks which jobs are required, which are allowed to fail, and whether the relevant runtime-feature combination is actually exercised by the project’s continuous integration.

Step 3: trace the required path through the source

A full line-by-line audit is rarely necessary. The agent follows the path that the application intends to use:

1. Locate the exported symbol shown in the documentation.
2. Trace construction, validation, and the main operation.
3. Identify imports activated on that path.
4. Inspect global state, caches, background work, and cleanup.
5. Record exceptions, return values, timeouts, and retry behavior.
6. Check whether input limits are enforced or delegated to the caller.

rg "class Parser|def parse|export .*parse|pub fn parse" .
rg "subprocess|exec\(|eval\(|requests\.|fetch\(|open\(" src lib
rg "global|singleton|cache|thread|async|lock|timeout|retry" src lib

Search results are leads, not findings. A reference to exec in a test fixture is different from a production path that launches a shell. Each material concern must point to a file, symbol, and condition under which it becomes relevant.

Step 4: resolve dependencies without adopting them

The next question is whether the candidate can coexist with the application’s current graph. The safest first attempt uses a copy of the project metadata or the package manager’s dry-run and lockfile-checking facilities.

# Illustrative patterns; use the repository's actual package manager.
python -m pip install --dry-run "./candidate"
npm install --package-lock-only --ignore-scripts ./candidate
cargo tree --duplicates
cargo update --dry-run

Installation scripts deserve special care. Package managers may execute lifecycle hooks or build code during dependency resolution. The agent begins with scripts disabled when supported, reads any required hooks, and only then evaluates them inside an isolated sandbox with minimal permissions.

The evidence captured here includes version conflicts, unexpected optional packages, native build requirements, new network downloads, lockfile changes, and duplicate foundational libraries. A successful resolver run proves only that constraints can be solved; it does not prove runtime behavior.

Step 5: build the smallest meaningful integration probe

The probe should exercise the application boundary, not merely import the package. For the parsing case, that means using the same input representation, configuration shape, asynchronous wrapper, and error-handling contract expected in production.

# tests/compat/test_candidate_parser.py
import pytest

from integration.candidate_parser import CandidateParser

def test_valid_minimal_document():
    parser = CandidateParser()
    result = parser.parse(b"key=value")
    assert result.items == {"key": "value"}

def test_rejects_malformed_input():
    parser = CandidateParser()
    with pytest.raises(ValueError):
        parser.parse(b"\x00invalid")

def test_enforces_configured_size_limit():
    parser = CandidateParser(max_bytes=8)
    with pytest.raises(ValueError):
        parser.parse(b"more-than-eight-bytes")

The assertions above are a template, not claimed results. They must be adjusted to the application’s real contract and the candidate’s documented behavior. If the package raises a different exception, the adapter may translate it—but the review card should record that translation as application-owned code.

A useful probe normally covers the smallest valid input, malformed input, resource limits, repeated calls, cleanup, and the required concurrency mode. It should remain small enough to review and rerun after dependency upgrades.

Step 6: verify in layers

Verification proceeds from cheap, narrow checks to broader ones. A practical order is:

1. Confirm the checked-out revision and a clean working tree.
2. Build or install the candidate in a clean temporary environment.
3. Run the focused integration probe.
4. Run static analysis on the adapter and touched application code.
5. Run the application’s relevant subsystem tests.
6. Run the complete project test suite if adoption remains viable.
7. Build the real deployment artifact and inspect its dependency contents.

git diff --check
pytest -q tests/compat/test_candidate_parser.py
pytest -q tests/parser tests/compat
mypy integration/candidate_parser.py
ruff check integration tests/compat

# Then use the project's normal full verification command.
make test
make build

Record the exact commands, environment identifiers, exit status, and relevant output. Do not convert an unexecuted command into a check mark. If infrastructure prevents a test, mark it “not run” and state why.

Failure cases the review must expose

The package installs but cannot run

Optional native components, missing shared libraries, or imports activated only on the required code path can survive a basic installation check. A clean-environment integration probe exposes this gap.

The dependency graph resolves by replacing a critical version

A package manager may produce a valid graph that silently upgrades or downgrades a foundational dependency. The agent compares lockfile changes and reruns the affected subsystem rather than treating resolution as harmless.

The documented API differs from the selected revision

Examples may describe the latest release while evaluation targets an older tag or commit. Documentation, source, and installed artifact must refer to the same revision.

The happy path works, but failure behavior does not

Unexpected exception types, unlimited retries, absent timeouts, partial writes, or retained global state can make an otherwise functional library operationally unsafe.

Repository tests pass outside the application only

The candidate’s tests validate its assumptions. They do not validate interaction with the application’s dependency versions, adapters, runtime policies, or deployment image.

The review executes untrusted automation too early

Build hooks, task runners, test fixtures, and installation scripts are executable code. Reading their entry points and isolating execution is part of compatibility work, not a separate security exercise.

The external project review card

The final artifact should be short enough to use in a change review while retaining reproducible evidence.

A strong decision statement is specific: “Suitable for a bounded trial on the parsing path after the deployment-image build and concurrent-call probe pass.” A weak statement such as “The repository looks maintained” neither defines evidence nor constrains risk.

Limitations of this method

A repository review is a compatibility assessment, not a guarantee of correctness or long-term maintenance. Focused source tracing may miss behavior outside the intended path. Tests demonstrate behavior only for the examined inputs and environment. A clean dependency resolution does not prove supply-chain integrity, and visible source does not prove that a registry artifact contains identical files.

The method also cannot answer organizational questions by itself: whether the team can maintain an adapter, accept the license, respond to an abandoned dependency, or carry a private patch. Those require explicit owners and policy decisions.

For high-impact components, extend the card with artifact comparison, vulnerability review, performance measurement using representative workloads, and a maintenance plan. Keep these as separate evidence sections so that “not performed” remains visible.

What makes the review repeatable

The agent’s value is not that it can read a repository quickly. It is that it can preserve the chain from requirement to evidence to decision. The repository revision is immutable, commands are recorded, the integration probe is retained, unknowns stay visible, and recheck triggers are defined.

That turns “this library seems useful” into an engineering statement another person can challenge and reproduce. If the evidence is insufficient, the correct result is not forced adoption or automatic rejection. It is a precise review card showing what is known, what failed, and which next check would change the decision.