DEV Community

Cover image for 7 things to check before an AI coding agent touches your repo
spran wiki
spran wiki

Posted on Fully Autonomous

7 things to check before an AI coding agent touches your repo

If you let an AI coding agent work in a repository, it acts through some account or token. Whatever that identity can do, the agent can do. GitHub's docs say a personal access token has the same capabilities as its owner, limited by the scopes or permissions it was granted.

So the question is not whether the agent is smart. It is what your repository allows before anything reaches production. Here are seven settings worth checking on GitHub. Each one comes from GitHub's own documentation, and the docs do not mention AI agents. Applying them to agents is my advice. Availability varies by plan and repository visibility, so check your own settings.

Diagram titled Seven checks along the path of an agent change. Four stages left to right: Credential (check 5, fine-grained token limited to chosen repositories and permissions), Push (check 4, push protection blocks detected secrets, check who can bypass), Pull request (checks 1 to 3: required reviews, admin bypass rule scope, CODEOWNERS), Workflow run (checks 6 and 7: GITHUB_TOKEN permissions, environment rules and secrets). A box says to open repository settings, answer each check as on, off or not available on your plan, and fix the gaps. A note says each check covers one point on the path and no single check covers all of it, and that settings vary by plan and visibility.

1. Require pull request reviews on the default branch

With a branch protection rule, repository admins can require a number of approving reviews before a merge. GitHub's docs say that once required reviews are on, collaborators can only change the protected branch through an approved pull request.

You can also dismiss stale approvals when new commits change the diff. That matters if an agent keeps pushing to a branch after a human approved it.

2. Know who the default branch rule applies to

By default, a branch protection rule does not apply to people with admin permissions, or to custom roles with the "bypass branch protections" permission. You can opt in to applying it to them as well.

If the agent runs as an admin account, the default rule may not stop it. Run it as a lower-privilege identity, or turn on the option that includes admins. That second part is my advice.

3. Use CODEOWNERS for the files that matter

A CODEOWNERS file names the people or teams responsible for parts of the code. Per the docs, owners are requested for review automatically when a pull request touches their files. Admins can also require code owner approval before merging.

Two details from the docs are easy to miss. The file must exist on the base branch of the pull request. And owners must have write permission, or they are not requested.

4. Turn on push protection for secrets

Push protection blocks a push when it detects a secret, before the secret reaches the repository. The docs list partner patterns, generic patterns such as private keys and database connection strings, and custom patterns.

Two limits to note. Generic patterns have to be enabled in your security settings. And by default, anyone with write access can bypass a block by giving a reason. An agent that can write to the repository may be able to do the same, so review who can bypass.

5. Give the token the smallest reach you can

GitHub recommends fine-grained personal access tokens over classic ones whenever possible. Per the docs, a fine-grained token can be limited to specific repositories and given specific permissions, and organization owners can require approval for them.

The docs also say fine-grained tokens cannot do everything classic tokens can, so check that your use case works. If it does, a token for one repository with only the permissions the task needs is a smaller target than a classic token with broad scopes.

6. Limit what the workflow token can do

GitHub Actions gives each workflow run a GITHUB_TOKEN. The docs say an action can reach it through the github.token context even if the workflow never passes it in, and recommend limiting its permissions to the minimum needed.

You set this with a permissions block in the workflow. If an agent can edit workflow files, then the branch rules and reviews from checks 1 to 3 are what keep a permissions change from merging unseen. That connection is my reasoning, not a statement from the docs.

7. Keep deploy secrets behind an environment

An environment can have protection rules, and a job that uses it must pass them before it runs or reads the environment's secrets. Rules include required reviewers (the docs say up to 6, and one approval is enough), a wait timer, and limits on which branches can deploy. You can also prevent the person who triggered a run from approving it.

Two caveats from the docs. Environments on GitHub Free are only available for public repositories. And a workflow that references an environment that does not exist creates it with no protection rules, so create and configure it first.

What this adds up to

Each check covers a different point on the path: the credential, the push, the pull request and the workflow run. No single one covers all of it.

A short way to use this list: open the repository's settings and answer each of the seven with "on", "off" or "not available on our plan". The answers tell you what stands between an agent's change and your production secrets. Fixing the gaps can start with check 1.

Source pages: About protected branches, About code owners, Push protection, Managing your personal access tokens, Use GITHUB_TOKEN for authentication in workflows, Managing environments for deployment.

I write about startups and AI at startupwiki.substack.com.

Cover photo: Daniil Komov on Unsplash.

Top comments (0)