Developing an AI agent for code reviews

At Inventis, we developed an AI agent that reviews our pull requests. A developer can ask it to run a review, before a colleague looks at the code themselves. The same agent can also read tickets from Jira and prepare a technical answer.

A human always validates what the agent proposes. In this article we explain how we built it, what it delivers and where its limits lie.

Why we developed an AI agent

Code reviews are essential for code quality. They make sure the code meets our agreements and stays easy to maintain in the long run. But they also eat up a lot of time.

We already check our coding standards automatically with other tools. An AI agent is interesting for what such a tool doesn’t see: logic errors, slow database queries or pieces of code that have been written twice.

AI can’t be trusted blindly, though. That’s why we built an agent that helps us, while the decisions stay with our people. You can read more about that vision on our page Our view on AI.

An AI agent that knows what to look for

An AI model that reviews code without context gives generic feedback. That’s why our agent works with skills: rules for good code reviews, supplemented with the agreements that each project writes down itself.

What a tool already checks, the AI agent deliberately skips. It looks at what a tool doesn’t see:

  • pieces of code that have been written twice
  • logic errors
  • who is allowed to see and do what
  • database queries that are unnecessarily slow

An AI agent connected to Bitbucket

We connected the agent to Bitbucket, where our code lives. A developer starts the review themselves. So it doesn’t happen automatically.

The AI agent only places comments on the code if a developer explicitly asks for it. For each problem, it proposes a concrete fix. The developer decides what’s relevant and accepts a suggestion with one click. After that, the regular review by a colleague follows.

An AI agent connected to Jira

We then went one step further. We connected the agent to Jira, where we track tickets with questions from clients and tasks for our developers. The AI agent can read those tickets and compare them with the code of the project concerned. That produces a technical analysis for the developer, for example an overview of where in the code the question comes into play.

The agent also drafts a reply to the client. The project manager reads it through and sends it themselves. That way, nobody has to dig through the entire codebase for every first assessment.

What the AI agent doesn’t do

The agent doesn’t decide and doesn’t replace a reviewer. It doesn’t place comments unless someone asks for them, doesn’t change any code itself and doesn’t send anything to a client. Our developers stay responsible, and rightly critical.

What an AI agent delivers

Manual, repetitive work that used to take a lot of time, and wasn’t exactly a developer’s favourite task, is now largely taken over by the agent.

In our experience, it also spots things that reviewers often miss. A human builds up only limited context, a few levels deep at most. The agent sees the bigger picture and can unravel the whole thread. Our developers, in turn, have a better idea of the right architecture. That’s why they complement each other.

Our developers and project managers are left with more time for work where their expertise makes the difference: architectural choices, complex problems and talking things through with the client.

What we learned

The agent was sometimes sure of itself, but wrong. Every so often, the agent was convinced of an error that wasn’t there. This mostly happened with code from external parties that it couldn’t inspect. Now it has to verify every problem before reporting it. What it can’t check, it only raises as a question.

It has to wait until someone asks. At first, the agent placed comments without anyone asking for them. We switched that off. It now only responds when a developer asks.

Agreements per project. Projects can supply their own review agreements. These may only make the checks stricter.

More agents isn’t better. We tried having several agents do the same review at once. That didn’t produce better reviews, because our pull requests are already small. It even ran into usage limits. Now one agent does the full work of one review.

Frequently asked questions

What is an AI agent for code reviews?

An AI agent for code review reads the changes in a pull request and gives feedback on what a tool doesn’t see, such as logic errors, duplicate code, access rights and slow database queries. At Inventis, it only does this when a developer asks, and it proposes a fix for each problem.

Does an AI agent replace developers?

No. The agent takes over repetitive checking work. Architecture, security and final responsibility stay with our developers.

Is our code shared with an AI provider?

Yes, the files the agent needs are processed by an AI model from Google. We use Gemini through Google’s business cloud platform. By default, this runs in a European data centre (Belgium).

The agent only reads the files it needs. It doesn’t copy a project’s entire codebase. We keep a deleted conversation for another 30 days for analysis. After that, it is permanently erased.

Access credentials for Bitbucket and Jira are stored encrypted.

Curious what AI could mean for your organisation?

You don’t need a finished plan. Tell us where you’re losing time, and we’ll look together at whether AI can solve it. Sometimes the answer is yes. Sometimes the answer is that you’re better off not using AI.