Valudio has the AI tool CodeRabbit take the first pass on every pull request in Azure DevOps. For a well-scoped pull request our Tech Lead now counts on about 15 minutes of review time, where every round added together used to take over an hour. Setup took three working days, the trial that followed two weeks. Most of the time went into Azure’s permission model.
In conversation with Dennis Quitaquís Tamay, Tech Lead at Valudio. Leda Felix asked the questions.
“A pull request isn’t something you can easily plan for. It’s completely asynchronous: the moment a developer finishes something or addresses our feedback, it needs to be reviewed almost immediately, or it becomes a bottleneck before QA can even start testing.”
Dennis Quitaquís Tamay, Tech Lead, Valudio
A pull request is code a developer submits for a colleague to check. It arrives when the work is done, not when the reviewer has time. At the time, three frontend and two backend developers were working at a high pace, and most backend pull requests landed with Dennis for the first pass.
Where the code review bottleneck came from
Every pull request went through two review phases: first whether the code was clean and met our standards, then whether what was built matched the user story. Depending on the quality of the first version, that meant zero, one or several rounds of feedback, correction and another review.
SonarCloud and Snyk ran alongside, but they cover different ground: risk and vulnerabilities, not design and logic. The first pass stayed manual, and QA waited for it.
The detour through the AI chat
Before any tool was involved, the team pasted code changes by hand into a general-purpose AI chat to have them read over. The answers were useful. The workflow stayed the same, with one more manual step, and the first pass still sat with Dennis. We cover why many AI initiatives stall at exactly this point in “Why AI working groups deliver nothing”.
Three criteria for the tool search
The review should start automatically as soon as a pull request opens. It should run natively in Azure DevOps, the backbone of our Microsoft environment. And it should handle .NET, Node, TypeScript and React.
Dennis ran the search with the team. Besides CodeRabbit, they looked at CodeAnt AI and Aikido:
| Tool | Why it fitted, or did not | Decision |
|---|---|---|
| CodeRabbit | Native integration with Azure DevOps, starts automatically when a pull request opens. Complements SonarCloud and Snyk rather than duplicating them. | Adopted |
| CodeAnt AI | Appears oriented toward Jira and GitLab, not Azure DevOps. Not tested in our environment. | Not pursued |
| Aikido | Not tested: the overlap with the security scans we already run through SonarCloud and Snyk settled it before we got that far. | Not pursued |
CodeRabbit was the first result in a search for an AI pull request tool for Azure. What made it stick was the two-week free trial on real pull requests from our own projects.
“Its integration is oriented mainly toward Microsoft, so every step of the setup involved things we already recognized and could execute ourselves. There wasn’t a single strange or unidentifiable step.”
Three days of setup, two weeks of trial
“The full integration, I’d say, didn’t take more than three continuous days: first finding the solution, second doing the testing, and third putting it into practice and configuring it.”
The three days were the setup: one day to find the right way to integrate, one to check that it runs, one to configure and roll out. After that, CodeRabbit ran for two weeks in the free trial on real pull requests before we talked about a budget.
Most of the setup time went into Azure’s permission model. We run many client projects in Azure DevOps, and the team first had to work out which user with which subscription had access to the projects to be reviewed. Then CodeRabbit received read access to the code, and it has commented on every new pull request since.
What changed for the team
Today CodeRabbit takes the first pass: it flags quality issues, suggests other variable names or recommends splitting a method. The developer decides what to take from it.
“Now we go directly to understanding the code, directly to the architecture, directly to the requirements of the user story. We look less at the detail and more at the design the developer chose for the solution.”
For a well-scoped pull request, Dennis now counts on about 15 minutes instead of over an hour. Both figures are his estimate from day-to-day work, not a measurement, and vary with the size and complexity of the pull request. Complex cases still get a conversation, about logic and hidden bugs. Nothing has piled up in front of QA since.
What the team had to get used to was the number of comments.
“The AI’s suggestions are entirely recommendable, but not obligatory, and that’s important for us. We can choose not to follow them. It can feel a bit overwhelming at first, seeing 10 comments appear on a single file you’ve changed. But those are 10 comments you don’t have to apply: you read them, interpret them, and then decide.”
What we take from it
The bottleneck was a single step, and the tool fit the environment we already run. We decided after two weeks on real pull requests. That is how we approach AI projects with clients too; here we were our own client. Where AI does not help in our daily work is the subject of “The downsides of AI in everyday development”.
Frequently asked questions
Does AI code review work with Azure DevOps?
Yes. CodeRabbit integrates natively with Azure DevOps and starts as soon as a developer opens a pull request. At Valudio it needed read access to the code. The largest effort was working out which user, under which subscription, actually reaches the client projects to be reviewed.
Does AI code review replace human reviewers?
No. At Valudio the AI takes the first pass: quality issues, naming, methods that have grown too long. Approval stays with the Tech Lead or a colleague and turns to architecture and whether the result matches the user story.
How long does it take to implement an AI code review tool?
At Valudio, three continuous working days of setup, followed by a two-week free trial on real pull requests before the budget decision. The tool itself was quick to set up; the time went into Azure’s permission model.
Can developers ignore the AI’s suggestions?
Yes. CodeRabbit’s comments are recommendations. The developer reads them, interprets them and decides what to apply. Ten comments on one file is a lot at first, but they are ten comments nobody has to apply.
Let’s talk about your bottleneck
If your team has a similar bottleneck, we are happy to talk it through. First call, 30 minutes. We will tell you honestly whether your plans and budget fit together, and what a discovery and concept phase would look like for you. We reply within 24 hours. Book a call.
