Detecting failures in software development is no easy task. As the code grows more complex, requirements expand, and the number of people involved increases, the probability of new bugs and edge cases appearing also increases.
Ubisoft is known (apart from major franchises like Assassin's Creed, Far Cry, Prince of Persia, or Tom Clancy) for the number of bugs that slip into their games (numerous videos on this can be found on YouTube; some people call them "Bugisoft").
This is why they are working hard to increase the technical quality of their games. At the latest Ubisoft conference in Montreal, the company presented a new AI-powered assistant for their developers.
The Assistant: Commit Assistant#
Ubisoft Montreal's R&D division, La Forge, has trained the assistant (called Commit Assistant) with nearly 10 years of code extracted directly from their repositories, causing it to learn from past mistakes and theoretically reducing the number of bugs that reach the final testers.
The premise of the assistant is very simple: if it automatically detects patterns in the code similar to those that occurred in the past, this indicates that the developer has introduced (or is about to introduce) a bug similar to one that was already fixed in the past. In the words of division director Yves Jacquier:
It's all about comparing lines of code that we've created in the past, the bugs that were created in them, the bugs that were corrected, and finding a way to link everything together to provide a super AI to developers.
Ubisoft hopes to reduce one of the most costly tasks in video game development with this assistant. The company claims that eliminating bugs during the development phase requires a lot of staff and can account for 70% of project costs (here I assume these are not fixed costs, but rather the more errors there are to correct, the longer development takes, and therefore more salaries have to be paid, additional staff hired, etc.).
That said, Commit Assistant is still at a very early stage. A great deal of computing power and a lot of data are needed to make increasingly accurate predictions and to iterate on the mathematical models to make them more effective. Ubisoft is beginning to use it in small development groups, but there is still no data on the real impact it is having on the development of their video games.
On top of all this there is also the human factor: will developers look favorably upon an artificial intelligence telling them "what you are programming has a bug"? Ubisoft wants developers to see the assistant as a help rather than a problem. In Jacquier's words:
We want that when we show a programmer statistics saying "hey! apparently you are creating a bug," they are able to understand that it is a tool to help them go faster. We have conceived it as just another tool. If you don't want to use it, don't; it's just another tool.
The company is betting on artificial intelligence as a method to speed up development processes. In addition to Commit Assistant, they have also developed other programs for simpler tasks such as preventing agents from colliding with each other while walking (which we can see in any open-world game like Assassin's Creed or Watch Dogs).
Commit Assistant was developed jointly with Concordia University, and the paper on how it works has been published. Despite the merits of the assistant, it is fairly obvious that it is not for everyone.
The main problem (as in many Artificial Intelligence problems) is the data that must be fed into the system to make it effective. Possessing billions of lines of code from years and years of development is no small feat, so the main focus of the assistant will likely be large studios like Rockstar or Electronic Arts. But if this type of technology matures enough, in the future we could see shorter release times, and perhaps more polished games.
