The DevOps shortcut to AI

In today’s rapidly evolving technology landscape around AI, organizations and leaders face a pressing need to adapt and thrive in the face of rapid innovation. In a keynote at DevTalks Romania, we presented our experience and insights into getting ML and AI solutions operational quickly. The keynote highlighted the benefits we get from applying best practices from DevOps to efficiently deploy, monitor, retrain, and govern ML/AI models in production (commonly referred to as model operations or MLOps).

Cutting across the hype cycle

There’s always a lot of hype and excitement surrounding emerging technologies, and now the time has come for ML and AI. Technologies typically go through stages of inflated expectations before reaching productivity. While hype is neither good nor bad, it’s important to promote learning, processes, and critical thinking within organizations to navigate through the hype to arrive at the best use of technologies in their business environment.

Three approaches to MLOps

Based on our client experience, three approaches to MLOps have emerged among large and midsized enterprises.

  1. Build in-house with ML specialists
  2. Buy a platform solution
  3. Leverage existing DevOps capability

The first involves hiring data scientists and specialists to handle MLOps within the data function.

The second is to invest in a specialized software product or a platform that manages the propagation of data and algorithms.

The third approach is to adapt existing processes and skills from DevOps. By integrating MLOps within the DevOps framework, organizations can build upon familiar practices and accelerate their journey towards successful AI solutions.

Novelty and distinction bias

When options are compared side by side, there’s a tendency to overemphasize differences rather than similarities. This is known to psychologists as distinction bias. And it can possibly explain why MLOps is seen as something completely novel that needs new skillsets as per the first approach and/or new tools to manage.

Investing in a specialized software product or platform solution to manage the lifecycle of AI/ML solutions also suffers from the novelty factor but in a slightly different way. Any large investment in a software solution, while standards and best practices evolve, is a risky proposition.

Instead, we believe in the third approach. It’s best to learn fast and leverage existing practices and experience as the starting point for new practices.

By avoiding the fallacy of setting up MLOps processes and practices as something completely new, we can easily avoid common mistakes and wasted efforts.

Figure 1 – Avoidable pains and obtainable gains for MLOps from DevOps practices

But what’s more exciting is what we gain from re-using existing know-how, which in addition to validated processes and practices, are established metrics and stakeholder understanding.

Building MLOps from DevOps

To quickly explain how we approach MLOps at Emergn, we must start with a top-down perspective of DevOps. There are two primary objectives of a business: developing products and services and running and operating them.

We follow a set of three principles for the successful development of digital products and services called Value, Flow, Quality. It brings a common language to our and our clients’ organizations and a set of scalable tools that deliver business results according to each context.

By understanding the commonalities between DevOps and MLOps, organizations can leverage proven tools, practices, and ways of working to accelerate their journey toward successful AI deployments.

Drawing on our Value, Flow, Quality principles with the key attributes of DevOps as illustrated below, we recommend forming MLOps practices on a culture of automation, self-service, continuous delivery, and observability with short feedback cycles.

Figure 2 – Good DevOps practices

This is the shortcut to efficient deployment, operation, and improvement of AI models. Let’s look at a couple of practical examples.

Observability and instrumentation

We built and delivered an automation solution for court administrators to anonymize verdicts before they are published.

The solution is integrated with the court office system and allows court staff to automate the redaction and substitutions for real names and other Personal Identifiable Information (PII) before the verdicts are made public. Clearly, the precision of the AI model for entity recognition needs to be very high.

With instrumentation built into the webservice, what we quickly discovered was that user behaviors anonymizing old court documents in test cycles were dramatically different from when the same users anonymized new court documents.

To build user confidence and trust, we built a dedicated UI for the court administrators not just to test but also to teach the model to control and improve the precision in specific model training sessions. From this dedicated UI, we found that:

  • Instrumentation and observability are key to capturing benefits
  • Fast feedback from real users in production is much more informative than from any QA
  • DevOps specialists are faster at setting up environment configurations and backups
Figure 3 – Personal Identification Information for a court administration solution


We built and delivered an automation solution for a legal services firm to automate the processing of claims forms. Using pre-built AI models from Microsoft, the solution reduces the expected time spent on case administration by 70%.

One of the challenges is that forms from claimants and third-party insurers change very frequently. We found the automated solution proved that:

  • Self-service is easily achievable
  • Power users need a process to configure new versions of forms as they are received
  • End-users can judge and decide when to (re-)train the model when the precision is visible and traceable from the UI
Figure $ – Claims automation for a legal services solution

Our recommendations

Recognize that AI is on the hype curve. Data scientists have extremely attractive skills, but putting models into operations is not the best use of their skill sets.

In our opinion, with the current rapid development of technology, it’s too early to buy a platform to manage MLOps. Leveraging know-how from DevOps is currently the best shortcut through the hype to establish a solid foundation for ML and AI solutions.

Our recommendations for setting up MLOps are summarized in the table below.

Automate and instrumentRush to define new processes
Release updates oftenExpect data scientists to be good operators
Design for self-service and frequent releasesLeave traceability and monitoring
Leverage and upskill DevOps expertiseUnderestimate how much users rely on UI for users acceptance

If you want to learn more about our Data & Analytics or DevOps capabilities, please get in touch. For more on this topic, see also: Explaining DevOps to Executives, or watch the full 45-minute keynote video from DevTalks here.