top of page
Untitled design - 2025-06-05T172104.079.png

How to Build a CI/CD Pipeline for ML on Azure ML

  • Writer: Nicole Mocskonyi
    Nicole Mocskonyi
  • Jun 9
  • 5 min read

Implementing Continuous Integration and Continuous Delivery (CI/CD) in machine learning (ML) projects is crucial for ensuring rapid, reliable, and reproducible model deployment. Azure Machine Learning (Azure ML) offers a robust platform to integrate CI/CD practices, enabling teams to automate the ML lifecycle from data ingestion to model deployment.


In this blog, Cyann experts walk through a step-by-step guide to building a CI/CD pipeline for ML on Azure ML, leveraging Azure DevOps and other Azure services. By following this guide, engineering teams can streamline their ML workflows, reduce manual errors, and accelerate time-to-production.


Prerequisites and Initial Setup


Before constructing the CI/CD pipeline, ensure the following prerequisites are met:


  • Azure Subscription: An active Azure subscription is required to access Azure services.

  • Azure Machine Learning Workspace: Set up an Azure ML workspace to manage ML assets.

  • Azure DevOps Organization: Create an Azure DevOps organization to manage repositories and pipelines.

  • Service Principal: Establish a service principal with the necessary permissions to allow Azure DevOps to interact with Azure resources.

  • Azure CLI and Azure ML CLI Extension: Install the Azure CLI and the Azure ML CLI extension to interact with Azure services from the command line.

  • Azure DevOps Extension for Azure ML: Install the Azure Machine Learning extension for Azure Pipelines from the Visual Studio Marketplace to integrate Azure ML with Azure DevOps.


Setting Up the Azure DevOps Project and Repository


With the prerequisites in place, proceed to set up the Azure DevOps project and repository:


Step 1: Create a New Project in Azure DevOps


  • Navigate to the Azure DevOps portal.

  • Select "New Project" and provide a name and description for the project.


Step 2: Import or Create a Repository


  • Within the newly created project, go to the "Repos" section.

  • You can either import an existing repository or create a new one to store your ML code, including training scripts, data processing scripts, and configuration files.


Step 3: Configure Service Connections


  • In the project settings, navigate to "Service connections."

  • Create a new service connection of type "Azure Resource Manager" using the service principal credentials.

  • This connection allows Azure DevOps pipelines to deploy resources and interact with Azure services securely.


Authoring the Training and Scoring Scripts


A well-structured ML project repository is foundational to a successful CI/CD pipeline. Azure ML expects specific script components to handle model training and scoring.


  1. Training Script (train.py)

    • This script defines how your model is trained. It includes loading the dataset, initializing the model, training, evaluating metrics, and saving the model artifacts.

    • Use argparse to pass in parameters dynamically from the pipeline.


Example snippet:


import argparse

import joblib

from sklearn.ensemble import RandomForestClassifier

from azureml.core import Run


parser = argparse.ArgumentParser()

parser.add_argument('--data-folder', type=str, dest='data_folder')

args = parser.parse_args()


# Load data

# Train model

# Save model


joblib.dump(model, 'outputs/model.joblib')


  1. Scoring Script (score.py)


    • This script is used during deployment for inference.

    • It must define init() and run() functions per Azure ML requirements.


Example template:

def init():

    global model

    model = joblib.load('model.joblib')


def run(data):

    result = model.predict(data)

    return result.tolist()


These scripts should be placed in a structured repository, often along with a conda.yaml file for environment definition.


Building the CI Pipeline (Continuous Integration)


The CI pipeline in Azure DevOps ensures that code is validated and artifacts are created automatically with every commit or pull request.


Step 1: Create a ci-pipeline.yml File


  • Place this YAML file in the root of your repository.

  • It should include steps to install dependencies, lint the code, and register the model with Azure ML.

    Basic CI example:

        trigger:

   branches:

     include:

       - main


pool:

  vmImage: 'ubuntu-latest'


steps:

- task: UsePythonVersion@0

  inputs:

    versionSpec: '3.8'


- script: |

    pip install -r requirements.txt

    python -m flake8 train.py

  displayName: 'Install dependencies and lint'


- task: AzureCLI@2

  inputs:

    azureSubscription: '<your-service-connection>'

    scriptType: 'bash'

    scriptLocation: 'inlineScript'

    inlineScript: |


      az ml model register --name mymodel --path outputs/model.joblib --resource-group <rg> --workspace-name <workspace>


Step 2: Commit and Monitor


  • Commit the pipeline file to the repo.

  • Navigate to Pipelines > New Pipeline in Azure DevOps and connect it to your repo.

  • Azure DevOps will automatically trigger the build on commit.


5. Building the CD Pipeline (Continuous Delivery)


The CD pipeline handles model deployment and endpoint testing.


Step 1: Create cd-pipeline.yml


This YAML pipeline typically includes steps for:


  • Creating an inference environment

  • Registering the environment

  • Deploying the model to Azure ML using bicep or CLI


Basic CD template:

trigger:

  branches:

    include:

      - main


pool:

  vmImage: 'ubuntu-latest'


steps:

- task: AzureCLI@2

  inputs:

    azureSubscription: '<your-service-connection>'

    scriptType: 'bash'

    scriptLocation: 'inlineScript'

    inlineScript: |

      az ml environment create --file conda.yaml --resource-group <rg> --workspace-name <workspace>

      az ml online-endpoint create --name my-endpoint --file endpoint.yml

      az ml online-deployment create --name my-deployment --endpoint-name my-endpoint --file deployment.yml


Step 2: Validate Deployment


  • The endpoint configuration can include readiness probes and health checks.

  • Add a final task in the pipeline to send a test payload to the endpoint and validate the response.


Monitoring and Managing the Deployed Model


Once the model is deployed, continuous monitoring is critical to ensure it remains performant and reliable under production conditions.


Enable Application Insights


Azure ML integrates with Application Insights to collect metrics such as latency, request volume, error rates, and custom logs. You can enable it during endpoint creation by adding the appropriate telemetry configurations to your deployment.yml file.


Example configuration:

Yaml

properties:

  app_insights_enabled: true


Track Model Performance


Use Azure ML’s built-in tools to monitor:

  • Data drift: Identify when input data starts to diverge from training data distributions.

  • Model accuracy over time: Revalidate models periodically against new labels.

  • Service health: Get alerts when the endpoint fails or becomes unavailable.


Automate Retraining


You can set up Azure ML pipelines or schedule jobs via Azure DevOps to retrain and redeploy the model when performance thresholds are breached.


How Can Cyann Help?


Cyann.ai specializes in building scalable, secure, and automated MLOps pipelines using Microsoft Azure services. Our experts help engineering and data science teams accelerate model delivery without sacrificing governance or reliability.


Our services include:


  • CI/CD Blueprinting and Implementation: We design and deploy fully automated pipelines using Azure DevOps and Azure ML.

  • Secure Environment and Compliance Setup: Implement enterprise-grade governance with Azure Policy, role-based access control (RBAC), and network isolation.

  • Model Lifecycle Automation: We enable automatic model retraining, promotion, and rollback using Azure ML pipelines and GitOps principles.

  • Monitoring and Observability: Configure advanced telemetry using Application Insights, Prometheus, and custom health checks.


Conclusion


Building a CI/CD pipeline for machine learning on Azure ML enables automation, consistency, and operational efficiency. By integrating Azure DevOps, Azure CLI, and Azure ML’s native capabilities, teams can deliver models faster and more reliably.


From training scripts to automated deployment and telemetry, the process outlined in this guide empowers engineering teams to treat ML systems like software, versioned, tested, and continuously improved.


Cyann’s deep expertise in Azure AI ensures organizations can implement these workflows securely and at scale, accelerating their MLOps maturity.


Contact us to discover how Cyaan assists organizations in maximizing their Azure capabilities.


References:

bottom of page