Microsoft AI/ML - The Pre-Built Part of the Spectrum

An overview of the pre-built AI and ML services provided by Microsoft Azure

Posted by Ciprian on April 15, 2019

Azure Cognitive Services provide the left side of the spectrum, the pre-built AI and ML services. The capabilities of these services range from fully trained models to models whose training can be customized using specific, domain-related data. All models are exposed as REST APIs as well as containers that can be deployed on a wide range of environments. The following diagram displays a generic view of the capabilities:

The AI-ML Pre-Built Services

The Cognitive Services are exposed as a set of ready to use REST APIs that cover the following major areas:

The Cognitive Services documentation home page is a great place to get you started. Some of the most widely used programming languages (like C#, Python, JavaScript, and Java) have their own SDKs that simplify the use of the services. In addition to all these, you can get familiarized with emerging technologies at the Cognitive Services Labs.

Once you reach the point where the existing (pre-trained) models are no longer suited to meet your requirements, you can start customizing the Cognitive Services. You can achieve this by providing your own labeled data and training the models on it. The Custom Vision, Custom Speech, and Custom Translator services are examples of customizable services. A particularly interesting example is the Language Understanding service which supports a more extensive approach to customization through its own, dedicated dashboard.

Once you are satisfied with the way your particular Cognitive Service performs, you can decide on how to use it. Historically, the first available option was to call the services as REST APIs. Lately, Microsoft has been adding support for packaging the APIs in Docker containers and enabling you to distribute them even outside the public cloud. When it comes to deployment in the public cloud, Azure Kubernetes Service (AKS) or Azure Container Instances (ACI) are the obvious choices. You will typically choose AKS when you need the full power of orchestrated clusters and ACI when an approach that is closer to "true serverless" is more desirable. If your game is running on premise, a Kubernetes cluster running on your Azure Stack is a powerful option. In addition to these, you can even bring your Cognitive Services containers to the edge (provided of course that your edge device/environment supports running Docker containers). A great point to start exploring this particular scenario is here.

Currently, container support is available for the Computer Vision, Face, Text Analytics, and Language Understanding services. The list evolves at a fast pace so I recommend you check periodically Microsoft's official container support page.


So, this is the high level overview of the current Pre-Built part of the AI/ML spectrum of services available in Microsoft Azure. The major categories of services are Vision, Speech, Knowledge, Search, Language, and Anomaly Detection. You can use the default trained versions of the models behind the services or you can customize their training by providing your own labeled data. Once you are satisfied with the training, you can either call the services as REST APIs or export them as Docker containers to be hosted in the public cloud, on-premise, or even on the edge. Follow The Hitchhiker's Guide to Microsoft AI to get the rest of the story in detail.