Examples of running Machine Learning Model on Device using Qualcomm AI HUB

Getting started

Installation

We recommend using Miniconda to manage your python versions and environments.

Step 1: Python environment

Install miniconda on your machine.

Windows: When the installation finishes, open Anaconda Prompt from the Start menu.

macOS/Linux: When the installation finished, open a new shell window.

Set up an environment for Qualcomm® AI Hub:

conda create python=3.8 -n qai_hub
conda activate qai_hub

Warning

If any of these steps fail with SSL: CERTIFICATE_VERIFY_FAILED errors, you have an SSL interception and traffic inspection tool installed. Ask your IT department for instructions on how to set up certificates for Python pip and the Python Requests library. They should provide you with a new certificate file. You will also need to add this certificate to Miniconda prior to creating your environment. This can be done with conda config --set ssl_verify <path to certificate>.

Step 2: Install Python client

pip3 install qai-hub

Step 3: Sign in

Go to Qualcomm® AI Hub and sign in with your Qualcomm ID to view information about jobs you create.

Once signed in navigate to Account -> Settings -> API Token. This should provide an API token that you can use to configure your client.

Step 4: Configure API Token

Next, configure the client with API token using the following command in your terminal:

qai-hub configure --api_token INSERT_API_TOKEN

You can check that your API token is installed correctly by fetching a list of available devices. To do that, you can type the following in a Python terminal:

import qai_hub as hub
hub.get_devices()

If you run into any issues, please contact us.

Quick example (PyTorch)

Once you have set up your Qualcomm® AI Hub environment, the next step is to submit a profiling job. First, install the dependencies of this example:

pip3 install "qai-hub[torch]"

Warning

If any of the snippets fail with an API authentication error, it means that you do not have a valid API token installed. Please see Installation to learn how to set this up.

Submit a performance analysis of the MobileNet v2 network:

from typing import Tuple

import torch
import torchvision

import qai_hub as hub

# Using pre-trained MobileNet
torch_model = torchvision.models.mobilenet_v2(pretrained=True)
torch_model.eval()

# Trace model
input_shape: Tuple[int, ...] = (1, 3, 224, 224)
example_input = torch.rand(input_shape)
pt_model = torch.jit.trace(torch_model, example_input)

# Profile model on a specific device
compile_job, profile_job = hub.submit_compile_and_profile_jobs(
    pt_model,
    name="MyMobileNet",
    device=hub.Device("Samsung Galaxy S23 Ultra"),
    input_specs=dict(image=input_shape),
)

This will submit a compilation job and then a profiling job, printing the URLs of both jobs. View all your jobs at /jobs/.

You can programmatically query the status of the job:

status = profile_job.get_status()
print(status)

You can access the results of the job using the snippet below. There are three main parts

• Profile: Results of the profiling in JSON format.

• Target Model: Optimized model ready for deployment.

• Results: Folder containing all the artifacts of the job (including logs).

Note that these are blocking API calls that wait until the job finishes:

# Download profile results as JSON (blocking call)
profile = profile_job.download_profile()
print(profile)

# Download an optimized model (blocking call)
model = profile_job.model.download()
print(model)

# Download results to current directory (blocking call)
profile_job.download_results(".")