Quick Start

This guide assumes you have deployed your own empty instance of SINGA-Auto and you want to try a full train-inference flow as the Super Admin:

  1. Authenticating on SINGA-Auto
  2. Submitting models
  3. Uploading datasets
  4. Creating a model training job
  5. Creating a model serving job after the model training job completes
  6. Making predictions

Follow the sequence of examples below to submit the Fashion MNIST dataset for training and inference. Alternatively, refer and run the scripted version of this quickstart ./examples/scripts/quickstart.py.

To learn more about what else you can do on SINGA-Auto, explore the methods of singa_auto.client.Client.

Note

If you haven’t set up SINGA-Auto on your local machine, refer to Quick Setup before continuing.

Note

Installing the client

  1. Install Python 3.6 such that the python and pip point to the correct installation of Python (see Installing Python)

  2. Clone the project at https://github.com/nusdbsystem/singa-auto (e.g. with Git)

  3. Within the project’s root folder, install SINGA-Auto’s client-side Python dependencies by running:

    pip install -r ./singa_auto/requirements.txt
    

Initializing the client

Example:

from singa_auto.client import Client
client = Client(admin_host='localhost', admin_port=3000) # 'localhost' can be replaced by '127.0.0.1' or other server address
client.login(email='superadmin@singaauto', password='singa_auto')

See also

singa_auto.client.Client.login()

Creating models

To create a model, you’ll need to submit a model class that conforms to the specification by singa_auto.model.BaseModel, written in a single Python file. The model’s implementation should conform to a specific task (see tasks).

Refer to the parameters of singa_auto.client.Client.create_model() for configuring how your model runs on SINGA-Auto, and refer to Model Development Guide to understand more about how to write & test models for SINGA-Auto.

Example:

client.create_model(
    name='TfFeedForward',
    task='IMAGE_CLASSIFICATION',
    model_file_path='examples/models/image_classification/TfFeedForward.py',
    model_class='TfFeedForward',
    dependencies={ 'tensorflow': '1.12.0' }
)

client.create_model(
    name='SkDt',
    task='IMAGE_CLASSIFICATION',
    model_file_path='examples/models/image_classification/SkDt.py',
    model_class='SkDt',
    dependencies={ 'scikit-learn': '0.20.0' }
)

See also

singa_auto.client.Client.create_model()

Listing available models by task

Example:

client.get_available_models(task='IMAGE_CLASSIFICATION')
# While leave the "task" unspecified, the method will retrieve information of all uploaded models
client.get_available_models()

Output:

[{'access_right': 'PRIVATE',
 'datetime_created': 'Mon, 17 Dec 2018 07:06:03 GMT',
 'dependencies': {'tensorflow': '1.12.0'},
 'id': '45df3f34-53d7-4fb8-a7c2-55391ea10030',
 'name': 'TfFeedForward',
 'task': 'IMAGE_CLASSIFICATION',
 'user_id': 'fb5671f1-c673-40e7-b53a-9208eb1ccc50'},
 {'access_right': 'PRIVATE',
 'datetime_created': 'Mon, 17 Dec 2018 07:06:03 GMT',
 'dependencies': {'scikit-learn': '0.20.0'},
 'id': 'd0ea96ce-478b-4167-8a84-eb36ae631235',
 'name': 'SkDt',
 'task': 'IMAGE_CLASSIFICATION',
 'user_id': 'fb5671f1-c673-40e7-b53a-9208eb1ccc50'}]

See also

singa_auto.client.Client.get_available_models()

Creating datasets

You’ll first need to convert your dataset into a format specified by one of the tasks (see tasks), and split them into two files: one for training & one for validation. After doing so, you’ll create 2 corresponding datasets on SINGA-Auto by uploading them from your filesystem.

Example (pre-processing step):

# Run this in shell
python examples/datasets/image_files/load_fashion_mnist.py

Example:

client.create_dataset(
    name='fashion_mnist_train',
    task='IMAGE_CLASSIFICATION',
    dataset_path='data/fashion_mnist_train.zip'
)

client.create_dataset(
    name='fashion_mnist_val',
    task='IMAGE_CLASSIFICATION',
    dataset_path='data/fashion_mnist_val.zip'
)

Output:

{'id': 'ecf87d2f-6893-4e4b-8ed9-1d9454af9763',
'name': 'fashion_mnist_train',
'size_bytes': 36702897,
'task': 'IMAGE_CLASSIFICATION'}

{'id': '7e9a2f8a-c61d-4365-ae4a-601e90892b88',
'name': 'fashion_mnist_val',
'size_bytes': 6116386,
'task': 'IMAGE_CLASSIFICATION'}

See also

singa_auto.client.Client.create_dataset()

Note

The code that preprocesses the original Fashion MNIST dataset is available at ./examples/datasets/image_files/load_mnist_format.py.

Creating a train job

To create a model training job, you’ll specify the train & validation datasets by their IDs, together with your application’s name and its associated task.

After creating a train job, you can monitor it on SINGA-Auto Web Admin (see Using SINGA-Auto’s Web Admin).

Refer to the parameters of singa_auto.client.Client.create_train_job() for configuring how your train job runs on SINGA-Auto, such as enabling GPU usage & specifying which models to use.

Example:

client.create_train_job(
    app='fashion_mnist_app',
    task='IMAGE_CLASSIFICATION',
    train_dataset_id='ecf87d2f-6893-4e4b-8ed9-1d9454af9763',
    val_dataset_id='7e9a2f8a-c61d-4365-ae4a-601e90892b88',
    budget={ 'MODEL_TRIAL_COUNT': 5 }
    model_ids='["652db9f7-d23d-4b79-945b-a56446ceff33"]'
)
# Omitting the GPU_COUNT is the same as letting GPU_COUNT equal to 0, which means training will be hosted on CPU only
# MODEL_TRIAL_COUNT stands for number of trials, minimus MODEL_TRIAL_COUNT is 1 for a valid training
# TIME_HOURS is assigned training time limit in hours.
# train_args={} could be left empty or unspecified, if not in use
client.create_train_job(
    app='fashion_mnist_app',
    task='IMAGE_CLASSIFICATION',
    train_dataset_id='ecf87d2f-6893-4e4b-8ed9-1d9454af9763',
    val_dataset_id='7e9a2f8a-c61d-4365-ae4a-601e90892b88',
    budget={'TIME_HOURS': 0.01,
            'GPU_COUNT': 0,
            'MODEL_TRIAL_COUNT': 1}
    model_ids='["652db9f7-d23d-4b79-945b-a56446ceff33"]',
    train_args={}
)

Output:

{'app': 'fashion_mnist_app',
'app_version': 1,
'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}
Using distributed training:
refer to https://pytorch.org/docs/stable/distributed.html

Example:

Output:

{'app': 'DistMinist',
'app_version': 1,
'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}

See also

singa_auto.client.Client.create_train_job()

Listing train jobs

Example:

client.get_train_jobs_of_app(app='fashion_mnist_app')

Output:

[{'app': 'fashion_mnist_app',
'app_version': 1,
'budget': {'MODEL_TRIAL_COUNT': 5},
'datetime_started': 'Mon, 17 Dec 2018 07:08:05 GMT',
'datetime_stopped': None,
'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8',
'status': 'RUNNING',
'task': 'IMAGE_CLASSIFICATION',
'val_dataset_id': '7e9a2f8a-c61d-4365-ae4a-601e90892b88',
'train_dataset_id': 'ecf87d2f-6893-4e4b-8ed9-1d9454af9763'}]

See also

singa_auto.client.Client.get_train_jobs_of_app()

Creating an inference job with the latest train job

To create an model serving job, you’ll have to wait for your train job to stop. Then, you’ll submit the app name associated with the train job (with a status of STOPPED). The inference job would be created from the best trials from that train job.

Example:

client.create_inference_job(app='fashion_mnist_app')
# Or with more details specified, such as Number of GPU 'GPU_COUNT'
client.create_inference_job(app='fashion_mnist_app', app_version=1, budget={'GPU_COUNT': 1} )

Output:

{'app': 'fashion_mnist_app',
'app_version': 1,
'id': '0477d03c-d312-48c5-8612-f9b37b368949',
'predictor_host': '127.0.0.1:30001',
'train_job_id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}

See also

singa_auto.client.Client.create_inference_job()

Listing inference jobs

Example:

client.get_inference_jobs_of_app(app='fashion_mnist_app')

Output:

{'app': 'fashion_mnist_app',
  'app_version': 1,
  'datetime_started': 'Mon, 17 Dec 2018 07:15:12 GMT',
  'datetime_stopped': None,
  'id': '0477d03c-d312-48c5-8612-f9b37b368949',
  'predictor_host': '127.0.0.1:30000',
  'status': 'RUNNING',
  'train_job_id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}

See also

singa_auto.client.Client.get_inference_jobs_of_app()

Making predictions

Send a POST /predict to predictor_host with a body of the following format in JSON:

{
    "query": <query>
}

…where the format of <query> depends on the associated task (see tasks).

The body of the response will be of the following format in JSON:

{
    "prediction": <prediction>
}

…where the format of <prediction> depends on the associated task.

Example:

If predictor_host is 127.0.0.1:30000, run the following in Python:

predictor_host = '127.0.0.1:30000'
query_path = 'examples/data/image_classification/fashion_mnist_test_1.png'

# Load query image as 3D list of pixels
from singa_auto.model import utils
[query] = utils.dataset.load_images([query_path]).tolist()

# Make request to predictor
import requests
import json
res = requests.post('http://{}/predict'.format(predictor_host), json={ 'query': query })
print(res.json())

Output:

{'prediction': [0.9364003576825639, 1.016065009906697e-08, 0.0027604885399341583, 0.00014587241457775235, 6.018594376655528e-06, 1.042887332047826e-09, 0.060679372351310566, 2.024707311532037e-11, 7.901770004536957e-06, 1.5299328026685544e-08],
'predictions': []}

Prediction for QuestionAnswering

The query question should be uploaded by the following format
data={"questions": ["How long individuals are contagious?"]}
res = requests.post('http://{}/predict'.format(predictor_host), json=data)
To print out the prediction result, you should use ‘res.text’
print(res.text)

Prediction for SpeechRecognition

The query data is passed using the following steps

data = [‘data/ldc93s1/ldc93s1/LDC93S1.wav’] data = json.dumps(data) res = requests.post(’http://{}/predict’.format(predictor_host), json=data[0])

To print out the prediction result, you should use ‘res.text’
print(res.text)

If the SINGA-Auto instance is deployed with Kubernetes, all the inference job are at the default Ingress port 3005 with the format of <host>:3005/<app>, where <host> is the host name of the SINGA-Auto instance, and <app> is the name of the application prodvided when we submit train jobs.

Stopping a running inference job

Example:

client.stop_inference_job(app='fashion_mnist_app')

See also

singa_auto.client.Client.stop_inference_job()