Quick Start¶
Table of Contents
- Quick Start
- Installing the client
- Initializing the client
- Creating models
- Listing available models by task
- Creating datasets
- Creating a train job
- Listing train jobs
- Creating an inference job with the latest train job
- Listing inference jobs
- Making predictions
- Prediction for QuestionAnswering
- Prediction for SpeechRecognition
- Stopping a running inference job
This guide assumes you have deployed your own empty instance of SINGA-Auto and you want to try a full train-inference flow as the Super Admin:
- Authenticating on SINGA-Auto
- Submitting models
- Uploading datasets
- Creating a model training job
- Creating a model serving job after the model training job completes
- Making predictions
Follow the sequence of examples below to submit the Fashion MNIST dataset for training and inference. Alternatively, refer and run the scripted version of this quickstart ./examples/scripts/quickstart.py.
To learn more about what else you can do on SINGA-Auto, explore the methods of singa_auto.client.Client
.
Note
If you haven’t set up SINGA-Auto on your local machine, refer to Quick Setup before continuing.
Note
- For Model Developers just looking to contribute models, refer to Quick Start (Model Developers)
- For Application Developers just looking to train and deploy models, refer to Quick Start (Application Developers)
- For Application Users just looking to make predictions, refer to Quick Start (Application Users)
Installing the client¶
Install Python 3.6 such that the
python
andpip
point to the correct installation of Python (see Installing Python)Clone the project at https://github.com/nusdbsystem/singa-auto (e.g. with Git)
Within the project’s root folder, install SINGA-Auto’s client-side Python dependencies by running:
pip install -r ./singa_auto/requirements.txt
Initializing the client¶
Example:
from singa_auto.client import Client client = Client(admin_host='localhost', admin_port=3000) # 'localhost' can be replaced by '127.0.0.1' or other server address client.login(email='superadmin@singaauto', password='singa_auto')
See also
singa_auto.client.Client.login()
Creating models¶
To create a model, you’ll need to submit a model class that conforms to the specification
by singa_auto.model.BaseModel
, written in a single Python file.
The model’s implementation should conform to a specific task (see tasks).
Refer to the parameters of singa_auto.client.Client.create_model()
for configuring how your model runs on SINGA-Auto,
and refer to Model Development Guide to understand more about how to write & test models for SINGA-Auto.
Example:
client.create_model( name='TfFeedForward', task='IMAGE_CLASSIFICATION', model_file_path='examples/models/image_classification/TfFeedForward.py', model_class='TfFeedForward', dependencies={ 'tensorflow': '1.12.0' } ) client.create_model( name='SkDt', task='IMAGE_CLASSIFICATION', model_file_path='examples/models/image_classification/SkDt.py', model_class='SkDt', dependencies={ 'scikit-learn': '0.20.0' } )
See also
singa_auto.client.Client.create_model()
Listing available models by task¶
Example:
client.get_available_models(task='IMAGE_CLASSIFICATION') # While leave the "task" unspecified, the method will retrieve information of all uploaded models client.get_available_models()Output:
[{'access_right': 'PRIVATE', 'datetime_created': 'Mon, 17 Dec 2018 07:06:03 GMT', 'dependencies': {'tensorflow': '1.12.0'}, 'id': '45df3f34-53d7-4fb8-a7c2-55391ea10030', 'name': 'TfFeedForward', 'task': 'IMAGE_CLASSIFICATION', 'user_id': 'fb5671f1-c673-40e7-b53a-9208eb1ccc50'}, {'access_right': 'PRIVATE', 'datetime_created': 'Mon, 17 Dec 2018 07:06:03 GMT', 'dependencies': {'scikit-learn': '0.20.0'}, 'id': 'd0ea96ce-478b-4167-8a84-eb36ae631235', 'name': 'SkDt', 'task': 'IMAGE_CLASSIFICATION', 'user_id': 'fb5671f1-c673-40e7-b53a-9208eb1ccc50'}]
See also
singa_auto.client.Client.get_available_models()
Creating datasets¶
You’ll first need to convert your dataset into a format specified by one of the tasks (see tasks), and split them into two files: one for training & one for validation. After doing so, you’ll create 2 corresponding datasets on SINGA-Auto by uploading them from your filesystem.
Example (pre-processing step):
# Run this in shell python examples/datasets/image_files/load_fashion_mnist.py
Example:
client.create_dataset( name='fashion_mnist_train', task='IMAGE_CLASSIFICATION', dataset_path='data/fashion_mnist_train.zip' ) client.create_dataset( name='fashion_mnist_val', task='IMAGE_CLASSIFICATION', dataset_path='data/fashion_mnist_val.zip' )Output:
{'id': 'ecf87d2f-6893-4e4b-8ed9-1d9454af9763', 'name': 'fashion_mnist_train', 'size_bytes': 36702897, 'task': 'IMAGE_CLASSIFICATION'} {'id': '7e9a2f8a-c61d-4365-ae4a-601e90892b88', 'name': 'fashion_mnist_val', 'size_bytes': 6116386, 'task': 'IMAGE_CLASSIFICATION'}
See also
singa_auto.client.Client.create_dataset()
Note
The code that preprocesses the original Fashion MNIST dataset is available at ./examples/datasets/image_files/load_mnist_format.py.
Creating a train job¶
To create a model training job, you’ll specify the train & validation datasets by their IDs, together with your application’s name and its associated task.
After creating a train job, you can monitor it on SINGA-Auto Web Admin (see Using SINGA-Auto’s Web Admin).
Refer to the parameters of singa_auto.client.Client.create_train_job()
for configuring how your train job runs on SINGA-Auto, such as enabling GPU usage & specifying which models to use.
Example:
client.create_train_job( app='fashion_mnist_app', task='IMAGE_CLASSIFICATION', train_dataset_id='ecf87d2f-6893-4e4b-8ed9-1d9454af9763', val_dataset_id='7e9a2f8a-c61d-4365-ae4a-601e90892b88', budget={ 'MODEL_TRIAL_COUNT': 5 } model_ids='["652db9f7-d23d-4b79-945b-a56446ceff33"]' ) # Omitting the GPU_COUNT is the same as letting GPU_COUNT equal to 0, which means training will be hosted on CPU only # MODEL_TRIAL_COUNT stands for number of trials, minimus MODEL_TRIAL_COUNT is 1 for a valid training # TIME_HOURS is assigned training time limit in hours. # train_args={} could be left empty or unspecified, if not in use client.create_train_job( app='fashion_mnist_app', task='IMAGE_CLASSIFICATION', train_dataset_id='ecf87d2f-6893-4e4b-8ed9-1d9454af9763', val_dataset_id='7e9a2f8a-c61d-4365-ae4a-601e90892b88', budget={'TIME_HOURS': 0.01, 'GPU_COUNT': 0, 'MODEL_TRIAL_COUNT': 1} model_ids='["652db9f7-d23d-4b79-945b-a56446ceff33"]', train_args={} )Output:
{'app': 'fashion_mnist_app', 'app_version': 1, 'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}
- Using distributed training:
- refer to https://pytorch.org/docs/stable/distributed.html
Example:
Output:
{'app': 'DistMinist', 'app_version': 1, 'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}
See also
singa_auto.client.Client.create_train_job()
Listing train jobs¶
Example:
client.get_train_jobs_of_app(app='fashion_mnist_app')Output:
[{'app': 'fashion_mnist_app', 'app_version': 1, 'budget': {'MODEL_TRIAL_COUNT': 5}, 'datetime_started': 'Mon, 17 Dec 2018 07:08:05 GMT', 'datetime_stopped': None, 'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8', 'status': 'RUNNING', 'task': 'IMAGE_CLASSIFICATION', 'val_dataset_id': '7e9a2f8a-c61d-4365-ae4a-601e90892b88', 'train_dataset_id': 'ecf87d2f-6893-4e4b-8ed9-1d9454af9763'}]
See also
singa_auto.client.Client.get_train_jobs_of_app()
Creating an inference job with the latest train job¶
To create an model serving job, you’ll have to wait for your train job to stop.
Then, you’ll submit the app name associated with the train job (with a status of STOPPED
).
The inference job would be created from the best trials from that train job.
Example:
client.create_inference_job(app='fashion_mnist_app') # Or with more details specified, such as Number of GPU 'GPU_COUNT' client.create_inference_job(app='fashion_mnist_app', app_version=1, budget={'GPU_COUNT': 1} )Output:
{'app': 'fashion_mnist_app', 'app_version': 1, 'id': '0477d03c-d312-48c5-8612-f9b37b368949', 'predictor_host': '127.0.0.1:30001', 'train_job_id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}
See also
singa_auto.client.Client.create_inference_job()
Listing inference jobs¶
Example:
client.get_inference_jobs_of_app(app='fashion_mnist_app')Output:
{'app': 'fashion_mnist_app', 'app_version': 1, 'datetime_started': 'Mon, 17 Dec 2018 07:15:12 GMT', 'datetime_stopped': None, 'id': '0477d03c-d312-48c5-8612-f9b37b368949', 'predictor_host': '127.0.0.1:30000', 'status': 'RUNNING', 'train_job_id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}
See also
singa_auto.client.Client.get_inference_jobs_of_app()
Making predictions¶
Send a POST /predict
to predictor_host
with a body of the following format in JSON:
{ "query": <query> }
…where the format of <query>
depends on the associated task (see tasks).
The body of the response will be of the following format in JSON:
{ "prediction": <prediction> }
…where the format of <prediction>
depends on the associated task.
Example:
If
predictor_host
is127.0.0.1:30000
, run the following in Python:predictor_host = '127.0.0.1:30000' query_path = 'examples/data/image_classification/fashion_mnist_test_1.png' # Load query image as 3D list of pixels from singa_auto.model import utils [query] = utils.dataset.load_images([query_path]).tolist() # Make request to predictor import requests import json res = requests.post('http://{}/predict'.format(predictor_host), json={ 'query': query }) print(res.json())Output:
{'prediction': [0.9364003576825639, 1.016065009906697e-08, 0.0027604885399341583, 0.00014587241457775235, 6.018594376655528e-06, 1.042887332047826e-09, 0.060679372351310566, 2.024707311532037e-11, 7.901770004536957e-06, 1.5299328026685544e-08], 'predictions': []}
Prediction for QuestionAnswering¶
- The query question should be uploaded by the following format
data={"questions": ["How long individuals are contagious?"]} res = requests.post('http://{}/predict'.format(predictor_host), json=data)
- To print out the prediction result, you should use ‘res.text’
print(res.text)
Prediction for SpeechRecognition¶
- The query data is passed using the following steps
data = [‘data/ldc93s1/ldc93s1/LDC93S1.wav’] data = json.dumps(data) res = requests.post(’http://{}/predict’.format(predictor_host), json=data[0])
- To print out the prediction result, you should use ‘res.text’
print(res.text)
If the SINGA-Auto instance is deployed with Kubernetes, all the inference job are at the default Ingress port 3005 with the format of <host>:3005/<app>, where <host> is the host name of the SINGA-Auto instance, and <app> is the name of the application prodvided when we submit train jobs.
Stopping a running inference job¶
Example:
client.stop_inference_job(app='fashion_mnist_app')
See also
singa_auto.client.Client.stop_inference_job()