Quickstart Tutorial

Open In Colab         DownloadDownload this Notebook

Quickstart Tutorial

In this short introduction to Masterful, you will train and evaluate a model start to finish with the Masterful CLI Trainer. This example will walk you through setting up your data and model and then using them to train and evaluate a model.

Prerequisites

Ensure you have Tensorflow installed, then install Masterful. For more details, follow the Masterful installation instructions.

[1]:
!pip install --upgrade pip --quiet
!pip install masterful --quiet &> /dev/null

TL;DR

Don’t want to read the rest of this guide, and want to start training immediately? The following command shows you how to start training with Masterful, using a configuration file and dataset on S3.

The sections following this will go into more detail on what is happening underneath the covers, and explain the dataset and configuration file formats.

For more information on the configuration file and dataset formats, see Masterful Configuration File and Masterful Dataset Format.

From Docker Install:

$ docker run -v $HOME/model_output:/root/model_output:rw --rm --gpus all masterful/masterful:latest masterful-train --config=https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/training.yaml

From Pip Install:

$ masterful-train --config=https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/training.yaml

Setup the Dataset

The first step in any training project is to collect the data you will train with. Masterful has a very simple, flexible CSV based format for images and labels that should make it trivial to prepare your data for training.

A typical dataset consists of a set of images and their labels. These images and labels are then split into different sets, typically called training, validation, and test/holdout sets. The training set is used to train your model, validation set is used to measure the performance of your model during training, and the test/holdout set is used to measure the generalization performance of your model on data is has never seen before. Masterful only requires a training set. If there is no validation set, then Masterful will create one from the training set. If there is no test/holdout set, then Masterful will not evaluate your model on that set and will not report evaluation metrics on the model.

Masterful uses a very simple CSV file format to describe the images and labels in your dataset. Typically you create a separate CSV file for each of the dataset splits (training, validation, and test) that you want to use during training. For this tutorial, you will use a simple Flowers dataset (the same dataset used in the Keras Image Classification tutorial). The images and labels for this dataset are stored in the public AWS bucket s3://masterful-public/datasets/quickstart/. This bucket has the following structure:

quickstart/
  daisy/
  dandelion/
  roses/
  sunflowers/
  tulips/
  training.yaml
  test.csv
  train.csv
  validation.csv

As you can see, there are CSV files created called train.csv, test.csv, and validation.csv, which hold the description of the training, test, and validation dataset splits respectively. You can also see a training.yaml file, which is a YAML formatted configuration file which defines all of the information necessary for Masterful to train on the above dataset. You will learn more about the configuration file below.

For more information about the Masterful dataset format, see Masterful Dataset Format.

Explore the Data

There are 5 classes in the dataset listed above, corresponding to different types of flowers. You can see examples of each class in the plot below.

[2]:
import matplotlib.pyplot as plt
import PIL
import requests
from io import BytesIO

daisies = [
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/daisy/100080576_f52e8ee070_n.jpg",
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/daisy/10140303196_b88d3d6cec.jpg",
]

dandelions = [
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/dandelion/10043234166_e6dd915111_n.jpg",
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/dandelion/10200780773_c6051a7d71_n.jpg",
]
roses = [
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/roses/10090824183_d02c613f10_m.jpg",
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/roses/102501987_3cdb8e5394_n.jpg",
]
sunflowers = [
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/sunflowers/1008566138_6927679c8a.jpg",
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/sunflowers/1022552002_2b93faf9e7_n.jpg",
]
tulips = [
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/tulips/100930342_92e8746431_n.jpg",
    "https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/tulips/10094729603_eeca3f2cb6.jpg",
]

images = [daisies, dandelions, roses, sunflowers, tulips]
ROWS = 2
COLUMNS = len(images)

f, axarr = plt.subplots(ROWS, COLUMNS, figsize=(15, 5))
curr_row = 0

for col, image_col in enumerate(images):
    for row, image_row in enumerate(image_col):
        with requests.get(image_row) as response:
            image = PIL.Image.open(BytesIO(response.content))
        axarr[row, col].imshow(image)
../_images/notebooks_tutorial_quickstart_cli_9_0.png

Configure Training

Masterful uses a simple YAML configuration file to setup the training parameters. The configuration file has five major sections: dataset, model, training, evaluation, and output. The configuration file used in this tutorial is here.

For more information on the configuration file format, see Masterful Configuration File.

Let’s Get Training

Simply point Masterful to the configuration file you created, and Masterful will begin training on your data. At the end of training, Masterful will summarize the performance metrics after evaluating the model on your test/holdout dataset.

[3]:
!masterful-train --config=https://masterful-public.s3.us-west-1.amazonaws.com/datasets/quickstart/training.yaml
MASTERFUL: Your account has been successfully registered. Masterful v0.5.0 is loaded.
MASTERFUL [12:04:47]: Training with configuration 's3://masterful-public/datasets/quickstart/training.yaml'.
MASTERFUL [12:04:49]: Using model efficientnetb0_v1 with:
MASTERFUL [12:04:49]:     4055976 total parameters
MASTERFUL [12:04:49]:     4013953 trainable parameters
MASTERFUL [12:04:49]:     42023 untrainable parameters
MASTERFUL [12:04:50]: Caching I/O optimized dataset split 'train' to /home/yaoshiang/.masterful/datasets/36158d44fb4f0c63ac240db51678fac4f63ba328...
100%|███████████████████████████████████████| 2936/2936 [05:44<00:00,  8.52it/s]
MASTERFUL [12:10:42]: Caching I/O optimized dataset split 'validation' to /home/yaoshiang/.masterful/datasets/0cc422076c324bbadaf751dfa2cc5845df0f8cb4...
100%|█████████████████████████████████████████| 367/367 [00:40<00:00,  9.07it/s]
MASTERFUL [12:11:29]: Caching I/O optimized dataset split 'test' to /home/yaoshiang/.masterful/datasets/3877e18299c65143f40085b01eeb88ebe52f5930...
100%|█████████████████████████████████████████| 367/367 [00:53<00:00,  6.91it/s]
MASTERFUL [12:12:27]: Dataset Summary:
MASTERFUL [12:12:27]:   Training Dataset: 2936 examples.
MASTERFUL [12:12:27]:   Validation Dataset: 367 examples.
MASTERFUL [12:12:27]:   Test Dataset: 367 examples.
MASTERFUL [12:12:27]:   Unlabeled Dataset: 0 examples.
MASTERFUL [12:12:28]: Training Dataset Analysis:
100%|██████████████████████████████████████| 2936/2936 [00:14<00:00, 199.18it/s]
MASTERFUL [12:12:42]: Training dataset analysis finished at 12:12:42 in 15 seconds (15s), returned:
------------------  ----------------------------------------
Total Examples      2936
Label Counts        dandelion  713
                    rose       510
                    tulip      634
                    daisy      514
                    sunflower  565
Label Distribution  dandelion  0.242847
                    rose       0.173706
                    tulip      0.21594
                    daisy      0.175068
                    sunflower  0.192439
Balanced            Yes
Per Channel Mean    [118.7795914  108.89870996  76.78918076]
Per Channel StdDev  [64.67416621 57.99644922 59.40810724]
Min Height          180
Min Width           143
Max Height          442
Max Width           1024
Average Height      272
Average Width       366
Largest Image       (442, 500, 3)
Smallest Image      (240, 143, 3)
Duplicates          1
------------------  ----------------------------------------
MASTERFUL [12:12:42]: WARNING: Duplicates detected in dataset split 'train'.
MASTERFUL [12:12:42]: WARNING: You can find the duplicate entries using the tool:
MASTERFUL [12:12:42]: WARNING: python -m masterful.data.duplicate_detector --config=s3://masterful-public/datasets/quickstart/training.yaml
MASTERFUL [12:12:42]: Validation Dataset Analysis:
100%|████████████████████████████████████████| 367/367 [00:01<00:00, 206.93it/s]
MASTERFUL [12:12:44]: Validation dataset analysis finished at 12:12:44 in 2 seconds (2s), returned:
------------------  ----------------------------------------
Total Examples      367
Label Counts        dandelion  95
                    rose       74
                    daisy      55
                    sunflower  62
                    tulip      81
Label Distribution  dandelion  0.258856
                    rose       0.201635
                    daisy      0.149864
                    sunflower  0.168937
                    tulip      0.220708
Balanced            No
Per Channel Mean    [116.63373761 106.30352827  74.48895039]
Per Channel StdDev  [62.93288427 55.68255178 56.71276635]
Min Height          180
Min Width           157
Max Height          429
Max Width           640
Average Height      268
Average Width       355
Largest Image       (429, 500, 3)
Smallest Image      (240, 157, 3)
Duplicates          0
------------------  ----------------------------------------
MASTERFUL [12:12:44]: Test Dataset Analysis:
100%|████████████████████████████████████████| 367/367 [00:01<00:00, 200.34it/s]
MASTERFUL [12:12:46]: Test dataset analysis finished at 12:12:46 in 2 seconds (2s), returned:
------------------  ----------------------------------------
Total Examples      367
Label Counts        dandelion  90
                    rose       57
                    sunflower  72
                    daisy      64
                    tulip      84
Label Distribution  dandelion  0.245232
                    rose       0.155313
                    sunflower  0.196185
                    daisy      0.174387
                    tulip      0.228883
Balanced            Yes
Per Channel Mean    [112.55877415 105.24692186  76.16738029]
Per Channel StdDev  [64.2522064  57.23987375 58.24934888]
Min Height          159
Min Width           159
Max Height          404
Max Width           1024
Average Height      271
Average Width       363
Largest Image       (404, 500, 3)
Smallest Image      (240, 159, 3)
Duplicates          0
------------------  ----------------------------------------
MASTERFUL [12:12:46]: Cross-Dataset Analysis:
MASTERFUL [12:12:46]: Cross-Dataset analysis finished at 12:12:46 in 0 seconds (0s), returned:
----------  -------------
train       train       1
            validation  1
            test        1
validation  train       1
            validation  0
            test        0
test        train       1
            validation  0
            test        0
----------  -------------
MASTERFUL [12:12:46]: Meta-Learning architecture parameters...
MASTERFUL [12:12:49]: Architecture learner finished at 12:12:49 in 3 seconds (3s), returned:
------------------------------  -----------------------------
task                            Task.CLASSIFICATION
num_classes                     5
ensemble_multiplier             1
custom_objects                  {}
model_config
backbone_only                   False
input_shape                     (180, 180, 3)
input_range                     ImageRange.ZERO_255
input_dtype                     <dtype: 'float32'>
input_channels_last             True
prediction_logits               True
prediction_dtype                <dtype: 'float32'>
prediction_structure            TensorStructure.SINGLE_TENSOR
prediction_shape                (5,)
total_parameters                4055976
total_trainable_parameters      4013953
total_non_trainable_parameters  42023
------------------------------  -----------------------------
MASTERFUL [12:12:49]: Meta-learning training dataset parameters...
MASTERFUL [12:12:50]: Training dataset learner finished at 12:12:50 in 1 seconds (1s), returned:
-------------------------  -----------------------------
num_classes                5
task                       Task.CLASSIFICATION
image_shape                (180, 180, 3)
image_range                ImageRange.ZERO_255
image_dtype                <dtype: 'float32'>
image_channels_last        True
label_dtype                <dtype: 'float32'>
label_shape                (5,)
label_structure            TensorStructure.SINGLE_TENSOR
label_sparse               False
label_bounding_box_format
-------------------------  -----------------------------
MASTERFUL [12:12:50]: Meta-learning validation dataset parameters...
MASTERFUL [12:12:51]: Validation dataset learner finished at 12:12:51 in 1 seconds (1s), returned:
-------------------------  -----------------------------
num_classes                5
task                       Task.CLASSIFICATION
image_shape                (180, 180, 3)
image_range                ImageRange.ZERO_255
image_dtype                <dtype: 'float32'>
image_channels_last        True
label_dtype                <dtype: 'float32'>
label_shape                (5,)
label_structure            TensorStructure.SINGLE_TENSOR
label_sparse               False
label_bounding_box_format
-------------------------  -----------------------------
MASTERFUL [12:12:51]: Meta-learning test dataset parameters...
MASTERFUL [12:12:52]: Test dataset learner finished at 12:12:52 in 1 seconds (1s), returned:
-------------------------  -----------------------------
num_classes                5
task                       Task.CLASSIFICATION
image_shape                (180, 180, 3)
image_range                ImageRange.ZERO_255
image_dtype                <dtype: 'float32'>
image_channels_last        True
label_dtype                <dtype: 'float32'>
label_shape                (5,)
label_structure            TensorStructure.SINGLE_TENSOR
label_sparse               False
label_bounding_box_format
-------------------------  -----------------------------
MASTERFUL [12:12:52]: Meta-Learning optimization parameters...
Callbacks: 100%|███████████████████████████████| 8/8 [01:14<00:00,  9.31s/steps]
MASTERFUL [12:14:07]: Optimization learner finished at 12:14:07 in 75 seconds (1m 15s), returned:
-----------------------  -----------------------------------------------------------------
batch_size               64
drop_remainder           False
epochs                   1000000
learning_rate            0.0017677668947726488
learning_rate_schedule
learning_rate_callback   <keras.callbacks.ReduceLROnPlateau object at 0x7f7b3c2d74f0>
warmup_learning_rate     1e-06
warmup_epochs            5
optimizer                <tensorflow_addons.optimizers.lamb.LAMB object at 0x7f7ad8150280>
loss                     <keras.losses.CategoricalCrossentropy object at 0x7f7ad8150790>
loss_weights
early_stopping_callback  <keras.callbacks.EarlyStopping object at 0x7f7ad81508b0>
metrics                  [<keras.metrics.CategoricalAccuracy object at 0x7f7ad8150ee0>]
readonly_callbacks
-----------------------  -----------------------------------------------------------------
MASTERFUL [12:14:09]: Meta-Learning Regularization Parameters...
MASTERFUL [12:14:10]: Warming up model for analysis.
MASTERFUL [12:14:14]:   Warming up batch norm statistics (this could take a few minutes).
MASTERFUL [12:14:17]:   Warming up training for 510 steps.
100%|██████████████████████████████████████| 510/510 [01:32<00:00,  5.54steps/s]
MASTERFUL [12:15:49]:   Validating batch norm statistics after warmup for stability (this could take a few minutes).
MASTERFUL [12:15:52]: Analyzing baseline model performance. Training until validation loss stabilizes...
Baseline Training: 100%|█████████████████| 2340/2340 [07:10<00:00,  5.43steps/s]
MASTERFUL [12:23:19]: Baseline training complete.
MASTERFUL [12:23:19]: Meta-Learning Basic Data Augmentations...
Node 1/4: 100%|██████████████████████████| 1040/1040 [03:06<00:00,  5.57steps/s]
Node 2/4: 100%|██████████████████████████| 1040/1040 [03:08<00:00,  5.51steps/s]
Node 3/4: 100%|██████████████████████████| 1040/1040 [03:08<00:00,  5.53steps/s]
Node 4/4: 100%|██████████████████████████| 1040/1040 [03:08<00:00,  5.52steps/s]
MASTERFUL [12:36:40]: Meta-Learning Data Augmentation Clusters...
Distance Analysis: 100%|███████████████████| 143/143 [01:00<00:00,  2.38steps/s]
Node 1/10: 100%|█████████████████████████| 1040/1040 [03:33<00:00,  4.87steps/s]
Node 2/10: 100%|█████████████████████████| 1040/1040 [03:33<00:00,  4.86steps/s]
Node 3/10: 100%|█████████████████████████| 1040/1040 [03:33<00:00,  4.86steps/s]
Node 4/10: 100%|█████████████████████████| 1040/1040 [03:34<00:00,  4.86steps/s]
Node 5/10: 100%|█████████████████████████| 1040/1040 [03:33<00:00,  4.86steps/s]
Distance Analysis: 100%|█████████████████████| 66/66 [00:28<00:00,  2.31steps/s]
Node 6/10: 100%|█████████████████████████| 1040/1040 [03:38<00:00,  4.77steps/s]
Node 7/10: 100%|█████████████████████████| 1040/1040 [03:38<00:00,  4.76steps/s]
Node 8/10: 100%|█████████████████████████| 1040/1040 [03:38<00:00,  4.76steps/s]
Node 9/10: 100%|█████████████████████████| 1040/1040 [03:38<00:00,  4.77steps/s]
Node 10/10: 100%|████████████████████████| 1040/1040 [03:38<00:00,  4.77steps/s]
MASTERFUL [13:16:20]: Meta-Learning Label Based Regularization...
Node 1/2: 100%|██████████████████████████| 1040/1040 [03:39<00:00,  4.74steps/s]
Node 2/2: 100%|██████████████████████████| 1040/1040 [03:39<00:00,  4.75steps/s]
MASTERFUL [13:24:06]: Meta-Learning Weight Based Regularization...
MASTERFUL [13:24:07]: Analysis finished in 69.93716329336166 minutes.
MASTERFUL [13:24:07]: Learned parameters harrier-onyx-cobweb saved at /home/yaoshiang/.masterful/policies/harrier-onyx-cobweb.
MASTERFUL [13:24:07]: Regularization learner finished at 13:24:07 in 4200 seconds (1h 10m 0s), returned:
-------------------------  -----------------------------------------------
shuffle_buffer_size        2936
mirror                     1.0
rot90                      1.0
rotate                     0
mixup                      0.0
cutmix                     0.0
label_smoothing            0
hsv_cluster                4
hsv_cluster_to_index       [[ 2  4  6 11 11 11]
                            [ 2  3  4  6  9 11]
                            [ 1  2  4  5  6 11]
                            [ 1  2  3  6  9 11]
                            [ 2  2  2  2  5 11]]
hsv_magnitude_table        [[  0  10  20  30  40  50  60  70  80  90 100]
                            [  0  10  20  30  40  50  60  70  80  90 100]
                            [  0  10  20  30  40  50  60  70 100  90  80]
                            [  0  10  20  30  40  50  60  70  80  90 100]
                            [100   0  10  90  80  50  20  40  60  30  70]]
contrast_cluster           4
contrast_cluster_to_index  [[ 4 11 11 11 11 11]
                            [ 1  1  1  1  7 11]
                            [ 4  6  6  7  9 11]
                            [ 1  2  5  9 11 11]
                            [ 1  2  5  9 11 11]
                            [ 1  2  4  6  8 11]]
contrast_magnitude_table   [[  0  10  20  30  40  50  60  70  80  90 100]
                            [  0  10  20  30  50  40  60  70 100  80  90]
                            [  0  10  20  30  40  50  60  70  80  90 100]
                            [  0  10  20  30  40  50  60  70  80  90 100]
                            [  0  10  20  30  40  50  60  70  80  90 100]
                            [  0  10  20  30  40  50  60  70  80  90 100]]
blur_cluster               4
blur_cluster_to_index      [[ 1  4 11 11 11 11]
                            [ 3  7 10 11 11 11]]
blur_magnitude_table       [[  0  10  20  30  40  50  60  70  80  90 100]
                            [  0  50  10  20  40  30  60  70  80  90 100]]
spatial_cluster            5
spatial_cluster_to_index   [[ 1  3  5  6  7 11]
                            [ 1  3  4  5  7 11]
                            [ 2  3  8 10 11 11]
                            [ 1  4  6  8 11 11]
                            [ 4  7  7 10 11 11]
                            [ 2  2  3  5  8 11]]
spatial_magnitude_table    [[  0  20  10  30  40  50  60  70  80  90 100]
                            [  0  20  10  30  40  50  60  70  80  90 100]
                            [  0 100  10  20  90  30  80  70  40  50  60]
                            [  0  10 100  20  90  80  30  70  40  60  50]
                            [  0  50  20  30  60  10  40  70  80  90 100]
                            [  0  10  20  30  40  50  60  70  80 100  90]]
synthetic_proportion       [0.0]
-------------------------  -----------------------------------------------
MASTERFUL [13:24:07]: Learning SSL parameters...
MASTERFUL [13:24:08]: SSL learner finished at 13:24:08 in 1 seconds (1s), returned:
----------  --
algorithms  []
----------  --
MASTERFUL [13:24:09]: Training model with semi-supervised learning disabled.
MASTERFUL [13:24:09]: Performing basic dataset analysis.
MASTERFUL [13:24:10]: Training model with:
MASTERFUL [13:24:10]:   2936 labeled examples.
MASTERFUL [13:24:10]:   367 validation examples.
MASTERFUL [13:24:10]:   0 synthetic examples.
MASTERFUL [13:24:10]:   0 unlabeled examples.
MASTERFUL [13:24:10]: Training model with learned parameters harrier-onyx-cobweb in two phases.
MASTERFUL [13:24:10]: The first phase is supervised training with the learned parameters.
MASTERFUL [13:24:10]: The second phase is semi-supervised training to boost performance.
MASTERFUL [13:24:12]: Warming up model for supervised training.
MASTERFUL [13:24:16]:   Warming up batch norm statistics (this could take a few minutes).
MASTERFUL [13:24:19]:   Warming up training for 510 steps.
100%|██████████████████████████████████████| 510/510 [01:40<00:00,  5.08steps/s]
MASTERFUL [13:25:59]:   Validating batch norm statistics after warmup for stability (this could take a few minutes).
MASTERFUL [13:26:09]: Starting Phase 1: Supervised training until the validation loss stabilizes...
Supervised Training: 100%|███████████████| 2756/2756 [10:59<00:00,  4.18steps/s]
MASTERFUL [13:37:25]: Semi-Supervised training disabled in parameters.
MASTERFUL [13:37:27]: Training complete in 13.261691228548687 minutes.
MASTERFUL [13:37:46]: ************************************
MASTERFUL [13:37:46]: Evaluating model on 367 examples from the 'test' dataset split:
MASTERFUL [13:37:46]:   Loss: 0.1692
MASTERFUL [13:37:46]:   Categorical Accuracy: 0.9482
MASTERFUL [13:37:49]:   Average Precision: 0.9476
MASTERFUL [13:37:49]:   Average Recall:    0.9447
MASTERFUL [13:37:49]:   Confusion Matrix:
MASTERFUL [13:37:49]:              |     daisy| dandelion|      rose| sunflower|     tulip|
MASTERFUL [13:37:49]:         daisy|        61|         1|         0|         1|         1|
MASTERFUL [13:37:49]:     dandelion|         1|        87|         0|         2|         0|
MASTERFUL [13:37:49]:          rose|         0|         0|        50|         0|         7|
MASTERFUL [13:37:49]:     sunflower|         0|         0|         1|        71|         0|
MASTERFUL [13:37:49]:         tulip|         0|         1|         4|         0|        79|
MASTERFUL [13:37:49]:     Confusion matrix columns represent the prediction labels and the rows represent the real labels.
MASTERFUL [13:37:49]:
MASTERFUL [13:37:49]:   Per-Class Metrics:
MASTERFUL [13:37:49]:     Class daisy:
MASTERFUL [13:37:49]:       Precision: 0.9839
MASTERFUL [13:37:49]:       Recall   : 0.9531
MASTERFUL [13:37:49]:     Class dandelion:
MASTERFUL [13:37:49]:       Precision: 0.9775
MASTERFUL [13:37:49]:       Recall   : 0.9667
MASTERFUL [13:37:49]:     Class rose:
MASTERFUL [13:37:49]:       Precision: 0.9091
MASTERFUL [13:37:49]:       Recall   : 0.8772
MASTERFUL [13:37:49]:     Class sunflower:
MASTERFUL [13:37:49]:       Precision: 0.9595
MASTERFUL [13:37:49]:       Recall   : 0.9861
MASTERFUL [13:37:49]:     Class tulip:
MASTERFUL [13:37:49]:       Precision: 0.9080
MASTERFUL [13:37:49]:       Recall   : 0.9405
MASTERFUL [13:37:49]: Saving model output to /home/yaoshiang/model_output/session-00000.
MASTERFUL [13:37:49]:     Saving saved_model output to /home/yaoshiang/model_output/session-00000/saved_model
MASTERFUL [13:38:10]:     Saving onnx output to /home/yaoshiang/model_output/session-00000/onnx
MASTERFUL [13:38:42]: Saving evaluation metrics to /home/yaoshiang/model_output/session-00000/evaluation_metrics.csv.
MASTERFUL [13:38:42]: Saving regularization params to /home/yaoshiang/model_output/session-00000/regularization.params.
MASTERFUL [13:38:42]: Saving confusion matrix to /home/yaoshiang/model_output/session-00000/confusion_matrix.csv.
MASTERFUL [13:38:42]: Total elapsed training time: 94 minutes (1h 33m 55s).
MASTERFUL [13:38:42]: Launch masterful-gui to visualize the training results: policy name 'harrier-onyx-cobweb'

Pro Tip!

If you are training on a remote machine over an SSH connection, you will want to ensure that your training session does not die if your SSH connected gets killed. You can use nohup to help here!

nohup masterful-train --config=s3://masterful-public/datasets/quickstart/training.yaml &> training_log.txt & while ! test -f training_log.txt; do :; done && tail -f training_log.txt

The above command will launch the job in the background, and then tail the output as it comes. You can CTRL-C the output at any time, and the job will continue to you. Simply tail the output file again to pick up the training log in progress.

You have now finished training your first model with the Masterful AutoML platform!

Next Steps

Now that you have finished this basic tutorial, we suggest exploring the rest of the documentation for more advanced examples and use cases. For example, you can learn how to use unlabeled data to further improve the performance of your model. Or you can learn about other vision tasks such as object detection and segmentation. Don’t see a topic that you are interested in? Reach out to us directly on email at learn@masterfulai.com or join our Slack commmunity. We are happy to help walk you through whatever challenge you are facing!