Quickstart Tutorial¶
Quickstart Tutorial¶
In this short introduction to the Masterful AutoML Platform, you will train a model start to finish with Masterful. This example will walk you through creating a simple dataset and model, and using that dataset and model with the Masterful AutoML platform. This example is intended to demonstrate workflows, not train a state of the art model.
Prerequisites¶
Please follow the Masterful installation instructions here in order to run this Quickstart.
Imports¶
First, import the necessary libraries and register the Masterful package.
[1]:
import tensorflow as tf
import masterful
masterful = masterful.register()
MASTERFUL: Your account has been successfully registered. Masterful v0.4.1 is loaded.
Setup Training Data¶
In this example, you will use the CIFAR-10 which is a common training dataset of 10 classes used in a lot of research papers.
Masterful supports many different types of data input formats. The recommended format is to use tf.data.Dataset objects constructed from your training and validation datasets. Masterful also supports:
A 2-tuple of numpy N-dimensional arrays, containing the images and the labels as the two items in the tuple.
A tf.keras.utils.Sequence object, which generates individual batches of images and labels.
A 2-tuple containing a generator function and the output signature of the items returned by the generator.
See the Data Input Types guide for more information about the different data input formats supported by Masterful.
[2]:
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()
Create the Model¶
Build a simple model to classify CIFAR-10 data. This is a toy model for demonstration purposes only, and should not be used in a production environment. It has a few convolutional layers and outputs logits directly, rather than using a softmax layer at the end. Outputting logits directly allows Tensorflow the ability to use a more numerically stable version of some loss functions.
[3]:
NUM_CLASSES = 10
def get_model():
model = tf.keras.models.Sequential()
model.add(
tf.keras.layers.experimental.preprocessing.Rescaling(
1.0 / 255, input_shape=(32, 32, 3)
)
)
model.add(
tf.keras.layers.Conv2D(
16,
(3, 3),
activation="relu",
)
)
model.add(tf.keras.layers.MaxPooling2D((2, 2)))
model.add(
tf.keras.layers.Conv2D(
32,
(3, 3),
activation="relu",
)
)
model.add(tf.keras.layers.MaxPooling2D((2, 2)))
model.add(
tf.keras.layers.Conv2D(
64,
(3, 3),
activation="relu",
)
)
model.add(tf.keras.layers.MaxPooling2D((2, 2)))
model.add(tf.keras.layers.GlobalAveragePooling2D())
model.add(tf.keras.layers.Dense(NUM_CLASSES))
return model
model = get_model()
Setup Masterful¶
The Masterful AutoML platform learns how to train your model by focusing on five core organizational principles in deep learning: architecture, data, optimization, regularization, and semi-supervision.
Architecture is the structure of weights, biases, and activations that define a model. In this example, the architecture is defined by the model you created above.
Data is the input used to train the model. In this example, you are using a labeled training dataset - CIFAR-10. More advanced usages of the Masterful AutoML platform can take into account unlabeled and synthetic data as well, using a variety of different techniques.
Optimization means finding the best weights for a model and training data. Optimization is different from regularization because optimization does not consider generalization to unseen data. The central challenge of optimization is speed - find the best weights faster.
Regularization means helping a model generalize to data it has not yet seen. Another way of saying this is that regularization is about fighting overfitting.
Semi-Supervision is the process by which a model can be trained using both labeled and unlabeled data.
The first step when using Masterful is to learn the optimal set of parameters for each of the five buckets above. You start by learning the architecture and data parameters of the model and training dataset. In the code below, you are telling Masterful that your model is performing a classification task (masterful.enums.Task.CLASSIFICATION) with 10 labels (num_classes=NUM_CLASSES), and that the input range of the image features going into your model are in the range [0,255]
(input_range=masterful.enums.ImageRange.ZERO_255). Also, the model outputs logits rather than a softmax classification (prediction_logits=True).
Furthermore, in the training dataset, you are providing sparse labels (sparse_labels=True) rather than dense (one-hot) labels.
For more details on architecture and data parameters, see the API specifications for ArchitectureParams and DataParams.
[4]:
model_params = masterful.architecture.learn_architecture_params(
model=model,
task=masterful.enums.Task.CLASSIFICATION,
input_range=masterful.enums.ImageRange.ZERO_255,
num_classes=NUM_CLASSES,
prediction_logits=True,
)
training_dataset_params = masterful.data.learn_data_params(
dataset=(x_train, y_train),
task=masterful.enums.Task.CLASSIFICATION,
image_range=masterful.enums.ImageRange.ZERO_255,
num_classes=NUM_CLASSES,
sparse_labels=True,
)
Next you learn the optimization parameters that will be used to train the model. Below, you use Masterful to learn the standard set of optimization parameters to train your model for a classification task.
For more details on the optmization parameters, please see the OptimizationParams API specification.
[5]:
optimization_params = masterful.optimization.learn_optimization_params(
model,
model_params,
(x_train, y_train),
training_dataset_params,
)
MASTERFUL: Learning optimal batch size.
MASTERFUL: Learning optimal initial learning rate for batch size 32.
The regularization parameters used can have a dramatic impact on the final performance of your trained model. Learning these parameters can be a time-consuming and domain specific challenge. Masterful can speed up this process by learning these parameters for you. In general, this can be an expensive operation. A rough order of magnitude for learning these parameters is 2x the time it takes to train your model. However, this is still dramatically faster than manually finding these parameters yourself. In the example below, you will use one of the many sets of pre-learned regularization parameters that are shipped in the Masterful API. In most instances, you should learn these parameters directly using the learn_regularization_params API.
For more details on the regularization parameters, please see the RegularizationParams API specification.
[6]:
# This is a set of parameters learned on CIFAR10 for
# small sized models.
regularization_params = masterful.regularization.parameters.CIFAR10_SMALL
The final step before training is to learn the optimal set of semi-supervision parameters. For this Quickstart, we are not using any unlabeled or synthetic data as part of training, so most forms of semi-supervision will be disabled by default.
For more details on the semi-supervision parameters, please see the SemiSupervisedParams API specification.
[7]:
ssl_params = masterful.ssl.learn_ssl_params(
(x_train, y_train),
training_dataset_params,
)
Training¶
Now, you are ready to train your model using the Masterful AutoML platform. In the next cell, you will see the call to masterful.training.train, which is the entry point to the meta-learning engine of the Masterful AutoML platform. Notice there is no need to batch your data (Masterful will find the optimal batch size for you). No need to shuffle your data (Masterful handles this for you). You don’t even need to pass in a validation dataset (Masterful finds one for you). You hand Masterful a model and a dataset, and Masterful will figure the rest out for you.
[8]:
# For demonstration purposes only, only train for 5 epochs.
# For production training, you do not need to override these
# parameters.
optimization_params.epochs = 5
optimization_params.warmup_epochs = 1
# Train with Masterful
training_report = masterful.training.train(
model,
model_params,
optimization_params,
regularization_params,
ssl_params,
(x_train, y_train),
training_dataset_params,
)
MASTERFUL: Training model with semi-supervised learning disabled.
MASTERFUL: Performing basic dataset analysis.
MASTERFUL: Masterful will use 5000 labeled examples as a validation set since no validation data was provided.
MASTERFUL: Training model with:
MASTERFUL: 45000 labeled examples.
MASTERFUL: 5000 validation examples.
MASTERFUL: 0 synthetic examples.
MASTERFUL: 0 unlabeled examples.
MASTERFUL: Training model with learned parameters orangutan-pretty-garden in two phases.
MASTERFUL: The first phase is supervised training with the learned parameters.
MASTERFUL: The second phase is semi-supervised training to boost performance.
MASTERFUL: Warming up model for supervised training.
MASTERFUL: Warming up batch norm statistics (this could take a few minutes).
MASTERFUL: Warming up training for 1407 steps.
100%|██████████| 1407/1407 [01:47<00:00, 13.05steps/s]
MASTERFUL: Validating batch norm statistics after warmup for stability (this could take a few minutes).
MASTERFUL: Starting Phase 1: Supervised training until the validation loss stabilizes...
Supervised Training: 100%|██████████| 7820/7820 [06:06<00:00, 21.32steps/s]
MASTERFUL: Semi-Supervised training disabled in parameters.
MASTERFUL: Training complete in 9.582347786426544 minutes.
The model you passed into masterful.training.train is now trained and updated in place, so you are able to evaluate it just like any other trained Keras model.
[9]:
model.compile(
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=tf.keras.metrics.SparseCategoricalAccuracy(),
)
model.evaluate(x_test, y_test)
313/313 [==============================] - 3s 8ms/step - loss: 1.3584 - sparse_categorical_accuracy: 0.5161
[9]:
[1.3614277839660645, 0.5127999782562256]
You have now finished training your first model with the Masterful AutoML platform!
Next Steps¶
Now that you have finished this basic tutorial, we suggest exploring the rest of the documentation for more advanced examples and use cases. For example, you can learn how to use unlabeled data to further improve the performance of your model. Or you can learn about other vision tasks such as object detection and segmentation. Don’t see a topic that you are interested in? Reach out to us directly on email at learn@masterfulai.com or join our Slack commmunity. We are happy to help walk you through whatever challenge you are facing!