{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "# Quickstart Tutorial\n", "\n", "**Author:** [sam](mailto:sam@masterfulai.com) \n", "**Date created:** 2022/03/29 \n", "**Last modified:** 2022/05/08 \n", "**Description:** Quick guide to getting started with Masterful." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)][1]        [![Download](images/download.png)][2][Download this Notebook][2]\n", "\n", "[1]:https://colab.research.google.com/github/masterfulai/masterful-docs/blob/main/notebooks/tutorial_quickstart.ipynb\n", "[2]:https://docs.masterfulai.com/0.4.1/notebooks/tutorial_quickstart.ipynb\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "## Quickstart Tutorial\n", "\n", "In this short introduction to the Masterful AutoML Platform, you will\n", "train a model start to finish with Masterful. This example will walk\n", "you through creating a simple dataset and model, and using that\n", "dataset and model with the Masterful AutoML platform. This example\n", "is intended to demonstrate workflows, not train a state of the\n", "art model.\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "## Prerequisites\n", "\n", "Please follow the Masterful installation instructions [here](../tutorials/tutorial_installation.md)\n", "in order to run this Quickstart." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "## Imports\n", "\n", "First, import the necessary libraries and register the Masterful package." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:00:24.920791Z", "iopub.status.busy": "2022-05-09T17:00:24.919924Z", "iopub.status.idle": "2022-05-09T17:00:55.207919Z", "shell.execute_reply": "2022-05-09T17:00:55.208417Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MASTERFUL: Your account has been successfully registered. Masterful v0.4.1 is loaded.\n" ] } ], "source": [ "import tensorflow as tf\n", "\n", "import masterful\n", "\n", "masterful = masterful.register()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "## Setup Training Data\n", "\n", "In this example, you will use the [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html) which\n", "is a common training dataset of 10 classes used in a lot of research\n", "papers.\n", "\n", "Masterful supports many different types of data input formats. The\n", "recommended format is to use [tf.data.Dataset](https://www.tensorflow.org/api_docs/python/tf/data/Dataset)\n", "objects constructed from your training and validation datasets. Masterful\n", "also supports:\n", "\n", "* A 2-tuple of numpy N-dimensional arrays, containing the images and the labels as the two items in the tuple.\n", "* A [tf.keras.utils.Sequence](https://www.tensorflow.org/api_docs/python/tf/keras/utils/Sequence) object, which generates individual batches of images and labels.\n", "* A 2-tuple containing a generator function and the output signature of the items returned by the generator.\n", "\n", "See the [Data Input Types](../concepts/guide_data_input_types.md) guide for\n", "more information about the different data input formats supported by Masterful." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:00:55.214622Z", "iopub.status.busy": "2022-05-09T17:00:55.213758Z", "iopub.status.idle": "2022-05-09T17:00:55.825150Z", "shell.execute_reply": "2022-05-09T17:00:55.825668Z" } }, "outputs": [], "source": [ "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "## Create the Model\n", "\n", "Build a simple model to classify CIFAR-10 data. This is a toy model\n", "for demonstration purposes only, and should not be used in a production\n", "environment. It has a few convolutional layers and outputs logits\n", "directly, rather than using a softmax layer at the end. Outputting\n", "logits directly allows Tensorflow the ability to use a more numerically\n", "stable version of some loss functions." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:00:55.836284Z", "iopub.status.busy": "2022-05-09T17:00:55.835345Z", "iopub.status.idle": "2022-05-09T17:00:55.919372Z", "shell.execute_reply": "2022-05-09T17:00:55.919957Z" } }, "outputs": [], "source": [ "NUM_CLASSES = 10\n", "\n", "\n", "def get_model():\n", " model = tf.keras.models.Sequential()\n", " model.add(\n", " tf.keras.layers.experimental.preprocessing.Rescaling(\n", " 1.0 / 255, input_shape=(32, 32, 3)\n", " )\n", " )\n", " model.add(\n", " tf.keras.layers.Conv2D(\n", " 16,\n", " (3, 3),\n", " activation=\"relu\",\n", " )\n", " )\n", " model.add(tf.keras.layers.MaxPooling2D((2, 2)))\n", " model.add(\n", " tf.keras.layers.Conv2D(\n", " 32,\n", " (3, 3),\n", " activation=\"relu\",\n", " )\n", " )\n", " model.add(tf.keras.layers.MaxPooling2D((2, 2)))\n", " model.add(\n", " tf.keras.layers.Conv2D(\n", " 64,\n", " (3, 3),\n", " activation=\"relu\",\n", " )\n", " )\n", " model.add(tf.keras.layers.MaxPooling2D((2, 2)))\n", " model.add(tf.keras.layers.GlobalAveragePooling2D())\n", " model.add(tf.keras.layers.Dense(NUM_CLASSES))\n", " return model\n", "\n", "\n", "model = get_model()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "## Setup Masterful\n", "\n", "The Masterful AutoML platform learns how to train your model by\n", "focusing on five core organizational principles in deep\n", "learning: architecture, data, optimization, regularization,\n", "and semi-supervision.\n", "\n", "**Architecture** is the structure of weights, biases, and activations\n", "that define a model. In this example, the architecture is defined by the model you created above.\n", "\n", "**Data** is the input used to train the model. In this example, you\n", "are using a labeled training dataset - [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html).\n", "More advanced usages of the Masterful AutoML platform can take into account unlabeled and synthetic\n", "data as well, using a variety of different techniques.\n", "\n", "**Optimization** means finding the best weights for a model and\n", "training data. Optimization is different from regularization because\n", "optimization does not consider generalization to unseen data. The\n", "central challenge of optimization is speed - find the best weights\n", "faster.\n", "\n", "**Regularization** means helping a model generalize to data it has\n", "not yet seen. Another way of saying this is that regularization is\n", "about fighting overfitting.\n", "\n", "**Semi-Supervision** is the process by which a model can be trained\n", "using both labeled and unlabeled data.\n", "\n", "The first step when using Masterful is to learn the optimal set of\n", "parameters for each of the five buckets above. You start by learning\n", "the architecture and data parameters of the model and training dataset. In the code below, you are telling Masterful that your model is performing a classification task (`masterful.enums.Task.CLASSIFICATION`) with 10 labels (`num_classes=NUM_CLASSES`), and that the input range of the image features going into your model are in the range [0,255] (`input_range=masterful.enums.ImageRange.ZERO_255`). Also, the model outputs logits rather than a softmax classification (`prediction_logits=True`).\n", "\n", "Furthermore, in the training dataset, you are providing sparse labels\n", "(`sparse_labels=True`) rather than dense (one-hot) labels.\n", "\n", "For more details on architecture and data parameters, see the API\n", "specifications for [ArchitectureParams](../api/api_architecture.rst#masterful.architecture.ArchitectureParams) and\n", "[DataParams](../api/api_data.rst#masterful.data.DataParams)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:00:55.926514Z", "iopub.status.busy": "2022-05-09T17:00:55.925646Z", "iopub.status.idle": "2022-05-09T17:00:56.243705Z", "shell.execute_reply": "2022-05-09T17:00:56.244252Z" } }, "outputs": [], "source": [ "model_params = masterful.architecture.learn_architecture_params(\n", " model=model,\n", " task=masterful.enums.Task.CLASSIFICATION,\n", " input_range=masterful.enums.ImageRange.ZERO_255,\n", " num_classes=NUM_CLASSES,\n", " prediction_logits=True,\n", ")\n", "training_dataset_params = masterful.data.learn_data_params(\n", " dataset=(x_train, y_train),\n", " task=masterful.enums.Task.CLASSIFICATION,\n", " image_range=masterful.enums.ImageRange.ZERO_255,\n", " num_classes=NUM_CLASSES,\n", " sparse_labels=True,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "Next you learn the optimization parameters that will be used to train\n", "the model. Below, you use Masterful to learn the standard set of\n", "optimization parameters to train your model for a classification task.\n", "\n", "For more details on the optmization parameters, please see the [OptimizationParams](../api/api_optimization.rst#masterful.optimization.OptimizationParams) API specification." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:00:56.248875Z", "iopub.status.busy": "2022-05-09T17:00:56.247996Z", "iopub.status.idle": "2022-05-09T17:01:02.327131Z", "shell.execute_reply": "2022-05-09T17:01:02.327798Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MASTERFUL: Learning optimal batch size.\n", "MASTERFUL: Learning optimal initial learning rate for batch size 32.\n" ] } ], "source": [ "optimization_params = masterful.optimization.learn_optimization_params(\n", " model,\n", " model_params,\n", " (x_train, y_train),\n", " training_dataset_params,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "The regularization parameters used can have a dramatic impact on the\n", "final performance of your trained model. Learning these parameters can\n", "be a time-consuming and domain specific challenge. Masterful can speed\n", "up this process by learning these parameters for you. In general, this\n", "can be an expensive operation. A rough order of magnitude for learning\n", "these parameters is 2x the time it takes to train your model. However,\n", "this is still dramatically faster than manually finding these\n", "parameters yourself. In the example below, you will use one of the\n", "many sets of pre-learned regularization parameters that are shipped\n", "in the Masterful API. In most instances, you should learn these\n", "parameters directly using the [learn_regularization_params](../api/api_regularization.rst#masterful.regularization.learn_regularization_params) API.\n", "\n", "For more details on the regularization parameters, please see the\n", "[RegularizationParams](../api/api_regularization.rst#masterful.regularization.RegularizationParams) API specification." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:01:02.332629Z", "iopub.status.busy": "2022-05-09T17:01:02.331864Z", "iopub.status.idle": "2022-05-09T17:01:02.333733Z", "shell.execute_reply": "2022-05-09T17:01:02.334325Z" } }, "outputs": [], "source": [ "# This is a set of parameters learned on CIFAR10 for\n", "# small sized models.\n", "regularization_params = masterful.regularization.parameters.CIFAR10_SMALL" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "The final step before training is to learn the optimal set of\n", "semi-supervision parameters. For this Quickstart, you are not using\n", "any unlabeled or synthetic data as part of training, so most forms\n", "of semi-supervision will be disabled by default.\n", "\n", "For more details on the semi-supervision parameters, please see the\n", "[SemiSupervisedParams](../api/api_ssl.rst#masterful.ssl.SemiSupervisedParams) API specification." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:01:02.338764Z", "iopub.status.busy": "2022-05-09T17:01:02.337717Z", "iopub.status.idle": "2022-05-09T17:01:02.340077Z", "shell.execute_reply": "2022-05-09T17:01:02.340621Z" } }, "outputs": [], "source": [ "ssl_params = masterful.ssl.learn_ssl_params(\n", " (x_train, y_train),\n", " training_dataset_params,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "## Training\n", "\n", "Now, you are ready to train your model using the Masterful AutoML\n", "platform. In the next cell, you will see the call to\n", "[masterful.training.train](../api/api_training.rst#masterful.training.train),\n", "which is the entry point to the meta-learning engine of the Masterful AutoML\n", "platform. Notice there is no need to batch your data (Masterful will\n", "find the optimal batch size for you). No need to shuffle your data\n", "(Masterful handles this for you). You don't even need to pass in a\n", "validation dataset (Masterful finds one for you). You hand Masterful\n", "a model and a dataset, and Masterful will figure the rest out for you." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:01:02.345992Z", "iopub.status.busy": "2022-05-09T17:01:02.345198Z", "iopub.status.idle": "2022-05-09T17:10:53.463051Z", "shell.execute_reply": "2022-05-09T17:10:53.463538Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MASTERFUL: Training model with semi-supervised learning disabled.\n", "MASTERFUL: Performing basic dataset analysis.\n", "MASTERFUL: Masterful will use 5000 labeled examples as a validation set since no validation data was provided.\n", "MASTERFUL: Training model with:\n", "MASTERFUL: \t45000 labeled examples.\n", "MASTERFUL: \t5000 validation examples.\n", "MASTERFUL: \t0 synthetic examples.\n", "MASTERFUL: \t0 unlabeled examples.\n", "MASTERFUL: Training model with learned parameters orangutan-pretty-garden in two phases.\n", "MASTERFUL: The first phase is supervised training with the learned parameters.\n", "MASTERFUL: The second phase is semi-supervised training to boost performance.\n", "MASTERFUL: Warming up model for supervised training.\n", "MASTERFUL: \tWarming up batch norm statistics (this could take a few minutes).\n", "MASTERFUL: \tWarming up training for 1407 steps.\n", "100%|██████████| 1407/1407 [01:47<00:00, 13.05steps/s]\n", "MASTERFUL: \tValidating batch norm statistics after warmup for stability (this could take a few minutes).\n", "MASTERFUL: Starting Phase 1: Supervised training until the validation loss stabilizes...\n", "Supervised Training: 100%|██████████| 7820/7820 [06:06<00:00, 21.32steps/s] \n", "MASTERFUL: Semi-Supervised training disabled in parameters.\n", "MASTERFUL: Training complete in 9.582347786426544 minutes.\n" ] } ], "source": [ "# For demonstration purposes only, only train for 5 epochs.\n", "# For production training, you do not need to override these\n", "# parameters.\n", "optimization_params.epochs = 5\n", "optimization_params.warmup_epochs = 1\n", "\n", "# Train with Masterful\n", "training_report = masterful.training.train(\n", " model,\n", " model_params,\n", " optimization_params,\n", " regularization_params,\n", " ssl_params,\n", " (x_train, y_train),\n", " training_dataset_params,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "The model you passed into [masterful.training.train](../api/api_training.rst#masterful.training.train)\n", "is now trained and updated in place, so you are able to evaluate it\n", "just like any other trained Keras model." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab_type": "code", "execution": { "iopub.execute_input": "2022-05-09T17:10:53.489506Z", "iopub.status.busy": "2022-05-09T17:10:53.488689Z", "iopub.status.idle": "2022-05-09T17:10:56.179709Z", "shell.execute_reply": "2022-05-09T17:10:56.180199Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "313/313 [==============================] - 3s 8ms/step - loss: 1.3584 - sparse_categorical_accuracy: 0.5161\n" ] }, { "data": { "text/plain": [ "[1.3614277839660645, 0.5127999782562256]" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.compile(\n", " loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n", " metrics=tf.keras.metrics.SparseCategoricalAccuracy(),\n", ")\n", "model.evaluate(x_test, y_test)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ "You have now finished training your first model with the Masterful\n", "AutoML platform!\n", "\n", "## Next Steps\n", "\n", "Now that you have finished this basic tutorial, we suggest exploring\n", "the rest of the documentation for more advanced examples and use cases.\n", "For example, you can learn how to use unlabeled data to further\n", "improve the performance of your model. Or you can learn about other\n", "vision tasks such as object detection and segmentation. Don't see a\n", "topic that you are interested in? Reach out to us directly on email\n", "at learn@masterfulai.com or join our Slack [commmunity](https://www.masterfulai.com/community).\n", "We are happy to help walk you through whatever challenge you are facing!" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "tutorial_quickstart", "private_outputs": false, "provenance": [], "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.9" } }, "nbformat": 4, "nbformat_minor": 0 }