# Installation Tutorial ## Prerequisites ### Supported Linux Distributions Masterful is supported on most Linux distributions that are supported by Tensorflow and PyTorch. ### Python Masterful supports Python 3.6+, which is installed by default in most Linux distributions. ### Pip Masterful is currently distributed on the Python Package Index [here](https://pypi.org/project/masterful/). While Python 3.x is installed by default on most Linux distributions, _pip_ is not installed by default, so make sure to install it if it is not already installed. ``` sudo apt install python3-pip ``` ### Tensorflow 2 Installing the Masterful package does not automatically install Tensorflow, so as not to overwrite the existing version of TF. Therefore Tensorflow, Tensorflow Datasets, and Tensorflow Addons must be installed separately from Masterful. Note that the tensorflow-datasets and tensorflow-addons packages are built for specific versions of the tensorflow package, so make sure to install the appropriate version of each according to their compatibility matrices. Masterful requires Tensorflow versions `2.4.1` or greater. ``` pip install --upgrade pip pip install tensorflow pip install tensorflow-datasets pip install tensorflow-addons ``` ## Masterful Installation Install Masterful with Python's _pip_ package manager. ``` pip install --upgrade pip pip install masterful ``` Verify your installation by running: ``` python -c "import masterful; print(masterful.__version__)" ``` If this returns a version number like `0.5`, your installation was successful. ## Next Steps Now it's time to get started! Go to the [quickstart](../notebooks/tutorial_quickstart_cli) train your first model with Masterful. ## Troubleshooting For the quickest access to help, support, and advice, please join the [masterful-community slack](http://www.masterfulai.com/community). ### I see a warning from Tensorflow Addons that my version of tensorflow is not supported You may see the following warning if there is a version mismatch between Tensorflow and Tensorflow Addons. Typically this happens if you install a specific version of Tensorflow, but install the latest version of Tensorflow Addons without looking at the compatibility matrix. If this happens, you will see a warning in the console that looks like this: ```console ... site-packages/tensorflow_addons/utils/ensure_tf_install.py:53: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.5.0 and strictly below 2.8.0 (nightly versions are not supported). The versions of TensorFlow you are currently using is 2.4.1 and is not supported. Some things might work, some things might not. If you were to encounter a bug, do not file an issue. If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version. You can find the compatibility matrix in TensorFlow Addon's readme: https://github.com/tensorflow/addons warnings.warn( ``` In order to fix this, you can uninstall Tensorflow Addons, and re-install the specific version of Tensorflow Addons that is compatible with your version of Tensorflow. The latest compatibility matrix can be found at https://github.com/tensorflow/addons. For convenience, here is the compatibility matrix for Tensorflow Addons: | TensorFlow Addons | TensorFlow | Python | |:----------------------- |:---|:---------- | | tfa-nightly | 2.5, 2.6, 2.7 | 3.7, 3.8, 3.9 | | tensorflow-addons-0.15.0 | 2.5, 2.6, 2.7 |3.7, 3.8, 3.9 | | tensorflow-addons-0.14.0 | 2.4, 2.5, 2.6 |3.6, 3.7, 3.8, 3.9 | | tensorflow-addons-0.13.0 | 2.3, 2.4, 2.5 |3.6, 3.7, 3.8, 3.9 | | tensorflow-addons-0.12.1 | 2.3, 2.4 |3.6, 3.7, 3.8 | | tensorflow-addons-0.11.2 | 2.2, 2.3 |3.5, 3.6, 3.7, 3.8 | | tensorflow-addons-0.10.0 | 2.2 |3.5, 3.6, 3.7, 3.8 | | tensorflow-addons-0.9.1 | 2.1, 2.2 |3.5, 3.6, 3.7 | | tensorflow-addons-0.8.3 | 2.1 |3.5, 3.6, 3.7 | | tensorflow-addons-0.7.1 | 2.1 | 2.7, 3.5, 3.6, 3.7 | | tensorflow-addons-0.6.0 | 2.0 | 2.7, 3.5, 3.6, 3.7 | ### Activating Masterful raises an AlreadyExistsError when using TensorFlow 2.6 Ensure the Keras version matches the Tensorflow version. Starting in Tensorflow 2.6, Keras was moved out of Tensorflow and back into a separate package, leaving a stale copy of Keras packages under tensorflow/python/keras, which have been removed in Tensorflow 2.7. Because of this TensorFlow 2.6 is known to erroneously use Keras 2.7 as a dependency. Although this bug is not related to Masterful, registering Masterful will expose the incompatibility during their import. Downgrading Keras to 2.6 resolves this issue. ### I see too much Tensorflow logging cluttering up my console By default, Tensorflow sets the C++ logging level to `INFO`, so you will see lots of console output like: ``` 2022-01-17 11:16:11.651649: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2022-01-17 11:16:11.651685: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11 2022-01-17 11:16:11.651710: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11 2022-01-17 11:16:11.651734: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2022-01-17 11:16:11.651757: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2022-01-17 11:16:11.651781: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 20 ``` You can set the environment variable to 1 in your shell, before running your code, to eliminate these `INFO` logs. ``` export TF_CPP_MIN_LOG_LEVEL=1 ``` If setting directly in your python script, make sure to set this variable *before* importing Tensorflow. Otherwise, it won't be picked up. ``` # Must set BEFORE importing Tensorflow (including dependencies that might also # import Tensorflow). import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1' # Now we can import Tensorflow for the first time. import tensorflow ``` ### I receive an AttributeError trying to access masterful.XXX If you try to access any Masterful APIs before activating with your account ID and authorization key, you will receive an `AttributeError` similar to the following: ``` Traceback (most recent call last): File "test.py", line 38, in model_spec, data_spec = masterful.spec.create_model_and_data_specs( AttributeError: module 'masterful' has no attribute 'spec' ``` This is expected, and can be fixed by activating before accessing any Masterful APIs. ``` masterful = masterful.activate() # Now you access all Masterful API calls. ``` ### I see an error related to IProgress when running Masterful within a notebook When Running masterful within a jupyter notebook you might encounter with the following error: ``` ... ~\anaconda3\envs\masterfulAI\lib\site-packages\tqdm\notebook.py in status_printer(_, total, desc, ncols) 113 # Prepare IPython progress bar 114 if IProgress is None: # #187 #451 #558 #872 --> 115 raise ImportError( 116 "IProgress not found. Please update jupyter and ipywidgets." 117 " See https://ipywidgets.readthedocs.io/en/stable" ImportError: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html ``` Masterful uses `tqdm` library in order to print out progress of the trainig loop. Nevertheless for some older versions of `jupyter` and `ipywidgets` it can generate this conflict. Thus, If you happen to run into this issue, please, just update `jupyter` and `ipywidgets`library to the current version. You can see more details on this related thread: https://stackoverflow.com/questions/67998191/importerror-iprogress-not-found-please-update-jupyter-and-ipywidgets-although ### I see an error related with Opencv (cv2) library when using masterful API You could come across with the following error: ``` ... ~/anaconda3/envs/tensorflow2_p38/lib/python3.8/site-packages/cv2/__init__.py in 7 8 from .cv2 import * ----> 9 from .cv2 import _registerMatType 10 from . import mat_wrapper 11 from . import gapi ImportError: cannot import name '_registerMatType' from 'cv2.cv2' (/home/ubuntu/anaconda3/envs/tensorflow2_p38/lib/python3.8/site-packages/cv2/cv2.cpython-38-x86_64-linux-gnu.so) ``` Materful uses opencv-python-headless as a dependency, however this error can appear because the version of `openv-python-headless` must match the version of `opencv-python`. Thus, you can downgrade `opencv-python-headless` to version `4.1.2.30` and the problem will be solved. Just uninstall running `pip uninstall opencv-python-headless` and install it again using the command: `pip install opencv-python-headless==4.1.2.30`. You can see more information on this issue [here](https://stackoverflow.com/questions/70537488/cannot-import-name-registermattype-from-cv2-cv2) ### I receive a NotImplementedError about symbolic Tensors If you see an error similar to the one below in the stack trace: ``` NotImplementedError: Cannot convert a symbolic Tensor (model/masterful_rotate/rotate/_angles_to_projective_transforms/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported ``` then your numpy version is incompatible with your Tensorflow version. Typically this means your numpy version is too high, and you should try downgrading your numpy version to a previous version. For example, with Tensorflow 2.4.1, `numpy==1.19.5` is known to work.