GitHub link

This lab gives an overview of some of the fundamental mlops concepts using mlflow:

  1. starter.ipynb: mlflow - autolog(), logging a model, loading a saved model
  2. linear_regression.ipynb: mlflow - logging parameters and metrics
  3. serving.ipynb: mlflow - serving models
  4. serving.py: mlflow - creating custom artifacts(e.g extra_pip_requirements.txt)

Prerequisites

  1. Github Lab-1
  2. Jupyter Lab installed in your system (important to have it installed)

Setting up the lab

  1. Open the local mlops_labs directory using Visual Studio Code(VSC) and open a new terminal inside VSC.
  2. Create a new working directory for this lab(e.g. mlflow_lab1).
  3. Move into the working directory using the command cd mlflow_lab1 in your terminal.
  4. Create a requirements.txt file and add packages similar to this.
  5. Create a virtual environment(e.g. mlflow_lab1_env).
  6. Activate the environment and install the required packages using pip install -r requirements.txt.

Cloning the necessary files

Copy the following files into your working directory.

  1. starter.ipynb
  2. linear_regression.ipynb
  3. serving.ipynb
  4. serving.py

Project Structure:

    mlops_labs/
    └── mlflow_lab1/
        ├── requirements.txt
        ├── mlflow_lab1_env/
        ├── starter.ipynb
        ├── linear_regression.ipynb
        ├── serving.ipynb
        └── serving.py

1. starter.ipynb

  1. To run starter.ipynb in jupyter lab, you have to link your virtual environment to jupyter lab.
  2. Run the following command in your terminal to do this. Pass the name of your virtual environment to –name. 
     python -m ipykernel install --user --name=mlflow_lab1_env
  3. Run jupyter lab 
     jupyter-lab
  4. Select your virtual environment as your notebook’s kernel by going to the Kernel menu -> Change kernel -> select your virtual environment.
  5. Continue with the jupyter notebook.

Note:

  • In case you cannot run the !mlflow ui commands inside your jupyter lab, open a new terminal in VSC, activate your virtual environment and run the same command in your new terminal.

2. linear_regession.ipynb

  1. Make sure you selected your virtual environment as your notebook’s kernel.
  2. Continue with the jupyter notebook.
  3. If you are unable to run the mlflow commands inside your notebook, open a new terminal inside your VSC, activate your virtual environment and run them there.

3. serving.ipynb

  1. Make sure you selected your virtual environment as your notebook’s kernel.
  2. Once your model is ready to serve, get it’s run id from mlruns/0/.
  3. Open a new terminal inside your VSC and activate your virtual environment.
  4. Run the following command replacing run_id. 
     mlflow models serve --env-manager=local -m mlruns/0/<run_id>/artifacts/model -h 127.0.0.1 -p 5001
  5. Make sure the url, port in your notebook and in the above command matches. 
     url = 'http://127.0.0.1:5001/invocations'

4. serving.py

  1. Make sure you selected your virtual environment as your notebook’s kernel.
  2. Run the file 
     python serving.py
  3. Run the mlflow ui 
     mlflow ui
  4. Check the artifacts.