Managing end-to-end Machine Learning Lifecycle with MLflow

18.03. | 13:45-15:15 | work.shop(1)

The idea of DevOps has revolutionized the whole software development lifecycle by introducing continuous integration and continuous delivery, resulting in faster iteration cycles. ML development, on the other hand, brings many new challenges beyond the traditional software development lifecycle: for example, ML developers try a lot of algorithms, tools, and parameters to get the best results, and they need to track all this information to reproduce them. This makes it even harder to put their work into production.

MLOps approach seeks to bring the same benefits of reduced iteration cycles and increased productivity to machine learning by applying ideas that have worked well for software development and adding new ones specific to ML.

MLflow project simplifies the whole ML lifecycle by introducing simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.

In this talk, we will show how MLflow can be used to build an automated CI/CD pipeline that can deploy a new version of the model and code around it to production. In addition, we will show how the same approach can be used in the data training pipeline that will retrain model on arrival of new data and deploy the new version of the model if it satisfies all requirements.

This is an interactive workshop, please make sure to bring your laptop. We will use the Databricks Community Edition for the lab exercises. To prepare for the lab please create an account for yourself (no credit card required). Learn more about MLflow and Managed MLflow

Speaker:

Michael Shtelma (Databricks GmbH)