Перейти к содержимому

Data Version Control (DVC) for ML | DeployBytes 7 - Line by Line

Dr. Sandeep Grover

0:00 / 0:00

Data Version Control (DVC) for ML | DeployBytes 7 - Line by Line

22 просмотра · 13 дней назад
Dr. Sandeep Grover
81 подписчик
22 просмотра · 13 дней назад
Real MLOps, step by step. Part 7 versions 4.3 GB of model weights with DVC: a DVC remote pointed at MinIO, 103-byte pointer files tracked in git, and the md5 blobs living in the bucket instead of the repo. Why this series exists: This video is part of a series created to make central concepts easier to understand and hold onto. It was built with accessibility in mind, particularly for learners with learning differences, so the pace and style are kept simple and supportive. There may be a few mispronunciations in the spoken narration, and occasionally a small mistake in the content. These do not take away from the purpose of the series, which is simply to help the learning stick. Work with me:   / sandeep-grover-b3192a16   Productionizing a real 7-encoder multimodal Rakuten classifier (84,916 products, weighted-F1 0.9147), from git to Kubernetes. Chapters: 0:00 Step 7 - DVC for the model weights 1:58 The shape of this step 3:43 Why DVC: git for code, DVC and MinIO for big binaries 5:08 The DVC configuration idea 7:02 The six substeps, in execution order 8:15 Live run 9:12 7a - dvc init 13:25 7b - dvc remote add the MinIO bucket 15:24 7c - dvc add the model weights 18:03 7d - dvc push (weights go to MinIO) 19:07 7e - hardlink cache and dvc checkout --relink 21:46 7f - commit the pointers 24:22 Verification: not in git, in MinIO 27:59 DVC internals, drilled in 30:51 The real blobs inside the MinIO dvc bucket 32:32 Walkthrough: .dvc/config 35:46 Walkthrough: .dvc/.gitignore and .dvcignore 38:05 Walkthrough: the .pt.dvc pointer files 46:49 Reality check: two errors, and the local MinIO question 48:45 Lessons worth keeping 50:47 Takeaway, and on to Step 8 Series stack (all 12 episodes): Python, cookiecutter-data-science, Git/GitHub, Docker, Docker Compose, PostgreSQL, MinIO (S3), MLflow, DVC, FastAPI, Prometheus, pytest, Nginx, Apache Airflow, Kubernetes, Argo Rollouts, PyTorch, Hugging Face Transformers. #MLOps #DVC #DataVersionControl #MinIO #Git #MachineLearning #Python #DataScience #DevOps #MLEngineering