Data Version Control (DVC) for ML | DeployBytes 7 - Line by Line
Dr. Sandeep Grover
0:00 / 0:00
Data Version Control (DVC) for ML | DeployBytes 7 - Line by Line
22 просмотра · 13 дней назад
Dr. Sandeep Grover
81 подписчик
22 просмотра · 13 дней назад
Real MLOps, step by step. Part 7 versions 4.3 GB of model weights with DVC: a DVC remote pointed at MinIO, 103-byte pointer files tracked in git, and the md5 blobs living in the bucket instead of the repo.
Why this series exists:
This video is part of a series created to make central concepts easier to understand and hold onto. It was built with accessibility in mind, particularly for learners with learning differences, so the pace and style are kept simple and supportive. There may be a few mispronunciations in the spoken narration, and occasionally a small mistake in the content. These do not take away from the purpose of the series, which is simply to help the learning stick.
Work with me: / sandeep-grover-b3192a16
Productionizing a real 7-encoder multimodal Rakuten classifier (84,916 products, weighted-F1 0.9147), from git to Kubernetes.
Chapters:
0:00 Step 7 - DVC for the model weights
1:58 The shape of this step
3:43 Why DVC: git for code, DVC and MinIO for big binaries
5:08 The DVC configuration idea
7:02 The six substeps, in execution order
8:15 Live run
9:12 7a - dvc init
13:25 7b - dvc remote add the MinIO bucket
15:24 7c - dvc add the model weights
18:03 7d - dvc push (weights go to MinIO)
19:07 7e - hardlink cache and dvc checkout --relink
21:46 7f - commit the pointers
24:22 Verification: not in git, in MinIO
27:59 DVC internals, drilled in
30:51 The real blobs inside the MinIO dvc bucket
32:32 Walkthrough: .dvc/config
35:46 Walkthrough: .dvc/.gitignore and .dvcignore
38:05 Walkthrough: the .pt.dvc pointer files
46:49 Reality check: two errors, and the local MinIO question
48:45 Lessons worth keeping
50:47 Takeaway, and on to Step 8
Series stack (all 12 episodes): Python, cookiecutter-data-science, Git/GitHub, Docker, Docker Compose, PostgreSQL, MinIO (S3), MLflow, DVC, FastAPI, Prometheus, pytest, Nginx, Apache Airflow, Kubernetes, Argo Rollouts, PyTorch, Hugging Face Transformers.
#MLOps #DVC #DataVersionControl #MinIO #Git #MachineLearning #Python #DataScience #DevOps #MLEngineering