Python Package Structure for ML | DeployBytes 2 - Line by Line
Dr. Sandeep Grover
0:00 / 0:00
Python Package Structure for ML | DeployBytes 2 - Line by Line
100 просмотров · 1 мес. назад
Dr. Sandeep Grover
81 подписчик
100 просмотров · 1 мес. назад
Why this series exists: This video is part of a series created to make central concepts easier to understand and hold onto. It was built with accessibility in mind, particularly for learners with learning differences, so the pace and style are kept simple and supportive. There may be a few mispronunciations in the spoken narration, the occasional formatting error, and sometimes a small mistake in the content. None of these take away from the purpose of the series, which is simply to help the learning stick.
Real MLOps, step by step. Part 2 turns a flat folder of ML code into a proper Python package (cookiecutter src/ layout) so it is importable, testable, and reproducible - the layout Docker, DVC, CI, and Airflow all assume.
Recommended Learning Order:
The Big Picture (visual overview): • Python Package Structure for ML | DeployBy...
Line by Line (full code walkthrough) - you are here
Why this series exists:
This video is part of a series created to make central concepts easier to understand and hold onto. It was built with accessibility in mind, particularly for learners with learning differences, so the pace and style are kept simple and supportive. There may be a few mispronunciations in the spoken narration, and occasionally a small mistake in the content. These do not take away from the purpose of the series, which is simply to help the learning stick.
Work with me: / sandeep-grover-b3192a16
Productionizing a real 7-encoder multimodal Rakuten product classifier (84,916 products, weighted-F1 0.9147), from git to Kubernetes.
Chapters:
00:00 Step 2 - From a script to a package
00:33 The shape of this step
01:33 Why a package, not a folder
02:07 The problem with the flat folder
02:40 live run
03:19 1 - Create the package skeleton
03:41 2 - Move the code and fix the imports
04:20 3 - Weights to models/, container config to docker/
04:46 ls -la models/
05:24 4 - Commit
05:42 Next: the four modules
05:54 src/features/featurizer.py
06:18 src/features/featurizer.py
06:41 src/features/featurizer.py
07:05 src/features/featurizer.py from line 1
08:24 src/features/featurizer.py from line 11
09:08 src/features/featurizer.py from line 18
10:17 src/features/featurizer.py from line 24
11:20 src/features/featurizer.py from line 29
11:56 src/features/featurizer.py from line 34
13:13 src/features/featurizer.py from line 48
14:21 src/features/featurizer.py from line 61
15:11 src/features/featurizer.py from line 70
15:39 src/features/featurizer.py from line 74
16:15 src/features/featurizer.py from line 80
17:04 src/features/featurizer.py from line 89
17:59 src/features/featurizer.py from line 99
19:00 src/features/featurizer.py from line 110
19:38 src/features/featurizer.py from line 117
20:48 src/features/featurizer.py from line 131
21:31 src/features/featurizer.py from line 140
22:24 predict_model.py
23:02 src/models/predict_model.py from line 1
24:37 src/models/predict_model.py from line 12
25:18 src/models/predict_model.py from line 16
26:02 src/models/predict_model.py from line 20
28:28 src/models/predict_model.py from line 36
30:06 src/models/predict_model.py from line 45
30:56 train_model.py
31:18 src/models/train_model.py
31:39 src/models/train_model.py
32:00 src/models/train_model.py from line 1
33:29 src/models/train_model.py from line 17
34:48 src/models/train_model.py from line 28
35:42 src/models/train_model.py from line 33
36:55 src/models/train_model.py from line 38
37:30 src/models/train_model.py from line 43
37:54 src/models/train_model.py from line 47
38:37 src/models/train_model.py from line 55
39:49 src/models/train_model.py from line 62
40:42 src/models/train_model.py from line 66
41:26 src/models/train_model.py from line 73
42:19 src/models/train_model.py from line 76
43:25 src/models/train_model.py from line 83
44:27 src/models/train_model.py from line 88
45:18 src/models/train_model.py from line 93
46:22 src/models/train_model.py from line 102
46:42 src/models/train_model.py from line 106
47:40 src/models/train_model.py from line 112
48:12 src/models/train_model.py from line 114
49:18 src/models/train_model.py from line 122
49:52 src/models/train_model.py from line 125
50:38 src/models/train_model.py from line 132
51:21 api.py
51:48 src/serving/api.py from line 1
52:09 src/serving/api.py from line 2
52:54 src/serving/api.py from line 6
53:19 src/serving/api.py from line 8
53:40 src/serving/api.py from line 11
54:25 src/serving/api.py from line 17
55:24 src/serving/api.py from line 24
56:29 src/serving/api.py from line 32
58:03 Takeaway + what comes next
Series stack (across all 12 episodes): Python, cookiecutter-data-science, Git/GitHub, Docker, Docker Compose, PostgreSQL, MinIO (S3), MLflow, DVC, FastAPI, Prometheus, pytest, Nginx, Apache Airflow, Kubernetes, Argo Rollouts, PyTorch, Hugging Face Transformers.
#MLOps #MachineLearning #MLEngineering #Python #Docker #Kubernetes #MLflow #DVC #FastAPI #Airflow #PyTorch #ModelDeployment #DataScience #DevOps