Перейти к содержимому

Python Package Structure for ML | DeployBytes 2 - Line by Line

Dr. Sandeep Grover

0:00 / 0:00

Python Package Structure for ML | DeployBytes 2 - Line by Line

100 просмотров · 1 мес. назад
Dr. Sandeep Grover
81 подписчик
100 просмотров · 1 мес. назад
Why this series exists: This video is part of a series created to make central concepts easier to understand and hold onto. It was built with accessibility in mind, particularly for learners with learning differences, so the pace and style are kept simple and supportive. There may be a few mispronunciations in the spoken narration, the occasional formatting error, and sometimes a small mistake in the content. None of these take away from the purpose of the series, which is simply to help the learning stick. Real MLOps, step by step. Part 2 turns a flat folder of ML code into a proper Python package (cookiecutter src/ layout) so it is importable, testable, and reproducible - the layout Docker, DVC, CI, and Airflow all assume. Recommended Learning Order: The Big Picture (visual overview):    • Python Package Structure for ML | DeployBy...   Line by Line (full code walkthrough) - you are here Why this series exists: This video is part of a series created to make central concepts easier to understand and hold onto. It was built with accessibility in mind, particularly for learners with learning differences, so the pace and style are kept simple and supportive. There may be a few mispronunciations in the spoken narration, and occasionally a small mistake in the content. These do not take away from the purpose of the series, which is simply to help the learning stick. Work with me:   / sandeep-grover-b3192a16   Productionizing a real 7-encoder multimodal Rakuten product classifier (84,916 products, weighted-F1 0.9147), from git to Kubernetes. Chapters: 00:00 Step 2 - From a script to a package 00:33 The shape of this step 01:33 Why a package, not a folder 02:07 The problem with the flat folder 02:40 live run 03:19 1 - Create the package skeleton 03:41 2 - Move the code and fix the imports 04:20 3 - Weights to models/, container config to docker/ 04:46 ls -la models/ 05:24 4 - Commit 05:42 Next: the four modules 05:54 src/features/featurizer.py 06:18 src/features/featurizer.py 06:41 src/features/featurizer.py 07:05 src/features/featurizer.py from line 1 08:24 src/features/featurizer.py from line 11 09:08 src/features/featurizer.py from line 18 10:17 src/features/featurizer.py from line 24 11:20 src/features/featurizer.py from line 29 11:56 src/features/featurizer.py from line 34 13:13 src/features/featurizer.py from line 48 14:21 src/features/featurizer.py from line 61 15:11 src/features/featurizer.py from line 70 15:39 src/features/featurizer.py from line 74 16:15 src/features/featurizer.py from line 80 17:04 src/features/featurizer.py from line 89 17:59 src/features/featurizer.py from line 99 19:00 src/features/featurizer.py from line 110 19:38 src/features/featurizer.py from line 117 20:48 src/features/featurizer.py from line 131 21:31 src/features/featurizer.py from line 140 22:24 predict_model.py 23:02 src/models/predict_model.py from line 1 24:37 src/models/predict_model.py from line 12 25:18 src/models/predict_model.py from line 16 26:02 src/models/predict_model.py from line 20 28:28 src/models/predict_model.py from line 36 30:06 src/models/predict_model.py from line 45 30:56 train_model.py 31:18 src/models/train_model.py 31:39 src/models/train_model.py 32:00 src/models/train_model.py from line 1 33:29 src/models/train_model.py from line 17 34:48 src/models/train_model.py from line 28 35:42 src/models/train_model.py from line 33 36:55 src/models/train_model.py from line 38 37:30 src/models/train_model.py from line 43 37:54 src/models/train_model.py from line 47 38:37 src/models/train_model.py from line 55 39:49 src/models/train_model.py from line 62 40:42 src/models/train_model.py from line 66 41:26 src/models/train_model.py from line 73 42:19 src/models/train_model.py from line 76 43:25 src/models/train_model.py from line 83 44:27 src/models/train_model.py from line 88 45:18 src/models/train_model.py from line 93 46:22 src/models/train_model.py from line 102 46:42 src/models/train_model.py from line 106 47:40 src/models/train_model.py from line 112 48:12 src/models/train_model.py from line 114 49:18 src/models/train_model.py from line 122 49:52 src/models/train_model.py from line 125 50:38 src/models/train_model.py from line 132 51:21 api.py 51:48 src/serving/api.py from line 1 52:09 src/serving/api.py from line 2 52:54 src/serving/api.py from line 6 53:19 src/serving/api.py from line 8 53:40 src/serving/api.py from line 11 54:25 src/serving/api.py from line 17 55:24 src/serving/api.py from line 24 56:29 src/serving/api.py from line 32 58:03 Takeaway + what comes next Series stack (across all 12 episodes): Python, cookiecutter-data-science, Git/GitHub, Docker, Docker Compose, PostgreSQL, MinIO (S3), MLflow, DVC, FastAPI, Prometheus, pytest, Nginx, Apache Airflow, Kubernetes, Argo Rollouts, PyTorch, Hugging Face Transformers. #MLOps #MachineLearning #MLEngineering #Python #Docker #Kubernetes #MLflow #DVC #FastAPI #Airflow #PyTorch #ModelDeployment #DataScience #DevOps