Clean Messy Data in Python — Fix Missing Values Fast

Clean a genuinely messy dataset with pandas: missing values, a ghost row, and double-pasted duplicates — fixed step by step until pandas confirms zero gaps and zero duplicates. Episode 9 of the AI/ML with Python beginner series. Real data is never clean. Learn the professional workflow: diagnose first, dedupe, fill honestly, drop last — and save a clean file you can trust. In this beginner Python tutorial for machine learning, you'll learn df.isna().sum() to find missing values, df.duplicated().sum() to find copied rows, drop_duplicates, fillna with "unknown" and the median (and why the median beats the mean), the dropna trap that deletes good data, and dropna(subset=...) — the scalpel fix. Why learn this? Data people spend around 80% of their time cleaning data, and Python appears in 81.4% of 10,704 machine learning job postings (Huntr 2026 analysis). A model trained on messy data learns the mess: garbage in, garbage out. FULL CODE — data_cleaner.py (copy, run, put it on GitHub; save videos_messy.csv in the SAME folder): """Data Cleaner — Episode 9: Cleaning data with pandas. Portfolio project #9: save this file and videos_messy.csv in the SAME folder, run it, put both on GitHub. Real data is messy: rows go missing, rows get pasted twice. Cleaning is the real 80% of a data job — find the mess, then fix it on purpose: fill what you can reasonably fill, drop what you truly cannot use. """ import pandas as pd Load the MESSY dataset: 12 videos with gaps and double-pasted rows. videos_messy.csv must be in the same folder as this script. df = pd.read_csv("videos_messy.csv") print("Before cleaning:", df.shape) STEP 1 — FIND the mess. Never clean blind. print(df.isna().sum()) # missing values, column by column print("Duplicate rows:", df.duplicated().sum()) STEP 2 — Remove exact duplicate rows (keep the first copy). df = df.drop_duplicates() STEP 3 — Fill gaps you can reasonably fill. A missing category becomes "unknown" — honest, not invented. df["category"] = df["category"].fillna("unknown") Missing numbers get the MEDIAN (the typical value), not the mean — one viral video would drag the mean up and lie to you. df["views"] = df["views"].fillna(df["views"].median()) df["likes"] = df["likes"].fillna(df["likes"].median()) STEP 4 — Drop what you cannot use: a row with no title tells us nothing. df = df.dropna(subset=["title"]) STEP 5 — Prove it is clean before you trust it. print("After cleaning:", df.shape) print(df.isna().sum()) print("Duplicate rows now:", df.duplicated().sum()) Save the clean dataset — this is the file you hand to your team. df.to_csv("videos_clean.csv", index=False) print("Saved videos_clean.csv") print(df.head(10)) Get the messy videos_messy.csv dataset from the pinned comment code reply and try it yourself. CHAPTERS: 0:00 Hook: clean a genuinely messy dataset 0:52 Why employers care: cleaning is 80% of the job 1:50 Meet the mess: gaps, ghosts, double-pasted rows 2:54 Find the mess: isna and duplicated 4:14 Fix 1: drop_duplicates + the df = trap 5:24 Fix 2: fill honestly — unknown + the median 7:00 The dropna trap + the subset scalpel 8:27 Pause & build: portfolio project #9 9:59 Recap + tomorrow: SQL from Python Watch the full series:    • AI/ML with Python — Beginner Series   Subscribe for daily Python-for-ML tutorials: new episode every day. Question for you: fill or drop — what's the messiest dataset you've ever opened, and what was hiding in it? Tell me in the comments. I read every comment. Job stat cited: Huntr analysis of 10,704 ML/AI job postings (2026) — https://huntr.co/resume-skills/machin... #python #machinelearning #pandas #datacleaning #pythonforbeginners