Textual Data cleaning in pandas then preprocess and EDA
Nikunj jha
0:00 / 0:00
Textual Data cleaning in pandas then preprocess and EDA
8 просмотров · 2 мес. назад
Nikunj jha
3 подписчика
8 просмотров · 2 мес. назад
Data Cleaning
Remove unwanted items like HTML tags, emojis, and special symbols
Fix spelling mistakes if needed.
Convert all letters to lowercase so the computer treats "Cat" and "cat" the same.
Preprocessing
Tokenization: Split long sentences into smaller pieces, or words.
Stop word removal: Take out common words like "the," "is," and "and" that add little meaning.
Stemming and Lemmatization: Reduce words to their basic root form (like changing "running" to "run").
Vectorization
Change words into numbers using methods like TF-IDF or word embeddings
.Feed these numbers into a machine learning model to train it.