Set the cleaning rules first

Module 2 : Spreadsheets, Python and BI with AI

Plan du cours

Cette leçon n'est pas encore dans votre langue, la voici donc en anglais.

Petit test

Guess first, no penalty. You ask an AI to "remove duplicates" from a customer list, and 340 of 5,000 rows disappear. What should have come first?

A bigger file, so the loss looks smallerA written rule for what counts as a duplicateNothing, since duplicates are always safe to remove

Cleaning means fixing messy data: extra spaces, mixed date formats, blanks and repeats. AI is quick at it. But every cleaning step is a decision about what to keep, and those decisions change your totals.

So write the rules before the AI touches anything:

Duplicates. Which columns must match for two rows to be the same?
Blanks. Leave them empty, fill them, or drop the row? In SQL, AVG and COUNT(column) skip blanks (NULLs) without telling you, while COUNT(*) counts every row.
Formats. Which date format is the source in? 03/04 is 3 April in one country and 4 March in another.

Then keep the raw file untouched, and count rows before and after each step.

La suite de cette leçon fait partie de Learn Pro

Le premier module de chaque cours est gratuit. Learn Pro ouvre toutes les leçons de tous les cours pour 10 $ par mois. Tous les outils JobRise restent gratuits.