Set the cleaning rules first

Module 2: Spreadsheets, Python and BI with AI

Course outline

Quick check

Guess first, no penalty. You ask an AI to "remove duplicates" from a customer list, and 340 of 5,000 rows disappear. What should have come first?

Cleaning means fixing messy data: extra spaces, mixed date formats, blanks and repeats. AI is quick at it. But every cleaning step is a decision about what to keep, and those decisions change your totals.

So write the rules before the AI touches anything:

  • Duplicates. Which columns must match for two rows to be the same?
  • Blanks. Leave them empty, fill them, or drop the row? In SQL, AVG and COUNT(column) skip blanks (NULLs) without telling you, while COUNT(*) counts every row.
  • Formats. Which date format is the source in? 03/04 is 3 April in one country and 4 March in another.

Then keep the raw file untouched, and count rows before and after each step.

The rest of this lesson is part of Learn Pro

The first module of every course is free. Learn Pro opens every lesson in every course for $10 a month. All JobRise tools stay free.