CSV, JSON and Parquet

Data arrives in a few formats, and each has quirks. CSV is plain text that every tool can open, but it stores no types. So dates, IDs with leading zeros and missing values all need care when you read it: pass dtype and parse_dates to read_csv.

JSON holds nested records, like a customer with a list of orders inside, and pd.json_normalize flattens them into columns. Parquet is a compressed binary format stored column by column. It keeps each column's type and lets you read only the columns you need.

A file that is 1 GB as CSV is often 100 to 200 MB as Parquet, and it loads far faster. Convert a large CSV to Parquet once and work from that.