8 Tips & Tricks for Pandas

1. Use pandas >= 1.0.0

Performance boosts above this version, so upgrade your package asap

2. Specify the types of the columns when loading csv

Save a lot of memory in large files

3. Specify the necessary columns when loading

Do not load columns you don't need

4. Delete Dataframes you not longer need

Free memory when possible

5. Read a file and process per chunks

Useful for large files

6. Explicitly convert certain columns to datetime objects when loading

This helps you to do easy transformations on datetimes columns

7. Use multi-index when possible

Groups values and compress data, saving a lot of memory

8. Avoid Apply method, use built in functions

Save CPU processing time avoiding Apply()