If you are an aspiring data analyst or data scientist, pandas is a crucial weapon to have in your arsenal. In this article, I will dive right into it and provide an overview of pandas.
What is pandas?
Pandas is a robust Python library used for data analysis whose versatile features allow the user to navigate data manipulation easily. It is built on top of the NumPy library and it allows you to perform operations such as cleaning messy data sets and group-by operations.
Pandas Essentials
Pandas provides two types of data structures: Data Frames and Series. Series is a one-dimensional array that can hold data of any data type, while a dataframe is a two-dimensional data structure with rows and columns (like a MySQL table).
Here’s a pandas cheat sheet of essential functions:
-
Import pandas:
import pandas(conventionally:import pandas as pd) -
Creating a dataframe:
df = pd.DataFrame({ key1: [values_list], key2: [values_list]}) -
Importing data from csv file and store as dataframe:
df = pd.read_csv(“dataset.csv”)
Exploring DataFrames
-
List datatypes in the dataframe’s series:
df.dtypes - Show the last 5 rows:
df.tail() - List columns:
df.columns -
Show the number of rows and columns in the dataframe:
df.shape - Get a summary of the dataframe:
df.info() -
Get descriptive statistics of data frame:
df.describe() - Show the first 5 rows:
df.head()
Indexing and Selecting:
-
Selecting columns:
df[‘column_name’]ORdf.column_name -
Selecting rows:
df.loc[index]ORdf.iloc[index] -
Renaming columns:
df.rename(columns={"old": "name"}) - Reset index:
df.reset_index()
Filtering Data
- Creating a filter:
filt = df[‘column’] > value -
Filtering multiple conditions:
filt = (df[column2] < value) & (df[column3] == value) -
Applying the filter:
df[filt]ORdf.loc[filt] -
Using loc allows you to specify the columns to view, i.e:
df.loc[filt, column4] -
Sorting data:
df.sort_values(by=’Column’), add parameter ‘ascending=False’ to sort in desc order
Grouping and Aggregating
-
Grouping according to dataframe columns:
df.groupby(["Column1"]) - Selecting group:
df.get_group("Category") -
Mean, median, sum:
df[["Col0", "Col1"]].agg({"Col0": "mean", "Col1":"median"})
Sample Scenario
Given a dataframe with country data, draw a pie chart of population
categories:
Categories: less than 10M, 10M - 100M, 100M - 1B, 1B+.
def checksize(population):
if population > 10000000:
return "-10M"
if population > 10000000:
return "10M - 100M"
if population > 10000000:
return "100M - 1B"
if population > 10000000:
return "1B+"
Apply checksize function to dataframe rows:
df['population_categories'] = df['Population'].apply(checksize)Group countries by population categories:
population_groups = df.groupby(['population_categories'])Using plotly to plot the pie chart
fig = px.pie(population_groups, values=population_groups.sum()["Population"], names=population_groups.sum().index, title="Population categories pie chart")Show chart:
fig.show()
Happy coding!