Today we will learn how to check whether a specific text value exists across a DataFrame columns and rows.
Step # 1: Create Example DataFrame
We will start by creating a simple DataFrame that you can use to follow along with this example.
import pandas as pd
office = ['Toronto', 'Paris', 'Rio de Janeiro', 'Buenos Aires', 'Paris']
salary = [192.0, 217.0, 230.0, 203.0, 117.0]
mydf = pd.DataFrame(dict(office = office, salary = salary))
mydf.head()
Here’s our DataFrame:
office | salary | |
---|---|---|
0 | Toronto | 192.0 |
1 | Paris | 217.0 |
2 | Rio de Janeiro | 230.0 |
3 | Buenos Aires | 203.0 |
4 | Paris | 117.0 |
Step #2: Check if string exists in a DataFrame column
You can use the standard pandas str accessor to check whether a string exists in a column:
search_str = 'Rio'
filt = mydf['office'].str.contains(search_str)
mydf[filt].head()
This returns the following DataFrame subset:
office | salary | |
---|---|---|
2 | Rio de Janeiro | 230.0 |
Step #3: Check if DataFrame row contains value
We now would like to check whether a string exists in any of the DataFrame rows:
search_str = 'Paris'
filt = mydf.apply (lambda row: row.str.contains(search_str).any(), axis = 1)
mydf[filt].head()
This will return the following rows:
office | salary | |
---|---|---|
1 | Paris | 217.0 |
4 | Paris | 117.0 |
Step # 4: Check multiple items are in DataFrame columns
Here we will use the Series isin method, and loop across the columns of our DataFrame (therefore, axis=0), then we’ll filter our DataFrame to show only columns containing values from the list:
search_lst = ['Paris', 'Buenos Aires']
filt = mydf.apply (lambda cell: cell.isin(search_lst), axis = 0)
mydf.loc[:,mydf[filt].any()]
office | salary | |
---|---|---|
1 | Paris | 217.0 |
3 | Buenos Aires | 203.0 |
4 | Paris | 117.0 |
Step # 5: Show cell containing specific value from list
In the next case we would like to find the exact Dataframe cells (row and column intersection) that contain a specific set of values. Here we go:
row = mydf.apply (lambda cell: cell.isin(search_lst).any(), axis = 1)
col = mydf.apply (lambda cell: cell.isin(search_lst).any(), axis = 0)
mydf.loc[row, col]
Here’s the result:
office | |
---|---|
1 | Paris |
3 | Buenos Aires |
4 | Paris |
Step #6: Check if a column contains values greater than a constant
For completeness – we can find rows that contains values greater than a specific number . In this example, we will search for all rows with salaries bigger than the mean salary. Here we go:
filt = mydf['salary'] > mydf['salary'].mean()
mydf[filt]