In this tutorial we’ll cover everything you might need in order to add new rows into an existing DataFrame. We’ll look specifically into a step-by-step process to append lists, dictionaries and Pandas Series objects into DataFrames.
1. Preparing the Data
We’ll first start by defining a few data objects that we’ll use throughout our examples.
import pandas as pd
# Create dummy data
survey_dict = { 'language': ['C++', 'VB.NET', 'Go', 'PHP', 'Perl'],
'avg_salary': [80,75,70,90,80],
'num_candidates': [122,154,138,120,145]
}
# Build the Pandas DataFrame
survey = pd.DataFrame(survey_dict)
survey
Here’s our initial DataFrame:
language | avg_salary | num_candidates | |
---|---|---|---|
0 | C++ | 80 | 122 |
1 | VB.NET | 75 | 154 |
2 | Go | 70 | 138 |
3 | PHP | 90 | 120 |
4 | Perl | 80 | 145 |
2. Insert row from Dictionary
Let’s define a new dictionary and insert it to the DataFrame using the append() DataFrame method. Note that we are able to pass the dictionary itself as a parameter of append().
new_dict = {'language': 'R', 'avg_salary': 80, 'num_candidates': 90 }
survey = survey.append(new_dict, ignore_index=True)
survey
Let’s take a look at the result:
language | avg_salary | num_candidates | |
---|---|---|---|
0 | C++ | 80 | 122 |
1 | VB.NET | 75 | 154 |
2 | Go | 70 | 138 |
3 | PHP | 90 | 120 |
4 | Perl | 80 | 145 |
5 | R | 80 | 90 |
3. Append multiple rows without index
In this case we’ll use the pd.concat() method to join our survey DataFrame and a new DataFrame comprising multiple row.
# without index
new_dict = {'language': ['VB', 'HTML'], 'avg_salary': [70, 50], 'num_candidates': [100, 110] }
survey2 = pd.DataFrame(new_dict)
survey3 = pd.concat([survey, survey2],ignore_index=True)
survey3
Note :if you are passing only a scalar in your dictionary, you might get the following error: If using all scalar values, you must pass an index. In that case, make sure you pass a list as your dictionary value or use the parameter ignore_index=True.
Here’s our output:
language | avg_salary | num_candidates | |
---|---|---|---|
0 | C++ | 80 | 122 |
1 | VB.NET | 75 | 154 |
2 | Go | 70 | 138 |
3 | PHP | 90 | 120 |
4 | Perl | 80 | 145 |
5 | R | 80 | 90 |
6 | VB | 70 | 100 |
7 | HTML | 50 | 11 |
4. Insert list at a specific index
In this example we’ll use the loc indexer to insert a list into a specific index. In this case we would like to append the new row as the last in our DF.
# insert row using index
new_row = ['Python', 120, 170]
survey = pd.DataFrame(survey_dict)
survey.loc[len(survey.index)] = new_row
survey
Here’s our output:
language | avg_salary | num_candidates | |
---|---|---|---|
0 | C++ | 80 | 122 |
1 | VB.NET | 75 | 154 |
2 | Go | 70 | 138 |
3 | PHP | 90 | 120 |
4 | Perl | 80 | 145 |
5 | Python | 120 | 17 |
5. Insert row at top of Pandas DataFrame
In this example we’ll simply append two DataFrames, the trick would be to append the existing DF to the new one.
new_dict = {'language': 'R', 'avg_salary': [80], 'num_candidates': [90] }
survey = pd.DataFrame(survey_dict)
survey2 = pd.DataFrame(new_dict)
survey3 = survey2.append(survey, ignore_index=True)
survey3
Here’s the output:
language | avg_salary | num_candidates | |
---|---|---|---|
0 | R | 80 | 90 |
1 | C++ | 80 | 122 |
2 | VB.NET | 75 | 154 |
3 | Go | 70 | 138 |
4 | PHP | 90 | 120 |
5 | Perl | 80 | 14 |
6. Add Pandas series as a column
In this example we’ll first create a Pandas Series from a dictionary and then join it to our DataFrame.
#add series to dataframe as row
new_dict = {'language': 'R', 'avg_salary': 80, 'num_candidates': 90 }
new_s = pd.Series(new_dict)
survey.append(new_s, ignore_index=True)
7. New row to empty DataFrame
In our last example for today, we’ll first create an empty Dataframe and then append the new data:
new_dict = {'language': 'R', 'avg_salary': 80, 'num_candidates': 90 }
survey4 = pd.DataFrame(columns = survey.columns)
survey4.append(new_dict,ignore_index=True)
And the output will be:
language | avg_salary | num_candidates | |
---|---|---|---|
0 | R | 80 | 90 |