Adding new rows to Pandas DataFrames: from list, from dictionary, from Series

In this tutorial we’ll cover everything you might need in order to add new rows into an existing DataFrame. We’ll look specifically into a step-by-step process to append lists, dictionaries and Pandas Series objects into DataFrames.

1. Preparing the Data

We’ll first start by defining a few data objects that we’ll use throughout our examples.

import pandas as pd


# Create dummy data


survey_dict = {            'language': ['C++', 'VB.NET', 'Go', 'PHP', 'Perl'],
                          'avg_salary': [80,75,70,90,80],
                          'num_candidates': [122,154,138,120,145]
                                       }
# Build the Pandas DataFrame
survey = pd.DataFrame(survey_dict)

survey

Here’s our initial DataFrame:

languageavg_salarynum_candidates
0C++80122
1VB.NET75154
2Go70138
3PHP90120
4Perl80145

2. Insert row from Dictionary

Let’s define a new dictionary and insert it to the DataFrame using the append() DataFrame method. Note that we are able to pass the dictionary itself as a parameter of append().

new_dict = {'language': 'R', 'avg_salary': 80, 'num_candidates': 90 }
survey = survey.append(new_dict, ignore_index=True)
survey

Let’s take a look at the result:

languageavg_salarynum_candidates
0C++80122
1VB.NET75154
2Go70138
3PHP90120
4Perl80145
5R8090

3. Append multiple rows without index

In this case we’ll use the pd.concat() method to join our survey DataFRame and a new DataFrame comprising multiple row.

# without index
new_dict = {'language': ['VB', 'HTML'], 'avg_salary': [70, 50], 'num_candidates': [100, 110] }
survey2 = pd.DataFrame(new_dict)
survey3 = pd.concat([survey, survey2],ignore_index=True)


survey3

Note:if you are passing only a scalar in your dictionary, you might get the following error: If using all scalar values, you must pass an index. In that case, make sure you pass a list as your dictionary value or use the parameter ignore_index=True

Here’s our output:

languageavg_salarynum_candidates
0C++80122
1VB.NET75154
2Go70138
3PHP90120
4Perl80145
5R8090
6VB70100
7HTML5011

4. Insert list at a specific index

In this example we’ll use the loc indexer to insert a list into a specific index. In this case we would like to append the new row as the last in our DF.

# insert row using index

new_row = ['Python', 120, 170]
survey = pd.DataFrame(survey_dict)
survey.loc[len(survey.index)] = new_row
survey

Here’s our output:

languageavg_salarynum_candidates
0C++80122
1VB.NET75154
2Go70138
3PHP90120
4Perl80145
5Python12017

5. Insert row at top of Pandas DataFrame

In this example we’ll simply append two DataFrames, the trick would be to append the existing DF to the new one.

new_dict = {'language': 'R', 'avg_salary': [80], 'num_candidates': [90] }
survey = pd.DataFrame(survey_dict)
survey2 = pd.DataFrame(new_dict)
survey3 = survey2.append(survey, ignore_index=True)
survey3

Here’s the output:

languageavg_salarynum_candidates
0R8090
1C++80122
2VB.NET75154
3Go70138
4PHP90120
5Perl8014

6. Add Pandas series as a column

In this example we’ll first create a Pandas Series from a dictionary and then join it to our DataFrame.

#add series to dataframe as row
new_dict = {'language': 'R', 'avg_salary': 80, 'num_candidates': 90 }
new_s = pd.Series(new_dict)
survey.append(new_s, ignore_index=True)

7. New row to empty DataFrame

In our last example for today, we’ll first create an empty Dataframe and then append the new data:

new_dict = {'language': 'R', 'avg_salary': 80, 'num_candidates': 90 }
survey4 = pd.DataFrame(columns = survey.columns)
survey4.append(new_dict,ignore_index=True)

And the output will be:

languageavg_salarynum_candidates
0R8090