AppDividend
Latest Code Tutorials

How to Replace NaN Values With Zeros in Pandas DataFrame

Pandas DataFrame contains all kinds of values, including NaN values, and if you want to get the correct output, then you must need to replace all NaN values with zeros. The interpreter sometimes does not understand the NaN values and our final output effect with these NaN values, that is why we have to convert all NaN values to Zeros.

Steps to replace nan values with zeros in DataFrame

Depending on your particular scenario, you may use one of the below four methods to replace NaN values with zeros in Pandas DataFrame.

  1. Replace NaN values for a specific column using df.fillna()
  2. Replace NaN with zeros a single column using df.replace()
  3. Replace with zeros for an entire DataFrame using df.fillna()
  4. Replace with zeros for an entire DataFrame using df.replace()

Replace NaN values for a specific column

Pandas DataFrame can be constructed from Python Dictionary. So let’s create DataFrame from Dictionary.

# app.py

import pandas as pd

dict = {'price': ['100', 'KDL100', 400, 'ADL100']}

df = pd.DataFrame(dict)

print(df)

Output

    price
0     100
1  KDL100
2     400
3  ADL100

The DataFrame column’s values are String in our example. There is no numeric format. Also, some values are the combination of data types like Numeric + String like ‘KDL100′ and ‘ADL100′.

Now, we will use a Pandas DataFrame.to_integer() function to convert String values to Numeric, and if the values are like a combination of String+Number, then those values will be converted to NaN values.

Convert Pandas DataFrame String to Number

We can convert String to Numeric values using df.to_integer() function.

# app.py

import pandas as pd

dict = {'price': ['100', 'KDL100', 400, 'ADL100']}
df = pd.DataFrame(dict)
df['price'] = pd.to_numeric(df['price'], errors='coerce')
print(df)

Output

 price
0  100.0
1    NaN
2  400.0
3    NaN

And now you will get with the NaN values.

Later, we will see how to replace all the NaN values with zeros in Pandas DataFrame.

Finally, to replace the NaN values with zeros for a column using Pandas, you may use the df.fillna() method.

Pandas df.fillna()

DataFrame.fillna() function fills the NA/NaN values in Pandas DataFrame. In our example, we will replace NaN values with zeros values.

See the following code.

# app.py

import pandas as pd

dict = {'price': ['100', 'KDL100', 400, 'ADL100']}
df = pd.DataFrame(dict)
df['price'] = pd.to_numeric(df['price'], errors='coerce')

df['price'] = df['price'].fillna(0)
print(df)

Output

   price
0  100.0
1    0.0
2  400.0
3    0.0

Now, you will see that the previous two NaN values became 0’s.

Replace NaN values with zeros using df.replace()

Pandas DataFrame replace() method accomplish the same task of replacing the NaN values with zeros by using np.nan property. The DataFrame replace() method replaces with other values dynamically. This differs from updating with .loc or .iloc, which requires you to specify a location to update with some value.

To use numpy nan, we need to import numpy library and then use it inside our program.

See the following code.

# app.py

import pandas as pd
import numpy as np

dict = {'price': ['100', 'KDL100', 400, 'ADL100']}
df = pd.DataFrame(dict)
df['price'] = pd.to_numeric(df['price'], errors='coerce')

df['price'] = df['price'].replace(np.nan, 0)
print(df)

Output

   price
0  100.0
1    0.0
2  400.0
3    0.0

We got the same output, as we have gotten in the df.fillna() method.

Replace NaN values with zeros in entire DataFrame

In the above example, we have created a single column in DataFrame. Now, we will create a DataFrame with multiple columns and will replace NaN values with zeros in multiple columns.

In the above examples, we have converted String values to Number values for a particular column. In this case, we need to convert that for the whole DataFrame. So, we will use the df.apply() function to convert the entire DataFrame values.

Then we will use the df.fillna() function to replace with zeros.

See the below code.

# app.py

import pandas as pd

dict = {'price': ['100', 'KDL100', 400, 'ADL100'],
        'discount': ['50', '50%', '30%', '20']}

df = pd.DataFrame(dict)
df = df.apply(pd.to_numeric, errors='coerce')

df = df.fillna(0)
print(df)

Output

   price  discount
0  100.0      50.0
1    0.0       0.0
2  400.0       0.0
3    0.0      20.0

In the output, we can see that all the values, like 50%, 30%, KDL100, and ADL100 converted to zeros.

Replace with zeros for an entire DataFrame

In this case, we will use import numpy and use np.nan property to target all the NaN values and then use the df.replace() function to replace the NaN values with zeros in entire DataFrame. See the following code.

# app.py

import pandas as pd
import numpy as np

dict = {'price': ['100', 'KDL100', 400, 'ADL100'],
        'discount': ['50', '50%', '30%', '20']}

df = pd.DataFrame(dict)
df = df.apply(pd.to_numeric, errors='coerce')

df = df.replace(np.nan, 0)
print(df)

Output

   price  discount
0  100.0      50.0
1    0.0       0.0
2  400.0       0.0
3    0.0      20.0

That is it. We have seen all the ways to convert NaN to zeros in DataFrame.

See also

Pandas DataFrame dropna()

Iterate columns in Pandas DataFrame

Pandas DataFrame assign()

Pandas DataFrame transpose()

Pandas DataFrame Pivot table

Leave A Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.