Latest Code Tutorials

How To Remove Duplicates From List in Python Example


You can remove duplicates from Python list using the following methods.

  1. Python Dictionary
  2. Python list comprehension or
  3. Write the user-defined function that can remove the duplicate entry and gives the unique items list.

Let’s see the solution one by one.

Python remove duplicates from list

Content Overview

In this tutorial, we will see How to remove duplicates from list in Python.

Pretty much in every project, you need to write a code to check if the list has any duplicates and if it has copies, then we need to remove them and return the new list with the unique items.

Okay, let us see the simple solution.

See the following code.


GoT = ['Tony', 'Rocket', 'Scott', 'Steve', 'Rocket', 'Natasha', 'Tony']
mylist = list(dict.fromkeys(GoT))

First, we have created a dictionary, using the List items like keys.

Python dictionary fromKeys() method automatically remove any duplicates because the dictionary data structure cannot have duplicate keys.

The final step will be to convert the dictionary to list.

Now we have the List without any copies, and it has the same order as the original List.

Print the List to demonstrate the result.

See the output.

Remove Duplicate Items From a Python List

So, it removes the duplicate elements from Python List and gets the list of unique items.

Python remove duplicates using dict.fromKeys()

If you like to have the function where you can pass your lists as an argument, and get back the list without duplicates, you can create the function.

See the below code.


def unique(data):
  return list(dict.fromkeys(data))

GoTB = ['Tony', 'Rocket', 'Scott', 'Steve', 'Rocket', 'Natasha', 'Tony']
finalList = unique(GoTB)

Here, we have written the logic inside the unique function, and that function returns the unique values of the list. It will give us the same output.

We have used the Dictionary.fromKeys() method.

Remove duplicate items using not in on List

We can use not in on the list to find out the duplicate items.

We create a result list and insert only those that are not in the unique list.

See the following code.


def duplicate(items): 
    unique = [] 
    for item in items: 
        if item not in unique: 
    return unique 

GoT = ['Tony', 'Rocket', 'Scott', 'Steve', 'Rocket', 'Natasha', 'Tony']

It gives the same output with the unique values of the list.

Remove duplicate elements from a List using a set

Python Set is an unordered data structure that contains unique elements.

See the following code.


GoTC = ['Tony', 'Rocket', 'Scott', 'Steve', 'Rocket', 'Natasha', 'Tony']
finalData = list(set(GoTC))

The set will remove the duplicate items from the list, but the output will be unordered. See the output.

Remove duplicates from a List using a set

Remove duplicates using list comprehension

This is just a one-liner shorthand of longer method done with the help of list comprehension.

See the following code.

listA = [11, 21, 19, 11, 19, 11, 29, 46] 
print("The original list is : " +  str(listA))
unique = [] 
[unique.append(x) for x in listA if x not in unique] 
print ("The list after removing duplicates : " + str(unique))

See the output.

➜  pyt python3
The original list is : [11, 21, 19, 11, 19, 11, 29, 46]
The list after removing duplicates : [11, 21, 19, 29, 46]
➜  pyt

Finally, How To Remove Duplicates From List in Python Example is over.

Recommended Posts

Python List Copy Example

Python List Reverse Example

Python List Sort Example

Python List Pop Example

Python List Index Example

Leave A Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.