Python Code Example: Removing Duplicates from a List

Removing duplicates from a list is a common task in data processing and analysis. In Python, there are several ways to achieve this, such as using sets, list comprehensions, or libraries like collections. Using a set is one of the simplest and most efficient methods, as sets inherently do not allow duplicate values. Here, we’ll demonstrate how to remove duplicates from a list using a set and also maintain the original order of elements.

Code Example

# Define a list with duplicate elements
numbers = [1, 2, 2, 3, 4, 4, 5, 6, 6, 7, 8, 8, 9, 10, 10]

# Use a set to remove duplicates while maintaining order
def remove_duplicates(lst):
    seen = set()
    unique_list = []
    for item in lst:
        if item not in seen:
            unique_list.append(item)
            seen.add(item)
    return unique_list

# Remove duplicates from the list
unique_numbers = remove_duplicates(numbers)

# Print the resulting list of unique numbers
print(f"Unique numbers: {unique_numbers}")

Code Explanation

Defining a List with Duplicates

A list numbers is defined, containing several duplicate elements.

numbers = [1, 2, 2, 3, 4, 4, 5, 6, 6, 7, 8, 8, 9, 10, 10]

Removing Duplicates Function

A function remove_duplicates is defined that takes a list as its argument and returns a new list with duplicates removed while maintaining the original order.

def remove_duplicates(lst):
    seen = set()
    unique_list = []
    for item in lst:
        if item not in seen:
            unique_list.append(item)
            seen.add(item)
    return unique_list

• Tracking Seen Elements: A set seen is used to keep track of elements that have already been encountered.
• Maintaining Order: An empty list unique_list is used to store the unique elements in their original order.
• Iterating Through the List: The function iterates through each item in the input list lst.
  • Adding Unique Elements: If the item is not in the seen set, it is appended to unique_list, and the item is added to the seen set.

Removing Duplicates

The remove_duplicates function is called with the numbers list to remove duplicates.

unique_numbers = remove_duplicates(numbers)

Printing the Result

The resulting list of unique numbers is printed.

print(f"Unique numbers: {unique_numbers}")