DeleteDuplicates

DeleteDuplicates[data]

deletes all duplicates from data.

DeleteDuplicates[data,test]

applies test to pairs of elements to determine whether they should be considered duplicates.

Details

  • DeleteDuplicates[data] deletes all but the first occurrence of each distinct element that appears in data.
  • DeleteDuplicates also works on associations, removing entries whose values duplicate those earlier in the association.
  • The data can have the following forms:
  • {e1,e2,…}list of values, including numbers, quantities, dates, ...
    Association[…]association of values
    QuantityArray[…]quantity array or other structured array
    Tabular[…]type-consistent tabular data
    TabularColumn[…]type-consistent column data
    Dataset[…]general hierarchical data
  • DeleteDuplicates never reorders elements, but only removes them.
  • DeleteDuplicates[data] is equivalent to DeleteDuplicates[data,SameQ]. »

Examples

open allclose all

Basic Examples  (2)

Delete duplicated elements:

Delete elements whose values duplicate those earlier in the association:

Scope  (6)

The data need not have head List:

Delete elements that are preceded by at least one larger element:

Treat numbers as equal if they differ by 0.01 or less:

Delete duplicate rows in a Tabular object:

Delete rows whose first column value is larger than a previous value:

Delete duplicate rows in a Dataset object:

Properties & Relations  (4)

DeleteDuplicates[list] is equivalent to DeleteDuplicates[list,SameQ]:

DeleteDuplicates is similar to Union without sorting:

Avoiding the sort improves the speed substantially:

DeleteDuplicates[list,test] applies test to two elements to determine if they are duplicates:

DeleteDuplicatesBy[list,f] transforms each element using f, then compares the results:

CountDistinct[list] is equivalent to Length[DeleteDuplicates[list]]:

CountDistinct[list,test] is equivalent to Length[DeleteDuplicates[list,test]]:

Possible Issues  (1)

If the comparison function returns neither True nor False, elements are assumed to be distinct:

Compare with the function that always reports elements as distinct:

Wolfram Research (2008), DeleteDuplicates, Wolfram Language function, https://reference.wolfram.com/language/ref/DeleteDuplicates.html (updated 2025).

Text

Wolfram Research (2008), DeleteDuplicates, Wolfram Language function, https://reference.wolfram.com/language/ref/DeleteDuplicates.html (updated 2025).

CMS

Wolfram Language. 2008. "DeleteDuplicates." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2025. https://reference.wolfram.com/language/ref/DeleteDuplicates.html.

APA

Wolfram Language. (2008). DeleteDuplicates. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/DeleteDuplicates.html

BibTeX

@misc{reference.wolfram_2024_deleteduplicates, author="Wolfram Research", title="{DeleteDuplicates}", year="2025", howpublished="\url{https://reference.wolfram.com/language/ref/DeleteDuplicates.html}", note=[Accessed: 17-January-2025 ]}

BibLaTeX

@online{reference.wolfram_2024_deleteduplicates, organization={Wolfram Research}, title={DeleteDuplicates}, year={2025}, url={https://reference.wolfram.com/language/ref/DeleteDuplicates.html}, note=[Accessed: 17-January-2025 ]}