# cfxdm - dm:dedup

**dm:dedup:** This cfxdm tag allows the user to remove the duplicate values from the queried data for a selected column or columns.

It can be used to find the unique values from a selected column or unique values from a more than one selected columns by looking at them as a combined value.

**dm:dedup** syntax**:** 

* **columns (**optional**)**. Specify a column or columns (comma separated) on which de-duplication of the data to be applied.

In the below example, for a reference, we are going to use **VMware vROps** as an extension to query the data and ingest it into **dm:dedup** to remove any duplicate values from a selected column or columns.

Enter the below command to select **VMware vROps Datastore Summary tag (\*vrops:datastore\_summary**). (In this example, vrops name is used as a label to identify VMware vROps extension and it's tags. The label is defined while adding the extension in cfxdx configuration file or through UI)

```
tag *vrops:datastore_summary
```

![](https://2978683539-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LhoMVYxiQlKXh6OxX98%2F-MVdeYcgylnhey0GcBbm%2F-MVdqUxBzyGEMfB4Lc_y%2FScreen%20Shot%202021-03-12%20at%208.30.29%20PM.png?alt=media\&token=265449c6-4b98-4e4a-90c0-a4d6c09a2132)

**Example 1:** Remove duplicate datastore names from a selected column

Get the data from VMware vROps Datastore Summary tag (**\*vrops:datastore\_summary**) and remove duplicate datastore names from a selected column '**name**'

```
data --> dm:dedup columns = 'name'
```

![](https://2978683539-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LhoMVYxiQlKXh6OxX98%2F-MVhtxvGH2gdoTdVRiPa%2F-MVhw_SLT9mboBdnAClp%2FScreen%20Shot%202021-03-13%20at%203.35.30%20PM.png?alt=media\&token=e1b0d20c-3984-49c1-be21-94c987ee56fa)

**Example 2:** Find the unique values in the combination of two selected columns (**name** & **parent\_datacenter**)

When more than one column is selected, **dm:dedup** looks at the values of the selected columns as a single data string and find unique values.

Get the data from VMware vROps Datastore Summary tag (**\*vrops:datastore\_summary**) and find the unique values in the combination of two selected columns (**name** & **parent\_datacenter**)

```
data --> dm:dedup columns = 'name,parent_datacenter'
```

![](https://2978683539-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LhoMVYxiQlKXh6OxX98%2F-MVhtxvGH2gdoTdVRiPa%2F-MVhz9tKCBwvnKDxBgXs%2FScreen%20Shot%202021-03-13%20at%203.47.05%20PM.png?alt=media\&token=359ae990-20d2-4c7c-9f64-a5e0420e0a0b)