Data Mapping cfxdm - dm:describe
Show column statistics
Last updated
Show column statistics
Last updated
dm:describe: This cfxdm tag allows the user to select one or more columns and provides below summarized statistics on the values within them.
count: Total count of the values within a column
unique: Total unique count of the values within a column
top: The top value out of all unique values which are repeated compared to others
freq: The number of times the top unique is repeated
mean: The mean out of all of the values (applies to rows that only has numeric values)
std: The standard deviation out of all of the values (applies to rows that only has numeric values)
min: The minimum value out of all of the values (applies to rows that only has numeric values)
25%: 25% percentile out of all of the values (applies to rows that only has numeric values)
50%: 50% percentile out of all of the values (applies to rows that only has numeric values)
75%: 75% percentile out of all of the values (applies to rows that only has numeric values)
max: The maximum value out of all of the values (applies to rows that only has numeric values)
dm: describe syntax:
columns (optional). Specify a column or columns (comma separated) on which 'describe' to be applied.
This section explains how users can use a CSV file loaded into a dataset. This saved dataset will be used to explain how the dm: describe function to display the stats of the dataset.
Step 1: Download 'incidents.csv' to the AIOps RDA environment as shown below from the local file system.
Step 2: Upload the file 'incidents.csv' to AIOps studio using file-browser (as shown below)
Step 3: Add a new empty pipeline with the name "dm_describe_example_1" as shown below and click the "Save" button (this step will create an empty pipeline and saves it to AIOps studio).
Step 4: Add the following pipeline commands into the empty pipeline text field that you have created in above Step 3.
You can copy the below code into your pipeline and execute that in your environment.
##### This pipeline loads incidents.csv file into AIOps Studio.
##### AIOps studio stores the data loaded from incidents.csv file
##### into local dataset named 'incident-summary'.
##### prints the data that was stored
@files:loadfile filename = "incidents.csv"
--> @dm:save name = 'incidents-summary'
--> *dm:filter *
Step 5: Check the data from incidents.csv by executing the pipeline and verifying using inspect data as shown below (screenshot -1 & screenshot-2)
Step 6: Now, add the following additional pipeline code to use the dm: describe function to the previously created pipeline from Step-4 as shown below (Edit and add the following pipeline code) and click verify to verify the pipeline code as shown below.
##### This pipeline loads incidents.csv file into AIOps Studio.
##### AIOps studio stores the data loaded from incidents.csv file
##### into local dataset named 'incident-summary'.
##### prints the data that was stored
@files:loadfile filename = "incidents.csv"
--> @dm:save name = 'incidents-summary'
--> *dm:filter *
--> @dm:describe
Step 7: Click execute button and execute the pipeline. RDA will execute the pipeline without any errors (as shown below)
Step 8: RDA uses the dm describe function to describe the statistics of the selected 'incidents.csv' dataset that was loaded earlier (and saved) as shown in the following screenshot.
Now, add the following additional pipeline code to use the dm: describe function to the previously created pipeline from Step-4 as shown below (Edit and add the following pipeline code) and click verify to verify the pipeline code as shown below.
##### This pipeline loads incidents.csv file into AIOps Studio.
##### AIOps studio stores the data loaded from incidents.csv file
##### into local dataset named 'incident-summary'.
##### prints the data that was stored
@files:loadfile filename = "incidents.csv"
--> @dm:save name = 'incidents-summary'
--> *dm:filter *
--> @dm:describe columns = 'Summary,Source'
Once the above pipeline code is added and executed, stats are provided to selected two columns as shown below screenshot.
