Metadata - cfxdm-dm:metadata

Metadata related cfxdm functionality using cfxdm-dm:metadata

dm:metadata: This cfxdm bot allows the user to use this function to check/display metadata user-selected selected/loaded dataset.

This bot is useful when a user wants to analyze metadata of the selected dataset (and the type of each column within the dataset.

The current example will use "surveys.csv" provided as a dataset to explain the functionality.

Step 1: Download the "surveys.csv" file to the local machine as shown below using a standard web browser.

Step 2: Upload (load) the above file into your RDA system as shown below.

• Using a web browser, connect to the RDA system <https://<rda-ipaddress>:9998>

• Click on the CFXDX Python 3 icon from the main landing page of the RDA launcher (as shown below)

• Launch AIOps studio using the RDA provided text field to start AIOps studio.

Type "studio()" into the text field of RDA to load AIOps studio UI

Step 3: Upload the above downloaded CSV file into AIOps studio as shown below.

Select the upload option from the studio environment (as shown below).

Select the file and upload it into the studio environment (as shown below).

Select 'surveys.csv' file for upload

The studio will show the uploaded file in the left panel (as shown below).

Uploaded file is visible on the left panel of studio environment

Step 4: Create an empty dataset 'cfxdm-metadata-example" pipeline and save that in AIOps studio.

Create an empty pipeline from studio environment

Step 5: Add the following pipeline code/commands into the empty pipeline text field that you have created in the above Step 4.

You can copy the below code into your pipeline and execute that in your environment. ##### This pipeline loads surveys.csv file into AIOps Studio. ##### AIOps studio stores the data loaded from surveys.csv file ##### into local dataset named 'cfxdm-metadata'. ##### prints the metadata from the dataset @files:loadfile filename = "surveys.csv" --> @dm:save name = 'cfxdm-metadata' --> *dm:filter --> @dm:metadata

In the above pipeline code, we added the '@dm:metadata' bot/function to the pipeline to fetch/extract metadata details of the 'cfxdm-metadata' that was created during the runtime from the surveys.csv file.

Step 6: Click the button 'Verify' and Click the 'Save' button (AIOps studio verifies the pipeline syntax and stores the pipeline into the earlier created 'cfxdm-metadata-example' pipeline as shown below.

Add pipeline code in the empty pipeline that was created and save the code.

Step 7: Users can now execute the above-created pipeline by clicking the 'Execute' button. Once the users click the 'Execute' button, AIOPs studio executes the pipeline that will load the user-defined file, save the loaded file into a local dataset named 'cfxdm-metadata' as shown below.

Sucessful execution of pipeline

Step 8: Verify the data using AIOps Studio using 'Inspect --> Output --> Get Data'

Output of the pipeline is providing metadata of the dataset

Step 9: @dm:metadata has additional filtering capabilities using which users can query specific metadata details/data as shown below.

Include filter - Column name regex pattern to include in the output (include patterns are matched first and then exclude)

Following examples provide usage of include attribute using a simple regular '| (OR)' operation.

Example a:

You can copy the below code into your pipeline and execute that in your environment. ##### This pipeline loads surveys.csv file into AIOps Studio. ##### AIOps studio stores the data loaded from surveys.csv file ##### into local dataset named 'cfxdm-metadata'. ##### prints the metadata from the dataset using a simple regular '|' @files:loadfile filename = "surveys.csv" --> @dm:save name = 'cfxdm-metadata' --> *dm:filter --> @dm:metadata include = 'plot_id|record_id'

Prints the metadata from the dataset using column name regex pattern 'plot_id|record_id'

Once you execute the above pipeline using the 'include' filter, the output will apply the filter and show the data based on the filter (as shown below).

Verify the data using AIOps Studio using 'Inspect --> Output /Data --> Get Data'

Prints the metadata from the dataset using column name regex pattern to include in the output.

Example b:

##### This pipeline loads surveys.csv file into AIOps Studio. ##### AIOps studio stores the data loaded from surveys.csv file ##### into local dataset named 'cfxdm-metadata'. ##### prints the metadata from the dataset using '.*id' regular expression @files:loadfile filename = "surveys.csv" --> @dm:save name = 'cfxdm-metadata' --> *dm:filter --> @dm:metadata include = '.*id'

Prints the metadata from the dataset using column name regex pattern to include '.*id'

Exclude filter - Column name regex pattern to exclude in the output.

##### This pipeline loads surveys.csv file into AIOps Studio. ##### AIOps studio stores the data loaded from surveys.csv file ##### into local dataset named 'cfxdm-metadata'. ##### prints the metadata from the dataset using '.*id' regular expression @files:loadfile filename = "surveys.csv" --> @dm:save name = 'cfxdm-metadata' --> *dm:filter --> @dm:metadata exclude = '.*id'