Metadata - cfxdm-dm:metadata
Metadata related cfxdm functionality using cfxdm-dm:metadata
dm:metadata: This cfxdm bot allows the user to use this function to check/display metadata user-selected selected/loaded dataset.
This bot is useful when a user wants to analyze metadata of the selected dataset (and the type of each column within the dataset.
The current example will use "surveys.csv" provided as a dataset to explain the functionality.
Step 1: Download the "surveys.csv" file to the local machine as shown below using a standard web browser.
Step 2: Upload (load) the above file into your RDA system as shown below.
Using a web browser, connect to the RDA system <https://<rda-ipaddress>:9998>
Click on the CFXDX Python 3 icon from the main landing page of the RDA launcher (as shown below)
Launch AIOps studio using the RDA provided text field to start AIOps studio.
Step 3: Upload the above downloaded CSV file into AIOps studio as shown below.
Select the upload option from the studio environment (as shown below).
Select the file and upload it into the studio environment (as shown below).
The studio will show the uploaded file in the left panel (as shown below).
Step 4: Create an empty dataset 'cfxdm-metadata-example" pipeline and save that in AIOps studio.
Step 5: Add the following pipeline code/commands into the empty pipeline text field that you have created in the above Step 4.
You can copy the below code into your pipeline and execute that in your environment.
##### This pipeline loads surveys.csv file into AIOps Studio.
##### AIOps studio stores the data loaded from surveys.csv file
##### into local dataset named 'cfxdm-metadata'.
##### prints the metadata from the dataset
@files:loadfile filename = "surveys.csv"
--> @dm:save name = 'cfxdm-metadata'
--> *dm:filter
--> @dm:metadata
In the above pipeline code, we added the '@dm:metadata' bot/function to the pipeline to fetch/extract metadata details of the 'cfxdm-metadata' that was created during the runtime from the surveys.csv file.
Step 6: Click the button 'Verify' and Click the 'Save' button (AIOps studio verifies the pipeline syntax and stores the pipeline into the earlier created 'cfxdm-metadata-example' pipeline as shown below.
Step 7: Users can now execute the above-created pipeline by clicking the 'Execute' button. Once the users click the 'Execute' button, AIOPs studio executes the pipeline that will load the user-defined file, save the loaded file into a local dataset named 'cfxdm-metadata' as shown below.
Step 8: Verify the data using AIOps Studio using 'Inspect --> Output --> Get Data'
Step 9: @dm:metadata has additional filtering capabilities using which users can query specific metadata details/data as shown below.
Include filter - Column name regex pattern to include in the output (include patterns are matched first and then exclude)
Following examples provide usage of include attribute using a simple regular '| (OR)' operation.
Example a:
You can copy the below code into your pipeline and execute that in your environment.
##### This pipeline loads surveys.csv file into AIOps Studio.
##### AIOps studio stores the data loaded from surveys.csv file
##### into local dataset named 'cfxdm-metadata'.
##### prints the metadata from the dataset using a simple regular '|'
@files:loadfile filename = "surveys.csv"
--> @dm:save name = 'cfxdm-metadata'
--> *dm:filter
--> @dm:metadata include = 'plot_id|record_id'
Once you execute the above pipeline using the 'include' filter, the output will apply the filter and show the data based on the filter (as shown below).
Verify the data using AIOps Studio using 'Inspect --> Output /Data --> Get Data'
Example b:
##### This pipeline loads surveys.csv file into AIOps Studio.
##### AIOps studio stores the data loaded from surveys.csv file
##### into local dataset named 'cfxdm-metadata'.
##### prints the metadata from the dataset using '.*id' regular expression
@files:loadfile filename = "surveys.csv"
--> @dm:save name = 'cfxdm-metadata'
--> *dm:filter
--> @dm:metadata include = '.*id'
Exclude filter - Column name regex pattern to exclude in the output.
##### This pipeline loads surveys.csv file into AIOps Studio.
##### AIOps studio stores the data loaded from surveys.csv file
##### into local dataset named 'cfxdm-metadata'.
##### prints the metadata from the dataset using '.*id' regular expression
@files:loadfile filename = "surveys.csv"
--> @dm:save name = 'cfxdm-metadata'
--> *dm:filter
--> @dm:metadata exclude = '.*id'
Last updated