cfxEdgeCollector Auto Export

cfxEdgeCollector Auto Export Functionality

The auto-export functionality allows users to create a schedule configuration for auto-export of the data to target systems. The data could have been collected by creating a single job or multiple jobs.

By creating a (Cron) schedule with a specified time, the cfxEdgeCollector starts collecting the data and merge it into a single zip file. The merged zip file will be exported to the target system at a scheduled time. The zip file name normally is formatted as shown below:

<jobname>_<number of jobs count>_<date>_<timestamp>.zip

Once the export is completed, the system keeps track of data upload status in the database table with details like zip file name, time of upload, the status of upload (failure/success).

In case of failure in uploading the file, the cfxEdgeCollector tries again up to 10 times with a delay of 5 minutes between tries. All failure reasons are captured along with other details like how long the cfxAIA access (in days), access key expiry details, etc.

How to schedule/configure auto-export?

Step 1: cfxedgecollector CLI (register option) is used to configure cfxAIA access keys to the cfxEdgeCollector.

The command line details can be obtained using --help option (as shown below).

>>> cfxedgecollector register -–help

Register the cfxAIA REST access key with the cfxEdgeCollector using the CLI command with options as shown below with an example data (IP address and Key-Value)

>>> cfxedgecollector register --target 10.95.117.30 
--key-name “Realize1” --key-value “acab04f8-50e8-4485-88b4-abafcda9cfe9” --add

>>> cfxedgecollector register --target 10.95.104.40 
--key-name “Realize2” --key-value “ccab02f8-10e6-1382-25b4-abafcda9cfrc” –-add

Users can update or modify the access key or IP address at any time by using the option 'update' in place of 'add'. The configuration can be deleted by using the 'remove' as an option by passing key-name which is unique across the cfxEdgeCollector.

In order to list all the configured keys, use the 'list' CLI option as shown below.

>>>cfxedgecollector register --list    

The following screenshot shows the CLI command and its result in a demo environment.

Step 2: Select 'Auto Export' checkbox and enter access key on UI as shown for each of the job configurations.

Step 3: The following CLI options are available for auto export.

One can export all the jobs data immediately by using the following command.

> cfxedgecollector --auto-export

There will be a check immediately to see the export option is selected and necessary access keys are provided. Then a zip file is created of all related to the key name.

The status of all data uploads can be obtained by using the following command

> cfxedgecollector --export-list    

By default, it will show the last 24 hours, however by changing the option to '--days' or '--months' as shown below. Similarly, there is an option to limit the number of records.

Step 4: The following command will export all the jobs data instantly to the target cfxAIA system.

>>> cfxedgecollector –auto-export

For auto-export, use cron schedule using the following command

>>> cfxedgecollector --jobname export1 
--schedule templates/schedule.json --auto-export-schedule

schedule.json has the following input fields for the Cron schedule.

{
  "minutes": "0",
  "hours": "*/6",
  "day-of-month": "1-15",
  "months": "*",
  "day-of-week": "SUN-SAT"
}

The following is an example screenshot of the output of a cron job list (that was configured in a demo instance).

Cron schedule created to run for every 6 hours

Note: All the logs related to auto-export are logged in the file exporter.log and path as shown below.

<HOME_DIR>/data/exportlogs/exporter.log