Qlik Application Automation: Incrementally export audit trail logs from Qlik Enterprise Manager to Splunk

[AfeefaTk]

This article provides an overview of the template configured in Qlik Application Automation to export Audit trail data from Qlik Enterprise Manager and log it into Splunk.

You can make use of the template which is available in the template picker. You can find it by navigating to Add new -> New automation -> Search templates and searching for 'Incrementally export audit trail logs from Qlik Enterprise Manager to Splunk' in the search bar.  Click on the Use template option to use it in the automation.

You will find a version of this automation attached to this article: "Incrementally-export-audit-trail-logs-from-Qlik-Enterprise-Manager-to-Splunk.json".

Limitations

• The 'List Audit Trails Incrementally' block is currently limited to returning only 12MB of audit trail data. 
• By default, Enterprise Manager retains audit files for 1 week or until they reach a total size of 100 MB. The audit trail retention size and age can be changed through the Enterprise Manager command line interface.
• This block will become inefficient if the audit trail retention settings have been changed to accommodate maximum retention size and age.
• If the block fails with an Out of Memory or Server error:
  • Set 'Fetch all records on first run' in the settings tab of the block to No.
  • Schedule the automation to execute this endpoint at least every 5 or 10 minutes.
  • Alternatively, schedule using another interval that accommodates the number of logs that are continuously generated by your Enterprise Manager servers.

Full Automation

Step-by-step description

1. data: Variable used to count the number of audit trail data logged into Splunk. The initial value is 0.
2. Add the 'List Audit Trails Incrementally' block to retrieve audit trail records from all servers incrementally.
3. Add a condition block to check if the response from the 'List Audit Trails Incrementally' block is not empty. If the condition block outcome evaluates to true:
   1. Add a 'Send Event' block from the Splunk connector to log audit trail data into Splunk.
   2. Add 1 to the 'data' variable during each iteration. This variable will provide the total number of audit trail logs that have been logged into Splunk.
4. Add a condition block to check if audit trails have been logged into Splunk.
5. If yes, use the 'Update Run Title' block to specify the count of audit trail data that has been processed to Splunk during the run. The job title is visible when looking at the automation history. For more info: https://help.qlik.com/en-US/cloud-services/Subsystems/Hub/Content/Sense_QlikAutomation/advanced/upda...

Related Content 

• How to: Get started with the Qlik Enterprise Manager Connector

[jorono]

Can be used this template and write the logs in elastic?

[AfeefaTk]

Hi @jorono 

This template will allow you to export audit trail logs from Qlik Enterprise Manager and write it to Splunk

I am not sure what you mean by elastic here.

Could you please explain a bit?

Thanks!

Afeefa TK

[AfeefaTk]

Hi @jorono 

Did you mean elastic logging platform?

If yes, currently we don't have elastic connector available in QAA. So unfortunately that won't be possible as of now

Maybe you can use our ideation board on the community to create new connector request 

https://community.qlik.com/t5/Suggest-an-Idea/idb-p/qlik-ideas

Thanks!

Afeefa TK

[Purushothaman]

@AfeefaTk ,

How to perform the same in Task using Qlik Sense Enterprise. In my use case, Instead of feeding into splunk, I need the incremental logs to stored  it as CSV.

Please help.

Thank you,

[AfeefaTk]

Hi @Purushothaman 

You can replace Splunk connector blocks with  Microsoft Excel connector blocks or use any cloud storage connectors to store the csv file and insert data into that file.

Refer to below links:

1. Microsoft Excel
2. Cloud Storage Connectors
   1. Amazon S3
   2. Dropbox
   3. Microsoft OneDrive

Hope this helps!

Thanks