topic Re: Qlik Replicate Automated Integration with Repositories in Qlik Replicate

Qlik Replicate Automated Integration with Repositories

Jet — Thu, 25 Jun 2020 15:50:13 GMT

Hi, does Qlik Replicate have automated Integration with any repository like GitHub, Bitbucket, etc? Or our only options is to manually put the json files on the repository?

Re: Qlik Replicate Automated Integration with Repositories

lyka — Thu, 25 Jun 2020 16:34:24 GMT

Hi Jet,

Currently, Replicate don't have any plugins to support source control tools like Github.

You can submit feature requests thru our Qlik Community "Product Insight and Ideas"

Thanks,

Lyka

Re: Qlik Replicate Automated Integration with Repositories

Jet — Fri, 26 Jun 2020 08:40:51 GMT

Thanks! I also heard from one of Qlik replicate's consultant that this has a workaround by using API but I am not yet familiar with using API and I also do not have a plan to do it. Do you have any process and APIs that you can share so I can base my deployment on that? Thanks!

Re: Qlik Replicate Automated Integration with Repositories

JitenderR — Fri, 26 Jun 2020 16:44:06 GMT

API's are supported via QEM. Below is the link for the latest version Developer/API user guide. You first need to define how you want to integrate replicate with github, for example - maintain separate metadata at task level, have tasks and endpoint definition metadata separated etc. and based on that use the corresponding API to extract the metadata and write to git-hub, similarly write to git-hub. Use programming language like Python for integration.

https://help.qlik.com/en-US/enterprise-manager/Content/EnterpriseManager/April%202020/Developers_Guide.pdf

Regards

Re: Qlik Replicate Automated Integration with Repositories

jeboal — Wed, 12 Jan 2022 16:57:00 GMT

this link to the developers guide is bad, can we have a new one please?

Re: Qlik Replicate Automated Integration with Repositories

Heinvandenheuvel — Wed, 12 Jan 2022 18:28:42 GMT

>> his link to the developers guide is bad, can we have a new one please?

Just start at the top and drill down (one step).

https://help.qlik.com/en-US/enterprise-manager

Re: Qlik Replicate Automated Integration with Repositories

mattdevdba — Wed, 12 Jan 2022 20:33:23 GMT

I wrote this blog back in 2018 which used the QEM API's. It was for getting metrics out into Elastic but the process of interacting with the API's for other CI/CD purposes would be similar. I hope if is useful. https://mattdevdba.medium.com/using-the-attunity-aem-rest-api-to-send-metrics-to-elastic-stack-c8ed479fdf8

Re: Qlik Replicate Automated Integration with Repositories

jeboal — Wed, 12 Jan 2022 23:47:05 GMT

Thanks Matt, good example of using the REST APIs - but am not sure this helps me do automated / CD deployments out of a repo into Attunity or Compose DW. I can imagine coding something up to leverage the QEM REST API - but was hoping someone had conquered this already with perhaps wrapping it all up in a chef script or other mechanism. Don't think we're the first ones to want to do CD for our Qlik products... thanks

Re: Qlik Replicate Automated Integration with Repositories

Heinvandenheuvel — Thu, 13 Jan 2022 03:46:43 GMT

@jeboal wrote "I can imagine coding something up to leverage the QEM REST API - but was hoping someone had conquered this already with perhaps wrapping it all up in a chef script or other mechanism. "

Well, I did something along those lines several years ago when it was still called AEM (Attunity). It helped me automate task promotion from dev to qa to prod and other extensive JSON manipulations in files or 'up in the air on the fly extracting and providing.

It will certainly need tweaks but it may be close to what you want or serve as an example. Tricky parts were JSON formatting as PowerShell json manipulations were not very predictable. Text is sometimes easier.

Anyway, here is the Powershell code from 2017 attached as .TXT fill and the 'help' below.

PS C:\Users\hein.HEINV-LPW10> C:\scripts\AEM_Replicate_JSON.ps1 -help AEM_Replicate_JSON.ps1 - Hein van den Heuvel - Attunity - 24-Dec-2017 This script is used obtain an overview of tasks and endpoint for a replicate server through the Attunity Enterprise Manager (AEM) RestAPI or from a provided JSON file. The script can also be used to EXPORT Replicate Server/Task JSON files with optional tweaks. For example, when moving defintions from one server to the next, typically the Endpoints must NOT be carried over as existing endpoint on the targetted server will be used with a different target database(server) and different credentials. We also expect that the endpoint name as well as the taskname will need to be changed according to the naming standards in place to match the new environment. This scripts can adapt the following elements in a Replication Definition - Remove Databases section ( Endpoints ) should they exist, failing to remove could make existing endpoint unoperational on import. - Sort explicit included tables by owner,name - Remove original UUID if present. - Change Task Name - Adaption of End-Points to provided new source and target names - Remove Tasks section - List top-level sections. Typically: tasks, databases, replication_environment, notifications, error_behavior, scheduler - Remove specified top-level section - Add or Replace Explicitly selected tables from CSV file in NAME,OWNER syntax. Owner will be defaulted from the first usage, or provided on commandline. IF the switch "Reference" is used, then NO contents change is done. The purpose of this is to generate a template going through the same power-shell defined formatting for easy comparisons with tools like WinDiff Options: -InputJson Required input file, with single task or full Replication_Definition or Obtain from Attunity EnterPrise Manager (AEM) -Credential Object obtained with : Get-Credential -Credential (or use System.Management.Automation.PSCredential Object) -AemURL Default https://localhost/attunityenterprisemanager -RepServer Targetted Replicate Server through AEM. Default: localhost -RepServer Targetted Replicate Server through AEM. Default: localhost -UserName To allow script to prompt for PassWord. Default: HEINV-LPW10\hein (should be: user@x.y) -OutputJson Name of output Json file formatted similar to Replicate, but not exactly the same. -SourceEP New Name for Source Endpoint (aka "database") IF single task. -TargetEP New Name for Target Endpoint IF single task. -TaskName TaskName to act on, if not the first and only task. -RenameTaskTo New Name for Task IF single task. -SectionName Name of json section to act on. -AddTables_CSV_file Name of CSV file with tables to ADD to the current tables in task (name,owner) -NewTables_CSV_file Name of CSV file with tables to REPLACE the current tables in task with (name,owner) -EPname + -EPdbsettingName + EPdbsettingValue To change End-point setting like "password" -- Switches -- -Reference Generate unchanged contents with script specifix formatting for comparisons -NotSortTables Stop the script from Sorting the explicitly included table list. -DeleteEP Remove the "databases" section from the InputJson -DeleteTasks Remove the "tasks" section from the InputJson -DeleteSection Request removal of json section identified by -SectionName -Import Import using AEM -TaskOverview List Tasks in the json file. -EPOverview List Databases (End Points) section in the json file -SectionOverview List the sections in the json file, useful as quick verification. -help This text