topic Re: GETLineage API in Integration, Extension & APIs

GETLineage API

PhillG — Mon, 27 Feb 2023 11:37:38 GMT

Looking to use the GetLineage API to get a hold on what is and isn't being used...

Not ready to go to SaaS and historically have used Nodegraph, so neither available options, and other than loading through Script Logs, was hoping this may be viable.

Have the connection to the Engine API through Powershell, just not sure how to then call the GetLineage method, to return the results.

Any ideas, pointers, or examples gratefully received, or if there is another way, open to ideas thanks!

Re: GETLineage API

Marc — Tue, 28 Feb 2023 01:20:29 GMT

So there are two options here.

Option 1 is using the QlikSenseSDK, which I assume you are doing already based on the "connection to the Engine API" in which case once you have the App you can just call the GetLineage method like so

#Get the Engine AppIdentifier using the App ID $QEAppIdentifier = [Qlik.Engine.LocationExtensions]::AppWithId($Location,$AppID) #Get the Engine App $QEApp = [Qlik.Engine.LocationExtensions]::App($Location, $QEAppIdentifier) #Call the GetLineage method $QEApp.GetLineage()

the second option is calling the REST API. (Example uses QlikSenseCLI)

#Get the Apps Lineage data $JSONAppLineage = Invoke-QSGet "https://$($QSECMHost)/api/v1/apps/$($AppID)/data/Lineage" -Raw $AppLineage = $JSONAppLineage |ConvertFrom-Json $AppLineage

Example 1 Full:

$AppID = 'ef47c8d8-9049-4ce9-880e-0289422f82c1' $QSEURI = "https://$($env:COMPUTERNAME)" #Need to be added once per Powershell Process add-type -path ".\bin\Newtonsoft\6.0.8\lib\net45\Newtonsoft.Json.dll" add-type -path ".\bin\QlikSenseSDK\16.1.0\Qlik.Engine.dll" add-type -path ".\bin\QlikSenseSDK\16.1.0\Qlik.Sense.JsonRpc.dll" add-type -path ".\bin\QlikSenseSDK\16.1.0\Qlik.Sense.Client.dll" # Set the Engine URL $Location = [Qlik.Engine.Location]::FromUri($QSEURI) $Location.AsNtlmUserViaProxy() #Get the Engine AppIdentifier using the App ID $QEAppIdentifier = [Qlik.Engine.LocationExtensions]::AppWithId($Location,$AppID) #Get the Engine App $QEApp = [Qlik.Engine.LocationExtensions]::App($Location, $QEAppIdentifier) #Call the GetLineage method $QEApp.GetLineage() $QEApp.Dispose() $Location.Dispose()

Example 2 Full:

$AppID = 'ef47c8d8-9049-4ce9-880e-0289422f82c1' $QSECMHost = "$($env:COMPUTERNAME)" if ($null -eq $(Get-Module -Name QlikSenseCLI -ListAvailable)){ Install-Module -Name QlikSenseCLI -Scope CurrentUser -Force } Import-Module QlikSenseCLI Connect-QlikSense -Hostname $QSECMHost -TrustAllCertificates #Get the Apps Lineage data $JSONAppLineage = Invoke-QSGet "https://$($QSECMHost)/api/v1/apps/$($AppID)/data/Lineage" -Raw $AppLineage = $JSONAppLineage |ConvertFrom-Json $AppLineage

Re: GETLineage API

PhillG — Wed, 01 Mar 2023 11:54:18 GMT

Great... Thanks for the great examples @Marc

Appreciated

Re: GETLineage API

PhillG — Wed, 01 Mar 2023 16:19:06 GMT

@Marc I'm finding the results a bit hit and miss, and not as expected.

Some apps are returning null or only a sub section of the actual loads/stores... Is there a particular pattern this method does not support?

The documentation doesn't seem to indicate any limitations.

Re: GETLineage API

Marc — Wed, 01 Mar 2023 21:54:57 GMT

generally you will need to have reloaded the app since the feature was included as the metadata & lineage is recorded as part of the reload process.

Re: GETLineage API

PhillG — Thu, 02 Mar 2023 10:38:08 GMT

Thanks Marc... unfortunately the method doesn't appear to produce consistent outputs.

Whilst a neater and more efficient method, Comparing to the reload script output it doesn't appear to be complete/reliable, unless I'm missing something