Extract complete json of an object, disregarding paging

[sergiogiogio]

I am using the qlik .net API I need to extract the complete json of the object, including all rows. Currently when I use <object>.Layout.ToString() I only have the 50 first rows in the json (qPivotDataPages).

I suspect that what I am observing is an "initial data fetch limit" meant for optimization purposes and I have the below questions:

1- Is there a way to configure the object to load all rows, or at least more rows, so that <object>.Layout.ToString() returns more data? Is there a way to increase this "initial data fetch limit" for the object or for the session altogether?

2- Alternatively, I suppose I can fiddle with GetData methods and iterate page by page, etc. but I suppose it will be very difficult to recreate the json. When I export a visualization to png, I see the complete json of the object sent to the printing service so I suspect there may be a more straightforward way?

Thanks in advance for any advice!

[Øystein_Kolsrud]

Hi! You are right in that you are probably only getting the part of the data covered by the "initial data fetch" setting. That setting is part of the hypercube definition and is intended to be used as convenient way of getting the data required for the initial rendering. The setting is defined here:

https://help.qlik.com/en-US/sense-developer/February2023/Subsystems/NetSDKAPIref/Content/Qlik.Engine...

One option for you could be to modify that setting and simply use a larger page that covers all your data. But keep in mind that the engine has a page size limit of 10 000 cells, so if your table has more cells than that then you'll have to use the "GetHyperCubeData" method to retrieve all your data. Or since you're dealing with a pivot table, you'll probably need the method "GetHyperCubePivotData".

The .NET SDK has a convenience class called "DataPager" that is intended to make such data extraction simpler. It only supports straight tables though, not pivot tables, so it might not be of interest to you in this case. But that class contains for instance a method called "GetAllData" that iterates through the entire set of data so that you can process it, and it does so lazily one page at a time so that you can avoid loading millions of rows into memory before you can start working through the data:

https://help.qlik.com/en-US/sense-developer/November2022/Subsystems/NetSDKAPIref/Content/Qlik.Engine...

[sergiogiogio]

Thanks @Øystein_Kolsrud , your first proposal worked for me but was more difficult than planned.

The below straightforward powershell code did not work (still only 50 records in the json output):

$GenericObject.Properties.HyperCubeDef.InitialDataFetch[0].Height = 100

However the below code did work (more then 50 records generated in the json output):

$initialDataFetchPatches = New-Object System.Collections.Generic.List[Qlik.Engine.NxPatch]
$initialDataFetchPatch = New-Object Qlik.Engine.NxPatch
$initialDataFetchPatch.Op = "Replace"
$initialDataFetchPatch.Path = "/qHyperCubeDef/qInitialDataFetch/0/qHeight"		
$initialDataFetchPatch.Value = "100"
$initialDataFetchPatches.Add( $initialDataFetchPatch )
$GenericObject.ApplyPatches($initialDataFetchPatches, $true) # soft patch

This might be a basic question about the qlik engine .net API, but is there a way to make the first code work?

[Øystein_Kolsrud · Accepted solution]

Just setting the property "Height" in that hypercube won't have an effect until you run a SetProperties call to the engine. Typically you  would do something like this (in C#, not sure how using works in PowerShell):

using (o.SuspendedLayout)
{
    o.Properties.HyperCubeDef.InitialDataFetch[0].Height = 100;
}

The using statement has two effects, it blocks any change notifications from affecting the properties while they are being edited and it ensures that a "SetProperties" is sent for the object when the using clause is exited.

Another option would be to simply do a Get/Set combination on the properties like this:

var props = o.GetProperties();
props.HyperCubeDef.InitialDataFetch[0].Height = 100;
o.SetProperties(props);

[sergiogiogio]

Thanks again! Powershell apparently has no equivalent of Using statement so requires to call Dispose() explicitly which can be confusing. So I went with the second option as it is easier to understand, everything works perfectly.