Rest connector Received message larger than max

[anderslinden]

(Connector error: Received message larger than max (8504391 vs. 4194304))

I get this message when fetching data from XLEDGER services using the REST Connector in Qlik Cloud.

Is there a way to get around it in the script or other ways?

Now I need to loop through a lot of requests with variables for selections, and not beeing certain if one day one of them will also be too large and the script will fail, or not get the data I need.

It's only balance data from a general ledger and not even the transactional data, but the response I get is obviously too large anyway when recieved. I get the full response in just one record from the service, which probably complicates this.

Any ideas/suggestions?

[Damien_V]

Hello @anderslinden,

The limitation is on the Qlik Engine side (4MB). It's recommended to load the data in smaller chunks (by applying filters or by paging) if you have data bigger than that.

Further explanation:

When selected from a database, the connector streams data to engine in chunks. 
but in some situations when a data field itself will exceed the 4MB limit, that cannot be streamed without getting truncated.

For the REST connector also, the connector applies the same streaming to the Qlik engine. But when the whole data is in one single data field, instead of rows, then it may result in the mentioned error or lossy data.

[anderslinden]

Thx for the reply!

Teh issue is in the REST connector. The "answer" I get from the Connector puts the full xml-response in one single data field as you wrote. Even if I say that it is an xml answer it still ends up in one single data field, and in a later stage I now have to enterpret the xml-data with subield and textbeween statements to extract the data. This is unforunate since it really is a simple xml response, that should be enterpreted as such already n the rest connector. The api is from XLEDGER. For now I use GetBudgetDetailsData and GetGeneralLedgerProfitAndLossData. I can't even consider transactional data on voucher level, since it will be much larger data sets, and this problem will multiply....

/Anders

Can you se another way I could do this?