Understanding GetTableData's qOffset and qRows

[wingmanzz]

Hello All,

I have been building  some sample apps for my instution using the Qlik Engine JSON RPC API with Python.

Everything seemed to be working ok with my GetTableData calls, until I realized that I was getting duplicated data. I would make the first call like this for example:

{"jsonrpc": "2.0",

"id": 4,

"handle": 1,

"method": "GetTableData",

"params": { "qOffset": 0, "qRows": 100,

"qSyntheticMode": false,

"qTableName": "Indexes" }

}


and I would faithfully get back 100 rows I would expect...

but when I made the next call to get 100 more rows like so...

{"jsonrpc": "2.0",

"id": 4,

"handle": 1,

"method": "GetTableData",

"params": { "qOffset": 100, "qRows": 100,

"qSyntheticMode": false,

"qTableName": "Indexes" }

}

I would get the SAME rows back. In fact the next call with qOffSet set to 200 would also return the same rows. I have verifed this behavior in the Qlik Sense Engine API explorer..i get the same rows. What am I missing about this API endpoint or parameters?

Happy to provide more details if needed. All the calls are made over the python websockets library and I have also verified that the socket is "empty" and Im not just picking off the same message over and over..

[alex_colombo · Accepted solution]

Hi @wingmanzz seems you are using the method in the correct way. Which Qlik Sense version are you using? I tested on QS Aug 2023 patch 8 and it works correctly.

[wingmanzz]

Hi Alex—we are using November 2022 release, patch 13 We will be upgrading to November 2023 this month. I couldnt find a known issue with this mentioned in patch notes or anything, but maybe I missed it.

the table in question has 130k rows. My app admin tells me we are upgrading our dev server on wed to Nov 23. I will test again an post results here.

[wingmanzz]

Hello, Alex--

we upgraded to November 2023 patch 4 yesterday and I retested. We got the same behavior. Dupe records. I am including a zipped up test harness that reproduces the problem in python. Any help in diagnosing that is going on here would be most helpful!

[Øystein_Kolsrud]

Your use of GetTableData looks correct to me, and I ran that program (admittedly after translating it to C#) and it runs just fine. I see you are only fetching one row at a time though. When you say you're getting dupe records, is it your if-condition that is triggering? Could it simply be that your table indeed has two identical records?

That "this_chunk_recs" variable doesn't seem to be computed correctly though. And I assume it's supposed to be used in the GetTableData call, but it isn't. But that doesn't affect the general GetTableData functionallity though.

[wingmanzz]

Thank Alex--yeah, the harness I pulled together was cribbed together from a larger project--please ignore the this_chunk_recs issue

But to your question: yes absolutely sure this table does NOT have duplicate records. it has 130k records. If I request them all, I get them ALL without dupes. if I request them in chunks i get dupes. I only request 1 records chunks here to show the error in the simplest form. yes, it triggers that if condition--in fact I included a log in the zip (I think) which shows the python output--2nd record is the same as the first and the if condition is met.

Also, please not if I just post these request in my engine api explorer web app, I also get dupe records.

This is beginning to sound like this is specific to our app OR our installation of Qliksense. 

I will test against a different app for completeness sake and post results here.

[wingmanzz]

Hi Alex--

So we created a simpler app--one that only has a SINGLE table of 5000 unique integers. The API  worked correctly! It still fails with the "complicated app" though. I'm also  including a new harness and log files (just to clean a few things up). Log_1.txt is the log with complicated app (fails) Log_2.txt is the successful, simple app.


so its something about that particular app. Some things we have conjectured:

1) the app with the issue has multiple tables--could it be related to the fact table duplicates?

2) the app in question has some pretty gnarly section access on it

We are going to add a fact table to the simple app and retest, and then add section access if its still working and retest again.

Thanks for all the attention and eyeballs as we work through this!