Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi folks ,
I am trying to extract data from Bigquery to Qlikview .Record count in BQ table is over 60 Million.
The data extraction activity runs fine for smaller data pulls but throws below error while pulling data for over 2 hours.
Error: ERROR [28000] [Qlik][DriverOAuthSupport] (8711) Authentication failed: invalid_grant
Please advise.
If it worked with smaller datasets and failed with bigger ones which may run for hours you may hit any timeout. This may not only on the driver + db side else each part within your environment/network could cause it even those which aren't the first address to think of like proxy-server, load-balancer and so on.
It might not easy to detect the real cause and even if you might not be able/allowed to adjust the settings like you need them. Therefore I suggest to split your query into several smaller parts and/or not to load the whole dataset else using an incremental approach and just pulling the most current data.
- Marcus
Thanks for the suggestion . In my data set , i don't have a valid column like (City, Geo unit,Business Unit etc) on which i can split the data. Date field is also causing same issue .
it would be helpful if you could suggest a way to split the data considering row count .
any way to extract the data from BQ part by part like 10 M at a time and saving it in qvd and then next 10 M records and saving it in next lot .
thanks in advance
Any period-field like a date is suitable. With it you could set appropriate where-clauses within loops, maybe something like the following:
for i = 1 to 5
let vStart = date(floor(yearstart(today(), -$(i))), 'YourFormat');
let vEnd = date(floor(yearend(today(), -$(i))), 'YourFormat');
let vYear = year('$(vStart)');
table: sql select * from source where DATE >= '$(vStart) and DATE <= '$(vEnd)';
store table into table_$(vYear).qvd (qvd);
drop tables table;
next
- Marcus