topic Re: Random store error with Google Drive connector Saas in Connectivity & Data Prep

Random store error with Google Drive connector Saas

achampire — Fri, 09 Dec 2022 10:04:50 GMT

Hello guys,

I am facing since the 6th december 2022 a problem with applications using Store instruction on a Google Drive folder.

Everything worked fine until this date. Applications runned hourly without any problems and all my QVDs were generated correcly.

Since the 6th december at 08:00 PM CEST, my reload tend to fail on a random Store instruction. By random, I mean that it is not especially one QVD.

I have 2 types of error thatI am facing

Data has not been loaded. Please correct the error and try loading again.
The following error occurred:
Cannot open file: 'lib://Google drive connector/<idfolder>/accounts.qvd' (Connector error: Error completing job: Object reference not set to an instance of an object.)
The error occurred here:
Store accounts into [lib://Google drive connector/<idfolder>/accounts.qvd] (qvd)

Data has not been loaded. Please correct the error and try loading again.
The following error occurred:
Cannot open file: 'lib://Google drive connector/<idfolder>/projet .qvd' (Connector error: Exception in reading response chunk for upload: Status(StatusCode="Unknown", Detail="Error during upload: Object reference not set to an instance of an object."))
The error occurred here:
Store projet into [Google drive connector/<idfolder>/projet .qvd] (qvd)

Do you guys have an idea about why do I have this errors? Are you facing the same problem since the 6th december too?

Re: Random store error with Google Drive connector Saas

onebot88 — Fri, 09 Dec 2022 19:28:22 GMT

I'm getting the exact same error. I don't know what to think, sometimes my script completes the stores without any issue and sometimes the error appears on table A or table B. I didn't change anything from my script and it was working fine until December...

Qlik, do you know something about it?

Best regards.

Re: Random store error with Google Drive connector Saas

ChannaK — Mon, 12 Dec 2022 02:18:16 GMT

i do have same issue from 6th december 2022

failed to upload into google drive.qlik suggest to use sleep command but i still see same issue

Re: Random store error with Google Drive connector Saas

d_abbatelli — Mon, 12 Dec 2022 07:31:34 GMT

Same issue here, about 50% of reloads using store have started failing on store since 6th December (in my case, 6 PM CET).

Re: Random store error with Google Drive connector Saas

barsssk — Mon, 12 Dec 2022 11:39:36 GMT

Same here, since Dec 6th.

As I heard, google drive changed its authentication method. I think Qlik Connectors couldnt catch the change yet, unfortunately..

All of my dashboards rely on qvd import/export to google drive. Please fix this ASAP.

Re: Random store error with Google Drive connector Saas

achampire — Mon, 12 Dec 2022 11:43:06 GMT

Hell @barsssk

Do you have any documentation, help page or whatever regarding the Google Drive modification on its authentication method ?

Re: Random store error with Google Drive connector Saas

barsssk — Mon, 12 Dec 2022 11:53:54 GMT

@achampire

Sadly no, I couldnt confirm it, still working on it. Thats why I said "I heard", but it made sense, since nothing changed qliksense side.

Re: Random store error with Google Drive connector Saas

Albert_Candelario — Mon, 12 Dec 2022 13:02:05 GMT

Hello all,

Thanks for highlighting this issue.

Our engineering team is investigating it on high priority.

When having a meaningful update we will update.

Cheers,

Albert

Re: Random store error with Google Drive connector Saas

barsssk — Tue, 13 Dec 2022 07:48:44 GMT

@achampire Mail I got from support:

"According to the R&D the root cause of this issue has been found and an according fix has been released.

Can you please test on your side and let me know if the result?"

I dont get error anymore, can you try it too?

Re: Random store error with Google Drive connector Saas

achampire — Tue, 13 Dec 2022 08:43:37 GMT

@barsssk I got the same answer from a case that I created yesterday on that topic.

I activated my reload schedule this morning (it was deactivated by the 5th time on failed).

It seems good for the first reload. I will take a look today to confirm that.

Re: Random store error with Google Drive connector Saas

ChannaK — Tue, 13 Dec 2022 09:16:37 GMT

@barsssk

i am getting below error

Connector error: <html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"/><title>Sorry...</title><style> body { font-family: verdana, arial, sans-serif; background-color: #fff; color: #000; }</style></head><body><div><table><tr><td><b><font face=sans-serif size=10><font color=#4285f4>G</font><font color=#ea4335>o</font><font color=#fbbc05>o</font><font color=#4285f4>g</font><font color=#34a853>l</font><font color=#ea4335>e</font></font></b></td><td style="text-align: left; vertical-align: bottom; padding-bottom: 15px; width: 50%!"(MISSING)><div style="border-bottom: 1px solid #dfdfdf;">Sorry...</div></td></tr></table></div><div style="margin-left: 4em;"><h1>We're sorry...</h1><p>... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.</p></div><div style="margin-left: 4em;">See <a href="https://support.google.com/websearch/answer/86640">Google Help</a> for more information.<br/><br/></div><div style="text-align: center; border-top: 1px solid #dfdfdf;"><a href="https://www.google.com">Google Home</a></div></body></html>)

Re: Random store error with Google Drive connector Saas

barsssk — Tue, 13 Dec 2022 09:30:58 GMT

@ChannaK

I haven't seen this issue before but I don't think it is related to the one in this topic.

But if i were you, I would test my connection, verify it or recreate it again and check if my google acount is accessible from Qlik Connectors.

Re: Random store error with Google Drive connector Saas

achampire — Tue, 13 Dec 2022 13:24:47 GMT

@barsssk

I faced the same error as ChannaK 10 minutes ago.

At the moment I have a 87% success on my extraction reloads which is clearly better than less than 25% before the fix.

Here below 2 new kinds of error that I am facing.

Data has not been loaded. Please correct the error and try loading again.
The following error occurred:
Cannot open file: 'lib://DataConnection/folder_id/jira.qvd' (Connector error: Error completing job: Failed on attempt 6 to PATCH. (Error while copying content to a stream.))
The error occurred here:
Store jira into [lib://DataConnection/folder_id/jira.qvd] (qvd)

Data has not been loaded. Please correct the error and try loading again.
The following error occurred:
Cannot open file: 'lib://DataConnection/folder_id/jira.qvd' (Connector error: <html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"/><title>Sorry...</title><style> body { font-family: verdana, arial, sans-serif; background-color: #fff; color: #000; }</style></head><body><div><table><tr><td><b><font face=sans-serif size=10><font color=#4285f4>G</font><font color=#ea4335>o</font><font color=#fbbc05>o</font><font color=#4285f4>g</font><font color=#34a853>l</font><font color=#ea4335>e</font></font></b></td><td style="text-align: left; vertical-align: bottom; padding-bottom: 15px; width: 50%!"(MISSING)><div style="border-bottom: 1px solid #dfdfdf;">Sorry...</div></td></tr></table></div><div style="margin-left: 4em;"><h1>We're sorry...</h1><p>... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.</p></div><div style="margin-left: 4em;">See <a href="https://support.google.com/websearch/answer/86640">Google Help</a> for more information.<br/><br/></div><div style="text-align: center; border-top: 1px solid #dfdfdf;"><a href="https://www.google.com">Google Home</a></div></body></html>)
The error occurred here:
Store jira into [lib://DataConnection/folder_id/jira.qvd] (qvd)

I copied paste the HTML code "for fun" and attached the result.

I guess it is due to a Google protection.

Re: Random store error with Google Drive connector Saas

Albert_Candelario — Tue, 13 Dec 2022 13:33:46 GMT

Hello all,

Thanks for confirming that the issue has been mitigated.

Our teams are still working in this topic.

Thanks for your collaboration.

Cheers,

Albert

Re: Random store error with Google Drive connector Saas

barsssk — Tue, 13 Dec 2022 13:50:00 GMT

@achampire Just tried 20 reloads and couldnt get the error (fortunately 🙂 )

I hope it will being solved for you soon.

Re: Random store error with Google Drive connector Saas

Albert_Candelario — Tue, 13 Dec 2022 13:50:49 GMT

Thanks for letting us know @barsssk

Re: Random store error with Google Drive connector Saas

achampire — Wed, 14 Dec 2022 08:40:05 GMT

Hello.
Here facts since the mitigation mechanism implementation.

I had since yesterday 99 major reloads split between 4 applications (2 with a hight amount of store instruction , 1 with less than 10 store instruction, 1 with less than 3 store instruction).

I faced 3 type of error:
- Error 1: html code with "The server encountered a temporary error and could not complete your request."
- Error 2: html code with "We're sorry...</h1><p>... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now."
- Error 3: error "Connector error: Error completing job: Failed on attempt 6 to PATCH. (The remote server returned an error: (429)"
Error 3 is split between Too Many Requests and Service Unavailable

On 99 reload, 9 failed that is 9% of failed reload.
Below the repartition.
error 1 error 2 error 3
1 2 6

I faced an average of 4 errors for applications with hight amount of store instruction, only one error for the application with less than 3 store instruction and 0 for the application with less than 3 store instruction).

Re: Random store error with Google Drive connector Saas

Albert_Candelario — Wed, 14 Dec 2022 09:54:16 GMT

Hello all,

We have delivered a new code that should reduce the failure rate even more.

Cheers,

Albert

Re: Random store error with Google Drive connector Saas

achampire — Thu, 15 Dec 2022 09:08:50 GMT

Hello @Albert_Candelario

Thank you for this information.

There is one day since this new code and I have errors since its deployment.

I am wainting today's reload in order to make some analysis and report the result here.

I do the same on a current case that I created.

Thx.

Alexandre CHAMPIRÉ

Re: Random store error with Google Drive connector Saas

Albert_Candelario — Thu, 15 Dec 2022 13:52:45 GMT

Thanks @achampire. Please, keep updating your support case and done.

Appreciated your feedback.

Cheers,

Albert