topic Re: Using OAuth 2.0 with Talend to Access Google APIs in Talend Studio

Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Wed, 12 Sep 2018 23:19:50 GMT

This tutorial was originally written in 2014 for my website. Although the Google screenshots may be slightly out of date and I wrote this using version 5.5.1 of Talend, you should still be able to make use of this. I have attached a copy of the job and a copy of the parameters file I used for this.

Google make use of the OAuth 2.0 protocol for authentication for (I think, but am happy to be corrected) all of their services. They do a pretty good job of describing the protocol. Once you have read through the documentation you should have a better idea of what you are doing, which should make this easier. The Google documentation is here.

The first thing that needs to be done is to create a Google Project. This is described below....

Create a Google Project

To see the whole post, download it here

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Wed, 12 Sep 2018 23:21:11 GMT

...Continued

15) "Get Access Token and Refresh Token" (tRESTClient)

This component is used to retrieve an access token and refresh token using the authorisation code retrieved from the component before.

To configure this component copy the configuration shown below. Ensure that the sections circled in red are set correctly. To add the "Query parameters" use the green plus symbol circled in red.

The values required can be seen above, but you can find them below so that you can copy and paste them.....

URL: ""<

To see the whole post, download it here
OriginalPost.pdf

Re: Using OAuth 2.0 with Talend to Access Google APIs

somersst — Thu, 14 Feb 2019 08:55:28 GMT

Hi,

I followed the different steps, but I receive the following message :

https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/drive&state=123456789qwertyui&redirect_uri=https://developers.google.com/oauthplayground&response_type=code&client_id=XXXX-t701hme370kfagcvo1052id7e8jblo8b.apps.googleusercontent.com&approval_prompt=auto&include_granted_scopes=true&access_type=offline
s://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/drive
400|{
"error": "invalid_grant",
"error_description": "Malformed auth code."
}

Any idea on what I can be doing wrong??

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Thu, 14 Feb 2019 11:15:38 GMT

I can see that your redirect_uri is incorrect. I used "http://localhost" (or "http%3A%2F%2Flocalhost" with the address url encoded).

https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/drive&state=123456789qwertyui&redirect_uri=https://developers.google.com/oauthplayground&response_type=code&client_id=XXXX-t701hme370kfagcvo1052id7e8jblo8b.apps.googleusercontent.com&approval_prompt=auto&include_granted_scopes=true&access_type=offline

Re: Using OAuth 2.0 with Talend to Access Google APIs

somersst — Tue, 19 Feb 2019 08:12:47 GMT

Hi,

I've corrected this.

I now have the following url :

When I do this and put the correct access-token it works.

But unfortunateley after an hour this doesn't work anymore. I've also included the refresh token, but this doesn't seem to work.

I presume this is something that needs to be set up on the google account??

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Tue, 19 Feb 2019 09:46:13 GMT

It has been a while since I wrote this job (I believe I did it in 2014), so the flow may have changed slightly. But the point of this job is that once the first token is received (access token), the refresh token is saved into a CSV file. The next time the job fails and requires a new access token, the refresh token will automatically be used to create one.

Using the refresh token to retrieve a new access token has a slightly different flow from the original flow to generate both a refresh token and access token. This may be what is tripping you up. However, if you have received a refresh token with your first access token, you should be able to use it to generate a new access token.

Have you tried downloading the job?

Re: Using OAuth 2.0 with Talend to Access Google APIs

somersst — Tue, 19 Feb 2019 12:37:01 GMT

I was using the job. Probably I have forgotten this step.

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Thu, 09 May 2019 15:33:18 GMT

Not able to view/download the image

Re: Using OAuth 2.0 with Talend to Access Google APIs

powerchip — Fri, 28 Jun 2019 13:12:24 GMT

Hi,

Just tried your scenario it's great and well documented

I made some changes to retrieve the code

in the context file

replace : redirect_uri;http%3A%2F%2Flocalhost
with : redirect_uri;urn:ietf:wg:oauth:2.0:oob

In the steps
13.

replace : redirect URL
with : code

14.
comment/remove the substring part
15.
in query parameters

replace : http://localhost
with : context.redirect_uri

Result :
In the browser you can directly copy-paste the code
It's less disturbing than the error page

Now I need to figure out how to retrieve data from Google with TOS and use it

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Fri, 28 Jun 2019 16:44:35 GMT

Nice additions. This was written a long time ago and things have moved on a bit with Google OAuth 2.0 functionality. I'm glad this still works and that you have been able to improve upon it @powerchip

Re: Using OAuth 2.0 with Talend to Access Google APIs

powerchip — Wed, 03 Jul 2019 08:26:56 GMT

I juste found this in the palette tGoogleDriveConnection

Maybe this can be easier

It's a Drive connection but I think it will work with other apis from Google, I'll need time to test it

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Thu, 04 Jul 2019 12:23:37 GMT

Hi @rhall

Thanks for this helpful tutorial,

I have a probelem that when i get the bearer from Trestclient, i want to enter the bearer and send a json file(content informations) to the api but i don't know how?

you will find attached a screen of my job.

Thanks in advance.

bearer.PNG

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Thu, 04 Jul 2019 12:42:23 GMT

Hi @shima,

I am not sure that this is directly related to this tutorial. Could I ask you to post a new question and give a bit more information as to what you have done and what you wish to achieve?

Regards

Richard

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Sat, 20 Jul 2019 15:16:00 GMT

Actually, we are trying to make a post (json) api call using Oauth2.0 authentication. Please find below step, what we have done with curl command. Exactly, same thing needs to be implemented in Talend.

Step1:-
Passed curl command to fetch access token below:-

curl -v -H "Content-Type: application/x-www-form-urlencoded" -X POST --data "client_id=8BYA26jcZ4cYcivjX0oUABvefmmeqErN&client_secret=lWRoPehoFCmypPls&grant_type=client_credentials&scope=DHub" https://slot1.org009.t-dev.corp.nutty.com/v2/oauth/token --cert /usr/local/certs/nutty.com.crt --key /usr/local/certs/nutty.com.key -v –k

Step2:-
From first step, we will be getting access token:-

{"access_token":"k420G8LGaNQSUXNhih0T3xgAIwyE","token_type":"Bearer","expires_in":"3599"}

Step3:-
Using this access token in curl command we have to make another POST API call:-

curl -v -X POST -H "Content-Type: application/json" -H "Authorization: Bearer k420G8LGaNQSUXNhih0T3xgAIwyE" -H "Correlation-Id: b8905354-a164-b480-9fb2-75c2d1a8498x" -H "Source-System: Dhub" --data @usage2.json https://slot1.org009.t-dev.corp.nutty.com/application/b2b-bds-dev/v1.0/billing-events/700001276364/subscriptions/6790fac7-3ae9-405f-be26-20c5ac317297/usages --key /usr/local/certs/nutty.com.key --cert /usr/local/certs/nutty.com.crt -v –k

Please see a snapshot of data, what we are trying to post through usage2.json file:-

{
"instanceId": "b4ee9fc9-7f50-49f4-a90f-32b0ffcf3c73",
"units": 120,
"Source-System": "D-Hub",
"effectiveDate": "2019-07-05T06:00:16.000Z",
"billingSpecId": "GSS_01",
"eventType": "GMM"
}

Final Output, which we expect:-

{"code":201,"status":201,"message":"Created"}

This is what we are trying to achieve in Talend. Is it possible to first authenticate using Oauth2.0 authenticationa nd then pass the access token to make a POST api call.

Re: Using OAuth 2.0 with Talend to Access Google APIs

indrajeet183 — Thu, 26 Sep 2019 14:11:32 GMT

Hi,
Thanks for wonderful guide for oauth, I'm currently referring this for my project.
I'm having difficulty in tMsgBox on linux(Fedora 29), I don't know if this is default behaviour of tMsgBox but whenever I click outside(want to copy URL) anywhere or do ALT+TAB (for opening browser) the msgbox get's disappear. But the job doesn't get killed it stays running.

Is there any workaround for this or this is only because I'm on linux?

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Thu, 26 Sep 2019 14:41:13 GMT

It sounds like the message box has lost focus. Try minimising the Studio or using alt+tab until the message box is viewable.

Re: Using OAuth 2.0 with Talend to Access Google APIs

indrajeet183 — Thu, 26 Sep 2019 15:04:05 GMT

Hi, I already checked that too, I thought it's minimized but I can't find it in ALT+TAB, also I can't find it in my task bar too.

I will try to switch desktop environment just to see if it's working or not.

Also just for confirmation I tried same on Windows and voila the problem is not there .....

Re: Using OAuth 2.0 with Talend to Access Google APIs

indrajeet183 — Thu, 26 Sep 2019 15:25:00 GMT

Got it, ALT+TAB doesn't work but if I minimise only using mouse I can see the msgbox. Thanks for the help.
Also I wanted to ask one more question, right now I'm doing all this in local environment so can you please tell me how can this deployed on production and if I'm able to only access production instance using SSH how I can deal with auth as there wont be messagebox.

Re: Using OAuth 2.0 with Talend to Access Google APIs

Anonymous — Thu, 26 Sep 2019 15:31:30 GMT

You will need to separate this job into 2 parts; the part which uses a refresh token to update the live auth token (+ the rest of your job) to be run on your server, the part which generates the refresh token will need to be done manually. Once the refresh token is generated, this should continue to run until your refresh token is removed or unauthorised.

Re: Using OAuth 2.0 with Talend to Access Google APIs

indrajeet183 — Thu, 26 Sep 2019 16:08:03 GMT

Yes, that makes sense I was thinking a hard way like maybe creating service(same redirect url) on server where it will save the auth code in existing file but I don't know if that's feasible or not.

Also I'm quite new to Talend so I don't know the best practices so can you please tell me any resources or your own suggestion for deploying and VCS?
I mostly work with integration stuff but using code so this is all new and I'm having hard time figuring out how can I deploy to production how to maintain error logs.

Also whenever you are having free time please check this https://community.talend.com/t5/Design-and-Development/Data-integration-for-Magento-to-Quick-Book/m-p/202719#M110760