topic Getting error in TMC for tsystem component trying to fetch GCP token in Talend Cloud

Getting error in TMC for tsystem component trying to fetch GCP token

syadav1622638282 — Fri, 15 Nov 2024 22:50:22 GMT

Hi ,

I have created a job which connect to GCP using token, I am using tsystem component to run gcloud.cmd print auth-access-token. job is running fine in studio and giving proper output but when I am trying to run it from TMC it is giving me below error.

Exception in component tSystem_6 (test_gcs_connection) java.io.IOException: Cannot run program \"gcloud.cmd\" (in directory \"C:\\Program Files (x86)\\Google\\Cloud SDK\"): CreateProcess error\u003d2, The system cannot find the file specified", .

Job structure is attached .

Can you provide resolution on the same.

Re: Getting error in TMC for tsystem component trying to fetch GCP token

Anonymous — Tue, 08 Jun 2021 06:44:07 GMT

The job is trying access a local folder, you should use an remote engine rather than Talend Cloud as runtime engine.

Regards

Shong

Re: Getting error in TMC for tsystem component trying to fetch GCP token

syadav1622638282 — Tue, 08 Jun 2021 06:58:10 GMT

Hi Shong ,

I am running job on remote engine itself from TMC, path "C:\\Program Files (x86)\\Google\\Cloud SDK\") is given in the tsystem component's home directory.

If I do not provide the home directory still it gives me below error.

java.io.IOException: Cannot run program "gcloud.cmd": CreateProcess error=2, The system cannot find the file specified.

It is giving issue while executing "gcloud.cmd auth print-access-token" command from tsystem componenet which is 1st component of the job .

Re: Getting error in TMC for tsystem component trying to fetch GCP token

Anonymous — Tue, 08 Jun 2021 07:53:44 GMT

Is the remote engine installed on the same machine where your studio is installed? You mentioned the job works fine in studio.

Re: Getting error in TMC for tsystem component trying to fetch GCP token

syadav1622638282 — Tue, 08 Jun 2021 08:20:31 GMT

No remote Machine is separate its not the same where studio is installed, but from TMC(connects to talend remote server) I am able to run jobs which are having DB connectivity, only GCP command is not working.

Re: Getting error in TMC for tsystem component trying to fetch GCP token

Anonymous — Tue, 08 Jun 2021 09:31:07 GMT

This could be for a number of reasons. First of all, is the gcloud.cmd installed on the remote engine machine? If it is, then maybe this KB article will help you....

https://community.talend.com/s/article/tsystem-does-not-work-as-expected-on-Remote-Engine-server-u0CDq

Re: Getting error in TMC for tsystem component trying to fetch GCP token

syadav1622638282 — Tue, 08 Jun 2021 09:34:46 GMT

Yes gcloud .cmd is installed on remote engine it is able to generate token on gcloud sdk .

The given link is not opening show invalid page.

Re: Getting error in TMC for tsystem component trying to fetch GCP token

Anonymous — Wed, 09 Jun 2021 09:43:30 GMT

This is an internal KB article, I copy the contents here, hope it helps.

Problem Description:

Tsystem does not work on the Remote Engine server though we have a big query cloud SDK installed on the server which uses the query passed by the job. We have verified that the big query command line utility works fine from every path and have tested multiple times. But when the job tries to use the same utility it gives me error like 'bq is not recognized as the internal or external command' which is a typical cmd error for not finding this utility.

Root Cause:

This behavior is because the Remote Engine is probably running with a different (system) account, it seems that the Path variable is not correctly set.

Workaround:

A workaround for this is to set the home directory in the tSystem component. You can also set the Path once for all jobs in the following file for example: E:\TalendRemoteEngine\etc\Talend Remote Engine-wrapper.conf and, add this property: set.PATH=E:\Google Cloud SDK\google-cloud-sdk\bin;%PATH%, and restart the Talend Remote Engine service.

The big query command should then run successfully against the sdk, either using a single or array command.

Regards

Shong