topic Re: Talend tAzureAdlsGen2Output component datetime output format issue in Talend Studio

Talend tAzureAdlsGen2Output component datetime output format issue

vbadri — Sat, 16 Nov 2024 00:14:26 GMT

Hello there,

I'm using 'tAzureAdlsGen2Output' component to write some csv files from oracle (via Dynamic schema), it is auto converting datetime columns to a format that was never defined within the job. If we send the same date as string/varchar it is writing as is.

'2011-10-21T09:09:36Z[UTC]' instead of '2011-10-21 09:09:36' even though I set the required format as "yyyy-MM-dd HH:mm:ss"

Any advice would be really helpful, thanks in advance.

Re: Talend tAzureAdlsGen2Output component datetime output format issue

Anonymous — Thu, 22 Apr 2021 06:19:08 GMT

Hello,

Could you please show us your tAzureAdlsGen2Output component setting screenshots?

Best regards

Sabrina

Re: Talend tAzureAdlsGen2Output component datetime output format issue

vbadri — Fri, 23 Apr 2021 14:51:21 GMT

Thanks for your prompt reply, please find the below screenshots, try to replicate it on your machine, thanks.

In this I'm trying to write a DateTime & String_DateTime values into a csv file on Azure Container.

tFixedFlow config

tAzureAdlsGen2Output config

Job Log

Output from Azure

Please let me know if i am missing anything from the config or guide me where to set the format the 'tAzureAdlsGen2Ouput' will pick the date format from.

Re: Talend tAzureAdlsGen2Output component datetime output format issue

vbadri — Fri, 07 May 2021 21:55:35 GMT

Hello Sabrina,

did you had a chance to look this issue?

Re: Talend tAzureAdlsGen2Output component datetime output format issue

Anonymous — Wed, 12 May 2021 06:13:40 GMT

Hello,

The data of Date/Time type is converted from local timezone(e.g, GMT+8 timezone) to UTC timezone in Azure?

Could you please try to add '-Duser.timezone=SGT' to the JVM setting. (Talend job - Run tab - Advanced settings) to see if it works?

Best regards

Sabrina

Re: Talend tAzureAdlsGen2Output component datetime output format issue

vbadri — Fri, 14 May 2021 09:29:37 GMT

Thanks for your reply.

The data of Date/Time type is converted from local timezone(e.g, GMT+8 timezone) to UTC timezone in Azure?

Yes, no matter what ever the Date/Time type input is given, the component is converting it to UTC in azure.

Could you please try to add '-Duser.timezone=SGT' to the JVM setting. (Talend job - Run tab - Advanced settings) to see if it works?

I have tried this earlier from one of the posts, it didn't work, I am in GMT, I tried with SGT but the output on Azure is still appear to be in UTC.

Re: Talend tAzureAdlsGen2Output component datetime output format issue

Anonymous — Mon, 17 May 2021 09:54:23 GMT

Hello,

Could you please clarify in which Talend version/edition you are?

Best regards

Sabrina

Re: Talend tAzureAdlsGen2Output component datetime output format issue

vbadri — Tue, 18 May 2021 01:15:55 GMT

I'm currently using Talend Data Fabric (R2021-04-7.3.1)

Re: Talend tAzureAdlsGen2Output component datetime output format issue

Anonymous — Tue, 18 May 2021 03:33:34 GMT

Hello,

I'm suspecting that Azure DB stores times in UTC by default and will convert any local time representations into this form. We will make an investigation on it and then come back to you as soon as possible.

Best regards

Sabrina

Re: Talend tAzureAdlsGen2Output component datetime output format issue

vbadri — Tue, 18 May 2021 15:14:50 GMT

Yeah, maybe Azure doing that, it would be nice if we can control that from Talend while providing the output to Azure.

EDIT : Because of that, we had go through so many workarounds!!

Re: Talend tAzureAdlsGen2Output component datetime output format issue

Anonymous — Wed, 26 May 2021 07:31:28 GMT

Hello,

I found a KB article about: https://community.talend.com/s/article/Talend-Data-Types-explained

Feel free to let us know if it helps.

Best regards

Sabrina

Re: Talend tAzureAdlsGen2Output component datetime output format issue

vbadri — Wed, 26 May 2021 09:21:27 GMT

Thanks for your response.

That link wasn't working as expected, please double check.

Thanks.

Re: Talend tAzureAdlsGen2Output component datetime output format issue

Anonymous — Thu, 27 May 2021 04:16:20 GMT

Hello,

Could you please create a work item jira issue on talend bug tracker?

https://jira.talendforge.org/secure/Dashboard.jspa

Best regards

Sabrina

Re: Talend tAzureAdlsGen2Output component datetime output format issue

TGorman — Mon, 18 Nov 2024 16:37:42 GMT

Is there any update regarding this issue as this is something that we are also having difficulties with.