topic How to load data from a file in s3 to redshift without headers in Talend Studio

How to load data from a file in s3 to redshift without headers

Anonymous — Sat, 16 Nov 2024 05:48:35 GMT

Hi,

I placed a csv file from my local to s3 using tS3Put component. Now, I want to load the data from that file to a redshift table. I want to load from second row. How can I do this?

Thanks in advance.

Thank You,

Alekhya.

Re: How to load data from a file in s3 to redshift without headers

vapukov — Fri, 17 May 2019 09:10:40 GMT

Hi,

you could use tRedshiftBulkExec component

in advanced setting you can add Redshift COPY parameters, like IGNOREHEADER (number of rows)

Re: How to load data from a file in s3 to redshift without headers

Anonymous — Fri, 17 May 2019 12:42:48 GMT

Hi,

When I run the job, I am facing below error for the column 'effective_from_dt'.

Invalid Date Format - length must be 10 or more

This is the date value in csv -- 12-31-4712 00:00:00

In the schema also, I had defined correctly.

Any idea why is it showing like that?

issue.png

Re: How to load data from a file in s3 to redshift without headers

vapukov — Fri, 17 May 2019 12:47:02 GMT

This is the date value in csv -- 12-31-4712 00:00:00

at least because it is the wrong date 🙂

because you are load (and possibly prepare) csv file self - need to investigate why you have wrong dates in csv

and finally - you can manage date format - redshift copy date format

Re: How to load data from a file in s3 to redshift without headers

Anonymous — Fri, 17 May 2019 13:06:27 GMT

Thanks.

It worked when I declared date format.