Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in NYC Sept 4th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Talend Redshiftrow Error

Hi ,

 

Can anybody help me,while i am running talend job with copy command its throwing error like.

insufficient data left in message during my COPY from S3 to Redshift.

 

Can anybody help me.

 

 

Regards,

Prasanth

 

Labels (1)
  • Other

6 Replies
vapukov
Master II
Master II

It is not a Talend error message
it is Redshift (Postgres) 

You can check,  what happens exactly by select data from sql_load_errors:

select 
     d.query, 
     substring(d.filename,14,20), 
     d.line_number as line, 
     substring(d.value,1,16) as value,
     substring(le.err_reason,1,48) as err_reason
from stl_loaderror_detail d, stl_load_errors le
where d.query = le.query and d.query = pg_last_copy_id(); 

https://docs.aws.amazon.com/redshift/latest/dg/r_STL_LOAD_ERRORS.html

Anonymous
Not applicable
Author

Hey Vapukov Thank you for your reply.

 

But,i am able to run same copy command from Redshift and i am able to load data into redshift from S3. Where as using Talend  i am unable to load data into redhsift,throwing error like : [Amazon](500310) Invalid operation: insufficient data left in message;

 

Could you suggest me on this.

 

Regards,

Prasanth

vapukov
Master II
Master II

same data? 0683p000009MA9p.png

Same == prepared by talend, transferred to S3 and then imported by Redshift? or it "similar" data? (which You trust look same ... but we not trust nobody 0683p000009MACn.png )

 

 

did You run command from links? what it show?

Anonymous
Not applicable
Author

Yes Vapukov.

Same data i used.When i am using copy command which is ran successfully into redshift.while i am trying to load same data using talend,some how it throwing an error on talend.

source file delimiter is '~' .

Also,please find below copy command which i used for redshift load.

 

copy xxxx.xxxxxxxxxxxxxxxxxxxxxxx
from 's3://xxxxxxxxxxxxxx/xxxx.txt'
iam_role 'xxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxx'
ESCAPE ACCEPTANYDATE ACCEPTINVCHARS EMPTYASNULL FILLRECORD IGNOREBLANKLINES IGNOREHEADER 1 TRIMBLANKS dateformat'auto' NULL AS '\0' delimiter '~'

 

Please help me how to run this job through talend.

Thank you.

 

Regards,

Prasanth

 

 

 

 

 

Anonymous
Not applicable
Author

Thank you for ypur reply.

 

i Just changed group.

 

Regards,

Prasanth

Anonymous
Not applicable
Author

Hi,

 

While i am loading data from s3 bucket to redshift using copy command its throwing error as : Extra column(s) found .

 

Could please help me on this.

 

Thank you.