Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Talend Redshiftrow Error

Hi ,

 

Can anybody help me,while i am running talend job with copy command its throwing error like.

insufficient data left in message during my COPY from S3 to Redshift.

 

Can anybody help me.

 

 

Regards,

Prasanth

 

Labels (1)
  • Other

6 Replies
vapukov
Master II
Master II

It is not a Talend error message
it is Redshift (Postgres) 

You can check,  what happens exactly by select data from sql_load_errors:

select 
     d.query, 
     substring(d.filename,14,20), 
     d.line_number as line, 
     substring(d.value,1,16) as value,
     substring(le.err_reason,1,48) as err_reason
from stl_loaderror_detail d, stl_load_errors le
where d.query = le.query and d.query = pg_last_copy_id(); 

https://docs.aws.amazon.com/redshift/latest/dg/r_STL_LOAD_ERRORS.html

Anonymous
Not applicable
Author

Hey Vapukov Thank you for your reply.

 

But,i am able to run same copy command from Redshift and i am able to load data into redshift from S3. Where as using Talend  i am unable to load data into redhsift,throwing error like : [Amazon](500310) Invalid operation: insufficient data left in message;

 

Could you suggest me on this.

 

Regards,

Prasanth

vapukov
Master II
Master II

same data? 0683p000009MA9p.png

Same == prepared by talend, transferred to S3 and then imported by Redshift? or it "similar" data? (which You trust look same ... but we not trust nobody 0683p000009MACn.png )

 

 

did You run command from links? what it show?

Anonymous
Not applicable
Author

Yes Vapukov.

Same data i used.When i am using copy command which is ran successfully into redshift.while i am trying to load same data using talend,some how it throwing an error on talend.

source file delimiter is '~' .

Also,please find below copy command which i used for redshift load.

 

copy xxxx.xxxxxxxxxxxxxxxxxxxxxxx
from 's3://xxxxxxxxxxxxxx/xxxx.txt'
iam_role 'xxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxx'
ESCAPE ACCEPTANYDATE ACCEPTINVCHARS EMPTYASNULL FILLRECORD IGNOREBLANKLINES IGNOREHEADER 1 TRIMBLANKS dateformat'auto' NULL AS '\0' delimiter '~'

 

Please help me how to run this job through talend.

Thank you.

 

Regards,

Prasanth

 

 

 

 

 

Anonymous
Not applicable
Author

Thank you for ypur reply.

 

i Just changed group.

 

Regards,

Prasanth

Anonymous
Not applicable
Author

Hi,

 

While i am loading data from s3 bucket to redshift using copy command its throwing error as : Extra column(s) found .

 

Could please help me on this.

 

Thank you.