topic Re: Copy into running fine but not loading data gzip file in Talend Studio

Copy into running fine but not loading data gzip file

rommy — Sat, 16 Nov 2024 01:45:53 GMT

Hi All,

I am using talend bulk execution to load data from s3 to snowflake. Talend tFileArchive converts the file to gzip format , file.csv.gz and upload it to s3 bucket. Copy into which gets executed through talend bulk component looks like below. It does nt throw an error or something but does nt load data either. If I try to load csv file without zip , it works fine.

File: file.csv.gz

Copy into table

from 's3://bucket/'

credentials=(aws_key_id='' aws_secret_key='')

FILE_FORMAT=(type=csv compression=gzip field_delimeter=',' skip_header=1 field_optionally_enclosed_by='\"' empty_field_as_null=true)

force=true

Can someone point wheres the issue ? Even if I execute above command through snowflake UI , it says ran successfully but does not load. File has data.

if I keep csv file to same s3 bucket location, it picks up csv file with out an issue and loads it , it also picks csv.gz file but does bt load data , row_parsed =0 shows there,

Thank you

Re: Copy into running fine but not loading data gzip file

rommy — Fri, 31 Jul 2020 17:36:28 GMT

After researching we came to know that gzip files are empty , I am using tFileArchive component to gzip the file.

Options are below:

SourceFile:"file.csv"

DestinationFile:"file.csv.gz"

ArchiveMethod:gzip

compression level : normal ,

Can someone point me why file is gziping blank ?

Re: Copy into running fine but not loading data gzip file

Anonymous — Mon, 03 Aug 2020 05:55:17 GMT

Hello,

Is there any error message printed on console? We will appreciate it a lot if you could post your job design screenshots here.

Best regards

Sabrina

Re: Copy into running fine but not loading data gzip file

rommy — Mon, 17 Aug 2020 19:22:47 GMT

Sorry for being late, there was an issue with logic hence it was creating zip file with header only, hence it was not loading any information , Thank you for getting back to me ,