topic Re: tAzureSqlDWBulkExec Not Working with gzip files in Talend Studio

tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Sat, 16 Nov 2024 09:28:14 GMT

tAzureSqlDWBulkExec has been incredibly helpful with my Azure DW migration, but I cannot get the component to work with a gzip file. I am using tFileArchive to create the file in a gzip format. My file is around 35 gb so the compression would be really helpful. This is the error that I am seeing.

Exception in component tAzureSqlDWBulkExec_1
com.microsoft.sqlserver.jdbc.SQLServerException: Invalid option ‘null’ specified while creating DelimitedText file format.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1535)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:845)
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:752)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:219)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:199)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStatement.java:729)

I am not seeing any issues with the compression turned off.

Here is the link to the component in the Talend Exchange: https://exchange.talend.com/#marketplaceproductoverview:marketplace=marketplace%252F1&p=marketplace%252F1%252Fproducts%252F1507&pi=marketplace%252F1%252Fproducts%252F1507%252Fitems%252F2604

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Mon, 31 Jul 2017 19:48:42 GMT

@dsoulalioux

have you tried with a smaller size gzip file?

Usman

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Mon, 31 Jul 2017 19:50:04 GMT

Yeah, I added (top 100) to my sql query. It was not working even with that very small file.

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Mon, 31 Jul 2017 19:54:43 GMT

what is the error you are getting when using the GZIP option?

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Mon, 31 Jul 2017 20:40:35 GMT

The error I am getting is:

com.microsoft.sqlserver.jdbc.SQLServerException: Invalid option ‘null’ specified while creating DelimitedText file format.

I have seen this error before when the file does not exist, but the gzipped file is being placed in azure storage and the file name matches the name of the file in the tAzureSqlDWBulkExec component.

@uzubair

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Mon, 31 Jul 2017 21:43:37 GMT

@dsoulalioux

I have updated the connector. There was an issue with the compression option. Please check the exchange for version 1.1.

I plan to add ADLS support in the coming days with version 2.0.

Usman

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Mon, 31 Jul 2017 21:45:46 GMT

Fantastic! Thank you!

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Tue, 01 Aug 2017 01:52:12 GMT

Do you know when this change will be live? I have been checking once an hour or so and it is still version 1.0. I am not sure if there is an approval process with the talend exchange.

Re: tAzureSqlDWBulkExec Not Working with gzip files

Anonymous — Tue, 01 Aug 2017 01:55:25 GMT

@dsoulalioux

i have submitted it for approval...

attaching it for reference as well

tAzureSqlDWBulkExec.zip