tAzureSqlDWBulkExec Not Working with gzip files

[Anonymous]

tAzureSqlDWBulkExec has been incredibly helpful with my Azure DW migration, but I cannot get the component to work with a gzip file. I am using tFileArchive to create the file in a gzip format. My file is around 35 gb so the compression would be really helpful. This is the error that I am seeing.

Exception in component tAzureSqlDWBulkExec_1 
com.microsoft.sqlserver.jdbc.SQLServerException: Invalid option ‘null’ specified while creating DelimitedText file format. 
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1535) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:845) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:752) 
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151) 
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:219) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:199) 
at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStatement.java:729) 

I am not seeing any issues with the compression turned off.

Here is the link to the component in the Talend Exchange:  https://exchange.talend.com/#marketplaceproductoverview:marketplace=marketplace%252F1&p=marketplace%...

[Anonymous]

@dsoulalioux

have you tried with a smaller size gzip file?

Usman

[Anonymous]

Yeah, I added (top 100) to my sql query.  It was not working even with that very small file.

[Anonymous]

what is the error you are getting when using the GZIP option?

[Anonymous]

The error I am getting is:

com.microsoft.sqlserver.jdbc.SQLServerException: Invalid option ‘null’ specified while creating DelimitedText file format. 

I have seen this error before when the file does not exist, but the gzipped file is being placed in azure storage and the file name matches the name of the file in the  tAzureSqlDWBulkExec component.  

@uzubair

[Anonymous · Accepted solution]

@dsoulalioux

I have updated the connector.  There was an issue with the compression option.  Please check the exchange for version 1.1.

I plan to add ADLS support in the coming days with version 2.0.

Usman

[Anonymous]

Fantastic! Thank you!

[Anonymous]

Do you know when this change will be live?  I have been checking once an hour or so and it is still version 1.0.   I am not sure if there is an approval process with the talend exchange.

[Anonymous]

@dsoulalioux

i have submitted it for approval...

attaching it for reference as well

tAzureSqlDWBulkExec.zip