I am currently trying to bulk load into a Snowflake table that includes an Auto Increment column. The column is the 1st column (non-null) in the table which is 8 columns in totalI am then creating a load file that is 6 fields long (relating to columns 2-7).
The file is then being put into an internal stage folder.
Note: I am also using the additional manual copy command option of 'error_on_column_count_mismatch=false' (as well as the Talend default ones) to get around the column count mismatch error (file columns vs table columns)
Unfortunately the Talend component does not appear to give me the ability to utilize the Snowflake workaround to copy into select columns
I.E. the Talend command being generated is this ...
copy into "NA_ADM_TXN" from '@~/ians_temp_int_stage/entitlement/' ON_ERROR='continue' FILE_FORMAT=(type=csv field_delimiter=',' error_on_column_count_mismatch=false compression=gzip field_optionally_enclosed_by='"')
But the only way to get the Talend generated file to load (via manually running this command in Snowflake) is this ...
copy into "NA_ADM_TXN" (VIEW_NAME, ROW_ID, OPERATION, TXN_TIMESTAMP, MODEL_MAJOR_VERSION, MODEL_MINOR_VERSION) from ( select $1, $2, $3, $4, $5, $6 from '@~/ians_temp_int_stage/entitlement/')
ON_ERROR='continue' FILE_FORMAT=(type=csv field_delimiter=',' error_on_column_count_mismatch=false compression=gzip field_optionally_enclosed_by='"')
It is mentioned here:
https://docs.snowflake.com/en/user-guide/data-load-transform.html# include-autoincrement-identity-columns-in-loaded-data
Any help would be appreciated on how to accomplish this using Talend components etc.
Thanks
