Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
hello:
I noticed that the snowflake tDBOutput always creates a temp table them loads the data in snowflake.
My benchmarks show faster performance and less credit usage NOT using a temp table.
Is there a way to over-ride this behavior and have it forgo creating a temp table and have it load directly to my target table?
Is there a reason for the loading temp table step (just for error handling?) ?
thanks!
I have confirmed that it is the Snowflake API that is doing this and the output component has no control over this. Thank you for investigating. I will look at the API Spec, & may end up contacting Snowflake to see if this behavior can be controlled thru the API. Until then, I will control this behavior by creating a joblet without the component.
Thanks for the help.
@mhodent ,when you were trying to load data into main table,is it creating a first temp table and loading to temp first and then loading to main table ?
@mhodent ,can you post your job design ?
attached - thanks. the tdboutput components create the temp tables using native snowflake commands after the put.
@mhodent ,the attachment is export of the code ? if yes i am getting error like There is no valid items to import, and make sure the selected path contain valid items.
no, that was a export > generate HTML Doc.. unzip and look at the design.
the problem is that the tdboutput generates a temp table as an intermediary step (on snowflake instance), and I would like to bypass creating the temp table & have the output just do a put, then a load directly to my target table
thanks
@mhodent ,useally we will not specify any temp table in talend job,may be that will create in snowflake and need to check the snowflake settings? can you show me where it was creating?
when I use the tdboutput to push the file into snowflake, I can see that a temp table is created for before the insert statement when looking at query history within snowflake history interface.
thanks
@mhodent ,it may be work in that way at snowflake DB side,we can not do in Talend side. need to verify at snowflake settings.