Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Is there a situation when 2 identical tables don't concatenate?
I am building a table from external sources which allow only 100 records to be extracted at a time. So I am using a loop to extract batches of 100 at a time into a RESULTS table.
Of course each batch has identical fields, but I'm getting many tables called RESULTS-n, joined by a synthetic table.
Advice please on what is stopping me getting a single table?
I am having a feeling that somehow there is either a noconcatenate in the script or you have additional fieldnames which gets dropped at a later point... might be easier to point out by looking at the script
Hi Stefan
Yes I am. Might this cause an issue ?
Yes, could break the expected auto-concatenation.
As already suggested, could you please post your relevant script part?
It will be quite a bit of work for me to share the code.
So I will use your hint to try to re-organise how I load the data and see if I can avoid the issue first.
Thanks
Certain script structures such as preceeding load or joins will "break" auto concatenation. You can usually fix this by using the Concatenate keyword to force concatenation.
-Rob
Thanks Stefan (and Sunny)
Based on your hints on where to look, I think I have found the issue: I was dropping fields within the loop, which I guess would confuse the AUTOconcatenate.
Hi Rob,
do you know if there is any official material when you can find more details regarding how auto concatenation may be affected by preceding loads, joins, etc. ?
Cheers
Piotr
I get that when I use a path and loop, if I also add NoConcatenate...
vPath = '\\qv_user_files\'
For each File in FileList('$(vPath)*_My_Data_Files.csv')
First 100
data:
NoConcatenate
LOAD
Field1,
Field2,
Field3
From $(File);
NEXT
It works perfectly if I do not.