Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I have a number of (thousands) files to load into a master table.
These files are very similar in nature, though can have differing field names.
for example:
File1:
field_a, field_b, field_b1, field_b2, field_c
File2:
field_a, field_b, field_b1, field_b2, field_b3, field_d
etc.
My intent is to have one table with all possible fields - concatenating the same fields, joining any new ones.
I'm assuming a left join would be in order? Something like:
For each file in filelist('lib://mylib\*.csv')
MasterTable:
Load
*
From [$(file)];
Join(MasterTable)
Load
*
From[$(file)];
Next file
This would work for the 2nd (and subsequent) files, but obviously not the first.
Is there an autogenerate or counter that I could employ maybe?
....
you would need to see if in your table 1 the data matches your table2 so you can add a left join.
since table 2 will be joined with table 1 and if the data in table 2 do not match those in table 1, your data will be shown as null if the data does not coincide and otherwise the data will show the data that coincide.
Table 1:
load * from tab1;
left join (Table 1)
load * from tab2;
I think it would be more correct to concatenate something like this:
Table 1:
load * from tab1;
Concatenate (Table 1)
load * from tab2;
I hope this clarifies your doubt
Regards!
I don't think you want Join. You want Concatenate, which appends rows to the an existing table.
Concatenate (MasterTable)
Load
*
From[$(file)];