Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
See why IDC MarketScape names Qlik a 2025 Leader! Read more
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Load multiple files with some missing some colums

I don' t seem to find the answer to this.

I want to load multiple CSV files, in which the columns are named, but some of them miss some columns

Ex.

file 1 to n have columns (a,b,c,d)

file n+1 to p have columns (a,b,c,d,e)

if my load scripts tries to read column 'e' I get an error for files 1 to n.

How can I prevent this?

Thanks in advance

12 Replies
simenkg
Specialist
Specialist

Table:

LOAD * FROM

[.\Source\*.csv]

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);


QlikView should do an automatic concatenate and the missing fields will be populated with Nulls().

If none of the files contain field e, but you use field e later in the script, then you could do something like this:

Table:
Load null() as a,

        null() as b, 

        null() as c,

        null() as d,

        null() as e

autogenerate 0;

concatenate(Table)

LOAD * FROM

[.\Source\*.csv]

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

Anonymous
Not applicable
Author

OK this allows me to load everthing correctly in the table "table", but as I do some transformation at load time, I should do that in two steps I guess like this :

//First solution provided by maxgro to load in temp table

set concat='';

for Each file in FileList('*.csv')

  table:

  $(concat)

  load    *,   '$(file)' as filename

  from $(file);

  set concat=' concatenate (table) ';

NEXT;


//Then do my transformations

noconcatenate

realtable:

load

     a as [field a],

     b as [field b],

     (if c=1, 'OK', 'KO') as [field c], //stupid transformation example

     d as [field d],

     if(isnull(e), 'No field e', e) as [field e] //Works now

resident table;

drop table table;

Anonymous
Not applicable
Author

Implemented maxgro solution (see answer above) and it works.

qlikviewwizard is also right though, as you point it "There is no way inside from a load to check if fields are existing or not"