Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Only at Qlik Connect! Guest keynote Jesse Cole shares his secrets for daring to be different. Learn More!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Load multiple files with some missing some colums

I don' t seem to find the answer to this.

I want to load multiple CSV files, in which the columns are named, but some of them miss some columns

Ex.

file 1 to n have columns (a,b,c,d)

file n+1 to p have columns (a,b,c,d,e)

if my load scripts tries to read column 'e' I get an error for files 1 to n.

How can I prevent this?

Thanks in advance

12 Replies
simenkg
Specialist
Specialist

Table:

LOAD * FROM

[.\Source\*.csv]

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);


QlikView should do an automatic concatenate and the missing fields will be populated with Nulls().

If none of the files contain field e, but you use field e later in the script, then you could do something like this:

Table:
Load null() as a,

        null() as b, 

        null() as c,

        null() as d,

        null() as e

autogenerate 0;

concatenate(Table)

LOAD * FROM

[.\Source\*.csv]

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

Anonymous
Not applicable
Author

OK this allows me to load everthing correctly in the table "table", but as I do some transformation at load time, I should do that in two steps I guess like this :

//First solution provided by maxgro to load in temp table

set concat='';

for Each file in FileList('*.csv')

  table:

  $(concat)

  load    *,   '$(file)' as filename

  from $(file);

  set concat=' concatenate (table) ';

NEXT;


//Then do my transformations

noconcatenate

realtable:

load

     a as [field a],

     b as [field b],

     (if c=1, 'OK', 'KO') as [field c], //stupid transformation example

     d as [field d],

     if(isnull(e), 'No field e', e) as [field e] //Works now

resident table;

drop table table;

Anonymous
Not applicable
Author

Implemented maxgro solution (see answer above) and it works.

qlikviewwizard is also right though, as you point it "There is no way inside from a load to check if fields are existing or not"