Qlik Community

New to Qlik Sense

Discussion board where members can get started with Qlik Sense.

Announcements
QlikWorld 2020: Join us May 11 - 14, 2020 in Phoenix, AZ. Register early and save $400. Learn More
dmg17188
New Contributor II

Load multiple csv files into one file

I searched for and tried two suggestions below for loading multiple CSV files at once into a larger single file, but not having success. 


If someone can point out what I'm missing I would really appreciate it!

Attempt #1:

vPath = 'lib://AttachedFiles';

For each file in FileList('$(vPath)\example_data*.csv');

example_data_1234:

Load * FROM $(File)[lib://AttachedFiles/example_data*.csv]

(txt, codepage is 28591, embedded labels, delimiter is ',', msq);


Outcome:  The load runs, but Qlik Sense is creating a synthetic key using all the fields instead of making one table and appending with all the following files.  I expected it just be one table and not need a key of any kind.  Is there a way to avoid the synthetic key or does it matter?


Attempt #2

example_data_0398:


LOAD *

FROM [lib://AttachedFiles/example_DATA_APR1_15.csv];

Concatenate (example_data_0398);

LOAD *

FROM [lib://AttachedFiles/example_DATA_APR16_30.csv]

(txt, codepage is 28591, embedded labels, delimiter is ',', msq);


Outcome: I'm getting an error that says "Unknown statement: Concatenate (example_data_0398)"


Tags (3)
1 Solution

Accepted Solutions
MVP & Luminary
MVP & Luminary

Re: Load multiple csv files into one file

If you want to handle it with a load *, that is, you want to pick all the fields in each file even if they are different:

Data:

LOAD 0 as dummyfield AutoGenerate 0;


Concatenate (Data)

Load * FROM [lib://AttachedFiles/example_data*.csv]

(txt, codepage is 28591, embedded labels, delimiter is ',', msq);


DROP FIELD dummyfield;


Note there is no For Loop, just a wildcard in the filename.


-Rob

http://masterssummit.com

http://qlikviewcookbook.com

View solution in original post

7 Replies
ogster1974
Honored Contributor II

Re: Load multiple csv files into one file

Include the file name in your new table to avoid synthetic keys. Also rather than * I would identify the fields to load explicitly so you know the files are using the same file structure.

Partner
Partner

Re: Load multiple csv files into one file

delete the ; after the concatenate

dmg17188
New Contributor II

Re: Load multiple csv files into one file

Hi, Andy!

I'm sorry, I'm not sure what you mean by "Include the file name in your new table to avoid synthetic keys".  Would you please provide an example?


Thank you!

dmg17188
New Contributor II

Re: Load multiple csv files into one file

I knew it would be something simple like that and it did work!  Thank you!!

I'm interested to see how to make the other option work too so I'll have both options available.

MVP & Luminary
MVP & Luminary

Re: Load multiple csv files into one file

If you want to handle it with a load *, that is, you want to pick all the fields in each file even if they are different:

Data:

LOAD 0 as dummyfield AutoGenerate 0;


Concatenate (Data)

Load * FROM [lib://AttachedFiles/example_data*.csv]

(txt, codepage is 28591, embedded labels, delimiter is ',', msq);


DROP FIELD dummyfield;


Note there is no For Loop, just a wildcard in the filename.


-Rob

http://masterssummit.com

http://qlikviewcookbook.com

View solution in original post

dmg17188
New Contributor II

Re: Load multiple csv files into one file

Ahh, perfect.  Thank you for the help, Everyone!

gdisalvo2017
New Contributor

Re: Load multiple csv files into one file

Thank you. This worked fine except it gave an error for the last statement -> DROP FIELD dummyfield;

Once removed it loaded perfect.