Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
See why IDC MarketScape names Qlik a 2025 Leader! Read more
cancel
Showing results for 
Search instead for 
Did you mean: 
xx_emanis
Contributor
Contributor

Loading multiple CSV files to one CSV file and then to Datawarehouse table

Hi All,

I have a requirement where I need to load 10 CSV files in to one CSV file and from that one CSV file I need to load the table in Datawarehouse.

All the 10 CSV files have the same column structure and will not be changing.

 

Can you please let me know the job flow of how to achieve this.

Thank You.

Labels (3)
1 Solution

Accepted Solutions
TRF
Champion II
Champion II

3 main steps ti build the output file:
- tFileList to iterate over the 10 input files
- tFileInputDelimited to read the content of the current file using tFileList_1_CURRENT_FILEPATH to get the full pathname
- tFileOutputDelimited with append mode to generate the output file

Then you can get this file as the input to load data into your DWH. Details depend on the engine you are using for this DWH.

View solution in original post

8 Replies
TRF
Champion II
Champion II

3 main steps ti build the output file:
- tFileList to iterate over the 10 input files
- tFileInputDelimited to read the content of the current file using tFileList_1_CURRENT_FILEPATH to get the full pathname
- tFileOutputDelimited with append mode to generate the output file

Then you can get this file as the input to load data into your DWH. Details depend on the engine you are using for this DWH.
xx_emanis
Contributor
Contributor
Author

Thanks for the quick reply.

Do I need to create 10 different metadata for the 10 different CSV I have?

TRF
Champion II
Champion II

Of course not as soon as all files have the same structure
xx_emanis
Contributor
Contributor
Author

Thanks TRF I was able to acheive my requirement based on the flow you provided.

 

The other thing I was asked today was to create a LOAD_DATE column in the output CSV file and the load date should be based on the date which is a part of the name of the source CSV files.

 

For Instance name of one of the source csv files is All_VM_Cluster_Stats_2018-11-04.

 

When the data from this file is loaded to the target output csv file, LOAD_DATE column should be populated with '2018-11-04'

 

Please explain how to achieve this.

TRF
Champion II
Champion II

New question, thank's to open à new topic
xx_emanis
Contributor
Contributor
Author

sorry dint get you..Do you want me to open a new topic for this question?
TRF
Champion II
Champion II

Yes, it's better to have just 1 question per topic
Anonymous
Not applicable

- tFileList to iterate over the 10 input files
- tFileInputDelimited to read the content of the current file using https://fetlife.vip/ https://downloader.vip/imvu/ https://downloader.vip/canva/