Qlik Community

Ask a Question

QlikView App Dev

Discussion Board for collaboration related to QlikView App Development.

Announcements
QlikView forum consolidation is complete. Labels are now required. LEARN ABOUT LABELS
cancel
Showing results for 
Search instead for 
Did you mean: 
markodonovan
Specialist
Specialist

Single QVD Loader or Multiple loaders

Hi there,

I have come across a question that I would be very interested in hearing other qlikviewers opinions on.

The questions is to do we creating qlikview loaders.

If you were creating a loader that was going to be reading 100+ tables from 1 system would you put the code that would generate the initial source and transformed qvds into 1 qvw file ?

Each table would have a separate tab in the loadscript.

Have you any loadscripts with more than 100 tabs?

My gut instinct is to try and separate the scripts logically in some way but different people have different ideas.

What is your opinion ?

Thanks

Mark

1 Solution

Accepted Solutions
marcus_sommer
MVP & Luminary
MVP & Luminary

I would split these loads into several qvw's then the handling and the overwiew with so many loads and script-tabs is possible but surely it isn't really good. Further qv will be significant slower while editing, scrolling and so on in large and complex load-statements.

Also with several qvw's you could parallelize these loads and updating as well.

- Marcus

View solution in original post

7 Replies
marcus_sommer
MVP & Luminary
MVP & Luminary

I would split these loads into several qvw's then the handling and the overwiew with so many loads and script-tabs is possible but surely it isn't really good. Further qv will be significant slower while editing, scrolling and so on in large and complex load-statements.

Also with several qvw's you could parallelize these loads and updating as well.

- Marcus

View solution in original post

giakoum
Partner
Partner

I guess that when reading 100+ tables, they all or most of them contain fact that you concatenate into one large fact table. So I guess that many of these tables have the same fields. If yes, then I would not create 100 tabs, I would create one tab with code that iterates over the tables and loads them. Something like a for loop :

For each table in tables

     [$(table)]:

     load fields from $(table);

     store [$(table)] into $(table).qvd (qvd):

next

markodonovan
Specialist
Specialist

Hi Ioannis,

The script in each tab is currently doing an incremental qvd load.

The tables are being kept separate and all the fields are different in each table.

I nice idea though.

Thanks

Mark

giakoum
Partner
Partner

Is it that the names of the fields are different but they contain the same data?

For example customerID in Table1 is called FieldA and in table2 it is called CustID?

markodonovan
Specialist
Specialist

Hi Ioannis,

Thanks for the reply.

All the tables have different data.  I wish it was that simple.

My main worry was a code bloat scenario where there was a desire to put the code for all the source qvd loads and transformed qvds in 1 qvw document. 

The system we are getting the data from is quite large and having 100 tabs would easily be possible.

It is interested to hear if anyone else encounters such problems.

I appreciate the fast reply to this question.

Thanks

Mark

giakoum
Partner
Partner

I see.

Then I would suggest you use an Excel file to keep tables and their respective fields and use a for loop to iterate through them, load and store the data in qvds:

Excel sample:

Table, Field

Table1, Field1

Table1, Field2

Table1, Field3

Table2, FieldA

Table2, FieldB

etc.

So instead of maintaining 100 tabs, you maintain one large excel sheet and one tab in QV

Not applicable

Can you elaborate on this approach? I'm in the process of deciding this myself, and like this idea, but am not sure exactly how to implement it. I'm worried about adding multiple tabs vs also tracking multiple qvd generator qvws. My one qvd generator right now is a bit slow with a little over 6M rows...