Best practice for large QVD development

[#1]

Hello to all in the Qlik Community!

My company is currently looking into QlikView.  We've been running SAP for about 5 years, so our finance tables have grown fairly large.  We have extracted such tables as BSEG, FAGLFLEXA and BKPF.  The QVDs are partitioned by year and even just one year for BSEG is 8GB.

The thought right now is that we would give access to the QVD files and data models via a file share.  What if the QV superusers (the ones building apps and data models) wanted to create a new data model using QVDs?  What's the best practice for them to use those QVDs to build a data model?  They would either have to have a pretty beefy machine or develop directly on the QV server.

I apologize if I haven't explained well enough.  If I haven't please ask some questions so I can elaborate.

Thanks!!!

[Peter_Cammaert]

Some quick thoughts:

• Buy two QlikView machines. One will be production (full license), the other will be an acceptance/test system (test license = cheaper)
• Never grant your developers access to the production machine. That one should be kept atlive at all costs. Developers can do things in test applications that eat every byte of RAM or run off with all CPU cycles..
• Implement an ODS-QDS-QVW data flow or a variation thereof. ODS makes a copy of SAP tables and uses incremental LOADing where needed (BSEG, CE1000, GLPCA, MSEG, Sales tables like VBAP/VBAK/...)
• Determine when you will update the ODS layer. If you run the big sync at night, then there will probably be time left to copy the synced ODS files on production to acceptance/development. If not, consider placing the QVDs on a high-speed NAS or file server.
• Make yourself member of the QlikView Deployment Framework group here on the Community. There are valuable documents available about rolling out QlikView development in an enterprise environment.

Peter

[#3]

Thanks for the thoughts Peter.

• Right now (since we're evaluating) we've only purchased the full production license.  We have also had the thought of having a test machine.  The problem we have with that is we're limited to only 2 RDP connections to a server.
• I completely agree.  We would only give access to the "test" server.
• This is the model we've taken and we've done exactly that.
• We update the tables at night.
• I'll join that group

The issue that we saw was a super user (an IT person) was trying to consume a QVD that was 8GB and couldn't handle it on his PC. I wasn't sure how others have dealt with the "LARGE QVD" issue.

[Peter_Cammaert]

Well, you could add RAM to developer's laptops (32GB max but only for some high-end machines I think).

However, development in QlikView is cyclic: develop - load - check - correct/develop again etc. For making that cycle as short as possible, more RAM will not really help much. IMHO during development you better reduce the amount of data to load to, say 1 yr or even 1 mo. You can use variables and date fields for that in your scripts. Another trick is to aggregate as much as possible in the QDS layer.

I myself use a SAP test system wherever available. Often has less bulky transaction tables, and you're allowed to create specific test situations without compromising your business operations.

Peter