for some weeks now I dived into QDF and am amazed: I think it is a great solution! Right now I am trying to find a suitable QVD-Strategy. After lots of redesign, the picture below shows my current structure:
In order to enable a proper ETL-process, I created the subfolders 1.Extract and 2.Transform for 2.QVD of every container. I also modified the Example QVD-Generators so that they atomatically create one more level of subfolders like "1.Stammdaten" (Master Data in German): These subfolders are meant to group data within one sphere of application together.
The process goes like this;
1) Start 1st QVD-Generator in 1.Application\1.Stammdaten and extract e.g. data from 8.Import\1.Stammdaten
--> Create raw QVD-files in 2.QVD\1.Extract\1.Stammdaten
2) Start 2nd QVD-Generator in 1.Application\1.Stammdaten and transform these raw QVDs from 2.QVD\1.Extract\1.Stammdaten
--> Create transformed QVDs in 2.QVD\1.Transform\1.Stammdaten
3) Start the Mart under 4.Mart\1.Stammdaten
--> Load all transformed QVDs from 2.QVD\1.Transform\1.Stammdaten to build the data model
4) Start the final GUI under 1. Application\1.Stammdaten
--> Do a binary load on the Mart under 4.Mart\1.Stammdaten --> The App is ready for use!
My question now is: Are those subfolders 1.Extract and 2.Transform under 2.QVD really necessary? The QDF Development Guide mentions that you should use subfolders under 2.QVD, but maybe it is meant like this:
The advantage of this model is in my opinion that you do not need the 1.Extract and 2.Transform folder in cases when no transformation is necessary. Then all data files, QVWs and QVDs are just grouped in the project-specific subfolder like "1.Stammdaten" in this case. The QDF Development Guide also mentions that the Index Monitor is the key to locate and manage all QVDs on different levels. But if all QVDs reside under 2.QVD\subfolder, how do you differentiate between raw and X-Form QVDs and that they do not overwrite each other? Do you suggest a naming convention like TableName_EXT and TableName_TRA? I want to make sure that there is enough flexibility for self-service users to create apps without exhausting staging and a long process for non-critical Apps, but I also want to be able to follow this process for important apps where a lot of data cleansing and modeling is necessary.
Thank you in advance!