Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi Team,
I am looking to optimize and automate our current data refresh process for the Products_Query workflow. Currently, the process requires significant manual intervention, and I’d like to move toward a more dynamic, automated solution in QlikView.
Current Workflow (Manual)
Centralized File: We use a single Excel file (Products_Query) with one tab.
Manual Updates: Whenever any of our 150 SQL queries need updating, I manually paste the new query into this sheet.
Manual Union: If multiple queries are updated, I manually perform a SQL Union within the Excel cell/tab.
Load: QlikView reads this consolidated query to fetch sales data from AWS and appends it to our historical QVD.
Proposed Workflow (Automated)
I want to eliminate the manual "copy-paste-union" steps by implementing the following:
Multi-Sheet Configuration: Create an Excel file where each of the 150 products has its own dedicated sheet containing its specific SQL query.
Conditional Execution: On a scheduled daily refresh, QlikView will check if a sheet's SQL text has changed.If changed: QlikView executes that specific SQL against the AWS database and updates the historical QVD. If unchanged: QlikView skips the AWS call for that product to save resources.
I need your help and welcomes alternate ideas also.
Thanks in advance,
This shouldn't be too hard to do, assuming the sheet names are fixed. Separating it into chunks, your algorithm would be something like:
For each tab in tablist
Load current query as text
Check if previous version exists (if it does, we'll have a QVD for it)
If exists, load previous verison and compare them
If identical, stop here and loop to next sheet
Store current query into QVD named as per the sheet name (e.g. Sheet1_Query)
Run query and store result to QVD based on sheet name (e.g. Sheet1_Data)
Drop result.
Loop 150 time;
Full load: Load 150 QVDs (Some will be new, others will be old because nothing changed)
Feeding this into GPT should give you the exact code, so I'm not including it as it's a tad on the long side.
You could also do this by loading data from QVD if identical and from query if not, but I personally prefer this approach. Either way works, though.