Skip to main content

New to Qlik Sense

If you’re new to Qlik Sense, start with this Discussion Board and get up-to-speed quickly.

QlikWorld 2023, a live, in-person thrill ride. Save $300 before February 6: REGISTER NOW!
Showing results for 
Search instead for 
Did you mean: 

ETL Loop with On/Off Switch as Variable

Hello QC!

I need to write a loop for our ETL process in order to aid in development/testing. We'd like to separate out Extract and Transform steps, but we want to keep them in the same app to reduce maintenance. (Side note, if you have thoughts on a better ETL process, please let me know. We're all pretty new to this.)

Because some of our large datasets can take 30-40 minutes to load from a database, we don't want to load data every time we load. The only two methods I'm aware of is Incremental Loading and using a Loop to determine with parts of the script get loaded. And since I'm not sure which is a better practice, I thought I'd come here.


I think I would like for the loop to do the following: Check for a variable (vLightSwitch) and see if it's On or Off. If it's On run our Extract section of script (which stores the raw data into QVDs that are accessed by the Transform phase) then carry on to load the rest of the script. If vLightSwitch is off, skip the Extract portion of script and go straight to Transform. So the Transform and store statements will always run, but the Extract section will only run if the vLightSwitch is turned On.

I'm not familiar with writing loop syntax at all so I don't know the best way to do this.

Thanks for the help!

(and let me know if there's a better way to manage the ETL process)

0 Replies