Please can anyone help me with the error handling in scripts.
I have 5 data load scripts. If one script is encountered with error i want to skip th error and execute the remaining 4 scripts.I am using gcp bigquery as a datasource. Below is the logic i tried. it is working partially. For example, if have an error in 4th script till 3rd data is loading and stopped loading at the 4th script. even though there is no error in 5th script it is also considered as error. I want to skip the 4th and jump on to execute the 5th script
SET ErrorMode = 0; // Allow script to continue on error
[Table]: SELECT * FROM `XXXX`;
IF ScriptErrorCount > 0 THEN TRACE ⚠️ Table load failed. Skipping...; // DROP TABLE [Table]; ENDIF
Thanks Mark. Tried the same but still data loads till the error is encountered and exiting the script. eventhough 5th script is vaild it is also failed along with the error
Like hinted above try to investigate which errors are occurring because like the help for ScriptError indicates only certain (the most common ones) are fetched from the ErrorMode and others will further cause an unmanageable exception. Further the knowledge of the possible errors enables the possibilities to develop alternative approaches, for example to prevent the errors in beforehand instead of ignoring them afterwards.
Beside the above the hint to be aware that other script-parts on the outside and/or in between of the ErrorMode logic may have an impact respectively causing side-effects, for example if these loads doesn't create new tables else being (auto-)concatenated to resident-tables and/or any load-prefixes are applied (join/keep/crosstable/...) and/or any ALIAS/QUALIFYING/... or similar stuff. Means these script-parts are interacting with other script-measurements on the outside. A simple check would be to create a complete new app and applying only the first load, then the second one and so on - without doing anything else.
All the 5 scripts are independent to each other. Straight away extracting the 5 different tables and not used any joins,concate etc. simply loading the tables from bigquery. some of the frequent errors we face everyday while re-loading the app are operation timeout-polling the job,max retries and rare cases if there are any changes in the already available column name(field mismatch or naming convention) from source.
I think it means the errors are mostly on the outside from the load and don't return a (in Qlik) defined error-code. I assume there are not many measurements possible within the ErrorMode to prevent these errors.
You may look if the connector could be adjusted/configured to return a defined error-code if the work-load on the source and/or the network is too high or enabling a data-transfer within intervals or at least any intermediates state-message (I'm alive) to prevent the timeout.
Helpful may also to split the queries and/or to implement incremental approaches to reduce the data-sets and/or the complexity of the queries.
Much more pragmatic would be not to chain n loads within a single app else to use n independent apps which provides also benefits in regard to different times and frequency to pull the data.
