Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
We are loading data to Qlik PRD, The throughput was decreased suddenly. Please help to check what is happening and how to resolve and speed up
We are doing the mid-size data (1-10 Billion) . At the very beginning of our loading the data throughput are around 20K-30K/s. . I was closely monitor the speed and assume the speed will increase as the fewer and fewer parallel runs. But things are going in the opposite direction, with fewer and fewer tables running at the same time, and throughput is getting lower and lower
We reload the filter 4 tables and the issue seems comes again. Around 6:28PM BJT the throughput was about 30K, now it drop suddenly under 10K. Here is the new one.
the diagnostics package and Trend graphs as follow。
Hi there. You provide some great data for folks trying to help, but at the same time you don't.
I see no easy graphs. I see diagnostics zip file with the task json (great) and some logs but no indication of where to look. So I just picked one log: "HP_LS_Z_MASTER-KafkaEHP__220129153631.log" and that seems fine or at least nothing obvious to worry about. 10 streams parallel, done in a minute, one large table defining most of the elapsed time (Z_MASTER.TCURR). Total 2571802 rows in 75 tables. 00h01m22s Elapsed. 31363 Records/second
No notion of billions of rows, no suggestion of how to compare subtask selection settings.
We are not mind readers. Assume we know nothing about your deployment - because we don't - and re-read your posting with that mindset. How can one possibly help? Please help us help you!
Cheers,
Hein