Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Streamlining user types in Qlik Cloud capacity-based subscriptions: Read the Details
cancel
Showing results for 
Search instead for 
Did you mean: 
shidong2
Contributor
Contributor

Qlik Replicate Throughput Issue

We are loading data to Qlik PRD, The throughput was decreased suddenly. Please help to check what is happening and how to resolve and speed up 

 

 

 

  • During Jan 30th-31th (1st wave of 1-10billion data tables)

 

We are doing the mid-size data (1-10 Billion) . At the very beginning of our loading the data throughput  are around 20K-30K/s. .  I was closely monitor the speed and assume the speed will increase as the fewer and fewer parallel runs. But things are going in the opposite direction, with fewer and fewer tables running at the same time, and throughput is getting lower and lower

 

 

 

 

 

  • Feb 2nd (2nd wave of 3-10billion data tables)

    We reload the filter 4 tables and the issue seems comes again. Around 6:28PM BJT the throughput was about 30K, now it drop suddenly under 10K.  Here is the new one.

 

 

 

Labels (1)
2 Replies
zhangsq24
Contributor II
Contributor II

the diagnostics package and Trend graphs as follow。

Heinvandenheuvel
Specialist III
Specialist III

Hi there. You provide some great data for folks trying to help, but at the same time you don't.

I see no easy graphs. I see diagnostics zip file with the task json (great) and some logs but no indication of where to look. So I just picked one log: "HP_LS_Z_MASTER-KafkaEHP__220129153631.log" and that seems fine or at least nothing obvious to worry about. 10 streams parallel, done in a minute, one large table defining most of the elapsed time (Z_MASTER.TCURR). Total 2571802 rows in 75 tables. 00h01m22s Elapsed. 31363 Records/second

No notion of billions of rows, no suggestion of how to compare subtask selection settings.

We are not mind readers. Assume we know nothing about your deployment - because we don't - and re-read your posting with that mindset. How can one possibly help? Please help us help you!

Cheers,

Hein