Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Announcements
Qlik and ServiceNow Partner to Bring Trusted Enterprise Context into AI-Powered Workflows. Learn More!
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable
‎2020-05-23 11:12 AM

efficient way of doing SCD on huge tables

Hi,

 

I'm having a data source table which contains 30 million records. On a given day, there could changes done only around 1000 records. I need to write a SCD type 2 table flow to fetch these changes. Can anyone please suggest the efficient way of reading the data from a huge table.

 

Currently I'm using tmysqlSCD for loading scd type 2 table

Labels (2)
Labels
308 Views
0 Likes
1 Reply
Raghuram_Puram
Contributor III
Contributor III
‎2020-05-23 02:32 PM

Quick thought: Bulk Load a certain no.of records into Temp table and use this temp table as source.

Parallelization might also do the trick. Can you try and post your findings?

308 Views
0 Likes