Mechanism for calculating source latency when using SAP Extractor as a source
Premise: SQLConnector is being used.
My understanding is that source capture in the extractor is conducted through the following two processes:
1.Execution of the GCDEXEXXXXX job: Data is extracted from the data source and stored in the ZQLIKDTCAXXXXXX table. 2.Execution of the /QTQVC/READ_DATA-XXXXX job: Data is retrieved from the ZQLIKDTCAXXXXXX table and transferred to Qlik Replicate.
Could you please explain how the source latency is calculated in Qlik Replicate in this scenario?
My customer wants to understand how the source latency is calculated to confirm whether the reading of the differential data generated by the batch processing is complete.
Latency is mainly calculated when operation on the data for the participating table is initiated on Source and applied/committed on the target side.
As replication tools to design to replicate committed data only . Table data type and volume plays a vital role in that .
if table have PK and don't have Lob's then throughput is always good, and latency is normal as per task settings.
if table don't have pk and lob then you may encounter latency.
** latency is also pertained to the way task is configured and involved endpoints as well.
Consider that if you have task and have different tables have lobs and don't have pk. tasks are prone to have higher latency and effect the overall throughput of task.
Thank you for the reply. I understand the general concept of latency. Given that, my question is specifically about how Qlik Replicate calculates source latency when the source is an SAP Extractor.
There is no change in the Data in the way Qlik works. it follows the same principals. the only difference will be they way its interact with the source endpoint Database.
in normal Rdbms it requires some privileges on certain tables to capture the tables metadata and changes stored in the log files or transaction logs.
In case of SAP as QR cannot connected with Rdbms Directly SAP application layers is in between so Qr submits request to backend Db and extractor pass it via some code to sap engine and then it processed . as Qr dont have Access to the directly it rely on the supplementary tables which were created when configuring extractor and table addition to task and full load and cdc put in action.
mainly latency also depend upon how fast Sap engine provide required info to the Qr in case of SAP extractor.
What I would like to know is how Qlik Replicate calculates source latency when using SAP Extractor as a source endpoint.
For example, I would like to know the specific calculation method, such as "If the data in the ZQLIKDTCAXXXXXX table received by the Qlik Replicate server was registered in the HANA Database 10 minutes earlier, the source latency would be calculated as 10 minutes."
Source latency is calculation done when the transaction is initiated on the participating table. And once the Commit is found it will qualify to be replicated.
you will see this under the monitor tab. as until source Commit and open transaction
I explained how log-based replication works as Databases contain transaction logs (also called redo logs) that store all database events allowing for the database to be recovered in the event of a crash. Withlog-based change data capture, new database transactions – including inserts, updates, and deletes – are read from source databases’ native transaction logs. in case of SAP application layer provide info when configured.
Latency is when the data is updated on source and applied on target. everything in between added up into the latency.
I understand that overall latency refers to the time from when the data is updated at the source until it is applied at the target. My question, however, is specifically about the source latency.
Additionally, I'm not asking for the general meaning of source latency; I want to know which part of the extraction process in the case of an SAP Extractor source is being calculated as source latency. Please provide an answer linked to the detailed extraction process.
