topic Re: Cannot refresh source table in Qlik Replicate

Cannot refresh source table

zhangsq24 — Fri, 04 Feb 2022 04:47:23 GMT

A task contains many tables, other tables load data normally, only this one table has problems, what causes this problem and how to solve it?

Re: Cannot refresh source table

Heinvandenheuvel — Fri, 04 Feb 2022 06:28:30 GMT

Dear @zhangsq24 . Most readers here would like to help, but have only feeble mindreading powers.

Please help us help you by indicating:

>> only this one table has problems, --- WHICH TABLE?

>> what causes this problem and how to solve it? --- WHAT PROBLEM ? Error message? Slowness (quantify!)

The attached diagnostics package is nice, but there are so many places to look. Please give a hint

-- which specific log to start

-- which source (+version), target (Kafka), Replicate (version). - Yes, that's all in the log but please help us help you.

Cheers,

Hein

Re: Cannot refresh source table

zhangsq24 — Fri, 04 Feb 2022 07:51:03 GMT

Hi HeinvandenheuveL

@Heinvandenheuvel

1、table name: Z_LOGISTICS_Z.ZDISC_BO_2

2、only the table cannot refresh metadata

3、source：DB2, target : apache kafka 2.0 . the task in prod environment，This task has a table running, and cannot be started or stopped at will.

the error infomation as follow:

]I: ODBC driver version: '11.01.0405 (ar_odbc_conn.c:612)
]I: ODBC driver version:"11.01.0405 (ar_odbc_conn.c:612)
]I: Source endpoint 'DB2 LUW' is using provider syntax '<default>'(providersyntax_manager.c:873)
]I: Reload for table Id 2 is requested (sorter transaction.c:2405)
]I: Start collecting changes for table id=2(sorter_transaction.c:2280)
]I: start 1oading tab1e 'Z_LOGISTICS_Z'.'ZDISC_B0_2' (Id = 2) by subtask 2. Start 1oad timestamp 0005D71655A9D880 (rep1icationtask_u
]E: Cannot refresh source table (2) metadata [1021802] (endpointshe11.c:3727)
]I:
Target endpoint 'Kafka' is using provider syntax'Kafkasyntax'(provider_syntax_manager.c:879)
]W: Table 'Z LOGISTICS_Z'.'zorsc_80 2' (subtask 2 thread 0) is suspended. cannot refresh source table (2) metadata (replicationtask.]I: Unload finished for table ''."!(Id =0). 0 rows sent.(streamcomponent.c:3663)]I: Reload for table Id 3 is requested(sorter_transaction.c:2405)
]I: Start collecting changes for table id=3(sorter_transaction.c:2280)]I:
start 1oadingtab1e'ZLOGISTICSZ'.'ZFICOEBREXTEND’(Id = 3) by subtask 2. Start 1oad timestamp 0005D71655A9D880 (rep1icationt
]I: kafka client version: 1.5.2 (queue_imp.c:995)
]I: resolve table orig db: table def orig db id of table 'sAPEHD.ZFICO EBREXTEND’= 000814A2 (db21uw endpoint metadata.c:266)]I: Metadata from broker 2:sas1_ss1://n2-mkp-by.lenovo.com:9092/2):(kafka_client.c:284)7I:8 brokers:(kafka_client.c:287)

Re: Cannot refresh source table

lyka — Wed, 23 Feb 2022 00:58:26 GMT

Hello,

I checked the metadata file and i dont see the table ZDISC_BO_2.

Was the table dropped? Can you remove the table from the task then resume.

Once you see eveything is running fine, can you stop the task and the table back.

Set source_unload and target_load to TRACE (in case you encounter any errors during full load)

Please then resume the task and it should trigger a full load.

Thanks

Lyka

Re: Cannot refresh source table

Heinvandenheuvel — Wed, 23 Feb 2022 21:08:00 GMT

This appears to be a LOGSTREAM setup. From the provided json (thank you) :

"logstreamstagingtask": "EHP_APP_DB-EHP_LS_Z_TRANSCATION",

Is the requested table (still) available in the logstream task?

If you were to run a test version of this task, with just this ZDISC_BO_2 table, and directly against a main source end point without logstream does it work? Now with the logstream endpoint and just that table. Does it still work?

ZDISC_BO and ZDISC_BO_2 are in the task_export.json and are the only two tables with Chinese characters in the description. That shouldn't matter and loading ZDISC_BO works, but maybe it does matter. Is the description functional for you (pre or post processing)? If not, you might want to remove that at least as test.

This is still in development right? Do yourself and your environment a little favor and reload after changing the naming from TRANSCATION to TRANSACTION. Obviously that does not matter for the Replicate system at all, but it will stop silly questions and miss-typed searches.

In the included log fragments is from reptask_S_EHP_LS_Z_TRAN_LOGIS_Z-KafkaEHP__220203221657.log right? The problem table load failed early on, and then was retried 3 more times. The included text in this topic shows 'Z_LOGISTICS_Z'.'ZDISC_B0_2' and target 'Z LOGISTICS_Z'.'zorsc_80 2' which is odd, and not in the log that I can tell. There are funky characters in the name in the log which may explain that.

hth,

Hein.