Databricks cdc errors and latency

MoeE — Fri, 17 Nov 2023 04:58:24 GMT

Hi guys,

I just wanna check if anyone has seen this issue before?

Been having cdc issues and high latency. We check the logs and we saw this error a bunch of times for insert, update, delete, merge which has been occuring for about 1 month. We've restarted the cluster and refreshed the tables as suggested in the error message but this error still persists. source and handling latency are about 8 hours each. Next step I might increase source_capture, target_apply, sorter and stream to try find more info as to the cause of these errors.

Failed (retcode -1) to execute statement: 'INSERT INTO `table`

RetCode: SQL_ERROR SqlState: HY000 NativeError: 35 Message: [Simba][Hardy] (35) Error from server: error code: '0' error message: 'org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 3583754.0 failed 4 times, most recent failure: Lost task 0.3 in stage 3583754.0 (TID 6325042) (10.30.144.133 executor 560): com.databricks.sql.io.FileReadException: Error while reading file abfss:REDACTED_LOCAL_PART@someplace.dfs.core.windows.net/someplaceelse/attrep_changes6305620283B82FB1/CDC00000002.csv.gz. [DEFAULT_FILE_NOT_FOUND] It is possible the underlying files have been updated. You can explicitly invalidate the cache in Spark by running 'REFRESH TABLE tableName' command in SQL or by recreating the Dataset/DataFrame involved. If disk cache is stale or the underlying files have been removed, you can invalidate disk cache manually by restarting the cluster.

Regards,

Mohammed

Re: Databricks cdc errors and latency

john_wang — Fri, 17 Nov 2023 05:50:33 GMT

Hello @MoeE ,

Thanks for reaching out to Qlik Community!

What's the Qlik Replicate version please? and also please show more lines around the error, I'd like to see what's the SQL or operations Qlik Replicate is trying to do now.

Thank you,

John.

topic Re: Databricks cdc errors and latency in Qlik Replicate

Databricks cdc errors and latency

Re: Databricks cdc errors and latency