Need some information on the log info

[suvbin]

Hi,

we are getting this error in one of the task, source is IBM db2 and target is google big query  --  this is child task .

00008612: 2023-10-26T12:20:38 [TARGET_APPLY    ]I: CSV files will not be kept (odbc_endpoint_imp.c:2133)

00008612: 2023-10-26T12:20:38 [TARGET_APPLY    ]I: Error CSV files will not be kept (odbc_endpoint_imp.c:2142)

can you please let me know why we are getting this error. 

[john_wang]

Hello @suvbin ,

Thanks for the post.

These 2 lines does not indicate any error, this is regular information: Qlik Replicate will use 'copy' or 'load' utility to upload the CSV files (which is used for batch load) to target DB first, then merge the change (in CDC stage), or data (in Full Load stage) to final target table. the .CSV files are interim usage only. By default, Replicate will clean these files automatically. If needed for example troubleshooting purpose we may let Qlik Replicate keep these interim files. The Internal Parameter names are keepCSVFiles & keepErrorFiles. 

hope this helps.

Regards,

John.

[kojagi3]

One important thing with logging is context. You should be able to search for a uuid primary key from your database and find all the relevant logging information later when you need. Like searching a user_id, order_id or whatever else should find you relevant logs, Having a log that has nothing but a string message saying "the code got here" isn't particularly helpful.

[Heinvandenheuvel]

>>>   .... [TARGET_APPLY    ]I: .... can you please let me know why we are getting this error. 

@suvbin - you have been working with Replicate quit a lot now. It would seem that by now you should know that this is NOT AN ERROR, but and INFORMATIONAL.

The letter between the closing bracket for the thread name and the colon indicates the 'level' of the the message.

I = Informational - NOT AN ERROR

W = Warning - Might be undesirable behaviour, or a performance issue, but not necessarily an error.

E = Error 

Double-checking: Did these informationals pop up once, early in the log, before actual applies (context!) if so then it should be very clear it is just confirming what it was asked it to, implicitly or explicitly. If if happens multiple times during processing it could indicate something which it would have liked to do, but could not but did not have an impact on the replication as such.

>>> Context -  Having a log that has nothing but a string message saying "the code got here" isn't particularly helpful.

@kojagi3  Agreed, but I do not see the relevance in the context of this topic. Replicate has failed many many opportunities to provide meaningful additional information. Just a few - "working" every 30 minutes is nice, but why not add event count and latency. 'processing oracle archive log' is fine, but why not add the time inside the log and the number of events read on closure like you done for SQL Server CDC. Even more so for errors - when you report, add some tangible, traceable information which is internally available more often than not.

Hein.

[suvbin]

@Heinvandenheuvel : Thank you for your response. Am aware that this is information. But until unless we don't get the context of this message,  there will be a impression that this may be a error. 

[Heinvandenheuvel]

>>  But until unless we don't get the context of this message,  there will be a impression that this may be a error. 

I beg to disagree. Informational is informational. Read it and decide whether it is as you want or not.

For the context - My best guess is that this is a message during initialization before any changes are processed. That makes it even more of a 'maybe you'd like to know' type message. If it happened during processing after some time with the message then I agree with you it starts to look like and error.

Hein.