Getting details of a bulk apply error in Google Big Query
While replicating data from Salesforce to Google Big Query we are frequently having data errors.
Most of the time we are not able to identify the cause of those errors since the logs are not detailed enough (we cannot keep verbose logs always active in production right now) and the errors are not written in attrep_apply_exceptions since it would require the transactional/1:1 mode that is not compatible with Big Query.
We already discussed the thing with Support and SMEs but they concluded that we do not have a way to view this data errors on the exceptions table, and this would be by design.
We would have the opportunity to understand what causes those errors in some way, it is not relevant if in the logs or in a table. It is the only way to go to the team managing the source system and work with them to solve the root cause of the errors.
As I understand the request, you would like to see some more information in the logs at INFO level to help troubleshoot since it is (very understandably) not possible to constantly run in VERBOSE mode in production.
We will collect feedback from others here and consider as we do our future planning.
NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.
