Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello Team,
I am trying to read csv file and store its content in a database table.
If a line in csv file is not properly formed, it throws an exception. I want to log all these lines in log file and process all the valid records.
Please see attached a snapshot of what I have so far. I use FileInputDelimited to read file content, then I process it line by line using flowToIterate and FixedFlowInput, passing data to tMap and then to database.
To log, I use tLogCatcher component attached to tLogRow. tLogCatcher is configured to catch java exceptions.
I have two problems here.
1. Assuming I have an invalid records in my csv file, error log file will not be created unless I check "Die on error" on the very first component (tFileInputDelimited). When I check "Die on error", log file containing only one error will be created and execution of the job will stop - not the desired scenario. I need to just log this record and keep processing the rest of the file.
2. Even when this one record is created in log file, it does not contain the line from csv file. It only contains java error message. I need to have entire line from csv file that caused this error. I don't see how I can configure tLogRow and tLogCatcher with different schema to log what I need. I can probably get away with tJava components but is this a right way?
Your help would be very much appreciated.
Svetlana
I am trying to read csv file and store its content in a database table.
If a line in csv file is not properly formed, it throws an exception. I want to log all these lines in log file and process all the valid records.
Please see attached a snapshot of what I have so far. I use FileInputDelimited to read file content, then I process it line by line using flowToIterate and FixedFlowInput, passing data to tMap and then to database.
To log, I use tLogCatcher component attached to tLogRow. tLogCatcher is configured to catch java exceptions.
I have two problems here.
1. Assuming I have an invalid records in my csv file, error log file will not be created unless I check "Die on error" on the very first component (tFileInputDelimited). When I check "Die on error", log file containing only one error will be created and execution of the job will stop - not the desired scenario. I need to just log this record and keep processing the rest of the file.
2. Even when this one record is created in log file, it does not contain the line from csv file. It only contains java error message. I need to have entire line from csv file that caused this error. I don't see how I can configure tLogRow and tLogCatcher with different schema to log what I need. I can probably get away with tJava components but is this a right way?
Your help would be very much appreciated.
Svetlana