UTF-8 (csv) files that have issues with \n newline issues within the files (see attached) See Row 3 of the Preview where the \n is causing the file to go to new line. How to fix?
Notepad++ works to find/replace /n to " " but i don't want to keep doing this every month for 50 files to upload.
I tried tmap row3.COLUMN.replaceAll("\\r\\n|\\r|\\n"," ") and did this for every column, however, then I am getting a nullexception after it runs half of the file.
Update: I got the replace to work now and the data looks good, but i had to add a tFilterRow_1 before tMap because there are 2 rows at the bottom of every file in the first column (summary info of the report) and I filtered them out. Is there a better way of filtering out these 2 rows in every file, because it made the length of uploading 4xs longer????
Notepad++ works to find/replace /n to " " but i don't want to keep doing this every month for 50 files to upload.
I tried tmap row3.COLUMN.replaceAll("\\r\\n|\\r|\\n"," ") and did this for every column, however, then I am getting a nullexception after it runs half of the file.
Update: I got the replace to work now and the data looks good, but i had to add a tFilterRow_1 before tMap because there are 2 rows at the bottom of every file in the first column (summary info of the report) and I filtered them out. Is there a better way of filtering out these 2 rows in every file, because it made the length of uploading 4xs longer????
Latest Update: Instead of using tfilter, i am using tMAP to filter out the 2 rows and it runs pretty fast. However, I am trying to iterate through every file and have each file automatically loaded into oracle, but it doesn't seem to be working. When i copy/paste the last 3 processes and create them individually they run perfectly (see individual processes in green in bottom right). The only different is the file path in the csv and one does a create table and the other is doing a insert into and i am not seeing anything wrong with the tMap.
Any Suggestions?
Any Suggestions?
Thanks JoRoesecke, I didn't think about that, that may help for one part of it. I am using tMAP because i need to remove /n newline characters from the csv file.
My latest with this is that I have found that I can get the Job to run fine (see attached) when i have my File Delimited set to a specific file path on the share drive, but once i put ((String)globalMap.get("tFileList_1_CURRENT_PATH")) that is when the tMAP is crapping out on me.
I tried to put footnote to 2 and I tried tJavaRow instead of tmap to replace my /n characters, but i am not getting NULL Pointer Exception.
My latest with this is that I have found that I can get the Job to run fine (see attached) when i have my File Delimited set to a specific file path on the share drive, but once i put ((String)globalMap.get("tFileList_1_CURRENT_PATH")) that is when the tMAP is crapping out on me.
I tried to put footnote to 2 and I tried tJavaRow instead of tmap to replace my /n characters, but i am not getting NULL Pointer Exception.