topic File Handing in Talend Studio

File Handing

Anonymous — Thu, 15 Jun 2017 22:46:07 GMT

Hi ,

I have below logfile(ProcessLog_06062017.txt ) and wanted to load into the table by application wise. how to achieve this :

FirstApp

12345 Records FileA

1345678 Records FileB

SecondApp

1111 Records FileC

7687 Records FileD

.....

I want to load the above into the table which is having below structure

App Name FileName RecordCount LdDate

Firstapp FileA 12345 06062017

Firstapp FileB 1345678 06062017

SecondApp FileC 1111 06062017

Its a logFile (So No column delimiter)

Re: File Handing

Anonymous — Fri, 16 Jun 2017 09:20:09 GMT

OK, I am going to make a suggestion based on what I am seeing. First of all, you say there is no column separator. There is. You have a space. That should do.

1) Use a tFileInputDelimited, set the separator as a space. Create 3 columns

2) You know when you have a header row as only the first column will be populated. Use this logic to identify it. You can use a tMap (or a tJavaFlex if your Java is reasonably good) to find this column and store it between rows under the header. In the tMap use a tMap variable for example (they can hold values between rows). Add this column to your "AppName" column

3) For the date field I am assuming you are using the date of the file or the date of processing. This should be relatively easy

I've not given a detailed explanation, but a high level method. It is much better you struggle on the right path, than get given the answer. The hardest part here is storing the value between rows and ensuring they are updated as and when required. I have a tutorial based on this sort of logic (https://www.rilhia.com/quicktips/quick-tip-compare-row-value-against-value-previous-row)

Re: File Handing

Anonymous — Fri, 16 Jun 2017 23:06:03 GMT

Thanks for the direction. I have achieved the solution by comparing previous row vs current row.

I have not consider space as delimiter because there is a space in file name and application name. Hence i used tfileinputPositional .