topic Re: split header and data in to different output files in Talend Studio

split header and data in to different output files

Anonymous — Tue, 22 Mar 2016 19:48:13 GMT

Hi All,

I am new to talend and trying to get my hands as dirty as possible.
Could someone please help me with the below need:
I've a employee.DAT file with members details as shown below
---- this line is not present in the DAT file ----
employeeID|firstName|lastName|birthDate|address1|city|state
Ndeggk|Jimmy|Hoover|16-02-2007|North Erringer Road|Albany|Alabama
UNTeEQ|Bill|Johnson|22-12-2007|San Marcos|Baton Rouge|Vermont
eh1nB8|Warren|Truman|29-12-2007|Santa Rosa North|Boise|Tennessee
---- this line is not present in the DAT file ----

Now, the requirement is to split the ..\inputFile\employee.DAT into two files

Step 1: save all employee records in the same format without header into a file say ..\outputFile\employee.DAT
this step was simple and I got it right

Step 2: save header info (employeeID|firstName|lastName|birthDate|address1|city|state) into ..\outputFile\employee.META

(a) catch is that each header field need to be written as a row in the output file without delimiter "|"

headerField
employeeID
firstName
lastName
birthDate
address1
city
state

(b) a new column must be added called fieldType and set to String by default for each row as shown below

headerField fieldType
employeeID String
firstName String
lastName String
birthDate String
address1 String
city String
state String

I accomplished step 2 (a) with tSplitRow as below

In this case, I had to manually add the column mapping and it works as long as the header fields in the file are always same or fixed, which is NOT in my case. The copy of employee.DAT file that I receive today might have same or different fields from that any other day. but the format is same (each header field is separated by "|" delimiter)

so basically I could succeed with entire step2

appreciate any kind of help .. thanks for your time.

* there is no restriction in how to accomplish this ... I mean it could be just by using talend components, or routines or tJava related components or any script

Regards,
P.R

Re: split header and data in to different output files

Anonymous — Tue, 22 Mar 2016 20:50:07 GMT

so basically I could NOT succeed with entire step2

Re: split header and data in to different output files

Anonymous — Wed, 23 Mar 2016 06:14:23 GMT

Hi, you can achieve it using dynamic feature of talend, which is only available in enterprise version. Please refer the below link-

Re: split header and data in to different output files

Anonymous — Wed, 23 Mar 2016 06:15:35 GMT

link
thread

Re: split header and data in to different output files

Anonymous — Wed, 23 Mar 2016 14:27:16 GMT

Thank you Shruti .. I'll go through the link and try to use it on Enterprise trail version.

I was successful in Step 2 by using tTurnRow component, a custom component downloaded from Talend Exchange.
Referred to the post:

Re: split header and data in to different output files

Anonymous — Wed, 23 Mar 2016 14:37:47 GMT

more details on how to use tTurnRow at the link

Re: split header and data in to different output files

Anonymous — Wed, 23 Mar 2016 14:39:00 GMT

not sure why the link is not getting displayed..

incase it repeats again .. take a look at this related topic
https://community.talend.com/t5/Design-and-Development/resolved-tTurnRow-not-showing-on-pallet-in-talend-5-3-0/m-p/109840