topic How to parse a fixed length using tFileInputPositional component with its Pattern option. in Talend Studio

How to parse a fixed length using tFileInputPositional component with its Pattern option.

Anonymous — Fri, 15 Feb 2019 06:22:14 GMT

i tried to parse a fixed length file. File sample data is:

GClog OP 00000099140001047454 2018-10-26 0

length pattern is:2,5,30,12,6.

I want to parse above line like this: GC;log;OP;00000099140001047454;2018-10-26;0

For getting output like above, i have first tried to simply parse the fixed length file first using this job:5 Rows are processing

Schema description: as every time schema is keep on changing and file having multiple lines also, so i tried to define schema as line only.like this:

when i ran this job,output is :

as in first image it is showing that 5 rows processing, while in tLogRow it is displaying only one row record. Why it is displaying only one record not all.

So kindly help me where i am doing mistake and a s i mentioned we do not ahve any fixed schema so we can not use Filepositional metadata option. Thanking you in advance. Kindly guide me and help me.

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

vapukov — Fri, 15 Feb 2019 06:48:02 GMT

Hi,

I not sure why you remove line separator, but even if file contains 1 row, leave it as default "\n"
in your case pattern must be - "2,4,3,20,11,2" or "2,4,3,21,10,1"
you need define column for each group - 6 columns
finally when you add at the end tFileOutputDelimited with separator ";" you will have string as expected

GC;log;OP;00000099140001047454;2018-10-26;0

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

Anonymous — Fri, 15 Feb 2019 07:26:35 GMT

Hi ,

Thanks for your reply first.

I not sure why you remove line separator, but even if file contains 1 row, leave it as default "\n".running with default"/n" option, giving above result
2. in your case pattern must be - "2,4,3,20,11,2" or "2,4,3,21,10,1":
ans: sample string : GClog OP 00000099140001047454 2018-10-26 0.Length specification as follows:
(a)GC=2
(b)log =5(3 alphabet+2 space)
(c)OP 00000099140001047454 =30(2 alphabet+3 space+20 digits+5 space)
(d)2018-10-26 =12(10 count for date format+2 space)
(e) 0=(5 space+1 digit)
3. you need define column for each group - 6 column.

ans: if suppose i have 50 lines, and each having different number of column. than how can i need to mange this?

I can not define column in a static manner, column number may be change as well as name also. Fixled length file having 1 line or can be more than 100 lines.

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

vapukov — Fri, 15 Feb 2019 07:33:12 GMT

2. length calculated for each field, but not from begin

3. you can not use this component for flexible structure - positional files is fixed, this is not doubt - how you plan to change (and guess) pattern if it different for each row?

3 just mean - you choose the wrong component

upd, you can attach full file, maybe we could suggest the best way (maybe ... )

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

Anonymous — Fri, 15 Feb 2019 07:38:03 GMT

Which component then i should have to check.Please suggest.

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

vapukov — Fri, 15 Feb 2019 07:44:43 GMT

possible - there is no one of components which resolve your task, but it not mean not possible to find a solution

it always depends from global patterns - what is common for your rows? how you will guess - rules for row1 and row37?

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

Anonymous — Fri, 15 Feb 2019 08:35:02 GMT

using regular expression, i am filtering same row data and write it to a positional file and then using there respective fixed length description, i am trying to parse. For file sample purpose you can view this:

GCULOG 9914OP - General - Holdings - 002 21487 1047454545819320181121130053
GClog QA 00000099140001047454 2018-10-26 0

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

Anonymous — Fri, 15 Feb 2019 08:47:20 GMT

For your reference attaching sample file. Please check it.

fixedFileSample.txt

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

vapukov — Fri, 15 Feb 2019 09:03:00 GMT

@vinayak019 wrote:

using regular expression, i am filtering same row data and write it to a positional file and then using there respective fixed length description, i am trying to parse. For file sample purpose you can view this:

GCULOG 9914OP - General - Holdings - 002 21487 1047454545819320181121130053
GClog QA 00000099140001047454 2018-10-26 0

this what I mean - if you understand based on more than 2 rows, how many patterns you have

you could sort rows by - length, substring, regex to proper components with defined structure and pattern, then you could merge them back (if need)

all depends on how many this patterns you are expected

but this is never possible to decide based on 2 rows only

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

Anonymous — Fri, 15 Feb 2019 09:41:40 GMT

that is correct. that file having five different type of datasets, which will be filtered using regular expression in tFilterRow Advanced options.but how can i break those datasets on the basis of their length.

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

vapukov — Fri, 15 Feb 2019 09:46:31 GMT

the simplest way (possible may be another, but this is easy to implement)

read file and iterate row by row
based on RunIf trigger append row to 1 from 5 files
run 5 subjobs - 1 for each pattern

it is not a universal solution, but it could be designed very fast and it will work

you can add one more filter - not matched to any of patterns and store rejected rows for future analysis

Re: How to parse a fixed length using tFileInputPositional component with its Pattern option.

Anonymous — Mon, 18 Feb 2019 07:13:17 GMT

I have implemented, but still not able to parse the fixed length file. Still same issue persists , on the basis of length pattern it is only reading 1 record .Not able to cross this barrier.can you ponder and showed me a designed job using length pattern for already shared fixed length file to delimiter file. Looking forward for your guidance.