topic [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component in Talend Studio

[resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

Anonymous — Mon, 23 Jul 2012 13:07:39 GMT

Hi
I am having files with different type of delimiters, i don`t know where and which file has delimiter issue, for that i am creating Job, which reads file in full row mode and then checking how many delimiters we have in a row, using delimiter count tjavaRow comp. and storing that delimiter count in a context variable to used in tMap to out put the row.
my job is look like this:
tFileList----TFileInputFullRow ---tJavaRow------------- tMap-----------tFileOutPutPositionall (Good Records)
|
|
tFileOutPutPositionall (Bad Records)
in tJavaRow code
context.vDelCount = line.Split(delimiter).length;
in tMap
I have created two out put one is for good and second one is for Bad using
context.vDelCount ==defaultCount then send to good
context.vDelCount !=defaultCount then send to bad
but some file delimiter is in " " quoted string i am able how to figured it out and resolve this issue. some times file has nested quoted string like below
" some |text " inner|text " "|xyz string|pqr string"|""
is there any way to put text qualifier and quoted string in tFileInputFullRow?
please suggest how to parse above row to validate the delimiter count as well nested quoted string issue.

Re: [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

Anonymous — Tue, 24 Jul 2012 02:54:52 GMT

but some file delimiter is in " " quoted string i am able how to figured it out and resolve this issue.

Sorry. Could you explain the above sentence in another way?
Besides, there isn't a way to put text qualifier in tFileInputFullRow.
Regards,
Pedro

Re: [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

Anonymous — Tue, 24 Jul 2012 05:32:20 GMT

sorry it has been missed some text.
i mean to say, some delimiters comes in quoted string . In that case how to skip quoted delimiter, to validate delimiters count in a row.
if there is no way to put text qualifier in tFileInputFullRow. then could you suggest me how to accomplish this task.

Re: [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

Anonymous — Tue, 24 Jul 2012 06:31:05 GMT

Hi
Got you.
I think you can use tReplace to replace " " with a space.
Regards,
Pedro

Re: [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

Anonymous — Tue, 24 Jul 2012 11:30:19 GMT

Re: [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

alevy — Wed, 25 Jul 2012 01:39:51 GMT

I suggest you try the opencsv parser, which gives you a very flexible way to parse strings to arrays with variable delimiters, text enclosures and escape characters.

Re: [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

Anonymous — Thu, 26 Jul 2012 07:57:42 GMT

sorry we can not use any third party utilities, i have found the solution on google to skip the delimiter from quoted string
but now i got new problem, i need to replace quote inside quote string. see below example
INPUT String:
439.00 01/28/2012 65 39284 7218063470 "xyz string"" with pqr string" 78.00 0000505585 my test string 2.06 history be 29.00
OUTPUT Line:
439.00 01/28/2012 65 39284 7218063470 "xyz string with pqr string" 78.00 0000505585 my test string 2.06 history be 29.00
any suggestion.

Re: [resolved] How To Validate Delimiters in Row Using tFileInputFullRow component

Anonymous — Thu, 26 Jul 2012 11:28:28 GMT

Thank You All of you, Got the solution on below link.
Hope it may help others thank you..
http://stackoverflow.com/questions/11649389/java-reg-ex-to-replace-double-quote-inside-double-quote