topic Re: CSV Splitter or Filter with Talend Java in Talend Studio

CSV Splitter or Filter with Talend Java

_AnonymousUser — Fri, 12 Oct 2012 21:28:10 GMT

We had a requirement today about splitting CSV which as you can see from http://thinkinginsoftware.blogspot.com/2012/10/the-data-team-was-in-need-of.html we resolved with a tJavaFlex component but ideally I think it should be either a new component in Talend or added as a future in existing ones.
The need is to parse an $inputFile using a $delimiter, and output only the lines where the $lookupColumn has a specific $lookupValue (four parameters).
Will this make sense to be added as a feature request? Or the recommended way to go would be to build an in house component? Probably Talend should open a project to host contributions as "talend custom components library"? A google search reveals tutorials on how to build custom components but I believe an open project to host the additional components will only help.
Thanks!
-Nestor

Re: CSV Splitter or Filter with Talend Java

Anonymous — Sat, 13 Oct 2012 00:15:10 GMT

tMap can certainly return matching rows only as that is one of the basic functionality of the application. On the fly schema is a different matter. Did you do a search at all and/or read the documentation?

Re: CSV Splitter or Filter with Talend Java

_AnonymousUser — Sat, 13 Oct 2012 01:56:26 GMT

Yes. I did the research. We have been using talend for a while. The only reason you see me as a guest is that I cannot longer login in http://www.talendforge.org
When I login I land in talend.com and when I try to get back to http://www.talendforge.org I look like logged off and I can only post as guest.
I know what a tmap can do but that cannot provide the solution we need.
I searched on the web and this forum, I looked into the manual, I tool a look at all possibly related components before going the tJavaFlex route.

Re: CSV Splitter or Filter with Talend Java

_AnonymousUser — Sat, 13 Oct 2012 02:18:53 GMT

I saw you kindly pointed out to the contextualized Help in Talend GUI in http://www.talendforge.org/forum/viewtopic.php?pid=96420#p96420 so I am researching on that now as "Dynamic Schema". I will get back with my findings. Thanks!

Re: CSV Splitter or Filter with Talend Java

_AnonymousUser — Sat, 13 Oct 2012 03:07:07 GMT

Please correct me if I am wrong but all I see about "Dynamic Schema" refers to TalendIntegartionSuite_Studio_UG.pdf which is only available with a paid subscription meaning is not open source. So you must be a user of Talend Integration Suite to read the chapter from that book called "dynamic schema" .
So given the fact that as I posted in my tutorial it is fairly simple to build a CSV Splitter or Filter do you think it makes sense to request the addition of this component (or similar) to the open source version?
Thanks!
-Nestor

Re: CSV Splitter or Filter with Talend Java

Anonymous — Sat, 13 Oct 2012 03:49:04 GMT

Hi Nestor
Thanks for your posts!

Probably Talend should open a project to host contributions as "talend custom components library"?

Talend Exchange is the best place for all community users to share custom components, you can always find some useful components developed by community users.

A google search reveals tutorials on how to build custom components but I believe an open project to host the additional components will only help.

A good tutorial about how to create custom component is written by saburo, see
http://community.talend.com:80/t5/Design-and-Development/Writing-a-component-creation-tutorial/m-p/5212#M1576

Please correct me if I am wrong but all I see about "Dynamic Schema" refers to TalendIntegartionSuite_Studio_UG.pdf which is only available with a paid subscription meaning is not open source.

Yes, the dynamic schema feature is only available in commercial subscription version.

Re: CSV Splitter or Filter with Talend Java

_AnonymousUser — Sat, 13 Oct 2012 05:19:34 GMT

Thanks Shong! I will check Talend Exchange next.
Great community here at Talend. Way to go, keep it up guys!
Best,
-Nestor

Re: CSV Splitter or Filter with Talend Java

nestoru — Thu, 08 Nov 2012 14:19:34 GMT

I ended up creating a tFileInputCSVFilter available in github together with a tutorial to build components https://github.com/nestoru/talend-custom-components

Re: CSV Splitter or Filter with Talend Java

Anonymous — Thu, 08 Nov 2012 18:26:38 GMT

I ended up creating a tFileInputCSVFilter available in github together with a tutorial to build components https://github.com/nestoru/talend-custom-components

Hi nestoru
Great job! Thanks for your sharing to Talend community!
Why don't you upload an example job to github? It help the users who want to use this component to learn the component quickly.

Re: CSV Splitter or Filter with Talend Java

nestoru — Thu, 08 Nov 2012 20:08:32 GMT

Hi Shong,
I have added a sample on how to use it in my blog http://thinkinginsoftware.blogspot.com/search?q=csvfilter#talendComponent
This component is really simple. There are 4 params to be used and the picture should be self explanatory.

Cheers,
-Nestor

Re: CSV Splitter or Filter with Talend Java

Anonymous — Sat, 10 Nov 2012 00:28:38 GMT

Hi Shong,
I have added a sample on how to use it in my blog http://thinkinginsoftware.blogspot.com/search?q=csvfilter#talendComponent
This component is really simple. There are 4 params to be used and the picture should be self explanatory.

Cheers,
-Nestor

Cool, Thank you!

Re: CSV Splitter or Filter with Talend Java

nestoru — Sun, 11 Nov 2012 19:47:45 GMT

I have added two new features to the component BTW, now it is possible to have the header somewhere in the file other than on the top row. This is handy for some files containing garbage at the beginning. I am also pushing the headers to the output as this is a component to just filter the raw CSV file.
Cheers,
-Nestor