topic Re: Split a file with Talend ESB in Talend Studio

Split a file with Talend ESB

Anonymous — Sat, 16 Nov 2024 10:28:52 GMT

Hello everybody,
I am new to the forum
I use Talend ESB and I have a question.
I have this flat file:
11886 1855 0000004309000
11886 1855 0000057370000
11886 1856 0000057374001
11886 1856 0000057375000

And i want 2 files (messages), a filter of "1855" and "1856":
11886 1855 0000004309000
11886 1855 0000057370000
And:
11886 1856 0000057374001
11886 1856 0000057375000
I tested the cfilter but i doesn't work.
Do you have any idea ?
Thank you in advance.
Natacha.

Re: Split a file with Talend ESB

Anonymous — Mon, 01 Aug 2016 05:04:11 GMT

Hi,
Have you already checked tMap component in DI perspective and set an expression for a filter of "1855" and "1856"?
Best regards
Sabrina

Re: Split a file with Talend ESB

Anonymous — Mon, 01 Aug 2016 14:24:49 GMT

Hi,
Have you already checked tMap component in DI perspective and set an expression for a filter of "1855" and "1856"?
Best regards
Sabrina

Thanks for you answer Sabrina !
I am in Talend Mediation, there is no Tmap.

Re: Split a file with Talend ESB

Anonymous — Mon, 01 Aug 2016 15:01:32 GMT

There are a few ways you could do this. First, please explain what you mean when you say cFilter does not work. Could you share a screenshot showing how you are trying to use it?
So if you know in advance that there are a fixed number of alternatives then you can use a content based router. For example, if you know the only record types you will receive are "1855" and "1856" then you can extract them from the input message, put them into a message header, and then route based on the header.
You would use the following Talend components:
cFile --> cSetHeader --> cMessageRouter --> (the rest of your route)
Note that cMessageRouter can take multiple outputs. You want to use the "When" path for most of your distinct routes.
An alternative approach would be to use cMulticast along with cFilter. It has the "advantage" of potentially being multi-threaded and it will still preserve rel ative order.
If the number of output alternatives is unknown (e.g. put all messages into distinct files based on the value in the second column, then use cSetHeader to set the CamelFileName header. CamelFileName is a special header for the cFile component. You will also need to set the fileExist option on the cFile output component. (Note that this is the output, not the input cFile).
You may want to check the performance if you use the cFile approach. I am not sure how cFile caches its output file handles and if it will do this efficiently. I think it will, but verify.

Re: Split a file with Talend ESB

Anonymous — Mon, 01 Aug 2016 16:22:15 GMT

Indeed, the number of outputs is unknown (my two files are an example). I have a number of unknown orders.
But I don't know how to extract the order number in the file in the cSetHeader. How to extract the order according to the positional?
This is a flat file.
Thank you in advance.

Re: Split a file with Talend ESB

Anonymous — Mon, 29 Aug 2016 20:35:22 GMT

Natacha, you can solve the problem by using the following route:

It is based on a fixed Length bindy unmarshaling and a simple Splitter that generates a bean instance for each file record; then, the cSetHeader component creates a new header property ( here called "group" ) using the second column value; finally you can find the cFile ( output FixedFiles ) with "fixedOut-${header.group}.txt"; so one file per group value is automatically generated ( with the cFile output component the advance setting "fileExist"->"append" is configured to insert records into the corresponding file.
I ran it using your pasted data geting the following two files:
fixedOut-1855.txt
11886 1855 0000004309000
11886 1855 0000057370000
fixedOut-1856.txt
11886 1856 0000057374001
11886 1856 0000057375000

Hope this helps, please let me know if you need assitance on getting the unmarshaling works.
Esteban.