tFileInputDelimited / tEDIFACTtoXML with variable file names

[Anonymous]

Hi,
We are currently evaluating different ESBs / integration soluations for our company.
One of our main requirements is the conversion of a large number of customer specific input formats into our internal canonical format. The customer data arrives on different routes: Some of it is JMS but most of them are traditional file drops (i.e. files transmitted via FTP).
The tFileInputDelimited and tEDIFACTtoXML components seem to be the components meant for this task, but both of them take a fixed file name as input. As our customers send thousands of documents with different names every day this is obviously no good.
I saw that the tFileInputDelimited can also take an inputStream which in combination with tFileList can 'scan' a directory and process the files contained therein, but that doesn't strike me as a very good solution. (Small question on the side: How do these components deal with files that are 'in transit', ie not fully transferred yet?)
I would prefer to have a solution where the customers' files are picked up and posted to a queue or webservice where the actual mapping and further processing is done.
What is the best way to achieve this with Talend's solution?
Thanks,
Kai

[Anonymous]

Hi Kai,
The best way to achieve this in Talend is to use the Talend ESB, which also has an open source offering freely available for download. From there you have support for file polling as well as FTP polling. You also have support for web services and JMS interactions. So you could easily build an ESB route to pick up those files (not limited by name), and drop them in a JMS queue or invoke a web service. The ESB components have various ways to deal with files that are not fully written. You can get into that documentation by actually doing a Google search on "Apache Camel file2" or "Apache Camel FTP".
In order to get into full transformation capabilities within ESB routes, I suggest taking a look at our enterprise offerings.
Ben

[Anonymous]

Hi Ben,
Thanks for your reply. Can you please explain in a bit more detail how to achieve the 'full transformation capabilities within ESB routes'? That's exactly what a colleague and I have been trying to find out but failed to do so far.
Do we need the 'Platform for Enterprise Integration' for this? If so, is it possible to get an evaluation version?
Thanks,
Kai

[Anonymous]

Kai,
for your use case with 'One of our main requirements is the conversion of a large number of customer specific input formats into our internal canonical format. ' we have a new feature which we will only fully introduce with our next version 5.4 (endof 2013) but which we can make availble to you as a preview if you like. Additional to the more limited tEDIFACTtoXMl component and tXMLMap component which serve the purpose of specific transformations only we have acquired the Oakland Data Transformer recently which provides exactly what you need in combination with the ESB RouteBuilder (which Ben mentioned above and which provides flecible ways to poll/read file use the transformation in between and then send the data e.g. to a Talend Data Service or any other Web_Service ) - As the transformation stuff is quite new (at least witin Talend - it is proven and used in production already within the above senario with Apache Camel so the new part is only tht it is now part of our product offering) it would be good if you could get in contact with us so we can see how we can support your evaluation. I'm pretty sure that this is really what you need and what you like to see but we can not just point you to a download and we might need to add some presales support for you to make it easier for you to evaluate it. You find the nearest location here: http://www.talend.com/contact
Dietmar

[Anonymous]

or send directly an email to sales.de@talend.com so that the closest office get in touch with you.
Elisa