EDI Mapping

[c0utta]

Hi there,
I'm developing an EDI solution using talend, and would appreciate some direction on best practice. I have used SQL Server DTS extensively so am used to data transformation tools, but talend seems to have more going for it. Biztalk scares the hell out of me, and I've had some bad experiences with other tools.
We are receiving EDI messages in a set format from our customer, which must then be formatted into a Header, Detail and Footer records to load into our ERP system.
My understanding is that I need to create Metadata objects for each record type in an EDI message, together with 3 Metadata objects for the Header, Detail and Footer. The incoming file must be linked to multiple Metadata objects, and the outgoing file must be linked to the Header, Detail and Footer Metadata objects. Or that's how I imagine it anyway.
It's easier to show the source and the intended result and obtain suggestions as to what is the best way to approach this task.
Input file
----------
BGM+220+R-07102N+9'
DTM+137:20070412:102'
DTM+2:20070419:102'
NAD+BY+9313938000631::9'
NAD+ST+9418GR::92'
NAD+SU+665487::92'
LIN+1++19310441008069 RV'
QTY+21:64'
DTM+15:20070430:102'
PRI+1E:17.2738'
UNS+S'
MOA+86:362.7498'
CNT+2:1'
Output file
------------
H,9313938000631,R-07103N,20070412,20070419,20070419,9418GR,665487
D,19310441008069,21,20070430,17.2738
S,21
Suggestions gladly received, and thanks for reading this post.
c0utta

[c0utta]

A follow up.
I ended up working out a way to do this myself with Talend, although I had to learn a bit of Perl and Regex to do so (good skills to have though!).
Using a combination of tFlexRow and tMap I was able to suppress lines and consolidate headers and footers. This post ( https://community.talend.com/t5/Installing-and-Upgrading/Change-Code/td-p/57144) helped me a lot, particular the hint regarding the constraint in tMap. Setting of the @output_row within the tFlexRow to be suppressed in the subsequent tMap was vital.
From an EDI perspective I used regular expressions to parse values by extracting matches.
Success for Talend will rely upon a vibrant community, so please make sure when you resolve an issue or have a breakthrough, report back to the community so we can all learn.
Cheers,
c0utta

[Anonymous]

Hi c0utta,

We are currently working on components that should simplify EDI mapping and are very interested to see how your jobs use current TOS components. Thank you for your support and feedback.

[mamaf]

Hi everybody
I'm also interested in EDI components.
Are these components out of the box ?
Thanks in advance for your reply.
Jean-Marc

[Anonymous]

Any update on EDI components for Talend?

[_AnonymousUser]

Surely out of the box EDI parsing components will boosts Talend in enterprises and so paying customers, and bundling them in talend will be a surprise in the open source ETL community ( the first ones to have them ).
surely this parsing can be done with regexs, but the whole point of an ETL tool is objects reusing, so i'm looking forward to a support in talend.
thanks

[_AnonymousUser]

For an EDI component, one should look at an SEF file that describes the loops, segments and what is valid in the EDI document. This way you do not have to worry about what transaction it is but simply if you have a valid SEF file and does the input data match the SEF file.
Way to thing of SEF files is almost like an XSD for XML.
Maybe someone could pull
http://sourceforge.net/projects/edival/
and see how they do there stuff.
Jon

[_AnonymousUser]

Hello all,
There is an open source ready solution for working with EDI files in Talend.
http://code.google.com/p/soperadi-smooks
We developed it for one of our customers who needs EDI-like structures and using Talend. It works quite well, you can check the screencasts.
Renat

[Anonymous]

Small update on the progress on the Talend EDI (smooks-based) components:
http://blog.sopera.com/2010/07/unedifact-support-overview/
Renat