topic Re: Custom File Processing? in Talend Studio

Custom File Processing?

Anonymous — Mon, 24 Oct 2016 14:05:53 GMT

I need to download and process bounced email messages. Here is what I have so far. I have a pop component that downloads all emails to a folder. Then I setup a new process to trigger when pop completes. This new process iterates the files with .mail extension. And this is where I run into issues. I need the Status code from the email file and the email address that the message was being sent to. The mail input file component seems to be very limited when it comes to what data it can extract from mail files. My thought was to use a custom parser to process each mail file and extract the values I need. This is where I'm lost and need some direction. Am I not using the mail file input component right? How do you import and process a folder of custom files? I tried using regex, but i don't understand how to process a group of files in a folder.

Re: Custom File Processing?

vapukov — Mon, 24 Oct 2016 15:15:29 GMT

If You question just about process folder - it is tFileList
in Your component You could use - ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) as filename
at least email address You can extract easy by tFileInputMail, not clean - what You mean for Status code

Re: Custom File Processing?

Anonymous — Mon, 24 Oct 2016 15:45:18 GMT

Filelist is what I'm using. It's more about the mail file itself. I can't get the fields I want out of it. A mail file has many more attributes in it then the mail file input will extract. How can I get to them all? I guess I just want to extract values from a whole file. I would like to extract a two column row, with email and error code.

Re: Custom File Processing?

vapukov — Tue, 25 Oct 2016 01:26:02 GMT

rizenine wrote:
I would like to extract a two column row, with email and error code.

What You mean - error code?
at least email (From and To) - easy extracted by native tFileInputMail
what parts of email (how it look in email header?) You try to extract?

Re: Custom File Processing?

Anonymous — Tue, 25 Oct 2016 17:30:35 GMT

Error code is just an attribute in the file. Status code is what it's called. I just want to check that an email was 5.0.0 error so I know to mark it invalid. Then I need the original email address to look it up and mark it.

I have an email file full of the below style information. Just need to grab two of them. The Final-Recipient and Status.
Final-Recipient: rfc822; someemailaddress@yahoo.com
Original-Recipient: rfc822;someemailaddress@yahoo.com
Action: failed
Status: 5.0.0

Re: Custom File Processing?

vapukov — Tue, 25 Oct 2016 22:18:03 GMT

I not have Status in my emails, so I just add Status: 5.0.0 manually for 1 email

Re: Custom File Processing?

Anonymous — Wed, 26 Oct 2016 00:58:09 GMT

There must be something with the format of these email files. I have status field and did the exact thing you did here, and it's not pulling it. I did end up just using a raw file input. Convert to string. Then chain together two regex units to get the two values I needed out. That seems to work, but I'm still not sure why the email files come down in a non-standard format. It's a Lotus Notes email server, so I'm sure that's the issue. It's always an issue.

Re: Custom File Processing?

vapukov — Wed, 26 Oct 2016 05:55:15 GMT

You can post 1 test email file, will be more easy to test

Re: Custom File Processing?

Anonymous — Wed, 26 Oct 2016 16:56:51 GMT

It does seem to work using regex. But here's the sample mail file. thanks.
20161026-114836_1.mail.txt.txt

Re: Custom File Processing?

vapukov — Wed, 26 Oct 2016 21:30:52 GMT

nothing 🙂