topic Re: process multiple files in Talend in Talend Studio

process multiple files in Talend

Anonymous — Mon, 14 Nov 2016 08:22:37 GMT

can someone tell how to process multiple files in Talend and store the output in a oracle table??
I tried using tfileList component but i was unable to insert data into table.
My file extension is also ".unl" which no file related component is reading data..
So could someone help me with this??

Re: process multiple files in Talend

Anonymous — Mon, 14 Nov 2016 09:59:35 GMT

Hi,
Can you convert your .unl files to flat files(excel) and then use tfilelist to achieve your goals?
The work flow looks like:tfilelist-->tfileinputexcel-->output
Best regards
Sabrina

Re: process multiple files in Talend

vapukov — Mon, 14 Nov 2016 10:15:20 GMT

most important not extension, but what inside
open it with any flat text editor - notepad++, textedit and etc, than You (and we are) will have idea - what You try to read?

Re: process multiple files in Talend

Anonymous — Tue, 15 Nov 2016 09:17:51 GMT

Hi,
Thanks for the replies.
I should not change the file format as it is production environment.What ever the .unl files I get in the directory differed by a sequence number I should read them and process them and store the required fields in the oracle table as output.
The data in the .unl file is pipe delimited (for example : 12345|active||P|hello) like this.
But when I iterate the files in the directory its reading only first file and not reading other files.
The sample file names in the soure directory will be like
sample20161115_SOMECODE1_SEQNUMBER.unl
sample20161115_SOMECODE2_SEQNUMBER.unl,etc
So how can i iterate??I mean what filemask should i keep in tfilelist component??
Any help would be greatful

Re: process multiple files in Talend

vapukov — Tue, 15 Nov 2016 10:13:07 GMT

of course - tFileList (use as pattern "*.unl") -> tFileInputDelimited -> tMap ->output
all must work as switzerland watches

Re: process multiple files in Talend

Anonymous — Wed, 16 Nov 2016 07:31:49 GMT

Hi,
Thanx for the reply.
But still my problem was not resolved.
I have three files in a directory
mon20161113_103_941462.unl
mon20161113_103_941463.unl
mon20161113_103_941464.unl
But when I gave filemask as "mon20161113_103_*.unl" in tfilelist and in tfileinputdelimited I created metadata with mon20161113_103_941462.unl file.
But when i run my job its showing 3 execs finished but when i see output the first file data is only stored 3 times but not 2 and 3 files.
Please help me with this.

Re: process multiple files in Talend

vapukov — Wed, 16 Nov 2016 08:27:59 GMT

it mean - You have error somewhere also 😉
we of course can try to guess, but will be much better if You show Your Job ... much more easy
without information, just trust me - it work! even with hundreds of files, sure simple error

Re: process multiple files in Talend

Anonymous — Wed, 16 Nov 2016 08:50:59 GMT

Attached is the job...pls help me with this

Re: process multiple files in Talend

Anonymous — Thu, 17 Nov 2016 10:06:40 GMT

Hi,
We can't see the screenshot on our side. Could you attach it on the forum, please? That would be great.
Screenshots works only if you drag&drop the image directly in the editor window.
Best regards
Sabrina

Re: process multiple files in Talend

Anonymous — Fri, 18 Nov 2016 06:34:10 GMT

PFA

Re: process multiple files in Talend

Anonymous — Fri, 18 Nov 2016 08:01:37 GMT

HI,
Thanks.Faced some issue while uploading

Re: process multiple files in Talend

Anonymous — Fri, 18 Nov 2016 08:21:07 GMT

Hi,
Can you click "UPLOAD" button to attach your screenshot?
Best regards
Sabrina