topic Re: Split a input file into multiple output files in Talend Studio

Split a input file into multiple output files

una_xikilla — Mon, 18 Sep 2017 12:21:18 GMT

Hello,

I have an input file and I want to make "n" output files using distinct value.

Eg:

Input File

HZARA 123

L123456HY

L98798676

L54354254

HCUSTO456

L98687687

L64746689

I need n different files based on distinct values in Col1 with "H".

HZARA.txt

L123456HY

L98798676

L54354254

HCUSTO.txt

L98687687

L64746689

Any help in this direction would be vey helpful.

Re: Split a input file into multiple output files

TRF — Mon, 18 Sep 2017 12:55:08 GMT

Add a tJavaRow to set a context variable with the filename as soon the first character is H. Add a tFilterRow to exclude rows with a H as a first character and finally call a child job to write rows in the output file according to the context variable.

Re: Split a input file into multiple output files

una_xikilla — Mon, 18 Sep 2017 13:42:12 GMT

Hi!

I'm learning Talend and i don't know how do that in the tJavaRow...

Re: Split a input file into multiple output files

TRF — Mon, 18 Sep 2017 14:24:01 GMT

Here is the main job:

- tFixedFlowInput is here to mimic your input file:

- tJavaRow set the needed context variables (filename and line):

output_row.line = input_row.line;
output_row.type = input_row.line.substring(0, 1);

context.line = input_row.line;

if(output_row.type.equalsIgnoreCase("H"))
	context.filename = input_row.line.substring(1, 5);

- tFilterRow is here to exclude header lines from result:

- tRunJob call the external job called "writer" (check the option "Transmit whole context" on this component to transmit the context variables "filename" and "line" you to define for both jobs)

Here is the external subjob "writer" :

- tFixedFlowInput is here to convert context variable "line" into a flow (required for the next component):

- tFileOutputDelimited is here to write the flow into the current file (designed by filename context variable (check option "Append" to write at the end of the file):

It works.

Re: Split a input file into multiple output files

una_xikilla — Mon, 18 Sep 2017 14:51:46 GMT

Thank you! I'm trying

But I get error -> Error: Could not find or load main class.

Re: Split a input file into multiple output files

TRF — Mon, 18 Sep 2017 14:54:55 GMT

Did you ever run a job successfully with this installation?
If so, please share a capture of your design

Re: Split a input file into multiple output files

una_xikilla — Mon, 18 Sep 2017 15:11:18 GMT

Now it's ok.

But i can't choose "type" in the filter... why?

Re: Split a input file into multiple output files

TRF — Mon, 18 Sep 2017 15:43:52 GMT

Is the tFilter schema synchronized with the tJavaRow schema?

Re: Split a input file into multiple output files

una_xikilla — Tue, 19 Sep 2017 08:18:48 GMT

Hi,

I have synchronized tFilter schema with the tJavaRow schema but the field "type" not...

Re: Split a input file into multiple output files

una_xikilla — Tue, 19 Sep 2017 08:51:32 GMT

Hi again

I have create the out fields but the job says:

Re: Split a input file into multiple output files

TRF — Tue, 19 Sep 2017 09:19:48 GMT

Hi,

You cannot use "!=" operator for a string.

Try to import the attached job.

splitFiles.zip

Re: Split a input file into multiple output files

una_xikilla — Tue, 19 Sep 2017 10:08:41 GMT

Thank you very much!! You are the best!

I have trying do the same job but i have errors. Could you tell me why please?

Pruebas.zip

Re: Split a input file into multiple output files

TRF — Tue, 19 Sep 2017 12:47:33 GMT

I cannot import your job (due to the version).
Can you share components configuration for those you are not sure?

Re: Split a input file into multiple output files

una_xikilla — Tue, 19 Sep 2017 13:14:39 GMT

¡Now work!

I forget to declare the variables in "Contexts"...

Thank you for all (especially for your patience)

Re: Split a input file into multiple output files

TRF — Tue, 19 Sep 2017 13:25:46 GMT

Great!

You're welcome.

Think to select the answer which help you most and mark the question as solved.

Kudos also accepted

Re: Split a input file into multiple output files

premv258 — Tue, 07 Aug 2018 08:30:41 GMT

If you just want to split the input file based on row count. Then Use tFileOutputDelimited.

Go to Advanced setting and tick (Split output in several files) and you can mention number of row you need in each output file.

asas.JPG

Re: Split a input file into multiple output files

mclee — Wed, 05 Sep 2018 08:38:12 GMT

Hi TRF,

I have the similar scenario but only the input is different, where the keywords is after the records. How do I achieve the similar output?

Input File

L123456HY

L98798676

L54354254

HZARA 123 <-- the Hxxxx indicator is after the data

L98687687

L64746689

HCUSTO456 <-- the Hxxxx indicator is after the data

Output just like una:

HZARA.txt

L123456HY

L98798676

L54354254

HCUSTO.txt

L98687687

L64746689

Regards,

MC Lee

Re: Split a input file into multiple output files

TRF — Wed, 05 Sep 2018 08:56:14 GMT

You should have a look to tMemorizeRows / tJavaFlex to solve your case.

Have a look to the tMemorizeRows documentation which contains an example.

Re: Split a input file into multiple output files

billimmer — Thu, 29 Nov 2018 17:32:02 GMT

Hi TRF,

I ran across your reply while working on a similar problem. But there is something I don't understand. Your example works as it should, but if instead of using a subjob for the tFileOutputDelimited, you filter directly to the tFileOutputDelimited, then only one .txt file is generated and the file is named ".txt", as if the context.filename is blank.

What is different about putting the output in a subjob vs. after the filter?

Thanks!

Re: Split a input file into multiple output files

TRF — Thu, 29 Nov 2018 18:17:30 GMT

Because you cannot change the output file from one row to an other, you must break the flow sequence. Another way to do that is to use tFlowToIterate and tIterateToFlow components.