topic Re: How to stop processing file or switch mid job in Talend Studio

How to stop processing file or switch mid job

mobmsc2 — Thu, 08 Jun 2017 14:49:39 GMT

Using open studio 6.4 data integration is it possible to stop processing a file midway through a job if certain conditions are met? I need to re-design a talend job which attempts in ingest excel files but where the structure of the data can change between files.

At the moment my job attempts to recover from these changes by re-reading the file in an attempt to match the ingested file against another schema but this way seems to end with me repeatedly executing components x times where x is the number of rows incorrectly matching the schema.

I am ingesting my data as strings but I'm finding it difficult to understand the best way of handling working with data where the data within a file is consistent when compared to a schema but not necessarily consistent across files

Re: How to stop processing file or switch mid job

Anonymous — Thu, 08 Jun 2017 14:59:43 GMT

This might help you figure out a way to identify/handle the switching schema

https://www.rilhia.com/tutorials/dynamic-column-order

Re: How to stop processing file or switch mid job

mobmsc2 — Fri, 09 Jun 2017 13:35:23 GMT

The problem is I can't be sure the column names would stay the same, some reports have more columns than others, a column may disappear for a few months then return back

Is there a way to break out of/away from processing a particular flow and start another?

Re: How to stop processing file or switch mid job

Anonymous — Mon, 12 Jun 2017 00:37:03 GMT

Hi,

In tjavarow component, you can check the condition and if condition occurs, you can exit there only using System.exit statement. your job will fail.