topic Re: tJava to tFileOutputDelimited returns null for initial value in Talend Studio

tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 10:54:09 GMT

Hi,

I have a job which parses a CSV output from an API. I am trying to dynamically set the file name, however, the first result of order number is always null, and then rest is 1 off.

The orderNumber shows correctly using System.out.println() in the tJava, so I'm assuming its something to do with the order that things are executed?

I have attached screenshots of both components & the workflow of the job.

Is there a way around this? I can't extract the order number directly from the JSON because its just a data object with csv inside of it.

Thanks

Re: tJava to tFileOutputDelimited returns null for initial value

TRF — Thu, 29 Aug 2019 11:02:05 GMT

Because tJava is the 1rst component fired when the subjob starts.
Replacing tJava by a tIterateToFlow should be enough to avoid null value for the filename.

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 11:11:54 GMT

Thanks for the quick reply.

The tJava is responsible for using substring to get the filename.

String orderString = row1.data.toString();
String orderNumber = orderString.substring(0, orderString.indexOf(','));

Is there a way to replicate this inside a tIterateToFlow? I tried below but it only returns null. Is this because of how I'm referencing row3.fileName in the tFileOutputDelimited or something else?

Re: tJava to tFileOutputDelimited returns null for initial value

TRF — Thu, 29 Aug 2019 11:20:08 GMT

Can you share the new job design?

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 11:22:10 GMT

Job currently looks like below;

Thanks

Re: tJava to tFileOutputDelimited returns null for initial value

TRF — Thu, 29 Aug 2019 11:36:13 GMT

If you have used default for tFlowToIterate, try to replace row1.data by (String)globalMap.get("row1.data"). If you don't have used default, use your variable name instead of "row1.data" for the globalMap.get.

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 11:57:14 GMT

Hi @TRF

The tFlowToIterate is set to default, I've managed to get the fileName passing through row3 using below;

I can see it using trace debug, but when ALL of the files are created now, they are created as null.csv.

Any idea why it doesn't seem to want to use the value even though we can see it during trace debug?

Re: tJava to tFileOutputDelimited returns null for initial value

TRF — Thu, 29 Aug 2019 12:25:16 GMT

Try to reuse (String)globalMap.get("row1.data") for the tFileOutputDelimited too

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 12:31:21 GMT

if I put;

File Name: "C:/Users/user/Desktop/Ax_Orders/" + (String)globalMap.get("row1.data").toString().substring(0, globalMap.get("row1.data").toString().indexOf(',')) + ".csv"

inside of the tFileOutputDelimited then it just creates one CSV, with the name of 27.csv. Interestingly, it now seems to only be evaluating this once rather than once per row.

I need a CSV per row which was the original reason for trying to use tJava to handle the row & filename

Re: tJava to tFileOutputDelimited returns null for initial value

TRF — Thu, 29 Aug 2019 12:58:18 GMT

My fault.
This design works:

Fill in the fields in tJavaFlex and set filename using global variables from tFlowToIterate.

In my case I use (String)globalMap.get("row17.data").

tFixedFlowInput is there to replace your tExtractJSONFields (4 rows with 1 field each = 01, 02, 03, 04).

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 13:10:53 GMT

Doesn't this have the same issue as before? The tJavaFlex returns the following files..

instead of 1 to 27.

What does your file name look like in the job above? Mine is currently set to "C:/Users/users/Desktop/Ax_Orders/" + row2.fileName + ".csv"

Re: tJava to tFileOutputDelimited returns null for initial value

TRF — Thu, 29 Aug 2019 13:14:19 GMT

The value for the 27th row is probably null

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 13:20:02 GMT

It's not, I can see the data clearly using postman, and if I open the file which has been saved as null.csv, the row starts as below...

Is this not the same problem that we had at the start, where you said that tJava runs first?

I'm open to changing the layout of the job I just need a resolution, I don't feel like this should be as hard as its proving to be!

Thank you for your help so far, I appreciate it

edit; the first row it coming through as "null" and then the following ones are a number off, eg 1.csv SHOULD be 2 if you do the substring manually

Re: tJava to tFileOutputDelimited returns null for initial value

TRF — Thu, 29 Aug 2019 14:47:18 GMT

"Is this not the same problem that we had at the start, where you said that tJava runs first?"

No, when using tJavaFlex, initial code starts first and only once, main code starts for each row and final code starts last and only once.

Well... can you share your JSON?

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 15:35:44 GMT

Here's a snippet of the JSON, the first two records (of the 27 which were being used previously)

{
    "message": "Fetched order(s)",
    "data": [
        "1,,,,,,,,,,,\"Admin Test\",\"11 Avenue\",City,UK,,POSTCODE,,2019-07-26T09:39:21+00:00,,,,,,,,,,,,,,2019-07-26T09:39:21+00:00,Y,,,,,Y,,,,,,,,,,,,\n1,,DELIVERY,DELIVERY,1,,,5.00,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n2,,SUPPu18024,AXb98017,1,,,12.54,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n3,,SUPPLEFTSLEEVE,LEFTSLEEVE,1,,,2.50,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n4,,LOGOSETUPFEE,LOGOSETUPFEE,1,,,10.00,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n",
        "2,,,,,,,,,,,\"Admin Test\",\"11 Avenue\",City,UK,,POSTCODE,,2019-07-26T09:39:21+00:00,,,,,,,,,,,,,,2019-07-26T09:39:21+00:00,Y,,,,,Y,,,,,,,,,,,,\n1,,DELIVERY,DELIVERY,1,,,5.00,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n2,,SUPPu18024,AXb98017,1,,,12.54,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n3,,SUPPLEFTSLEEVE,LEFTSLEEVE,1,,,2.50,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n4,,LOGOSETUPFEE,LOGOSETUPFEE,1,,,10.00,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n"
    ]
}

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 16:01:26 GMT

It looks like this is caused by your filename naming method. You say you have used this....

"C:/Users/users/Desktop/Ax_Orders/" + row2.fileName + ".csv"

@TRF used this....

"C:/Users/users/Desktop/Ax_Orders/" + ((String)row2.fileName) + ".csv"

If this fixes the problem please accept @TRF's solution

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 16:10:17 GMT

Unfortunate this still has the same issue. The first row is exported as null.csv, second row is 1.csv.. etc.

They are all off by one number

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 16:20:59 GMT

Can you take a screenshot of your job, your tJavaFlex and your file component settings please?

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 16:28:29 GMT

No problem.. A few of the row names have changed since this morning but the job is still the same. See below;

Full job,

tFlowToIterate;

tJavaFlex;

OutputDelimited;

Re: tJava to tFileOutputDelimited returns null for initial value

Anonymous — Thu, 29 Aug 2019 16:48:57 GMT

OK, I see the issue. It's caused by the way in which file components work. When you iterate, everything after the iterate link is treated as a subjob in its own right. A file component is initiated at the beginning of a subjob....before any data has been processed. So, as you are creating your filename at the beginning of the subjob, the file component for the first run is not given a name (the data has not been processed by the tJavaFlex component. The way to get around this is to put a tJava before your tJavaFlex and link to the tJavaFlex with an iterate link. You there therefore have two iterate links; one before your tJava and one before your tJavaFlex. In the tJava component put the following code.....

String orderString = ((String)globalMap.get("extractedDate.data"));
String orderNumber = orderString.substring(0, orderString.indexOf(','));
globalMap.put("filename", orderNumber);

Then you can either leave the tJavaFlex data exactly the same (maybe remove the filename bit) and use this in your filename field....

"C:/Users/users/Desktop/Ax_Orders/" + ((String)globalMap.get("filename")) + ".csv"

That should do it.

Again, this is what @TRF was suggesting, so it is probably only fair to give him the solution 🙂