topic Re: tFileOutput, get filename used in Talend Studio

tFileOutput, get filename used

Anonymous — Sat, 16 Nov 2024 05:25:15 GMT

Hi, after having created a file with tFileOutputPositional, how do i access the filename used? When writing the file i use a unique name, 'C:/in/work/" + (String)globalMap.get("tFileList_1_CURRENT_FILE") + "_" + java.util.UUID.randomUUID()', and the plan is to OnSubJubOK rename the file with tFileCopy. But i cannot find the filename used by tFileOutputPositional anywhere. In the 6.3 documentation it seems that it was added to the globalMap but I'm using 7.1 and it no longer seems to be apart of it.

Re: tFileOutput, get filename used

iamabhishek — Mon, 01 Jul 2019 15:05:00 GMT

Before passing the data to tFileOutputPositional you could process it through tJavarow like -

//Code generated according to input schema and output schema
output_row.newColumn = input_row.newColumn;
output_row.newColumn1 = input_row.newColumn1;
output_row.newColumn2 = input_row.newColumn2;

and after that you could use your logic to derive the filename and store it in context or globalMap which you could use in the tFileOutputPositional - "File Name" property as well in the tFileCopy in a different subjob.

String uuid = UUID.randomUUID().toString();
String filename = "C:/Talend/"+ uuid +"/out.txt";
globalMap.put("filename", filename);
System.out.println("Filename: "+ (String)globalMap.get("filename"));

Re: tFileOutput, get filename used

Anonymous — Mon, 01 Jul 2019 15:09:02 GMT

Have tried some similar to that, but couldn't get it to work as the filename part of tFileOutputPositional is initialized on subjob start. So directly after my tFileList component has run, the filename of tFileOutputPositional is set :/.

Re: tFileOutput, get filename used

Anonymous — Mon, 01 Jul 2019 15:31:23 GMT

Hi @anpe ,

Instead of onSubjob Ok from tFileLIst you should put OnComponent Ok from the component just befor tFileCopy.

As you are using OnSubjobOk from tFileList, the CURRENT_FILE globalMap will be flushed, hence you will get null value in that.

On Component Ok link from the component before tFileCopy will still keep the globalMap of tFileList till the execution is finished from tFileCopy and the second iteration will start from tFileList then only.

Thanks and Regards,

Subhadip

Re: tFileOutput, get filename used

Anonymous — Mon, 01 Jul 2019 15:39:19 GMT

Hi subhadip13, think you have misunderstood my problem. The problem is not the CURRENT_FILE variable. Put the filename that tFileOutputPositional use. My job can be seen below. It polls a folder and then does some mapping changes before writing a new file to disk. After that I want to move the newly created file, but need to know what the filename used was, as it is randomly set.

Re: tFileOutput, get filename used

Anonymous — Mon, 01 Jul 2019 16:36:43 GMT

Hi @anpe ,

Sorry if I have misunderstood the problem.

You can do an "OnComponentOk" from tFileOutputPositional and use that link to tFileCopy.

Thanks and Regards,

Subhadip

Re: tFileOutput, get filename used

iamabhishek — Mon, 01 Jul 2019 16:45:08 GMT

Based on the screenshot of the job shared, I think you could complete the flow in a single sub job where you could use tFileCopy after using tFileOutputPositional in that case you would be able to get/store the file names for every iteration.

Re: tFileOutput, get filename used

Anonymous — Tue, 02 Jul 2019 08:37:40 GMT

Ï don't see how that helps, tFileOutputPositional still doesn't output the filename it uses. So why would it help to connect the tFileCopy to it directly?

Re: tFileOutput, get filename used

Anonymous — Tue, 02 Jul 2019 10:10:54 GMT

For now i ended up in below solution, adding a tJava after tFileList to set the filename and add it to the globalMap (thus before tFileOutputPositional is initialized). It works, but is not exactly what I want. The current solution doesn't allow me to use any data inside the file when setting the filename, which I might want in the future.

Re: tFileOutput, get filename used

iamabhishek — Tue, 02 Jul 2019 11:43:04 GMT

That's exactly what I had suggested at the beginning to use tJavarow to derive the filename and use it later for your tFileOutputPositional task and also to have the tFileCopy immediately after and not in separate subjob.
Regarding your "use any data inside the file when setting the filename" - this could be done after your file is written, by reading the same file not sure though what you had meant apart from this.