topic Re: developer environment specific defaults in tMap: store on disk tempdir in Talend Studio

developer environment specific defaults in tMap: store on disk tempdir

Anonymous — Sat, 16 Nov 2024 14:07:26 GMT

Hi there,
as soon as you put a tMap component in your job, its property TEMPORARY_DATA_DIRECTORY gets populated with a default temp directory (e.g. "C:\Documents and Settings\JohnDoe\Desktop\MyTalendInstallation\workspace\temp"). This works like a charm on the developer's machine. But when you exchange jobs with other developers or deploy jobs on other machines, things start to get messy. Of course, it is well documented in this forum not to forget to set a specific temp dir. But sometimes we are lazy and think that some tMap instances probably won't need "store on disk" and tend to leave the defaults unchanged. At this very moment, we have "stained" a job's properties with developer seat specific information.
Therefore, I suggest a "convention over configuration"-approach. This property should be empty by default. If it is still empty at execution time, the generated code should try to determine a sensible default temp dir (e.g. java.io.File.createTempFile).
Best regards,
Michael

Re: developer environment specific defaults in tMap: store on disk tempdir

Anonymous — Tue, 23 Dec 2008 14:26:59 GMT

Hi,
For all static path or static value properties; Talend best practice is to use context variables to do that.
Context variable will be setted for multiple environment when you deploy and run on a different machine.
For further information about context variables; refer to the Talend Documentation.
Best regards;

Re: developer environment specific defaults in tMap: store on disk tempdir

Anonymous — Mon, 05 Jan 2009 15:43:58 GMT

Hi Christophe Antoine,
I agree with you that context variables should be used, if you want to explicitly configure such a temp dir.
But the situation with TEMPORARY_DATA_DIRECTORY in tMap is different. If I put a tMap component on the job, this property is automagically prefilled. It just works out of the box. But as soon as I copy the job on another machine it stops working.
So I think, Talend should either force the user to set this property explicitly (perhaps with a context variable) if he or she wants to use this functionality. Or it should work implicitly *on every machine*. My problem is, that Talend introduces developer specific information implicitly.
Best regards,
Michael

Re: developer environment specific defaults in tMap: store on disk tempdir

amaumont — Mon, 05 Jan 2009 16:50:58 GMT

What do you think about display the error icon (with a precise message) on the component when the temp folder has the default value or is empty, and "Store on disk" is enabled?

Re: developer environment specific defaults in tMap: store on disk tempdir

Anonymous — Mon, 05 Jan 2009 17:11:36 GMT

Hi amaumont,
showing an error icon is a very good idea.
But depending on the implementation, such an error icon would probably only be displayed on the original developer's machine. If the default directory on my machine is "C:\Documents and Settings\JohnDoe\Desktop\MyTalendInstallation\workspace\temp", how could a Talend instance on another machine determine that this is the default value? The temp directory on another machine could be "/home/janedoe/myTalendInstallation/workspace/temp". This is exactly the reason, why a default value should *not* be materialized. In a perfect solution, this property should be null as a default. This would illustrate that the developer has not yet explicitly set the property. And if you want this feature to work out of the box, Talend could and should automatically use a sensible temp dir (discovered at runtime), if it encounters such a null default.
Best regards,
Michael

Re: developer environment specific defaults in tMap: store on disk tempdir

amaumont — Tue, 06 Jan 2009 08:58:20 GMT

I agree with you, the empty value method is the most sure solution.
Please, add a feature in the Bugtracker.
Thank you for your support.
amaumont

Re: developer environment specific defaults in tMap: store on disk tempdir

Anonymous — Wed, 14 Jan 2009 07:59:22 GMT

Hi there,
sorry for the delay. I have now reported this as a feature request in issue #0006090.
http://www.talendforge.org/bugs/view.php?id=6090
Best regards,
Michael

Re: developer environment specific defaults in tMap: store on disk tempdir

Anonymous — Wed, 15 Apr 2009 19:28:08 GMT

Hi there,
thanks for fixing issue #006090 in version 3.0.3.
Branch 3.1 introduces a similar problem with the component tUniqRow and its new "Use of disk" capability: The property TEMP_DIRECTORY is prefilled with the workspace-directory. I have reported this as issue #0006983
http://www.talendforge.org/bugs/view.php?id=6983
Best regards,
Michael

Re: developer environment specific defaults in tMap: store on disk tempdir

Anonymous — Tue, 21 Apr 2009 10:06:40 GMT

Hi there,
I had a look on the changesets that should fix these bugs. There still seems to be an issue there. Let's have a look on tMap_begin.inc.javajet lines 46 - 50:
http://talendforge.org/trac/tos/browser/trunk/org.talend.designer.components.localprovider/components/tMap/tMap_begin.inc.javajet?rev=21209#L46

                String tempFolder = ElementParameterParser.getValue(node, "__TEMPORARY_DATA_DIRECTORY__");
	                if (tempFolder.equals("")) {
 	                        tempFolder=ElementParameterParser.getValue(node.getProcess(), "__COMP_DEFAULT_FILE_DIR__") + "/temp";
 	                }
 	                String rowsBufferSize = ElementParameterParser.getValue(node, "__ROWS_BUFFER_SIZE__");

"__COMP_DEFAULT_FILE_DIR__" is evaluated during code generation and *not* during runtime of the job. Furthermore, the content of the variable tempfolder is used as java code in line 860, so the string literal would have to be enclosed by quotes.
Thus, we have moved the developer specific information from the .item-file to the .java/.class-file of the job. This does not solve the issue. Unfortunately, it makes such problems even harder to debug.
Instead, code should be generated that determines an appropriate temp path at runtime (e.g., using System.getProperty(?java.io.tmpdir?)).
Best regards,
Michael