System.getenv() when run Job from TAC

[vapukov]

Hi,

with community edition we are use System.getenv() for retrieve OS variables, it work fine everywhere:

- windows, linux, macOS 

- from Studio

- from shell

also same function work under Talend ESB Runtime as expected

but when Job implemented on TAC, this function stop work and return null (in Studio for subscription version - all fine)

variables defined in:

/etc/environment (normally for Community and Runtime)

.bachrc (duplicated when try to resolve)

setenv.sh (duplicated when try to resolve)

server of course was restarted after any changes

look like TAC start processes without passing environment, any recommendations how to fix this would be pretty appreciated !

Thanks, Vlad

[vapukov]

we use of course

but question about - how to read OS variables

[Anonymous]

Does your job work on the TAC if you manually run it via the command line on that machine? For example, compile on Studio and take the zip, explode it and run it using the script file provided. Is that able to retrieve the system environment variables?

[vapukov]

---

@rhall wrote:

I use this routine code in my Implicit Context settings (they can run code as well as accept hardcoded values) and within my jobs sometimes....although I tend to load everything from the Implicit Context if it is suitable.

    /**
     * getEnvironmentVariable: used to retrieve Environment Variables
     * 
     * {talendTypes} String
     * 
     * {Category} Implicit Context Load
     * 
     * {param} string("TalendContextPassword") variableName: the parameter name to be returned

     * 
     * {example} getEnvironmentVariable("TalendContextPassword") # returns "My Password"
     */
	public static String getEnvironmentVariable(String variableName) {
		String returnVal = System.getenv(variableName);

		if (returnVal == null) {
			System.out.println(variableName
					+ " does not exist or holds no value");
		}

		return returnVal;
	}

This all works without fault and n my current project I have had no problems with this across 3 environments for the last 6 months. As I said, there is nothing in Talend stopping this from working. It is an implementation problem.

---

I use similar construction (little different, but still - it work last 2.5 year) ... until this case

Tell me please - what implementation problem it could be? what could restrict read settings available for 100% users on this server?

[vapukov]

---

@rhall wrote:

Does your job work on the TAC if you manually run it via the command line on that machine? For example, compile on Studio and take the zip, explode it and run it using the script file provided. Is that able to retrieve the system environment variables?

---

thank You, will test and this also

[vapukov]

---

@rhall wrote:

Does your job work on the TAC if you manually run it via the command line on that machine? For example, compile on Studio and take the zip, explode it and run it using the script file provided. Is that able to retrieve the system environment variables?

---

all work from ssh - as it must

and not correct from TAC:

and again correct from karaf:

[Anonymous]

Did you say that your TAC, Job Server and Runtime are all running on the same server? 

Also, where is it getting the context_filepath from? That is the same in all 3 tests.

[vapukov]

---

@rhall wrote:

Did you say that your TAC, Job Server and Runtime are all running on the same server? 

Also, where is it getting the context_filepath from? That is the same in all 3 tests.

---

yes - all 3 on same server

yes and no - this is different case (and as You can see - on TAC - variable is NULL, not the same)

on each server and/or machine defined 2 variables:

- talend_env - dev/prod/local

- talend_context_path - base folder for talenduser config files, it could be not only context file, it could be templates for emails, some other files and etc

if variables defined, we have combined name for current config file:

• /opt/talenduser/config/context_prod.csv
• /opt/talenduser/config/context_local.csv
• /opt/talenduser/config/context_dev.csv

if variables - not defined (or not accessible) - default context file:

• /opt/talenduser/config/context_dev.csv

[Anonymous]

Can you add a call to retrieve the PATH variable in your job? This should exist for all your users. My suspicion is that this will be returned. If that is the case it suggests that the user running the job (or the Job server) does not have access to those variables. Sorry to go back to this, but that can be the only reason for this. The Java that works in the ESB is exactly the same (I assume) as the Java that is running in the job in the job server and on the Studio. All the Job Server does is start the job. The job runs in its own JVM as essentially a standalone application. The only thing that can possibly cause this is a permissions/user issue. It is incredibly easy to have different users running Talend services, particularly on Linux.

[vapukov]

PATH is work

and much more interesting - it work definitely from /etc/environment: (edited) ... it strange

root@talend-prod-02:/opt/test/testEnvVariables# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/TalendTools-6.4.1/mongodb/bin
root@talend-prod-02:/opt/test/testEnvVariables# su talenduser
talenduser@talend-prod-02:/opt/test/testEnvVariables$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
talenduser@talend-prod-02:/opt/test/testEnvVariables$ cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
talend_env="prod"
talend_context_path="/opt/talenduser/config/"

so, it read /etc/environment ... it take it from ...

PATH from /etc/environment:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PATH under talenduser shell:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PATH from TAC
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games|

[vapukov]

AND TAC - do not show HOME

----------+---=|
|talend_env|talend_context_path|context_filepath                      |system_path                                             |home|
|=---------+-------------------+--------------------------------------+--------------------------------------------------------+---=|
|null      |null               |/opt/talenduser/config/context_dev.csv|/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games|null|

it must be:

|talend_env|talend_context_path    |context_filepath                        |system_path                  |home           |
|=---------+-----------------------+----------------------------------------+-----------------------------+--------------=|
|local     |/opt/talenduser/config/|/opt/talenduser/config/context_local.csv|/usr/bin:/bin:/usr/sbin:/sbin|/Users/vladimir|
'----------+-----------------------+----------------------------------------+-----------------------------+---------------'

 (samples for local) 

Warning: Parameter "dwh_export_path" is a new parameter of Implicit_Context_Context
.----------+-----------------------+---------------------------------------+----------------------------------------------------------------------------------------+----------------.
|                                                                                     tLogRow_1                                                                                      |
|=---------+-----------------------+---------------------------------------+----------------------------------------------------------------------------------------+---------------=|
|talend_env|talend_context_path    |context_filepath                       |system_path                                                                             |home            |
|=---------+-----------------------+---------------------------------------+----------------------------------------------------------------------------------------+---------------=|
|prod      |/opt/talenduser/config/|/opt/talenduser/config/context_prod.csv|/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games|/home/talenduser|
'----------+-----------------------+---------------------------------------+----------------------------------------------------------------------------------------+----------------'