System.getenv() when run Job from TAC

[vapukov]

Hi,

with community edition we are use System.getenv() for retrieve OS variables, it work fine everywhere:

- windows, linux, macOS 

- from Studio

- from shell

also same function work under Talend ESB Runtime as expected

but when Job implemented on TAC, this function stop work and return null (in Studio for subscription version - all fine)

variables defined in:

/etc/environment (normally for Community and Runtime)

.bachrc (duplicated when try to resolve)

setenv.sh (duplicated when try to resolve)

server of course was restarted after any changes

look like TAC start processes without passing environment, any recommendations how to fix this would be pretty appreciated !

Thanks, Vlad

[vapukov]

and one more test, whoami from TAC:

[INFO ]: sugarcrm.testenvvariables_0_1.testEnvVariables - tSystem_1 - Command to execute: 'whoami '.
[INFO ]: sugarcrm.testenvvariables_0_1.testEnvVariables - tSystem_1 - The command has been executed successfully.
talenduser

little more extend example:

prepare .sh file:

#!/bin/sh

whoami
echo $talend_env
echo $talend_context_path

run from talenduser:

root@talend-prod-02:/opt/test/testEnvVariables# su talenduser
talenduser@talend-prod-02:/opt/test/testEnvVariables$ /opt/test/testEnvVariables/check_env.sh
talenduser
prod
/opt/talenduser/config/

 run from TAC:

[INFO ]: sugarcrm.testenvvariables_0_1.testEnvVariables - tSystem_2 - The command has been executed successfully.
talenduser


[statistics] disconnected

and we have:

- user correct and same

- job run from proper username

- user have permission for file

- but JobServer - do not pass OS variables to this process

all variables (printenv) from run with JobServer:

UPSTART_INSTANCE=
SHLVL=2
OLDPWD=/opt/TalendTools-6.4.1/jobserver/agent
runlevel=2
UPSTART_JOB=rc
_=./start_rs.sh
TERM=linux
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
RUNLEVEL=2
PREVLEVEL=N
UPSTART_EVENTS=runlevel
TALEND_HOME=/opt/TalendTools-6.4.1
PWD=/opt/TalendTools-6.4.1/jobserver/agent/TalendJobServersFiles/repository/SUGARCRM_testEnvVariables_20180606_125412_CWdfx/testEnvVariables
JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre
previous=N

[fdenis]

it's a good question, but you are using an an ETL software. So you are accepting some abstraction level. Talend is generating Java ,and give you the possibility to add java code into Talend job's. environment variables exist to share common data on a lower execution level. if you want to respect ETL minding think about data not about java,

if you want to have local variable on each server, you can use load local context file.

but you cannot say Talend is not working. Talend is not an java IDE but is using a java based IDE   

Regards

[vapukov]

@fdenis wrote:

it's a good question, but you are using an an ETL software. So you are accepting some abstraction level. Talend is generating Java ,and give you the possibility to add java code into Talend job's. environment variables exist to share common data on a lower execution level. if you want to respect ETL minding think about data not about java,

if you want to have local variable on each server, you can use load local context file.

but you cannot say Talend is not working. Talend is not an java IDE but is using a java based IDE   

Regards

Thank You

for all time working with Talend, I do not remember case when I tell something similar
 

but in any case - I try to find the way, how to achieve what I want?
I don't try to find how to achieve this by other ways  - I now them.

I just use this construction long time, and I have it many projects pre-defined, and I want be sure - this is not possible with TAC,

Plus some other scripts connected to Talend (not my) - also use same or similar variables, and I do not want ask 3rd party developers to change  this without 100% sure - it is no other ways.

What I expect to find:

- or confirmation - this is not work art all, but as You can read above - Richard report he use this construction with TAC, so what difference and what wrong with my TAC? (karaf is also top level java container - but it work with ebb runtime)
- or some tricks - how to achieve this, like may be need edit some JobServer settings for mange him

[Anonymous]

I believe this is actually a Linux *feature* rather than a Talend bug after some reading. The quickest piece to look at (for getting on the right track of the problem) can be found here: https://serverfault.com/questions/374404/service-command-and-environment-variables

I recalled that I did have an issue with environment variables a long time back, but the sys-admin guys fixed it for me pretty quickly by "sorting the service initialisation" or something to that effect. My assumption was that they had set it up against the wrong user (a situation I have seen on many an occasion on Windows systems). I suspect that the issue was that the environment variables were stripped out by the service command.

The reason this does not happen with the Karaf (ESB Runtime) is that this is run as a daemon process and I suspect that is the case for Tomcat (TAC) as well. 

Sorry if it sounded like I wasn't paying attention to what you were saying or that I didn't believe you, I was just trying to rule out everything I could think of first. It is kind of hard to resolve these sorts of things without access to "see for yourself". Hopefully this will help you find a workaround. 

I'd also like to say there is absolutely nothing wrong with using Java with Talend. In fact I would argue that used correctly, it is one of the biggest pros of using Talend. There is NOTHING I cannot interact with using Talend because of the ability to build in other people's APIs. The trick is to get the usage right and don't reinvent the wheel.....unless the wheel (contexts in this case) is already pretty broken of course 🙂

[vapukov]

Thank You Richard - no any issues

troubleshooting is always pain and any idea is useful

most important to know and be sure - at least one person have expected result ... that all, its mean solution is somewhere near just need to find it

I also ask my system admin to check, may be we found what wrong

and other case - when just have answer "it is not possible, it not supported" ... this is also answer 

thanks again, regards, Vlad

[vapukov · Accepted solution]

ok, so "partial" solution

"partial" - because I fix the issue, but right now I have not time to check - what exactly step was only needed

1. JobServer start as not interactive process (.sh script), and for not interactive processes OS environment do not pass

2. for this reason I add variables into:

• /etc/environment
• ~/.bashrc
• jobserver/agent/start_rs.sh
• /etc/bash.bashrc
• ~/.bash_profile

this is not proper way, just because I have not time - I add variables in any place where it could be taken. proper do it 1 by 1 and reboot after each change for check, but I was very limited in time.

as result, all work as expected:

.----------+-----------------------+---------------------------------------.
|                                tLogRow_1                                 |
|=---------+-----------------------+--------------------------------------=|
|talend_env|talend_context_path    |context_filepath                       |
|=---------+-----------------------+--------------------------------------=|
|prod      |/opt/talenduser/config/|/opt/talenduser/config/context_prod.csv|
'----------+-----------------------+---------------------------------------'