Skip to main content
Announcements
A fresh, new look for the Data Integration & Quality forums and navigation! Read more about what's changed.
cancel
Showing results for 
Search instead for 
Did you mean: 
vapukov
Master II

System.getenv() when run Job from TAC

Hi,

 

with community edition we are use System.getenv() for retrieve OS variables, it work fine everywhere:

- windows, linux, macOS 

 

- from Studio

- from shell

 

also same function work under Talend ESB Runtime as expected

 

but when Job implemented on TAC, this function stop work and return null (in Studio for subscription version - all fine)

 

variables defined in:

/etc/environment (normally for Community and Runtime)

.bachrc (duplicated when try to resolve)

setenv.sh (duplicated when try to resolve)

 

server of course was restarted after any changes

 

look like TAC start processes without passing environment, any recommendations how to fix this would be pretty appreciated !

Thanks, Vlad

Labels (4)
1 Solution

Accepted Solutions
vapukov
Master II
Author

ok, so "partial" solution

"partial" - because I fix the issue, but right now I have not time to check - what exactly step was only needed

 

1. JobServer start as not interactive process (.sh script), and for not interactive processes OS environment do not pass

2. for this reason I add variables into:

  • /etc/environment
  • ~/.bashrc
  • jobserver/agent/start_rs.sh
  • /etc/bash.bashrc
  • ~/.bash_profile

this is not proper way, just because I have not time - I add variables in any place where it could be taken. proper do it 1 by 1 and reboot after each change for check, but I was very limited in time.

 

as result, all work as expected:

.----------+-----------------------+---------------------------------------.
|                                tLogRow_1                                 |
|=---------+-----------------------+--------------------------------------=|
|talend_env|talend_context_path    |context_filepath                       |
|=---------+-----------------------+--------------------------------------=|
|prod      |/opt/talenduser/config/|/opt/talenduser/config/context_prod.csv|
'----------+-----------------------+---------------------------------------'

View solution in original post

25 Replies
Anonymous
Not applicable

I suspect this is because your job servers are not located on your TAC machine and/or you do not have the system environment variables set up on all of your job servers. The jobs do not take any environment settings from the TAC since they do not run on the TAC (unless you have a job server on there as well). When you use a virtual job server (made up of several job servers) you will need to have these variables set up on ALL job servers or the jobs will only work intermittently.

vapukov
Master II
Author

it is single server installation - all on same server 0683p000009MACn.png only SVN on remote

Anonymous
Not applicable

Then this is likely to do with permissions and OS environment variable settings. Which user is running your services (Talend Job Services, etc) and are the environment variables configured for them? What you are trying to do certainly works. I use this all of the time. However it can get frustrating trying to track down what is wrong sometimes. Can you echo the variables from the command line on the machine using the user which runs the Job server service. That is the first thing to try.

vapukov
Master II
Author

- /etc/environment - for all users

- talenduser - /home/talenduser/.bashrc

this user from which Talend start

 

same user used for run ESB Runtime with only /etc/environment settings - and function work from OSGI jobs

 

P.S.

of course echo from Talenduser show correct variables, it was first what tested

Anonymous
Not applicable

This is strange.

 

"look like TAC start processes without passing environment, any recommendations how to fix this would be pretty appreciated !" -  This is not the case as environment variables are never passed to a Talend job. Using the code you have demonstrated shows you are requesting them from within an already started job. This is an issue with the Java and/or (more likely) your server config. TAC has absolutely no effect on this. It literally just passes the executable to the appropriate jobserver and starts the job. That is all. I suspect your Jobserver service is not running under the user you suspect it is. You can test this by creating a job which uses a tSystem control to try your echo statements.

 

fdenis
Master

for me, environment variables are load on logon.

TAC is used to Schedule background tasks on multiples runtime servers. you did not need to be logged to run task on Talend runtime environment. Local service have to run but independent sub processes did not need any environment variables.

fdenis
Master

With tac the best way is to use context variables.
Anonymous
Not applicable

I use (and have done for a few years, across versions from 5 forward) a procedure where my implicit context load (for DI jobs) retrieves context variables from a database. So that there is absolutely no difference required from first DEV compilation, through QA to PROD, I configure each of my environments with context database credentials using system variables. This means I can configure each of my environments on creation and do not have to touch my jobs again no matter which environment they run on. I only use the Default context because having multiple contexts will end up with issues no matter how thorough you think you....especially if there is a team of developers. I do this with my ESB work as well, but use PropertyPlaceholder to receive my "context"" variables so that I have exactly the same values across the board. I also add another layer of complexity in many environments as many of my customers do not like human readable values to be stored even in environment settings. For this I use JASYPT. I do not experience this issue unless service users are incorrect OR there is a new Job Server that has not been configured. I have used this on Windows and Unix and apart from the configuration of the servers, the implementation is exactly the same.

 

I use this routine code in my Implicit Context settings (they can run code as well as accept hardcoded values) and within my jobs sometimes....although I tend to load everything from the Implicit Context if it is suitable.

 

    /**
     * getEnvironmentVariable: used to retrieve Environment Variables
     * 
     * {talendTypes} String
     * 
     * {Category} Implicit Context Load
     * 
     * {param} string("TalendContextPassword") variableName: the parameter name to be returned

     * 
     * {example} getEnvironmentVariable("TalendContextPassword") # returns "My Password"
     */
	public static String getEnvironmentVariable(String variableName) {
		String returnVal = System.getenv(variableName);

		if (returnVal == null) {
			System.out.println(variableName
					+ " does not exist or holds no value");
		}

		return returnVal;
	}

This works for running on my DEV Studio (my machine is configured with environment variables on Windows), running jobs from the command line (Windows or Linux) and running by starting on TAC. To add even more of a potential for problems to this mix, I automatically load tasks into the TAC using my ESB runtime (when a file is dropped in a folder and the required job is not yet live) by using the MetaServlet API to find the required job in my Nexus and load to the TAC. No contexts are set up in the TAC because this process renders doing that unnecessary. It also enables me to have new jobs that supersede already existing jobs to be used as soon as they are logged in my background database as being the "live" job.

 

This all works without fault and n my current project I have had no problems with this across 3 environments for the last 6 months. As I said, there is nothing in Talend stopping this from working. It is an implementation problem.

 

 

vapukov
Master II
Author


@rhall wrote:

This is strange.

 

TAC has absolutely no effect on this. It literally just passes the executable to the appropriate jobserver and starts the job. That is all. I suspect your Jobserver service is not running under the user you suspect it is. You can test this by creating a job which uses a tSystem control to try your echo statements.

 


thank You, sometime brain storm help 0683p000009MACn.png (not in this case :-))) )

 

just for close all ideas about users

 

file /etc/environment - affected for all users 

https://stackoverflow.com/questions/13046624/how-to-permanently-export-a-variable-in-linux