topic Re: How to include a file in Project Resource in Talend Studio

How to include a file in Project Resource

Sanjay5 — Tue, 11 Jul 2017 20:08:53 GMT

Hi,

I want to include a file which i am referring in my Talend Job.

I created a talend Routine which uses a file, I have loaded the file to routine along with jar file.

Now when i run the job in Local mode, Job is able to fetch the data from file, but when i run in job Server, It does not get the file.

help needed to resolve the issue

How to include the file in my project resources for remote run?

Re: How to include a file in Project Resource

Anonymous — Wed, 12 Jul 2017 07:12:50 GMT

Hi
It does not support relative file path. You can have a shared directory (through NFS or SMB mount) to have all necessary files accessible for all servers where your job runs.

Regards
Shong

Re: How to include a file in Project Resource

Sanjay5 — Fri, 14 Jul 2017 18:28:02 GMT

Hi Shong,

How to point my job to look for the files which are needed for Project Resource?

In other words, How my job will understand the location of the file.

Re: How to include a file in Project Resource

Sanjay5 — Fri, 14 Jul 2017 18:36:32 GMT

Please find attached email for detailed explanation on what is the issue i am trying to resolve

Question on Project Resource location for Talend Job.zip

Re: How to include a file in Project Resource

vapukov — Sat, 15 Jul 2017 08:31:38 GMT

You can change path to files for context variables

Spoiler

context.path_to_files+"/file_name1.csv"
context.path_to_files+"/file_name2.xls"

and etc

with implicit context loading it allow refer to different path on different servers

Re: How to include a file in Project Resource

Sanjay5 — Tue, 18 Jul 2017 21:40:56 GMT

Thanks for the information.

I am not looking to share the files between different servers.

I want to access a file which is located in Java Project Resource folder. (Same as Eclipse)

See my attached screenshot. (File Name is vmkeys.xml)

I want to access this file when i run the job in JobServer. Not sure where should i keep the file.

Eclipse Screen shot.png
Question on Project Resource location for Talend Job.zip

Re: How to include a file in Project Resource

Anonymous — Wed, 19 Jul 2017 03:08:18 GMT

This functionality exists for ESB services since we have resources there. In DI, it doesn't. You will need to put the file somewhere on disk on your jobserver, and read the file as the first operation of your job.

Re: How to include a file in Project Resource

vapukov — Wed, 19 Jul 2017 03:14:05 GMT

I also mean - not share file between servers, but have unique location for same file on different environment

for example - when Dev machine windows, and Prod Server Linux it is hard to have same PATH to resources

and context variables - help to resolve this

Re: How to include a file in Project Resource

Sanjay5 — Wed, 19 Jul 2017 06:15:07 GMT

My Java code routine in the job automatically searches for the file "vmkeys.xml" in project resources folder:

I.e. C:\Talend\workspace\.Java\src\main\resources

In my local workspace file is located under below path

C:\Talend\workspace\.Java\src\main\resources\vmkeys.xml

I do not have to specifically change anything in my job to read data from this file, for local run.

Now for remote run, If i keep the file in job server in below location

/ABC/vmkeys.xml

I am not sure how i change my job to look for this file in any other location in job server:

I am not sure how to make this work, I believe in Eclipse as well files are read from "Project Resources" folder.

Any suggestions would be really helpful.

Re: How to include a file in Project Resource

vapukov — Wed, 19 Jul 2017 13:36:28 GMT

what is the problem with context variables?

no one fixed value, in properties:

and csv file on each machine:

that all
when Job start, it read OS variable, it it not NULL - take config file from there, if not from default (for me) place

Configuration could be changed without re-build the Job

Re: How to include a file in Project Resource

Sanjay5 — Wed, 19 Jul 2017 18:13:07 GMT

Thanks for detailed explanation,

I think, I am not able to make my issue clear:

I have a xml file named vmkeys.xml, which is required for my routine.

My routine code is accessing this file when ever i run my job.

I think my code routine always expects the file to be at Java Project Resources folder

I.e. C:\Talend\workspace\.Java\src\main\resources

If i remove the file from above directory my routine will fail

Now if i keep my file to any other directory say C:\Talend\workspace\.Java\src\main\resources1

In this case also my routine will fail

Question:

I want to know if i can change my job routine to start looking for the file "vmkeys.xml" in any other directory.

The reason i want this is, because in job server i dont have any corresponding project resource directory.

Re: How to include a file in Project Resource

Anonymous — Wed, 19 Jul 2017 18:41:08 GMT

Generally Routines require static methods. Hence, yes, you can pass the path of the file to your routines. It can be an extra parameter to your routine method, which is the path of the file, that you pass when calling the routine.

If you call the routine frequently, you can setup a static private variable, create a method to set that variable, and call that method first thing in your job. Then every other routine methods can refer to the static variable which contains the path of the file. And then you load it as usual in your routine.

You will need a folder on your jobserver or on a shared drive (locked down so that only the job can access it if the content needs to be secured) where you will put this file.

Re: How to include a file in Project Resource

MahiwalVinit — Mon, 16 Apr 2018 15:04:36 GMT

what is best practice
1. shared directory (through NFS or SMB mount)
2. Create the folders on job server and keep the file?