topic Re: Docker file for talend job - pass context file path in Talend Studio

Docker file for talend job - pass context file path

tanjaj — Mon, 16 Dec 2019 08:57:47 GMT

Hello,

I'm trying to write a docker file to generate docker image for talnd job. And I followed this topic to write a docker file :

https://community.talend.com/t5/Design-and-Development/Generic-Dockerfiles-for-Talend-Jobs/ta-p/46317

So, my talend job looks the same as in the topic above, except the value nbrows is in the context file, stored in my local folder, and when I'm running container i want to pass only path to this context file, not value of nbrows. How to write a docker file to replace hard coded path (in the Default.properties file of talend build folder) with new path to context file?

Best,

Tanja

Re: Docker file for talend job - pass context file path

Anonymous — Mon, 16 Dec 2019 13:06:14 GMT

Hello @tanjaj,

Thanks for trying to build Docker images with Talend

First, you are following a tutorial that shows you how to build Talend jobs from a Dockerfile. I just want to remind you that you can natively build Docker images in Studio if you have a "platform" license.

Now for your question, I am not sure I've fully understood what you want.

It looks like you want to supply a context file at runtime to your container.

If so, you can achieve this in different ways.

I'll give you an example:

You can use Implicit Context Load option (in the parameters of your jobs, in the "extra" tab) to load an external context file. As you build your job as a container, you need to share this file with your host machine for the container to be able to access it. You can do so like this:

docker run -v /host_machine_path/context.properties:/context.properties -it talendjob

Of course, the implicit context load would point to /context.properties . This way you can choose any context file at runtime.

Please let me know if this is not what you were looking for.

Thanks,

Thibaut

Re: Docker file for talend job - pass context file path

tanjaj — Tue, 17 Dec 2019 10:06:42 GMT

Hello,

Thank you very much for replaying on this topic.

I have tried your solution but unfortunately it's not working. Also, I have Talend Open Studio, so no build Docker file for me

I'm new to the docker and containers so maybe I didn't explain very well.

This is what I want:

- I have file called Default.properties which stores path to my context file, and it looks something like this:

 context_file_path=C:\Users\Admin\Desktop\demo\context_file.xml

This variable context_file_path, my job will use to find context_file and read values from it. Structure of my context_file looks like this:

<?xml version="1.0" encoding="ISO-8859-15"?>
<root>
<row>
<key>nbrows</key>
<value>12</value>
</row>
</root>

When I run container it gives me an error that it can't find this path: C:\Users\Admin\Desktop\demo\context_file.xml and that is the path hard coded in Default.properties. So, I guess, I need some command like

sed

in linux, which will edit Default.properties file and pass to it new path.

Let me know if I was any clear or I made bigger confusion.

Best,

Tanja

Re: Docker file for talend job - pass context file path

Anonymous — Wed, 18 Dec 2019 10:45:36 GMT

I think you need to rethink this requirement a little. How are you using context variables in your job? I presume you are using a tFileInputXML and a tContextLoad. Why does the path to the file need to change? Can you not simply mount a volume and place the file in there? As far as your job is concerned, it would always be available in the same place. But it would make it easy to set context variables without having to modify your image. Take a look here: https://docs.docker.com/storage/volumes/

Re: Docker file for talend job - pass context file path

tanjaj — Wed, 18 Dec 2019 12:13:00 GMT

Hi,

You were right, it's working!

I finally understood what you guys were saying

I mount the volume, put the file there as you said and pass the path as a parameter:

-e ARGS="--context_param context_file=/data/"

Thanks a lot!

Best,

Tanja

Re: Docker file for talend job - pass context file path

Anonymous — Wed, 18 Dec 2019 14:38:16 GMT

No problem. Glad you got it working! I think that @tgourdel may deserve the "accepted solution" on this one though. If you want to add him or change it to him (you can apply more than one "accepted solution"), that would be fair 🙂