Generate Java sources out of Talend sources

[Anonymous]

Hello,

Using the Git Plugin, I was able to export my Talend Sources (with myJob.screenshot, myJob.item, myJob.properties in the process folder) into my git repository.

Is it possible to generate the java sources out of that? And how?

Thanks for the help.

[Anonymous]

You need to forget about the idea of generating Java sources unless you want to build the Open Source Edition from source and make changes to enable you to just build sources. Talend is not built to give you the Java source code. The Talend "source" is the XML, the Java source is an intermediary step between the XML and the compiled job. Continuous Integration is very possible, but it has to be done in the way described in the link I sent you.

[Anonymous]

In the link you sent me, there is this sentence : "Gets the sources (potentially at each commit) and converts them to Java classes with the Talend CI Builder". This is exaclty what I'am trying to achieve but I would like to know if it is possible to do this step only by using the CommandLine.

Like when I build a Job in the Talend Studio, I have a src/main/java/project_name/job_name containing the java source code in the zip file generated. Isn't that the same java code that would be generated by the CI Builder? If yes, isn't it possible to generate it only by using the CommandLine?

I'am very new to all this and I'am very thankful for any help I can get.

[Anonymous]

The sources they are talking about are the Talend sources. If "source" is mentioned anywhere in Talend documentation it is talking about the very basic Talend source code (which is XML). From the XML it generates Java (which is in no way available to you unless you copy it from the Studio or retrieve it from a built zip file) and then it compiles. The Continuous Integration functionality requires the Command Line at the moment. You configure Jenkins to use the Command-Line. The Command-Line builds from the XML sources, it won't build from Java sources. While it obviously uses Java sources, it's purpose is to convert from the XML. There is absolutely no other supported way. Obviously you can hack it to get the Java source and hack it to compile from the Java source, but that will add a massive unsupported and unnecessary step into the process.

The best way of implementing CI (in my opinion) is to use what is described in that link. Essentially you use the Command-Line to compile your code into artefacts, store them in Nexus, deploy them and test, if the test passes promote from your "TEST" Nexus repository to your "PREPROD" or "PROD" Nexus repository, etc.

Can I ask what you think the benefits would be of trying to configure Jenkins to compile from Java Sources? It adds a lot of extra steps and I cannot see the benefit. If anything it adds a mass of configuration and maintenance when Talend patch and you have to reconfigure a load of Jars (at the very least).

[Anonymous]

"Can I ask what you think the benefits would be of trying to configure Jenkins to compile from Java Sources?"

This is not how it's done in the documentation, compiling from java sources?

https://help.talend.com/reader/mpVLyjeAbBh7eHWdz_0SlQ/HF8MMjUq3bllDlzOz2lqxw

Here you can read what the first Jenkins job does : "Gets the sources (potentially at each commit) and converts them to Java classes with the Talend CI Builder. This project calls the CommandLine application that generates all the sources of the Jobs and Tests you have created in your project."

So this is how I understood the workflow from the documentation :

Jenkins gets the Talend sources (the XML files) from git, then calls the commandline and the CI Builder to generate Java sources from those XML. Then with Maven we could run our test cases (at this point we need java sources to run test cases right?).

Am I mistaken at some point? If no, my purpose is to keep the same workflow but whitout the Talend CI Builder if possible. If it is not possible, I would like to know what the CI Builder does that the CommandLine can't do.

[Anonymous]

The CI Builder is essentially a plugin for Maven. It works WITH the Command-Line. This is described in the link below.... 

https://community.talend.com/t5/Architecture-Best-Practices-and/Continuous-Integration-with-Talend-C...

With regard to testing, this video shows how this is intended to be handled (https://www.youtube.com/watch?v=Ygeg8Xt1x88). 

Why do you not want to use the CI Builder? Is this because you want to achieve continuous integration with the open source edition? If this is the case, the Command-Line will not be available to you either. 

You can probably hack this if you really want, but it won't be supported. You should find all of your Java sources in your Workspace. I believe these are up to date with each compilation (build) of each job. The folder is here (workspace\.Java\src\main\java\{project name}\{job_name}) in 6.2. As long as you have the related Jars available you will be able to compile from this code. However, as @fdenis said, you will need to make sure you update this code whenever you change a job in your Studio. You could save the job and this code would therefore be out of date with the saved Talend sources. It should be noted that CI Builder will also have a workspace folder where these files will be generated, but I do not have CI Builder installed so cannot check. I just want to reiterate that this is NOT the supported way of doing things.

[Anonymous]

"Why do you not want to use the CI Builder? Is this because you want to achieve continuous integration with the open source edition?"

If you already have a Talend license and a Command-Line, do you get automatically the CI Builder with it or do you have to pay for the additional plugin?

[Anonymous]

That I am not sure about. Talend do change their pricing models quite regularly. I imagine you do have to pay extra for it, but you should check with your Talend contact

[Anonymous]

Ok. I already asked this question in another topic but I since I haven't received a convincing answer so I'am asking again here :

In the doc I can read "The CommandLine offers the same basic functionalities as Talend Studio and it is used to generate and export the Jobs developed with Talend Studio".

If it offers the same functionalities, why it can't transform Talend sources into Java sources and needs a a external plugin (Talend CI Builder) to do that?

[Anonymous]

The Studio doesn't provide an option to just generate the Java Sources......that happens as a by-product of generating the built jobs (....if you know where to look). The Command-Line is exactly the same. Check out where it is installed and look at the folder structure. In fact you can even start it as a proper Studio......or at least you could when I last tried it. I no longer work with the Command-LIne and prefer to get my team to publish to our Nexus repository from their Studios manually. 

CI Builder is a Maven plugin built to work with Talend in the CI environment. From experience of playing around with it (not using it for very long I might add), it simply plugs the gaps between what Jenkins can do out of the box, what Command-Line offers out of the box and what Nexus (your Maven repository) can do. For a detailed breakdown of everything it does, you should speak to your Talend account manager.