SVN in Talend TAC

[Anonymous]

Hi, 

I have small confusion regarding SVN configuration working with Tac.

My question is, Is that possible we have svn server and deployment server is same server ?

We configure our svn in Tac and use tac url in Talend's web app url field in manage connections.

If we do settings like this, what's the difference between when we deploy our jobs from Tac on deployment server and committed talend code via svn in same deployment server ? If yes, Do they share same location or different in server ?

And we deploy the build of talend job in Tac, where it keeps on deployment server ? Do we define this location in Tac configuration ?

I'm little confuse here how these two things work all together, SVN with TAC with Deployment. 

Any help will be appreciated !!

[Anonymous]

To be honest, your question is a bit confusing.

Anyway. You can run your SVN server where ever you want! It does not matter. In most cases the SVN server is also used by other projects than Talend and therefore the server often already exists in the company when we install Talend.

The only thing you should check is the SVN server version - must be supported by Talend.

The svn server url in the TAC will directly be delivered to the studios and the studios talks directly to the svn server (same applies to git).

To separate development and production you should use different branches - just the normal case for svn or git.

[Anonymous]

Hi Jan, thanks for your response.

So you mean to say, My deployment server can be my svn server as well. Correct ?

So please correct me if am wrong, we take a build from studio connected to svn and deploy it from tac. And tac keeps that build in nexus artifact repository. Is it correct ?

Second, I have never used publish of jobs in Talend. How its different from normal deployment explained above, like take a build of job from studio and then deploy that zip file in TAC ?

My assumption, correct me If am wrong, When we publish the job from studio with snapshot option, it pushes the latest build of job from studio directly to nexus artifact repository which we have already configured in tac.

If that's a case, then even after doing publishing why we go to tac and find the job from Nexus option to deploy in Execution task (which appears when we click on ADD option in job conductor) ? What it does extra when we select the job from Nexus repository to deploy ? Dont you think so once its published, job should be automatically appears in job conductor ?

Please correct me if am going in wrong direction..!

[Anonymous]

What do you mean with development server? A TAC or the Jobserver or what?

[Anonymous]

Jan I have attached the same image with my understanding, please have a look and share your words.

I have couple of more doubst Jan on same topic that where the Nexus artifact repository actually sits physically ? Like SVN folder structure is in Machine 2 for example, Is it the same way Nexus sits in Machine 3 ?

Here is my complete understanding Jan, What I'm thinking is,

1) When we build a job from Talend studio and import a zip file into Tac's execution task, Tac place that zip file to Nexus repository. And according to me Nexus artifact repository is always present in job server, the server where we want to deploy our Talend jobs and run as well.

2) When we publish our job from Talend studio, the zip file of job goes to Nexus artifact repository in job server. And then after that we point that job from Tac as a execution Task. Is my understanding correct Jan in point 1 and 2 mentioned above ?

These are two ways of deployment, that's how I'm think. Please put my understanding in right direction if am wrong.

Secondly, If am thinking right in above mentioned points why we have different ways of deployment, like first is import zip file in Tac and second is publishing a job ?

Third, What is the difference between Add normal task and Artifact task when we add a job to deploy in job conductor ?

Repo.png

[Anonymous]

Hi Kate,

I think you are confusing the old system with the new system. In the old days (and it is still used by some customers now) the CommandLine would generate and build the job from the SVN when you published a job using the TAC. This meant recompiling every time (...a very poor way of working). What the Nexus does is allow us to compile once (from the Studio when you publish) and store the compiled artefact. What the TAC does with this is simply pull the compiled artefact from the Nexus when it is needed. 

You can still pull the code from SVN and compile it using the Commandline (orchestrated by the TAC interface) if you want, but I would not. In fact, I would advocate completely removing the CommandLine (which is just a headless Studio) and publish your jobs to the artefact repository (Nexus) directly from the Studio. 

[Anonymous]

Thanks rhall for joining conversation.

Can you please share your answers to below points ?

Like you said when we publish the job from studio, it goes to Nexus repository. I agree. 

Here are small doubts:

1) Is Nexus repository is also a compiled code managing directory like structure like SVN has ?

2) When we build Talend job from studio and import .zip file into Tac, Does it goes to same Nexus repository ?

3) When we publish our job from Talend studio, the compiled code zip file of job goes to Nexus artifact repository in job server. And then after that we point that job from Tac as a execution Task ? Is it correct understanding ?

4) What is the difference between Add normal task and Artifact task when we add a job to deploy in job conductor ?

5) If am thinking right in above mentioned points then why we have different ways of deployment in TAC, like first is to import zip file in Tac and second is publishing a job ?

[Anonymous]

---

@kate wrote:

Thanks rhall for joining conversation.

Can you please share your answers to below points ?

Like you said when we publish the job from studio, it goes to Nexus repository. I agree. 

Here are small doubts:

1) Is Nexus repository is also a compiled code managing directory like structure like SVN has ?

 The Nexus repository is just compiled code. Its structure is related to the Group ID, Artifact ID and Version values you specify when you publish. The structure is not related to the SVN.

2) When we build Talend job from studio and import .zip file into Tac, Does it goes to same Nexus repository ?

 This sounds like you are by-passing the Nexus. When you "build" in the Studio, it does not publish to the Nexus repository. I don't believe jobs published on Tac like this go to the Nexus repository. There is not enough information supplied to put them in a the Nexus (I've just tested and cannot see them in the Nexus). Once you deploy the job you will find the zip file's folder structure underneath the /jobserver/agent/TalendJobSeversFiles/repository folder.

3) When we publish our job from Talend studio, the compiled code zip file of job goes to Nexus artifact repository in job server. And then after that we point that job from Tac as a execution Task ? Is it correct understanding ?

This seems correct. Although the Nexus does not have to be on the JobServer. The Nexus repository can be anywhere in your architecture.

4) What is the difference between Add normal task and Artifact task when we add a job to deploy in job conductor ?

 "Add normal" allows you to select where to add the job from (zip, SVN...using commandline, and Nexus). "Artifact task" just makes it easier if you are always adding from a Nexus repository. This way is the recommended way now.

5) If am thinking right in above mentioned points then why we have different ways of deployment in TAC, like first is to import zip file in Tac and second is publishing a job ?

This is to support older implementations. Talend used to make you compile your code from SVN using the CommandLine. Now it works in a better way. However, many users will have built CommandLine scripts to automate this process. If Talend stopped supporting this mechanism over night, then it would alienate a lot of customers and lead to a lot of rework. 

Unfortunately it has not been made very clear to newer customers that the CommandLine is pretty much deprecated now. Most things you could do with the commandline, you can now do with the metaservlet (not quite everything, but most things).  

---

[Anonymous]

Thanks for this wonderful sharing rhall.

I have just one last doubt about the deployment we do using build jobs from Talend studio.

When we build the job from talend studio, it generates the zip file. And after that when we import this zip file in TAC's execution task where it goes in the back-end ? I mean where that job is kept in job server after importing zip file into TAC in job conductor ? Like in case of publish it goes to artifact repository, but what happen in the case of importing zip file type of deployment in Tac ? I hope it makes sense

[Anonymous]

Hi all,

The Publisher in the TAC creates so called artefacts and put them in the Nexus. 

The publisher must be used for services and can be used for batch jobs.

I strongly recommend using the Publisher for all kind of jobs.

The advantage compared to the build of the job in the Job Conductor (with its generate-function) is, the artefact in the Nexus is stable and (in case of using a release repository) it cannot be overwritten.

If you build a job within the Job Conductor you cannot rely on it because the Job Conductor can decide to build a job again. The Job Conductor will never build a job if you have created the Job Conductor entry based on a Nexus artefact.

We have also 2 different TACs (at least). One TAC is dedicated to the development and has a command-line connected to it. Here we only build artefacts and organize the projects and the developer rights.

The other TAC is only to run jobs. We setup here the same projects like in the Dev-TAC with exactly(!) the same name but typed as "no storage" - means there is no source repository. This way we cannot build a job in production and always build the artefacts in the development - mostly using different branches for the different stages (feature-branches, release branches etc).