In TAC Architecture , Nexus , Git , databases

[Mahesh6]

Hi All,

I am kind of new to TAC . 

While going through the TAC architecture I had some doubts.

In TAC ,

we have 2 repositories  & 3 databases: 

1. NEXUS

2. GIT

1. Audit

2. Admin

3 . Monitoring

So My Questions are  :

1. What  are exact roles of these 2 repositories in TAC .

2. Whenever we schedule Job in TAC we point TAC to deploy job from NEXUS Artifact repo, We have all versions of job there so why do we need to maintain source code ?
3. How they are inter connected to each other inside TAC ?

4. If we remove old versions of job from NEXUS repo (Only maintain New version),  does it affect my TAC server ? If Yes then How ?
(For ex. We have publish two version of Job (0.1 & 0.2) and if we remove 01. and only maintain 0.2 in Nexus.)
5. What are the role of databases in TAC?

6. how TAC server communicate with all repo & Database.

Thanks in advance. 

[Anonymous · Accepted solution]

Hi,

We have now moved to Talend Cloud - but from memory of using TAC the best answer I can think of to 1) and 2) would be

- GIT has the 'source code' ( xml, code snippets, contexts etc) and represents the editable version of the jobs in Studio

- Nexus has the built/compiled version of a job - akin to a deployable version ( as well as stores other dependencies/third party libraries etc) 

3) TAC has configuration to both the nexus server ( from our experience usually 1:1 with TAC instances ) and Via Workspaces/Projects to GIT - in our case we also have GIT configured on each user account - so we can keep audit trail of users updating jobs.

4) Removing Jobs from Nexus isn't ideal - suspect if you removed all references (running tasks/scheduled jobs etc it might be ok) but if you have jobs calling other jobs this could get messy. One option might be to maintain multiple nexus repositories ( i.e. dev / staging / preprod / prod ) probably a way of doing this - likely easier with paid version of nexus. So that you can safely remove artifacts in dev which were not pushed to staging etc. 

5) Databases are mostly monitoring, audit trail and runtime config ( what jobs should run when, with what contexts )  useful for managing production environments, used by TAC to show job runtime history etc.

BTW I would thoroughly suggest moving to Talend Cloud. We moved across last year with help of Talend Professional services. Most jobs moved with very little change.Administering cloud much easier - TAC is a fairly complicated bit of engineering!

Anyway hope that helps,

AndrewB @ Harmoney

[Mahesh6]

Hi Andrew
Appreciated your help.
Just one query :
1. How TAC manage SVN /GIT when we publish the job ? because we publish deploy-able version of jobs to NEXUS repo right ?

Is it when we deploy job to NEXUS that time Studio also push source code etc to SVN/GIT ...?

[Anonymous]

Hi
Deploy job to Nexus will not push source code to SVN/GIT.
To push source code to SVN/GIT, it depends on the commit model and lock model, these two options are
configured when you create a new project on TAC.

Regards
Shong

[Anonymous]

Hi, 

Yes Studio only really interacts with GIT.

We then have a couple of alternative approaches.

- From GITHUB we then trigger a build pipeline ( in our case Jenkins ) which using Talend Command Line turns the source files ( XML + Code + Libraries + Magic) into a binary executable ( or JAR file ) which is stored in Nexus. We have also wrapped routes into a Spring Boot based docker instance for example.

- Talend Admin Console can also do this on the fly - we pick job to run by scanning Git repo/branches etc, if you try and deploy/run a Task it uses its own copy of Talend Studio  ( in headless - no GUI) mode to checkout code + generate the binary/runable version - if it hasn't already done so. In this case no build server required. I'm not sure if Nexus is required for this mode - I suspect it is because we tend to build on TAC server but run on runtimes ( so assume runtimes must pull built version out of nexus) 

As per previous - I'm a little bit rusty as been a while since I used TAC - Talend cloud abstracts most of this away!

Shong, hopefully thats somewhat correct - please correct as required!