topic Local Nexus or Cloud Nexus- Need Help with CICD Setup in Talend Studio

Local Nexus or Cloud Nexus- Need Help with CICD Setup

sarora1 — Sat, 16 Nov 2024 04:34:04 GMT

I am following this link to setup build and deploy process

https://community.talend.com/t5/Architecture-Best-Practices-and/Continuous-Integration-and-Continuous-Deployment-of-data/ta-p/142359#toc-hId-125132543

In the setup directions,

For Maven_OPTS, fill in the location of your local CommandLine, and remote Nexus repository URLs.

-Dgeneration.type=local
-Dproduct.path=/opt/talend/Talend-7.1.1/cmdLine/cmdline/studio
-Xmx3076m
-DaltDeploymentRepository=releases-ci::default::http://adminserver:8081/repository/releases-ci/
-Dproject.distributionManagement.repository.url=http://adminserver:8081/repository/releases-ci/
-Dproject.distributionManagement.repository.id=releases-ci

I am confused about the URL

-DaltDeploymentRepository=releases-ci::default::http://adminserver:8081/repository/releases-ci/
-Dproject.distributionManagement.repository.url=http://adminserver:8081/repository/releases-ci/
-Dproject.distributionManagement.repository.id=releases-ci

Questions:

http://adminserver:8081/repository/releases-ci/ appears to be on-prem Nexus?

Why it's not being deployed to TMC ( a.k.a Cloud Nexus)?

What is the use of deploying to on-prem Nexus? My understanding is that TMC can not talk to on-prem Nexus to schedule and run the jobs.

Another question about CICD process,

Is this the correct CICD flow:

- pull code from Git

- build jars

- run tests

- if tests succeed, deploy to TMC (right to TMC not to on-prem nexus?)

- if tests fails, stop build and report as failure

Re: Local Nexus or Cloud Nexus- Need Help with CICD Setup

Anonymous — Wed, 25 Sep 2019 20:53:04 GMT

The build options you mentioned are for a local Nexus. It is the default profile which you can also specify using -Pnexus as part of your mvn deploy command. In case you want to deploy to Talend Cloud instead, then you must use -Pcloud-publisher as the profile. Still a local Nexus is required for user libraries, as well as a P2 update site.

All Maven build options are documented here: https://help.talend.com/reader/o2DMkp1_WVLFHSyNkjxn4w/X~3zX5XhKeW3Qqui8wNoAw

Good luck

Re: Local Nexus or Cloud Nexus- Need Help with CICD Setup

sarora1 — Thu, 26 Sep 2019 15:32:12 GMT

Thanks @jclarysse

That was helpful. Few more questions:

1. Currently, I am building and deploying one job like this

deploy -Pcloud-publisher -X -pl jobs/process/marketing_jobs/name_of_job_0.1 -fae -amd

Is it possible to build all the jobs under folder marketing_jobs with something like this

deploy -Pcloud-publisher -X -pl jobs/process/marketing_jobs/* -fae -amd

2. What step in the Jenkins Job tells Jenkins to run the tests?

3. Is there a way in Jenkins to skip the execution of tests?

Re: Local Nexus or Cloud Nexus- Need Help with CICD Setup

Anonymous — Thu, 26 Sep 2019 16:44:24 GMT

This is Maven standard process. When you run the deploy phase then it also runs all previous phases inclusive test.

https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

According to the documentation, you should be able to skip tests with parameter -DskipTests

https://maven.apache.org/plugins-archives/maven-surefire-plugin-2.12.4/examples/skipping-test.html

Re: Local Nexus or Cloud Nexus- Need Help with CICD Setup

sarora1 — Thu, 26 Sep 2019 16:50:46 GMT

@jclarysse Thanks again.

Looks like I edited my question after you answered it. I will appreciate if you can look at my previous message in this chain. I had one question about building all the jobs in a folder.