topic Need help with Jenkins pipeline build in Talend Studio

Need help with Jenkins pipeline build

sarora1 — Sat, 16 Nov 2024 05:01:17 GMT

[ERROR] Plugin org.talend.ci:builder-maven-plugin:7.1.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor 
for org.talend.ci:builder-maven-plugin:jar:7.1.1: Could not transfer artifact org.talend.ci:builder-maven-plugin:pom:7.1.1 
from/to plugins_snapshot (http://XXXXXXXXX:9081/repository/talend-custom-libs-snapshot/😞 
Failed to transfer file http://XXXXXXXX:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7.1.1/builder-maven-plugin-7.1.1.pom 
with status code 400 -> [Help 1]

Am I missing some plugins or missing some settings in my pipeline script on both?

More info...

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Failure to transfer 
org.talend.ci:builder-maven-plugin:pom:7.1.1 from http://10.190.3.207:9081/repository/talend-custom-libs-snapshot/ 
was cached in the local repository, resolution will not be reattempted until the update interval of plugins_snapshot 
has elapsed or updates are forced. Original error: Could not transfer artifact org.talend.ci:builder-maven-plugin:pom:7.1.1 
from/to plugins_snapshot (http://10.190.3.207:9081/repository/talend-custom-libs-snapshot/😞 
Failed to transfer file http://10.190.3.207:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7.1.1/builder-maven-plugin-7.1.1.pom with status code 400

env.PROJECT_GIT_NAME = 'CICD_POC_PROJECT'
env.PROJECT_NAME = env.PROJECT_GIT_NAME.toLowerCase()
env.ARTIFACT = 'test_job'
env.VERSION = '0.1'
env.GIT_URL = 'https://github.com/YYYYYY/XXXXXX.git'
env.CLOUD_URL = 'https://tmc.us.cloud.talend.com/inventory/'
env.TYPE = "" // if big data = _mr
env.IMAGE_NAME = 'XXX'   
// Credentials IDs (Manage Jenkins => Credentials)
env.GIT_CREDENTIALS_ID = 'git_user'
env.TMC_CREDENTIALS_ID = 'tmc_user'

node {
 	// Clean workspace before doing anything
    try {
        def userInput
        def deployprod
        stage('Initialize') {
            sh '''
                echo "PATH = ${PATH}"
                echo "M2_HOME = ${M2_HOME}"
            ''' 
        }
        stage ('Git Checkout') {
            git(
                url: "${GIT_URL}",
                credentialsId: "${GIT_CREDENTIALS_ID}",
                branch: 'master'
            )       
            mvnHome = tool 'M3'
        }
        stage ('Build, Test and Publish artifacts to Cloud') {
            withCredentials([usernamePassword(credentialsId: "${TMC_CREDENTIALS_ID}", usernameVariable: 'Dusername', passwordVariable: 'Dpassword')]) {
					withMaven(
                            // Maven installation declared in the Jenkins "Global Tool Configuration"
                            maven: 'M3',
                            // Maven settings.xml file defined with the Jenkins Config File Provider Plugin
                            // Maven settings and global settings can also be defined in Jenkins Global Tools Configuration
                            mavenSettingsConfig: 'maven-settings-file',
                            mavenOpts: '-Dlicense.path=~/Documents/files/license -Dproduct.path=~/Documents/cmdline -Dgeneration.type=local -Dservice.url=${CLOUD_URL} -Dcloud.publisher.screenshot=true -Dcloud.publisher.skip=false -Dservice.accelerate=true -Dcloud.publisher.updateFlow=true -Dservice.username=$Dusername -Dservice.password=$Dpassword -Dcloud.publisher.environment=Development -Dcloud.publisher.workspace=YYYY -Xms1024m -Xmx3096m') 
                            {
                    
                        // Run the maven build
                        sh "mvn -X -f $PROJECT_GIT_NAME/poms/pom.xml -Pcloud-publisher clean deploy -fn -e -pl jobs/process${TYPE}/${ARTIFACT}_${VERSION} -am"
                    
                        }    
		        }
        }
        
    } catch (err) {
        currentBuild.result = 'FAILED'
        throw err
    }
}

Re: Need help with Jenkins pipeline build

sarora1 — Fri, 09 Aug 2019 15:33:46 GMT

Replying to my own question:

I see issue with local studio unable to push libraries to Nexus talend-custom-libs-snapshot

here are some errors from my local studio log

org.talend.commons.exception.PersistenceException: Studio can't access the resource from server, please check the resource url.
	at org.talend.repository.remoteprovider.client.AbstractClient.getArtifactLocation(AbstractClient.java:822)
	at org.talend.designer.publish.di.preferences.PublishPreferences.getArtifactRepositoryFromServer(PublishPreferences.java:141)
	at org.talend.designer.publish.di.preferences.PublishPreferences.getArtifactRepository(PublishPreferences.java:75)
	at org.talend.designer.publish.di.preferences.PublishPreferences.getArtifactRepository(PublishPreferences.java:61)
	at org.talend.designer.publish.di.model.NexusService.getPublishNexusServerBean(NexusService.java:153)
	at org.talend.updates.runtime.nexus.component.NexusServerManager.getLocalNexusServer(NexusServerManager.java:65)
	at org.talend.updates.runtime.engine.factory.ComponentsNexusInstallFactory.getLocalNexusFeatures(ComponentsNexusInstallFactory.java:79)
	at org.talend.updates.engine.EditionComponentsNexusInstallFactory.getAllExtraFeatures(EditionComponentsNexusInstallFactory.java:41)
	at org.talend.updates.runtime.engine.factory.ComponentsNexusInstallFactory.retrieveAllExtraFeatures(ComponentsNexusInstallFactory.java:182)
	at org.talend.updates.runtime.engine.ExtraFeaturesUpdatesFactory.retrieveAllComponentFeatures(ExtraFeaturesUpdatesFactory.java:71)
	at org.talend.updates.runtime.feature.FeaturesManager.retrieveAllOfficalFeatures(FeaturesManager.java:156)
	at org.talend.updates.runtime.feature.FeaturesManager.getFeaturesCache(FeaturesManager.java:141)
	at org.talend.updates.runtime.feature.FeaturesManager.getUpdates(FeaturesManager.java:163)
	at org.talend.updates.runtime.feature.FeaturesManager.searchUpdates(FeaturesManager.java:214)
	at org.talend.updates.runtime.ui.feature.job.FeaturesCheckUpdateJob.run(FeaturesCheckUpdateJob.java:54)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: Studio can't access the resource from server, please check the resource url.

Re: Need help with Jenkins pipeline build

sarora1 — Fri, 09 Aug 2019 19:21:46 GMT

@xdshi any help here?

Re: Need help with Jenkins pipeline build

Anonymous — Mon, 12 Aug 2019 03:53:13 GMT

Hello,

Are you using talend open studio or subscription solution?

You can find the integration documentation here:
https://help.talend.com/reader/o2DMkp1_WVLFHSyNkjxn4w/lHXK~e0ZTKbbL6HDnQVofA

The missing CI builder jar , you showed in the question, is now part of the subscribed 7.1.1 Talend Studio.

Best regards

Sabrina

Re: Need help with Jenkins pipeline build

sarora1 — Mon, 12 Aug 2019 13:57:00 GMT

Subscription version 7.1.1

Error message says

[ERROR] Plugin org.talend.ci:builder-maven-plugin:7.1.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for 
org.talend.ci:builder-maven-plugin:jar:7.1.1: Could not transfer artifact org.talend.ci:builder-maven-plugin:pom:7.1.1 from/to plugins_snapshot 
(http://XXXXXX:9081/repository/talend-custom-libs-snapshot/😞 Failed to transfer file 
http://XXXXXX:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7.1.1/builder-maven-plugin-7.1.1.pom 
with status code 400 -> [Help 1]

Based on above message the location its looking for CI builder jar at http://XXXXXX:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7.1.1/builder-maven-plugin-7.1.1.pom

but I I dont see org/talend/ci under talend-custom-libs-snapshot

But I do see it's available where the the cmdline is installed on server

/home/Documents/cmdline/configuration/.m2/repository/org/talend/ci

Where do you configure location to look for there libraries?

And if I change localRepository in maven settings.xml to where TalendStudio (cmdline) is installed

then I get this error message

[ERROR] Could not create local repository at /home/Documents/cmdline/configuration/.m2/repository -> [Help 1]org.apache.maven.repository.LocalRepositoryNotAccessibleException: Could not create local repository at /home/Documents/cmdline/configuration/.m2/repository

Re: Need help with Jenkins pipeline build

sarora1 — Mon, 12 Aug 2019 20:25:09 GMT

Talend consultant was able to help us out. The problem was the permissions on cmdline folder(or whatever folder you have studio installed) were causing the failure. Resolution: change permission on cmdline folder so that Jenkins can create stuff in that folder.

Marking this as resolved.