Talend Custom Component - Question and Issue

[mrauf]

Following are the questions we are seeking help and details.

1. How to publish the component compiled library, and how to install compiled lib in talend studio?

so other talend developers and use it.

2. How we can publish custom component using gradle commands

Like, for maven

.\mvnw talend-component:deploy-in-studio -D talend.component.studioHome="C:/Users/mrauf/tools/talend/8.0.1/"

3. How to pass tDBConnection(Snowflake) to a custom component?

4. How to mock all database interaction in unit test for our talend custom components?

5. How to implement @Required for parameters? We are getting error when use this annotation

[ERROR] Failed to execute goal org.talend.sdk.component:talend-component-maven-plugin:1.43.0:validate (talend-validate) on project crd-talend-translation: Some error were detected:

[ERROR] - No source instantiable without adding parameters for @DataSet("CustomDataset") (com.crd.talend.components.dataset.CustomDataset), please ensure at least a source using this dataset can be used

just filling the dataset information.

Please let me know if need any details for any of these points

[undx]

Hi @mrauf,

1. We've no exchange/marketplace for Talend Component Kit connectors yet.

Anyway, the easiest way to exchange connectors with others is for now using the component classifier aka the `car` file produced during build.

This artifact can be deployed to studio using the studio-deploy command.

See https://talend.github.io/component-runtime/main/latest/studio-from-car.html for details.

2. Please see documentation https://talend.github.io/component-runtime/main/latest/build-tools-gradle.html

Once setup, you'll have the following tasks available

```

Talend Component Kit tasks

--------------------------

talendComponentKitComponentArchive - Creates a Component ARchive (.car) based on current project.

talendComponentKitDependencies - Creates the Talend Component Kit dependencies file used by the runtime to build the component classloader

talendComponentKitDeployInStudio - Deploys the module components to the Studio.

talendComponentKitDocumentation - Generates an asciidoc file with the documentation of the components.

talendComponentKitSVG2PNG - Converts the SVG into PNG when needed (icons).

talendComponentKitValidation - Validates that the module components are respecting the component standards.

talendComponentKitWebServer - Starts a web server allowing you to browse your components (requires t`

```

3. Could you refine your question? At first understanding, I'll say there's no way.

4. you should rely on interfaces and implement your mocking classes for testing.

Here's some documentation that may help about mocks provided by framework (emitters/collectors):

https://talend.github.io/component-runtime/main/latest/index-sub-testing-runtime.html

5. Here the issue is a failing model validation, more precisely the `ValidateDataSet` goal which checks:

- Ensures that any @DataSet has a unique name.

-*-> Ensures that there is a source instantiable just filling the dataset properties (all others not being required).

- Finally, the validation checks that each input or output component uses a dataset and that this dataset has a datastore.

Is your configuration class used in any Source component?

Best regards

[mrauf]

Thanks for the details. I am still trying to incorporate all.

For #1: Please correct me if following command is right to publish the car to talend studio. I tried but looks like its not publishing the component to talend studio

java -jar crd-component2.car studio-deploy C:/tools/talend/

For #2: How to set the talend studio path for task "talendComponentKitDeployInStudio"

[undx]

Hello @Muhammad Mahmood Rauf​ ,

the recommended usage is the following

```

java -jar crd-component2.car studio-deploy --location C:/tools/talend/Talend-Studio-X -f

```

apply the `-f`` flag (force) is component was already deployed previously.

br

[mrauf]

For #5, Attached is a sample component code.

Can you please review suggest the fix for following #5 error

Some error were detected:

- No source instantiable without adding parameters for @DataSet("CustomDataset") (com.company.talend.components.dataset.CustomDataset), please ensure at least a source using this dataset can be used just filling the dataset information.

[undx]

Hello @Muhammad Mahmood Rauf​ ,

clear that the error message is not understandable is that case! We should refine it to a more comprehensible one...

The issue is coming from `@Required` option in your mapper conf. As connectors should be platform agnostic (meaning Studio vs Cloud), some requirements should be applied.

With such config, in Cloud environment, we cannot generate a sampling automatically due to required/missing (in that case) option, that why the validation fails.

Anyway, if you target Studio only, you may remove the `@Required` annotation or disable this validation with the system property set as below

```

mvn clean install -Dtalend.validation.dataset=false

```

See also this document about validation properties : https://talend.github.io/component-runtime/main/latest/build-tools-maven.html#_validating_the_component_programming_model

BR

[mrauf]

Setting validation.dataset to false is not helping. Please suggest

..........> .\gradlew clean build -D talend.validation.dataset=false

> Task :jar FAILED

No source instantiable without adding parameters for @DataSet("CustomDataset") (com.company.talend.components.dataset.CustomDataset), please ensure at least a source using this dataset can be used just 

filling the dataset information.

FAILURE: Build failed with an exception.

* What went wrong:

Execution failed for task ':jar'.

> java.lang.IllegalStateException: Some error were detected:

 - No source instantiable without adding parameters for @DataSet("CustomDataset") (com.company.talend.components.dataset.CustomDataset), please ensure at least a source using this dataset can be used j

ust filling the dataset information.

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.

Use '--warning-mode all' to show the individual deprecation warnings.

See https://docs.gradle.org/6.0-rc-1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3s

5 actionable tasks: 5 executed

[mrauf]

Related to #4, Mocking

The attached component example uses dal.getSampleData

And we have one unit test for this case.

Can you please help me with how to mock dal.getSampleData, where its being used by the component internally.

[undx]

Don't know if it's a copy/paste error, but if not, the `--define` ( `-D`) should not have space!

ie:

```

.\gradlew clean build -Dtalend.validation.dataset=false

```

PS: Also, I'm not not a gradle expert, don't know if `--define` is working (same syntax) as maven.

[mrauf]

Tried both with or without space, same error

Can you please try it on the sample code I shared and confirm if we are looking at the right solution here?