Dataset and Datastores still not working with Output components

[Os1]

Greetings,

Please last year i posted(on my private account @mbocquet ) that Fresh generated component with the sdk version greater than 1.1.1 could not pass the Datasets and Datastores validation.  As of sdk 1.1.7 and api 1.1.5, Only  Input components seems to pass this validation test, Output Components still fail and can only work with 1.1.1 where the Datasets and Datastore rules where not enforced while  on the otherhand, I had to delete Datasets and Datastore to make a processing component work with the latest version of the sdk. This means any one who is new to the sdk and following your get started tutorial page on git hub will still not have a working component. I am using open studio 7.1.2 to test the components.  Please could you update the page about this issue or maybe provide a work around?

Best Regards

Os

[florentlb]

Hello,

The tutorial (https://talend.github.io/component-runtime/main/1.1.8/tutorial-create-my-first-component.html) has been updated and tested manually a few weeks ago, with the new version of the Starter. Can you let me know at which point it fails? This tutorial only shows a processor and does not need to activate the IO mode

There is also a simpler and shorter version with input and output components that only shows the starter configuration: https://talend.github.io/component-runtime/main/1.1.8/tutorial-generate-project-using-starter.html, maybe you are referring to that one?

We're working on a new tutorial showing a more complete set of components but it is not ready yet.

Thanks,

Florent

[Os1]

Hello Florent,

thanks for the quick reply,  i m referring to this https://talend.github.io/component-runtime/main/1.1.8/tutorial-create-my-first-component.html ,  With version 1.1.7 of the sdk, and version 1.1.5 of the api,  when the  processor is being defined as an output component  i.e with only input rows defined like so

@ElementListener
    public void onNext(@Input final Record defaultInput) {
    }

. The Datasets validations fails, regardless whether Datasets are defined or not. But when it is being defined as a processor with both @Inputs  and  @Ouputs, then it will work as long as Datasets are not defined. In my Use case i want to use a processor as a pure output component with no output rows,  And this seems to work only with version 1.1.1 of the sdk any version above wont pass the Datasets validation.  Just a side note,  the current plugin does not allow to change  Category during project generation. 

Best regards

Os

[florentlb]

Hello again,

Ok, datasets/datastores are defined. Can you make sure that they are also referenced in the component configuration? What the validation does is that it checks that all input and output components have a dataset specified in their respective configuration, and that these datasets reference a datastore. I'm saying this just in case you defined them but somehow did not reference them, or the right ones, in your components.

If that is the case and it still fails, can you push your project on github so the team can grab it and try to reproduce, and then check if there is anything to be fixed?

Thanks,

Florent

[Os1]

Hello  Florent,

thanks for the reply once more, I  reference the datastore in the  dataset and finally the dataset into  the configuration but still have this problem.  here is a link  https://github.com/sambaf/TableauExtractOutput

to the source code that i m trying to configure.  And here is the error log 

[ERROR] - No source instantiable without adding parameters for @DataSet("TdataSet") (com.tableau.talend.components.dataset.TdataSet), please ensure at least a source using this dataset can be used just filling the dataset information.

[florentlb]

Hello again,

According to the log error, it seems that your component family is missing a source. Check here:

• https://talend.github.io/component-runtime/main/1.1.8/component-configuration.html#dataset-and-datas..., especially the note saying "each dataset has a corresponding source (mapper or emitter) which has a configuration that is usable if the software only fills the dataset part. All other properties must not be required."
• https://talend.github.io/component-runtime/main/1.1.8/build-tools-maven.html#_validating_the_compone... (validateDataSet). You can disable this validation if you want to use the component in the Studio only, but there's no guarantee that it will work in the future.

Hope this helps,

Florent

[Os1]

Hello Florent,

Once more thanks for your quick reply, 

Please correct me if i m wrong, isn't Emitter and Source, the way input Components are Implemented i.e  a method annotated with @Emitter  in the Mapper class creates and starts a new instance of the source(Which is where the logic of  the input component is implemented)? Are you suggesting that every output component must have a corresponding input component with at least the dataset defined in it?     If this is the case then why is such a crucial information missing on this tutorial  https://talend.github.io/component-runtime/main/1.1.8/tutorial-create-my-first-component.html, that basically creates an output component?   Please could you kindly pass an eye on my configuration here  https://github.com/sambaf/TableauExtractOutput/tree/master/src/main/java/com/tableau/talend/componen... see where i m going wrong?

Best regards

Os

Best Regards

[florentlb · Accepted solution]

Hi,

Yes you are right on your different points. You need an input or to set validateDataSet to false. The easiest is to disable that validation as it does not make sense to have it for your specific case. Add the plugin below to the plugins section of your project pom file:

<plugin>
        <groupId>org.talend.sdk.component</groupId>
        <artifactId>talend-component-maven-plugin</artifactId>
        <version>1.1.7</version>
        <executions>
          <execution>
            <id>talend-component-validate</id>
            <phase>process-classes</phase>
            <goals>
              <goal>validate</goal>
            </goals>
            <configuration>
              <validateDataSet>false</validateDataSet>
            </configuration>
          </execution>
        </executions>
      </plugin>

I also get your point on the tutorial but it is meant to show a simple processor that does not have these validation/dataset/datastore constraints and mainly focuses on the workflow from the starter to the Studio. I'll make this clearer in the content.

Florent

[Os1]

Hello Florent,

thanks for your time and solution, Please could the Talend team look into this issue and maybe insist that  at the moment, a fresh generated output component can only work if a corresponding source/emitter is provided with datasets defined or dataset Validation is dis-activated in pom.xml? Its quite hard for some one getting started to see this.  Just a side note, its not possible to change dataset names as well as Categories in the current version of the intellij plugin as well as browser version.  

Thanks for your time and responses.

best regards 

Os

[florentlb]

Hello,

Thanks, and thanks as well for sharing your feedback! Yes the team is aware of these inconveniences as well as about the bugs in the starter. I'll still forward the message to make sure

Florent