topic Re: @Producer and @DiscoverScheme in Talend Studio

@Producer and @DiscoverScheme

Anonymous — Fri, 02 Jan 2026 14:45:42 GMT

Hi Talend team,

Please post a simple project that shows the use of @Prodcuer and @DiscoverScheme.

1. What are the best practice to use them?

2. How to run the function that annotated with @Producer only once?

3. What is the full flow of a custom component? Which class/function is read first and which last?

4. Please don't give me the example with the Hazelcast. I already tried the tutorial and it isn't so clear to me. Moreover, I finished the tutorial with lots of errors.

I'm using Talend studio 7.1, and my custom component was genertaed using the Component Kit Starter.

Re: @Producer and @DiscoverScheme

florentlb — Mon, 04 Mar 2019 09:23:54 GMT

Hello Dani,

Sorry for not getting back to you sooner.

I will work on checking/updating the tutorials and making them clearer, and providing more examples as well.

In the meantime, here are some links about what we already have in the docs:

- DiscoverSchema: https://talend.github.io/component-runtime/main/1.1.8/services-actions.html#_schema

- Input logic, including Producer: https://talend.github.io/component-runtime/main/1.1.8/component-define-input.html

There is more information that you can find on that site about the general process for building custom components, let me know if you are looking for other specific pieces of information.

Florent

Re: @Producer and @DiscoverScheme

igorbeslic — Wed, 27 Mar 2019 11:45:02 GMT

Hi @fleborgne! Can I ask questions regarding to SchemaDiscovery, Record Building and Component Kit Starter?

I noticed that docs for 1.1.8 advise user to build records using:

org.talend.sdk.component.api.service.record.RecordBuilderFactory#newRecordBuilder(org.talend.sdk.component.api.record.Schema)

that method was introduced in SDK 1.1.5

If one generates component project with Web Component Kit Starter 1.1.8 , generated component project depend on SDK 1.1.8. Deploying such component into latest TS 7.2.1 M3 is futile as TS 7.2.1 comes with component SDK 1.1.4 and running such component ends up with

java.lang.IllegalStateException: java.lang.NoSuchMethodError: org.talend.sdk.component.api.service.record.RecordBuilderFactory.newRecordBuilder(Lorg/talend/sdk/component/api/record/Schema;)Lorg/talend/sdk/component/api/record/Record$Builder;

(maybe this is no place to ask why latest TS doesn't use latest SDK)

I'm new Talend user and I'm quite frustrated with discrepancies in Docs, examples and what actually Talend does. Can you give me an advise, like developer friendly combination of:

- Talend SDK documentation version

- Talend Component Kit Starter version

- Talend Studio version

which would provide developer with pleasant development environment?

Thank you very much!

Re: @Producer and @DiscoverScheme

florentlb — Wed, 27 Mar 2019 13:29:58 GMT

Hello,

Yes the forum is always the place to ask .

The main reason for this delta is that Talend Studio and the Component Kit have distinct release cycles (mainly because the Component Kit is not only tied to the Studio but also to other apps).

I will add the corresponding versions for Studio 7.2.1 M3 to our compatibility page. But yes, it uses the version 1.1.4 of the framework. You should be able to go through your use case using the 1.1.4 documentation: https://talend.github.io/component-runtime/main/1.1.4/services-actions.html#_schema

One last point, the Component Kit Starter always reflects the latest version of the framework, which I understand can cause some uncertainty. However, the team tries to ensure backward compatibility so that even projects generated with newer versions of the starter can still work with older versions of the framework.

How to change the version of the framework you use is described in this document: https://talend.github.io/component-runtime/main/1.1.8/compatibility.html

Let me know if that is not clear and be sure that the team is trying to make the process smoother with every release.

Florent

Re: @Producer and @DiscoverScheme

igorbeslic — Wed, 27 Mar 2019 20:55:58 GMT

Hi @fleborgne thank you for your documentation links. It was very useful. I did some tests and research and I have couple more questions, just to clear my thoughts:

1) Documentation says to annotate method that returns Schema with @DiscoverSchema but it says not in which class, so I put it in class annotated with @Service (actually it is in documentation services section so I concluded it should go there). Now I have method that accepts parameter that is DataSet configuration and name that matches class name of the class that is annotated with DataSet annotation. Something like this:

// Service class
@Service
public class EBusService{
    @DiscoverSchema("EBusDataSet")
    public Schema discoverSchema(EBusDataSet eBusDataSet) {
         // code skipped to be concise
    }
}

// DataSet class
@DataSet("EBusDataSet")
@GridLayout({
    @GridLayout.Row({ "_eBusConnection" })
})
@Documentation("Additional EBus properties and directives needed for processor")
public class EBusDataSet implements Serializable {
    // code skipped to be concise
}

The problem I see here, I miss

RecordBuilderFactory

Should I update discoverSchema method signature by adding RecordBuilderFactory as parameter like this:

@DiscoverSchema("EBusDataSet")
public Schema discoverSchema(EBusDataSet eBusDataSet, RecordBuilderFactory recordBuilderFactory) {
// code skipped to be concise
}

Is it expected that my EBusService is injected into component class via constructor? Once deployed in Talend Studio, when discoverySchema method is called? How should one initiate discoveryMethod? By clicking some element on Component configuration UI?

2) As nothing from 1 worked, I had to play with Talend Studio debug mode, spend some time and I figure out next: If one clicks Guess Schema button in Talend Studio component view, Talend studio would invoke component method annotated with @Producer (in my case placed in Source class referred from component via @Emitter annotated method and reverse engineer returned record and display discovered schema via UI.

Im confused... why Documentation mentioned something that doesn't work? I expected that discoverSchema would be called by Talend Studio out-of-the box, not that it is useless? Or maybe I miss something that is not written in documentation.

3) Back to documentation. Documentation gives example:

{
  "entries":[
    {
      "comment":"The column 1",
      "name":"column1",
      "nullable":false,
      "type":"STRING"
    },
    {
      "comment":"The int column",
      "name":"column2",
      "nullable":false,
      "type":"INT"
    }
  ],
  "type":"RECORD"
}

It also says, method return type is org.talend.sdk.component.api.record.Schema. This JSON structure is not org.talend.sdk.component.api.record.Schema... If https://talend.github.io/component-runtime/main/1.1.4/record-types.html describes how to build Schema with RecordBuilderFactory, can somebody update this section with instructions how to build schema described with given JSON without RecordBuilderFactory... or tell how to inject RecordBuilderFactory into Service class (I did it but still I don't see purpose, look 4th question)

4) I bumped into SalesForce component implementations... Patterns found there work without usage of annotation proposed by Talend documentation:

Service component:

https://github.com/Talend/connectors-se/blob/master/salesforce/src/main/java/org/talend/components/salesforce/service/SalesforceService.java

Abstract emitter:
https://github.com/Talend/connectors-se/blob/master/salesforce/src/main/java/org/talend/components/salesforce/input/AbstractQueryEmitter.java#L99

Particular implementor that takes fields metadata from DB:
https://github.com/Talend/connectors-se/blob/master/salesforce/src/main/java/org/talend/components/salesforce/input/ModuleQueryEmitter.java

Do sales force components rely to some super old API version? Anyway, same principle works form me with both API 1.1.4 and API 1.1.7... Only confusing is should one read documentation or decompile talend native components to figure out what is true there?
5) Can one propose documentation change, or suggest changes? Does Talend team reviews those suggestions (I ask to be sure I wouldn't waste my time if I do so)?

Thank you very much for patience and long message. I had really lot of troubles with this.

Re: @Producer and @DiscoverScheme

florentlb — Thu, 28 Mar 2019 09:31:53 GMT

Hi,

Can you try adding '@Option' as follows and let me know if you still encounter the same issue you described in your 2nd point:

discoverSchema(@Option EBusDataSet eBusDataSet, RecordBuilderFactory recordBuilderFactory)

I'll wait for your feedback but if this is indeed what was missing, I'll make it clearer in the documentation.

And yes, suggestions of doc enhancements are more than welcome as we are in the process of making it more accessible and easy to understand for external developers (it was originally created as an internal doc).

The connectors you are referring to are in the process of being migrated from the old framework to the new one, so some of them have been done using older versions of the Component Kit framework.

Again, thanks for your feedback,

Florent

Re: @Producer and @DiscoverScheme

igorbeslic — Thu, 28 Mar 2019 11:02:26 GMT

Hi @fleborgne I updated code but no any progress. Please can you tell me what triggers

@DiscoverSchema

method?

Do I have to perform any action in UI or it should called automatically by component runtime during Job execution?

Thank you!

Re: @Producer and @DiscoverScheme

florentlb — Thu, 28 Mar 2019 13:49:04 GMT

Hi,

It creates a clickable button in the component UI. It's up to the user to click it or not.

Florent

Re: @Producer and @DiscoverScheme

igorbeslic — Thu, 28 Mar 2019 14:39:06 GMT

Hi @fleborgne it does not confirmed independently by another two developers...

Thank you!

Re: @Producer and @DiscoverScheme

igorbeslic — Fri, 29 Mar 2019 20:27:38 GMT

@fleborgne wrote:

Hi,

It creates a clickable button in the component UI. It's up to the user to click it or not.

Florent

Hi @fleborgne please, can you describe me how clickable button looks like? Does it look like this one:
component view screenshoot

Thank you very much!!!!!