Guess schema for the Output component

[Anonymous]

Hello folks, 

I have created Input component using TaCoKit, where I can discover the schema of my DTOs (based on which endpoint user selected) using the @DiscoverSchema annotated method: 

@DiscoverSchema(family = "Family", value = "guessInputSchema")
public Schema guessInputSchema(
final InputDataSet inputDataSet, // contains endpoint info
final RecordBuilderFactory recordBuilderFactory) { // for schema builder

return _service.getEndpointTalendSchema(
inputDataSet, recordBuilderFactory);
}

Now I'm working on Output component where user can choose an endpoint based on action he wants (e.g. DELETE, UPDATE, CREATE), and my idea was that using another method like the one I showed above, I can generate new Schema for my Output component. For example, if user selects DELETE method and appropriate endpoint then my output component schema would only have 1 entry, e.g. name: ID, type:long (the only info I need for my DELETE endpoints). In that case, a user would map its record ID to my output component's ID, and all other fields would be ignored.

Unfortunately, a new method annotated with @DiscoverSchema never gets triggered when I click "guess Schema" for the Output component (in TOS 7.1.1). Is this the intended behavior, am I breaking some patterns by trying to set a schema for Output component, and if not, how can I programmatically set the schema for the Output component?

Thank you, 

Matija 

[Anonymous]

Hi @matijapetanjek

Yes, output schema comes from previous component output link by propagation.

If you want to provide a mutation, you can provide a mapper component (processor with one input and one output). You can also just ignore other fields in your implementation.

Hope it helps.

[Anonymous]

Hi @rmannibucau ,

We want to achieve similar functionality that the tSalesforceOutput component has.

1) User select a Resource / Module

2) The component dynamically creates a schema for it based on the @GuessSchema annotated method. 

	@ElementListener
	public void onNext(
		@Input final Record inputRecord,
		@Output final OutputEmitter<Record> mainFlowEmitter,
		@Output("REJECT") final OutputEmitter<Record> rejectFlowOutputEmitter) {
		//...
	}

The use-case is the following:

1) We cannot really depend on the schema propagation, as the previous component's schema may not provide enough information for the output component to be able to construct the request for the endpoint. 

2) Output component gets the Record from inputRecord. Of course it has the schema, but we should make sure that the record has a schema that the output component can understand to be able to construct the payload for the request. 

3) That's why we want first to Guess the Schema for the selected resource / operation in the Output component that would be a hint for the User what are the expected field names to be able to construct the payload (a JsonObject in the backend).

Our question is, is it possible to make these buttons work in the Output component? If so, what's the proper way to do it?

Thank you,

Zoltán

[Anonymous]

Ok, so first this is *not* an output for talend component kit since you emit records so it is a mapper or transformation component from its topology.

Currently these components only support dynamic evaluation - guess schema - to enforce schema accuracy since they are structurally generally not having their own schema.

[Anonymous]

Hi @rmannibucau, thank you for answering.

Can you please clarify one thing...How can I work with Record in my Output component if I don't know its schema in front?

If I can only propagate schema from a previous input component then, based on input component, each time I will get a Record with different schema. 

For example, I want to have output component that will delete a record entry:

@ElementListener
public void onNext(@Input final Record inputRecord) {
long id = inputRecord.getLong("I_NEED_TO_KNOW_THE_NAME_OF_THIS_FIELD");
_myRESTservice.delete(id);
} 

 I'm just trying to understand the concept before getting into implementation details...If every user can pass a record that has a different "id" column name, how can I read that long value if I don't know by which name to fetch that long value? 

Thank you!

[Anonymous]

Record provides a getSchema where you have entries and you properties/configuration can contain a built in suggestable to select the column to use for your logic.

[Anonymous]

HI @matijapetanjek ,

Yes, Me also facing same issue and also I'm not able to share my input some one for testing.When I'm sharing maven project folder for them,they getting dependency files are missing errors.Because I have added some jar files into project folder,all jars are showing  them  missing like.

Failed to execute goal on project vtigercrm-components: Could not resolve dependencies for project org.talend.components:vtigercrm-components:jar:0.0.1-SNAPSHOT: The following artifacts could not be resolved: com.vtiger:vtwslip:jar:1.5, com.apache-mime4j:apache-mime4j:jar:0.6: Could not find artifact com.vtiger:vtwslip:jar:1.5 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

Thanks,

Venkat

Error.png

[Anonymous]

Hi @Venkat_C ,

Did you install the component - not just the jar but the .car? If you are in global mode for your maven repository you must ensure this dependency is in your local maven repository. If not, it must be located in the studio repository (in configuration folder). Finally, ensure that in the project they are in scope compile (or worse case "runtime") otherwise they are ignored.

Romain

[Anonymous]

Hi @

Yes, I have installed successfully into TOS using  two commands 

1.mvnw clean install

2.`mvnw talend-component:deploy-in-studio -Dtalend.component.studioHome="<path to Talend Open Studio home>"

for me working when will change .m2 folder in TOS configuration folder.for others facing jars are missing like above.

Thanks,

Venkat

[Anonymous]

Hi,

Deploy in studio goal only supports 

studioHome/configuration/.m2/repository/

as maven repository.

1.1.10 will enable to customize it but is not yet released - a snapshot should be available onhttps://oss.sonatype.org/service/local/repositories/snapshots/content tomorrow.

Romain