Talend Salesforce Schema refresh

[DTaylor]

Hi,

We are working on a Salesforce implementation and as a result, extra fields are being added into existing module schemas... yet the schema within Talend does not automatically refresh... i.e. if an extra field is added then it is not visible in Talend.

I can remove the module and re-add and it does appear but that is not practical as any job that uses that module needs reworking.

The only solution I have found that works is to manually edit the schema by adding in the extra field in manually... this then does pull the data through.

This is not practical though as fields will be missed and also it is time consuming.

Does anyone know how I can refresh the schema to reflect the latest underlying database please?

I found the article below but that dates back from 2011 and does not help...

https://community.talend.com/s/question/0D53p00007vCpKMCA0/salesforce-metadata-how-to-refresh-schemas-correctly

Kind regards,

David

[Anonymous]

Hello,

Could you please clarify in which Talend version/edition you are?

We met a similiar issue before on V 7.3.1 that the new columns are not showing in Talend. It is a combination of the new columns being formula fields and columns having restrictions on viewing on them. It was not an issue from talend side.

Best regards

Sabrina

[DTaylor]

Hi,

We are on 8.0.1 Build Id: R2023-04.

In this particular example the two new fields are just text fields... no formulas or look ups associated with them. I do have visibility in the database... if I delete and re-add the module then they will appear as have done that with another example that was not being used anywhere else. I also know that if I add the fields manually they will appear and will pull the data through.

Kind regards,

David

[Anonymous]

Hello David,

Could you please have a look at this jira issue to see if it is what you are looking for?

https://jira.talendforge.org/browse/TDI-40013

Best regards

Sabrina

[DTaylor]

Hi Sabrina,

I've reviewed that document and no... the two additional fields that have been added to the table are not formula fields... they both are purely twenty character text fields.

Kind regards,

David

[Anonymous]

Hello,

Is your solution 8.0.1 Build Id: R2023-04 open studio or subscription solution?

Best regards

Sabrina

[DTaylor]

Hi,

It is the Studio and have the licensed version. I have raised a ticket with Talend directly and have a call set up for tomorrow afternoon.

Thanks for your help.

Kind regards,

David

[Anonymous]

Hello David,

Thanks for your prompt reply and we will appreciate it a lot if you could post the root cause and solution here.

Feel free to let us know if there is any further help we can give.

Best regards

Sabrina

[DTaylor]

Hi,

I received an answer back from Talend directly although due to other project commitments I have not reviewed in detail.

It would appear there is a way via inserting a dynamic column.

The response I received is detailed below...

Talend Studio allows you to add a dynamic column to the schema of certain components in a Job. The dynamic column may constitute the only column in the schema, or it may be added after known columns, as the last column in the schema.

The dynamic column retrieves the columns which are undefined in the schema. This means that source columns which are unknown when the Job is designed, become known at runtime and are added to the schema. This can make Job design much easier as it allows for simple one to one mapping of many columns. There are many uses for dynamic columns.

====

Following are the documents which may help you :

https://help.talend.com/r/en-US/8.0/studio-user-guide/dynamic-schema

https://help.talend.com/r/en-US/8.0/studio-user-guide/defining-dynamic-schema-columns

https://help.talend.com/r/en-US/8.0/open-studio-user-guide/centralizing-salesforce-metadata