topic Update Generic Schema from Map Structure in Talend Studio

Update Generic Schema from Map Structure

McDon — Fri, 15 Nov 2024 22:08:51 GMT

During the creation of a new Map, the output structure is created and a generic schema is created for that output structure.

As I continue through development I realized I want to add additional fields to the output structure.

These fields are not based on the input structure but will be derived from information in the input structure.

I have already added functions and logic to the output structure so I don't wan to just re-create it.

The Ask: I would like to regenerate the Generic schema from the output structure without having to manually update the schema?

When finished I will have a great many output structures and maps so a manual-sync is not practical.

Re: Update Generic Schema from Map Structure

Anonymous — Wed, 01 Feb 2023 21:46:59 GMT

A trick to do this is to do the following. Take a backup of your job first, just in case it goes wrong.

With your backup, go to output table in the tMap and change your Schema Type from Repository to Built-in. It will remain the same, but will be built-in.
Add your new columns manually to this.
At the very right at the bottom of the tMap window (underneath where you edit your schema) you will see a blue disk. Click it and follow the steps to save your schema.
Go back to the Schema Type and change this from Built-in to Repository. Underneath that, select the new schema you just saved.

All of your expressions should remain the same and you will be using your newly updated schema.

Re: Update Generic Schema from Map Structure

McDon — Wed, 01 Feb 2023 22:11:07 GMT

@Richard Hall - thank you so much for the quick response - greatly appreciated.

I like this approach and have a follow-up question.

I'm adding to my structures nested inherited structures.

I want them nested so they maintain inheritance.

Specifically

ADT A01 HL7 Message
- MSH Segment
  - edw fields
    - edw field 1
    - edw field 2
    - ...
  - MSH fields from the thmap creation

I found when I regenerate in the way you mentioned, the edw fields are added but they are not nested and lose their inheritance.

This causes a problem for future maintenance.

My objective was to keep the inheritance and support future modifications to the EDW by only changing the master edw_fields structure.

Please tell me if I've explained this clearly enough.

Re: Update Generic Schema from Map Structure

Anonymous — Wed, 01 Feb 2023 23:30:54 GMT

Are you talking about XML schemas here? Sorry, I was just thinking about relatively flat schemas. When editing XML schemas you will need to click on the parent element and add your new element as a child of that.

.....or have I completely misunderstood?

Re: Update Generic Schema from Map Structure

McDon — Thu, 02 Feb 2023 02:41:44 GMT

@Richard Hall - I wasn't clear.

I'm reading from a predefined Talend EDI structure and ultimately going to a flat Snowflake strcuture.

However, the output structure from the map I had as structured similar to XML.

This may be my problem in that I'm "crossing the streams".

Re: Update Generic Schema from Map Structure

Anonymous — Thu, 02 Feb 2023 16:30:57 GMT

I tell you what, can you post a screenshot of what you are starting with and what happens when you make the change? This will probably be a little easier for me to see the issue and a possible workaround.