idea Re: Add data warehouse function to reinitialize the table[s] for a given entity or group of entities - Status changed to: Closed - Archived in Suggest an Idea

Re: Add data warehouse function to reinitialize the table[s] for a given entity or group of entities

Nisarh — Sat, 27 Mar 2021 16:30:22 GMT

We are in the initial stage of Development and this will be a huge issue for us after production deployment. We definitely do not want to lose history in the WH tables. This is a must have for us.

Thanks!

Re: Add data warehouse function to reinitialize the table[s] for a given entity or group of entities - Status changed to: Open - Collecting Feedback

Tzachi_Nissim — Thu, 08 Apr 2021 16:15:10 GMT

Hi,

The reason there is no support for dropping a single entity or several select entities because when reloading these entities there is no guarantee that the surrogate key of records will be preserved. Since other entities could include a relationship to the dropped entity with the surrogate key stored for that relationship, reloading could result in incorrect relationships.

A way to work around this would be to preserve the business-key to surrogate key mapping in the hub and then reload the table. This is obviously more complex than just drop&create. I'm opening this up for community feedback and voting.

Regards,

Tzachi

Re: Add data warehouse function to reinitialize the table[s] for a given entity or group of entities

jtompkins — Thu, 15 Apr 2021 13:57:38 GMT

Hi Tzachi,

I would like to suggest a new idea for Compose to preserve surrogate key values, both at the DWH level in the event of reinitializing entities, as well as the Data Mart level when dropping or truncating TDMA tables, preserving the VIDs. Would such a request even be plausible / realistic to implement?

Thanks

Re: Add data warehouse function to reinitialize the table[s] for a given entity or group of entities

plexpro52 — Thu, 15 Apr 2021 15:06:43 GMT

Hello Tzachi,

>> A way to work around this would be to preserve the business-key to surrogate key mapping in the hub and then reload the table.

I think that this approach should be plausible, and actually not that complex, at least conceptually; implementation in SQL may be a little tricky, but perhaps not either.

René

Re: Add data warehouse function to reinitialize the table[s] for a given entity or group of entities

JorgeUriarte — Thu, 05 Aug 2021 21:00:46 GMT

I also would like to add that if we already have a table alone, and its at the bottom of the granularity, we should be able to drop and recreate it. and if its in the middle of a hierarchical relationship, then automatically we should be able to drop and recreate all the related tables.

I hit an issue where a DWH had to be fully recreated because 1 single new table which wasnt even loaded failed to be adjusted, and we couldnt even create the adjust script. (which is another problem). a simple solution would have been to drop and recreate that table alone. and not reload 60 million records for no reason.

From now on, please track this idea from the Ideation por...

Meghann_MacDonald — Wed, 02 Aug 2023 14:57:01 GMT

From now on, please track this idea from the Ideation portal.

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.

Re: Add data warehouse function to reinitialize the table[s] for a given entity or group of entities - Status changed to: Closed - Archived

Ideation — Wed, 02 Aug 2023 14:57:02 GMT