idea Re: Replicate generic log stream target connector - Status changed to: Open - Collecting Feedback in Suggest an Idea

Replicate generic log stream target connector

RJGoulet — Wed, 24 Feb 2021 15:13:51 GMT

Currently in Replicate a log stream connector that is being used as a target cannot be used more than once. The reasoning for that given is that since log stream is a type of file connector each task requires a distinct file location.

The truth behind it is somewhat misleading. If you create a file stream connector target as f:\log_stream and than a task as LOG_STREAM1 the actual used directory by the task is f:\log_stream\LOG_STREAM1\ which will be unique to the task.

Therefore it should be possible to create a generic log stream connector called "LOG_STREAM_TARGET" which points to f:\log_stream and use it in multiple tasks as "LOG_STREAM1", and "LOG_STREAM2" which is turn point to F:\log_stream\LOG_STREAM1\" and "f:\log_stream\LOG_STREAM2" respectively due to the internal mechanisms already in place. Granted you have to create individual source connectors as these need not only the task name but the origin database as well.

The benefit is that you don't need to create as many connectors which reduces some of the complexity.

Re: Replicate generic log stream target connector

Shelley_Brennan — Mon, 22 Mar 2021 14:54:37 GMT

Can you provide a specific use case / example of how you would like to use log stream so we can better understand the request? Thank you!

Re: Replicate generic log stream target connector

RJGoulet — Fri, 26 Mar 2021 15:08:27 GMT

I've got 6 different source databases that we will be replicating some data from into another on prem oracle database plus azure sql and adls gen 2 data lake. Having one log stream target connector is much easier to manage vs 6 different named connectors that all look otherwise alike. It's really a matter of trying to limit the number of objects that one has to manage and potentially mess up.

Re: Replicate generic log stream target connector - Status changed to: Open - Collecting Feedback

Shelley_Brennan — Tue, 06 Apr 2021 14:10:05 GMT

Thank you for the clarification. We would like to continue collecting feedback from others as well.

From now on, please track this idea from the Ideation por...

Meghann_MacDonald — Wed, 02 Aug 2023 16:18:36 GMT

From now on, please track this idea from the Ideation portal.

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.

Re: Replicate generic log stream target connector - Status changed to: Closed - Archived

Ideation — Wed, 02 Aug 2023 16:18:38 GMT