idea Re: Schedule the load data in Data Catalyst in Suggest an Idea

Schedule the load data in Data Catalyst

alexandru — Mon, 04 May 2020 13:26:00 GMT

Currently only the QVD ingest data has the possibility to schedule the data load/refresh it would be nice to have it for all source types

Re: Schedule the load data in Data Catalyst - Status changed to: Open - Collecting Feedback

Christopher_Ortega — Wed, 10 Jun 2020 18:06:36 GMT

Alexandru and those who liked this post:

Please provide more feedback on how you would like to see this work.

There is a deprecated, hidden module called Scheduler that allows you to schedule entity loads, prepare dataflows, or/and publish jobs into a serial process. Note it is entity loads.

Your request mentions scheduling for sources.

Would you like to see something different than the deprecated scheduler or the scheduler? If the scheduler, is it sufficient for what you seek or are you looking for something in addition to that?

Thanks!

Re: Schedule the load data in Data Catalyst

Carlton_Lo — Thu, 11 Jun 2020 18:16:08 GMT

Chris,

I think re-open the hidden scheduler is a start. Recently, we have more clients looking for a scheduling capability for data load as well as prepare workflow. The idea of calling API is also not well received since they have already spent $ on software and now they have write extra code / script.

Thanks

Re: Schedule the load data in Data Catalyst

Christopher_Ortega — Thu, 11 Jun 2020 18:17:04 GMT

Thanks. I understand that the lack of scheduling of loads is a gap and you say that the scheduler module is a start, but you didn't really address my question.

If it is a start, it is not in and of itself enough. What more would you like to see?

Thanks!

Re: Schedule the load data in Data Catalyst

alexandru — Fri, 12 Jun 2020 08:39:24 GMT

Hi @Christopher_Ortega , I really appreciate that you're asking for more details.

So from a functionality point of view, it would be nice to have

1. something similar to Qlik Sense, where you can Schedule whatever Task you want, both Extract/Prepare/Publish

2. be able to aggregate the triggers in cascade, example: after "Extract_Mkt" finishes, to trigger in automatic the "Prepare_Mkt"

3. integrate it the Design Flow together with the Lineage 🙂

So in my opinion, having these features will make our customers very happy! 😄

Re: Schedule the load data in Data Catalyst

Peter_Kim — Mon, 22 Jun 2020 14:10:48 GMT

I think we ought to look at a pan-product scheduler and job tracker - across Replicate, Compose, Catalog, and Sense. That seems to make the most sense.

Re: Schedule the load data in Data Catalyst

Christopher_Ortega — Tue, 30 Jun 2020 11:18:34 GMT

I agree with @Peter_Kim on this topic and will explore this further.

Re: Schedule the load data in Data Catalyst - Status changed to: Open - On Roadmap

Christopher_Ortega — Sun, 23 Aug 2020 18:16:35 GMT

Re: Schedule the load data in Data Catalyst - Status changed to: Delivered

Meghann_MacDonald — Mon, 18 Sep 2023 15:24:32 GMT

This feature has now been delivered in May 2023. Thank you for your contributions!