Best Data load option

[Anonymous]

Hi Community

I am looking better option for data reload for below scenario

In data model there are only two files A & B.

A has 20 million records and updates once in a day

B has only 50 to 100 records and updates every hourly.

So every hourly reloading data is of course time consuming, how we can tackle?

Thanks

[sunny_talwar]

I guess you can do a partial reload the data for the source B so that you don't have to keep reloading the source A.

[Anonymous]

Hi Sunny

Thanks for suggestion, but we need full load once in a day, rest partial.

This means conditional load (full / partial) based on time, is that possible?

[sunny_talwar · Accepted solution]

Yes it is. Although, I have never done that, but I am sure you might be able to add time based if statement and under them a partial vs. full reload statement. In worst case you might just need to add the same script twice with time based if statement. One will run throughout the day (every hour) whereas the other one runs once a day.

[Anonymous]

Make sense,

Thanks Sunny for your time