Which is faster for 1M records per day CSV or Rest API?

[mhmmd_srf]

Hi All,

I have incremental load to create QVD. It takes last one days data and append. But, the volume of last one day data is nearly 1M and this is taking nearly 10 hours to just read the data. My source is Service Now and I am connecting to Service Now through rest API.

If I get the data dump in cvs format, will be faster than fetching data directly from Service Now?

Thanks,

Sarif

[deepanshuSh]

Hi, Yes Excel file would definitely boost the loading time, but do consider that there is a limitation of the excel of maximum of 104876 ( approx 1M) so the data might get lost in the process if volume is high and records are more than the set limit.