Qlik Community

New to Qlik Sense

Discussion board where members can get started with Qlik Sense.

reshma_km
Contributor III

Data source for Qliksense

Hi,

If the data volume is huge, which approach is better? Our source data is stored in excel. So using the excel as data source or loading the data to a database like SQL server and using that as data source?

15 Replies

Re: Data source for Qliksense

If the data is in Excel and the objective is to get it into Qlik then I would do it directly Excel into Qlik, as I can see no benefit of adding a database load / extract into the processing chain.

I am curious what your definition of huge is.  I seem remember the maximum Excel can handle is about a million rows, but I could well be wrong about that.

reshma_km
Contributor III

Re: Data source for Qliksense

what about 1 million rows from excel source? Will that be an issue from the "app data reload" point of view? And what about storing the excel? I am thinking it has to be placed in a server folder or in a shared folder which can be accessed by both server and developer machines. Will that be a problem?

OmarBenSalem
Esteemed Contributor

Re: Data source for Qliksense

If ur data is excel, it cannot be qualified as huge. (since excel has a limit for about a million of max rows.., which is qualified as TINY for Qlik Sense)

Don't worry, Qlik will handle it just FINE.

reshma_km
Contributor III

Re: Data source for Qliksense

Hi,

I think there is a limit to an attached file as a data source.  Do you know what it is and if configurable?  Is it something like 200MB?

Luminary
Luminary

Re: Data source for Qliksense

JustinDallas
Valued Contributor II

Re: Data source for Qliksense

TINY?  What kind of Godzilla sized data sets are you working with?  The largest I've got is a 3MM recordset for logistics location breadcrumbs.

reshma_km
Contributor III

Re: Data source for Qliksense

Is there any way to increase the size from 200 mb?

Re: Data source for Qliksense

I think that limit is to do with attached files whilst using the Data Manager, which I have only played with and never used in vengeance.

For loading data from data sources such a files, databases etc. I am not aware of any Qlik imposed limits per se.

I reckon it will work fine, so why not give it a go at loading the data - I think you will be pleasantly surprised.

reshma_km
Contributor III

Re: Data source for Qliksense

Hi,

Currently I am not working with any huge files. This is for a future app proposed by client. They say they have data in excel files and the data size is huge. So I just want to be sure so that we don't run into problems later, both development and maintenance point of views.

Community Browser