Data Movement / Storage when migrating to Qlik Cloud

[kclark_eab]

Hi,

My team is working on moving from Qlik Sense hosted on premises to Qlik Cloud Analytics. Our data is not currently stored in any of the available cloud-enabled Qlik connected platforms. Our current data is a series of CSVs in structured folders. We are operating multiple tenants, but I have heard that the Direct Access Gateway does not work for that configuration.

We're weighing whether or not we should move all of our CSV data into Snowflake as a work around, invest in a Cloud storage option like S3 or Azure Blob, or purchase Talend.

1. Can anyone describe the data movement options that have worked for them with multi-tenancy?
2. Our internal IT team is advised against S3, but is that the best available option?
3. How does Talend compare to third party cloud options like S3?

Thanks,

Katie

[makunii]

Hello Katie,

You can use the Qlik Data Gateway Direct Access (Qlik file connector).
Below is more information:

https://help.qlik.com/en-US/cloud-services/Subsystems/Hub/Content/Sense_Hub/Gateways/direct-access-g...

And

https://help.qlik.com/en-US/cloud-services/Subsystems/Web_Storage_Provider_Connectors_help/Content/C...

This way, you are connecting the files directly to your on-premises file server. You are not transferring the file from on-prem to Qlik Cloud.

Regards,
Marco

[PabloLabbeImaps]

The direct access gateway is an option if the files are very small ( less than 1MB)  for larger files the direct access could be very slow. In this case an upload process to a S3 bucket is far faster to upload an once there, far faster to read from Qlik app. 

How large are the CSV files. how much storage is required to store all files ?

[chriscammers]

There is one very important limitation if you are considering using the Direct Access Gateway.

The Store Command is not supported in the gateway.

So if you are looking to write data you have some options

1. Cloud storage provider - Azure Blob, S3, Google Storage
2. Sharepoint
3. Store to DataFiles in a space

I think the key difference between the 3 is the user experience 

1. Cloud storage is the most restrictive in terms of barriers and technical complication for users to interact with the files
2. Sharepoint allows for reading/writing and historical versioning
3. Qlik Data Files provides the best exposure of the schema and metadata in data products to make your qlik environment metadata rich. But those files get locked in, they cannot be downloaded

[kclark_eab]

Does Direct Access gateway work across tenants? Does that require a separate gateway for each tenant?