Data file size exceeded file streaming buffer size. Adjust the StreamingBufferSizeMB setting.

[Sonja_Bauernfeind]

Loading files larger than 50MB from web storage connectors such Dropbox, Office 365SharePoint, SFTP, Amazon S3 and similar leads to: 

Data file size exceeded file streaming buffer size. Adjust the StreamingBufferSizeMB setting.

Environment

Qlik Sense Enterprise on Windows 
Qlik Cloud 

Resolution

Qlik Sense Enterprise on Windows (client-managed):

The solution requires us to modify the Qlik Sense Engine settings.ini. See How to modify Qlik Sense Engine settings.ini for details on the procedure.

1. Stop the Engine Server
2. Go to C:\ProgramData\Qlik\Sense\Engine
3. Create a backup of the Settings.ini 
4. Open the Settings.ini file
5. Add StreamingBufferSizeMB=1024 line  (higher values are not currently supported by connectors)
6. Add an empty line after the above parameter if the parameter is the last one to be added
7. Save
8. Restart the Engine Service

Qlik Cloud

The value cannot be configured in Qlik Cloud. A workaround is to split the big data files by e.g. months.

Internal Investigation ID(s) 

QLIK-83207
CP-4120
QLIK-82097

[Ken_T]

thanks for publishing this.  I can confirm this works for SharePoint on May 2022 Enterprise 

[elifapa]

hi Sonja, could you let me know higher values than 1024mb is still not supported by Qlik? we are on qlik sense may 2022. thank you!
KR, Elif

[Sonja_Bauernfeind]

Hello @elifapa 

Not at this point, no.

All the best,
Sonja 

[abTech]

i'm using qlik sense desktop version may 2023,

i'm trying to load a file from google cloud storage and i done the authentication process but when it comes to loading the file and error accrue says: adjust the streaming buffer size 

how can i fix this.

and there is no setting.ini file in installation directory 

[Sonja_Bauernfeind]

Hello @abTech 

The Desktop Client also has a settings.ini, it is stored in C:\Users\USERNAME\Documents\Qlik\Sense.

Here's an article which documents the change of a specific setting; you can use that as a template: How to enable Qlik Sense Desktop debug level logging 

All the best,
Sonja 

[klekovkinda]

By the way is there any technical details about this setting (StreamingBufferSizeMB)?
I mean how does it influence on performance etc. 
For example I have 24 running tasks at the same time, all of them are reading files. If I set StreamingBufferSizeMB to 1024 does it mean that the Qlik will reserve more memory  like 24*(1024-<DefaultStreamingBufferSizeMB>)Mb?
And what is the default StreamingBufferSizeMB?
Or I can set 1024 and the memory consumption will be the same as I have set it to 256?

[pauljohansson]

Its amazing to be able to stream from your google bucket into Qlik. Any approx date on when the max StreamingBufferSize will be increased to > 1024mb?

thanks in advance

Paul

[Dana_Baldwin]

Hi @pauljohansson 

I think your best approach to requesting this and/or checking the status is to submit this directly to our Product Management team here: https://community.qlik.com/t5/Ideas/idb-p/qlik-ideas

Thanks,

Dana

[pauljohansson]

Suggestion created - https://ideation.qlik.com/app/#/case/432087?currentProductId=9da99bc1-0dfb-4471-8e12-d23685578c43&se...

[damianbadura1]

What is a biggest available value for StreamingBufferSizeMB? 1024? Is it for each file that is retrieved from Dropbox, Office 365SharePoint, SFTP, Amazon S3 or sum for all running at the same time?