48 Replies Latest reply: Sep 11, 2017 4:45 AM by George Beaton Branched to a new discussion. RSS

    Load Web File dynamically

    Eddy Sanchez

      Hi all

       

      For load a web file in Qlik Sense is mandatory to define the connection with the specific link?

       

      I need to load a web file that the source is made by variables like that:

       

      [https://domain.com/$(vYear)/$(vMonth)/File.XLS]

       

      Any suggestion?

       

      thank you

       

      Att

       

      Eddy

        • Re: Load Web File dynamically
          Eddy Sanchez

          If I use the same script of QlikView, don't work and show the next message error:


          LOAD statement only works with lib:// paths in this script mode



            • Re: Load Web File dynamically
              Michael Tarallo

              Hi Eddy - in order to use the "legacy" type load statement (without lib) - you must change a setting - in the settings.ini file to disable Standard mode.

               

              The help document is here:

               

              https://help.qlik.com/sense/en-US/online/index.html#../Subsystems/WorkingWith/Content/LoadData/DisableStandardMode.htm?H…

               

              This will enable you to use the http call like it is done in QlikView and with the use of variables.

               

              Otherwise - you can still use variables with the lib statement - BUT you would need to make a connection for each URL.

               

              If you need a sample, please let us know.

               

              Please mark the appropriate replies as helpful / correct so our team and other members know that your question(s) has been answered to your satisfaction.

               

              Regards,

               

              Mike Tarallo

              Qlik

                • Re: Load Web File dynamically
                  Janusz Twardziak

                  Hi Michael,

                   

                  The "legacy" mode seems like a good idea to be a "default" one as it gives more flexibility in creating a connection strings to data sources.

                   

                  Would you pass this suggestion to the team, please?

                   

                  Best regards,

                  Janusz

                    • Re: Load Web File dynamically
                      Michael Tarallo

                      Hey Janusz -

                       

                      Yes - in fact that is the plan - direct from the product manager's mouth:

                       

                      "We are going to revert the default back to legacy to allow custom connect until the new connectors are available for Sense."

                       

                      Regards,


                      Mike

                        • Re: Load Web File dynamically
                          Eddy Sanchez

                          Michael, where I can send suggestions for Qlik Sense?

                           

                          I miss the next shorcut combinations:

                           

                          Ctrl+E = Data load Editor

                          Ctrl+R = Load data

                          Ctrl+Shift+R=Partial Reload (If still exist)

                          Ctrl+T = Data model viewer

                          Ctrl+Alt+V = Variables

                           

                          Att

                           

                          Eddy

                            • Re: Load Web File dynamically
                              Michael Tarallo

                              Hi Eddy

                               

                              This is the place for that as well - we will send ideas and suggestions to the R&D team.

                               

                              In the mean time here are some shortcuts you may find useful where applicable to Qlik Sense.

                               

                              Standard:

                               

                              https://help.qlik.com/sense/en-US/online/index.html#../Subsystems/Qlik_Management_Console_help/Content/ServerUserGuide/S…

                               

                              Additional - (after the gestures)

                              https://help.qlik.com/sense/en-US/online/index.html#../Subsystems/WorkingWith/Content/UserInterface/GesturesAndKeyboardS…

                               

                              Regards,

                               

                              Mike

                                • Re: Load Web File dynamically
                                  Chris Brain

                                  Hi Mike,

                                   

                                  This currently looks like a major issue for the many QVSource users out there (and general Qlik users loading from web urls) who rely on being able to construct URLs as the load script runs.

                                   

                                  Could you clarify a couple of things from this thread?

                                   

                                  • You mentioned 'Otherwise - you can still use variables with the lib statement - BUT you would need to make a connection for each URL.'
                                    • What does this mean exactly, can the URL behind a lib statement read variables (initial tests suggest not) or do you need to create a connection for every unique URL?

                                   

                                  • Yes - in fact that is the plan - direct from the product manager's mouth: "We are going to revert the default back to legacy to allow custom connect until the new connectors are available for Sense."
                                    • This sounds like the reversion back to the legacy mode as default will be temporary, is that right?
                                    • Re: Load Web File dynamically
                                      Michael Tarallo

                                      Hey Chris - nice to hear from you, in regards to your questions:

                                      • You mentioned 'Otherwise - you can still use variables with the lib statement - BUT you would need to make a connection for each URL.'
                                        • What does this mean exactly, can the URL behind a lib statement read variables (initial tests suggest not) or do you need to create a connection for every unique URL?

                                       

                                      If you use the lib statement - it is expecting a defined connection which is read from the list of connections stored in the repository - and if set up with security rules, it will respect those as well. So if a variable is used with the LIB statement such where the variable's values can be something like connection1 or connection2  - the actual connection names connection1 and connection2 must be defined to the correct data sources.

                                       

                                       

                                       

                                      • Yes - in fact that is the plan - direct from the product manager's mouth: "We are going to revert the default back to legacy to allow custom connect until the new connectors are available for Sense."
                                        • This sounds like the reversion back to the legacy mode as default will be temporary, is that right?

                                      Yes - you are correct, this will happen with the next minor update release, I am being told.

                                       

                                      HTH

                                       

                                      Mike

                                        • Re: Load Web File dynamically
                                          Chris Brain

                                          Hi Mike,

                                           

                                          Good to be speaking with you too, especially considering how familiar you are with our product.

                                           

                                          I am still not quite clear on this :-/

                                           

                                          Regarding:

                                          So if a variable is used with the LIB statement such where the variable's values can be something like connection1 or connection2  - the actual connection names connection1 and connection2 must be defined to the correct data sources.

                                          Can a variable, e.g. $(vSearchTerm), be passed through to a connection or would a connection need to be created for each possible/unique URL?

                                           

                                          And:

                                          Yes - you are correct, this will happen with the next minor update release, I am being told.

                                          But it sounded from your initial message that this reversion itself would be temporary, i.e. that again soon after the default will revert to the current position of a 'LIB://' only approach in the script - is that also your understanding?

                                            • Re: Load Web File dynamically
                                              Michael Tarallo

                                              Hi Chris:

                                               

                                              Can a variable, e.g. $(vSearchTerm), be passed through to a connection or would a connection need to be created for each possible/unique URL?

                                               

                                              If not using the legacy mode AND only using LIB - then each Web File connection-type would need to be defined as shown in the previous reply. The variable could only be the connection name.

                                               

                                              If using the Legacy mode - as set in the settings.ini - then you can use the same traditional approach with variables  as used with QlikView (the script that QVSource uses)

                                               

                                              But it sounded from your initial message that this reversion itself would be temporary, i.e. that again soon after the default will revert to the current position of a 'LIB://' only approach in the script - is that also your understanding?

                                               

                                              You are correct, I was not getting into too much detail. It would be temporary change until the full compatibility of connectors and variable support are available for Qlik Sense.

                                                • Re: Re: Load Web File dynamically
                                                  Chris Brain

                                                  So this means that in the default case (when the default mode stabilizes) Qlik users will no longer be able to do something like this?

                                                   

                                                  LET noRows = NoOfRows('Params');
                                                      
                                                  for i=0 to $(noRows)-1
                                                       
                                                       let vParam = peek('id', $(i), 'Params');
                                                       
                                                       WebAPI:
                                                       LOAD
                                                          '$(vParam)' as Param,
                                                          col1 as data1,
                                                          col2 as data2
                                                       FROM
                                                       [https://server/api?param=$(vParam)] (XmlSimple, Table is [XXXXXX]
                                                  next
                                                  

                                                   

                                                  Without potentially complex/confusing settings file changes (which may not even be permitted by their organisation)?

                                                   

                                                  This is something which is not only key to QVSource users, but also used commonly in our experience outside of QVSource, which is also evidenced by the OP.

                                      • Re: Load Web File dynamically
                                        chandni bhowmick

                                        Hi Mike,

                                         

                                        I am trying to connect to salesforce in Qliksense . I know that to use Custom Connect, we need to enable Legacy Mode in Qlik Sense Desktop. However, when I open settings.ini in a notepad it is empty.

                                        How do I change StandardReload=1 to StandardReload=0?

                                    • Re: Load Web File dynamically
                                      Jochem Zwienenberg

                                      Hi Mike,

                                       

                                      How about Qlik Sense Server, do we also need to change a setting.ini for legacy support?

                                      Where is this setting.ini?

                                       

                                      reg. Jochem

                                      • Re: Re: Load Web File dynamically
                                        Chris Brain

                                        Can Qlik comment on the reason for this with respect to loading data from http end points?

                                         

                                        The docs state:

                                        For security reasons, Qlik Sense in standard mode does not support absolute or relative paths in the data load script or functions and variables that expose the file system.

                                        However, since absolute and relative paths were supported in QlikView, it is possible to disable standard mode and use legacy mode in order to reuse QlikView load scripts.

                                        Disabling standard mode can create a security risk by exposing the file system.

                                        But this seems related to file system access, not web urls.

                                          • Re: Re: Load Web File dynamically
                                            Michael Tarallo

                                            Hi Chris - all legacy type connectors (including file access and web urls - pre Qlik Sense) that have not been updated yet to take advantage of the new  connector architecture (which includes the ability to secure the data connectors with security rules) will fall under this category.

                                             

                                            There is a new Web File connector as you can see - when created and used in Standard mode - the ability to create security rules is possible to control access and actions with the defined connections. When these connectors also allow variables to be passed to them, I could see the Legacy mode starting to fade out.

                                             

                                            HTH

                                             

                                            Mike

                                              • Re: Load Web File dynamically
                                                Chris Brain

                                                HI Mike,

                                                 

                                                So then regarding:

                                                When these connectors also allow variables to be passed to them, I could see the Legacy mode starting to fade out.

                                                 

                                                Can we assume something like my earlier example will be possible, e.g.:

                                                 

                                                LET noRows = NoOfRows('Params');
                                                   
                                                for i=0 to $(noRows)-1
                                                    
                                                     let vParam = peek('id', $(i), 'Params');
                                                    
                                                     WebAPI:
                                                     LOAD
                                                        '$(vParam)' as Param,
                                                        col1 as data1,
                                                        col2 as data2
                                                     FROM
                                                     'lib://my api data'
                                                next
                                                

                                                 

                                                Where the 'lib://my api data' will either be able to either access the script variables (e.g. vParam) or there will be some other mechanism to pass them through (i.e. it might not look exactly like the above)?

                                            • Re: Load Web File dynamically
                                              MARCO HADIYANTO

                                              Hi Michael,

                                               

                                              I tried this link

                                              http://help.qlik.com/en-US/sense/3.0/Subsystems/Hub/Content/LoadData/disable-standard-mode.htm

                                              but when i open setting.ini, the file is blank then i add StandardReload=0 then nothing happen.

                                               

                                              Thanks

                                              Marco

                                          • Re: Load Web File dynamically
                                            Jochem Zwienenberg

                                            What Chris mentioned is what we absolutely need.

                                            Passing variables is at the moment NOT possible with a LIB connection!

                                            So we have to go back to legacy mode.

                                              • Re: Load Web File dynamically
                                                Michael Tarallo

                                                Hi Guys - I have an inquiry into Product Management, I will update you as soon as I learn more.

                                                 

                                                Mike

                                                  • Re: Load Web File dynamically
                                                    Darren Ball

                                                    Hi Mike,

                                                    I was just wondering if you had any updates on this, as this is a critical point for a lot of users?

                                                     

                                                    Thanks,
                                                    Darren

                                                      • Re: Load Web File dynamically
                                                        Michael Tarallo

                                                        Hi Darren,

                                                         

                                                        I was recently told that our PM team was working with the QVSource team, but I do not have any information on how that conversation went. I do know that variable support for the connections is on the roadmap, but there isn't a time frame on it yet.

                                                         

                                                        Currently - using Legacy mode is the work-around for this.

                                                         

                                                        Regards,

                                                         

                                                        Mike

                                                          • Re: Load Web File dynamically
                                                            Chris Brain

                                                            HI Mike,

                                                             

                                                            We have not heard anything from your PM team, could you let us know where this info came from or who we should expect to be speaking with?

                                                             

                                                            Regards,

                                                             

                                                            Chris

                                                              • Re: Load Web File dynamically
                                                                Michael Tarallo

                                                                HI Chris,

                                                                 

                                                                I'll have the account manager QVSource contact you. I recall in an email one of our team members stating they were having a call with QVSource, I could have been mistaken.

                                                                 

                                                                Mike

                                                                  • Re: Load Web File dynamically
                                                                    Chris Brain

                                                                    Hi Mike,

                                                                     

                                                                    Thanks - we have not heard from anybody yet but welcome speaking further.

                                                                     

                                                                    In addition to that, as we know for sure that there are a lot of people which this will affect (and who are already asking us about this) could we get a response from Qlik on this thread as to the reasoning and benefits behind this change for end users with regards loading from http?

                                                                     

                                                                    As noted earlier, the docs state:

                                                                    For security reasons, Qlik Sense in standard mode does not support absolute or relative paths in the data load script or functions and variables that expose the file system.

                                                                    However, since absolute and relative paths were supported in QlikView, it is possible to disable standard mode and use legacy mode in order to reuse QlikView load scripts.

                                                                     

                                                                    Which I can understand with regards loading from the file system but what is the thinking behind locking this down by default for loads from http end points?

                                                                     

                                                                    "Legacy mode" also sounds like something which could be discontinued in the future which is another point of concern. Can we rely on this being available in the long term?

                                                                      • Re: Load Web File dynamically
                                                                        Michael Tarallo

                                                                        Hi Chris - absolutely, I will take of this request.

                                                                         

                                                                        Mike

                                                                        • Re: Load Web File dynamically
                                                                          Michael Tarallo

                                                                          Hi Chris - I just wanted to clarify a few things. I was just thinking about this and wanted you to make sure you understand that Qlik Sense is NOT a replacement for QlikView. QlikView will continue to be improved upon as well going into next year, so QVSource will continue to work with QlikView new releases.

                                                                           

                                                                          Qlik Sense specifically addresses the need for self-service data visualization first with the added capabilities to create mash-ups using its extensive APIs for both connectivity, visualizations, integration and automation.

                                                                           

                                                                          • Are there specific use cases pertaining to self-service data visualization where you are using or plan to use QVSource data sources with Qlik Sense over using QlikView?

                                                                          • Did you / your customers think you needed to migrate or move over QlikView Apps to Qlik Sense (which is not the case)

                                                                           

                                                                          Now, Qlik Sense uses newer API technology and protocols and is being improved upon with each release to add new capabilities. From my understanding, QVSource had developed connectors to originally work with QlikView utilizing the http endpoints and APIs. My assumption is that QVSource will look into the new Qlik Sense connector APIs and start to develop Qlik Sense specific connectors gradually. "Legacy mode" will support QlikView 11 connectors, and will not go away until we know that we have addressed prioritized issues.

                                                                           

                                                                          I have been informed that that variable support for the LIB connectivity is on the list, however it is not at the top of the list.

                                                                           

                                                                          That being said - if you can address any specific concerns that can help prioritize this further.

                                                                           

                                                                          I have contacted the Product Manager and your QVSource account rep to inform of them of your needs and I intend for them to contact you.

                                                                           

                                                                          Kinds Regards,

                                                                           

                                                                          Michael Tarallo

                                                                          Qlik

                                                                            • Re: Load Web File dynamically
                                                                              Chris Brain

                                                                              HI Mike,


                                                                              Thanks for your message - have tried to address all your points below:

                                                                              ...wanted you to make sure you understand that Qlik Sense is NOT a replacement for QlikView. QlikView will continue to be improved upon as well going into next year, so QVSource will continue to work with QlikView new releases.

                                                                              Yes - we fully understand this and we are keen to work with both products and preferably with a single version of QVSource.

                                                                               

                                                                              Qlik Sense specifically addresses the need for self-service data visualization first with the added capabilities to create mash-ups using its extensive APIs for both connectivity, visualizations, integration and automation.

                                                                              Agreed, and I feel these are the types of scenarios where users will want to load interesting data from external web APIs which is where QVSource adds a lot of support and value.

                                                                               

                                                                              • Are there specific use cases pertaining to self-service data visualization where you are using or plan to use QVSource data sources with Qlik Sense over using QlikView?

                                                                              This really comes down to our (and Qlik's) customers and how they wish to use QlikView and Qlik Sense. We want to continue to make it easy for both sets of users to access the growing list of APIs we support.

                                                                               

                                                                              We are already in conversations prospects making purchase decisions on Qlik Sense and we are keen to be able to give them a clear message about the interoperability with QVSource. Saying we are only able to run in Legacy mode is not reassuring unfortunately.


                                                                              • Did you / your customers think you needed to migrate or move over QlikView Apps to Qlik Sense (which is not the case)

                                                                              No - we didn't think this.

                                                                               

                                                                              Now, Qlik Sense uses newer API technology and protocols and is being improved upon with each release to add new capabilities. From my understanding, QVSource had developed connectors to originally work with QlikView utilizing the http endpoints and APIs. My assumption is that QVSource will look into the new Qlik Sense connector APIs and start to develop Qlik Sense specific connectors gradually. "Legacy mode" will support QlikView 11 connectors, and will not go away until we know that we have addressed prioritized issues.

                                                                              We are not currently planning to change the way QVSource works - our user base appears to be currently very satisfied with the usability, flexibility and performance. If there is a compelling reason to change and.or our users/partners show an interest in us changing then we will of course look into it.

                                                                               

                                                                              I have been informed that that variable support for the LIB connectivity is on the list, however it is not at the top of the list.

                                                                              This is good to hear (well, not that it's not top of the list ).

                                                                               

                                                                              That being said - if you can address any specific concerns that can help prioritize this further.

                                                                              I think I have done this now - specifically something like my post Oct 15, 2014 8:55 AM is what I currently feel we would like to see possible.

                                                                               

                                                                              I have contacted the Product Manager and your QVSource account rep to inform of them of your needs and I intend for them to contact you.

                                                                              This is much appreciated and I believe we now have a call scheduled.

                                                                               

                                                                              Thanks again Mike.

                                                                                • Re: Load Web File dynamically
                                                                                  Jochem Zwienenberg

                                                                                  It's not a replacement, but some customers like the natural UI/analytics and are serious thinking about using Sense for some replacement dashboards. I would not like to see another specific version of QVSource for Sense. Sense makes it possible to import existing QVW's (script 99% Compatible). So it would be nice that Qlik could solve the problem regarding the LIB Connections (Variables), i think it really needs your serious attention (priority) for the future (for now we can enable the legacy support).

                                                                                   

                                                                                  thnx again!

                                                                              • Re: Load Web File dynamically
                                                                                Ian Crosland

                                                                                Hi Chris

                                                                                 

                                                                                The limitation of the LIB function with http end points is not a design decision its just a limitation at the moment and is on the roadmap to include extra support for LIB/http connections including variable support.  Legacy mode will remain in the product to cater for these limitations and to overlap with people wishing to migrate QlikView scripts into Sense in a staged way

                                                                                 

                                                                                hope this clarifies

                                                                                 

                                                                                thanks

                                                                                 

                                                                                Ian

                                                                  • Re: Load Web File dynamically
                                                                    Pablo Labbe

                                                                    Hi ,

                                                                     

                                                                      Any updates on this thread ?