Skip to main content
Announcements
Introducing Qlik Answers: A plug-and-play, Generative AI powered RAG solution. READ ALL ABOUT IT!
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

HTTP LOAD

Hello,

I need to retrieve table informations through web site HTTP connection.

It works fine with anonymous (public) web site but I don't know how to pass login/password for protected web site

Best regards

Régis

13 Replies
nathanfurby
Specialist
Specialist

Not applicable
Author

Hi!

There are several ways to go about this. The ways I know of, you'll need a bit of macro code. You can choose to either save the web-site locally and then fetch the data from the tables, or run the macro within the script.

A logon macro that saves the HTML to files on disk could look like:


sub RetrieveData

dim FileName
dim FilePath

FilePath = "C:\Storage\HTML\"
Set objShell = CreateObject("WScript.Shell")

'Creates our IE object
Set IE = CreateObject("InternetExplorer.Application")

'Browse to our url
IE.Navigate "http://www.randomdomain.com/login.aspx"
IE.Visible = False 'True if you want to see the process

'wait for the page to load
Do While IE.Busy
Loop

'log in to the site - the name of these inputboxes needs to be identified in the HTML-code
IE.Document.All.Item("txtUsername").Value = "JohnDoe"
IE.Document.All.Item("txtPassword").Value = "Abcd1234"

'submit the page - the name of this button needs to be identified in the HTML-code
IE.Document.All.Item("LbtnNext").click

'wait while the form is submitted and the resulting page is loaded.
Do While IE.Busy
Loop

' Following code for each page to load and be saved to the proper folder
IE.Navigate "http://www.randomdomain.com/Data/Page1.aspx"

'wait for the page to load
Do While IE.Busy
Loop

'Assign the webpage results to a variable
sDocHTML = IE.Document.documentElement.innerhtml

'Assign path
FileName = FilePath & "SaveAsThisName.aspx.htm"

'Save the HTML
WriteFile FileName,sDocHTML

' Repeating for each page, can be made more dynamic in case of multiple page retrieval

IE.Navigate "http://www.randomdomain.com/Customers/Page2.aspx"

'wait for the page to load
Do While IE.Busy
Loop

'Assign the webpage results to a variable
sDocHTML = IE.Document.documentElement.innerhtml

'Assign path
FileName = FilePath & "AnotherSaveAsNameHere.aspx.htm"

'Save the HTML
WriteFile FileName,sDocHTML

IE.Quit

end sub

' Function for writing contents to file
function WriteFile(sFilePathAndName,sFileContents)

Const ForWriting = 2

Set oFS = CreateObject("Scripting.FileSystemObject")
Set oFSFile = oFS.OpenTextFile(sFilePathAndName,ForWriting,True)

oFSFile.Write(sFileContents)
oFSFile.Close

Set oFSFile = Nothing
Set oFS = Nothing

End function


I hope it helps you get started!

BR

Jakob Berglund

Not applicable
Author

Thanks a lot

I have to look at these 2 solutions

About curl, I have tried with qlikcommunity with my login/password but i don't succeed

NAthanFurby, have you ever try to go to retrieve qlikcommunity web page with your login/password with curl ?

Regards

Regis

nathanfurby
Specialist
Specialist

No I have not tried accessing QlikView Community.

Not applicable
Author

Just so you can try it for yourself, in order to log in to QlikCommunity replace the appropriate part of my code above with this :


'log in to the site
IE.Document.All.Item("ctl00$ContentPlaceHolder1$txtUser").Value = "YourUserNameHere"
IE.Document.All.Item("ctl00$ContentPlaceHolder1$txtPassword").Value = "YourPasswordHere"

'submit the page
IE.Document.All.Item("ctl00$ContentPlaceHolder1$Button1").click


If you want to see it working, you may also set the IE.visible to true instead of false.

BR

Jakob Berglund

Not applicable
Author

it works fine

Thanks a lot

Regis

Not applicable
Author

Hi,

qlikCommunity works without a password :-)But your browser uses a proxy . Pass than proxy to CURL and you're ready to go . As simple as


curl --proxy 10.10.10.10:8080 http://community.qlik.com/


-Alex

Not applicable
Author

Hi,

You can't schedule tasks in QEMC / Windows Task Manager to run when you're not logged in using the VBscript above.

You know, simple things should be easy. Complex things should be possible.

Curl is able to script out complex scenarios, since 2002 and earlier.

  • connecting to a web site
  • retrieve the needed form fields and random challenge (meant to prevent brute force password trials and spammers)
  • login and save cookie
  • navigate to inbox using the cookie above

http://curl.haxx.se/mail/archive-2002-03/0210.html

You can google for more curl examples using form authentication,cookies, SharePoint.

-Alex

Not applicable
Author

I'll definately check out curl. Always been looking for a better alternative 😉