Selecting File Content from GitHub

[peterkunhardt]

We have recently set up a GitHub repository for all our Business Intelligence work. It's made change management dramatically easier for us, but I'd like to take things to the next level. My hope is that rather than write SQL statements directly into my Qlik Apps, I can keep the SQL statement on GitHub, and then with each reload of an App, fetch the content via the Web Connector and pass that content to my data connection. This would allow us to maintain all SQL statements in a single repo, and if multiple Qlik Apps use the same script, changing the script in our GitHub will effectively change that script in all Qlik Apps that point to it.

My thinking on how that would work so far is:

1. Define a table as a single row with the content of a particular GitHub file as the content of the row.
2. PEEK() at the row and set a variable equal to that content.
3. Create a new table with the load statement defined as 'SQL' followed by the content of the variable.

Steps 2 and 3 are easy enough, I already use PEEK() and pass variables as a small part of an SQL statement all the time. However, I'm struggling to see how to get step one working. Anyone have any thoughts on how/whether it is possible to fetch the content of a file over the GitHub Web Connector? I see there is a "download file" option, perhaps my goal should be to use the Web Connector to fetch the file & save it locally, (overwriting the file if it already exists) then use a "Folder" connection to fetch the content of the resulting .txt?

Any thoughts or suggestions appreciated!

- Pete

[rwunderlich]

I haven't tried using the GitHub connector to pull file data, but I do pull data using $(include).  I make a Web File connection (named QvcWeb for example)  that references the raw.githubusercontent.com url like:

http://raw.githubusercontent.com/RobWunderlich/Qlikview-Components/master/Qvc_Runtime/Qvc.qvs

and then a script include like:

$(Must_Include=lib://QvcWeb);

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

http://www.easyqlik.com

[peterkunhardt]

Hi rwunderlich,

Thanks for this idea, this appears to work great! Unfortunately this is a private repo, meaning the raw.githubusercontent.com/ link only work when I include the "?token=<token>" parameter at the end of the link. (Otherwise it throws a 404 error, thankfully) Compounding this misfortune: that token gets updated any time a new commit affects the named file. This means it will not be durable enough to be committed to a data connection.

I did also find this,‌ which would seem to suggest that one can generate a personal OAuth token that can be used permanently to access the raw files through CURL, but having generated a token with the "repo" permissions and tested it with the simple URL method, it appears not to work.

Would be grateful for any other insights you may have.

- Pete