Qlik Community

QlikView Documents

Documents for QlikView related information.

MongoDB + QlikView.pptx

Not applicable

MongoDB + QlikView.pptx

Querying MongoDB directly with QlikView is pretty easy to set up.  You don't need any 3rd party extensions. If anyone is interested in learning more about it you can take a look at a basic presentation that I put together.

Comments
Not applicable

ok Thanks.

Thanks and Regards

Amol Khochare

Not applicable

Perfect thanks.

Could you tell us a bit more about the high volume performance ? What is your experience so far ?

Thanks

vadimtsushko
Contributor III

Interesting approach.

I personally probably would not use it, and would use standard mongodb tools instead.

So I would use mongoexport utility to get csv file from mongodb, for example:

mongoexport --db scratch --collection zips --csv --fieldFile fields.txt --out c:\export\zip.csv

and then load that csv file to QlikView

For me it is preferable way because:

  • No need to bother about running ruby script as a web server (propably as a windows service) and check if it is live on reload time
  • Eliminate convert from json to xml stage of whole operation
  • I suspect (though not sure) that qlikview is more performant in loading csv formatted data then xml data
  • REST API embedded in mongod is somewhat limited and do not include projection operators for example (you cannot select only some fields from collection with big number of fields). On the other hand mongoexport is fully capable utility
  • With mongoexport data are sent over network only one time and in efficient binary BSON format over TCP socket. In proposed scheme data are sent over network twice: as text format JSON and as text format XML, both time over high level HTTP network layer.

So on the whole in my opinion simple scheme with intermediate staging of data in csv file shoud  be more robust and performant

Not applicable

Bertrand, I have not done an extensive amount of performance testing but one nice feature of the MongoDB REST interface is that you can parameterize your queries like this:

http://127.0.0.1:28017/databaseName/collectionName/?filter_a=1&limit=-10

This way you can filter your data in MongoDB and only bring a smaller subset into Qlik. 

Not applicable

Yea but CSV is probably not great if your data is hierarchical and you would like to preserve the relationships in Qlik. 

But yea good points, I like your approach.

Employee
Employee

I would recommend looking at the Datascript Module available with QlikView Expressor:

http://community.qlik.com/docs/DOC-4433/version/5

vadimtsushko
Contributor III

olgaavram: Good point.

While one can easily flatten mongodb collection with embedded documents by mongoexport, flattening collection with embedded arrays require either mongodb aggregation framework (which is currently not supported by mongoexport) or scripting.

In that case I would use any available mongodb client (ruby in your case) to get flat csv data from mongodb.

I believe all efficiency and robustness considerations from my previous post (network utilization and so on) apply in that case too.

Additionally I suspect that writing in ruby aggregation framework's query or simple snippet with loop to get flat data from mongodb collection would be easier then to flatten deeply nested XML collection in QlikView script.

Obviously QlikView Expressor gives other options.


Not applicable

Well, thanks all. We finally used mongoexport tool which gives us a 3 huge csv (3 Go) but thank to Qlikview it was greatly compressed into a nice little qvd.

In fact, we didn't need to restrict the perimeter. We needed everything.

Thanks to all of you. Have a nice weekend.

Version history
Revision #:
1 of 1
Last update:
‎04-09-2014 12:44 AM
Updated by: