Data Load from virtul table in mongob via Qlik ODBC connector

[fabdulazeez]

Hi,

We are connecting to MongoDB via Qlik mongodb connector. For a denormalized collection in mongodb , driver provides  base table  with normal data from the collection and a virtual table for each column of denormalized data, expanding the nested data.

When script fetching filtered data from the virtual table is executed I am seeing a mongodb query in the driver logs which fetch all the data from the collection with projection as the child. Is this the expected behavior ? Since I believe fetching all the records from collection will put a heavy load on the database, can anyone confirm if this is the actual behavior of the driver or if the logs are providing wrong queries.

Jun 28 09:57:41.983 DEBUG 9592 Simba::MongoDBODBC::MDClient::Query: db.collection1.find({ }, { "skip" : 0, "limit" : 0, "batchSize" : 4096, "noCursorTimeout" : false, "projection" : { "child" : true } })

We have also tried joining the base table and virtual table. In that case I can see 2 query on the driver logs, one fetching filtered data from collection and another fetching whole collection with projection only for the child array(Virtual table).

Jun 29 09:29:17.721 DEBUG 13880 Simba::MongoDBODBC::MDClient::Query: db.collection1.find({ "$and" : [ { "$or" : [ { "client" : { "$eq" : "DXB" } }, { "client" : { "$not" : { "$type" : 2 } } } ] }, { "$or" : [ { "flag" : { "$eq" : "A" } }, { "flagAd" : { "$not" : { "$type" : 2 } } } ] }, { "$or" : [ { "bestKnownTime" : { "$gt" : { "$date" : 1640736000000 } } }, { "bestKnownTime" : { "$not" : { "$type" : 9 } } } ] }, { "$or" : [ { "bestKnownTime" : { "$lte" : { "$date" : 1656581356000 } } }, { "bestKnownTime" : { "$not" : { "$type" : 9 } } } ] } ] }, { "skip" : 0, "limit" : 0, "batchSize" : 4096, "noCursorTimeout" : false, "projection" : { "bestKnownTime" : true, "client" : true, "flag" : true, "home" : true, "rotKey" : true, "DateLocal" : true, "Rotation" : true } })

Jun 29 09:29:17.766 DEBUG 13880 Simba::MongoDBODBC::MDClient::Query: db.collection1.find({ }, { "skip" : 0, "limit" : 0, "batchSize" : 4096, "noCursorTimeout" : false, "projection" : { "child" : true } })

Hi @Vinay_B ,

Can you please guide me here?