topic How to ingest filtered datasets? in Catalog and Lineage

How to ingest filtered datasets?

ssamuels — Tue, 07 Dec 2021 22:35:04 GMT

Is it possible to create new entities in the Source module by loading filtered data from RDBMS through custom sql queries?

Re: How to ingest filtered datasets?

DaveHowland — Tue, 23 Jun 2020 01:24:21 GMT

@ssamuels ,

Yes, in the QDC Help, search for 'src.file.glob' in Qlik Data Catalyst Source Properties, and you will find what you need. Against RDBMS sources, that property drives the query that's executed to pull the data for that entity. When we build it anew, it's simply a list of the columns in the table, but it can be edited. Want to query a view? Add a sub-select, or a join? All are allowed.

Hope that helps,

David

Re: How to ingest filtered datasets?

DaveHowland — Tue, 30 Jun 2020 13:32:35 GMT

@ssamuels ,

Can I ask I this solution is the one you were looking for? If so, would you mind closing the case? Otherwise, any other questions are welcome.

Thanks, David

Re: How to ingest filtered datasets?

ssamuels — Tue, 30 Jun 2020 13:48:32 GMT

Thanks @DaveHowland for your answer.

Re: How to ingest filtered datasets?

ssamuels — Tue, 30 Jun 2020 13:53:11 GMT

@DaveHowland Does this mean that I can use this property to specify my own custom sql queries to pull the data from the database?

Re: How to ingest filtered datasets?

DaveHowland — Tue, 30 Jun 2020 13:59:54 GMT

@ssamuels

100% correct. That's how you can build an entity in QDC that will reflect any values you would like. Subqueries, LOJ's, Views, partitions, etc ... all are supported. If it's SQL-92, it should work.

thanks, David