IT amazed by Qlik's ETL capabilities. Asking if writeback to DB is possible?

[vkish16161]

All,

My IT team is amazed by Qlik's ETL capabilities. One colleague exclaimed "Qlik can do that?". Now, they want to do a POC on augmenting our ETL tool. Possible cost cutting measure.

They want to do all cleansing in Qlik and write it back to DB.

Is this natively possible in Qlik and if not, are there any third party tools that can do this?

P.S - Could storing the tables in QVX  format help?

Thanks!

rwunderlich‌ marcus_sommer‌ mto‌

Message was edited by: Vishnu Chakravaram

[Michael_Tarallo]

Hey Vishnu -

So - IMHO - I would highly caution using the term "eliminating" - ETL tools have there place and are more structured along the lines including orchestration and workflow - usually all done with a slick GUI. Qlik's has "ETL" capabilities, but I use the term loosely - as it greatly depends on the solution. You can use Qlik script to perform programmatic like functions to transform and augment data and write out . QVD files (in-memory representation on disk) - but writing back to the DB is not technically what it was designed to do. I am not saying it cannot be done, but there are better ways to achieve what you are looking for. If you want to continue this discussion let us know and we will do our best to assist.

If you are looking to cut costs, you may want to look at Pentaho Data Integration or the open source project it is based on Kettle - I worked for Pentaho for 5 years - and for a free ETL tool - it is quite amazing what it can do. There is a licensed or subscription based enterprise version as well.

https://sourceforge.net/projects/pentaho/files/Data%20Integration/

When applicable please mark the appropriate replies as CORRECT. This will help community members and Qlik Employees know which discussions have already been addressed and have a possible known solution. Please mark threads as HELPFUL if the provided solution is helpful to the problem, but does not necessarily solve the indicated problem. You can mark multiple threads as HELPFUL if you feel additional info is useful to others.

Regards,

Michael Tarallo (@mtarallo) | Twitter

Qlik

[vkish16161]

Mike,

Thanks for the honest answer. I have loosely worded my post. But in all reality, IT has some intrinsic red tape and they never "eliminate" a tool. Pardon me for inaccurate statements. (I have corrected my above post.)

IT's just interested in knowing if there are any Qlik 3rd party tools that can do a writeback to DB. Please let me know if you come across such tools now or in the future.

Is storing in a QVX file any better than QVD? (Probably a seperate thread)

Thanks

[Michael_Tarallo · Accepted solution]

- thanks Vishnu - always need to be a bit politically correct when it comes to IT and their toys!

So output to QVX is the open format used by partners / 3rd party tools - take a look at this post for more info:

Pentaho Data Integration QVX Writer Plugin

[jerifortune]

Hi Mike,

Most of us are aware that neither Qlik nor any other BI tool is designed to handle writeback. It is not even ideal as such functionality undermines data integrity. However, out of passion for your tool, many clients want to use Qlik as workflow tool. I have similar requests from Qlik clients whom I support. In my case, end users will trigger the writeback.

[Michael_Tarallo]

Thanks Jerry - agreed - we have partner solutions I believe that handle stuff like this as well.

[jerifortune]

Thank you Mike

I hope to present Nick's approach first. and will contact you if they are not satisfied with the process.

[rwunderlich]

After cleansing, in addition to storing the tables as QVDs, you can store a copy as CSV as well.

STORE mytable into mytable.csv (txt);

That CSV can be consumed by a database bulk import process.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

[vkish16161]

Rob,

Thanks that's what I demoed to my team.

[daniel_s-cubed]

Hi Vkish,

I hope you have found a solution to your request.

If not, then you should take a look at our native Qlik Sense write-back extension "Komment":

Komment 

It only use native Qlik functionality and APIs and hence, it can be installed right away without the involvement of IT. All your comments are stored to QVDs so there is no need for external services or databases.

Happy commenting 

All the best

Daniel