topic Re: Qlik Data Calatalyst and Sqoop - Database Import and Export in Catalog and Lineage

Qlik Data Calatalyst and Sqoop - Database Import and Export

brunnovb_in — Tue, 07 Dec 2021 22:35:04 GMT

Hello everyone!
I am trying to make use of sqoop to import data to QDC (DB -> QDC) and also to export QDC data (QDC -> DB), but I am not succeeding. I've used several sqoop command attempts, like the example below. Could you support? An example of trying sqoop command is below.

sqoop import --connect "jdbc: sqlserver: //apps.in1.com.br:1433; databaseName = DB_DEMO" --username qlik_demo --password 'qlik123' --warehouse-dir / demoqdc

Could you give me an example of sqoop import and export command?

Regards,

Brunno V.B.

Re: Qlik Data Calatalyst and Sqoop - Database Import and Export

Clever_Anjos — Tue, 29 Oct 2019 12:29:57 GMT

@brunnovb_in why are you using scoop to onboard data and not using the regular workflow?

About exporting data to RDBMS, you could use the OPENCONNECTOR target and please check the manual

http://<yourserver>:8080/podium-webhelp/v1/index.htm#t=SOURCE%2FOPENCONNECTOR_Usage_and_Tips.htm&rhsearch=OPENCONNECTOR&rhhlterm=OPENCONNECTOR&rhsyns=%20

Re: Qlik Data Calatalyst and Sqoop - Database Import and Export

lucassad — Tue, 29 Oct 2019 15:04:01 GMT

Hi Brunno,

Open Connector will run a sh script, so you will need to create a script to do a data import and additional properties to send the parameters to your script.

To create new properties and use in your sh script you will need to use QDC API to create them (PUT /propDef/v1/save).

You will find in QDC help 2 examples, DistCp script and Secure Copy (SCP/SSH) script, you can use the sample scripts to start your own.

---------

In case you are planning to insert data into an MS SQL DB you can also use the bcp tool in your sh script. I am adding some MS links here as a reference for you.

https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-migrate-bcp?view=sql-server-2017

https://docs.microsoft.com/pt-br/sql/linux/sql-server-linux-setup-tools?view=sql-server-ver15

Thanks,