Skip to main content
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
cancel
Showing results for 
Search instead for 
Did you mean: 
hisen22
Contributor
Contributor

New job - stuck with Qlik

Hey guys,

I started a new job couple of months ago and I have questions for you.

Company : Retail company with millions and millions of rows of sales. Thousand of sales daily

Currently : Each morning Qlik automations retrieve data for the DB and store it in .qvd files (qlik format) and then the different dashboard read those files

Scenario :

I'm the only BI person of the company and my job consist to retrieve data from the so called PostgreSQL "data warehouse" (but for me it's more of a database because it's straight raw facts and dimensions tables that we need to join) that our ERP company made available for us, transform that data and create different dashboards with Qlik.

At the moment, the company decided to do everything in Qlik, so the data is extract from the DB with qlik, transform in qlik load script and then use in the different reports.

I'm looking to modernize the stack, implement a more robust ELT/ETL process and create a more scalable approach because the company plan to grow a lot in the future.

What would you suggest to use at each moment of the process ? (preferably open-source)

Just so you know, intermediate person here (great ELT/ETL, SQL and Python knowledge).

Thanks

Labels (1)
1 Reply
AlexOmetis
Partner - Specialist
Partner - Specialist

I'd look at Qlik Cloud Data Integration (QCDI) if you're wanting to implement a highly efficient and customisable data warehouse in something like Snowflake. QCDI looks after the ingestion of data (using CDC where possible), building Operational & Historical Data Stores as well as allowing simple and customised push-down transforms. It's ELT, although I believe some ETL options may come to the product in the future.

This also integrates very well with Qlik Sense SaaS for analytics. 

Some useful links: