Building Streaming Data Pipelines with Qlik Open Lakehouse

[Troy_Raney]

This Techspert Talks session covers:

• Streaming data via AWS Kenisis
• Qlik Open Lakehouse capabilities
• From Pipelines to Analytics

Chapters:

1. 01:11 - Tour de Qlik IoT demo
2. 02:20 - End-to-End Architecture
3. 04:00 - Streaming Sources
4. 05:36 - Qlik Open Lakehouse optimization
5. 06:32 - Lakehouse Cluster Architecture
6. 07:28 - Possible use cases
7. 08:31 - How to build a Stream data pipeline project
8. 12:23 - Target table settings
9. 13:53 - The AWS S3 bucket environment
10. 15:35 - Data quality - Trust Score
11. 17:39 - Improving data quality
12. 21:44 - Setting up partitions
13. 22:31 - Look at Optimized S3 bucket
14. 23:32 - Creating analytics
15. 24:31 - Tour de Qlik app
16. 27:30 - Q&A: Can this be used with any analytics tool?
17. 28:20 - Q&A: What is a good monitoring tool?
18. 29:16 - Q&A: Where should transformations be done?
19. 31:04 - Q&A: Is Streaming possible with Replicate?
20. 31:54 - Q&A: What version of Kafka is supported?
21. 32:47 - Q&A: Can JSON data be extracted with transformations?
22. 36:12 - Q&A: Can we see this demo at Qlik Connect?

Resources:

• Build on Iceberg with Qlik Open Lakehouse
• Benchmarking Costs Qlik Open Lakehouse Vs a Data Warehouse
• Setting up Qlik Open Lakehouse
• Creating and managing data marts
• Qlik Learning: Qlik Talend Cloud  - Open Lakehouse
• Qlik Talend Data Integration Certification Exams

Click here to see transcript