Who's worked with dbt and how does it compare with Qlik load script or SQL & SSIS?

[clinton113]

I know it's the hottest thing right now in the ETL/ELT space, but is it really any good? Coming from a strong Qlik background, where Qlik's load script, if you know what you are doing can be really fast to get a solution going - is dbt just as fast to develop a solution in? How does dbt compare to using raw SQL and SSIS packages to build a simple star schema data warehouse? I've watched some YouTube videos on dbt, which are really showing me that it seems labour intensive, but perhaps once you know it, it's going to be really powerful. I do prefer ETL to ELT - so that could be where I just don't get it; all my Qlik load scripts Transform the raw data and we do the same with SQL. It looks like dbt is a good tool for data engineering and getting data from many sources into a, let's call it data repository. Should I invest the time to learn it?

[astro0]

Full disclosure - I am a huge fan of dbt.

Pros: super easy to learn (it’s just SQL and Jinja). There’s version control so it democratizes development of transformation code. There are tests you can input. Leverages your data warehouse for doing the transformations. Also there’s a free version

Cons: a lot of companies just plain out use it incorrectly, mainly by using it to implement band aid solutions when they should be engaging with other teams for better fixes. I blame the ease of use here as well as some of the marketing messaging. Lack of formal training by dbt Labs also plays a part in the above

I currently consult for a company that uses Qlik and I hate it. It’s super slow. They start going nuts when mentioning anything where a script transforms hundreds of MBs of data. I can’t wait to introduce them dbt