I do a lot of ETL/ELT. We have been exploring options for loading into BigQuery, and I'm curious to find what others are doing.
I see multiple products from Google, but not sure exactly where I should reach. Looks like there's overlap there.
Right now most of my data is batched loaded at a certain interval. Mostly hourly or daily.
Usually comes in via CSV's that we have to preprocess and MERGE, but other times we have to hit API's to extract data.
I have explored other things like Spark via Databricks and hosted like Rivery and Fivetran, but just trying to figure out what else can be done with native Google products.
So, how do you do load?
