I do a lot of ETL/ELT. We have been exploring options for loading into BigQuery, and I'm curious to find what others are doing.
I see multiple products from Google, but not sure exactly where I should reach. Looks like there's overlap there.
Right now most of my data is batched loaded at a certain interval. Mostly hourly or daily.
Usually comes in via CSV's that we have to preprocess and MERGE, but other times we have to hit API's to extract data.
I have explored other things like Spark via Databricks and hosted like Rivery and Fivetran, but just trying to figure out what else can be done with native Google products.
