-
Notes: Snowflake copy into
Be careful with copy into it has some nuances Copy into only loads changed files – running the command on the same set of files that have already loaded will do nothing. Supposedly the mechanism here is a checksum on the filename + contents, although I can’t find any docs explaining how this works in…
-
Creating a DAG Factory in Airflow
If you are using Airflow to orchestrate DBT, and if like me you prefer to isolate pipelines into logical units, then you will have noticed the inherent repetition that comes with this approach. For example, perhaps you have a set of models which read from operations_db source and write to a set of operations marts,…
-
Interfaces in Go
Lets first define two types, circle and rectangle. Both types have the method area(). In main() both objects t, c are created then the area() method is called on each. What if we wanted to loop over those objects? We cannot create a slice as t, c are of different types, and a slice must…
-
Cross account roles in AWS
A recent project involved connecting a service hosted on a third parties AWS cloud to our AWS cloud. The service needed to be able to read secrets from Secrets manager, and be able to read/write files to S3. To do this you need to setup a cross account role that the principal (the third party)…
-
Subscribe
Subscribed
Already have a WordPress.com account? Log in now.
oddments 