Loading a database table with "slowly changing dimensions" in Python

Question

Ok, I can load records into a table using to_sql in pandas. (Unfortunately, I cannot use bcp or bulk insert, because my (SQL Server) database server is remote). How about a table with (Type 2) slowly changing dimensions? In SSIS, I would use SCD Wizard - what is the Python alternative? I would like to avoid coding up SCD logic from scratch - hopefully, there is a package (perhaps one of these) which does that? Simply loading a table is fine and dandy, but in the world of dimensional datamarts one surely has to support SCD?

Wheelhouse

In my experience SQLalchemy does not care whether your schema changes. You can load it in no problem. If you need to upload data, are you hoping to append data to an existing table? That might pose a problem if you're removing or adding columns. Django handles this through migrations and requires default values where necessary. Can you give a small example for the SCD you're trying to do? – Latialatices

try this github.com/rtogo/pyscd – Moribund

chrthomsen.github.io/pygrametl – Moribund

pygrametl appears to be mature. There is a paper available (pdf) at: dbtr.cs.aau.dk/DBPublications/DBTR-25.pdf – Ramiform