toolkit 2.3.0

Posted January 12, 2023 ‐ 1 min read

Toolkit 2.3.0 released

Bugfixes

  • mark_deletes didn’t work correctly in some cases for Step.
  • Run transformation until... didn’t work in Flow UI Sandbox.
  • Fields, entity names and paths should be properly escaped now.
  • Notification email should now be sent properly even if the failing DAG didn’t produce any log file.
  • Adding columns to incremental table should now work better.

New features

  • BigQuery transformations now run in single session.
  • Airflow configuration tweaks to improve performance.
  • Fewer edges in DAGs should lead to better run times.
  • Added few unit tests.
  • SSH connection to worker can be initiated using both SSH key and password.
  • Several more exceptions are now caught and retried when connecting to worker.
  • All non-comment queries should now run in SQL transformations.
  • Error will be raised on duplicate tables on transformation input.
  • Few minor changes to improve stability and tasks' performance.