toolkit 2.3.0
Posted January 12, 2023 ‐ 1 min read
Toolkit 2.3.0 released
Bugfixes
mark_deletes
didn’t work correctly in some cases for Step.Run transformation until...
didn’t work in Flow UI Sandbox.- Fields, entity names and paths should be properly escaped now.
- Notification email should now be sent properly even if the failing DAG didn’t produce any log file.
- Adding columns to incremental table should now work better.
New features
- BigQuery transformations now run in single session.
- Airflow configuration tweaks to improve performance.
- Fewer edges in DAGs should lead to better run times.
- Added few unit tests.
- SSH connection to worker can be initiated using both SSH key and password.
- Several more exceptions are now caught and retried when connecting to worker.
- All non-comment queries should now run in SQL transformations.
- Error will be raised on duplicate tables on transformation input.
- Few minor changes to improve stability and tasks' performance.