Data Engineer | Data Pipelines & Lakehouse | Open-Source Contributor
I design and build reliable data pipelines — from ingestion to analytics-ready datasets. My day-to-day work is ETL/ELT in Python and SQL, large-scale processing on Apache Spark and Databricks, and dimensional modeling, with a focus on getting incremental ingestion right: CDC, Delta Lake MERGE, and clean dbt transformations.
- Languages: Python, SQL, Rust
- Data Processing: Apache Spark, Databricks, Delta Lake, dbt, Apache Airflow
- Databases: PostgreSQL, MS SQL Server, SSIS, SSAS
- Data Manipulation: Pandas, NumPy
- Architecture: Medallion Architecture, CDC, Delta MERGE, Dimensional Modeling
- Testing & Quality: pytest, Rust tests, SQL validation
- 🔵 Contributing to Apache, Delta Lake, dbt Labs, and other data-engineering open-source projects
- 🔵 Building reliable ETL/ELT pipelines and lakehouse architectures
- 🔵 Working with Delta Lake, Apache Spark, and Databricks at scale
- 🔵 Exploring query optimization, data quality monitoring, and incremental ingestion patterns
I contribute to bug fixes, documentation, tests, and performance improvements across data-engineering projects.
| Organization | Project | Status | Pull Requests | Contributions |
|---|---|---|---|---|
| Delta Lake | delta-rs | 4 merged | #4832, #4833, #4866, #4890 | Schema merging nullable fix (Rust), deletedFileRetentionDuration docs, TIMESTAMP UTC offset fix, multiple conditional clauses merge examples |
| Kedro | kedro | 2 merged, 1 open | #5809, #5808, #5828 | CachedDataset docs example, templating concept page rewrite, OmegaConfigLoader replace=True resolver fix |
| SQLFluff | sqlfluff | 1 merged | #8715 | Removed outdated Snowflake USING note from Rule ST07 |
| Airbyte | airbyte | 1 merged | #88468 | Fixed dead low-code CDK sample connector links |
| ClickHouse | ClickHouse | 1 merged | #125739 | Version note for Replicated engine default_replica_path settings |
| Apache | Iceberg | 1 open | #18468 | Spark branch write precedence docs |
| Apache | DataFusion | 2 open | #26198, #26489 | Validate pruning-ratio config at SET time, width_bucket bucket-computation dedup refactor (approved) |
| Apache | Airflow | 1 open | #74424 | Missing tests for cncf.kubernetes utilities |
| dbt Labs | dbt | 1 open | #16690 | Log effective thread count at startup |
| OpenMetadata | docs-om | 1 merged, 1 open | #615, #599 | Full Stack dev container setup fix, OpenSearch FGAC permissions guide |
| Canonical | craft-cli | 2 open | #504, #510 | TypeError for non-string items in humanize_list, strip Sphinx markup from plain help overview |
| Canonical | craft-providers | 1 open | #1026 | Typed error for unsupported Ubuntu version in base checks |
| Canonical | starbase | 1 merged | #638 | Reuse index.rst first sentence for llms.txt description |
| Canonical | rockcraft | 1 open | #1401 | Prefix base digest with sha256 in OCI annotation and rock metadata |
| lakeFS | lakeFS | 1 open | #10555 | Squash merge option in branch merge dialog |
| Bytewax | bytewax | 1 open | #637 | Expose bytewax.version, git_revision, git_dirty build info |
- Languages: Python, SQL, Rust
- Data Processing: Apache Spark, Databricks, Delta Lake, dbt, Apache Airflow
- Databases & Warehousing: PostgreSQL, MS SQL Server, SSIS, SSAS
- Data Manipulation: Pandas, NumPy
- Architecture: Medallion Architecture, CDC, Delta MERGE, Dimensional Modeling
- Testing & Quality: pytest, Rust tests, SQL validation
- 11 merged PRs across Delta Lake, Kedro, SQLFluff, Airbyte, ClickHouse, Canonical, and OpenMetadata
- 13 open PRs across Apache, dbt Labs, OpenMetadata, Canonical, lakeFS, and Bytewax
- Contributing to docs, tests, bug fixes, and performance improvements
- Data Engineer — Alphabridge Consulting Ltd, Lahore (Aug 2026)
- Teaching Assistant, DBMS & Data Warehousing / BI — PUCIT, Lahore (Jan 2025 – Jan 2026)
- BS Data Science — University of the Punjab, PUCIT (2022 – 2026)
- Email: shahzebali1218@gmail.com
- GitHub: Shahzeb-DataEngineer
Building reliable data pipelines, one PR at a time. 🚀