I’m Smars Hu. A data engineer & minimalist based in Toronto 🇨🇦

God help those who help themselves

自救者，人恒救之

🔨 3 Years of Experience in Data Engineering

💼 Data Engineer @ ScotiaBank, Toronto, Canada🇨🇦

🎓 Master of Science in Big Data Analytics @ Trent University, Ontario, Canada 🇨🇦

📝 My website

📞 Book a meeting with me

Simulated an enterprise-level on-premise self-managed big data distributed cluster using Docker containers. Integrated components include Hadoop, Zookeeper, Spark, Hive, MySQL, Airflow, Prometheus, ClickHouse, and Power BI. Developed a data warehouse for an e-commerce backend based on dimensional modeling theory and built a BI analytics system for reporting and data analysis.

Reproduced a modern enterprise-grade Azure cloud data engineering architecture widely adopted in North America. Leveraged technologies such as Databricks, PySpark, ADLS Gen2, Unity Catalog, Delta Lake, Power BI, and Azure Data Factory (ADF) to develop cloud-native data pipelines on Azure and perform exploratory data analysis (EDA).

☘️ Languages

☘️ Distributed Computation & Data Warehouse

☘️ Streaming & Lakehouse Architecture

☘️ Data Engineering Practices

• Theory:

• Data Orchestration:

• Data Quality:

• Data Governance:

• Data Visualization & BI:

• Data RESTful API Services & Integration:

☘️ Databases: OLAP, OLTP & NoSQL

(OLTP)

(OLAP)

(NoSQL/Search)

☘️ Cloud-Native Data Engineering, Containerization & Platform Tools

(Synapse, ADLS Gen2, Databricks, Data Factory)

(S3, Lambda)

☘️ DevOps & Monitoring:

☘️ Basic Tools

(AI)

(OS, Version Control, API, Dev environment)

(Project Management, Doc)