• Design and implement data pipelines using Apache Spark and Delta Lake.
• Perform data modelling, query optimization, and performance tuning.
• Develop Python-based transformations and automation scripts.
• Build and monitor ADF pipelines and data workflows.
• Implement unit testing and integrate with CI/CD practices.
• Collaborate with cross-functional teams for scalable data solutions.

Essential Skills:

• Apache Spark & Delta Lake
• SQL Server / Synapse
• Python
• Data Modelling
• Data Pipelines (ADF or equivalent)