Note: The job is a remote job and is open to candidates in USA. PHT Staffing LLC is assisting a client in hiring an experienced Data Engineer. The role involves designing and maintaining data pipelines, optimizing ETL workflows, and collaborating with teams to deliver data solutions.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks , PySpark , and SQL
- Build and optimize ETL/ELT workflows for large-scale data processing
- Implement data ingestion solutions from various sources into Azure Data Lake Storage (ADLS Gen2)
- Develop and manage Delta Lake architecture and Lakehouse solutions
- Collaborate with Data Architects, Analysts, and Business teams to deliver data solutions
- Ensure data quality, governance, security, and performance optimization
- Work with CI/CD pipelines and automation for data platform deployments
Skills
- 8+ years of Data Engineering experience
- Strong hands-on experience with Azure Databricks, PySpark, Spark SQL, and Delta Lake
- Experience with Azure Data Factory (ADF), ADLS Gen2, and Azure ecosystem
- Strong SQL and data modeling skills
- Experience with data warehousing and ETL/ELT frameworks
- Knowledge of Unity Catalog, data governance, and security best practices
- Experience with Git, CI/CD, and Agile methodologies
- Azure Certifications
- Experience with DBT, Power BI, or Microsoft Fabric
- Knowledge of streaming technologies such as Kafka or Event Hubs
- Experience with Lakehouse Architecture
- Exposure to AWS or GCP data platforms
- Healthcare, Finance, or Retail domain experience
Company Overview