Note: The job is a remote job and is open to candidates in USA. Perform is seeking a Senior Data Engineer who operates as a hybrid technical product owner, bridging data engineering and ML/AI. This role involves building and maintaining the data platform on Azure Databricks while collaborating with business stakeholders to drive impactful data solutions.
Responsibilities
- Design, build, and operate the data platform on Azure Databricks: ingestion, transformation, storage, and serving layers that power analytics, AI models, and operational reporting
- Build and maintain data pipelines across the ecosystem: Salesforce, SQL Server, Snowflake, third-party sources, and the new cloud-native payments platform
- Engineer for quality and trust with validation checks, anomaly detection, lineage tracking, and documentation that ensure every downstream consumer can rely on the data
- Write clean, version-controlled, production-grade code. Think like a software engineer building a product, not a script runner maintaining jobs
- Partner directly with business stakeholders across physician growth, member services, finance, and operations to understand how data drives decisions and then build for those decisions, not for abstract requirements
- Act as a technical product owner for your domain areas: own the backlog, prioritize based on business impact, and ship iteratively without waiting for a PM to sequence your work
- Translate ambiguous business questions into data models, feature tables, and curated datasets that analysts and data scientists can build on immediately
- Close the loop: follow your data through to the dashboard, the model, or the operational workflow and validate that it's actually driving the outcome
- Use Claude Code and agentic development as your primary workflow: AI-driven pipeline generation, automated testing, rapid prototyping to ship at a pace that would be impossible with traditional approaches
- Build data infrastructure that is AI-ready: well-documented, semantically clear, and structured so that AI tools and agents can reason over it effectively
- Scout, evaluate, and adopt emerging AI tools and platforms that make the data team faster by separating real value from hype with hands-on testing
- Share what you learn. Document patterns, run demos, and help the broader team adopt AI-first workflows with confidence
Skills
- BS in Computer Science, Data Science, or related field; 6+ years in data engineering or a hybrid data engineering/analytics role
- Deep hands-on experience with Azure Databricks: notebooks, Delta Lake, Unity Catalog, and production-scale pipelines. Databricks 3+ years is non-negotiable
- Strong Python and SQL; experience with PySpark and distributed data processing. Python is essential for validating AI-generated logic
- Built and operated data pipelines that serve analytics, ML models, and operational systems, not just batch ETL jobs
- Worked directly with business stakeholders to define requirements, shape data products, and deliver measurable outcomes
- Active, daily use of AI coding tools (Claude Code, Copilot, or similar) as a force multiplier. Claude and AI tooling in active daily use (team already works this way)
- Strong communication skills with a track record of presenting technical work to non-technical audiences
- Power BI familiarity for serving downstream BI team
- Salesforce/CRM data integration experience, directly relevant to use case
- Experience with engagement scoring or machine learning model pipelines
- Familiarity with agentic development patterns and AI-assisted coding
- Background in healthcare or mission-driven industries
- US-based preferred, not a hard requirement
Company Overview
Company H1B Sponsorship