Principal Data Engineer

We are seeking a highly experienced, hands-on Principal Data Engineer to define and evolve the architecture, engineering standards, and operating model for our enterprise data platform.

This role is for a technical leader who excels at complex system design: translating business and analytical needs into scalable, reliable, secure, and maintainable data products. The Principal Data Engineer will lead critical architectural decisions across data ingestion, storage, transformation, orchestration, governance, observability, and consumption, while raising the engineering bar through mentorship, design reviews, and pragmatic best practices.

You will work across engineering, analytics, platform, security, and business stakeholders to create systems that make trusted data accessible, timely, and actionable at scale.

What You’ll Do

Lead Data Architecture and System Design

• Own the technical vision and architecture for scalable, resilient, and cost-effective data platforms and pipelines.
• Design end-to-end data systems spanning batch and streaming ingestion, transformation, data modeling, storage, orchestration, serving layers, and consumption patterns.
• Make and document high-quality architecture decisions, including tradeoffs involving performance, reliability, scalability, latency, security, maintainability, and cost.
• Define reusable architectural patterns for common data engineering challenges, including CDC, event-driven data flows, schema evolution, slowly changing dimensions, data quality, and multi-tenant or domain-oriented data models.
• Establish clear boundaries and interfaces between operational systems, data platforms, analytics layers, and downstream applications.
• Lead technical design reviews for high-impact initiatives and provide clear, constructive direction on implementation approaches.
• Identify architectural risks early and drive plans to mitigate technical debt, single points of failure, data inconsistency, and operational fragility.

Set Engineering Standards and Best Practices

• Establish and continuously improve standards for data engineering development, testing, deployment, documentation, monitoring, incident response, and lifecycle management.
• Define best practices for version-controlled data transformations, infrastructure as code, CI/CD, code review, automated testing, and environment promotion.
• Build a culture of data reliability by implementing standards for data contracts, schema management, lineage, observability, freshness, completeness, accuracy, and reconciliation.
• Champion security, privacy, access control, governance, retention, and compliance requirements throughout the data lifecycle.
• Develop practical engineering guardrails that enable teams to move quickly without sacrificing quality, trust, or operational excellence.
• Drive adoption of well-defined SLIs, SLOs, alerting, runbooks, and post-incident learning practices for critical data products.

Drive Technical Decision-Making

• Serve as a trusted technical authority for complex, cross-functional decisions involving data architecture, tooling, platform capabilities, and engineering investments.
• Facilitate technical discussions that turn ambiguous requirements into durable, well-reasoned solutions.
• Evaluate build-versus-buy decisions, platform technologies, and vendor capabilities using clear criteria and documented recommendations.
• Align architecture decisions with organizational priorities, business outcomes, data strategy, and long-term platform sustainability.
• Create and maintain architecture decision records, reference architectures, roadmaps, and technical standards that provide clarity across teams.
• Influence leaders and stakeholders through concise communication of technical options, risks, tradeoffs, and recommended paths forward.

Build and Improve Data Products

• Lead the design and development of high-value, production-grade data pipelines, datasets, and platform capabilities.
• Partner with data analysts, data scientists, software engineers, and product teams to deliver well-modeled, discoverable, and trustworthy data products.
• Optimize data systems for reliability, usability, performance, scalability, and cost efficiency.
• Drive modernization initiatives, including migration from legacy platforms, consolidation of fragmented pipelines, and reduction of manual operational processes.
• Ensure data products are designed with clear ownership, documentation, lineage, quality expectations, and support models.

Mentor and Multiply Engineering Excellence

• Mentor data engineers at all levels, with a particular focus on growing senior engineers into strong technical leaders.
• Provide practical coaching on system design, data modeling, debugging, performance tuning, code quality, and production operations.
• Lead architecture reviews, technical workshops, design sessions, and knowledge-sharing forums.
• Establish reusable templates, reference implementations, and playbooks that improve consistency and accelerate delivery.
• Foster an inclusive, high-accountability engineering culture centered on thoughtful collaboration, continuous learning, and ownership.
• Act as a technical multiplier: helping teams make better decisions independently while maintaining alignment with shared architectural principles.

What you bring

Required

• 10+ years of experience in software engineering, data engineering, platform engineering, or a related technical discipline.
• 5+ years designing and operating large-scale data platforms, distributed data systems, or enterprise-grade data products.
• Demonstrated expertise in end-to-end system design, including defining requirements, evaluating tradeoffs, designing resilient architectures, and delivering production systems.
• Deep experience with modern data architecture concepts, such as lakehouse or warehouse platforms, batch and streaming processing, data modeling, orchestration, metadata management, and data governance.
• Strong proficiency in SQL and at least one general-purpose programming language, such as Python, Java, Scala, or similar.
• Experience with distributed data processing frameworks and technologies such as Spark, Kafka, Flink, Snowflake, Databricks, BigQuery, Redshift, or equivalent platforms.
• Strong knowledge of data modeling practices across analytical, operational, and domain-oriented use cases.
• Experience establishing engineering standards for testing, CI/CD, observability, infrastructure as code, reliability, and security.
• Proven ability to lead technical decisions across multiple teams without relying on direct authority.
• Exceptional written and verbal communication skills, including the ability to explain complex technical concepts and tradeoffs to technical and non-technical stakeholders.
• Demonstrated experience mentoring engineers and raising technical standards across an organization.

Preferred

• Experience with cloud-native data platforms on AWS, Azure, or GCP.
• Experience implementing data mesh, domain-oriented data product, or self-service platform patterns.
• Familiarity with data governance, cataloging, lineage, privacy, and access-control tooling.
• Experience supporting financial, revenue, operational, customer, or enterprise analytics domains.
• Experience with ML data pipelines, feature stores, or AI/LLM data foundations.
• Experience managing major data-platform migrations, modernization programs, or large-scale architectural transformations.

Leadership Competencies

• Systems thinker: Connects individual technical decisions to broader platform, organizational, and business outcomes.
• Pragmatic architect: Balances ideal-state architecture with incremental delivery and operational realities.
• Technical judgment: Makes clear recommendations amid ambiguity and communicates tradeoffs transparently.
• Builder and operator: Designs for production from the beginning, including reliability, security, observability, and supportability.
• Mentor and multiplier: Develops others and creates scalable mechanisms for technical excellence.
• Collaborative influencer: Builds alignment across engineering, analytics, security, product, and business teams.

Our Technology Stack

• Languages & Databases: Python, SQL, MySQL, NoSQL databases
• Orchestration & Transformation: Airflow, dbt
• Data Platforms: Snowflake, AWS, GCP, BigQuery, Redshift
• Data Integration: Fivetran, custom ETL/ELT frameworks
• Observability & Data Reliability: Datadog, monitoring, alerting, data-quality frameworks

Originally posted on Himalayas