- Data Architecture & Solution Design: Design end-to-end data, analytics, and Lakehouse solutions using Databricks. Translate business objectives and technical requirements into scalable architectures that support enterprise data processing, analytics, and reporting needs.
- Data Engineering & Pipeline Development: Develop, review, and optimize robust ETL/ELT pipelines and enterprise data platforms using Databricks, Apache Spark, PySpark, Python, and SQL. Establish best practices for data transformation, modeling, warehousing, and distributed data processing.
- Databricks Platform Engineering: Design and implement solutions using Delta Lake, Unity Catalog, Databricks SQL, and Databricks Workflows. Ensure data platforms are secure, reliable, maintainable, and aligned with enterprise architecture standards.
- Cloud Data Solutions: Architect scalable data environments across Microsoft Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP). Support cloud data platform modernization, migration, and transformation initiatives while optimizing infrastructure utilization and operational efficiency.
- Customer Engagement & Technical Consulting: Lead technical discovery sessions, architecture workshops, and solution discussions with enterprise customers. Engage with architects, CTOs, CDOs, engineering managers, and other senior stakeholders to understand requirements, present recommendations, and communicate architectural decisions, risks, and trade-offs.
- Architecture Documentation & Governance: Prepare high-level designs (HLDs), low-level designs (LLDs), architecture diagrams, technical proposals, and implementation documentation. Establish architectural guidelines and review solution designs to ensure consistency, scalability, and long-term maintainability.
- Proofs of Concept & Technical Demonstrations: Lead proofs of concept (POCs), technical demonstrations, architecture assessments, and solution validation activities. Evaluate alternative approaches, validate technical feasibility, and demonstrate how proposed solutions address customer requirements.
- Performance Optimization & Troubleshooting: Diagnose and resolve complex issues affecting Databricks and Spark workloads. Optimize data processing performance, resource utilization, reliability, scalability, and cloud costs while identifying opportunities for continuous improvement.
- Technical Leadership & Mentorship: Provide architectural direction, technical guidance, and mentorship to data engineering teams. Review code, data pipelines, and implementation approaches to promote engineering excellence and ensure alignment with architectural standards.
- Cross-Functional Collaboration: Partner with sales, pre-sales, delivery, product, and engineering teams to develop effective technical solutions. Contribute to solution positioning, technical feasibility assessments, and successful customer engagements throughout the solution lifecycle.
- Innovation & Continuous Improvement: Identify opportunities to improve data architecture, automation, development practices, and operational efficiency. Stay informed about advancements in Databricks, cloud data platforms, distributed computing, and modern data engineering technologies.
Requirements:
- Professional Experience: At least 8 years of experience in data engineering, data architecture, solution architecture, big data, or a related technical field, with demonstrated experience designing and delivering enterprise-grade data solutions.
- Databricks Expertise: Strong hands-on experience with Databricks and enterprise data platforms, including Lakehouse architecture, Delta Lake, Unity Catalog, Databricks SQL, and Databricks Workflows.
- Big Data & Distributed Computing: Deep knowledge of Apache Spark and PySpark, including distributed processing concepts, data transformation strategies, and the design of scalable data pipelines.
- Programming & Querying: Advanced proficiency in Python and SQL, with experience developing, maintaining, reviewing, and optimizing production-grade data engineering solutions.
- Data Engineering & Architecture: Proven experience designing scalable ETL/ELT pipelines, data models, data warehouses, and enterprise data platforms. Strong understanding of data architecture principles, data processing patterns, reliability, scalability, and maintainability.
- Cloud Platforms: Practical experience with at least one major cloud provider: Azure, AWS, or GCP. Ability to design and evaluate cloud-native data architectures and make informed decisions regarding performance, reliability, and cost.
- Performance Tuning & Optimization: Hands-on experience troubleshooting and optimizing Databricks and Spark workloads, improving execution efficiency, resource utilization, processing speed, and overall platform performance.
- Solution Architecture & Consulting: Demonstrated experience leading technical discovery, architecture workshops, POCs, technical demonstrations, solution validation, and architecture assessments. Ability to translate complex business requirements into practical technical designs and implementation roadmaps.
- Customer-Facing Communication: Strong consulting and stakeholder-management skills, with the confidence to engage senior technology leaders, architects, CTOs, CDOs, and engineering managers. Excellent communication, presentation, and technical storytelling abilities, including the capacity to explain complex concepts clearly to technical and non-technical audiences.
- Technical Leadership: Experience providing architectural guidance, mentoring data engineers, reviewing technical implementations, and promoting engineering best practices across teams.
- Analytical & Problem-Solving Skills: Strong analytical thinking and a structured approach to solving complex technical challenges. Ability to evaluate architectural alternatives, identify risks, and recommend solutions that balance business needs with technical constraints.
- Additional Technical Skills: Exposure to Snowflake, Kafka, Spark Streaming, dbt, Terraform, CI/CD pipelines, MLflow, MLOps, generative AI, large language models (LLMs), retrieval-augmented generation (RAG), or data migration is an advantage.
- Certifications: Databricks certifications are desirable and can strengthen your application.
- Ownership & Remote Collaboration: A proactive, independent working style with strong accountability and the ability to manage customer-facing responsibilities in a remote environment. Comfortable balancing hands-on technical work, strategic architecture decisions, and multiple stakeholder priorities.
Benefits:
- Competitive Compensation: Annual salary ranging from INR 40–70 lakhs per annum (LPA), depending on experience, expertise, and role alignment.
- Remote Flexibility: Work remotely from India, with the flexibility to contribute from your preferred location.
- Full-Time Employment: A full-time opportunity focused on enterprise data architecture and modern cloud-based solutions.
- Technical Impact: Lead the design and delivery of scalable data platforms and influence architecture decisions for enterprise customers.
- Professional Development: Expand your expertise in Databricks, Lakehouse architecture, cloud data engineering, distributed computing, and emerging AI technologies.
- Technical Leadership Opportunities: Guide engineering teams, mentor technical professionals, and establish best practices across data engineering projects.
- Customer Engagement: Work directly with senior enterprise stakeholders, helping organizations address complex data challenges and modernize their technology ecosystems.
- Exposure to Modern Technologies: Gain experience with advanced data platforms, cloud technologies, analytics solutions, and emerging capabilities in AI and machine learning.