Assistant Manager

Here's an overview of what it typically involves, along with the qualifications and skills needed

• Data Pipeline Development:Design, develop, and maintain scalable batch and streaming ETL pipelines using Databricks, PySpark, Python, and SQL.
• Lakehouse Implementation:Build and manage data solutions leveraging Delta Lake and medallion architecture principles to support trusted and reusable data products.
• Data Integration:Integrate data from enterprise applications, databases, APIs, files, and cloud platforms to enable unified analytics and reporting.
• Data Modeling & Transformation:Develop efficient data models and transformation frameworks that support business intelligence, advanced analytics, and AI/ML use cases.
• Platform Optimization:Monitor and optimize Databricks workloads, cluster configurations, and data processing performance to ensure scalability and cost efficiency.
• Data Quality & Governance:Implement data quality checks, monitoring, lineage, security controls, and governance practices in alignment with enterprise standards.
• DevOps & Automation:Contribute to CI/CD implementation, automated deployments, testing frameworks, and release management for data engineering solutions.
• Cross-functional Collaboration:Partner with business stakeholders, data architects, analytics teams, and AI engineers to deliver data products that support business outcomes.
• AI & Analytics Enablement:Enable downstream AI, machine learning, and Generative AI initiatives by building reliable, high-quality, and governed data foundations.
• Continuous Improvement & Innovation:Stay current with advancements in Databricks, cloud technologies, and modern data engineering practices to continuously enhance platform capabilities.

• Experience:4 - 6 years of experience in data engineering, ETL development, and cloud-based data platforms.
• Technical Skills:
• Proficiency in Python, PySpark and SQL
• Strong expertise in Databricks Data Engineering capabilities, including: Databricks Lakehouse Architecture, ETL/ELT Design and Development, Delta Lake, Data Modeling, Batch and Streaming Data Pipelines, Spark Optimization and Performance Tuning
• Experience in designing and implementing: Medallion Architecture, Data Quality Frameworks, Data Governance Controls, Metadata and Lineage Solutions
• Experience with orchestration and workflow management tools such as Databricks Workflows, Azure Data Factory, or equivalent.
• Experience implementing CI/CD pipelines, Git-based source control, automated testing, and deployment automation.
• Strong understanding of data warehousing, dimensional modeling, and modern Lakehouse architectures.
• Exposure to Databricks Mosaic AI, Vector Search, MLflow, feature engineering, or GenAI data preparation workflows (Preferred).
• Familiarity with data preparation for RAG, LLM, and AI/ML use cases (Preferred).
• Experience with infrastructure-as-code and containerization technologies (Preferred).
• Certifications:Relevant certifications in data engineering, such as Databricks Certified Data Engineer Associate/Professional, or similar. (Preferred not mandatory)
• Problem-Solving:Strong analytical and problem-solving skills, with a proven ability to design scalable data solutions and resolve complex data engineering challenges.
• Communication:Excellent communication skills with experience working directly with clients and business stakeholders to deliver data and analytics solutions.
• Stakeholder Management: Experience engaging with business stakeholders to understand data and analytics requirements and translate them into scalable data engineering solutions.