Senior Analyst

We are recruiting for a Databricks Application Developer in the Legal, Risk Management and Compliance (LRMC) team.

Roles & responsibilities

• Develop business applications and AI-enabled solutions using Databricks, translating solution requirements into working applications and integrations.

• Build Databricks applications and develop application logic, APIs, services, and integrations with enterprise systems.

• Design and implement scalable batch and streaming data pipelines using Python, SQL, PySpark, and Delta Lake.

• Apply Medallion Architecture principles across Bronze, Silver, and Gold layers to improve data quality, reusability, and consumption.

• Create and manage Databricks Workflows, jobs, task dependencies, parameters, scheduling, monitoring, and failure handling.

• Integrate governed data assets from Unity Catalog and apply appropriate access controls, lineage, and data governance practices.

• Design database objects and data models, optimize SQL queries, and improve performance of data processing workloads.

• Integrate Databricks applications with governed data, APIs, analytics services, and AI or machine learning capabilities.

• Use GitHub for source control and implement CI/CD pipelines with GitHub Actions for automated testing and deployment.

• Support unit testing, system integration testing, deployment, production readiness, troubleshooting, and technical documentation.

Mandatory technical & functional skills

• 3-5 years of experience in data engineering, data platform development, or Databricks application development.

• Hands-on expertise with Azure Databricks, Python, PySpark, Spark SQL, SQL, Delta Lake, and notebook-based development.

• Experience creating reliable ETL/ELT pipelines for batch and streaming data processing, including validation and error handling.

• Strong understanding of Medallion Architecture, dimensional data modelling, database design, indexing, query optimization, and performance tuning.

• Experience building Databricks applications, backend logic, REST APIs, and integrations with enterprise systems and data services.

• Hands-on experience with Databricks Workflows, jobs, scheduling, task orchestration, monitoring, logging, and production support.

• Experience with Unity Catalog, access controls, data lineage, governance, secrets, and secure data handling.

• Proficiency with GitHub source control, branching, pull requests, code reviews, GitHub Actions, and CI/CD deployment pipelines.

• Understanding of automated testing, Agile delivery, technical documentation, and operational readiness.

Preferred technical & functional skills

• Experience with Databricks Asset Bundles, Databricks SQL, Lakehouse architecture, and Infrastructure as Code.

• Exposure to Azure services such as Azure Data Lake Storage, Key Vault, Data Factory, Entra ID, and Azure DevOps.

• Experience integrating machine learning, generative AI, or governed AI services into Databricks applications.

• Knowledge of Power BI or other analytics and data visualization platforms.

• Understanding of risk management, compliance, governance, or other regulated business domains.