Databricks Lead

Job highlights 4-10 years experience in Data Engineering with expertise in Databricks, PySpark, Python, PL/SQL, and Azure Data Factory Design, develop, and maintain scalable data pipelines and ETL/ELT solutions; optimize data workflows; integrate data into warehouses and lakes; monitor pipeline performance; collaborate with stakeholders; support cloud data migration Job match score Early Applicant Keyskills Location Work Experience Job description We are looking for a skilled Data Engineer with expertise in Databricks, PySpark, Python, PL/SQL, and Azure Data Factory (ADF) to design, develop, and optimize scalable data pipelines and analytics solutions. The ideal candidate will have experience working with large-scale data processing, ETL/ELT frameworks, cloud-based data platforms, and data warehousing solutions. Key Responsibilities Design, develop, and maintain data pipelines using Databricks, PySpark, and Azure Data Factory (ADF). Build scalable ETL/ELT solutions for processing structured and unstructured data. Develop and optimize data transformation workflows using Python and PySpark. Write, maintain, and tune complex PL/SQL queries, stored procedures, and database objects. Integrate data from multiple sources into enterprise data warehouses and data lakes. Monitor and troubleshoot data pipeline performance, ensuring reliability and data quality. Collaborate with business stakeholders, analysts, and data scientists to understand data requirements. Implement data governance, security, and best practices across data platforms. Support cloud-based data migration and modernization initiatives. Participate in code reviews, testing, and production deployments. Required Skills 410 years of experience in Data Engineering or related roles. Strong hands-on experience with Databricks and PySpark. Proficiency in Python programming for data processing and automation. Strong experience in PL/SQL development and performance tuning. Experience with Azure Data Factory (ADF) for data integration and orchestration. Knowledge of data warehousing concepts and ETL/ELT processes. Experience working with SQL databases and large datasets. Understanding of Azure cloud services and data platform architecture. Strong analytical and problem-solving skills. Preferred Skills Experience with Azure Data Lake Storage (ADLS). Exposure to Delta Lake, Azure Synapse Analytics, or Snowflake. Experience with CI/CD and DevOps practices. Knowledge of Data Governance and Data Quality frameworks. Azure certifications are a plus. Role: Software Development - Other Industry Type: IT Services & Consulting Department: Engineering - Software & QA Employment Type: Full Time, Permanent Role Category: Software Development Education UG: Any Graduate read more Key Skills Skills highlighted with ‘‘ are preferred keyskills PL/SQLADFPySparkDatabricksPython Report this job