Databricks Developer
Position: Databricks Developer
Experience: 5–8 Years
Engagement: Contractual
Contract Duration: 6 Months
Joining: Immediate / Short Notice Preferred
About the Role
Appbay Technologies is looking for an experienced Databricks Developer to join our team on a 6-month contractual engagement. The ideal candidate will have strong hands-on experience in Databricks, data engineering, Spark, and cloud-based data platforms and will be responsible for designing, developing, and optimizing scalable data solutions.
Key Responsibilities
- Develop and maintain scalable data pipelines using Databricks and Apache Spark.
- Design and implement ETL/ELT workflows for processing large datasets.
- Develop data solutions using PySpark and/or SQL.
- Work with Delta Lake for reliable and scalable data storage.
- Optimize Spark jobs, queries, and data pipelines for performance and cost efficiency.
- Build and manage workflows using Databricks Workflows.
- Integrate data from multiple sources into Databricks.
- Implement data transformation, cleansing, and validation processes.
- Collaborate with data engineers, analysts, developers, and business stakeholders.
- Troubleshoot data pipeline issues and ensure data quality and reliability.
- Follow best practices for data security, governance, and development.
- Participate in code reviews, documentation, testing, and deployment activities.
Required Skills
- 5–8 years of experience in Data Engineering / Databricks development.
- Strong hands-on experience with Databricks.
- Strong knowledge of Apache Spark and PySpark.
- Excellent proficiency in SQL.
- Experience with Delta Lake and data lake architecture.
- Strong understanding of ETL/ELT concepts and data pipelines.
- Experience working with cloud platforms such as Azure, AWS, or GCP.
- Good understanding of data warehousing and data modeling concepts.
- Experience with Git/version control and CI/CD practices.
- Strong analytical and problem-solving skills.
Good to Have
- Experience with Azure Databricks.
- Knowledge of Azure Data Factory / AWS Glue / similar data integration tools.
- Experience with Unity Catalog and Databricks governance.
- Knowledge of cloud storage such as ADLS, S3, or GCS.
- Experience with streaming technologies such as Kafka.
- Databricks certifications will be an added advantage.
