ADF (Azure Data Factory) Developer

We are looking for a technically sound, hands-on Azure Data Factory (ADF) Developer with experience designing, implementing, and optimizing data integration and ETL processes.

This role requires a strong understanding of data flow, transformation, and orchestration in Azure Data Factory, with an emphasis on high-performance data pipelines and complex workflows.

The ideal candidate will have a keen eye for detail, a problem-solving mindset, and a proactive approach to optimizing ADF solutions for scale and efficiency.

Responsibilities:

  • Design, develop, and deploy high-performance ETL pipelines using Azure Data Factory to support data integration needs across various business units.
  • Create, monitor, and optimize complex data workflows, including data ingestion, transformation, and loading to and from various Azure data services (Azure SQL Database, Azure Data Lake Storage, etc.).
  • Collaborate with cross-functional teams to gather data integration requirements and translate them into technical solutions using ADF.
  • Develop and manage parameterized data flows, mapping data flows, and pipelines to enable reusable and flexible data processes.
  • Implement robust error handling, logging, and monitoring mechanisms to ensure data quality, reliability, and traceability.
  • Optimize existing ADF pipelines for performance, cost-effectiveness, and scalability, identifying areas for improvement and implementing best practices.
  • Leverage other Azure services such as Azure Data Lake, and Azure SQL Database in combination with ADF for complete end-to-end data solutions.
  • Implement data processing and ETL jobs with large datasets, ensuring that jobs run efficiently in high-volume data environments.
  • Document workflows, processes, and procedures for knowledge sharing and operational support.
  • Participate in code reviews, ensuring adherence to best practices and coding standards.
  • Troubleshoot and resolve technical issues within data workflows and ETL jobs to minimize downtime and maximize performance.

Required Skills and Experience:

  • 5-6 years of hands-on experience with Azure Data Factory in a production environment.
  • Strong proficiency in designing and building complex ADF pipelines, including parameterized pipelines, mapping data flows, and orchestrating data workflows.
  • Experience with various ADF integration components, including Linked Services, Datasets, Pipelines, Triggers, and Data Flows.
  • Expertise in SQL, with experience in querying, transforming, and analyzing large datasets.
  • Familiarity with other Azure services such as Azure Data Lake, and Azure SQL Database, with the ability to integrate them within ADF workflows.
  • Experience in performance tuning of ADF pipelines and understanding of data partitioning and distribution for optimized data flows.
  • Proficient in data modeling, data warehousing principles, and ETL best practices.
  • Knowledge of Azure DevOps for CI/CD in ADF, including version control and pipeline deployments.
  • Strong analytical and troubleshooting skills to resolve complex data integration and pipeline issues.
  • Effective communication skills to collaborate with stakeholders, technical and non-technical alike.

Additional Information:

  • Title: ADF (Azure Data Factory) Developer
  • Location:  Remote
  • Experience:  5 – 6 years
  • Shift Time:  UK shift

Send Resumes To: raj@email.xtglobal.com