Descriere
Key Responsibilities: Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as Azure DevOps, GitHub Actions or Databricks Repos. Monitor pipeline performance and troubleshoot data issues in real-time and batch environments. Document solutions, workflows, and technical standards. Required Skills: Experience in data engineering with a strong focus on Databricks and Apache Spark. Proficiency in PySpark, SQL, and Python. Experience with Delta Lake, Databricks SQL, and Unity Catalog. Hands-on experience with cloud platforms Familiarity with data lakehouse architecture, data warehousing and streaming data Strong understanding of ETL best practices, data partitioning, and performance tuning. Experience with CI/CD for data pipelines. Excellent problem-solving and communication skills. ##LI-DNIKey Responsibilities: Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as Azure DevOps, GitHub Actions or Databricks Repos. Monitor pipeline performance and troubleshoot data issues in real-time and batch environments. Document solutions, workflows, and technical standards. Required Skills: Experience in data engineering with a strong focus on Databricks and Apache Spark. Proficiency in PySpark, SQL, and Python. Experience with Delta Lake, Databricks SQL, and Unity Catalog. Hands-on experience with cloud platforms Familiarity with data lakehouse architecture, data warehousing and streaming data Strong understanding of ETL best practices, data partitioning, and performance tuning. Experience with CI/CD for data pipelines. Excellent problem-solving and communication skills.
A little about ADP: We are a comprehensive global provider of cloud-based human capital management (HCM) solutions that unite HR, payroll, talent, time, tax and benefits administration and a leader in business outsourcing services, analytics, and compliance expertise. We believe our people make all the difference in cultivating a down-to-earth culture that embraces our core values, welcomes ideas, encourages innovation, and values belonging. We've received recognition for our work by many esteemed organizations, learn more at ADP Awards and Recognition.
Diversity, Equity, Inclusion & Equal Employment Opportunity at ADP: ADP is committed to an inclusive, diverse and equitable workplace, and is further committed to providing equal employment opportunities regardless of any protected characteristic including: race, color, genetic information, creed, national origin, religion, sex, affectional or sexual orientation, gender identity or expression, lawful alien status, ancestry, age, marital status, protected veteran status or disability. Hiring decisions are based upon ADP’s operating needs, and applicant merit including, but not limited to, qualifications, experience, ability, availability, cooperation, and job performance.
Ethics at ADP: ADP has a long, proud history of conducting business with the highest ethical standards and full compliance with all applicable laws. We also expect our people to uphold our values with the highest level of integrity and behave in a manner that fosters an honest and respectful workplace. Click https://jobs.adp.com/life-at-adp/ to learn more about ADP’s culture and our full set of values.
