Senior Big Data Engineer (R-19335)

Key Responsibilities

• Design, build, deploy, and document scalable data pipelines within big data ecosystems
• Develop ETL/ELT pipelines using modern data processing frameworks and tools
• Work with data lake architectures and modern data warehousing concepts such as virtual data warehouses and push-down analytics
• Build and optimize data solutions using file formats such as Parquet, Iceberg, and Delta
• Develop high-performance solutions using Python, Databricks, and Apache Spark
• Work with cloud infrastructure, primarily on GCP Design and manage relational databases including DB2, Oracle, and SQL Server with strong SQL expertise
• Implement workflow scheduling and orchestration using Apache AirflowManage big data clusters including installation, configuration, monitoring, security, resource allocation, and performance tuning
• Create detailed design documents and proof of concepts for new capabilities
• Collaborate with platform and infrastructure teams to deploy solutions into production
• Optimize workloads, manage multi-tenant environments, and ensure SLA adherence
• Participate in planning, data science initiatives, and continuous platform improvement

Key Requirements

• Minimum 8 years of experience in ETL/ELT technologies
• 4+ years of hands-on experience with big data technologies such as Databricks and Apache Spark
• Minimum 6 years of experience working with cloud platforms, preferably GCP
• Strong programming expertise in Python
• Experience with relational databases and complex SQL development
• Exposure to NoSQL and graph databases is a plus
• Experience with access control mechanisms such as role-based and attribute-based access
• Familiarity with BI tools such as Tableau, Power BI, or Looker…