Big Data Engineer

Apple

Bangalore,India

Job Description

About the job

Job Summary Imagine what you could do here. At Apple, great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job and theres no telling what you could accomplish! This position is in the Big Data Engineering team that manages various state of the art open source technologies in Streaming, Data Science and Big Data Analytics areas, including Kafka, Hadoop, Spark and AI/ML. You should have experience developing/deploying large-scale big data analytics/streaming applications and services. The ideal candidate will bring a lot of energy, initiative and excitement! Will be ready to learn and explore new leading edge technologies.

Key Qualifications

• -What are we expecting? This position demands the best and brightest software engineers! You should have BS or MS in CS with strong Computer Science fundamentals and good troubleshooting and problem solving skills. The ideal candidate will bring a lot of energy, initiative and excitement! Will be ready to learn and explore new leading edge technologies.
• 4 - 12 years of professional experience in deploying and handling complex, distributed and multi-tier Big Data environments.
• Solid experience and knowledge in Big Data Technologies such as Hadoop, Spark, Kafka, Hbase, Presto etc.
• Core competency in Hadoop ecosystem with experience in Hadoop admin activities like deploying clusters, add/delete nodes, name node high availability, etc.
• Good proficiency in System platforms - Operating Systems, Networking, CPU, Memory, etc.
• Good exposure and expertise in Public Cloud offerings (AWS & GCP). Preferably experience in handling BigData platforms on Cloud (EMR, EKS, etc).
• Passionate about Problem solving and troubleshooting challenging engineering problems. Triage complex operational issues with various engineering and customer teams.
• Strong flair for automation. Develop tools/scripts to monitor server-side components and to streamline operational efficiency and procedures.
• Reasonable knowledge and experience in programming using Java/Python. Design and build tools and framework for managing and scaling big data technologies.
• Knowledge of DevOps methodology and tools and experience in handling CI/CD pipelines.
• Knowledge of deployments in Kubernetes containers highly desirable.
• Experience in handling architectural and design considerations such as performance, scalability, reusability and flexibility issues.
• Experience with processing large amounts of data.

Description

-Build a Big Data Platform. -Design, architect, and engineer Big Data technologies, platforms and applications. -Setup of Hadoop clusters with related technologies on hosts including a combination of bare metal systems, VMs, Containers and Cloud. -Setup of Kafka brokers, Kafka MirrorMakers and Kafka Zookeeper on hosts including a combination of bare metal systems, VMs, Containers and Cloud. -Develop scalable, robust systems that will be highly adaptable to changing business needs. Architect, improve, and scale applications to the next level. -Interface with application owners, developers and project managers. -Recommend and deploy tools and processes to enable rapid development, deployment and operations of big data solutions. -Be a guru for application teams faced with architectural decisions or complex technical problems, such as scaling and tuning. -Mentor other engineers in the team on the development of big data technologies and applications.

Education & Experience

BS Computer Science or equivalent (MS Preferred)