Big Data Engineer, Profit Intelligence

amazon

Chennai,tamil nadu

Job Description

About the job

DESCRIPTION

Job summary
Amazon’s Profit Intelligence is seeking a talented Big Data Engineer to join the Profit Intelligence team. We develop software solutions that are revolutionizing Amazon business intelligence through advanced algorithms running on big data technologies. The ideal candidate thrives in a fast-paced environment and relishes working with petabytes of extremely complex and dynamic data. In this role you will be part of a team of high caliber data and software engineers to build data pipelines using big data technologies such as Apache Spark, Hive/Hadoop, and distributed query engines. You should be passionate about working with big data and have the aptitude to incorporate new technologies and evaluate them critically. You must possess excellent business and communication skills and be able to work with business owners to analyze requirements and build solutions. You are a self-starter, has a proven track record of dealing with ambiguity and working in a fast-paced, highly dynamic environment. Working experience of any one of the programming languages such as Java, C#, C++, Scala, etc. is a big plus.

Major Responsibilities:

• Interface with PMs, business customers, and software developers to understand requirements and implement solutions
• Collaborate with both Retail Finance and IN business teams to understand the inter-dependencies and deliverables
• Design, develop, and operate highly-scalable, high-performance, low-cost, and accurate data pipelines in distributed data processing platforms with AWS technologies
• Recognize and adopt best practices in data processing, reporting, and analysis: data integrity, test design, analysis, validation, and documentation
• Keep up to date with big data technologies, evaluate and make decisions around the use of new or existing software products to design the data architecture

Key job responsibilities
You will be part of a team of creative, top-notch software developers to work hard, have fun, and make history. Software engineers at Amazon are more than just order takers; they see a problem and leverage innovative technology to address it. You will be working with very large data sets, well beyond the scalability limits of conventional relational databases. Were looking for people who innovate, love solving hard problems, and never take "no" for an answer.

Our team is within the Selling Partner Services organization and develops sophisticated tools for the Amazon Consumer businesses, supporting deep dive analysis, vendor negotiations and business planning towards enhancement at the bottom line. We also provide financial and operational reports to Amazon retail vendors worldwide.

BASIC QUALIFICATIONS

Bachelor’s degree in computer science, engineering, mathematics, or a related technical discipline

•  Hands-on experience and advanced knowledge of SQL and query performance tuning skills.
• Basic scripting skills using Python
• 1+ years of experience as a Data Engineer or in a similar role
•  Experience with data modeling, data warehousing, and building ETL pipelines
•  Experience using big data technologies (Hadoop, Hive, Hbase, Spark, EMR, etc.)

•  Experience working with AWS big data technologies (EMR, Redshift, S3, AWS Glue, Kinesis and Lambda)
•  Knowledge of data management fundamentals and data storage principles
• Knowledge of distributed systems as it pertains to data storage and computing
• Effective analytical, troubleshooting and problem-solving skills

PREFERRED QUALIFICATIONS

Masters in computer science, or related technical quantitative field

• 3+ years of experience as a Data Engineer in a company with large, complex data sources.
•  Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
•  Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations
•  ETL development experience on large datasets (terabytes in size)
• Creating queries in hive, scala, presto or other big data platforms
•  Development experience in one of the programming languages (Java) is a plus
• Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and data engineering strategy