Data Engineer

HARMAN International

Bengaluru,Karnataka

Job Description

About the job

About the job

• Relevant Experience - 5-8 years

Role

Design, implementation, and management of data platform, data pipelines to help fulfill data access and aggregation functionality for users in cloud environment.

Responsibilities:

• Designing, constructing, developing, installing, testing and maintaining architectures of large-scale data processing systems.
• Experiment with various data models to optimize the search queries and come up with the proposals.
• Build data pipelines that capture, process and store organized data into large databases which would be ready for consumption.
• Expertly use Query programming languages such as SQL and functional programming languages such as Python, for mining and storing data into databases.
• Perform ETL tasks on the Big Data, using highly scalable algorithms written in Scala / Java / PySpark so that in can utilize the power distributed computing.
• Have end- to- end responsibility for leading projects focused on extracting, merging, analyzing and managing large sets of data across multiple, disparate databases.
• Be able to transform unstructured raw data in to formats suitable for statistical modeling, visualization and machine learning environments.
• Establish methodologies for quickly rolling out new data analysis capabilities for standalone data- driven products and service to support our associates.
• Be able to understand varies data sources and may be able to implement automated data quality and auditing through standard tools and custom code.
• Demonstrate a deep knowledge of and ability to operationalize, leading data technologies and best practices
• Be responsible for maintaining project plans, clean code, and wellwritten documentation
• Be able to work in teams and collaborate with stakeholders to define requirements
• Make decisions independently on analytical problems & methods.
• Be able to identify and suggest novel areas of future work for themselves or the team
• Be able to work in a globally distributed team in an Agile/ Scrum approach
• Technical/ Functional:
• Hands-on data engineering or other data- intensive development experience
• Experience processing large amounts of structured and unstructured data
• Advanced knowledge of programming languages such as Python
• Experience building scalable data models and performing complex relational databases queries using SQL (Oracle, MySQL), etc.
• Experience with Big Data tools like Spark, Hadoop, Amazon EMR.
• Experience working on Cloud platforms like AWS,IBM,Azure including EMR, RDS, EC2, Vault, ECS, S3, EFS, etc.
• Behavioural:
• Effective team player.
• Study industry trends (global and domestic) and apply it in work-area.
• Good communication and presentation skill