Associate Data Engineer Jobs in Hyderabad - Aodh Lifesciences Private Limited

We’re looking for Data Engineers to help us grow our Data Lake, which is built using a serverless architecture, with 100% native AWS components including Redshift Spectrum, Athena, S3, Lambda, Glue, EMR, Kinesis, SNS, CloudWatch, and more! Who has the best practices in software engineering, data management, data storage, data compute, and distributed systems.

Responsibilities and Duties

• Develop and maintain automated ETL pipelines for big data using scripting languages such as Python, Spark, SQL and AWS services such as S3, Glue, Lambda, SNS, SQS, KMS.
• Develop and optimize data warehouse and data lake tables using best practice for DDL, physical and logical tables, data partitioning, compression, and parallelization.
•  Develop and maintain data warehouse and data lake metadata, data catalog, and user documentation for internal business customers.
•  Develop data objects for business analytics using data modeling techniques.
•  Develop, test, and deploy code using internal software development toolsets. This includes the code for deploying infrastructure and solutions for secure data storage, ETL pipelines, data catalog, and data query.
• Work with internal business customers and software development teams to gather and document requirements for data publishing and data consumption via data warehouse, data lake, and analytics solutions.
• Implement and support reporting and analytics infrastructure for internal business customers using AWS, services such as Athena, Redshift, Spectrum, EMR, and QuickSight.
• Develop and maintain automated ETL monitoring and alarming solutions using Python, Spark, SQL, and AWS services such as CloudWatch and Lambda.
• Help internal business customers develop, troubleshoot, and optimize complex SQL and ETL solutions to solve reporting, metrics, and analytics problems.
• Develop and maintain data security and permissions solutions for enterprise-scale data warehouse and data lake implementations including data encryption and database user access controls and logging.

Required Experience, Skills and Qualifications

Education: Any Degree

Experience: 0-2 years of experience in distributed system concepts from a data storage and compute perspective (e.g. data lake architectures).
0-2 years of experience with one or more query languages (e.g. SQL), schema definition languages (e.g. DDL), and scripting languages (e.g. Python) to build data solutions.