Data Engineer Jobs in Chennai - Merit Software Services Pvt Ltd

Data Engineer

Merit Software Services Pvt Ltd

Not disclosed

Expired

Posted: 01 Nov 19

Job Description

Role: Data Engineer

Roles and Responsibilities :

To develop Scala or Python scripts, UDFs using both Data frames/SQL/Data sets and RDD in Spark 2.3+ for Data Aggregation, queries and writing data back into the OLTP system through Sqoop.
Should have a very good understanding of Partitions, Bucketing concepts and designed both Managed and external tables, ORC files in Hive to optimize performance.
Wrote and Implemented Spark and Scala scripts to load data from and to store data into Cassandra/Hbase/ any NoSQL
Implementing SCD Type 1 and Type 2 model using Spark
Developed Oozie workflow for scheduling and orchestrating the ETL process
Experienced in performance tuning of Spark Applications for setting right Batch Interval time, the correct level of Parallelism and memory tuning
Streaming data into Elastic search for visualization using Kibana
Should have implemented the mapping parameters/variables in the mapping and the session level to increase the reusability of the code and parameterize the hardcoded values.

Additional skills:

Knowledge in AWS stacks AWS Glue, S3, SQS
Exposure to Elastic Search, Solr is a plus
Exposure to NoSQL Databases Cassandra, MongoDB
Exposure to Serverless computing

Location : Chennai

Job Particulars

Role it software engineer

Education Any Post Graduate, Any Graduate

Who can apply Freshers

Hiring Process Face to Face Interview

Employment TypeFull Time

Job Id741435

Job Category IT/Software

Locality Address

State Tamil Nadu

Country India

About Company

Merit is a tight-knit team of tech lovers and problem solvers who thrive on getting a job done well, on time and on budget. Our combination of exceptional talent, flexible resources and cost-effective delivery sees us power some of the world’s most trusted B2B data products. We buck the trend of off-shore companies by guaranteeing clear communications channels, ex-pat management teams and dedicated account managers based in the UK.

Jobs By Location

Chennai

Noida

Gurgaon

Pune

Delhi

Hyderabad

Mumbai

Bangalore

Kolkata

Ahmedabad