- Data Engineer
- SysMind Tech in New York, New York, United States
- jobs
- 21 hours ago
jobs description
Location: New York
Role:
Data Engineer with hands on Java experience a must. Customer
Location:
Remote is ok Duration: 12+ Months Contract
JOB DESCRIPTION:
Java, Python, Oozie, Pig, Scala, Sage Maker, Hive, Airflow, AWS
Nice to have:
Map Reduce, Unix shell scripting, SQL/No
SQL, Git or Maven, Spark, Kafka Big Data developers with hands-on experience in building data pipeline applications With Hands on Java Experience
-2+ yrs of hands-on experience in building Big Data applications using SQL/No
SQL technologies in the Hadoop ecosystem
-Strong Map Reduce/Java app development experience & UNIX shell scripting skill
-Prepare Env for science model training
-Investigation and initial work to connect with new system (Ex. key-value store, remote service)
-SHP, eSHP topology
-TnV web crawling pipeline
-SRT batch scoring pipeline
-SRT model updating pipeline
-Hive, Pig with custom UDF implementation in Java
-MySQL
-Oozie
-Git, Maven
-Spark
-Plus:
Kafka, Dev Ops, CI/CD