Data Engineering Team Lead

new york, New York

A global investment firm is currently seeking a Data Engineering Team Lead to join their team in New York, NY.

Within this role, you will be responsible for leading your team in designing a custom data pipeline solution for a multi-terabyte big data platform that will facilitate analysis of structured and unstructured data.



  • Must have at least a Bachelor's degree in Computer Science, or related field 
  • 3+ years of experience leading a team
  • 5+ years of experience building scalable and reliable data pipelines based on Big Data processing technologies like Hadoop, MapReduce, Spark, etc.
  • 5+ years of industry experience in Python, Scala or other functional programming language
  • 3+ working with cloud computing platforms, such as AWS
  • Experience processing large amounts of structured and unstructured data
  • Deep experience with ElasticSearch or similar big data platform
  • Highly analytical with good problem-solving skills and creativity; able to work independently in a fast-paced environment
  • Positive individual with strong communication skills; able to interface well with users and other development teams
  • Self-driven individual with an ability to work under pressure


  • Strong knowledge of and experience with statistics; potentially other advanced math as well
  • Experience with writing Kafka producers and consumers
  • Experience with Spark Streaming and Spark SQL in a production setting

Job keywords:

Files must be less than 2 MB.
Allowed file types: doc docx pdf.

Vertical Tabs