Back to Jobs
Big Data Engineer-Hive and SparkSQL
San Francisco CA
Long term Contract
As a member of the team, you will build systems to manage hundreds of petabytes of data and process tens of millions of events per second in real time. The services you build will integrate directly with Clients products, opening the door to new and cutting-edge features. You will work with open-source technologies such as Hadoop, Scalding, Heron, and Presto and be an active member of the open-source community. You will empower dozens of engineering teams, hundreds of co-workers, and millions of users to dream of new insights and new possibilities.
• Real-Time Compute Infrastructure - cutting-edge streaming and interactive compute technologies, including Presto is a plus
• Data Pipeline - tools and services that simplify data discovery, data management, and job scheduling for engineers and data scientists
Help us solve some of our biggest challenges!
• Integrate Scalding with next-generation frameworks such as Spark and Tez
• Optimize our batch and real-time compute stacks to improve efficiency and reliability
Who You Are:
You want to learn, work with, and contribute to cutting-edge open-source technologies. The ideal candidate has experience with and/or a history of contributions to Hadoop, Spark, Hive, Scalding, Parquet, or similar technologies. You are a strong Java, Scala, or C++ developer.
You have experience in distributed systems, database internals, Linux and networking fundamentals, or performance analysis
Thanks & Best Regards,