Big Data Solutions Engineer Leader
Description:
A sophisticated Data Services team are looking for a leader for their Information Management solutions.
The ideal person will come with a strong technical hand on background but has the necessary leadership kills to bring team on .
The technical background required :
Highly experienced Java Hadoop Big Data Engineer with experience of working with large-scale, distributed data pipelines.
Responsibilities:
span the full data engineering lifecycle from architecture and design, data analysis, software development,
QA, capacity planning and managing the analytics environment as a whole.
Job Responsibilities:
* Build distributed, scalable, and reliable data pipelines that ingest and process data at scale and in real-time.
* Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases.
* Perform offline analysis of large data sets using components from the Hadoop ecosystem.
* Evaluate and advise on technical aspects of open work requests in the product backlog with the project lead.
* Own product features from the development phase through to production deployment.
* Evaluate big data technologies and prototype solutions to improve our data processing architecture.
Desired Skills and Experience:
* BS in Computer Science or related area
* 10-12 years software development experience
* Minimum 2 Year Experience on Big Data Platform
* Proficiency with Java, Python, Scala, HBase, Hive, MapReduce, ETL, Kafka, Mongo, Postgres, Visualization technologies etc.
* Flair for data, schema, data model, how to bring efficiency in big data related life cycle
* Understanding of automated QA needs related to Big data
* Understanding of various Visualization platform (Tableau, D3JS, others)
* Proficiency with agile or lean development practices
* Strong object-oriented design and analysis skills
* Excellent technical and organizational skills
* Excellent written and verbal communication skills
Top skill sets / technologies in the ideal candidate:
* Programming language -- Java (must), Python, Scala, Ruby
* Batch processing -- Hadoop MapReduce, Cascading/Scalding, Apache Spark
* Stream processing -- Apache Storm, AKKA, Samza, Spark streaming
* NoSQL -- HBase, MongoDB, Cassandra, Riak,
* ETL Tools – Data Stage, Informatica,
* Code/Build/Deployment -- git, hg, svn, maven, sbt, jenkins, bamboo
Technologies that we use include:
* R
* Java
* Hadoop/MapReduce
* Flume
* Storm
* Kafka
* MemSQL
* Pig
* Hive
* Tableau Integration
* ETL
Excellent communication and decision making skills are essential.
Strong analytical, problem solving and decision-making skills.
Identify project risks and recommend mitigation efforts.
Identify project issues, communicate them and assist in their resolution.
Assist in continuous improvement efforts in enhancing project team methodology and performance.
Cooperative team focused attitude
If you are happy to be represented and be put forward for this role (or would like more information) please send me an email back to cv@solasconsulting.ie or call me (T: 01 2449520) indicating if you wish to be represented by Solas Consulting Group.
Email me jobs like this
