Senior Data Engineer with the responsibility to design data pipeline architecture in partnership with the Product owner to fit with product roadmap needs. Responsible for expanding and optimizing data and data pipeline architecture, as well as optimizing data flow and transformations.
Main Responsibilities
* Defining how to develop the pipeline and overall quality
* Building new and relevant data engineering features for enhancing the pipeline according to the roadmap
* Mentoring the team to ensure optimal application of best practices, coding conventions over the pipeline and assist data engineers in their duties upon request
* Providing technical guidance and direction as well as contributing hands-on to pipeline development
* Acting as an evangelist and guiding practitioner of agile and devops practices
Expertise Required:
* Bachelor's/Master's/PhD Degree in Computer Science, Applied Mathematics, Engineering, or any other technology related field
* Experience in optimizing data pipeline architecture, data flow and transformations
* Experience building big data applications and pipelines using Spark - Big Data Engineering
* Experience working with Reference Data or Master Data
* Experience building large-scale software systems and implementation of creative solutions to difficult computational problems (with emphasis on performance and near real-time data analytics)
* Senior level data engineering (e.g. ETL, Databricks or Spark experience)
* Python, R or Scala proficiency
* AWS / Azure cloud technologies and stack
* Strong knowledge in DevOps and coding conventions
We are also ideally seeking a Specialist Data Engineer with some of the following:
* Medical imaging solutions
* Technical ability to configure images systems to load and transfer files
* Knowledge of medical image formats and metadata
* Work closely with data scientists to solve technical issues in building analytical blueprints in ML, DL, and genomic sequence analysis
* Understanding of ML/DL and imaging processing in big data computing engine (e.g. Spark)Familiarity with performance tuning skills (Spark, Data I/O); the internals of Spark and other components in the Hadoop ecosystem (HDFS, Hive)
Contract Duration: 6 months, extendable - daily rates.
Ideally you will be based in Ireland but we will consider anyone with the above skillset who is based in Europe.
Please note that EU/EEA nationals or existing Irish work visa holders only need apply.
IRC is acting as an Employment Business in relation to this vacancy.
Email me jobs like this
