Our Client is a cutting edge technology company that has established itself as the preferred supplier in their field to industry leading enterprise software companies. They provide their clients with comprehensive telemetry & business intelligence solutions that allow them to protect their intellectual property and convert any licensing breaches into paying customers. They are now looking to recruit a Lead Data Engineer, for their Dublin development centre. This role will be responsible for the design, development and deployment of a large scale AWS Cloud based enterprise data integration and processing solution.
Core Role Responsibilities:
* Design and develop a cloud based enterprise solution to connect multiple data feeds and repositories as part of a scalable data processing solution.
* Manage a small team of data engineers to build production quality data processing solutions.
* Provide guidance to junior members of the team on technology choices, design and implementation of data integrations.
* Continuous re-evaluation of existing data processing solutions to maintain optimal performance, data security and availability.
* Assure data quality using data cleansing techniques.
* System troubleshooting and problem solving.
* Documenting the configuration and operation of the data integration platform.
* Evaluation and adoption of suitable new technologies, for example No SQL and Graph Databases, Big Data tools.
* Other activities and responsibilities that may be deemed appropriate as required.
Required Experience and Skills:
* AWS Cloud: Experience developing/deploying in an AWS Cloud environment.
* Big Data tools: Knowledge of big data tools, for example Elastic Search, Apache Kafka, Redis.
* Scripting Skills: Strong scripting and automation skills (Python preferred).
* Data Interchange: Developing with XML, JSON based technologies.
* Web Services: Developing and integrating web services (ReST, SOAP).
* Deployment Containerization: Knowledge of Docker.
Desirable Skills (Not essential but beneficial to have):
* Relational Database Skills: Experience writing SQL for data ETL.
* Scala Development: Scala development experience.
* Agile Methodologies: Experience working in an Agile development environment.
* Monitoring Tools: Experience in integration of monitoring systems (Zabbix, CloudWatch, Kibana) with ETL services.
* BSc in Computer Science or other equivalent technical discipline