Developer And Data Engineer

We are seeking a talented part-time Developer and Data Engineer who is experienced working with a range of technologies, including Kafka, Elasticsearch, Linux (CentOS, Ubuntu), databases, in-memory databases such as Redis, Python development, and developing RESTful APIs. The ideal candidate should have a strong background in software development and a deep understanding of data engineering principles. As a Developer and Data Engineer, you will play a critical role in designing, implementing, and maintaining data pipelines and applications that support our data-driven initiatives.

Details

• Hours:Up to 28 hours per week
• Schedule:To be determined between you and your supervisor
• Location:University of New Brunswick, Fredericton
• Dates:July 1stto August 31st, 2023

Responsibilities

• Build and maintain Elasticsearch clusters to enable fast and accurate search and analytics capabilities.
• Experience working with big data NoSQL databases such as Elasticsearch.
• Conduct thorough testing and debugging to ensure the reliability and quality of the developed solutions.
• Develop RESTful APIs to expose data and functionality to internal and external consumers.
• Design, develop, and maintain robust, scalable, and efficient data pipelines and applications.
• Write clean, maintainable, and efficient code, following best practices and coding standards.
• Develop, maintain, and optimize search queries for the Elasticsearch platform.
• Configure and manage Kafka topics and deploy producers and consumers to support real-time data streaming and event-driven architectures.
• Experience working with Message Brokers (such as Kafka) and publisher-subscriber architectures.
• Stay up to date with industry trends, emerging technologies, and best practices related to data engineering and software development.
• Monitor system performance and troubleshoot issues, providing timely resolutions.
• Work with in-memory databases such as Redis to enhance caching and improve system performance.
• Manage and optimize databases, ensuring data integrity, security, and high performance.
• Develop, maintain and debug codes written in python.
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications and solutions.

Job qualifications and skill sets

• Experience working with in-memory databases like Redis for caching and performance enhancement.
• Experience with Kafka, including configuring topics, consumers and producers and working with real-time data streaming.
• Strong proficiency in programming languages such as Python, with experience in developing robust, scalable applications.
• Solid understanding of Linux operating systems, particularly CentOS and Ubuntu.
• Familiarity with databases such as MySQL, PostgreSQL, or MongoDB, including data modelling and query optimization.
• Experience in developing RESTful APIs using frameworks like Flask or Django or FastAPI.
• Working knowledge in Elasticsearch, including cluster setup, indexing, querying, and performance optimization.
• Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.
• Excellent communication and collaboration skills, with the ability to work effectively in a team environment.
• In-depth knowledge of data engineering principles, including data integration, ETL processes, and data modelling.

Asset qualifications

• Knowledge of data visualization tools such as Kibana or Tableau.
• Familiarity with DevOps practices and tools for continuous integration and deployment.
• Knowledge of containerization technologies like Docker and orchestration tools like Kubernetes.
• Understanding of big data frameworks such as Hadoop and Spark.
• Experience with cloud platforms such as AWS, Azure, or GCP.

Application review will continue until the positions are filled.To apply please include a cover letter and your curriculum vitae.