Lead Date Engineer Jobs

ROLE : LEAD DATA ENGINEER

Location - needs to be in office 3 times per week in Toronto

Client - CPPIB

Rate - 80 CAD

MANDATORY - Terraform and Kubernetes

PositionLead Data Engineer,TorontoLocation

Toronto

OfficeAddress

ProjectDescription

The Data Engineering team is looking for people who are passionate about working in agile delivery environments and resolving the engineering challenges of building robust and scalable data systems aligned to enterprise data strategy.

As Lead, Data Engineer you will be responsible for developing, constructing, and testing large-scale data analytics systems based on AWS cloud that will help address the disparate analytics needs of a growing organization.

Responsibilities

• - Participate in the development life cycle from start to completion - requirements analysis, development, testing, and deployment.
• - Define and shape Investments' future technology and research process
• - Design solutions aligned with long-term architecture and technology strategy using Amazon Web Services (AWS) for Cloud development.
• - Ensure architecture will support the requirements of the Investments business.
• - Work in a fast-paced environment collaborating with developers, data engineers, architects, researchers, and data scientists.
• - Develop tools that prepare, transform, combine, and manage structured and unstructured data for use by Investments business users.
• - Conceptualize, design, and implement analytics products that enhance Investments analytics capabilities.

Skills

Must have

• - Knowledge & experience with driving IaC concepts within an organization, leveraging Terraform, Ansible, GitHub actions.
• - University degree in Engineering or Computer Science preferred.
• - AWS EMR, S3, Glue.
• - Overall IT experience 8+ years
• - Deep proficiency in Python with experience using Spark, Pandas or PySpark.
• - Hands-on expertise with building data pipelines and applications, leveraging Kubernetes, Airflow,
• - Hands-on experience building data exploratory interfaces using Notebooks (JupyterHub/Lab, Livy, SPARK, Hudi, Hive)
• - Experience of ETL pipelines, managing multiple datasets and providing necessary support.

Nice to have

• - Familiar with cloud technology best practices to enable the distribution and analysis of big data on the cloud (formatting/partitioning/etc.).
• (AWS SagemakerSparkmagic, Enterprise Gateway, matplotlib, Plotly/Dash, etc.).
• - Experience with front-end framework ie. Angular, React is a plus
• - Demonstrated ability to easily deal with both abstract and concrete concepts and be able to reconcile them for the appropriate audience and context preferred.
• - Ability to work in an entrepreneurial environment and be a self-starter.
• - Demonstrate strong facilitation, negotiation, interpersonal, communication and collaboration skills.
• - Quickly understand organizational dynamics and management priorities, and to be able to work effectively in a fast paced, results driven company.
• - Interests in the financial industry.
• - Experience with Cloud based data and analytics platforms, warehouses (Redshift/Spectrum, Databricks, Snowflake), BI Tools, OLAP systems (Clickhouse, Druid), including a mix of relational, non-relational, streaming and event-based architectures.

Please share the resumes to [email protected]

Thanks & Regards

SUMANJALI DHARAMKARI,

Technical Recruiter

Seven Hills Group Technologies Inc

E:[email protected]

Contact:+1(908)-275-4957

http://www.sevenhillsgrouptech.com//