Data Engineer Jobs

We are looking for Junior Data Engineer for our large telecom client who have experience in Hadoop and GCP migration.

Suitable candidate shall participate in all aspects of Big Data solution delivery life cycle including analysis, design, development, testing, production deployment, and support.

What are the top three functions of this role? (Top duties the contractor will be accountable for)

1. Coding for building an ETL pipeline, Assist on migrating platform to GCP

2. Develop standardized practices for delivering new products and capabilities using Big Data technologies, including data acquisition, transformation, and analysis

3. Ensure Big Data practices integrate into overall data architectures and data management principles (e.g. data governance, data security, metadata, data quality)

What will the contractor need to deliver in their initial 1–6 months? Deliver one data ingestion project independently

What are the top 3 skills sets you want to see on a candidate’s resume?

1. Hadoop Developer

2. Knowledge about GCP and Cloud native solutions

3. Analyst in Business Intelligence

Which qualifications are essential, if any, for the role? Knowledge of SQL and couple of programming language

:

As a member of the Network Big Data team, reporting to the Network Big Data CoE Manager, the Big Data DevOps will play a leading role in the development of new products, capabilities, and standardized practices using Big Data technologies. Working closely with our business partners, this person will be part of a team that advocates the use of Big Data technologies to solve business problems, and be a thought-partner in the Big Data space.

Primary Responsibilities:

· Participate in all aspects of Big Data solution delivery life cycle including analysis, design, development, testing, production deployment, and support

· Develop standardized practices for delivering new products and capabilities using Big Data technologies, including data acquisition, transformation, and analysis

· Ensure Big Data practices integrate into overall data architectures and data management principles (e.g. data governance, data security, metadata, data quality)

· Create formal written deliverables and other documentation, and ensure designs, code, and documentation are aligned with enterprise direction, principles, and standards

· Train and mentor teams in the use of the fundamental components in the Hadoop stack

· Assist in the development of comprehensive and strategic business cases used at management and executive levels for funding and scoping decisions on Big Data solutions

· Troubleshoot production issues within the Hadoop environment

· Performance tuning of a Hadoop processes and applications

· Proven experience as a Hadoop Developer/Analyst in Business Intelligence

· Strong communication, technology awareness and capability to interact work with senior technology leaders is a must

· Good knowledge on Agile Methodology and the Scrum process

· Delivery of high-quality work, on time and with little supervision

· Critical Thinking/Analytic abilities

Basic Qualifications:

· Bachelor in Computer Science, Management Information Systems, or Computer Information Systems is required.

· Minimum of 4 years of Building Java apps

· Minimum of 2 years of building and coding applications using Hadoop components - HDFS, Hive, Impala, Sqoop, Flume, Kafka, StreamSets, HBase, etc.

· Minimum of 2 years of coding Scala / Spark, Spark Streaming, Java, Python, HiveQL

· Minimum 4 years understanding of traditional ETL tools & Data Warehousing architecture.

· Strong personal leadership and collaborative skills, combined with comprehensive, practical experience and knowledge in end-to-end delivery of Big Data solutions.

· Experience in Exadata and other RDBMS is a plus.

· Must be proficient in SQL/HiveQL

· Hands on expertise in Linux/Unix and scripting skills are required.

Preferred Qualifications:

· Strong in-memory database and Apache Hadoop distribution knowledge (e.g. HDFS, MapReduce, Hive, Pig, Flume, Oozie, Spark)

· Past experience using Maven, Git, Jenkins, Se, Ansible or other continuous integration tools is a plus

· Proficiency with SQL, NoSQL, relational database design and methods

· Deep understanding of techniques used in creating and serving schemas at the time of consumption

· Identify requirements to apply design patterns like self-documenting data vs. schema-on-read.

· Played a leading role in the delivery of multiple end-to-end projects using Hadoop as the data platform.

· Successful track record in solution development and growing technology partnerships

· Ability to clearly communicate complex technical ideas, regardless of the technical capacity of the audience.

· Strong inter-personal and communication skills including written, verbal, and technology illustrations.

· Experience working with multiple clients and projects at a time.

· Knowledge of predictive analytics techniques (e.g. predictive modeling, statistical programming, machine learning, data mining, data visualization).

· Familiarity with different development methodologies (e.g. waterfall, agile, XP, scrum).

· Demonstrated capability with business development in big data infrastructure business

Candidates have to pass a written HackerRank test focussed on Python or Java.

Stable long term job with the telecom giant in Mississauga and lots of growth potential. Please send in your resume