EN | FR

293 Hadoop jobs in Canada

Data Engineer – AWS & Hadoop

Toronto, Ontario Astra North Infoteck Inc.

Posted 11 days ago

Job Viewed

Tap Again To Close

Job Description

Skills Required:NetezzaSQL Server 2012 Integration Services (SSIS)Role and Responsibilities:Understand requirements from product owners and translate them into requirement and scope documentsDecide on the best fitment of technologies/services available within scopeCreate solutions for data ingestion and transformation using Hadoop services such as Spark, Spark Streaming, Hive, etc.Create technical design documents to communicate solutions and mentor the team in solution developmentBuild solutions with AWS and Hadoop services as per design specificationsAssist teams in building test cases and support testing effortsCoordinate with upstream, downstream, and other supporting teams for production implementationProvide post-production support for implemented solutionsMandatory Skills and Experience:Strong working experience with AWS data services (EMR, S3, Glue)Hands-on experience with AWS EMR, Glue (Spark with Scala), and AWS S3Strong hands-on experience in Hadoop services like SparkExtensive experience in building batch workloads on AWS using AWS EMRAdept at analyzing and refining requirements, consumption query patterns, and choosing the right technology fit (RDBMS, data lake, data warehouse)Additional Technical Skills:Knowledge of analytical data modeling on any RDBMS/MPP platformKnowledge of PythonPractical experience migrating Hadoop-based data lakes from on-premises to AWS EMR on EMRFSNice-to-Have Skills and Experience:Experience handling terabytes/petabytes of data and millions of transactions per daySkills to develop ETL pipelines using AirflowKnowledge of Spark Streaming or other streaming jobsAbility to deploy code using AWS Code Pipeline and BitbucketExpertise in programming languages such as Scala, Java, and comfort working on Linux platformKnowledge of cloud-based MPP platformsExperience Required: 6-8 years
This advertiser has chosen not to accept applicants from your region.

Data Engineer - Bigdata,Hadoop,Python, ReactJS,PL/SQL

Toronto, Ontario Astra North Infoteck Inc.

Posted 26 days ago

Job Viewed

Tap Again To Close

Job Description

Location: TORONTO Exp Required: 8-10Skills Required: Big data and Hadoop Ecosystems, Python, ReactJS, PL/SQLJob Description: Resource for Senior Data Engineer Must-have:Experience with SQL Server Back-End development including: SQL Scripts, Stored-Procedures, Dynamic Stored-Procedures·Experience development Python scripts and libraries· Experience with Agile development, GitHub, SSMS, Dataiku·Experience working with databases such as SQL Server, HDFS· Experience with DBMS Data Modeling, ERD, Data Pipelines and DQ rules Implementation·Experience to Implement complex logic from Business Requirements using both SQL Stored Procedures and/or Python·Experience to produce Architecture Design and Technical documents for all Layers such as Front-End UI, API End Points, and Back-End Stored Procedures.Nice-to-have Experience ETL Work Flow and Python/SQL and Dataiku Recipes using Dataiku PlatformExperience designing front-end solutions with React.js, OpenShift, API End-Points (APIGEE)Experience with HR systems & workflow management tools such as Workday, SAP, ServiceNowExperience developing Tableau Dashboard and reusable Work Books.Good knowledge of API Endpoints development using Python Language.
This advertiser has chosen not to accept applicants from your region.

"AWS Data Engineer - Hadoop, Spark, Scala, AWS DataLake *Netezza"

Toronto, Ontario Astra North Infoteck Inc.

Posted 13 days ago

Job Viewed

Tap Again To Close

Job Description

AWS Data Engineer  - Hadoop, Spark, Scala, AWS DataLake *NetezzaRole and Responsibilities

Understand business requirements from product owners and convert them into technical scope and requirement documents

Design end-to-end data ingestion and transformation solutions using Hadoop ecosystem (Spark, Spark Streaming, Hive, etc.)

Create technical design documentation and mentor team members on implementation

Develop scalable solutions using Spark on AWS Data Lake

Build reusable frameworks for data engineering on AWS using services like S3, EMR, Glue, etc.

Coordinate with cross-functional teams (upstream/downstream) for production deployments

Provide post-production support and bug fixes as needed

Interpret and migrate existing Netezza/Hadoop features into AWS Data Lake architecture

Assist QA/SIT teams with unit testing, functional testing, and migration activities

Work with stakeholders to define reusable design patterns for data onboarding and integration

5 MUST-HAVE Skills & Experience

Strong hands-on experience with Hadoop ecosystem , particularly Hive and Spark with Scala

Extensive experience in data engineering on AWS , including S3, EMR, Glue, Redshift, Lake Formation, and Python

Proficiency in PySpark and building batch workloads on Hadoop and AWS platforms

Experience with code versioning and deployment tools like Bitbucket, Artifactory, and AWS CodePipeline

Understanding and implementation of data encryption techniques and secure data handling

 5 NICE-TO-HAVE Skills & Experience

Experience handling terabyte/petabyte-scale data and processing millions of transactions per day

Building and orchestrating ETL pipelines using Apache Airflow

Knowledge of Spark Streaming or similar streaming technologies

Proficiency in Scala or Java , and comfort with Linux-based environments

Familiarity with AWS services like Secrets Manager, KMS, Lambda , and Pythonic pipeline design principles

 Experience Required

6+ years of hands-on experience in data engineering, data warehouse, and data lake platforms

This advertiser has chosen not to accept applicants from your region.

Manager, Data Processing

Toronto, Ontario Sago

Posted today

Job Viewed

Tap Again To Close

Job Description

Job Description

Job Description

SAGO is a global leader in market research and data solutions, committed to connecting human answers to business questions. The world is transforming, and SAGO is continuously reaching new heights, redefining our solutions beyond our clients' wildest expectations, and making it smarter, faster, and easier to gain insights that matter.

We're a team of innovators, strategists, and problem-solvers, passionate and inspired thinkers, who celebrate our wins. Vibrant Leaders who empower our colleagues and innovate the Market Research industry. Curious learners, constantly finding new ways to transform. Bold thinkers and Authentic team members who show up to work every day being ourselves, never afraid of new ideas or opinions, because we believe they matter.

Requirements

The Manager of Data Processing will be responsible for managing Sago’s internal data team and relationship with vendors, in addition to working directly on data checks and tabulations. They will be asked to coordinate with programmers, quality assurance team members, project managers in managing timelines and responsibilities, ensuring Sago processes are being followed. They will be asked to represent the data team in discussions with clients, colleagues, and partners. They will be expected to help recruit, interview, and train the new members of the team, as well as inform processes and set standards for Sago’s data processing department.

Expectations would be to manage a highly dynamic team, processing and analyzing data to provide actionable insights into market trends, consumer behavior, and competitive dynamics. Must be able to manage a heavy workload with high expectations for quality and turnaround time. Must be able to delegate work, formulate processes for the entire team to follow, and both provide and accept constructive feedback on performance. Will be expected to manage systems for tracking error rates, performance, margin, and bandwidth. Must maintain low error rates across the team. Must be willing to learn new processes and systems with a positive mindset and eagerness to grow into the role.

The analyst uses statistical methods and visualization tools to interpret data and create detailed reports and dashboards. They collaborate with cross-functional teams to support strategic decision-making and continuously improve research methodologies. Strong analytical skills, proficiency in statistical software, and the ability to communicate findings effectively are essential for this position.

KEY JOB RESPONSIBILITIES:

  • Serves as advisor and consultant to members of other teams
  • Excellent Interpersonal skills, outstanding problem solving and decision-making skills.
  • Train, develop and provide on-going technical support to junior internal staff, as well as outsourced teams on best practices and Schlesinger SOPs
  • Perform data checks using Quantum, SPSS and Excel
  • Check Max-diff, Conjoint and segmentation setups
  • Prepare tables in Quantum.
  • Writing data cleaning scripts
  • Do data manipulation like labeling, stacking, recoding etc.
  • Resolving client queries and updating their requirements
  • Communicate and coordinate with clients and internal teams

CORE COMPETENCIES:

  • Have knowledge of Quantum and SPSS; renaming, recoding, labelling, stacking data and other similar types of data manipulation required by clients
  • Know how to QC tables
  • Adapt to working in high pressure environments with strict deadlines and multiple deliverables
  • Good English communication skills

QUALIFICATIONS

Education:

  • Bachelor’s degree

Experience:

  • 7+ years overall Data Processing experience in the Market Research industry
  • 3+ years management experience in the Market Research industry

Computer Skills:

  • Excellent with MS Teams, Office, Word, Excel, PowerPoint, and Outlook
  • SPSS
  • Quantum

This advertiser has chosen not to accept applicants from your region.

Principal Engineer - Distributed Data Processing System

L6G 0G1 Markham, Ontario Huawei Technologies Canada Co., Ltd.

Posted 10 days ago

Job Viewed

Tap Again To Close

Job Description

Huawei Canada has an immediate permanent opening for a Principal Engineer.


About the team:

Established in 2014, the Distributed Scheduling and Data Engine Lab is Huawei Cloud's technical innovation center in Canada. The lab focuses on researching and developing advanced cloud technologies, supporting the productization and iterative optimization of its technical achievements. Current research areas include cloud native databases, infrastructure resource scheduling and prediction, cloud-native middleware, media engines, and user experience studies. The lab fosters a robust technical environment, allowing collaboration with industry experts to create a highly competitive cloud platform. Our team has an immediate permanent opening for a Principal Software Engineer.


About the job:

  • Work with a team of architects and engineers to develop proof-of-concept distributed systems and product components.

  • Investigate and design data processing system focusing on extra-large volume data for real-time streaming, based distributed system with the latest and state of the art hardware technology.

  • Continuous enhancements on the AI-native data analysis system to fulfill customer requirements utilizing AI techniques for public cloud capacity management and scheduling.

This advertiser has chosen not to accept applicants from your region.

Intern, Aircraft Maintenance Data Analysis (Winter 2026)

Dorval, Quebec Bombardier

Posted 4 days ago

Job Viewed

Tap Again To Close

Job Description

**Innovation is in our DNA. is it in yours?**
Bombardier is a global leader in aviation, creating innovative and game-changing planes. Our products and services provide world-class experiences that set new standards in passenger comfort, energy efficiency, reliability and safety. We are a global organization focused on working together with a team spirit.
**Your boarding pass will include.**
Several conferences, including:
+ Meet An Executive
+ Women Taking Flight
Learning more about Bombardier, including:
+ Bombardier Products conference
+ Visits of the Bombardier sites
+ Bombardier Academy of Learning
Many social/networking opportunities, including:
+ Volunteering
+ Networking for Success
+ 5 à 7, Potluck, and much more!
**What are your contributions to the team?**
+ Support reliability data collection improvement process to ensure a high level of data accuracy and completeness.
+ Build new tools and interfaces to our data using Oracle, Access, Excel with VBA, and MS PowerBI.
+ Update and document processes and procedures as required.
**How to thrive in this role?**
+ You are currently completing a degree in Computer Science, IT/Software/Electrical/Aerospace Engineering or other related fields. Non-engineering fields will also be considered.
+ You have completed a minimum of 2 years of studies in this program.
+ You have strong analytical abilities.
+ You have in-depth technical knowledge of Microsoft Suite products
+ You have a strong understanding of how to manipulate data in MS Excel as required with VBA.
+ You have can create queries and filters in MS Access, for the purpose of engineering data analysis.
+ You have a good foundation with MS Power BI and PowerAutomate to build dashboards and reports (asset)
+ You have knowledge of SAP, Salesforce.com, Oracle, Azure, SQL, Maximo and Machine Learning (asset)
+ You possess excellent communication (English required), presentation and interpersonal skills.
+ You are a highly motivated team player with a sense of responsibility and good judgment.

**Boarding Information:**
+ Location: Administrative Centre (CA)
+ Duration: 8 months
+ Flexible workplace-Hybrid
_It is important to note that our internship opportunities are open to students only, not new graduates. All our interns may be required to occasionally travel outside of Canada for training/work purpose._
This advertiser has chosen not to accept applicants from your region.

Student Intern, Smart Energy - Pricing Management and Product Data Analysis

Toronto, Ontario Celestica

Posted 19 days ago

Job Viewed

Tap Again To Close

Job Description

Req ID:
Remote Position: Hybrid
Region: Americas
Country: Canada
State/Province: Ontario
City: Toronto
**16 Month Internship**
Celestica is a global electronics manufacturing services company, headquartered in Toronto, Canada, with over 28,000 employees across 40 sites in North America, Europe, and Asia.
Working with the world's leading technology companies across a broad range of industries, Celestica delivers a full suite of services - from design and engineering, to manufacturing and supply chain management - to help its customers develop and manufacture products that are transforming the global economy. These products span key areas such as data centre technology, green energy, healthtech, and autonomous vehicles.
Working at Celestica is more than a job. It's about being part of something bigger and shaping the future with your work. You'll take on exciting and challenging projects, get hands-on experience, collaborate with a global team, and learn from some of the greatest minds in the industry. And just as we aspire to imagine, develop and deliver a better future with our customers, we're just as committed to delivering the best employee experience.
**Benefits for Students Include:**
+ Experience working for one of the largest Canadian global technology companies
+ Possibility of a student casual contract position after completion of internship
+ Possibility of a full-time position after completion of degree
+ Access to Celestica's development programs and projects
+ Celestica's Time Off to Volunteer program - two paid days off to volunteer per year
+ Fun intern activities including celebration events and networking
+ Internal recognition programs and rewards for stellar performance
**What's in it for you?**
+ **Opportunity** : Job experience at a large Canadian-based global company
+ **Innovation** : We look to our employees to implement new ideas and improve the way we do things
+ **Collaboration** : Students work as part of global teams, enabled by collaborative technology
+ **Sustainability** : We uphold strong ethics, reduce our impact on the environment and empower employees to give back to the community with two paid days off per year to volunteer
+ **Development opportunities** : including soft skills courses, innovation projects and mentorship
+ **Networking** : Strategically build your professional network to advance your career and deepen relationships across your peers and colleagues.
+ **Fun** : Our interns have the opportunity to participate in fun activities including intern social events, Celestica's charitable initiatives and our Spirit Week activities
**About this opportunity:**
The **Smart Energy - Pricing Management and Product Data Analysis Intern** will work on the engineering manufacturing team to resolve quality issues and fix any quality issues. Work is guided by detailed instructions, routines, and procedures. In the first 3 months, the intern will work under close supervision and receive detailed instructions on all work assignments. Once proficiency in the tasks is achieved, the intern will work autonomously and be given more complex duties. The intern will also have the opportunity to work with colleagues and suppliers around the globe, gaining international experience.
**Preferred Skills:**
+ Advanced proficiency in MS Office (ex. Microsoft Excel functions such as VLOOKUP, PIVOT tables, Indexing, and data analysis).
+ Strong analytical skills.
+ Strong organizational skills.
+ Strong communication skills.
+ Project management experience.
+ Excel Macros and VBA are an asset.
+ Knowledge of SQL programming language is an asset.
**Activities:**
+ Understand content of customer Bill of Material (BOM) and Engineering drawings.
+ Read and process the customer "Approved Vendor List" (AVL).
+ Will be trained to analyze and deploy both the technical and commercial impacts of the Engineering Changes Notices (ECN).
+ Consolidate, tabulate, analyze and report on various topics, commitments, and performance metrics.
+ Communicate and report data issues and other requirements to internal and external contacts.
+ Understand and process supply chain related parameters like Lead Time (LT), Minimum Order Quantity (MOQ) and Estimated Annual Usage (EAU).
+ Participate in material bid pricing activities and interact with global customer business units.
+ Coordinate, send and negotiate quote tenders with suppliers to establish cost and set pricing.
+ Research Energy or other customers products and market trends.
+ Create presentations and reports using Excel & PowerPoint.
+ Support customers to resolve pricing management related issues for products.
+ Will have exposure to SAP and internal databases and applications. In some case will work with business and IT to maintain and develop new applications.
**Physical Demands, duties of this position are varied and will include:**
+ Duties of this position are performed in a normal office environment.
+ Duties may require extended periods of sitting and sustained visual concentration on a computer monitor or on numbers and other detailed data.
+ Repetitive manual movements are frequently required (e.g. data entry, using a computer mouse, etc).
**Additional Details:**
**Area:** Global Supply Chain Services
**Start Date:** May 2026
**Location:** 5140 Yonge Street, Toronto, ON
**Vacancies:** 1 position
**Experience:** No formal experience required
**Education:** Currently enrolled in an Internship program completing their 3rd or 4th year specializing in Business, Finance, Computer Science, or Engineering with the intention to return to school the following year.
***As this position involves work that is subject to the Canadian Government's Controlled Goods Program (CGP), your employment offer is conditional upon the successful completion of a security assessment by Celestica and/or its authorized agents, per the Controlled Goods Regulations.**
To apply, please visit **Please provide a copy of your transcript with your application. Please include your student email address in your application.**
Celestica is an equal opportunity employer. All qualified applicants will receive consideration for employment and will not be discriminated against on any protected status (including race, religion, national origin, gender, sexual orientation, age, marital status, veteran or disability status or other characteristics protected by law).
At Celestica we are committed to fostering an inclusive, accessible environment, where all employees and customers feel valued, respected and supported. Accommodations are available upon request for candidates throughout the hiring process. If you require an accommodation, we will work with you to meet your needs.
This position involves technologies regulated by the Canadian government's Controlled Goods Program (CGP). Candidates must be Canadian citizens or permanent residents of Canada and meet all other requirements under CGP. Job offers will be conditional upon candidates undergoing a successful security screening per the Controlled Goods Regulations.
Celestica would like to thank all applicants, however, only qualified applicants will be contacted.
Celestica does not accept unsolicited resumes from recruitment agencies or fee based recruitment services.
This advertiser has chosen not to accept applicants from your region.
Be The First To Know

About the latest Hadoop Jobs in Canada !

Student Intern, Smart Energy - Account Focused, Pricing Management and Product Data Analysis

Toronto, Ontario Celestica

Posted 26 days ago

Job Viewed

Tap Again To Close

Job Description

Req ID:
Remote Position: Hybrid
Region: Americas
Country: Canada
State/Province: Ontario
City: Toronto
**16 Month Internship**
Celestica is a global electronics manufacturing services company, headquartered in Toronto, Canada, with over 28,000 employees across 40 sites in North America, Europe, and Asia.
Working with the world's leading technology companies across a broad range of industries, Celestica delivers a full suite of services - from design and engineering, to manufacturing and supply chain management - to help its customers develop and manufacture products that are transforming the global economy. These products span key areas such as data centre technology, green energy, healthtech, and autonomous vehicles.
Working at Celestica is more than a job. It's about being part of something bigger and shaping the future with your work. You'll take on exciting and challenging projects, get hands-on experience, collaborate with a global team, and learn from some of the greatest minds in the industry. And just as we aspire to imagine, develop and deliver a better future with our customers, we're just as committed to delivering the best employee experience.
**Benefits for Students Include:**
+ Experience working for one of the largest Canadian global technology companies
+ Possibility of a student casual contract position after completion of internship
+ Possibility of a full-time position after completion of degree
+ Access to Celestica's development programs and projects
+ Celestica's Time Off to Volunteer program - two paid days off to volunteer per year
+ Fun intern activities including celebration events and networking
+ Internal recognition programs and rewards for stellar performance
**What's in it for you?**
+ **Opportunity:** Job experience at a large Canadian-based global company
+ **Innovation:** We look to our employees to implement new ideas and improve the way we do things
+ **Collaboration:** Students work as part of global teams, enabled by collaborative technology
+ **Sustainability:** We uphold strong ethics, reduce our impact on the environment and empower employees to give back to the community with two paid days off per year to volunteer
+ **Development opportunities:** including soft skills courses, innovation projects and mentorship
+ **Networking:** Strategically build your professional network to advance your career and deepen relationships across your peers and colleagues.
+ **Fun:** Our interns have the opportunity to participate in fun activities including intern social events, Celestica's charitable initiatives and our Spirit Week activities
**About this Opportunity:**
The **Smart Energy - Account Focused, Pricing Management and Product Data Analysis Intern** will work on the engineering manufacturing team to resolve quality issues and fix any quality issues. The intern will be assigned to work with a client for the duration of the internship. Work is guided by detailed instructions, routines, and procedures. In the first 3 months, the intern will work under close supervision and receive detailed instructions on all work assignments. Once proficiency in the tasks is achieved, the intern will work autonomously and be given more complex duties. The intern will also have the opportunity to work with colleagues and suppliers around the globe, gaining international experience.
**Preferred Skills:**
+ Advanced proficiency in MS Office (ex. Microsoft Excel functions such as VLOOKUP, PIVOT tables, Indexing, and data analysis).
+ Strong analytical skills.
+ Strong organizational skills.
+ Strong communication skills.
+ Project management experience.
+ Excel Macros and VBA are an asset.
+ Knowledge of SQL programming language is an asset.
**Activities:**
+ Understand content of customer Bill of Material (BOM) and Engineering drawings.
+ Read and process the customer "Approved Vendor List" (AVL).
+ Will be trained to analyze and deploy both the technical and commercial impacts of the Engineering Changes Notices (ECN).
+ Consolidate, tabulate, analyze and report on various topics, commitments, and performance metrics.
+ Communicate and report data issues and other requirements to internal and external contacts.
+ Understand and process supply chain related parameters like Lead Time (LT), Minimum Order Quantity (MOQ) and Estimated Annual Usage (EAU).
+ Participate in material bid pricing activities and interact with global customer business units.
+ Coordinate, send, and negotiate quote tenders with suppliers to establish cost and set pricing.
+ Researching Energy or other customers products and market trends.
+ Create presentations and reports using Excel & PowerPoint.
+ Support customers to resolve pricing management related issues for products.
+ Support internal customer requisitions and accurately input data on the products pricing information.
+ Report current, accurate data on internal databases.
+ Track prices for components, monitor the influx of data, and understand what drives cost reduction in daily activities.
+ Will have exposure to SAP and internal databases and applications. In some case will work with business and IT to maintain and develop new applications.
**Physical Demands, duties of this position are varied and will include:**
+ Duties of this position are performed in a normal office environment.
+ Duties may require extended periods of sitting and sustained visual concentration on a computer monitor or on numbers and other detailed data.
+ Repetitive manual movements are frequently required (e.g. data entry, using a computer mouse, etc).
**Additional Details:**
**Area:** Global Supply Chain Services
**Start Date:** May 2026
**Location:** 5140 Yonge Street, Toronto, ON
**Vacancies:** 1 position
**Experience:** No formal experience required
**Education:** Currently enrolled in an Internship program completing their 3rd or 4th year specializing in Business, Finance, Computer Science, or Engineering with the intention to return to school the following year.
***As this position involves work that is subject to the Canadian Government's Controlled Goods Program (CGP), your employment offer is conditional upon the successful completion of a security assessment by Celestica and/or its authorized agents, per the Controlled Goods Regulations.**
To apply, please visit **Please provide a copy of your transcript with your application. Please include your student email address in your application.**
Celestica is an equal opportunity employer. All qualified applicants will receive consideration for employment and will not be discriminated against on any protected status (including race, religion, national origin, gender, sexual orientation, age, marital status, veteran or disability status or other characteristics protected by law).
At Celestica we are committed to fostering an inclusive, accessible environment, where all employees and customers feel valued, respected and supported. Accommodations are available upon request for candidates throughout the hiring process. If you require an accommodation, we will work with you to meet your needs.
This position involves technologies regulated by the Canadian government's Controlled Goods Program (CGP). Candidates must be Canadian citizens or permanent residents of Canada and meet all other requirements under CGP. Job offers will be conditional upon candidates undergoing a successful security screening per the Controlled Goods Regulations.
Celestica would like to thank all applicants, however, only qualified applicants will be contacted.
Celestica does not accept unsolicited resumes from recruitment agencies or fee based recruitment services.
This advertiser has chosen not to accept applicants from your region.

Big Data Engineer

Mississauga, Ontario Motion Recruitment Partners

Posted 6 days ago

Job Viewed

Tap Again To Close

Job Description

Big Data Engineer
Mississauga, Ontario
**Hybrid**
Contract
$66/hr - $71/hr
Grow your career as a Big Data Engineer with an innovative global bank in Mississauga, Ontario. Contract role with strong possibility of extension or conversion. Will require working a hybrid schedule 3 days onsite per week.
Join one of the world's most renowned global banks and trusted brand with over 200 years of continuously evolving financial services worldwide. You will work alongside some of the smartest minds in the industry who are excited to share their knowledge and to learn from you.
Contract Duration: 11+ Months
**Required Skills & Experience**
+ 10+ years of Big Data development experience. At least 5 recent years of hands-on coding!
+ Must have strong Spark and Scala skills. Strong Scala development (with previous Java background) preferred.
+ ands-on experience working with Hive, Impala, Kafka, HBase, Spark for data curation/conformance related work.
+ Strong proficiency in Spark for development work related to curation/conformance.
+ Must have data streaming experience with Spark/Kafka or equivalent streaming/batch processing and event-based messaging.
+ Solid understanding of the Big Data architecture and the ability to trouble shoot performance and/or development issues on Hadoop (Cloudera preferably)
+ Experience working in a Continuous Integration and Continuous Delivery environment and familiarity with tools like Jenkins, TeamCity, SonarQube, OpenShift, ECS, or Kubernetes.
+ Proficient in industry-standard best practices such as Design Patterns, Coding Standards, Coding modularity, and Prototyping.
+ Experience with data visualization tools and techniques for presenting data insights effectively.
+ Agile development experience working in agile and scaled agile teams.
+ Strong data analysis skills and the ability to slice and dice the data as needed for business reporting.
+ Relational SQL (Oracle, SQL Server), NoSQL (MongoDB) and Cache (Couchbase) database integration and data distribution principles experience
+ Leadership & Mentorship: Ability to guide and mentor junior developers, fostering a collaborative team environment and promoting professional growth.
+ Strong communication skills, both written and verbal, with the ability to explain complex technical concepts to both technical and non-technical audiences.
+ Financial services is preferred (e.g., payments, ISO 222 format, posting mechanisms). E-commerce experience is also helpful.
**What You Will Be Doing**
+ Utilize in-depth knowledge and hands-on skills across multiple Applications Development areas to provide technical oversight across systems and applications
+ Will work very closely with and manage the work of a team of data engineers working on a Big Data Platform.
+ 70% hands-on development & coding, 30% delegation and leading a team of Applications developers.
+ Review/analyze/develop proposed technical solutions for projects
+ Contribute to formulation of strategies for applications development and other functional areas
+ Develop comprehensive knowledge of how areas of business integrate to accomplish business goals
+ Provide evaluative judgment based on analysis of factual data in complicated and unique situations
+ Impact the Applications Development area through monitoring delivery of end results, participate in budget management, and handling day-to-day staff management issues, including resource management and allocation of work within the team/project
+ Ensure essential procedures are followed and contribute to defining standards negotiating with external parties when necessary
Motion Recruitment Partners (MRP) is an Equal Opportunity Employer. All applicants must be currently authorized to work on a full-time basis in the country for which they are applying, and no sponsorship is currently available. Employment is subject to the successful completion of a pre-employment screening. Accommodation will be provided in all parts of the hiring process as required under MRP's Employment Accommodation policy. Applicants need to make their needs known in advance.
**Posted by:** Melissa Klein
**Specialization:**
+ Data Engineering
This advertiser has chosen not to accept applicants from your region.

Senior Big Data Engineer

Mississauga, Ontario Citigroup

Posted 17 days ago

Job Viewed

Tap Again To Close

Job Description

About the Role:
+ We are looking for a Senior Data Architectto build and maintain - Platform Modernization, TransformationPrograms
+ You'll be part of a Core Accounts Data team that's responsible for the full software development life cycle, upholding best-in-class engineering standards and modern methodologies.
+ This role involves writing code, collaborating with other developers,understanding business requirements, developing technical solutions, collaborating with other business partners, breaking down acceptance criteria to understand team backlog deliverables, complexities, and risks, and contributing effectively to an agile team.
Key Responsibilities:
+ Design, develop, and maintain robust, scalable, and high-performance applications
+ Partner with architects, product owners, and cross-functional teams to design scalable and distributed system
+ Follow best-in-class Big Data/ETL Applications design standards, security considerations, scalability, maintainability and resource utilization aspects
+ Ensure high levels of unit test coverage, DevOps principles and a culture of continuous delivery and improvement
+ Actively contribute to hands-on coding, code reviews, and refactoring to maintain high engineering standards
+ Ability of analyzing the results across the environment and finding potential or existing errors in code/builds
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
+ Mentor junior engineers and foster a culture of engineering excellence and continuous learning.
Skills and Qualifications:
+ 10+ years of experience
+ Excellent Understanding of Bigdata/ETL ecosystem
+ Proficiency in Apache Spark development with Scala
+ Experience in building Spark Streaming and Batch/Live streaming applications
+ Strong experience in creating applications for Big Data and other Data & Analytics technologies (Spark, Scala, Hadoop, Hive etc.)
+ Ability to analyze Spark job performance in Spark UI and apply required performance tuning
+ Experience of working with Linux Scripting, Kafka, Git, Maven, SSH
+ Well versed with creating and debugging Scala code in IntelliJ IDEA
+ Good problem solving skills
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Primary Location Full Time Salary Range:**
$94,300.00 - $141,500.00
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
This advertiser has chosen not to accept applicants from your region.
 

Nearby Locations

Other Jobs Near Me

Industry

  1. request_quote Accounting
  2. work Administrative
  3. eco Agriculture Forestry
  4. smart_toy AI & Emerging Technologies
  5. school Apprenticeships & Trainee
  6. apartment Architecture
  7. palette Arts & Entertainment
  8. directions_car Automotive
  9. flight_takeoff Aviation
  10. account_balance Banking & Finance
  11. local_florist Beauty & Wellness
  12. restaurant Catering
  13. volunteer_activism Charity & Voluntary
  14. science Chemical Engineering
  15. child_friendly Childcare
  16. foundation Civil Engineering
  17. clean_hands Cleaning & Sanitation
  18. diversity_3 Community & Social Care
  19. construction Construction
  20. brush Creative & Digital
  21. currency_bitcoin Crypto & Blockchain
  22. support_agent Customer Service & Helpdesk
  23. medical_services Dental
  24. medical_services Driving & Transport
  25. medical_services E Commerce & Social Media
  26. school Education & Teaching
  27. electrical_services Electrical Engineering
  28. bolt Energy
  29. local_mall Fmcg
  30. gavel Government & Non Profit
  31. emoji_events Graduate
  32. health_and_safety Healthcare
  33. beach_access Hospitality & Tourism
  34. groups Human Resources
  35. precision_manufacturing Industrial Engineering
  36. security Information Security
  37. handyman Installation & Maintenance
  38. policy Insurance
  39. code IT & Software
  40. gavel Legal
  41. sports_soccer Leisure & Sports
  42. inventory_2 Logistics & Warehousing
  43. supervisor_account Management
  44. supervisor_account Management Consultancy
  45. supervisor_account Manufacturing & Production
  46. campaign Marketing
  47. build Mechanical Engineering
  48. perm_media Media & PR
  49. local_hospital Medical
  50. local_hospital Military & Public Safety
  51. local_hospital Mining
  52. medical_services Nursing
  53. local_gas_station Oil & Gas
  54. biotech Pharmaceutical
  55. checklist_rtl Project Management
  56. shopping_bag Purchasing
  57. home_work Real Estate
  58. person_search Recruitment Consultancy
  59. store Retail
  60. point_of_sale Sales
  61. science Scientific Research & Development
  62. wifi Telecoms
  63. psychology Therapy
  64. pets Veterinary
View All Hadoop Jobs