43 Urgent Data Engineer Ruby Aws Python Apache Spark jobs in North Vancouver

Your Job

DarkVision, a Koch Engineered Solutions company, is seeking a Data Analysis Team Lead to join our fast growing In-line (pipeline) Inspection team in North Vancouver, BC . DarkVision leverages cutting-edge proprietary acoustic imaging technology to deliver industry leading asset inspection services, capable of detecting defects as small as a few millimeters. As an ILI (In-Line Inspection) Data Team Lead, you will manage a team of Data Analysts to produce accurate inspection reports critical to the safe operation of oil and gas pipelines.

The ideal candidate has experience leading technical teams to deliver mission-critical results in a fast-paced, entrepreneurial environment. You bring a passion for operational excellence, process optimization, and challenging the status quo. Join our innovative team - comprising experts in machine vision, imaging, computer graphics, and data analytics - as we continue building the world's most advanced platform for industrial imaging and pipeline diagnostics.

Our Team

Our Data Analysts utilize 3D data visualization software and human-in-the-loop machine learning processes to meticulously analyze ultrasound datasets. The outputs of these analyses result in critical insights that enable our clients to make data-informed decisions on the operations and maintenance of multi-million-dollar infrastructure assets.

What You Will Do

• Learn the delivery standards and workflow processes for your assigned ILI product line
• Regularly check in with team members to assess capacity and remove roadblocks or inefficiencies
• Through frequent 1:1 meetings, track team members' progression in their role, deliver constructive feedback, and ensure they receive proper recognition for their work while building a roadmap for their career development
• Facilitate communication between your team and other teams and upper management
• Ensure employees are aligned with the company vision, and that the goals and direction of the team help drive achievement of the vision
• Facilitate the improvement of workflow procedures and best practices; seek out new solutions, seek feedback, and investigate and implement improvement opportunities
• Review projects assigned to Analysts on your team; you are responsible for the content and delivery of projects
• Partner with Recruiting and other Team Leads to recruit and evaluate candidates; a high volume of growth and hiring is expected
• Make informed recommendations for hiring, raises/promotions, performance management, and further training
• Develop new and improve existing processes and standards
• Set project deadlines, assign project leads, and manage follow-up requests
• Forecast team capacity vs. projected projects, offering transparency to stakeholders

Who You Are (Basic Qualifications)

• At least 2 years of engineering or science study at university, college, or technical school
• Experience managing a team producing client deliverables (e.g. reports, analysis)
• Experience managing early to mid-career individuals
• Experience leading process improvement initiatives

What Will Put You Ahead

• Experience in the Oil & Gas industry (e.g. ILI, pipeline or piping integrity, NDE/NDT, well integrity, well logging) or industrial asset inspection services
• Experience managing teams with at least 5 team members
• Experience in rapidly scaling an operation or team
• Able to disseminate complex information and generate pragmatic solutions
• Able to assess and adjust work processes to meet scaling requirements; delegate and prioritize work effectively among the team

For this role, we anticipate paying $85,000 - $110,000 CAD per year. This role is eligible for variable pay, issued as a monetary bonus or in another form.

At Koch companies, we are entrepreneurs. This means we openly challenge the status quo, find new ways to create value and get rewarded for our individual contributions. Any compensation range provided for a role is an estimate determined by available market data. The actual amount may be higher or lower than the range provided considering each candidate's knowledge, skills, abilities, and geographic location. If you have questions, please speak to your recruiter about the flexibility and detail of our compensation philosophy.

Who We Are

As a Koch Engineered Solutions company, DarkVision provides high-resolution acoustic imaging technology that offers significant improvements over existing ultrasonic and other imaging technologies on the market today. We serve customers in a variety of industries, such as public infrastructure, energy and pipelines, power (renewables and conventional), aerospace, manufacturing, automotive and others.

At Koch, employees are empowered to do what they do best to make life better. Learn how our business philosophy helps employees unleash their potential while creating value for themselves and the company.

Additionally, everyone has individual work and personal needs. We seek to enable the best work environment that helps you and the business work together to produce superior results.

How to Apply

If you have the above qualifications, we would like to hear from you. We thank all applicants in advance, but please be advised that only those selected for an interview will be contacted.

We are an equal opportunity employer. If you require accommodation or assistance at any time during the application or selection processes, please submit a request by following the directions located in the FAQ section at the bottom of the kochcareers.com webpage.

Successful candidates will be required to complete a criminal background check.

keywords: team lead, supervisor, leadership, management, supervision, analyst, analysis, reporting, reports, pipeline integrity, in-line inspection, ILI, asset inspection, condition monitoring, anomalies, cracks, deformation, metal loss, geometry, well integrity, pipes, piping, oil & gas, upstream, midstream

#LI-JN1

Job Description
Day to Day
Insight Global is seeking a Data Engineer to join one of a Retail Client in Vancouver BC. This Data Engineer will sit within the People Enablement Technology team which is committed to enabling seamless experiences for their global workforce. The Data & Analytics Pillar is a newly formed team, focused on automating our core health & wealth offerings as well as work across the department to enable efforts driven by the People Systems, Digital Workplace & Learning, and Workforce Agility pillars. The project the successful candidate will join is bringing the learning data from various platforms such as LI learning into an existing data lake. This will allow for analytics insights to be generated from this people data for various teams such as the ESG team. The data engineer will support using data bricks for this ingestion, and use unity catalog for its categorization and potential encryption if required.
A successful candidate will be a problem solver and an expert in ETL programming/scripting, data modelling, data integration, SQL and have exemplary communication skills. The candidate will need to be comfortable with ambiguity in a fast-paced and ever-changing environment, and able to think big while paying careful attention to detail. The candidate will know, and love working with new technologies, can model multidimensional datasets, and can partner with cross functional business teams to answer key business questions. Apart from building data pipelines, you will be an advocate for automation, performance tuning and cost optimization. Be ready to question the status quo and bring forth intelligent solutions and proof of concepts.
Principal Duties and Responsibilities:
Support the design and implementation of complex data integration, modeling, and orchestration workflows using tools such as Databricks, Azure Data Factory, and Snowflake
Collaborate with architects, engineers, analysts, and business stakeholders to deliver secure, reliable, and high-performing data solutions
Apply DevSecOps principles to ensure secure development, deployment, and monitoring of data pipelines and infrastructure
Implement and maintain data encryption, access controls, and governance using Databricks Unity Catalog to protect sensitive information
Participate in operational support, including troubleshooting, performance tuning, and continuous improvement of data workflows
Create and maintain technical documentation and contribute to knowledge-sharing across the team
We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form ( . The EEOC "Know Your Rights" Poster is available here ( .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: .
Skills and Requirements
Must Haves:
5+ years of experience as a Data Engineer working with Databricks, SQL & Python
3+ years with Cloud Data Integration & Modelling (Azure Data Factory)
Strong experience with Unity Catalog for categorization and filing sensitive data
Experience with DevOps tools and building secure, scalable data workflows
Proficiency in Python, and Pyspark
Experience working with people centered and sensitive data to understand the complexities of encryption, decryption (such as health information, payment information, banking information).
Experience with data modeling, data warehousing, and building ETL pipelines.
Excellent problem-solving skills, combined with the ability to present your findings/insights clearly and compellingly in both verbal and written form.
Strong documentation and communication skills
Bachelors degree in Computer Science, Mathematics, Statistics, Operations Research. Plusses:
- Snowflake experience null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to

Job Description

Job Description

We are looking for an experienced Senior Data Engineer for our client. This is a permanent position that is completely remote! Our client is a global enterprise company with a product that you've likely used.

Within this role, you'll have the chance to work on an application that is used by millions of users daily. With your expertise in BI, Data warehousing and big data you'll contribute to the expanding expertise of the team and keep data components current!

You Have:

• Minimum 5+ years professional working experience as a Data Engineer
• Strong experience working with both batch and real-time ETL involving large data sets
• Experience mentoring and guiding Junior Data Engineers in previous roles
• Strong Experience working with SQL and large data sets
• Hands-on experience coding in Python is required
• Experience working with Business Intelligence and Data Warehousing
• Experience working with data services on the cloud such as AWS and/or Google Cloud Platform environments
• Experience with REST API's

Some AWESOME selling points:

• Remote + flexible work schedule
• North of six figures salary + Equity + Health Spending Account (it's competitive)
• Amazing opportunity for career progression to lead
• Amazing mentorship (seriously)
• Plenty of vacay
• RRSP Matching
• Team budget for education for employees
  
  

It's a seriously awesome opportunity! If this looks interesting to you, go ahead and click apply :)

* Unfortunately we are only able to consider candidates who either live in Canada, or currently have an active Canadian work visa or citizenship for this role.

Job Description

Job Description

Job Title: Data Engineer Intern

Location: Vancouver (Remote)
Duration: 4-8 Months
Salary: 20-22 CAD / hour

About the Company:

Join our exciting team, where innovation and collaboration are key to our success. Over the last six months, weve experienced tremendous growth, quadrupling our staff and continuing to scale rapidly. With a year-over-year revenue growth of 3x and a net profit increase of 4x, we are on an exceptional trajectory that reflects our commitment to excellence and innovation.

As part of our growing team, you'll have the chance to make a significant impact on the company's development. We believe every team member's voice is important, and we encourage you to share your ideas and help shape the companys direction. We are looking for individuals who are not only passionate about their work but who are also eager to grow into leadership roles within the company. With an open-door policy, youll work closely with top management, receive valuable mentorship, and have the opportunity to influence key business decisions. This is your chance to contribute to something meaningful and be a driving force behind the companys growth!

Position Overview:

As a Data Engineer Intern, you will work closely with an experienced data team to assist in designing, developing, and maintaining our data systems and infrastructure. This internship provides hands-on experience with large datasets, building data pipelines, and using tools for data processing and analysis. At our company, you wont just be learning the ropes you'll be contributing to projects that impact business decisions. Youll receive mentorship, training, and be an integral part of our journey.

Key Responsibilities:

• Assist in the design, implementation, and maintenance of data pipelines.

• Help with data ingestion, transformation, and integration tasks.

• Support the creation and optimization of databases, queries, and storage solutions.

• Collaborate with the team to ensure data is available and usable for analysis.

• Perform data cleaning, validation, and preprocessing.

• Write scripts and tools to automate data-related tasks.

• Troubleshoot and resolve data-related issues in the system.

• Document processes, procedures, and project progress for future reference.

Requirements:

• Currently pursuing a degree in Computer Science, Engineering, Mathematics, or a related field.

• Basic knowledge of SQL and database management.

• Familiarity with programming languages like Python, Java, or Scala.

• Understanding of data structures and algorithms.

• Exposure to cloud platforms (AWS, Azure, or Google Cloud) is a plus.

• Experience with managing and optimizing ETL (Extract, Transform, Load) processes.

• Strong analytical and problem-solving skills.

• Ability to work collaboratively in a team environment.

• Excellent communication skills and attention to detail.

Preferred Skills:

• Experience with data visualization tools like Power BI or Tableau.

• Familiarity with Big Data technologies such as Hadoop or Spark.

• Knowledge of version control tools like Git.

What Youll Gain:

• Hands-on experience with data engineering tools and platforms.

• Mentorship from experienced data engineers.

• Opportunity to work on real-world projects that impact business outcomes.

• Exposure to cutting-edge technologies and methodologies in data engineering.

• As a startup, we value your input. You'll be given the chance to contribute innovative ideas, which could help steer the direction of our data systems and business strategies. You'll work directly with leadership and have a unique opportunity to develop both professionally and personally.

Application Process:

Interested candidates should submit their resume and a cover letter outlining their interest and relevant coursework or project experience. We look forward to seeing how you can contribute to our company's growth and success while gaining invaluable hands-on experience.

Job Description

Job Description

We are looking for an experienced Junior Data Engineer for our client. This is a permanent position that is remote to start with later relocation to Calgary or Winnipeg . Our client is a global enterprise company with a product that you've likely used.

You Have:

• Minimum 2+ years professional working experience as a Data Engineer
• Strong Experience working with SQL and large data sets
• Hands-on experience coding in Python would be a strong asset
• Experience working with Business Intelligence and Data Warehousing
• Experience working with data services on the cloud such as AWS and/or Google Cloud Platform environments
• Experience with REST API's

Perks:

• Competitive Salary
• Individual performance bonus
• Health and dental benefits
• 3 weeks vacation
• Relocation Package
• RSP matching

If this opportunity sounds like something that fits your career path, please apply to this posting; we'd love to talk to you!

Job Description

Job Description

We are looking for an experienced Junior Data Engineer for our client. This is a permanent position that is remote to start with later relocation to Calgary or Winnipeg . Our client is a global enterprise company with a product that you've likely used.

You Have:

• Minimum 2+ years professional working experience as a Data Engineer
• Strong Experience working with SQL and large data sets
• Hands-on experience coding in Python would be a strong asset
• Experience working with Business Intelligence and Data Warehousing
• Experience working with data services on the cloud such as AWS and/or Google Cloud Platform environments
• Experience with REST API's

Perks:

• Competitive Salary
• Individual performance bonus
• Health and dental benefits
• 3 weeks vacation
• Relocation Package
• RSP matching

If this opportunity sounds like something that fits your career path, please apply to this posting; we'd love to talk to you!

Job Description

Job Description

We are looking for an experienced Senior Data Engineer for our client. This is a permanent position that is completely remote! Our client is a global enterprise company with a product that you've likely used.

Within this role, you'll have the chance to work on an application that is used by millions of users daily. With your expertise in BI, Data warehousing and big data you'll contribute to the expanding expertise of the team and keep data components current!

You Have:

• Minimum 5+ years professional working experience as a Data Engineer
• Strong experience working with both batch and real-time ETL involving large data sets
• Experience mentoring and guiding Junior Data Engineers in previous roles
• Strong Experience working with SQL and large data sets
• Hands-on experience coding in Python is required
• Experience working with Business Intelligence and Data Warehousing
• Experience working with data services on the cloud such as AWS and/or Google Cloud Platform environments
• Experience with REST API's

Some AWESOME selling points:

• Remote + flexible work schedule
• North of six figures salary + Equity + Health Spending Account (it's competitive)
• Amazing opportunity for career progression to lead
• Amazing mentorship (seriously)
• Plenty of vacay
• RRSP Matching
• Team budget for education for employees
  
  

It's a seriously awesome opportunity! If this looks interesting to you, go ahead and click apply :)

* Unfortunately we are only able to consider candidates who either live in Canada, or currently have an active Canadian work visa or citizenship for this role.

Job Description

Job Description

We are looking for an experienced Senior Data Engineer for our client. This is a permanent position that is completely remote! Our client is a global enterprise company with a product that you've likely used.

Within this role, you'll have the chance to work on an application that is used by millions of users daily. With your expertise in BI, Data warehousing and big data you'll contribute to the expanding expertise of the team and keep data components current!

You Have:

• Minimum 5+ years professional working experience as a Data Engineer
• Strong experience working with both batch and real-time ETL involving large data sets
• Experience mentoring and guiding Junior Data Engineers in previous roles
• Strong Experience working with SQL and large data sets
• Hands-on experience coding in Python is required
• Experience working with Business Intelligence and Data Warehousing
• Experience working with data services on the cloud such as AWS and/or Google Cloud Platform environments
• Experience with REST API's

Some AWESOME selling points:

• Remote + flexible work schedule
• North of six figures salary + Equity + Health Spending Account (it's competitive)
• Amazing opportunity for career progression to lead
• Amazing mentorship (seriously)
• Plenty of vacay
• RRSP Matching
• Team budget for education for employees
  
  

It's a seriously awesome opportunity! If this looks interesting to you, go ahead and click apply :)

* Unfortunately we are only able to consider candidates who either live in Canada, or currently have an active Canadian work visa or citizenship for this role.

Job Description

Job Description

We are looking for an experienced Junior Data Engineer for our client. This is a permanent position that is remote to start with later relocation to Calgary or Winnipeg . Our client is a global enterprise company with a product that you've likely used.

You Have:

• Minimum 2+ years professional working experience as a Data Engineer
• Strong Experience working with SQL and large data sets
• Hands-on experience coding in Python would be a strong asset
• Experience working with Business Intelligence and Data Warehousing
• Experience working with data services on the cloud such as AWS and/or Google Cloud Platform environments
• Experience with REST API's

Perks:

• Competitive Salary
• Individual performance bonus
• Health and dental benefits
• 3 weeks vacation
• Relocation Package
• RSP matching

If this opportunity sounds like something that fits your career path, please apply to this posting; we'd love to talk to you!