Software Guidance & Assistance

Big Data QA Engineer

  • Job Type: Contract W2
  • Industry Type: IT Sector
  • Industry Location: San Diego
  • Experience: NA
  • No. of Positions: 1
  • Primary Skills: Software QA Engineer Engineering
  • Secondary Skills: Python XML Linux
  • Job Location: San Diego, California
  • Posted Date: Posted today
Job Description

Software Guidance & Assistance, Inc., (SGA), is searching for an Big Data QA Engineer for a Contract assignment with one of our premier Information Technology Services clients in San Diego, CA.

Responsibilities :

  • The Data Engineering team has a need for a Big Data QA engineering professional who can assist with important test automation and QA tasks associated with mission critical projects.
  • This engineer will work closely with the team's primary test engineer and the broader team to align cohesively with ongoing projects.
  • Primary focus will be on QA test automation.
  • Will define a test automation strategy for new data pipelines or modify existing strategy to expand QA coverage of an existing pipeline.
  • Create data pipeline, specific input datasets, and expected datasets to implement QA automation.
  • Modify existing input datasets and expected datasets when business requirements for existing data pipelines change.
  • Create or modify Python scripts for triggering data pipeline specific QA tests and validating against the expected outputs.
  • Integrate with Jenkins CI/CD automation to run nightly QA tests automatically.
  • Perform manual testing where automation is not feasible, or QA tests need to be run on ad-hoc basis.



Required Skills:

  • The role requires an engineer who is data savvy and has an overall system quality mindset.
  • Good understanding of basics of probability and statistics. For example, ability to create representative smaller samples from larger datasets by applying various selection methods. Should know what a Normal distribution is and how to prove the randomness of a data sample. Able to apply descriptive statistics for data validation, troubleshooting.
  • Advanced Python scripting skills are must. For example, ability to work with ease using various data formats such as CSV, JSON, XML in Python and create modular/re-usable code. Should be able to write OO code and understand list comprehensions in Python.
  • Comfortable working in Linux environment with bash, grep, awk, ssh, xargs etc.
  • Critically think about corner cases in data pipelines and create test cases to simulate those conditions. Should understand the significance of test coverage. Should understand fault injection.
  • Understands the problems associated with processing large datasets (10's of TB) and is conceptually familiar with technologies available to solve those problems.
  • Not expected to know Hadoop or Spark but would-be a plus.



SGA is a Certified Women's Business Enterprise (WBE) celebrating over thirty years of service to our national client base for both permanent placement and consulting opportunities. For consulting positions, we offer a variety of benefit options including but not limited to health & dental insurance, paid vacation, timely payment via direct deposit. SGA accepts transfers of H1 sponsorship for most contracting roles. We are unable to sponsor for Right-to-Hire, Fulltime, or Government roles. All parties authorized to work in the US are encouraged to apply for all roles. Only those authorized to work for government entities will be considered for government roles. Please inquire about our referral program if you would like to submit a candidate for any of our open or future job opportunities. SGA is an EEO employer. We encourage Veterans to apply. To view all of our available job postings and/or to learn more about SGA please visit us online at www.sgainc.com .

Relevant Skill Jobs
System Analysts and System Administrators