Senior Web Data Engineer (Remote) Job at MAP SSG, Remote

NDJJVU9ETWRYL09XNEkyUnl3dE1RSnZv
  • MAP SSG
  • Remote

Job Description

Summary

We need someone with 4+ years of experience in web data engineering, API integration, or similar roles, who has strong experience with Python libraries (requests, web scraping frameworks (Scrapy, BeautifulSoup, Selenium), and modern browser automation tools (Playwright, Puppeteer). You should be comfortable designing and maintaining robust data collection pipelines from public APIs and web sources, and have a deep understanding of web technologies like HTML, CSS, JavaScript, REST APIs, and GraphQL. You should have up-to-date knowledge of proxy management, session handling, and TLS techniques. Bonus points if you have experience with Rust and/or Go web scraping frameworks for performance, and familiarity with distributed systems and job queues.

What you'll do :
  • Design and maintain robust data collection pipelines from public APIs and web sources.
  • Build and optimize web scraping systems for publicly accessible content.
  • Transform and normalize collected data into structured formats for downstream processing.
  • Monitor data quality, collection performance, and compliance with platform guidelines.
Responsibilities :
  • Design and maintain robust data collection pipelines from public APIs and web sources.
  • Build and optimize web scraping systems for publicly accessible content.
  • Transform and normalize collected data into structured formats for downstream processing.
  • Monitor data quality, collection performance, and compliance with platform guidelines.

Requirements :

  • Either or both of
    • Strong experience with Python libraries (requests, web scraping frameworks (Scrapy, BeautifulSoup, Selenium), modern browser automation tools (Playwright, Puppeteer)
    • Strong experience with Rust and/or Go web scraping frameworks for performance
  • Deep understanding of web technologies: HTML, CSS, JavaScript, REST APIs, GraphQL
  • Understanding of network protocols, DNS, and web infrastructure
  • Familiarity with data formats and serialization (JSON, Parquet, Protocol Buffers)
  • Knowledge of proxy management, session handling, and TLS techniques
  • Strong background in data validation, cleaning, and transformation pipelines
  • 4+ years of experience in web data engineering, API integration, or similar roles
  • Experience working in cross-functional teams with strong communication skills

Bonus requirements :

  • BS/MS in Computer Science, Engineering, or equivalent experience
  • Experience with distributed systems and job queues (Redis, Celery, Amazon SQS)
  • Knowledge of web scraping practices and compliance frameworks
  • Familiarity with Kubernetes for scalable data collection
  • Experience with database systems for storing and indexing collected data
  • Public GitHub repositories demonstrating web scraping or API integration projects

Benefits :

  • Medical, Dental, Vision, Life Insurance, STD and LTD Plans
  • FSA - Medical and Dependent Care
  • EAP and wellness programs
  • Productivity stipends
  • 13 Paid Holidays
  • Unlimited PTO
  • Flexible work environment - 100% remote
  • Bi-annual company/team meetups
  • 401(k) plan with employer matching contributions
  • Annual review for salary raises
Salary
$150-200K

Job Tags

Remote work, Flexible hours,

Similar Jobs

The World

1st Electrical Engineer Job at The World

The World, Residences at Sea, is the finest private mega yacht at sea.We offer the best of the best at every level of our organization. Employees of The World strive for excellence in everything we do. The primary responsibility of the 1st Electrical Engineer is the... 

Talent Masters Staffing

Machinist II Job at Talent Masters Staffing

 ...Job Summary: Machinist II position requires a solid understanding on CNC operations, the ability to perform complex setups, troubleshoot issues and ensure quality standards are met. Duties: Set up, adjust, and operate machine tools and many specialized or advanced... 

Amazon.com Services LLC

Lead Fulfillment Associate Job at Amazon.com Services LLC

 ...DESCRIPTION Lead Fulfillment Associate The Amazon operations team is looking for Lead Fulfillment Associates with a strong record...  ...receive/stow product, ensure inventory accuracy, and unload/load trucks. Lead Fulfillment Associates will also be responsible for... 

Providence Health and Services

Part-Time Associate Medical Assistant Job at Providence Health and Services

 ...members of the patient care team in our clinics. We currently have Part-time roles available in the following locations: ~ Centralized...  ...Forces Or Coursework/Training and minimum one year of work experience as a medical assistant that includes the following clinical... 

Laminar

Machine Learning (ML) Applications Engineer- Chemical/Process Engineering Job at Laminar

Machine Learning (ML) Applications Engineer- Chemical/Process Engineering At Laminar (formerly H2Ok Innovations), we're leading the charge in cleantech innovation, reshaping process industrials and manufacturing to drive operational efficiency and sustainability for...