Senior Data Infrastructure Engineer (Scraping & Scale)

About the role

We are looking for a Senior Data Infrastructure Engineer to design and scale high‑performance web scraping and data ingestion systems. This fully remote position will build the core infrastructure that collects millions of social‑media profiles, posts and video records daily.

Key responsibilities

• Lead development and maintenance of scalable data collection pipelines for platforms such as Instagram, TikTok, YouTube and X.
• Design and manage web‑scraping infrastructure using residential proxies, TLS fingerprinting, browser automation and session rotation.
• Implement distributed queues and workflow orchestration with Temporal, Ray, Apache Kafka and Celery.
• Architect storage solutions using Parquet, Apache Iceberg, Snowflake and Amazon S3.
• Set up monitoring, alerts and recovery mechanisms to handle platform changes, blocking patterns and proxy failures.

Required profile

• At least 4 years of experience in high‑volume web scraping, data engineering or reverse engineering.
• Strong expertise in overcoming advanced anti‑bot systems (Cloudflare, Akamai, PerimeterX) using TLS impersonation, browser automation and proxy management.

Required skills

• Advanced proficiency in Python or Go.
• Experience with browser‑automation frameworks such as Playwright, Puppeteer, Scrapy or Selenium.
• Knowledge of distributed systems and task queues (Temporal, Ray, Apache Kafka, Redis).
• Strong SQL skills and experience with modern analytical data lakes and warehouses.
• Familiarity with storage formats and platforms: Parquet, Apache Iceberg, Snowflake, Amazon S3.

What we offer

• Fully remote work arrangement.
• Full‑time employment.