Data Engineer (Detect & Track Distillation)

About the role

Harmattan AI, a defense prime building autonomous and scalable defense systems, is hiring a Data Engineer to own the data layer feeding its machine learning teams. Working out of Paris, Lausanne or Zurich, you will turn terabytes of raw, unstructured data into clean, documented, versioned datasets so modelers can focus on training. You join at an early stage with real influence over how field and public data is processed.

Key responsibilities

• Ingest, decode and store raw, unstructured field data (video and other sensor streams) from field logs into efficient, controlled formats.
• Align multiple data streams temporally and spatially so paired data is usable for training.
• Curate ingested data and public datasets: parsing, filtering, de-duplication and revision.
• Define what data to gather and how to label it, and own the dataset-construction workflow and labeling tooling, coordinating with data-gathering and labeling teams.
• Build task-specific datasets for training and evaluation with acquisition, annotation and product teams.
• Version datasets and maintain lineage so training runs stay reproducible.
• Store data in efficient formats such as columnar or sharded ones, and manage storage tiering to balance cost and latency.
• Deliver clean, documented datasets and the tools needed to load them, keeping training from being I/O-bound.

Required profile

• Degree in a STEM field, or equivalent practical experience; practical data engineering experience matters more than the degree.
• Experience building and maintaining pipelines for unstructured data at scale (video, images or sensor data), covering ingestion, decoding, storage, curation and versioning.
• Strong Python and data engineering skills, and comfort optimizing data loaders for common training frameworks such as PyTorch.
• Bonus: multimodal sensor data, labeling or dataset construction for ML, dataset versioning tooling and distributed data processing tooling.
• Systematic, quality-minded, pragmatic and service-oriented, with a knack for automating the handling of messy data.
• Full commitment to the company's mission of providing a defensive edge to allied nations through ethical, high-impact technology.