Senior Data Engineer

About the role

As a Senior Data Engineer you will own the pipelines and platform that move Nerdy’s data from source systems into a governed lakehouse. You will build streaming and batch ingestion, operate the Iceberg lakehouse on Starburst Galaxy, and deliver reliable, well‑tested dbt layers for analytics engineers and analysts, leveraging AI to accelerate development, QA, and operations.

Key responsibilities

• Build and run streaming and batch ingestion from Kafka and third‑party APIs into Iceberg tables, handling CDC, schema evolution and late‑arriving data.
• Operate the Iceberg lakehouse on Starburst Galaxy (Trino), managing table layout, partitioning, compaction, retention and cluster sizing.
• Develop bronze, silver and gold layers with dbt, ensuring incremental models, idempotent merges and safe backfills.
• Design dbt Cloud job schedules, dependencies, retries and alerting; maintain freshness SLAs and drive incidents to root cause.
• Implement data quality tests, contracts, freshness monitoring, lineage and access controls, including PII handling.
• Optimize query performance and cost; manage infrastructure as code with Terraform and CI/CD.
• Collaborate with analytics, product and business teams to turn source changes into dependable data products.

Required profile

• Bachelor’s degree in Computer Science, Engineering, Math/Stats or equivalent experience.
• 3+ years as a Data Engineer in high‑growth environments.
• Expert SQL and strong Python skills; experience building unattended production pipelines.
• Hands‑on with open table formats and lakehouse engines (Iceberg on Trino/Starburst Galaxy, Delta Lake, Hudi, Spark, Databricks, Presto/Athena).
• Streaming experience with Kafka/Confluent or similar (Kinesis, Redpanda), including schema registry, CDC (e.g., Debezium) and idempotent loading.
• Proficiency with dbt, dbt Cloud, Airflow or Dagster, and solid AWS experience (S3, IAM, Glue, catalog).
• Experience with infrastructure as code (Terraform) and Git‑based CI/CD.

Required skills

• SQL
• Python
• Iceberg, Trino, Starburst Galaxy
• Kafka, Confluent, Kinesis, Debezium
• dbt, dbt Cloud
• Airflow, Dagster
• AWS (S3, IAM, Glue)
• Terraform
• Git, CI/CD
• AI‑native productivity tools (Cursor, Make, Supabase, Netlify, Claude Code, n8n, Firecrawl, ChatGPT, Grok, Bolt, Vercel)