Skip to content

● India · open to remote & relocation

Tanmay Singh

Data Engineer · MLOps · AI systems

I build reliable data pipelines and the platforms around them, from 10M+ record banking systems to AI products.

01 / ingest

About

I'm a data engineer with a background in MLOps. At Capgemini I spent two and a half years on HSBC UK's application data management team, running Airflow pipelines over 10M+ customer credit records a month and cutting batch runtime by 40%.

After an MS in Data Science, Analytics and Engineering at Arizona State, I joined Vault IQ and worked across the stack on an AI brand-development platform: the Supabase data model, the backend and UI, and the agents that turn LLM and image-generation models into brand content and product shots.

Outside work I build things end to end, like CheckYourAQI, an air-quality forecaster that runs unattended on AWS, and Mettle, the workout tracker I'm building now.

02 / transform

Experience

  1. Dec 2025 – May 2026 · Remote

    Data Engineer (AI & Backend Systems)

    Vault IQ

    Full stack · data, backend and product UI

    • ›Worked across the stack, from the Supabase schemas for user, brand, product and social data (4+ modules) to the backend services and product UI built on them
    • ›Sped up the AI brand-generation pipeline, orchestrating OpenAI, Gemini and Flux Pro to produce brand content and product visuals
    • ›Built a photoshoot agent that takes generated product images and stages them in marketing-ready settings, from hero shots to styled scenes, while keeping the product true to the original
    • ›Built the production chatbot with a custom knowledge-base layer, plus the dashboard backend serving brand, catalogue and revenue metrics in real time, integrating 6+ external APIs
    • Supabase
    • PostgreSQL
    • OpenAI
    • Gemini
    • Flux Pro
  2. Aug 2023 – May 2025 · education

    MS, Data Science, Analytics and Engineering

    Arizona State University

  3. Oct 2020 – Mar 2023

    Data Engineer

    Capgemini

    HSBC UK · Application Data Management

    • ›Built and orchestrated Airflow ETL pipelines for HSBC's customer credit data systems
    • ›Processed 10M+ customer records a month across BigQuery and PostgreSQL environments
    • ›Cut batch pipeline runtime by 40% by optimising BigQuery queries and storage, and wrote SCD logic to meet ETL SLAs
    • ›Added PySpark validation and data-quality checks aligned with credit-risk models, plus monitoring that flags anomalies before they reach production
    • Airflow
    • BigQuery
    • PostgreSQL
    • Python
    • PySpark
    • SQL
  4. Aug 2016 – May 2020 · education

    BTech, Computer Science and Engineering

    Manipal University Jaipur

03 / serve

Projects

More on GitHub →

Tzing66/mettle

Mettle

● In development — not public yet

  • Mettle screenshot 1
  • Mettle screenshot 2
  • Mettle screenshot 3
  • Mettle screenshot 4

The idea

Most workout trackers are either fast but flat, or gamified but slow to log. Mettle tries to be both: logging comes first, the game second. Showing up earns steady XP, crossing real strength and running benchmarks earns a lot of it, and everyone climbs from E rank to S rank (six ranks, five levels each).

Benchmarks are relative, not absolute: they’re normalised for bodyweight, age and sex, so a 75 kg lifter benching bodyweight is compared fairly with a 120 kg one. The rules also avoid rewarding unhealthy behaviour: cardio XP has diminishing returns and rest days don’t break streaks.

How it works

  • Offline-first logging: workouts are written to SQLite on the phone (Drizzle ORM), because gyms often have bad reception. Signing in is optional; when you do, data syncs to Supabase.
  • One XP engine, two places: XP is derived from logged sets by a pure TypeScript engine. The same code runs in a Supabase Edge Function, so leaderboard XP is recomputed server-side and can’t be edited on the phone.
  • Real standards: 325 exercises (public-domain catalogue plus Mettle’s own), strength and rep standards, and WMA/USATF age-graded running tables decide benchmark tiers and your starting rank.
  • Social: friend groups with weekly leaderboards, profiles with goals, and routines you can share by code.
  • Security: Postgres row-level security on every cloud table, Google or email-code sign-in, and full account deletion.

Stack

React Native + Expo (TypeScript), expo-router, Reanimated, expo-sqlite + Drizzle, Zustand, Supabase (Postgres, Auth, RLS, Edge Functions), Jest with 220 tests across the engine, SQLite integration, sync and server.

Status

In active development and not public yet. Builds go to a small group of testers through EAS. If you’d like to try the beta, use the button below.

Tzing66/photobooth

Photobooth

● Try it live on this page ↓

  • Retro '70s booth: the front of the machine, with sample strips and the curtain
    Retro '70s booth: the front of the machine, with sample strips and the curtain
  • Darkroom theme, under the safelight
    Darkroom theme, under the safelight
  • Sketchbook theme, drawn with boiling hand-drawn lines
    Sketchbook theme, drawn with boiling hand-drawn lines
  • Pixel Arcade theme
    Pixel Arcade theme
  • Photo Boy handheld theme
    Photo Boy handheld theme

The idea

A vintage photobooth that lives in the browser. You step inside, strike four poses, and get back a developed photostrip, like the booths in old malls and train stations. You can use it in the Booth section of this page. Inspired by mysketchbooth.com.

How it works

  • Shoot or upload: four shots from a phone or laptop camera, each with a 3-2-1 countdown and a flash, or four uploaded photos cropped to 3:4.
  • Set up the shot: 16 frames (from Classic Black to Film Negative, Ticket Stub and Pressed Flowers) and backdrops: your own room, or a painted white wall, studio grey, black or velvet curtain, with a live preview of your own face.
  • Cut-out backdrops: a MediaPipe selfie segmenter, self-hosted and running on-device, separates you from the room so you can stand in front of a painted backdrop, with a flash shadow behind you.
  • The darkroom: a canvas pipeline develops the strip with flash glow, edge burn, black-and-white or film-colour toning, paper grain and dust. An “authentic” mode adds a harsh close-flash look: blown highlights, crushed blacks, halation and heavy grain.
  • Take it home: save it (straight to Photos via the share sheet on phones), share a 1080×1920 story image, print it on A4, or cut the strip into four single prints. Every strip you make is kept in a gallery in your browser.

Six booths

Retro ’70s, Sketchbook, Darkroom, Pixel Arcade, Photo Boy and Hello Kitty & Friends. Each theme is just decor: the pages own all the behaviour, so a new booth is one new file.

Privacy

Everything runs client-side. Photos are processed on your device and stored in your browser’s IndexedDB; nothing is ever uploaded.

Built with React, TypeScript, Vite, Tailwind CSS, Framer Motion, the Canvas API and MediaPipe.

Tzing66/checkyouraqi

CheckYourAQI

  • Dashboard: city average now, tomorrow's forecast and every station on a map
    Dashboard: city average now, tomorrow's forecast and every station on a map
  • Forecast accuracy page: live error against simple baselines
    Forecast accuracy page: live error against simple baselines
  • A station page during the OpenAQ outage, labelled as stale rather than hidden
    A station page during the OpenAQ outage, labelled as stale rather than hidden

The problem

Delhi’s air is among the worst of any major city, and it swings hard: PM2.5 can triple in a few days when crop-residue burning in Punjab and Haryana meets still winter air. Most public sites show what the air is like now. CheckYourAQI forecasts it for each of 70 government monitoring stations 24, 48 and 72 hours ahead, and is upfront about how accurate those forecasts are.

How it works

  • Ingest: hourly and daily pulls from OpenAQ (station PM2.5), Open-Meteo (weather and the CAMS air-quality model) and NASA FIRMS (satellite fire detections), orchestrated by Airflow on EC2.
  • Model: a bronze → silver → gold lakehouse on S3 and Iceberg, built with 27 dbt models on Athena and guarded by 44 data tests.
  • Forecast: one LightGBM model per horizon predicts every station every hour. Retraining runs weekly on GitHub Actions, and a challenger is promoted only if it beats production on data production has never seen.
  • Serve: a Streamlit dashboard and a FastAPI read API on Lambda, both reading a small public Parquet snapshot so neither touches AWS credentials.
  • Operate: Telegram alerts when a task fails or the data source goes down or recovers. The whole thing runs unattended for about $21 a month.

Results

Walk-forward validation over 14 months (647k training rows). Mean absolute error in µg/m³, lower is better:

Horizon LightGBM Persistence Same hour, last 7 days CAMS
24h 31.4 33.6 33.2 61.6
48h 33.8 38.5 34.1 62.6
72h 35.2 40.0 34.9 63.4

Design decisions

  • No leakage, enforced by a test. Features use only data available at prediction time, and a dbt test fails the build if any input post-dates the prediction. The same SQL macro builds training and live features.
  • UTC everywhere, IST only on screen.
  • Honest about stale data. Stations and the feed are classified live, delayed, inactive or outage, and forecasts built from stale inputs say so.

Tzing66/mesa-transit-expansion-emissions-analysis

Mesa Transit Expansion & Emissions

  • Census tracts with no bus service (dark), over Mesa's existing routes
    Census tracts with no bus service (dark), over Mesa's existing routes
  • Existing routes and stops
    Existing routes and stops
  • Five proposed routes and 45 stops covering the gaps
    Five proposed routes and 45 stops covering the gaps
  • 500 m walking buffers around the new routes, intersected with tracts
    500 m walking buffers around the new routes, intersected with tracts

Map data © OpenStreetMap contributors

Net CO₂ saved per year

Five proposed routes · 20 riders per bus · 12-mile average trip

  • Diesel bus
  • Electric bus
5% uptake
10% uptake
15% uptake
Data table
ScenarioDiesel busElectric bus
5% uptake5,1377,591
10% uptake10,27215,180
15% uptake15,40922,772

Unit: metric tons CO₂ / year. Average across all 24 modelled scenarios: 8,078 t CO₂ / year.

The problem

Large parts of Mesa, Arizona have no bus service at all. This graduate project at Arizona State University (FSE 570) asked where new routes would help most, and what they would do for emissions.

Method

  • Find the gaps: joined ACS population data to census tracts in QGIS and overlaid the existing network, which left 63,413 residents in tracts with no transit.
  • Design routes: proposed 5 new bus routes with 45 stops along major arterials, then used 500 m walking buffers intersected with tracts to check who each route actually reaches.
  • Cost it: brought the city’s transportation budget into the same pipeline for preliminary route-expansion and operating cost estimates.
  • Model emissions: compared diesel and electric buses across 5–15% ridership uptake, two trip lengths and two occupancy levels, using national per-mile emission factors.

Results

Across all 24 scenarios the routes save about 8,078 metric tons of CO₂ a year on average. Electric buses nearly double the benefit at high uptake: roughly 22,800 t a year at 15% uptake, versus 15,400 t for diesel (20 riders per bus, 12-mile trips).

Built with QGIS, Python, GeoPandas, pandas and Matplotlib on public data (ACS, OpenStreetMap, bus ridership records).

Tzing66/CheckYourData

CheckYourData

The problem

Writing data-quality checks by hand is tedious, so they often don’t get written. CheckYourData lets you upload a dataset and has an AI agent propose validation checks from its schema, which you review, edit and run.

How it works

  • Upload a CSV. The backend profiles its columns and stores the dataset.
  • Suggest: an agent sends the column schema plus a small sample to Claude, using forced tool-use so the response is always a structured list of check configs. Malformed suggestions are caught by validation and retried automatically.
  • Review: you approve or edit suggestions alongside preset checks. The agent never runs checks or saves anything itself.
  • Run and track: checks run against the data, and results are kept as history with drift over time.
  • Cache: suggestions are cached in Postgres keyed by a hash of schema + sample, so a repeat call returns in ~40ms instead of ~4s without re-hitting the API.

Architecture

A FastAPI backend with Postgres, a React + TypeScript frontend, and a single Docker service in production: FastAPI serves the built frontend from the same origin, and uploaded files live in Supabase Storage so they survive free-tier restarts.

Tzing66/Real-Time-Clickstream-Analytics-Pipeline-on-AWS

Real-Time Clickstream Analytics

The problem

Clickstream data arrives as a continuous stream of small events. This project builds a fully serverless pipeline on AWS that simulates that stream, lands it, transforms it and makes it queryable, with orchestration and alerting around it.

Pipeline

Lambda (simulate events)
  → Kinesis stream
  → Lambda (read stream, write raw JSON to S3)
  → Step Functions
      → Glue ETL (flatten geo fields, convert timestamps, partition by date)
      → Athena (MSCK REPAIR TABLE)
      → SNS (success / failure)

Services

  • Kinesis for real-time ingestion
  • Lambda, twice: one simulates events with faker, one reads from Kinesis and writes to S3
  • S3 for the raw and processed zones
  • Glue for cleaning and date partitioning
  • Athena to repair partition metadata and query the processed data
  • Step Functions to orchestrate the run, with SNS notifications on success or failure

Tzing66/Solar-Intensity-Model

Solar Intensity Model

Actual vs predicted daily solar output

Phoenix, 2006 · 37 held-out test days · linear regression on weather features

  • Actual
  • Predicted

Data table
DateActualPredicted
2006-01-044,825.84,686.6
2006-01-074,387.14,459.1
2006-01-105,454.65,045.8
2006-01-135,179.85,020.9
2006-01-143,622.24,150
2006-01-195,123.74,931.2
2006-01-275,415.75,326.3
2006-02-264,935.55,057.2
2006-03-044,412.45,132.4
2006-03-111,600.42,219.3
2006-04-177,883.27,304.4
2006-05-137,948.46,906.5
2006-05-146,374.16,936.1
2006-05-165,644.66,305.9
2006-05-207,416.66,983.2
2006-05-237,790.27,090.3
2006-06-244,982.44,100.7
2006-07-106,518.16,553.9
2006-07-235,882.55,429.5
2006-07-294,743.74,919.4
2006-08-015,613.36,054.4
2006-08-176,1156,521.2
2006-08-216,264.76,431.2
2006-08-296,554.56,354.7
2006-09-115,908.96,168.6
2006-09-146,041.95,828.1
2006-09-166,317.96,512.2
2006-09-216,898.26,440.8
2006-09-246,0496,129.8
2006-09-306,066.65,706.2
2006-10-016,6395,534.6
2006-10-144,021.45,357.5
2006-11-065,616.65,234
2006-11-125,004.24,959.4
2006-11-214,912.85,032.4
2006-12-085,099.75,178.1
2006-12-221,495.12,554.2

Unit: daily output, sum of 5-minute MW readings.

The problem

Predict how much power a solar plant in Phoenix, Arizona generates each day from the weather alone, and see whether automated feature selection beats simply using every feature.

Data

  • Daily weather (temperature, humidity, wind, solar radiation) for 2006 from the NASA POWER API.
  • Hours of daylight for the site, computed with pvlib.
  • Five-minute plant output from NREL, summed per day as the target.

Modelling

Three regression models, evaluated on the same 10% hold-out (37 days):

Model R² Avg. error
Linear regression, all features 0.825 9.3%
Polynomial (degree 2) 0.750 12.8%
Linear after RFECV feature selection 0.702 15.8%

Feature selection made things worse here: with only five weather features there was little noise to remove, and dropping any of them cost signal. A random-forest importance check agreed: solar radiation carries most of the signal (≈0.74), with daylight hours a distant second (≈0.12).

Tzing66/job_matching_agent

Job Matching Agent

The problem

Job boards return far more postings than anyone can read. This is a personal agent that fetches postings, filters them against my profiles, and has Claude score and rank the shortlist.

How it works

  • Fetch from the Adzuna API, with the country as a parameter and a configurable posting-age window applied server-side.
  • Search several keyword profiles independently, in a fixed priority order (Data Engineering first), so cross-profile duplicates resolve predictably.
  • Remember what’s already been seen in SQLite, keyed globally by job link, so each run only surfaces new postings.
  • Score each profile’s shortlist with one batched Claude call per run rather than one call per job, to keep token usage down.
  • Track results in a persistent Excel tracker that accumulates rows across runs, with an editable “Applied” column.

04 / compute

Stack

Languages

  • Python
  • SQL
  • TypeScript

Data & warehousing

  • PostgreSQL
  • BigQuery
  • Supabase
  • dbt

Pipelines & processing

  • Airflow
  • Spark
  • pandas

ML & AI

  • scikit-learn
  • LightGBM
  • Claude API
  • Gemini
  • OpenAI

Cloud & DevOps

  • AWS
  • Google Cloud
  • Docker
  • GitHub Actions
  • Git

Apps & APIs

  • FastAPI
  • React
  • React Native
  • Streamlit

05 / monitor

Now

Currently

  • ›Building Mettle
  • ›Grinding NeetCode 150
  • ›Reading up about something niche
  • ›Finding the next big breakthrough

GitHub activity

profile ↗

last push in Tzing66.github.io

0 contributions on 2026-04-260 contributions on 2026-04-270 contributions on 2026-04-280 contributions on 2026-04-290 contributions on 2026-04-300 contributions on 2026-05-010 contributions on 2026-05-020 contributions on 2026-05-030 contributions on 2026-05-040 contributions on 2026-05-050 contributions on 2026-05-060 contributions on 2026-05-070 contributions on 2026-05-080 contributions on 2026-05-090 contributions on 2026-05-100 contributions on 2026-05-110 contributions on 2026-05-120 contributions on 2026-05-130 contributions on 2026-05-140 contributions on 2026-05-150 contributions on 2026-05-160 contributions on 2026-05-170 contributions on 2026-05-180 contributions on 2026-05-190 contributions on 2026-05-200 contributions on 2026-05-210 contributions on 2026-05-220 contributions on 2026-05-230 contributions on 2026-05-240 contributions on 2026-05-250 contributions on 2026-05-260 contributions on 2026-05-270 contributions on 2026-05-280 contributions on 2026-05-290 contributions on 2026-05-300 contributions on 2026-05-310 contributions on 2026-06-010 contributions on 2026-06-020 contributions on 2026-06-030 contributions on 2026-06-040 contributions on 2026-06-050 contributions on 2026-06-060 contributions on 2026-06-070 contributions on 2026-06-080 contributions on 2026-06-090 contributions on 2026-06-100 contributions on 2026-06-110 contributions on 2026-06-120 contributions on 2026-06-130 contributions on 2026-06-140 contributions on 2026-06-150 contributions on 2026-06-160 contributions on 2026-06-170 contributions on 2026-06-180 contributions on 2026-06-190 contributions on 2026-06-200 contributions on 2026-06-210 contributions on 2026-06-220 contributions on 2026-06-230 contributions on 2026-06-240 contributions on 2026-06-250 contributions on 2026-06-260 contributions on 2026-06-270 contributions on 2026-06-280 contributions on 2026-06-290 contributions on 2026-06-300 contributions on 2026-07-010 contributions on 2026-07-020 contributions on 2026-07-030 contributions on 2026-07-040 contributions on 2026-07-050 contributions on 2026-07-060 contributions on 2026-07-070 contributions on 2026-07-080 contributions on 2026-07-090 contributions on 2026-07-100 contributions on 2026-07-110 contributions on 2026-07-120 contributions on 2026-07-130 contributions on 2026-07-140 contributions on 2026-07-150 contributions on 2026-07-160 contributions on 2026-07-170 contributions on 2026-07-180 contributions on 2026-07-190 contributions on 2026-07-200 contributions on 2026-07-210 contributions on 2026-07-220 contributions on 2026-07-230 contributions on 2026-07-240 contributions on 2026-07-250 contributions on 2026-07-260 contributions on 2026-07-270 contributions on 2026-07-280 contributions on 2026-07-290 contributions on 2026-07-300 contributions on 2026-07-310 contributions on 2026-08-010 contributions on 2026-08-020 contributions on 2026-08-030 contributions on 2026-08-040 contributions on 2026-08-050 contributions on 2026-08-060 contributions on 2026-08-070 contributions on 2026-08-080 contributions on 2026-08-090 contributions on 2026-08-100 contributions on 2026-08-110 contributions on 2026-08-120 contributions on 2026-08-130 contributions on 2026-08-140 contributions on 2026-08-150 contributions on 2026-08-160 contributions on 2026-08-173 contributions on 2026-08-180 contributions on 2026-08-190 contributions on 2026-08-200 contributions on 2026-08-210 contributions on 2026-08-220 contributions on 2026-08-233 contributions on 2026-08-243 contributions on 2026-08-250 contributions on 2026-08-260 contributions on 2026-08-270 contributions on 2026-08-280 contributions on 2026-08-290 contributions on 2026-08-300 contributions on 2026-08-310 contributions on 2026-09-010 contributions on 2026-09-020 contributions on 2026-09-030 contributions on 2026-09-040 contributions on 2026-09-050 contributions on 2026-09-060 contributions on 2026-09-070 contributions on 2026-09-080 contributions on 2026-09-090 contributions on 2026-09-101 contribution on 2026-09-110 contributions on 2026-09-120 contributions on 2026-09-130 contributions on 2026-09-145 contributions on 2026-09-150 contributions on 2026-09-167 contributions on 2026-09-172 contributions on 2026-09-180 contributions on 2026-09-190 contributions on 2026-09-203 contributions on 2026-09-210 contributions on 2026-09-226 contributions on 2026-09-232 contributions on 2026-09-242 contributions on 2026-09-250 contributions on 2026-09-264 contributions on 2026-09-2718 contributions on 2026-09-2810 contributions on 2026-09-290 contributions on 2026-09-3010 contributions on 2026-10-010 contributions on 2026-10-023 contributions on 2026-10-0311 contributions on 2026-10-040 contributions on 2026-10-058 contributions on 2026-10-0627 contributions on 2026-10-0711 contributions on 2026-10-080 contributions on 2026-10-090 contributions on 2026-10-100 contributions on 2026-10-118 contributions on 2026-10-1210 contributions on 2026-10-1353 contributions on 2026-10-143 contributions on 2026-10-150 contributions on 2026-10-160 contributions on 2026-10-170 contributions on 2026-10-1820 contributions on 2026-10-19
233 contributions · 26 weeks248 in the last year

06 / play

Booth

photobooth · ready

The Photobooth's retro '70s booth front

I was tired of finding photobooths in my own country, so I made one for myself. It is just a silly little project I whipped up on a Saturday afternoon. Try it!

Everything happens in your browser. Your photos never leave your device.

07 / sink

Contact

Hiring for a data or ML platform role, or just want to talk pipelines? Reach out.

Open to remote roles, or relocating internationally with visa sponsorship.

esc
↑↓ to move · ↵ to select