~

ML / AI Platform Engineer (SR I / II)

Vishnu Keshav Senthil

I build the platform behind production AI: low-latency Rust services over gRPC and HTTP, model serving and LLM gateway infrastructure, agent memory and orchestration, and the observability layer teams depend on. I came up modeling production systems as a data scientist, which is why I build platforms for the people who actually run models in production, backed by Postgres, ClickHouse, Bigtable, Kafka, Kubernetes, and whatever else the job needs.

Rust platform services

Low-latency gRPC and HTTP services in Rust for feature serving, agent memory, and transcription. Tokio, Postgres, and protobuf contracts shared with every client.

Model serving & LLM gateway

A governed LLM gateway with routing, budgets, and audit, plus a model registry and serving path the whole company ships on. Built for real inference traffic.

Observability & reliability

Tracing, metrics, and evals across the stack with Prometheus, OpenTelemetry, Datadog, and ClickHouse. Autoscaling, health probes, and schema guardrails that keep it boring.

Core stack

  • Rust
  • Tokio
  • gRPC (unary, REST mapping)
  • HTTP APIs
  • Protobuf
  • Distributed systems
  • PostgreSQL
  • ClickHouse
  • ClickHouse Keeper
  • Valkey
  • Bigtable
  • Kafka
  • Snowflake
  • Docker
  • Kubernetes
  • GCS
  • Prometheus
  • OpenTelemetry
  • Datadog
  • MLflow
  • Langfuse
  • OpsML
  • Scouter
  • Promptfoo
  • LiteLLM
  • Google ADK
  • LangGraph
  • A2A
  • MCP
  • Python
  • FastAPI
  • PyTorch
  • Operations research
  • Linear programming
  • Discrete-event simulation
  • CPLEX
  • Arena

Experience

ML / AI Platform Engineer (SR I / II)

Shipt / Platform

  • Build low-latency Rust services for agent memory and feature serving, exposing the same unary API over gRPC and HTTP from one protobuf contract, backed by Postgres, Bigtable, Kafka, and Snowflake.
  • Run shared platform infrastructure as horizontally scalable services: the model registry autoscales 2 to 10 pods on Kubernetes with health probes, artifact proxying, and zero-credential client access.
  • Own the observability and reliability layer: Langfuse tracing across web and worker roles on a 3-node ClickHouse backend, Prometheus and Datadog APM export, and Promptfoo eval gates in CI.
  • Operate the LLM gateway, turning a LiteLLM proxy into a governed gateway with routing, budgets, audit logs, guardrails, and multi-modal support.
  • Build agent orchestration foundations on Google ADK, LangGraph, A2A, and MCP: durable sessions and tasks plus reference templates other teams build on.
  • Make the operational calls that keep platform services boring: explicit schema-migration guardrails, autoscaling policies, and catching a runaway GenAI job queue before it shipped.
Oct 2023 - Present

Senior Data Scientist

Shipt / Target Last Mile Delivery

  • Shipped the first model trusted to automate last-mile surge pay in production, owning it from training through serving.
  • Rebuilt proof-of-delivery package detection on an owned Mask R-CNN stack with a SAM2-assisted Label Studio workflow, replacing a black-box detector.
  • Led the Target Last Mile geocode-correction launch, validating corrected pins against proof-of-delivery evidence and sizing the annual order exposure from bad source pins.
  • Built Insights Dashboard, the first visual operations platform for pay, store coverage, surge, order density, and driver supply.
  • Improved dispatch-time measurement with geo-exit signals, cutting geofence inflation out of the metric and filtering unreliable sessions before launch readout.
Apr 2022 - Oct 2023

Self-hosted infrastructure

Personal projects

  • Built Remora, a self-hosted Rust memory service that gives coding agents and local agents one namespaced memory plane over HTTP and MCP, with about 25 ms p50 recall.
  • Built lobsterbox and lobsterfleet, a Rust self-hosted remote-dev stack with Docker and Hetzner leases, browser terminal and VNC, and GitHub auth.
  • Run a multi-node homelab: a Proxmox cluster, Docker workloads, segmented networking, resilient DNS, and GPU-backed agent memory.

Research Associate (I, II)

Evidera (PPD / Thermo Fisher Scientific)

  • Ran statistical analysis and built data pipelines in Python and R for health-economics and outcomes-research studies.
  • Worked with large patient-level and claims datasets to support cost-effectiveness and real-world-evidence models.
Nov 2019 - Apr 2022

Research Assistant

The Ohio State University

  • Built a Python app that pulled trailer data and assigned incoming trailers to strip doors for a cross-dock scheduling window.
  • Formulated trailer-door assignment as a linear program, compared neighbor search, exhaustive, flow-shop, and branch-and-bound approaches, and built discrete-event simulations to study throughput.
Jan 2018 - Sep 2019

Selected Impact

Feature store rewrite

Reworked the shared feature store in Rust with gRPC, Bigtable, Kafka, and Snowflake paths, cutting read latency roughly in half.

LLM gateway hardening

Turned a LiteLLM proxy into a governed gateway with routing, budgets, audit logs, guardrails, and multi-modal support.

MLflow registry

Own MLflow registry infrastructure with Cloud SQL, GCS artifact proxying, 2-10 pod autoscaling, Prometheus metrics, Datadog APM, and schema upgrade guardrails.

Langfuse eval observability

Built the Langfuse platform path with web and worker roles, a 3-node ClickHouse backend, Prometheus export, and Promptfoo route-agent e2e checks in CI.

Agent platform foundation

Built durable memory and task storage for agents over gRPC and HTTP, then wired it into templates and reference systems for production use.

Self-hosted agent memory

Built Remora, a Rust memory service with scoped namespaces, HTTP and MCP access, hybrid recall, active forgetting, and about 25 ms p50 recall latency.

Self-hosted remote dev stack

Built lobsterbox and lobsterfleet, a Rust broker and Rust/WASM fleet UI for Docker or Hetzner leases, browser terminal/VNC, and GitHub auth.

Production-style homelab

Run a segmented Proxmox and Docker homelab with resilient DNS, identity, LLM access, and GPU-backed memory for home agents.

Surge pay automation

Shipped the first surge pay model trusted to automate last-mile delivery pay in production, owned end to end.

Computer vision rebuild

Moved proof-of-delivery detection onto an owned Mask R-CNN stack with a SAM2-assisted labeling flow and better recall.

Education

2019

The Ohio State University

M.S. Operations Research

2017

Anna University

B.E. Mechanical Engineering

Earlier Engineering Work

  • Cross-dock trailer scheduling with Python, linear programming, neighbor search, and discrete event simulation
  • Windfarm maintenance Markov model with a 20% modeled cost reduction
  • Courtside Cafe Arena simulation with 16% lower modeled waiting time and at least $288/day in savings
  • Census income classification with sampling strategies and 0.85 F1 from random forest with oversampling
  • Time-series forecasting comparison across SARIMA and a dilated convolutional neural network
  • BEML aluminum hangar simulation that found bottlenecks and showed a 6% throughput gain
  • Ford India CMM shop variability analysis and seat recliner measurement prototype
  • Ashok Leyland-funded hybrid composite bus flooring
  • Go-kart chassis design and procurement
  • Automatic seatbelt prototype