Mitanshu Goel is a robotics and Physical AI engineer based in Delhi, India. ECE graduate from MAIT (2026) with an AI/ML minor. At Nferent AI (Mar to Jun 2026) he built a bimanual VR teleoperation rig and robot-learning data pipeline: a Meta Quest 3 drives an Elite Robots CS66 pair and a Franka Research 3 over a real-time C++ control loop, a multi-sensor capture tool records MANUS gloves and RealSense cameras on one hardware clock, and the Franka rig recorded a 51-episode imitation-learning dataset exported to LeRobot format. He also wrote the control software for a Tesollo DG-5F dexterous hand that plays Rock-Paper-Scissors using a RealSense camera and a MediaPipe gesture classifier. Demo videos of the dual-arm teleoperation, the dexterous hand, and the SarthakAI voice assistant are embedded on this page. He has run three continued-pretraining experiments on a self-scraped Reddit corpus (a LoRA adapter on Mistral 7B, a QLoRA adapter on Qwen 2.5 through a hand-written distributed loop, and a small GPT from scratch). On an A.T.O.M. Robotics team hexapod he worked on the gait and inverse-kinematics control node and the ros2_control integration. He looks for Physical AI, Robotics SWE, ML Engineering, and Research Engineering roles.
Mitanshu Goel
Building robots, and the data and models that teach them.
Most recently at Nferent AI, where I built a bimanual VR teleoperation stack on real industrial arms and the pipelines that record imitation-learning data from it. The robot demos below are that work running. On the model side I train LLMs, from a small GPT written from scratch to continued-pretraining runs on Mistral and Qwen.
The robots I worked on, running. Each demo is explained in the experience section below.
- Dual-arm VR teleoperation. Built the stack that drives two Elite Robots CS66 arms from Meta Quest 3 controllers, a real-time C++ control loop per arm, written from scratch. It maps the operator’s hand motion to end-effector motion in the tool frame, smooths the noisy human input, and clamps sudden or near-singular moves so the arm cannot jerk. An anchor and clutch model lets the operator release, reposition, and re-grip without the arm jumping.
- DROID-style Franka teleop and a LeRobot dataset. Extended the teleoperation to a Franka Research 3 with a layered safety stack between the headset and the arm: a frame-jump guard rejects impossible VR readings, a leash keeps every command within a short reach of the arm’s measured pose, and a slew-rate limiter caps how fast the target can move. Used the rig to record a 10-task, 51-episode manipulation dataset, about 2.1 hours of two-camera RGB-D (color plus depth) video with synchronized robot state, exported to LeRobot format for imitation learning.
- Dexterous-hand Rock-Paper-Scissors. Wrote the control software that lets a Tesollo DG-5F five-finger hand, 20 motors over Modbus-TCP, play Rock-Paper-Scissors with a human. A RealSense camera and a MediaPipe gesture classifier read the operator’s hand at any orientation. Two fixes made the hand reliable: registering the vendor SDK’s callbacks before connecting, because the native library segfaults otherwise, and replacing the firmware’s unreliable arrived flag with a detector that waits for the joints to settle.
- MANUS multi-sensor capture. Built the capture tool that records two MANUS gloves and three RealSense cameras on a single hardware clock, with measured sync error under 15 milliseconds at the 95th percentile. A frame-uniqueness watchdog fails any episode in which a camera silently repeats a stale frame, so corrupted recordings never reach the training set. Recorded 9 manipulation tasks across 45 episodes, every one validation-passed.
- Diagnosed and patched a real-hardware crash in the robot’s URScript, a null handle on headless restart, that had been breaking teleop bring-up.
- Brought up and tested incoming commercial robots before dispatch: arms (Franka, Flexiv, Elite, Elephant Robotics), grippers (Robotiq, Tesollo, DH Robotics), and a Unitree quadruped.
- Built the PC-side system that turned a UBTech Yanshee humanoid into a voice and vision assistant for a logistics demo. A custom-trained YOLOv8 package detector ran over the robot's camera stream, and a NeMo speech-recognition (ASR) pipeline handled wake-word and command routing to either a chat service or a QR scanner.
- Designed resilient camera reconnection with exponential backoff and placeholder frames, so the web view never went blank when the stream dropped.
- Handled wake-word detection cheaply by matching ASR transcripts against about twenty phonetic spellings of the trigger phrase, instead of pulling in a separate wake-word engine.
- Built a sensor-network workstation that collected environmental data and ran predictive analysis on it to infer the state of the monitored equipment.
- Built a vision-driven detection and sorting line on a robotic arm and conveyor belt.
- Built MoveIt motion planning for a 6-DOF arm, validated first in simulation and then on the real hardware, focused on Cartesian-space planning for pick-and-place.
- Tracked down the URDF kinematic mismatches between simulation and hardware that were blocking stable trajectory execution.
- Hexapod. An 18-DoF hexapod built with ROS 2. I worked on the control side: the tripod-gait and analytic inverse-kinematics node, the ros2_control hardware interface, and the launch wiring. The URDF is CAD-exported. Team project, and an honest boundary with it: the hexapod walks in Gazebo and on a stand, and free-standing on the floor is the part that never got solved. GitHub ↗
- Web-controlled robotic arm. A browser interface that drives a robotic arm through a rosbridge WebSocket, with the arm's live camera feed streamed back into the UI.
- Represent the society at robotics competitions and hackathons, and run technical sessions for members.
- Minor: Artificial Intelligence & Machine Learning
- CGPA: 8.01 / 10
Three language models trained on a Reddit corpus I scraped and built the pipeline for: a 51M-parameter GPT trained from scratch, a LoRA adapter on Mistral 7B, and a QLoRA adapter on Qwen 2.5 that runs through a distributed training loop I wrote by hand, with token-offset sharding and checkpointing that resumes from the exact token count after a dropped cloud session. The from-scratch model is the measured win: perplexity 16.85 on held-out Reddit text against 24.68 for GPT-2, at under half the parameters. The half-trained Mistral adapter came out slightly worse than its base, and the repo reports it that way.
Hybrid retrieval over 20 industrial-safety PDFs: dense vectors plus BM25, a logistic-regression reranker, and a cross-encoder reference, on FastAPI. The real story is the evaluation. I caught that the original test set reused the training questions, so the headline number was the reranker grading its own homework. I rebuilt the eval around a disjoint held-out set with NDCG, MRR, and Recall@k, and once it was measured honestly the learned reranker did not clearly beat the plain hybrid baseline. The corrected eval is the result, not a leaderboard win.
A RAG system where hallucinations are structurally hard. If the model says it fully answered something but cites nothing to back it up, a Pydantic model-validator rejects the response at parse time, before it reaches the user. A retry loop feeds the parse failure back to the model. On a 9-question held-out rubric it returned 0 hallucinations, with 8 of 9 answerability calls correct.
A personal memory store that runs with no cloud dependency. Notes go in, and a local 4-bit Phi-3 model generates title, tags, and summary metadata at ingest, with regex fallbacks for when the model fumbles. Search is hybrid: dense vectors and keyword matches merged by best score. An ask endpoint answers questions over your own notes with retrieval, and an input guard strips chat-control tokens so a pasted note cannot inject instructions into the model.
Image generation treated like evolution. SDXL latent tensors are the genetic material: mutate them, cross two together, iterate across generations. A custom moment-preserving SLERP does the interpolation and restores the parent mean and variance, so a child latent stays on the noise manifold instead of decoding to grey mush. The diffusion loop is hand-written so bred latents can be fed straight in as initial noise.
A Cox proportional-hazards model for churn on about 10,000 Steam reviews, with risk signals pulled from the review text by an LLM. It reached a 0.874 hold-out C-index, but most of that lift came from features that quietly re-encode the outcome. So I decomposed it: separated the leaky polarity features from the forward-looking behaviour, kept the roughly +0.14 that actually predicts ahead of time, and added a contract test to block the leaky features from creeping back in. The smaller honest number is the one I report.
Batch retail analytics over 163 stores and five months. An XGBoost model forecasts next-month attach rate and is judged against persistence, last month carried forward, on a temporal holdout. It wins by a hair, 0.158 versus 0.160 RMSE, and the write-up reports exactly that margin instead of hiding the baseline. An earlier random split had leaked store identity between train and test, so the honest thin margin replaced a flattering wrong one. K-Means store segmentation and per-store rankings ride on the same pipeline.
A trader-behaviour study on 211k trades against the Bitcoin Fear and Greed index, with KMeans segmentation into trader archetypes. The honest centre of the repo is the post-mortem. A next-day classifier landed right at the base rate and caught only 2 of 44 actual loss days, and a follow-on volatility regressor came back worse than predicting the mean. I wrote up why it failed and stopped, instead of tuning a dead signal.
Do fundamentals explain annual returns for the five big Indian IT firms across seventeen years? Multivariate regression with Benjamini-Hochberg correction, leave-one-out cross-validation, and permutation tests says mostly no: no predictor survives correction, and that null is reported as the finding. What makes the null credible is a positive control, the method recovers the strong co-movement between the firms (correlations 0.84 to 0.94), so it does find real effects where they exist. Auditing the pipeline also caught a ticker-to-company mapping bug, now pinned by a test.
ROS 2 Humble, ros2_control, MoveIt, RViz, Gazebo.
Real-time C++ control loops, Cartesian servoing, inverse kinematics, SE(3) transforms.
VR teleoperation (Quest 3, OpenVR), RealSense RGB-D, multi-camera sync, LeRobot datasets, Elite CS SDK, Franka libfranka.
PyTorch, Unsloth, TRL, PEFT, LoRA, rsLoRA, BitsAndBytesConfig, Hugging Face Hub.
Mistral 7B, Qwen 2.5, nanoGPT, SDXL Lightning, Diffusers, NVIDIA NeMo, YOLOv8.
Sentence-Transformers, ChromaDB, llama.cpp.
lifelines (Cox PH), scikit-learn, XGBoost, K-Means, pandas, NumPy, SciPy, statsmodels.
F-tests, cross-validation, NDCG, MRR, Recall@k.
FastAPI, Uvicorn, SQLite FTS5, BM25.
Next.js 15, React 19, Firebase Firestore, TypeScript.
Docker, Linux, Git.
Python, C++, TypeScript, JavaScript, SQL, Bash.
Looking for full-time roles in Physical AI and Robotics SWE. Also open to research engineering and applied ML.