Developed joint angle estimation pipeline for Optimus Gen3 Hands. Using this data, performed system-id for Gen3 hands that was used in Robotaxi event. Designed a Probabilistic Roadmap planner for collision-free manipulation and whole-body control for enhanced safety. Improved C++ Manipulation Stack, supported event-critical tasks, and developed tracking and grasping algorithms for dynamic environments.
Specifics protected under NDA.

Joined JPL to work on Pose estimation of Lander for the Mars Sample Return Mission. I worked on Sample Recovery Helicopters in surface navigation subsystem, being part of Aerial Mobility Team. Designed the system from camera calibration to Pose estimation in C++. Successfully evaluated robustness tests under different stress conditions.
Additionally provided support for Stereo Visual Odometry algorithm.

Tried to develop a novel monocular event based visual inertial odometry using self supervised learning, namely contrast maximization. Engineered the training pipeline to run on multiple GPUs using Distributed Data Parallel (DDP) and deployed on SLURM cluster. Formulated a factor graph based backend using IMU preintegration and neural lie algebra constraints optimized by iSAM2 using GTSAM.
PS: Results were not impressive at all. But I implemented a lot of algorithms for event cameras and enjoyed working on this!

Developed a model zoo of lightweight architectures - LSTM, Modified Segformer Vision Transformer and CNNs for real-time pure visionbased obstacle avoidance, enabling the drone to autonomously navigate through dynamic obstacle fields at speeds of up to 7 m/s. [Paper]

Developed Sense and Avoid System (SAA) pipeline for Agricultural UAV where implemented LIDAR point cloud processing, obstacle avoidance and developed mavlink based support interface for lower level control. I was also the main Controls engineer for Indoor drone where successfully brought position accuracy to 5 cm in extreme low light conditions.

• Used Error State Kalman Filter to optimally combine SE3Tracknet and Classical Method based Pose estimation reducing angular error by 39% on YCB Video Dataset.
• Developed a modular and clean pipeline for SE3Tracknet removing bloatware in the orignal code while maintaining all functionality.
• Developed a novel Depth Simulator network that simulators real depth noise from synthetic depth resulting in better i.i.d approximation in training and inference data.

• This was a GTSAM learning Project! Formulated a factor graph using ICP based Odometry on Argo AI Driving Dataset.
• Performed incremental Smoothing and Mapping (iSAM) to optimize the odometry using GTSAM.

Description under development... See the code readme for more info!

•Developed and implemented an Operational Space Controller (OSC) for a 5-linked planar biped robot, formulating the problem as an instantaneous quadratic program (QP) to optimize actuator inputs and achieve precise tracking of the center of mass, torso angle, and swing foot position for stable walking.
• Leveraged the pydrake interface to construct the QP, with friction constraints, input limits, dynamic and contact constraints for robust biped locomotion.
• Implemented velocity and torso tracking objectives as quadratic cost, resulting in locomotion upto 0.7 m/s within actuator limits.

• Implemented Segformer's B0 architecture from scratch which mainly comprises of Attention Module, MixFFN layer and a MLP decoder.
• Trained on ADE20K dataset with weighted cross entropy loss to counter class imbalance and learning rate scheduler to speed up training.
• Achieved 0.45 mean IOU which is greater than 0.38 reported from official implementation. Implemented the same on CityScape dataset on a single sequence of 600 images.

• Depth estimation using Monocular Vision, by fusing information from multiple images of the scene at different angles.
• Utilized Pose-derived baselines to establish epipolar lines and performed NCC-based block matching for correspondence identification.
• Calculated depth using baselines, followed by integration into the Kalman Equation (only measurement update), resulting in enhanced depth fusion from multiple angled images.

• Applied Non Linear Bundle Adjustment using Horn Schunk Trick on the BAL dataset.
• Used Ceres and g2o for solving the problem.

The project aimed to achieve autonomous flight of a quadrotor in an obstacle map, given start and end coordinates.
• State estimation was performed using Stereo Visual Inertial Odometry with fusion using an Error State Kalman Filter.
• Path planning was implemented using A*, followed by generating a min snap trajectory through Ramer-Douglas-Peucker downsampling. To achieve rapid tracking, a cascaded non-linear geometric controller was implemented for the quadrotor.
• My algorithm achieved top 5 position in leaderboard in the class of size 100 based on speed, accuracy and robustness.

• Implemented a Quaternion based Unscented Kalman Filter(UKF) to track 3D orientation from Gyroscope and Accelerometer data
• Employed an optimization-based approach to process sigma points and estimate the state of the quaternion

• Integrated the inertial orientation and odometry with a 2D LIDAR scan to build the occupancy grid map of the environment while localising the robot using a particle filter

• Implementation of two-view stereo and multi-view stereo algorithms for dense 3d reconstruction of a scene.

• Estimated Camera Pose using PnP/P3P correspondence algorithm (without OpenCV). Backprojected the render mesh object to obtain an AR effect.
• Resolved Pose ambiguity problem by using depth as a criteria.

• Detected April tag pose using ROS wrapper and then used an least square optimization approach to refine the estimate.
• A bezier curve strategy was used to plan a path and then a velocity based PD controller was used to move the end effector.

• Developed a 12 state Kalman Filter that fuses MPU6050, BMP180 and Magnetometer, Neo6M GPS to get a reliable state estimate.
• Developed a gain scheduling based control strategy using adaptive gains based on state estimate.
• Used ESP32 as a flight controller, implemented RTOS to run State Estimator and Controller in different cores.

• Developed a physics based graphical Simulator for differential drives and quadrotors from scratch in python.
• Implemented Swarm algorithms for pattern formation using graph theory.
• Inbuilt path tracer, data logger and auto pid tuner.

Implemented an adaptive control strategy using Energy Shaping to reach the Equilibirum region and then using Linear Quadratic Regulator to stabilize.

Specialization in Vision and Controls.

Awarded Gold Medal and Award of Highest academic Excellence.