Shikun Ban

I am a final-year undergraduate at Peking University, advised by Prof. Yizhou Wang.

I am fortunate to have been introduced to academic research by Xiaoxuan Ma and Wentao Zhu at Center on Frontiers of Computing Studies, and I am grateful for the opportunity to collaborate closely with Prof. Yunzhu Li at Columbia University.

I am passionate about 3D Vision, Robotics, and Human-centered AI, particularly interested in crafting efficient & generalizable solutions for fundamental 3D vision tasks in embodied applications (e.g. robot pose estimation) while applying advances from vision community to enhance robot perception, manipulation, and human-robot interaction.

I have gained experience and am actively expanding my knowledge in these fields through comprehensive surveys and hands-on experiments.

Publications

Real-time Holistic Robot Pose Estimation with Unknown States

Shikun Ban, Juling Fan, Xiaoxuan Ma, Wentao Zhu, Yu Qiao, and Yizhou Wang

In European Conference on Computer Vision (ECCV), 2024

Abs arXiv PDF Video Code Website

Estimating robot pose from RGB images is a crucial problem in computer vision and robotics. While previous methods have achieved promising performance, most of them presume full knowledge of robot internal states, e.g. ground-truth robot joint angles. However, this assumption is not always valid in practical situations. In real-world applications such as multi-robot collaboration or human-robot interaction, the robot joint states might not be shared or could be unreliable. On the other hand, existing approaches that estimate robot pose without joint state priors suffer from heavy computation burdens and thus cannot support real-time applications. This work introduces an efficient framework for real-time robot pose estimation from RGB images without requiring known robot states. Our method estimates camera-to-robot rotation, robot state parameters, keypoint locations, and root depth, employing a neural network module for each task to facilitate learning and sim-to-real transfer. Notably, it achieves inference in a single feed-forward pass without iterative optimization. Our approach offers a 12-time speed increase with state-of-the-art accuracy, enabling real-time holistic robot pose estimation for the first time.
Learning Human-aware Robot Policies for Adaptive Assistance

Jason Qin, Shikun Ban, Wentao Zhu, Yizhou Wang, and Dimitris Samaras

In ArXiv Preprint, 2024