Shikun Ban

I am a final-year undergraduate at Peking University, advised by Prof. Yizhou Wang.

I am fortunate to have been introduced to academic research by Xiaoxuan Ma and Wentao Zhu at Center on Frontiers of Computing Studies, and I am grateful for the opportunity to collaborate closely with Prof. Yunzhu Li at Columbia University.

I am passionate about 3D Vision, Robotics, and Human-centered AI, particularly interested in crafting efficient & generalizable solutions for fundamental 3D vision tasks in embodied applications (e.g. robot pose estimation) while applying advances from vision community to enhance robot perception, manipulation, and human-robot interaction.

I have gained experience and am actively expanding my knowledge in these fields through comprehensive surveys and hands-on experiments.

Publications and Preprints

Real-time Holistic Robot Pose Estimation with Unknown States

Shikun Ban, Juling Fan, Xiaoxuan Ma, Wentao Zhu, Yu Qiao, and Yizhou Wang

In European Conference on Computer Vision (ECCV), 2024

Abs arXiv PDF Video Code Website

Estimating robot pose from RGB images is a crucial problem in computer vision and robotics. While previous methods have achieved promising performance, most of them presume full knowledge of robot internal states, e.g. ground-truth robot joint angles. However, this assumption is not always valid in practical situations. In real-world applications such as multi-robot collaboration or human-robot interaction, the robot joint states might not be shared or could be unreliable. On the other hand, existing approaches that estimate robot pose without joint state priors suffer from heavy computation burdens and thus cannot support real-time applications. This work introduces an efficient framework for real-time robot pose estimation from RGB images without requiring known robot states. Our method estimates camera-to-robot rotation, robot state parameters, keypoint locations, and root depth, employing a neural network module for each task to facilitate learning and sim-to-real transfer. Notably, it achieves inference in a single feed-forward pass without iterative optimization. Our approach offers a 12-time speed increase with state-of-the-art accuracy, enabling real-time holistic robot pose estimation for the first time.
Learning Human-aware Robot Policies for Adaptive Assistance

Jason Qin, Shikun Ban, Wentao Zhu, Yizhou Wang, and Samaras Dimitris

In arXiv preprint arXiv:2412.11913, 2024

Abs arXiv PDF Video Code Website

Developing robots that can assist humans efficiently, safely, and adaptively is crucial for real-world applications such as healthcare. While previous work often assumes a centralized system for co-optimizing human-robot interactions, we argue that real-world scenarios are much more complicated, as humans have individual preferences regarding how tasks are performed. Robots typically lack direct access to these implicit preferences. However, to provide effective assistance, robots must still be able to recognize and adapt to the individual needs and preferences of different users. To address these challenges, we propose a novel framework in which robots infer human intentions and reason about human utilities through interaction. Our approach features two critical modules: the anticipation module is a motion predictor that captures the spatial-temporal relationship between the robot agent and user agent, which contributes to predicting human behavior; the utility module infers the underlying human utility functions through progressive task demonstration sampling. Extensive experiments across various robot types and assistive tasks demonstrate that the proposed framework not only enhances task success and efficiency but also significantly improves user satisfaction, paving the way for more personalized and adaptive assistive robotic systems.