Distributed Perception Aware Safe Leader Follower System via Control Barrier Methods

Richie R. Suganda1, Tony Tran3, Miao Pan1, Lei Fan1,2, Qin Lin1,2, Bin Hu*,1,2
1Department of Electrical and Computer Engineering at University of Houston 2Department of Engineering Technology at University of Houston 3Cullen College of Engineering Research Computing at University of Houston
Paper arXiv Code
Cover Image

Abstract

This paper addresses a distributed leader-follower formation control problem for a group of agents, each using a body-fixed camera with a limited field of view (FOV) for state estimation.

The main challenge arises from the need to coordinate the agents' movements with their cameras' FOV to maintain visibility of the leader for accurate and reliable state estimation. To address this challenge, we propose a novel perception-aware distributed leader-follower safe control scheme that incorporates FOV limits as state constraints. A Control Barrier Function (CBF) based quadratic program is employed to ensure the forward invariance of a safety set defined by these constraints. Furthermore, new neural network based and double bounding boxes based estimators, combined with temporal filters, are developed to estimate system states directly from real-time image data, providing consistent performance across various environments.

Comparison results in the Gazebo simulator demonstrate the effectiveness and robustness of the proposed framework in two distinct environments. The benefits of proposed methods are also validated through experiments using Husarion ROSbot 2 Pro mobile robots.

Perception-aware CBF based Leader-follower Control

Block Diagram Image

We propose a novel perception-aware, distributed leader-follower control scheme that explicitly incorporates camera FOV limitations as state constraints. A CBF-QP framework is employed to enforce forward invariance of a safety set defined by these constraints. Deep Neural network (DNN) based and double bounding boxes (DBB) based estimators combined with temporal filters are introduced to obtain reliable estimates for leader-follower states across different environments.

DNN Angle Estimation

A DNN-based estimator, combined with a linear temporal filter, maps real-time image data to angle estimates by framing the estimation task as an image classification problem.

DNN Image

DBB Distance Estimation

The DBB method uses a CSRT tracker on the RGB image to generate a red bounding box that locates the leader, then scales it to half its dimensions on the depth image as a green bounding box to estimate the distance.

CSRT Image

Results from Gazebo Simulations

Simulation 1: Two-Robot Leader-follower Formation in Oval Track Environment

With CBF
(In-View Safe Tracking)
✅
Without CBF
(Out-of-View Crash)
❌

The first simulation evaluates the proposed perception-aware safe leader-follower control scheme in an oval track environment and is compared to a baseline that only applies the leader-follower formation controller. The system is evaluated in three stages: (i) an initial straight-line formation where the follower remains directly behind the leader, (ii) a formation transition where the follower moves to the side of the leader, introducing a conflict between control objectives and safety constraints, and (iii) a return to the initial straight-line formation. The proposed approach successfully resolve this conflict by adaptively activating the CBF-QP safety filter to maintain the leader within its FOV, while the baseline system fails to do so, resulting in the loss of the leader causing instability. These results demonstrate the control scheme's effectiveness in maintaining formation and safety constraints.

Simulation 2: Two-robot Leader-follower Formation in Residence Environment

With CBF
(In-View Safe Tracking)
✅
Without CBF
(Out-of-View Crash)
❌

The second simulation evaluates the generalization capability of the proposed perception-aware control scheme and the DNN-based estimator, the second experiment is conducted in an unseen outdoor residence environment. Despite being trained exclusively in the oval track setting, the estimator is able to generalize its environement effectively, providing an accurate estimation that enables the proposed controller to maintain desired formation and safety constraints. The baseline approach, fails to adapt to the environmental differences, leading to the loss of the leader causing instability. This simulation validates the robustness of the proposed estimator to ensure a robust and reliable leader-follower formation.

Simulation 3: Three-robot Leader-follower Formation in an Oval Track Environment

With CBF
(In-View Safe Tracking)
✅
Without CBF
(Out-of-View Crash)
❌

The third experiment evaluates the proposed control scheme in a three-robot leader-follower formation within the oval track environment. In this setting, the last robot is susceptible to the safety violations due to the error propagation and disturbances caused by the preceding robots. This demonstration shows that the robot not utilizing the proposed approach fails almost immediately in Stage 1. In contrast, the proposed approach successfully preserves the safety.

Experiments with Two ROSbot Pro Mobile Robots

With CBF
(In-View Safe Tracking)
✅
Without CBF
(Out-of-View Unsafe Tracking)
❌

Experiments using two ROSbot Pro 2 robots confirm that our perception-aware leader-follower control scheme consistently keeps the leader within the follower's field of view, ensuring safe, in-view tracking. In contrast, the traditional formation control strategy without CBF loses the leader, resulting in unsafe, out-of-view tracking. Note that these experiments employed an OptiTrack Motion Capture System for state estimation as a proof-of-concept, with future tests incorporating DNN and DBB-based estimators.

BibTeX

@inproceedings{suganda2024distributed,
  author    = {R. R. Suganda and T. Tran and M. Pan and L. Fan and Q. Lin and B. Hu},
  title     = {Distributed Perception Aware Safe Leader Follower System via Control Barrier Methods},
  booktitle = {Proc. IEEE Int. Conf. on Robotics and Automation (ICRA)},
  year      = {2025},
  address   = {Atlanta, US},
}