IEEE CDC 2026 · to appear

Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing

A closed-form adversary for decentralized separation policies

Alex Zongo, Filippos Fotiadis, Ufuk Topcu, Peng Wei

The George Washington University · The University of Texas at Austin, Oden Institute

Preprint Poster Cite

What if the positions a policy sees are wrong?

In cooperative surveillance, every aircraft broadcasts its GPS-derived position. When those broadcasts are corrupted, the whole observed traffic picture becomes unreliable. We train separation policies that stay safe under a worst-case corruption that we can compute in closed form, without adversarial co-training.

Robust separation under GPS spoofing (schematic)Two aircraft converge on a merge point. Aircraft B's broadcast position is corrupted inside a bounded box; aircraft A, running the robust policy, slows and keeps its separation buffer. merge B A robust policy true position spoofed report bound κ
Schematic. Aircraft B's broadcast position is spoofed inside a bounded set (dashed box); aircraft A, running the robust policy, slows and keeps its separation buffer.
NMACs up to 35% corruption
≈ 0
worst-case observation, no adversary training
One shot
accuracy of the closed-form worst case
2nd order

Simulation results averaged over 100 episodes in BlueSky with Amazon MK30 performance parameters, en-route package-delivery airspace. Not a flight-certified guarantee.

01 / The problem

Separation depends on a broadcast you cannot fully trust.

Each aircraft observes itself and up to five nearby intruders (position, speed, heading) within a 500 m detection radius, and a shared decentralized policy selects a speed change: slow down, hold, or speed up. GPS multipath, jamming, or spoofing corrupts those broadcasts, so the policy acts on a traffic picture that may not exist.

Local observation model and the effect of a corrupted broadcastAn ownship observes intruders within its detection radius. Their broadcast states form the policy input. When intruder 2's broadcast is corrupted, its reported position jumps and the policy selects a different speed change. detection radius (500 m) ownship local observation own: pos, speed, hdg intruder 1 intruder 2 intruder 2 · corrupted intruder 3 … up to 5 intruders π(· | o) speed change −δv 0 +δv a corrupted input changes the decision
Each aircraft observes intruders within 500 m. When one broadcast is corrupted (coral), the observation changes and so can the selected speed change.

02 / Adversarial formulation

A two-player game over observations.

With probability \(R\), an adversary replaces the true traffic state \(\mathbf{S}\) with a corrupted observation \(\Xi\) inside a bounded uncertainty set \(\Omega(\mathbf{S}) = \{\Xi : \lvert\Xi - \mathbf{S}\rvert \le \kappa\}\), chosen to minimize the agents’ value. The policy must perform well under both the clean and the corrupted observation.

$$\max_{u_t} V^{\pi}_{\mathrm{rob}}(\mathbf{S}_t) = \max_{u_t}\, \mathbb{E}_{\pi}\Big[\, r(\mathbf{S}_{t+1}, u_t) + \gamma\big((1-R)\,V^{\pi}(\mathbf{S}_{t+1}) + R \min_{\Xi \in \Omega(\mathbf{S}_{t+1})} V^{\pi}(\Xi)\big) \,\Big|\, \mathbf{S}_t \Big]$$

Robust value: the agents maximize return while the adversary corrupts the next observation with probability R.

Observation corruption as a two-player gameLeft: the adversary searches a bounded box around the true state and settles on the corner with the lowest value. Right: with probability one minus R the policy sees the clean state; with probability R it sees the adversary's observation. The adversary's move search a bounded set around the true state high V low V κ S Ξ* worst case The agents' move act well under both observations true state S 1 − R R clean obs S fake obs Ξ policy π speed change u
The adversary searches a bounded set around the true state for the lowest-value observation; the policy sees the clean state with probability 1 − R and the adversary's observation with probability R.

03 / Closed-form adversary

The worst case, without training an adversary.

Instead of learning an adversary through iterative, unstable co-training, we derive the worst-case observation directly. A first-order expansion of the value function moves the corrupted state to the lowest-value corner of the uncertainty box. The expression is one-shot, linear-time in the state dimension, and approximates the true worst case with second-order accuracy.

$$\Xi^{\star} = \mathbf{S} - \kappa \odot \operatorname{sign}\big(\nabla V(\mathbf{S})\big)$$

The gradient is taken through a frozen teacher value function.

The closed-form worst-case observationLeft: an iterative adversary needs many updates to find a bad observation. Right: the closed form takes the sign of the value gradient and moves the observation to the opposite corner of the bounded box in one step. Iterative adversary trained or searched: many updates, can be unstable many training updates Closed form (ours) one step, linear time, second-order accurate S ∇V Ξ* Ξ* = S − κ ⊙ sign(∇V(S))
An iterative adversary needs many updates; the closed form moves the observation to the box corner opposite the value gradient in one step.

04 / Robust training

Stay invariant to corruption, stay close to nominal.

Two regularizers shape the robust policy. An invariance term penalizes \(\mathrm{KL}\big(\pi(\cdot\mid\mathbf{S})\,\Vert\,\pi(\cdot\mid\Xi^{\star})\big)\), so the decision is the same whether the observation is clean or corrupted. An anchor term keeps the robust policy close to a nominal teacher, so robustness is not bought by giving up clean performance. If the invariance gap is bounded by \(B\), the performance loss under corruption is bounded by \(Q_{\max}\sqrt{2B}\); a smaller \(B\) means a tighter guarantee.

Phase 1: nominal

Train a teacher policy with no corruption (R = 0).

Phase 2: robust

Fine-tune with the closed-form adversary and a curriculum on R.

Execution

Centralized training, decentralized execution: each aircraft runs its own copy using only local observations.

Invariance and anchor regularizersLeft: during robust training, the action distribution under the corrupted observation is pulled toward the distribution under the clean observation. Right: a KL spring keeps the robust policy close to the nominal teacher. Bottom: the resulting performance bound. Invariance regularizer same decision, clean or corrupted observation −δv0+δv π(· | S) π(· | Ξ*) Anchor regularizer stay close to the nominal teacher policy robust π teacher π̄ KL divergence 1. keeps the robust policy near nominal behavior 2. avoids sacrificing clean performance KL(π(· | S) ‖ π(· | Ξ*)) ≤ B ⇒ performance loss ≤ Qmax·√(2B)
Invariance pulls the corrupted-observation action distribution toward the clean one; the anchor keeps the robust policy near the nominal teacher.
Policy architecture

A shared network encodes the ownship state and a stack of up to five neighbor states with multi-head attention and pooling, then decodes a value estimate and a speed-increment action.

Shared policy network: own state and neighbor states are encoded, combined through multi-head attention and pooling, and decoded into value and action heads.
Shared attention-based policy network; the frozen teacher's value gradient is used to compute the worst-case observation. Poster, panel 5

05 / Results

Graceful degradation as corruption grows.

The robust policy keeps near mid-air collisions (NMACs) near zero up to a 35% contamination rate and degrades gracefully beyond it, while the nominal policy deteriorates sharply. Ablations show that both regularizers are needed: invariance alone destabilizes training, and anchoring alone preserves nominal behavior but collapses at high corruption.

NMAC count and minimum separation versus contamination rate R for the full robust policy and the nominal policy.
Robust vs. nominal policy: NMACs (left) and minimum separation (right) as the contamination rate R increases. Poster, panel 6
Ablation: full robust policy, anchor without invariance, and invariance without anchor, showing NMACs and minimum separation versus contamination rate.
Ablation of the invariance and anchor regularizers. Shaded bands show variability across 100 episodes. Poster, panel 6
Simulation environment

BlueSky air traffic simulator with Amazon MK30 performance parameters, en-route package-delivery airspace.

BlueSky simulation of small UAS on structured en-route routes; aircraft are triangles with separation buffers drawn as circles.
High-density en-route sUAS traffic. Aircraft (triangles) follow structured routes; circles show separation buffers. Labels show flight ID, speed, and altitude. ACC 2026 poster

06 / Scope and next steps

Where this applies, and what comes next.

The approach applies to any cooperative multi-agent system whose shared observations can be corrupted. The evidence here is simulation-based. Follow-up work places a runtime safety filter around the deployed policy (DASC 2026); code will be released with the redesigned environment.

Reference

Cite this work

Zongo, A., Fotiadis, F., Topcu, U., & Wei, P. (2026). "Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing." arXiv preprint arXiv:2603.28900.

BibTeX 
@misc{zongo2026robust,
  title         = {Robust Multi-Agent Reinforcement Learning for Small UAS Separation
                   Assurance under GPS Degradation and Spoofing},
  author        = {Alex Zongo and Filippos Fotiadis and Ufuk Topcu and Peng Wei},
  year          = {2026},
  eprint        = {2603.28900},
  archivePrefix = {arXiv}
}

Acknowledgement. This work was supported by NASA under grant 80NSSC24M0070 and by ONR under grant N00014-22-1-2703.