Augmented Reality Aircraft Tracking
PlaneAR
A real-time aircraft tracking app for Android that uses augmented reality to overlay live flight data onto a camera feed — built as a senior capstone at Oregon State University and shipping a custom Vulkan graphics engine written from scratch in C++.
- Team
- OSU Senior Capstone
- Languages
- Kotlin, C++20
- Graphics API
- Vulkan
- Platform
- Android
Project Overview
PlaneAR is a real-time aircraft tracking application for Android that uses augmented reality to overlay live flight data onto a camera feed. Built as a senior capstone project at Oregon State University, PlaneAR was a collaborative team effort—and the result is a fully functional, polished app that ships a custom Vulkan graphics engine written from scratch in C++.
Point your phone at the sky, and PlaneAR identifies aircraft in your field of view, displaying their callsign, altitude, and heading in 3D-anchored cards that follow the planes as they move.
Technologies Used
- Languages
- Kotlin
- C++20
- Graphics API
- Vulkan
- Platform
- Android (Jetpack, NDK)
- Build Tools
- Gradle
- CMake
- CI/CD
- GitHub Actions
Key Features
Real-Time ADS-B Integration
Live flight data is fetched from public ADS-B APIs and continuously refreshed, tracking aircraft positions, altitudes, and callsigns within a configurable radius.
Custom Vulkan Rendering Engine
The AR overlay is powered by a hand-built native Vulkan pipeline in C++, delivering low-latency, high-fidelity rendering on mobile hardware with minimal CPU overhead.
3D Billboarded Flight Cards
Per-aircraft information cards are rendered as instanced 3D billboards that exist in world space, anchored to each aircraft's projected position and interactive on tap.
Aviation HUD
A professional-grade heads-up display features a smoothed compass tape and cardinal direction indicators—purpose-built for aircraft spotting.
Sensor Fusion
Gyroscope and accelerometer data are combined and smoothed with Euler-angle filtering to produce a stable, jitter-free AR experience.
Achievement System
An integrated milestone system rewards users for spotting rare aircraft types and hitting tracking goals.
Technical Highlights
Vulkan Graphics Pipeline
Full pipeline ownership—swapchain management, render passes, descriptor sets, GLSL shaders, and texture management are all handled natively with no middleware.
JNI Bridge
A low-overhead communication layer passes aircraft vectors, orientation matrices, and scene state from the Kotlin application tier down to the native renderer each frame.
Instanced Rendering
Billboard primitives (labels, glyphs, and compass elements) are batched into instanced draw calls with per-instance vertex buffer data, eliminating redundant API overhead.
Hybrid 2D/3D Rendering
The engine maintains an ordered draw list supporting interleaved 2D HUD and 3D world-space elements within a single frame, resolved via a batch marker sentinel approach.
AR Culling
A dot-product cull against the camera forward vector discards aircraft behind the viewer before they ever reach the GPU.
Camera Texture Pipeline
Live camera frames are imported as AHardwareBuffer and consumed in Vulkan via YCbCr external image conversion, enabling zero-copy camera-to-renderer frame delivery.
Demonstration
The AR overlay in action—3D-anchored flight cards tracking live aircraft in the user's field of view, including the aviation HUD with compass tape and cardinal direction indicators.
Learning Outcomes
Low-level graphics programming with the Vulkan API on mobile hardware.
Native Android development with JNI and the NDK.
Real-time sensor fusion and AR coordinate projection.
Performance optimization for GPU-bound rendering on constrained devices.
Cross-functional team collaboration on a complex, multi-layer codebase.
Learn More
Visit the project website or explore the source on GitHub.