Project Overview
PlaneAR is a real-time aircraft tracking application for Android that uses augmented reality to overlay live flight data onto a camera feed. Built as a senior capstone project at Oregon State University, PlaneAR was a collaborative team effort—and the result is a fully functional, polished app that ships a custom Vulkan graphics engine written from scratch in C++.
Point your phone at the sky, and PlaneAR identifies aircraft in your field of view, displaying their callsign, altitude, and heading in 3D-anchored cards that follow the planes as they move.
Key Features
- Real-Time ADS-B Integration: Live flight data is fetched from public ADS-B APIs and continuously refreshed, tracking aircraft positions, altitudes, and callsigns within a configurable radius.
- Custom Vulkan Rendering Engine: The AR overlay is powered by a hand-built native Vulkan pipeline in C++, delivering low-latency, high-fidelity rendering on mobile hardware with minimal CPU overhead.
- 3D Billboarded Flight Cards: Per-aircraft information cards are rendered as instanced 3D billboards that exist in world space, anchored to each aircraft's projected position and interactive on tap.
- Aviation HUD: A professional-grade heads-up display features a smoothed compass tape and cardinal direction indicators—purpose-built for aircraft spotting.
- Sensor Fusion: Gyroscope and accelerometer data are combined and smoothed with Euler-angle filtering to produce a stable, jitter-free AR experience.
- Achievement System: An integrated milestone system rewards users for spotting rare aircraft types and hitting tracking goals.
Technical Highlights
PlaneAR showcases several advanced engineering concepts across its hybrid Kotlin/C++ architecture:
- Vulkan Graphics Pipeline: Full pipeline ownership—swapchain management, render passes, descriptor sets, GLSL shaders, and texture management are all handled natively with no middleware.
- JNI Bridge: A low-overhead communication layer passes aircraft vectors, orientation matrices, and scene state from the Kotlin application tier down to the native renderer each frame.
- Instanced Rendering: Billboard primitives (labels, glyphs, and compass elements) are batched into instanced draw calls with per-instance vertex buffer data, eliminating redundant API overhead.
- Hybrid 2D/3D Rendering: The engine maintains an ordered draw list supporting interleaved 2D HUD and 3D world-space elements within a single frame, resolved via a batch marker sentinel approach.
- AR Culling: A dot-product cull against the camera forward vector discards aircraft behind the viewer before they ever reach the GPU.
- Camera Texture Pipeline: Live camera frames are imported as
AHardwareBufferand consumed in Vulkan via YCbCr external image conversion, enabling zero-copy camera-to-renderer frame delivery.
Demonstration
Pictured above: The AR overlay in action—3D-anchored flight cards tracking live aircraft in the user's field of view, including the aviation HUD with compass tape and cardinal direction indicators.
Technologies Used
- Languages: Kotlin, C++20
- Graphics API: Vulkan
- Platform: Android (Jetpack, NDK)
- Build Tools: Gradle, CMake
- CI/CD: GitHub Actions (automated APK release builds)
Learning Outcomes
This project demonstrates proficiency in:
- Low-level graphics programming with the Vulkan API on mobile hardware.
- Native Android development with JNI and the NDK.
- Real-time sensor fusion and AR coordinate projection.
- Performance optimization for GPU-bound rendering on constrained devices.
- Cross-functional team collaboration on a complex, multi-layer codebase.