project sunrise qantas test flight

The Cloud-Edge Architecture Behind Qantas's Historic Non-Stop Flight to London

When Qantas flight QF9 departed Perth for London in March 2018, the aviation world watched a 17-hour non-stop marathon that tested human endurance and aircraft engineering. But beneath the passenger experience and fuel calculations, Project Sunrise Qantas test flight was also a massive real-time data engineering and edge computing experiment - one that reveals how modern aviation is quietly becoming a proving ground for distributed systems, observability pipelines. And satellite-enabled cloud architectures.

As a software engineer who has built telemetry pipelines for maritime tracking systems, I find Qantas's Project Sunrise far more interesting for its backend than its cabin pressurization or seat design. The real story is how the airline-in partnership with Inmarsat, Collins Aerospace and AWS-architected a data infrastructure capable of streaming 100+ GB of flight data per hour from an aircraft moving at 560 mph over the Indian Ocean, with no terrestrial cell towers for 1,200 miles.

This article breaks down the technical systems behind the Project Sunrise Qantas test flight: the satellite edge nodes, the compressed telemetry protocols, the real-time anomaly detection pipelines. And the SRE lessons every platform engineer can steal. If you're building systems that must operate under extreme latency, intermittent connectivity. Or hostile environments, the architecture of a 17-hour flight is your new case study,

The Real Data Problem: Why 17 Hours of Flight Telemetry Is Hard

The Project Sunrise Qantas test flight wasn't just about proving the 787-9 could fly Perth to London non-stop. It was about proving that the airline could maintain real-time awareness of every system on that aircraft for the entire duration. Modern long-haul aircraft generate roughly 1. 2 TB of data per flight across engines, avionics - cabin systems. And crew tablets. Under normal operations, most of that data is stored locally and dumped after landing. But for a test flight pushing the limits of human and machine endurance, Qantas needed continuous, low-latency streaming to ground operations.

The challenge is physical. At cruising altitude over the Indian Ocean, the aircraft is beyond the range of any terrestrial radio or cellular network. The only link to the ground is satellite. And even modern L-band satellites (like Inmarsat's Global Xpress constellation) offer only 50 Mbps downlink and 5 Mbps uplink per aircraft. That's roughly the bandwidth of a 2005 home DSL connection-for a data pipeline processing hundreds of megabytes per minute.

This is where the engineering gets interesting. The Qantas team, with Collins Aerospace, implemented a lossy compression layer at the aircraft edge. Instead of transmitting raw sensor readings at 100 Hz, the onboard data aggregation unit (a Collins Aerospace Venue cabin server repurposed for telemetry) applied time-series compression using a variant of the Zstandard compression algorithm (RFC 8878)By grouping sensor readings into delta-encoded buckets and discarding redundant frames (e g., altitude readings that don't change for 30 seconds), they reduced the stream from 100 MB/min to roughly 8 MB/min-a 92% reduction that fit within the satellite uplink budget.

Satellite Edge Computing: Processing Data at 35,000 Feet

The Project Sunrise Qantas test flight relied on what cloud engineers now call "edge computing in motion. " The aircraft itself became a hybrid edge node-part data producer, part compute cluster. Onboard servers ran a modified version of AWS Greengrass (now AWS IoT Greengrass V2) to pre-process and filter telemetry before it ever touched the satellite link. This is a pattern I've seen in maritime tracking systems for cargo ships. But aviation's latency requirements are an order of magnitude stricter.

The edge pipeline performed three critical functions:

• Data deduplication: Multiple sensors (e, and g, engine temperature from left and right engines) were cross-referenced and merged into single time-series entries.
• Anomaly detection: A lightweight machine learning model (trained on 500+ hours of previous Qantas 787 flights) flagged any sensor reading that deviated more than 3 sigma from historical patterns. This triggered an immediate satellite priority channel.
• Payload prioritization: Cabin air quality, crew biometric data (from pilot wearables). And engine vibration spectra were given highest priority. In-flight entertainment logs were deprioritized and batched for post-landing sync.

This architecture is a textbook example of data gravity at the edge. By processing data where it was generated, Qantas avoided the satellite bottleneck entirely for 95% of normal operations. Only exceptions and summary statistics consumed the precious uplink bandwidth. For platform engineers building IoT or mobile systems, this is the same trade-off you face: do you stream every log event to the cloud,? Or do you filter, aggregate,? And only escalate?

Observability Pipelines Under Intermittent Connectivity

One of the trickiest engineering challenges in the Project Sunrise Qantas test flight was maintaining observability when the satellite link dropped out. While Inmarsat's Global Xpress constellation provides near-global coverage, there are gaps-especially over the southern Indian Ocean and near the equator where satellite handoffs occur. During these blackouts (which lasted 2-8 minutes), the aircraft's telemetry continued to flow,, and but ground operations had no visibility

The Qantas SRE team solved this with a store-and-forward queue backed by a local Redis instance on the aircraft. During connectivity loss, telemetry data was buffered in memory with a configurable TTL (time-to-live). If the blackout exceeded 10 minutes, the buffer was spooled to a local SSD. When satellite connectivity resumed, the queue drained in priority order-critical engine data first, then cabin metrics, then logs. This is essentially the same pattern used by Apache Kafka's log compaction and throttling but running on a 2U server in the cargo hold.

For the ground-side observability stack, Qantas used a combination of Prometheus (for time-series metrics) Grafana (for dashboards). The satellite link delivered a compressed protobuf stream that was decompressed and ingested into a custom Kafka topic. From there, the data flowed into a time-series database (TimescaleDB) for historical analysis and into a real-time alerting system that monitored 200+ thresholds-engine oil pressure, cabin CO2 levels, pilot heart rate variability. And even the number of steps taken by crew (tracked via wearable Fitbits).

The key lesson: observability under intermittent connectivity requires idempotent ingestion. Because the aircraft might re-send buffered data after a blackout, the ground pipeline had to handle duplicate timestamps without double-counting. This was achieved using a deduplication key composed of (flightID, sensorID, UnixTimestamp). Any duplicate with the same key was silently dropped by the Kafka consumer before it reached the database.

Real-Time Anomaly Detection and Pilot Alerting

Beyond infrastructure metrics, the Project Sunrise Qantas test flight was a testbed for real-time anomaly detection in human performance. Pilots wore SmartCap headbands that monitored EEG signals for fatigue. The data streamed to the edge server. Where a TensorFlow Lite model (trained on 200+ hours of simulator data) classified pilot alertness as "normal," "fatigued," or "critical. " This classification was packaged into a single-byte payload and transmitted via satellite every 30 seconds.

When the model detected "critical" fatigue in either pilot, the ground operations center received an alert within 2 seconds (including satellite latency). The protocol used a MQTT-SN (MQTT for Sensor Networks) broker running on the edge node. Which published to a ground-based RabbitMQ cluster. This is a classic pub/sub pattern, but adapted for the 600ms round-trip latency of geostationary satellite links. The Qantas team found that standard MQTT with QoS 2 was too chatty (too many ACK packets) for the limited uplink. So they downgraded to QoS 1 and implemented a custom retry mechanism that only re-sent if the ground didn't ACK within 3 seconds.

For developers building real-time alerting systems over unreliable networks, this is a critical optimization. QoS 2 (exactly-once delivery) is a myth in high-latency, low-bandwidth environments. You're better off with at-least-once delivery and idempotent consumers on the ground. The Project Sunrise test flight proved that even safety-critical alerts can be handled with QoS 1, as long as the ground system can deduplicate and the airborne system can prioritize.

Cloud Infrastructure: AWS and the Data Lake for Post-Flight Analysis

After the Project Sunrise Qantas test flight landed, the real work began. The 1. 2 TB of raw data stored on the aircraft's local SSDs was physically extracted and uploaded to an AWS S3 bucket in the Sydney region. This is a surprisingly low-tech step-despite all the satellite streaming, the bulk of the data was still moved by a human carrying a hard drive. But the cloud architecture that ingested and analyzed that data was anything but low-tech.

Qantas used AWS Glue to catalog and transform the raw binary sensor data into Parquet files partitioned by flight phase (takeoff, cruise, landing). The transformed data was loaded into Amazon Redshift for querying by data scientists and aviation engineers. They also built a QuickSight dashboard that visualized engine performance across the entire 17-hour flight, overlaying GPS coordinates with fuel flow, temperature. And vibration data.

What's notable here is the data lifecycle management. The airline set a 90-day retention policy for raw sensor data in S3 Standard, after which it was moved to S3 Glacier for archival. Only aggregated metrics (hourly averages, anomaly events) were kept indefinitely. This is a best practice for any organization dealing with high-volume telemetry: don't store raw data forever; store derived metrics that answer 95% of future questions.

Lessons for Platform Engineers: What Project Sunrise Teaches About Distributed Systems

The Project Sunrise Qantas test flight isn't just an aviation milestone-it's a distributed systems case study that every backend engineer should study. Here are the three takeaways I'm applying to my own work:

1, and edge filtering is non-negotiable for bandwidth-constrained systems Whether you're building an IoT sensor network or a mobile app with spotty connectivity, you can't stream everything to the cloud. The Qantas team achieved a 92% reduction by compressing and deduplicating at the edge, and add the same pattern with Mosquitto MQTT brokers on your edge nodes and you'll save cloud costs and improve reliability.

2. Idempotent ingestion is your safety net for unreliable networks. The deduplication key pattern (flightID + sensorID + timestamp) is trivial to add but saves you from data corruption when connections drop and retry. Every Kafka consumer or database inserter should have this logic,

3Real-time alerting over high-latency links requires protocol tuning. MQTT-SN with QoS 1 is a better fit for satellite or long-range radio than standard MQTT with QoS 2. If you're building for maritime, aviation. Or remote IoT, benchmark your protocol against real latency (600ms+), not localhost.

Frequently Asked Questions About the Project Sunrise Qantas Test Flight

What is Project Sunrise Qantas?

Project Sunrise is Qantas's initiative to operate non-stop commercial flights from Australia to London and New York. The test flights (starting in 2018) validated aircraft performance, crew endurance, and passenger comfort for 17-20 hour routes. The technology angle involves real-time satellite telemetry, edge computing. And cloud-based data analysis.

How did the test flight handle data transmission over the Indian Ocean?

The aircraft used Inmarsat's Global Xpress Ka-band satellite network for uplink, combined with an onboard edge server running AWS Greengrass. Data was compressed using Zstandard (RFC 8878), deduplicated, and prioritized before transmission. During satellite blackouts, data was buffered in a local Redis queue and re-sent when connectivity resumed.

What software stack was used for real-time monitoring?

Qantas used a combination of Prometheus for metric collection, Grafana for dashboards. And a custom Kafka pipeline for ingestion. The ground-side time-series database was TimescaleDB. Alerting was handled by a custom system monitoring 200+ thresholds, including engine parameters and pilot biometrics.

What was the biggest engineering challenge?

The most difficult aspect was maintaining observability during satellite handoffs and blackouts. The team solved this with a store-and-forward queue backed by local SSD storage, combined with idempotent ingestion on the ground to handle duplicate data packets after reconnection.

Can the same architecture be used for non-aviation applications?

Absolutely. The edge computing pattern (compress, deduplicate, prioritize) is directly applicable to maritime tracking, remote IoT sensor networks, autonomous vehicles. And even mobile apps in areas with poor connectivity. The deduplication key and MQTT-SN tuning are reusable patterns for any distributed system.

The Future of High-Latency, High-Volume Telemetry Systems

The Project Sunrise Qantas test flight demonstrated that modern aviation is as much a software engineering challenge as an aeronautical one. The systems that kept the aircraft visible to ground operations-edge compression, satellite-aware protocols, idempotent pipelines-are the same systems that will power autonomous ships, drone delivery networks, and remote industrial IoT. Every platform engineer should study the telemetry architecture behind that 17-hour flight. Because the next time you build for unreliable networks, you'll be solving the same problems.

If you're developing real-time systems for mobile or edge environments, start by auditing your data pipeline for compression opportunities. Can you reduce your stream by 90% without losing critical signals,? And can you make your consumers idempotentCan you switch from QoS 2 to QoS 1 without breaking your alerting? The answers will save you bandwidth - cloud costs. And operational headaches-just like they did for Qantas over the Indian Ocean.

What do you think?

How would you design a telemetry pipeline for a system that experiences 8-minute connectivity blackouts every hour? Would you use a different edge framework than AWS Greengrass, and why?

Is QoS 2 ever justified for safety-critical alerts over high-latency links,? Or should we accept at-least-once delivery with idempotent consumers as the standard pattern?

What's the most surprising engineering lesson you've learned from a non-software industry (aviation, maritime, automotive) that changed how you build distributed systems?