↓Skip to main content

Behind The Scenes

Table of Contents

Overview
#

This article aims to explain how we have re-designed and re-written the backend used to collect, enrich and display the ADSB data our community sends us.

Anatomy of a Feeder Connection
#

When you follow our Getting Started Guide:

plane.watch feeder connection anatomy

  1. pw-feeder opens two connections to Bordercontrol - one connection for BEAST, and another for MLAT.
  2. During connection establishment, Bordercontrol checks the validity of the feeder’s API key. If invalid, the connection is dropped. If valid, the connection is allowed, and bordercontrol starts a “feed-in” container for the client.
  3. Bordercontrol proxies BEAST traffic to the feed-in container.
  4. Bordercontrol proxies MLAT traffic to an mlat_server instance running on the regional multiplexer.
  5. pw_ingest runs within the feed-in container, decodes the BEAST data, and publishes the data as a message onto the NATS message bus.
  6. pw_ingest runs within the regional multiplexer containers, decodes the BEAST-formatted MLAT results, and publishes the data as a message onto the NATS message bus.
  7. The data is processed through the pw-pipeline, and is enriched before…
  8. …being displayed to the user via a websocket that connects the frontend website to the output of the pipeline.

Components
#

ADS-B Traffic Control (ATC)
#

ATC is plane.watch’s feeder portal. It provides:

  • User sign-up and account management
  • Feeder creation and management
  • A databases containing aircraft types, airports, routes, operators and waypoints

ATC provides both public and private APIs. The public API is used by pw-feeder to report the feeder’s status back to the client.

The private API is used to:

pw-feeder
#

pw-feeder is a BEAST and MLAT protocol specific stunnel client, that securely proxies data from the client to plane.watch’s servers, specifically bordercontrol.

pw-feeder runs on the feeder client computer, and receives BEAST protocol data from software such as dump1090 and readsb, or from hardware devices such as the Jetvision Radarcape. It also communicates with mlat-client software running on the client, and mlat-server software running within plane.watch.

Stunnel is used to ensure that the data you send to us is encrypted and tamper resistant. Furthermore, we send the feeder’s plane.watch API key in the TLS request’s Server Name Indication (SNI) field, allowing us to determine which data is associated to which feeder.

bordercontrol
#

Bordercontrol is our custom-written feed receiver. Its job is to handle the many incoming connections from feeders running our pw-feeder client software.

When a pw-feeder client connects:

  1. The feeder’s API key is read from the stunnel’s SNI information.
  2. Bordercontrol queries ATC to determine if the API key matches that of a valid feeder.
  3. If the API key is valid, a unique “feed-in” container is spun up for that client. If the API key is invalid, the connection is dropped.
  4. The incoming BEAST connection is proxied to the feed-in container.
  5. The incoming MLAT connection is proxied to an instance of mlat-server running on the regional multiplexer.

pw_ingest
#

pw_ingest runs on each feed-in container. It decodes BEAST protocol frames, and publishes the data as messages to NATS, for processing through the plane.watch pipeline. Messages are tagged with the feeder’s API key, so the source of the data can be tracked throughout the processing pipeline.

pw-enrichment
#

pw-enrichment, also known as EC (enrichment centre), consumes decoded ADS-B frames from the NATS message queue and adds additional information relating to the aircraft, route and other metadata. The EC maintains an internal memory-cache so as to not overwhelm the ATC database, given the message rate can be in the thousands of messages a second.

pw_router
#

pw_router takes enriched data and reduces it down to significant events, thus cutting the message rate significantly. The router also publishes two feeds to the NATS bus - a high resolution and a low resolution - these are used on the website to not overwhelm the browser with messages.

Router also connects to an instance of ClickHouse, a highly scalable columnar database designed for storing timeseries data - perfect for ADSB. Both high and low resolution feeds are stored to Clickhouse for later analysis and for drawing the historical paths in the UI.

pw_ws_broker
#

pw_ws_broker is a horizontally scalable service that provides a WebSocket interface to connect the public web interface with the backend pipeline. When a user loads the public website, a websocket is connected to this service to stream message updates from the pipeline directly to the user.

The broker also provides streaming APIs for querying ATC for more detailed route and airframe information, as well as querying ClickHouse for historical track information.

pw-ui
#

pw-ui is a lightweight Ruby on Rails app that serves the HTML/CSS/Javascript for the beta.plane.watch frontend website. Stimulus is the Javascript framework we chose as it models functionality through flexible controllers that integrate with HTML, in-line, to bind dynamic behaviour.

The map view is based on the OpenLayers framework which provides a high-performance interface to the HTML Canvas API or WebGL.

In Conclusion
#

We are aiming to launch our new UI (currently accessible via https://beta.plane.watch) in December of 2023. At this point in time, our private GitHub repositories will become public, making this project fully open source.

How Can I Get Involved?
#

  • Join our Discord. Come say G’day!
  • Become a Feeder. Share your ADSB data!
  • Contribute Financially. Help us fund our server and hosting costs!
  • Submit code to our repositories. We recommend discussing your proposed code changes in our Discord prior to submitting any pull requests.


Plane Watch
Author
Plane Watch