diff --git a/.gitignore b/.gitignore index 614e06c..e8e3c27 100644 --- a/.gitignore +++ b/.gitignore @@ -45,3 +45,5 @@ ilp-rest-service/ilp-cw-api/results.json target drone-black-box/drone-black-box drone-black-box/drone_black_box.db* + +*.pdf diff --git a/README.md b/README.md index 3b617b2..17b974d 100644 --- a/README.md +++ b/README.md @@ -17,3 +17,28 @@ chromium --user-data-dir="/tmp/chromium" --disable-web-security ``` since CORS is not handled in this setup. + +### Setup via Nix + +Ensure you have [Nix](https://nixos.org/download.html) installed, then run: + +```bash +nix develop +``` + +This will set up a development environment with all necessary dependencies. + +Then start each component separately: + +```bash +cd ilp-rest-service +./gradlew bootRun + +cd drone-black-box/ +go build +./drone-black-box + +cd drone-frontend/ +bun dev . +``` + diff --git a/docs/assets/architecture.png b/docs/assets/architecture.png new file mode 100644 index 0000000..e746181 Binary files /dev/null and b/docs/assets/architecture.png differ diff --git a/docs/assets/frontend-dbg.png b/docs/assets/frontend-dbg.png new file mode 100644 index 0000000..c13abce Binary files /dev/null and b/docs/assets/frontend-dbg.png differ diff --git a/docs/src/main.typ b/docs/src/main.typ new file mode 100644 index 0000000..9f54ad0 --- /dev/null +++ b/docs/src/main.typ @@ -0,0 +1,168 @@ +#import "@preview/codly-languages:0.1.10": * +#import "@preview/codly:1.3.0": * + +#show: codly-init.with() + +#let title = "The Drone Telemetry & \"Black Box\" System" +#let author = "Peikun Yang" +#let mailRaw = "s2522255@ed.ac.uk" +#let mail = link("mailto:" + mailRaw)[#mailRaw] +#let date = datetime.today().display("[year]-[month]-[day]") +#show raw: set text(font: "Maple Mono NF") + +#set page( + header: align(right)[#title], + numbering: "1", + paper: "a4", + // margin: (x: 20pt, y: 50pt) +) + + +#block( + width: 100%, + height: 100%, + align(center + horizon)[ + #text(size: 1.8em, weight: "bold")[CW3 Explanation #linebreak() #title] + + #v(0.5em) + + #text(size: 1.4em, weight: "bold")[#author] + + // #v(0.5em) + + #text(size: 1.1em, weight: "bold")[#mail] + ] +) + +#pagebreak() + +#set heading(numbering: "1.1") +#show link: it => emph(underline(it)) + += Introduction & Problem Statement + +The ILP Rest Service provides a complete backend solution for drone querying, availability +checking, and delivery path planning. However, as the service focuses on backend logic, +it provides very limited insight into the runtime behaviour of the system. +// From an operational and engineering perspective, this lack of observability makes it +// difficult to understand, validate, and debug complex dispatch and pathfinding decisions. + +In real-world scenarios, issues such as route inefficiencies, unexpected detours, +collisions, or battery-related failures cannot be effectively analysed without access +to a detailed execution trace. Currently, operators and developers only receive the +final output of endpoints such as `/api/v1/calcDeliveryPath`, which often consists of +extremely long coordinate sequences. Debugging solely based on these final results is +inefficient and error-prone, as the internal decision-making steps of the +pathfinding process remain hidden. + +This project introduces a standalone, sidecar "Black Box" Telemetry Service. Unlike a standard CRUD interface, +this system captures high-frequency flight data and enables +Time-Travel Debugging, allowing operators to visualize real-time telemetry and +engineers to replay historical states deterministically to identify algorithm flaws. + += System Innovation & Architectural Benefits + +To address the observability gap, this project implements a solution that deviates from a traditional monolithic CRUD design. The innovation lies in the adoption of a Heterogeneous Microservices Architecture and the application of Event Sourcing principles. + +== Heterogeneous Microservices + +Instead of forcing the Java application to handle high-frequency telemetry ingestion +, which may have negative impact on pathfinding workloads + +- *Java* remains focused on CPU-bound business logic (path calculation). +- *Go* as a lightweight, concurrency-optimized service handles the I/O-bound telemetry stream. + +This polyglot approach allows the system to leverage Go's low-latency garbage collection and Goroutines for handling thousands of concurrent telemetry writes, ensuring the monitoring subsystem does not become a bottleneck for the core delivery service. + +== Fault Isolation & Asynchronous Dispatch + +A critical innovation in this design is *Fault Isolation*. The Java service utilizes an asynchronous *Fire-and-Forget* pattern to dispatch telemetry. +Unlike tight coupling where a database failure could crash the entire dispatch system, this architecture ensures that the core business logic remains resilient. Even if the "Black Box" service is offline or experiencing high latency, the drone delivery calculations proceed uninterrupted. + +== Time-Travel Debugging via Event Sourcing + +Traditional systems often overwrite the "current state" of an entity (e.g., updating a drone's coordinates in a DB row). This destroys historical context. +This project implements Event Sourcing: rather than storing state, we persist the *stream of events* (timestamped location updates). +- *Benefit:* This enables deterministic replay. We can reconstruct the exact state of the fleet at any recorded time. +- *Application:* This transforms the system from a passive monitor into an active forensic tool, allowing engineers to "rewind" time to investigate collisions or route anomalies that are impossible to catch in real-time. + +#figure( + image("../assets/architecture.png"), + caption: [Architecture of Coursework 3] +) + += Implementation Details and Technologies + +== Data Protocol Optimization + +Instead of using verbose GeoJSON for internal telemetry transfer, a compact flat JSON schema, DroneEvent, is adopted for communication between the Go telemetry service and the Svelte frontend. This significantly reduces payload size and network overhead, minimising latency during high-frequency timeline scrubbing. The Java REST service is extended to emit compatible events for both real-time and historical path reconstruction. + +== Backend: Golang & SQLite + +The telemetry service is an I/O-bound application requiring high concurrency. +It is designed to handle thousands of ingestion requests allowing the front-end service +dragging the progress bar smoothly (real-time querying the backend). +If use the main ILP Rest Service, it will block request-response model under high I/O pressure. + +To achieve persistence without operational overhead, I used SQLite, which provides full +ACID guarantees while keeping the microservice self-contained and easy to deploy, +and edge-computing friendly. + +Concurrency control follows a reader–writer separation strategy to allow non-blocking snapshot queries during continuous telemetry ingestion. + +== Frontend: Svelte & Reactive DOM + +#figure( + image("../assets/frontend-dbg.png"), + caption: [Frontend with Debug Information] +) + +The visualization dashboard is built with *Svelte* to ensure rendering performance. + +Svelte compiles components into highly efficient vanilla JavaScript that directly manipulates the DOM. This #strong[Compiler-based approach] is crucial, reduces runtime overhead for the map interface, where the system must render moving drone markers at 60fps during timeline scrubbing without frame drops. + +For the mapping engine, I selected #strong[Leaflet.js] due to its lightweight footprint. I utilized Svelte's reactive statements (`$:`) to create a declarative data flow: when the time-slider input changes, it automatically triggers the data fetcher (querying the Go backend) and updates the Leaflet marker layer. This significantly reduced boilerplate code compared to an imperative DOM manipulation approach. + +== Quality Assurance & Validation + +I implemented a multi-layered quality assurance strategy: + +- Unit Testing +- Load Resilience, simulating burst traffic +- Reproducibility, the use of `docker-compose` for consistent environment setup + += Challenges & Trade-offs + +== Defining Velocity and Time Step + +Since the CW2 specification defines strictly spatial discretization (moves) without temporal parameters, a velocity model is required to enable deterministic time-series replay. + +We adopt a cruising speed of $v = 15 "m/s"$, based on the specifications of the #link("https://www.dji.com/global/mavic-3-enterprise/specs")[DJI Mavic 3 Enterprise]. + +Given the simulation is centered in Edinburgh ($approx 55 degree$N), we can only consider latitude. + +// we calculate the physical distance of a single logical `STEP` ($0.00015 degree$). We consider the meridional distance (latitude) as the upper bound for step length, as longitudinal distance decreases with the cosine of the latitude: + +// $ +// 1 degree_phi &approx 111.32 "km" \ +// 1 degree_lambda &approx 111.32 "km" times cos(55 degree) approx 63.85 "km" +// $ + +The maximum physical distance $d_"max"$ per logical move is therefore: + +$ d_"max" = 1 degree_phi times 0.00015 approx 16.698 "m" $ + +With velocity $v = 15 "m/s"$. // the time duration per step $Delta t$ is: + +// $ Delta t = d_"max" / v = (16.698 "m") / (15 "m/s") approx 1.113 "s" $ + +For the purpose of this _Proof of Concept_, we simplify the temporal resolution to *1.0s per step*. This provides a realistic yet computationally convenient baseline for the telemetry playback. + + += Conclusion + +This project demonstrates a robust, polyglot approach to system +observability. By leveraging Go for performance, SQLite for persistence, +and Svelte for interactivity, the \"Black Box\" provides a +professional-grade debugging tool that significantly enhances the +maintainability of the drone delivery ecosystem. diff --git a/docs/src/preamble.typ b/docs/src/preamble.typ new file mode 100644 index 0000000..3b3a504 --- /dev/null +++ b/docs/src/preamble.typ @@ -0,0 +1,37 @@ +#import "@preview/codly-languages:0.1.10": * +#import "@preview/codly:1.3.0": * + +#show: codly-init.with() + +#let title = "The Drone Telemetry & \"Black Box\" System" +#let author = "Peikun Yang" +#let mailRaw = "s2522255@ed.ac.uk" +#let mail = link("mailto:" + mailRaw)[#mailRaw] +#let date = datetime.today().display("[year]-[month]-[day]") +#show raw: set text(font: "Maple Mono NF") + +#set page( + header: align(right)[#title], + numbering: "1", + paper: "a4", + // margin: (x: 20pt, y: 50pt) +) + + +#block( + width: 100%, + height: 100%, + align(center + horizon)[ + #text(size: 1.8em, weight: "bold")[CW3 Explanation #linebreak() #title] + + #v(0.5em) + + #text(size: 1.4em, weight: "bold")[#author] + + // #v(0.5em) + + #text(size: 1.1em, weight: "bold")[#mail] + ] +) + +#pagebreak() diff --git a/drone-frontend/src/components/Sidebar.svelte b/drone-frontend/src/components/Sidebar.svelte index f778292..9e9f5fb 100644 --- a/drone-frontend/src/components/Sidebar.svelte +++ b/drone-frontend/src/components/Sidebar.svelte @@ -117,12 +117,6 @@ -
- > {status} -
-
@@ -213,29 +207,5 @@ {/if}
-
-

- Dispatch Payload (JSON) -

- -
- -
- -