Safety-critical AI
Safety-critical AI. AI whose every answer can be checked, replayed and timed.
Safety-critical AI is artificial intelligence used where a wrong or late answer can harm people, an aircraft or a mission. It has to give the same answer to the same input, show the sources behind it, answer inside a known time and leave the decision to a person. AeroBrain AI, developed by SkyDynamics, is a set of high-assurance AI technologies built to those requirements for aerospace and defence.
What it is
Judged by its worst answer. Four properties separate safety-critical AI from a general assistant.
A general assistant is judged by how useful its answers are on average. Safety-critical AI is judged by its worst answer: one unsupported statement, one answer that arrives too late, or one decision that nobody can reconstruct afterwards can be enough to cause harm. So the properties that matter are those that can be shown to an engineer, an auditor or an authority.
- A general assistant
- A plausible answer
- Safety-critical AI
- An answer with its evidence, on time
Deterministic
The same question on the same data returns the same evidence in the same order, so an answer can be replayed and examined later.
Traceable
Every statement points to the reading, document and revision it rests on. A statement with no source is withheld.
Bounded in time
For functions where a late answer counts as a failure, each stage is designed to run inside a fixed time budget.
Overseen by a person
The AI proposes and shows its reasons. The accountable person decides, and the decision is recorded with the evidence.
The four properties as SkyDynamics builds AeroBrain to them. The timing figures depend on the hardware and the function: they are set with the customer and shown for each product.
Safety-critical LLM or SLM
A small model, held to its evidence. Why AeroBrain uses a small language model, not a general-purpose LLM.
A large language model (LLM) is trained on a broad corpus so that it can answer almost anything. A small language model (SLM) is compact and can be bounded to a narrow task. For safety-critical work the properties in this table decide, and they favour a small model that answers only from the evidence retrieved for the question.
| Criterion | General-purpose LLM | Assurance Small Language Model |
|---|---|---|
| Where it runs | Usually in a data centre, reached over a network | On the device, at the edge, on premises or air-gapped |
| What it answers from | What it learned in training, plus what it is given | Only the evidence retrieved for the question |
| A statement with no source | Can be produced and sound convincing | Designed to be withheld |
| Model version | A hosted model can be updated by its provider | A fixed version, recorded with every answer |
| Response time | Varies with load and the network | Designed to fit a fixed time budget |
| Audit afterwards | Hard to bound and inspect | Input, evidence, model version and output recorded together |
A general comparison of the two kinds of model; individual LLM deployments differ. The Assurance Small Language Model is developed by SkyDynamics; its figures depend on the hardware and are set with the customer.
AeroBrain AI
Four technologies. Each one answers a requirement of safety-critical AI.
AeroBrain is the SkyDynamics set of high-assurance AI technologies, developed in-house. Each can be used alone; combined, they turn real-time avionics data and approved manuals into decision support that a person checks.
Secured RAG engine
Retrieval written in Rust and deterministic: live readings and approved passages, each with its source and time or revision, in the same order every time.
Learn moreContextual Composition Protocol
Holds the task and its context in memory and reads each event against it, so the answer is given with the whole situation in view. In service today behind every aeroEBT application.
Learn moreLLiDA real-time database
Captures avionics data as time series on the device, in RAM and on disk, and keeps every connected node synchronised.
Learn moreAssurance Small Language Model
A compact model that composes its answer only from the evidence it is given and withholds what has no source. Every output is recorded with the model version.
Learn more
Standards
Engineered like flight software. And framed by the rules written for AI.
AeroBrain is developed to high-assurance practice so that the certification evidence for a product can be produced. Certification is granted for a specific product and installation, with the authority.
MISRA C, Rust and Ada SPARK
C code follows MISRA C, the RAG engine is written in Rust for memory safety, and Ada SPARK is used where properties have to be proved.
The objectives of DO-178C
Requirements, design, code, tests and evidence are traceable in both directions, as the airborne software standard expects.
EASA AI Concept Paper
EASA guidance for machine learning applications. SkyDynamics classifies AeroBrain as Level 1B, human cognitive assistance: the AI helps a person analyse and decide, and the person decides.
EU AI Act
Regulation (EU) 2024/1689 sets the obligations for high-risk AI systems, including human oversight, record keeping and transparency.
IEC 61508 and IEC 62304
Functional safety in other industries and the life cycle of medical device software, for safety-critical uses outside aviation.
Records an auditor can replay
Input data and its time, evidence with source and revision, model version, output and the human decision are kept together for each answer.
Where it applies
Wherever a wrong answer has consequences. Specialised in aerospace and defence.
Next-generation avionics
Real-time decision support on the aircraft, from live avionics data and the operator's approved manuals. The crew decides.
Airline operations control
OpsEye ranks and costs recovery options during a disruption. A controller decides, with four-eyes approval on regulated actions.
Learn moreEBT and CBTA training
The Contextual Composition Protocol works behind the EBT Scenario Builder and the instructor app. The instructor grades.
Learn moreDefence and mission systems
Disconnected and air-gapped operation, with the data and the model on the device.
Mission-critical data
The LLiDA database for avionics, mission-critical operations and space operations.
Learn moreMedical and other fields
The same set can be specified for medical devices and other safety-critical systems, at the assurance level each requires.
How an answer is made
From an event to a decision. Five steps, each one recorded.
In AeroBrain, an answer is not generated from what a model remembers. It is assembled from evidence, step by step, and every step leaves a record.
An event or a question
A reading crosses a limit in LLiDA, or a person asks. The time and the input are recorded.
The context is already held
The Contextual Composition Protocol holds the phase, the configuration and the task in memory, so the event is read in its situation.
Evidence is retrieved
The RAG engine retrieves live readings and approved passages deterministically, each with its source and time or revision.
Statements are composed
The Assurance Small Language Model words the options from that evidence only, and withholds what has no source.
A person decides
The options and their evidence are shown. The decision, accepted or overridden, is recorded with everything that led to it.
How to evaluate
Eight questions for any AI supplier. Before AI goes near a safety-critical function.
These questions separate AI built for safety-critical work from a general assistant placed in front of it. AeroBrain's answers are on this page and on the AeroBrain page.
Does the same question give the same answer?
Ask whether retrieval is deterministic: the same question on the same data must return the same evidence, so an answer can be replayed.
Where does each statement come from?
Ask to see the source and revision behind every statement, and what happens to a statement that has none.
Which model version answered?
Ask whether the model version is fixed and recorded with each answer, or can change behind an interface.
How long can an answer take?
Ask for the time budget and how it is met on your hardware, not an average response time.
Does it work without a connection?
Ask whether the model and its data run on the device, on premises or air-gapped.
Who decides?
Ask where the person sits in the loop, and how acceptance or override is recorded. In EASA terms, ask for the level of assistance claimed.
How is the software developed?
Ask for the coding rules and the development standard, such as MISRA C and the objectives of DO-178C, and for traceability from requirement to test.
Can an auditor replay it?
Ask whether input, evidence, model version, output and decision are kept together for every answer.
Terms
The vocabulary, in plain words. Ten terms used on this page.
Safety-critical AI
AI used where a wrong or late answer can harm people, an aircraft or a mission.
Mission-critical AI
AI that supports work that must not stop, so it has to keep working on the device.
Large language model (LLM)
A model trained on a broad corpus to answer almost anything, usually run in a data centre.
Small language model (SLM)
A compact model bounded to a narrow task, small enough to run on the device.
RAG
Retrieval-augmented generation: the model answers from documents and data retrieved for the question, so each answer can cite its source.
Deterministic retrieval
Retrieval that returns the same evidence, in the same order, for the same question on the same data.
Unsupported statement
A statement a model produces that no source supports. Safety-critical AI withholds it.
Time budget
The fixed time inside which each stage must finish, for functions where a late answer counts as a failure.
Human oversight
A person sees the options and the evidence and takes the decision, which is recorded.
EASA AI Level 1B
In the EASA AI Concept Paper, human cognitive assistance: the AI helps a person analyse and decide. SkyDynamics classifies AeroBrain at this level; it is not an EASA approval.
Questions about safety-critical AI
What is safety-critical AI?
Safety-critical AI is AI used in work where an error or a late answer can harm people, an aircraft or a mission. It must be deterministic, traceable to its sources, bounded in time and overseen by a person, and it is developed to the rules used for critical software so that its behaviour can be shown to an authority.
Can a large language model (LLM) be used in a safety-critical system?
Not as a free-standing source of answers. A general-purpose LLM can produce a convincing statement that no source supports, its version can change and its response time varies. A language model can take part in a safety-critical function when it answers only from evidence retrieved for the question, its version is fixed and recorded, each statement is traced to that evidence, it runs inside a time budget and a person decides. AeroBrain is built that way, with a small language model.
What is a safety-critical SLM?
A small language model (SLM) built for safety-critical work: compact enough to run on the device in real time, bounded to its task, and auditable. The Assurance Small Language Model in AeroBrain composes answers only from the evidence it is given, withholds a statement that has no source, and records its input, evidence, version and output together.
What is AeroBrain AI?
AeroBrain is the set of high-assurance, auditable AI technologies developed by SkyDynamics for safety and mission-critical fields, specialised in aerospace and defence. It combines a secured RAG engine written in Rust, the Contextual Composition Protocol, the LLiDA real-time time-series database and the Assurance Small Language Model.
How is safety-critical AI certified?
Certification is granted for a specific product and installation, by the competent authority, against the standards that apply to it, such as DO-178C for airborne software. The AI technologies inside the product are developed so that the evidence can be produced: AeroBrain follows MISRA C and the objectives of DO-178C, with requirements, code, tests and evidence traceable in both directions.
Does safety-critical AI take the decision?
Not in AeroBrain. It proposes options, shows the evidence for each and records what was accepted or overridden; the accountable person decides. SkyDynamics classifies AeroBrain at EASA AI Level 1B, human cognitive assistance.
What is the difference between safety-critical AI and mission-critical AI?
Safety-critical AI supports work where an error can harm people or an aircraft, so its answers must be traceable and reproducible. Mission-critical AI supports work that must not stop, such as operations control or a defence mission, so it must keep working on the device. AeroBrain is designed for both.
What is deterministic RAG?
Retrieval-augmented generation in which retrieval is deterministic: the same question on the same document snapshot and the same recorded data returns the same evidence in the same order, with its source and time. An auditor can replay the answer, not only read it.
Why is a small language model better suited to safety-critical work than a large one?
Because the properties that matter are easier to show for it. A compact model can run on the device inside a time budget, be fixed to one recorded version and be bounded to the evidence it is given. A general-purpose LLM is harder to bound and to inspect.
Can safety-critical AI run without an internet connection?
It should be able to. In AeroBrain, the RAG engine and the Assurance Small Language Model run on the device or on premises, including air-gapped, and LLiDA keeps the data on the node.
Bring a safety-critical function.
Tell us the function, the hardware and the assurance level. A technical briefing shows which AeroBrain technologies apply and what evidence comes with them.
We respond within one business day.