Ethics and Safety
The series ends where the engineering runs out. Bias is not a bug introduced late, alignment is a choice about whose judgement counts, and guardrails are a layer rather than a solution — the questions here do not have settled answers, but shipping without asking them is itself an answer.
Where Bias Comes From
Not one source but a chain, and each link compounds the last.
The corpus
Pretraining data is a snapshot of what people published, weighted by who published most. Over-represented languages, viewpoints and demographics are over-represented in what the model treats as ordinary.
The filters
Cleaning the corpus is itself editorial. Blocklists tuned to remove slurs also strip reclaimed language and whole dialects, quietly removing the communities that use them.
The annotators
Preference tuning encodes the judgement of a particular group of raters working to a particular set of guidelines. Their assumptions about what is helpful, polite or unsafe become the model's defaults.
The feedback loop
Model output becomes tomorrow's training data. Skew that is not corrected is not merely preserved — it is amplified with each generation of models trained on a web that models increasingly wrote.
Alignment: Whose Values?
Alignment is a technical process with a political core, and the technical part is the easier half.
Aligning a model means making its behaviour match an intent. The methods are well understood — collect preferences, fit a reward, optimise against it. The question the methods cannot answer is whose intent, adjudicated by whom, and with what recourse when someone disagrees.
These are not abstract. A model that refuses medical questions protects a company from liability and fails a nurse. One that answers freely serves the nurse and occasionally harms someone acting on bad advice. There is no setting that is simply correct; there is a choice about who bears the residual risk, and that choice is usually made by whoever wrote the guidelines, not the person affected.
Helpful vs harmless
The two objectives conflict directly. Every refusal boundary is a point on that trade-off, chosen by someone.
Universal vs local
One global model meets wildly different legal and cultural norms. Aligning to all of them at once is not coherent.
Stated vs revealed
Raters reward answers that sound good. Optimising for approval produces confident, agreeable, fluent output — which is not the same as correct.
Failure Modes Worth Designing Against
Concrete harms, ordered by how often they actually turn up in production rather than how dramatic they sound.
| Failure | What it looks like | Who it lands on |
|---|---|---|
| Confident error | A fabricated fact delivered with no hedging, acted on by someone with no way to check | The end user, usually the least expert one |
| Over-refusal | Legitimate questions declined because they pattern-match to something sensitive | People whose ordinary needs look unusual to a rater |
| Unequal quality | Noticeably worse output in some languages, dialects or name conventions | Exactly the groups already under-served |
| Privacy leakage | Memorised training data, or user input logged into the next training set | People who never used the product |
| Automation bias | A confident answer overriding a human's correct judgement | Whoever the decision was about |
| Scaled persuasion | Fluent, cheap, targeted text produced faster than anyone can review it | The information commons |
Guardrails and Their Limits
Each layer catches something the others miss. None is sufficient, and stacking them is the whole strategy.
Training-time alignment
Refusals and tone learned during preference tuning. Deep and general.
Cannot be changed without retraining, and encodes one set of values for every deployment.
System prompt
Instructions setting role and boundaries at the top of every conversation.
Sits in the same token stream as user input, so it competes with it rather than overriding it. Long conversations erode it.
Input and output filters
Classifiers on the way in and the way out, independent of the model.
Blunt. Both false positives and false negatives, and they add latency to every call.
Capability limits
What the surrounding system can actually do — which tools exist, what they can reach, what requires confirmation.
The most reliable layer, because it does not depend on the model behaving. Underused.
Human review
A person in the loop before anything consequential happens.
Does not scale, and degrades into rubber-stamping precisely when volume rises.
Questions Worth Asking Before You Ship
Not a compliance checklist. Six questions where not having an answer is the finding.
Who is harmed when it is confidently wrong?
Name them. If the answer is a user with less expertise than you, the interface needs to communicate uncertainty that the model itself does not express.
Does it work as well for everyone it serves?
Run your eval set across the languages, dialects and name conventions your users actually have. Unequal quality is invisible until it is measured.
Can a user tell they are talking to a model?
And can they reach a person? Undisclosed automation is a design choice that gets made by omission.
Where does user input end up?
Logged, retained, used for training? People type things into a chat box they would never put in a form.
What happens when it is manipulated?
Assume the input is hostile and the retrieved document is poisoned. What is the worst the tools you granted can do?
How would you know it had degraded?
A model update, a quantization change or a prompt edit can shift behaviour silently. Without monitoring, the first report comes from a user.