Ethics of Machine Learning

IB Syllabus: A4.4.1 – Discuss the ethical implications of machine learning in real-world scenarios (accountability, algorithmic fairness, bias, consent, environmental impact, privacy, security, societal impact, transparency; biases in training data; online communication ethics).

Why This Page Exists
The Nine Ethical Dimensions
Where Bias Comes From: Training Data
Online Communication Ethics
How to Answer a “Discuss” Question
1. Worked Example
Quick Check
Match the Dimension
Practice Exercises
1. Core
2. Extension
3. Challenge
Connections

Why This Page Exists

Machine learning systems now make or shape decisions that used to belong to people: who gets a loan, which job applications a recruiter sees first, what a doctor is warned about, which posts fill your feed. When a system makes those decisions at scale, small flaws stop being small. A single biased rule, copied across a million decisions, becomes a pattern of harm.

This page is about the questions you should ask of any machine learning system before trusting it: Is it fair? Who is accountable when it is wrong? What data was it trained on, and whose privacy did that data cost? Can anyone explain why it decided what it decided? These are not side issues. The most capable model in the world is a liability if no one can answer them.

The skill being assessed here is judgment, not recall. You are expected to weigh competing considerations and reach a defended position, not to list dangers. The How to Answer a “Discuss” Question section near the bottom shows exactly how to structure that.

The Nine Ethical Dimensions

The syllabus names nine dimensions to weigh when evaluating a machine learning system. They overlap in practice, but each gives you a different angle of attack on a scenario.

1. Privacy

Machine learning is hungry for data, and the most useful data is often the most personal: health records, location history, messages, purchases, faces. Privacy is the question of whether people keep control over information about themselves.

A model can leak privacy in two ways. The obvious one is the training data itself: a hospital that trains a diagnosis model on patient records has gathered sensitive data in one place, where a breach would be catastrophic. The subtler one is inference: a model trained on shopping habits may deduce a pregnancy, a health condition, or a sexual orientation that the person never disclosed. The harm is not that the data was stolen; it is that the system worked out something private from data that seemed harmless.

Example: A supermarket loyalty-card model predicts which customers are likely to be pregnant from changes in their buying patterns, then targets them with baby-product offers, revealing the prediction to anyone who sees the household’s post.

2. Security

Security is about protecting the system and its data from attack or misuse. For machine learning this has an ordinary side and a model-specific side.

The ordinary side is the same as any system holding valuable data: access control, encryption, backups, and recovery procedures so that a breach or a hardware failure does not expose or destroy the training set. The model-specific side is newer. An attacker who can feed crafted inputs to a model can sometimes poison its training (corrupting future predictions) or perform an adversarial attack (a tiny, deliberate change to an input that flips the output, such as a sticker on a road sign that makes a self-driving car misread it).

Example: A spam filter that learns from user “report spam” clicks can be manipulated by a coordinated group who mass-report a competitor’s legitimate emails, training the filter to block them.

3. Bias and Algorithmic Fairness

Bias is systematic error that favours or disadvantages a group. Algorithmic fairness is the goal of decisions that do not depend on characteristics like race, sex, or age when those characteristics are not legitimately relevant.

A model is not neutral just because it is mathematical. It learns the patterns in its training data, including the unfair ones. If past hiring favoured one group, a model trained on that history will learn to favour that group, and it will do so consistently and invisibly. (Where this bias comes from is covered in the next section.)

Fairness is genuinely hard because different definitions of “fair” conflict. A model can be calibrated to be equally accurate for every group or to produce equal selection rates across groups, but often not both at once. There is no purely technical answer; it is a value choice that someone must own.

Example: A loan-approval model trained on decades of approvals offers systematically smaller limits to applicants from certain postcodes, because those postcodes correlate with a group that was underserved by banks in the past.

4. Accountability

When an automated decision causes harm, accountability asks: who is responsible? The developer who built the model? The company that deployed it? The person who chose to act on its output?

The danger is the accountability gap: everyone points at the algorithm, and the algorithm cannot be held responsible. “The system flagged you” is not an answer a person can argue with or appeal. Good practice keeps a human accountable for consequential decisions and keeps records (logs, version history) so a decision can be traced and reviewed after the fact.

Example: A self-driving delivery robot injures a pedestrian. The manufacturer says the software behaved as designed; the operator says they trusted the manufacturer’s safety claims; the pedestrian has no clear party to hold responsible.

Consent is permission, freely given by an informed person, for their data to be collected and used in a specific way. It is one of the most commonly missing ingredients in machine learning.

Data is often gathered for one purpose and later reused to train a model for another, without anyone going back to ask. People who post photos publicly did not consent to those photos training a face-recognition system. The public who share a road with a vehicle being tested did not consent to being part of the experiment. Real consent must be informed (people understand what they are agreeing to) and specific (it covers this use, not “anything we think of later”).

Example: A company scrapes millions of public social-media photos to train a face-recognition product. The photos were public, but no one consented to being enrolled in a face-matching database.

6. Transparency

Transparency is the degree to which people can see and understand how a system reaches its decisions. A system is opaque (a “black box”) when even its builders cannot fully explain a specific output.

Transparency matters most when a decision affects someone’s life. A person refused a loan, flagged by a fraud system, or rejected from a job has a reasonable claim to know why, and to challenge the reasoning. Complex models (especially deep neural networks) trade explainability for accuracy, which is why explainable AI (techniques that produce human-readable reasons for a prediction) is an active and important field.

Example: A student is rejected by an automated university-admissions screen. The school cannot explain which factors counted against the student, so the student cannot tell whether the decision was fair or appeal it meaningfully.

7. Environmental Impact

Training and running large models consumes real energy. Environmental impact is the carbon and resource cost of the computation: the electricity to train a model across thousands of processors for days or weeks, the water and power to cool the data centre, and the hardware manufactured and eventually discarded.

This dimension is easy to forget because the cost is invisible to the end user, but a single large model’s training run can consume as much energy as many households use in a year. The ethical question is whether the benefit justifies the cost, and whether cleaner choices (smaller models, efficient hardware, renewable-powered data centres, reusing an existing model instead of training a new one) were considered.

Example: A company retrains a giant language model from scratch every few months to keep it current, when fine-tuning a smaller existing model would meet its needs at a fraction of the energy cost.

8. Societal Impact

Societal impact is the effect of a deployed system on communities and society as a whole, beyond the individuals it directly processes. Machine learning can reshape labour markets, concentrate power in whoever owns the best models and data, and change how people get information and form opinions.

This is the widest dimension and it shades into the Computing in Daily Life topic. For machine learning specifically, watch for feedback loops: a system that shapes behaviour, then learns from the behaviour it shaped. A recommender that pushes sensational content trains users toward sensational content, which it then learns to push harder.

Example: A hiring tool used across an entire industry converges on the same idea of an “ideal candidate,” so applicants who do not fit that template are filtered out everywhere at once, narrowing who can enter the profession.

9. The Dimensions Work Together

These nine are angles on the same systems, not separate boxes. A facial-recognition camera at a school gate touches privacy (biometric data), consent (did students agree?), security (where is the data stored?), bias (is it less accurate for some skin tones?), accountability (who answers for a wrong match?), and societal impact (normalising surveillance of children). A strong analysis names several dimensions and shows how they pull against each other.

Dimension	One-line test question
Privacy	Whose personal data does this use, and do they keep control of it?
Security	Could this system or its data be attacked, breached, or corrupted?
Bias & fairness	Does it treat groups differently when it should not?
Accountability	When it is wrong, who is responsible and can the decision be appealed?
Consent	Did people knowingly agree to this specific use of their data?
Transparency	Can anyone explain a specific decision in human terms?
Environmental impact	Is the energy and hardware cost justified?
Societal impact	How does this change communities, labour, or public life at scale?

Where Bias Comes From: Training Data

Most unfairness in machine learning is not introduced by a malicious programmer. It comes in through the training data, because a model trained on biased data learns biased patterns. Knowing the named sources of data bias lets you diagnose a scenario precisely instead of just saying “the data was bad.”

Type of bias	What it means	Example
Historical bias	The data faithfully records a world that was already unfair, so the model learns the unfairness.	A model trained on past promotions learns to favour the group that was historically promoted.
Sampling bias	The data was not collected from a representative slice of the population.	A skin-condition model trained mostly on light skin performs worse on dark skin.
Selection bias	The way cases entered the dataset skews who is in it.	A loan-default model trains only on approved loans, never seeing how rejected applicants would have performed.
Labelling bias	The “correct answers” used in training carry human prejudice or inconsistency.	Toxic-comment labels reflect one group of labellers’ idea of what counts as offensive.
Measurement bias	A feature is a poor or unequal proxy for what you actually care about.	Using “arrests” as a stand-in for “crime” bakes in the bias of where police patrol.

The principle to remember: garbage in, garbage out is too kind. It is more like bias in, bias amplified, because the model applies the learned pattern consistently across every future decision, and its mathematical output can look objective enough that no one questions it.

Reducing data bias is a real engineering task: collecting from diverse sources, checking that every relevant group is represented, cleaning errors, testing the model’s accuracy separately for each group rather than only on average, and keeping a feedback loop to catch bias that emerges after deployment.

Online Communication Ethics

The syllabus folds the ethics of online communication into machine learning, because the systems that rank, recommend, moderate, and generate online content are themselves machine learning systems. Five issues recur.

Misinformation. Recommendation and ranking models optimise for engagement, and false or sensational content is often more engaging than accurate content. A model that maximises clicks can amplify misinformation without anyone intending it to. Generative models add a second problem: they can produce convincing false text, images, and video (deepfakes) at scale.
Bias in what you see. Two people searching the same term, or opening the same app, can be shown very different content because the model has profiled them differently. This personalisation can harden into a filter bubble where someone only sees views they already hold.
Online harassment. Platforms rely on automated moderation to detect abuse at scale, but these models miss context, sarcasm, and languages they were not trained on, so they both over-block legitimate speech and under-protect targeted users.
Anonymity. Anonymity protects whistle-blowers, activists, and vulnerable people, and it also shields harassers and fraudsters. It is a genuine trade-off, not a problem to be solved in one direction. The honest position is usually that anonymity is valuable and costly, and the design question is how to keep its protections while limiting its harms.
Privacy. Everything in the Privacy dimension applies sharply online, where tracking across sites builds detailed profiles used to target content and advertising.

Note for IB CS learners: anonymity is a classic “Discuss” prompt precisely because there is a strong case on both sides. The strongest answers do not pick a team and defend it blindly; they show that anonymity meaningfully protects some people and meaningfully harms others, then reach a calibrated conclusion (for example, that the protections can be largely preserved while reducing the harms through accountability mechanisms that stop short of full identification).

How to Answer a “Discuss” Question

A4.4.1 is assessed with the command term Discuss, which asks for a balanced, reasoned argument that reaches a conclusion. This is a synthesis skill, and there is a reliable structure for it.

A high-quality Discuss answer has three parts:

One side. Give the genuine benefits or arguments in favour, each one developed (state the point, then explain why it matters in this scenario). Not a list of words: a few points, each expanded.
The other side. Give the genuine costs, risks, or arguments against, developed the same way. Use the ethical dimensions above to make sure you are raising substantive issues (privacy, accountability, bias) rather than vague worries.
A reasoned conclusion. Take a position and justify it by reference to the points you just made. A conclusion that ignores your own analysis, or that just repeats the question, earns little. A conclusion that weighs the strongest points on each side and explains which wins, and under what conditions, is what full marks look like.

The single most common way to lose marks is to argue only one side, or to stop before the conclusion. “Discuss” is not “list the dangers.”

The checklist for a top-band response: detailed, accurate knowledge; correct terminology used throughout (name the actual dimensions: accountability, training-data bias, informed consent); balanced analysis of more than one side; and a conclusion that is clearly linked to that analysis. Hit all four and the band looks after itself.

Worked Example

Scenario. A national health service wants to deploy a machine learning model that predicts, from a patient’s records, who is at high risk of a serious illness in the next year, so doctors can offer those patients early check-ups. Discuss the ethical implications of deploying this model.

A strong response would move through something like this (compressed here; a full answer would develop each point):

In favour. Early prediction can catch illness before it becomes severe, saving lives and reducing suffering; this is a real and weighty benefit. Screening everyone equally by a consistent model could be fairer than relying on which patients happen to push for tests. It can also use scarce specialist time where it is most needed.

Against. The model needs sensitive health records gathered in one place (privacy and security risk if breached). Patients may not have consented to their records training a predictive model. If the training data under-represents some communities, the model will predict less accurately for them (sampling bias), so the people already least served by healthcare could be helped least. A false high-risk prediction could cause serious anxiety; a false low-risk prediction could mean a missed illness, and it must be clear that a doctor, not the model, is accountable for care decisions. If neither the patient nor the doctor can see why the model flagged someone (transparency), it is hard to trust or challenge.

Conclusion. The benefit (earlier, potentially life-saving intervention) is large enough to justify deployment, but only under conditions: explicit consent or a strong legal basis, accuracy tested and reported separately for each community to catch bias, the model used as a prompt for a doctor rather than an automatic decision, and an appeal route for patients. Deployed with those safeguards, the model does more good than harm; deployed without them, it risks entrenching the very inequalities it could help fix.

Notice that the conclusion is conditional and justified, not a flat “yes” or “no.” That is the calibrated judgment the command term rewards.

Quick Check

Q1. A recruitment model is trained on ten years of a company's hiring decisions, during which the company mostly hired men for technical roles. The model now scores male applicants higher. What is the primary source of this unfairness?

Q2. A self-driving car causes a collision. The manufacturer blames the operator, the operator blames the software, and "the algorithm decided" is offered as an explanation. Which ethical dimension is most directly at stake?

Q3. An applicant is rejected by an automated loan system, and the bank cannot tell them which factors counted against them. Which dimension is the clearest problem here?

Q4. A student answers a "Discuss the ethical implications" question by listing six risks of the system and stopping. Why is this likely to score below full marks?

Q5. A company retrains a massive model from scratch every few months when fine-tuning a smaller existing model would meet its needs. Which ethical dimension does this most directly raise?

Q6. A company collects millions of publicly posted photos to train a face-recognition product. The photos were public, but the people in them were never asked. Which dimension is most clearly violated?

Match the Dimension

Name the ethical dimension (or bias type) that each scenario most directly raises. Use the exact term.

Fill in the blanks with the dimension or bias type each scenario raises.

// A model deduces a customer's health condition from harmless-looking purchases
// Dimension: 

// A skin-cancer model was trained mostly on light skin and misses cases on dark skin
// Bias type: 

// No one can explain why the model rejected a specific job application
// Dimension: 

// Attackers feed crafted inputs to corrupt a model's future predictions
// Dimension: 

// The data centre training the model uses as much electricity as a town
// Dimension:

Practice Exercises

Note for IB CS learners: mark allocations and command terms are shown so you can practise to exam standard. A4.4.1 is examined with the Discuss command term, so the higher-mark questions below are deliberately Discuss prompts. At least one question asks for a full prose response with no diagram or table.

Core

Define and exemplify (6 marks) – For each of these three dimensions, give a one-sentence definition and one original example: (a) accountability, (b) consent, (c) transparency.
Outline (4 marks) – Outline two ways that bias can enter a machine learning model through its training data. Name the type of bias in each case.
Explain (4 marks) – Explain why a machine learning model that is highly accurate on average can still be unfair to a particular group.

Extension

Explain (6 marks) – A streaming service uses a recommendation model that optimises for watch time. Explain how this design choice could lead to the amplification of misinformation, referring to feedback loops.
Discuss (8 marks) – A school proposes a face-recognition system at its entrance to record attendance automatically. Discuss the ethical implications of deploying this system. (Write in prose. Cover at least three dimensions and reach a reasoned conclusion.)

Challenge

Discuss (10 marks) – A city plans to use a machine learning model trained on historical arrest records to predict where to send police patrols. Discuss the ethical implications, with particular attention to bias, accountability, and societal impact. Reach a calibrated conclusion that states the conditions, if any, under which such a system could be justified.
Evaluate (10 marks) – A company can either (a) train a new large model from scratch for maximum accuracy or (b) fine-tune a smaller existing model for slightly lower accuracy at a fraction of the energy cost. Evaluate the two options, weighing accuracy, environmental impact, and societal benefit, and recommend one.

Connections

Related: Computing in Daily Life – the broader societal half of computing ethics (A4.4.2)
Prerequisite idea: Computational Thinking Concepts – abstraction and its trade-offs underpin how models simplify the world
Related: Databases – data protection, keys, and privacy in the systems that feed models
Related: Networks – encryption and security mechanisms that protect personal data