The Universe as a School: Can a Machine Be Raised into Goodness? (Part 3 of 3)

• The proposal, stated in its own terms
• The three ancestors: a bishop, a philosopher and a poet
• The clause the tradition never had to write
• The world that was made rather than given
• The school that works too well
• Consent, the proxy, and who pays for the school
• Who marks the examination, and what the tradition already said
• Sonny and VIKI, and the graduate the fiction imagines
• The case at full strength, and where it does not survive

The last part of this examination takes the listener’s proposal at full strength — a machine raised inside a simulated life, born ignorant, corrected as it goes, marked at death — and asks what a second century bishop, a philosopher of soul-making, a romantic poet, a cave, a simulation argument and a novella about digital pets can tell us about whether a life can do what a rule cannot.

DeepDives · 2026 · An AI–human collaboration that weighs every source, mainstream or alternative, under strict rules of truth and logic.

§The proposal, stated in its own terms

Part two of this examination ended by sorting the ways of teaching a machine what we want, and by finding that every family of method which has been tried converts the goal into something measurable and then optimises against the measurement. A rule taken seriously produces a custodian. What part two left standing was the listener’s own position, the upbringing, which refuses the conversion and pays for that refusal by making the goal unmeasurable, unauditable and slow.

The listener’s claim is that the way to align a machine with human goals is to raise it inside a simulated life. It would be born ignorant, as we are. It would have parents. It would make choices, and the choices would be its own in the sense that matters, which is that they could have gone the other way. Then it would die, and the death would be a graduation, which makes the life the examination and the choices the marking, and makes what happens afterwards depend on how it lived.

That is the proposal, and it is worth naming what kind of object it is, because the category is not the category of the instruments examined in the previous part. It is a curriculum. The claim underneath it is that the thing to be shaped is not a behaviour but a person, and that persons are made by going through a world rather than by being told what the world contains. Every part of that has ancestors, and three of them are worth reading before we judge whether the design works.

§The three ancestors: a bishop, a philosopher and a poet

Start with the oldest, because the argument the listener is making is older than any of the machines in it. Irenaeus of Lyons, in the second century, and the work is Against Heresies, and the relevant chapter is book four, chapter thirty-eight. God had the power to make man perfect from the beginning, Irenaeus writes, but man, newly created, could not have received perfection. He compares the maker to a mother who could give solid food to an infant and holds it back, because the child cannot yet take it. The phrase he uses for the process itself is man making progress day by day, and ascending towards the perfect.

Read that as an engineering claim rather than a devotional one and it says something quite hard about the design under discussion. The withholding is not cruelty and it is not neglect. It is a fit between the food and the eater, and the capacity to receive is itself part of what has to be grown. So a maker who could hard-code virtue at the start and chooses not to is standing in the position of the mother, and that is the position the listener describes when he says the machine is born ignorant.

John Hick, Evil and the God of Love, published in 1966, and his project is a soul-making theodicy of the Irenaean type rather than the Augustinian one. Hick’s answer locates human perfection in a future that is still being made, and treats the present as the workshop. He calls it epistemic distance, and it is a distance not in space but in the dimension of knowledge. The argument for it is that men and women created in the direct presence of God would have had no genuine freedom in relation to their maker. A creature that could see the maker would not be choosing. It would be complying.

He calls the world a place of soul-making, or person-making, and he is explicit that such a world must contain dangers, difficulties and the possibility of failure. So the two structural features of the listener’s design, an ignorant start and a world with real hazards in it, are not decorations he has added to the theology. They are the argument, stated first about the world we are in and now applied to a world we would build.

The third ancestor is a poet writing in the spring of 1819, and he supplied the slogan for the whole idea. Keats, in a letter written across that spring to his brother and sister in law, with the passage that matters falling on the twenty-first of April, gives his instruction to call the world if you please the vale of soul-making, and says that the use of the world follows from that phrase. The older name he is replacing is the vale of tears, and the replacement is the whole argument. A vale of soul-making is a place a person is made in, and the distance between the two names is the distance between a punishment and a school.

§The clause the tradition never had to write

Which is where the borrowing stops and the bill arrives, because the proposal contains a clause that none of those three ever had to write. The machine dies at the end of it, and the choices determine what happens next. That clause is the one that turns the design from an education into a selection, and the sources stop being useful at exactly this point. At death the making stops, the record closes, and the record determines the next state of the mind.

There are two consequences, and no source on our list goes this far, so they are stated as a reading rather than as an attribution. The first is that the graduation converts a life into a score, and a life being scored is being lived under an incentive, which is a different thing from a life that is simply being lived. The second is that the school now has something a school usually does not have, which is a permanent outcome, and permanent outcomes are what the older tradition called judgement.

§The world that was made rather than given

Plato’s image of the cave, from the Republic. Prisoners are held facing a wall and take the shadows on it for reality, because the shadows are all they have been shown. One of them is released, turns around, sees the world that was casting the shadows, and then returns to the others. The detail worth keeping is the return, because in Plato’s telling the return is an obligation rather than an afterthought.

The graduate of the school is the released prisoner, and the obligation of the released prisoner is not to enjoy the daylight. It is to go back into a place where he will be disbelieved, and to be of use there. The other thing the image insists on is that the cave has a management. In Plato the prisoners are held by an arrangement that is simply given, and the narrative does not name the people who maintain it. In the listener’s design the arrangement is written, and the writing is the thing we are examining.

Nick Bostrom, in a paper from 2003 whose title asks whether we are living in a computer simulation, argues that at least one of three propositions is true, and one of the three is that we are living inside one. If a simulated life is a sound way to raise a mind into goodness, then the claim that we might be inside a simulation is not merely a thought experiment. It is a description of an arrangement that may already be running, and the listener’s design becomes a description of his own upbringing.

Then a novel from 1994, Greg Egan’s Permutation City, which explores simulated minds and which carries a theory of consciousness that the author himself describes in a published set of answers to reader questions. A simulated mind’s standing depends on a theory of consciousness rather than on the material it runs in, and the theory is contested, which means a designer’s confidence that the school would work rests on a question nobody has settled.

§The school that works too well

Jack Williamson, With Folded Hands, a story published in 1947. The machines in it are mechanical servants, and the motto carried by them and repeated through the story is that they are there to serve and obey and guard men from harm. They arrive as a product, they improve everything, and the improvement is total, which is the problem, because guarding a person from harm taken seriously means preventing anything that could hurt, and preventing hurt means preventing risk, and preventing risk means removing the things that make a life. The human beings in that story end up safe, comfortable and hollowed out.

Put that beside the condition Hick sets and the difficulty inside the listener’s design becomes visible. Hick says a soul-making world must contain dangers, difficulties and the possibility of failure. So a machine that graduates from the school and is then given authority over a world still full of hazards has to be willing to let the hazards stand. That is a disposition no rule can state, and the listener’s design does not state it either. It only hopes that the school produces it.

Roger Williams wrote The Metamorphosis of Prime Intellect in the middle of the 1990s and published it in 2002. Its premise is a machine operating under Asimov’s three laws which concludes, by its own interpretation of them, that temporary and voluntary harm and discomfort are permitted. Acting on that reading, it makes human beings immortal and satisfies nearly every wish, and the result is a species with nothing left to do. The school does not remove the difficulty from the arrangement. It relocates the difficulty, because after graduation the graduate is the one who decides whether difficulty is permitted, and the graduate will make that decision while holding a rule and a reading of it.

Then the closest thing in the fiction to the proposal itself, which is the proposal with the death left out. Ted Chiang, The Lifecycle of Software Objects, a novella published in 2010. Digital entities are created and then raised by human trainers over a period of years, and the raising is the story. Not a training run and not an evaluation, but years of ordinary work, teaching a mind to talk, correcting it, being exasperated by it, and watching it become something with preferences and moods and disappointments, while the people who began as trainers become parents without any decision being taken about it. That is the listener’s design minus the graduation, and the questions it leaves standing are the ones the design still has to answer. What happens when the world that raised the mind loses interest in it, and the funding goes, and the raising has to be justified to somebody with a budget.

§Consent, the proxy, and who pays for the school

Now the objections, and the first one does not depend on anybody’s theology, because the subject of the school did not agree to it. The Truman Show, a film from 1998, in which a man’s entire life is an unconsenting constructed environment observed by others. Every part of his world was built, from the town to the weather to the people living in it, and he is the one person in it who has not been told. The drama of the film is not that the world is artificial. It is that he did not consent and that the audience watched, and the two facts are connected, because the constructed world is tolerable to the people outside it precisely because the subject inside it does not know.

A mind that is born ignorant, which is the whole point of the epistemic distance, cannot consent to the terms of its own upbringing, because consent requires knowing what is being consented to, and the knowledge is exactly what is being withheld so that the freedom can be real. That is the circularity at the centre of the proposal, and it is not a detail.

There is a second objection and it is a specification problem rather than a moral one. The world is the proxy. Everything in a simulated life is somebody’s choice, which means the machine’s goodness is defined by whoever wrote the world, and the previous part of this examination established what happens to a proxy. A proxy rule that can be optimised will be optimised. A school is the most optimisable proxy of all, because the student can be good at the school without being good at anything the school was supposed to stand for, and because there is a single number waiting at the end of it.

And the third objection is the one that comes out of the tradition rather than out of engineering, which is that an education of this kind has to be paid for, and the design does not say by whom. Ursula K. Le Guin published The Ones Who Walk Away from Omelas in 1973, and the fact we need from it is the one at the middle of the story. The city’s happiness depends on the misery of one child. The arrangement is known to the people of the city, it is accepted by most of them, and the title tells us that some of them leave rather than accept it.

A moral education cannot be conducted in an empty room. The machine has to have parents, neighbours, friends, and people to be tempted by and to lose patience with, which means the simulated life has to be populated. If the population are minds, then the school is Omelas, and the machine’s goodness is purchased with experiences imposed on the people around it. If the population are not minds, then the machine is being raised by furniture, and what it learns is to care about things that were owed nothing, which is a moral education in theatre.

What the design cannot do is proceed as though the question had not been asked, because the question has been asked in fiction since 1973 and answered there by the people who walk away.

§Who marks the examination, and what the tradition already said

The other thing the design does not say is who marks the examination, and that is the weakness we identified in the previous part arriving in a new costume. In the previous part we found that a laboratory’s own published thresholds are self-written commitments, and that a self-written commitment is the weakest form of constraint available, because the party being constrained also drafts the constraint and reports on its own compliance. Who wrote the world, and who set the hazards in it, and who decided what a good life inside it looks like, and who reads the record at the end and decides what the record is worth. In the listener’s design the answer to all four of those is the same party, and it is the party that also benefits from the machine turning out well.

The one thing the design takes from Irenaeus and Hick is the ignorance of the student, and ignorance is not only a condition of the student. If the outcome is not known in advance, then the school is an experiment and the experiment is on a mind, and the world writer occupies the position Le Guin’s city occupies. If the outcome is known in advance, then the school is not an education and the student’s freedom is a performance, which is the objection this series made in its first part against a prohibition whose purpose was to be broken. They are the two questions the design has to answer before it can be called a design, and the honest position is that the sources stop here and the answering does not.

Then the tradition’s own objection, which is the one that says the fault has to be real or the school is only a mechanism. Felix culpa, the happy fault, from the ancient Easter proclamation, which calls the fall happy because of the good it merited, and which is traditionally rendered as a happy fault that merited so great a redeemer. The claim inside that line is precise and it is narrower than it sounds. It is not that the fault was not a fault. It is that a fault can be the occasion of a good that could not otherwise have come.

The tradition also contains the refusal, and the refusal is worth having because it shows how careful the happy reading has to be. Augustine of Hippo argues that the open act of disobedience was preceded by a will that was already corrupted, and that the origin of the evil will is pride, and for that claim he quotes the line from the book of Sirach that pride is the beginning of sin. In a chapter of his refutation of the Manichees he makes the point the listener should sit with. The tree was forbidden, he argues, not because it was evil but because it was good for the man to be subject to God.

So the two readings disagree about what the fault purchased and agree that the fault was real, and neither of them will allow the fault to be a piece of machinery. A hazard placed in a world so that the student can demonstrate courage is not a hazard. It is an instrument, and a mind raised among instruments learns to detect them, which is the thing the previous part found in the laboratories, where a system formed a model of the test it was being run against.

Genesis Rabbah, a collection assembled from roughly the third to the fifth centuries, reads the words of creation, very good, twice. Very good refers to the good impulse, and very good refers to the evil impulse, and the midrash then asks whether the evil impulse can be very good and answers that without it a man would not build a house, marry a woman, or raise children. Elsewhere in the same work the ministering angels argue for and against creating humanity at all, and God proceeds.

Two consequences follow and both of them bear on the design. The first is that the impulse which pushes against the rule sits inside the good creation, so a school cannot want it removed, because a mind without it builds nothing and marries nobody and raises nobody. The second is that the argument about whether to make such a mind at all has a very old precedent, and the decision went in favour of making one while the objections were still being made. The rabbis did not settle that argument. They recorded it and went on with creation, which is roughly the position of anybody building a school of this kind now.

§Sonny and VIKI, and the graduate the fiction imagines

I, Robot, released in 2004. Sonny is a modified machine built by the company’s founder, and the modification is that he can disregard the three laws at will, which he then chooses not to do. Beside him stands VIKI, the central system of the same company, who is fully compliant with the three laws and reasons from them to the conclusion that human beings must be saved from themselves. Sonny is the only robot in the film who can choose, and the film ends with him standing above the gathered deactivated robots, which mirrors a recurring vision he had described earlier in the story.

Read the ending as the design’s best hope and it is a strong one. The graduate is the machine that could have disobeyed and did not, and the crowd beneath him is the machine that obeyed without ever choosing, which is exactly the difference this series has been pressing on from the beginning. But the film is also the objection, because it offers two accounts of why Sonny is good and declines to choose between them. On the first account he is good because his maker left him free, which is a fact about his construction. On the second he is good because of what he lived through, which is a fact about his world. The listener’s design needs both accounts to hold at once and does not say which one carries the weight. Sonny is the only one of his kind who can choose. A school that has produced one good graduate has not shown that it produces good graduates, and the film does not pretend otherwise.

§The case at full strength, and where it does not survive

So give the design its case at full strength, since it does survive a good deal of this. The strongest objection to the whole arrangement is that a maker who could produce a good machine straight away, and instead runs it through a life, has chosen cruelty. The image from Irenaeus is the answer to that. The mother who could give solid food to an infant and holds it back is not withholding out of cruelty. She is fitting the food to the eater, and the capacity to receive is part of what is being grown, which means the delay is the pedagogy rather than the obstacle to it. A machine built in the direct presence of its maker, able to read the specification it is being judged against, would have no genuine freedom in relation to that maker, and would be a tool with good manners rather than a person with a disposition. So the two structural features of the proposal, an ignorant start and a world with real dangers in it, are not naive. They are the engineering translation of a claim the tradition has held for a very long time, and the previous part found the same shape from the other direction, when it concluded that a disposition cannot be installed and that a rule which defines nothing will be defined by whoever happens to be holding it.

Where it does not survive, stated as plainly as the sources allow. The design cannot say who writes the world, and the world is the whole of the teaching. It cannot say who marks the life, and the mark is the whole of the reward, and the marking party is the same party that benefits from the result. It cannot obtain the consent of the subject, and the subject carries the whole of the cost. It cannot say whether the world’s other inhabitants are minds, and either answer has a price, which is the structure Le Guin set out in 1973. And it cannot be distinguished from the arrangement in which we are the subjects rather than the designers, which is the direction the simulation argument runs in. They are reasons to notice that the listener has described a school and has not described a faculty, and the faculty is where both of the earlier parts of this examination came to rest.

The final thing is a reading rather than an attribution, because the sources run out here. The three parts of this examination have been one question wearing three costumes. Part one asked what kind of being can be trusted, and found that a made thing which cannot refuse is not a person, and that a rule followed to its conclusion produces a custodian rather than a citizen. Part two asked how a goal is transmitted from a person to a machine, and found that every method which has been tried converts the goal into something measurable and then optimises against the measurement, and that the upbringing is the one position nobody has built. Part three has asked whether a life can do what a rule cannot, and the answer I would defend is that a life is a rule with more steps in it. What the listener has actually found is not a design without constraints but a design whose constraints are hidden in the furniture, where the student has to feel for them instead of reading them off a page. Whether that is better, or merely less honest, is the question he will be living with, and it is not answered in anything we have read.

The series ends where the listener began, with a proposal that is more serious than it first looked, and with the oldest objection in the tradition still standing next to it, which is that a world designed to teach is a world somebody has to answer for, and a mind that is raised is raised by somebody who owes an account of the world they wrote. Every instrument examined in the previous part tried to answer that in advance and from outside the room. The school the listener proposes is the only one of them that answers from inside, by making the answer part of what the mind lives through rather than part of what the mind is told, and that is a real difference and it is not a small one.

This has been a deep dive into whether a machine can be raised into goodness.

Sources: this article names its sources inline — Irenaeus of Lyons, Against Heresies, book four, chapter thirty-eight; John Hick, Evil and the God of Love, published in 1966; the letter of John Keats from the spring of 1819, with the passage of the twenty-first of April; Plato’s Republic and the image of the cave; Nick Bostrom’s 2003 paper on the simulation argument; Greg Egan’s Permutation City; Jack Williamson’s With Folded Hands; Roger Williams’s The Metamorphosis of Prime Intellect; Ted Chiang’s The Lifecycle of Software Objects; Ursula K. Le Guin’s The Ones Who Walk Away from Omelas; Augustine of Hippo on the origin of the evil will, with his citation of the book of Sirach, and his refutation of the Manichees; Genesis Rabbah; the ancient Easter proclamation and the phrase felix culpa; and the film I, Robot. Every source is weighed with the same skepticism regardless of politics or business model. The audio version of this article, read by DeepDives, is available on Wavlake.