While leading the Dendarii Mercenaries on a hostage rescue mission, Miles Vorkosigan has a seizure — a recurring consequence of his death and resuscitation in Mirror Dance — which results in his accidentally (and non-fatally) severing the rescued hostage's legs with his weapon. Terrified of the consequences, Miles falsifies his mission report to his boss, Simon Illyan, the head of Imperial Security (ImpSec). Illyan finds out anyway, and Miles is forced to accept a medical discharge.

Miles becomes seriously depressed. His cousin Ivan Vorpatril, with the help of his friend and ImpSec Captain Duv Galeni (who encountered Miles during the events of Brothers in Arms), manage to get him to break out of his funk.

Duv Galeni is enamored of Laisa Toscane, a very wealthy heiress and member of a Komarran economic delegation. Miles gets them invitations to a party hosted by Emperor Gregor, hoping to help him to score points with Laisa, but unexpectedly she and Gregor fall in love. When their engagement is announced, Galeni is furious with Miles, calling him on a monitored line to express his rage at Barrayarans in general and the Vor class in particular.

When Illyan suffers a sudden, crippling mental impairment, Miles attempts to investigate, but receives no cooperation from Lucas Haroche, ImpSec's acting chief, so he asks Gregor to assign him an Imperial Auditor, a troubleshooter answerable only to the Emperor. Gregor unexpectedly decides that it would save many steps (and his time) by making Miles himself a temporary Auditor, even if Haroche considers Miles a prime suspect in Illyan's impairment.

Illyan's breakdown was caused by a malfunction in the memory device implanted in his brain when he was a young lieutenant. The device begins dumping random sets of memories into Illyan's mind at an accelerating pace, causing him to believe he is at different points in his life. After consulting with Gregor, Miles orders ImpSec's doctors to remove the device as soon as possible. Then he sets out to find out whether the breakdown was natural or artificial. He recruits "Dr. Waddell", formerly Hugh Canaba, a biotechnology expert he extracted from Jackson's Whole (in "Labyrinth"). Waddell discovers the culprit is a synthetic biological agent specially designed to target Illyan's implant.

Illyan moves into Vorkosigan House to recover, where he receives visits from Lady Alys Vorpatril, Gregor's social secretary, who was also at his bedside during his breakdown.

While making a thorough inspection of ImpSec headquarters, Miles finds the first clue: a false record that claims he entered ImpSec's storeroom recently. An inventory establishes that the weapon was commissioned by Ser Galen, Duv Galeni's now-deceased Komarran terrorist father. It was stolen by someone trying to frame Lieutenant Vorkosigan, not Lord Auditor Vorkosigan.

Thanks in part to his ill-advised call to Miles, Duv Galeni is arrested for the attack. However, Miles is certain that this is also a frameup. When Haroche offers to reinstate Miles and put him back in charge of the Dendarii Mercenaries, Miles realizes that he has been offered a (nearly irresistible) bribe and that Haroche committed the crime to become head of ImpSec. He maneuvers Haroche into incriminating himself. Gregor consults with the other Auditors and makes Miles' appointment permanent.

Miles makes one final attempt to persuade his second-in-command and lover, Elli Quinn, to marry him, but she cannot bear being planet-bound, so he gives her command of the Dendarii. Gregor and Laisa become betrothed. Illyan and Alys, meanwhile, have become secret lovers, to Ivan's dismay. Duv Galeni becomes engaged to Delia Koudelka.

Miles undergoes surgery to implant a device that can pre-emptively trigger a milder seizure at a time of his choosing as a palliative measure, the underlying condition being incurable.

Memory was nominated for the Hugo, Nebula, and Locus Awards for Best Novel in 1997.^[1]

In the New York Review of Science Fiction (October 1998, number 122), the novel is summarized as follows:

In force and intensity as well as this elegiac undertone, Memory is indeed a quantum jump ahead of The Warrior's Apprentice, although in Miles' double trajectory it fills the same position of excursion from "childhood" and foundation of a new personality. But where Mark subsumes and reconciles himself to his dark internal Others, Miles' integration is achieved by an excision - or, as Bujold put it, a "repossession" ("Letterspace", Letter 8^[2] ), with all the word's overtones, theological as well as financial - and a metamorphosis at very high cost. Nor is the cost limited to the characters. Reading Memory, I myself felt very much like Wordsworth seems to have when he wrote "Ode on the Intimations of Immortality": what we had here was remarkable, spectacular, far more powerful than Apprentice and its ilk, but it was also darker, less sparkling, without that adolescent, outrageous joie d'esprit.^[3]