Gieve served as Permanent Under-Secretary of State at the Home Office from 2001 to 2005. At the time, the Home Office was the government department responsible for law and order, including prisons and probation, policing, oversight of MI5, and immigration and nationality matters. As Permanent Secretary, Gieve was in charge of the Department's administrative functions and its civil servants, working to the Ministerial team headed by the Home Secretary. During Gieve's tenure at the Home Office, there were three Home Secretaries; Jack Straw (until 2001), David Blunkett (2001–2004), and Charles Clarke (from 2004). Blunkett was obliged to resign from government after a scandal involving accusations of abuse of his official position and misuse of government funds.

On 31 January 2006, after Gieve had left, the UK National Audit Office published a report, Home Office: 2004-05 Resource Account,^[2] which was highly critical of the Home Office's accounts during the period of Gieve's tenure; the accompanying press release stated that:

"Sir John Bourn, head of the National Audit Office, reported to Parliament today that the Home Office had not maintained proper financial books and records for the financial year ending 31 March 2005. Sir John Bourn therefore concluded that, because the Home Office failed to deliver its accounts for audit by the statutory timetable, and because of the fundamental nature of the problems encountered, he could not reach an opinion on the truth and fairness of the Home Office’s accounts".

Later in 2006, Charles Clarke was dismissed as Home Secretary and replaced by John Reid, who shortly after his appointment made a statement to Parliament in which he described the Home Office as "unfit for purpose".

In early January 2006 Gieve was appointed as the new Deputy Governor for Financial Stability at the Bank of England.^[1] The appointment was for a five-year term, and carried with it membership of the influential Monetary Policy Committee.

On 30 April 2006 the UK's The Independent newspaper reported that Gieve was being pressed to resign from the Bank of England because of financial mis-management at the Home Office during his period as Permanent Secretary, and his involvement in the released prisoners affair.^[3] There was concern that Gieve was not a City insider and that he did not have sufficient technical knowledge for the role. It is not clear who was "pressing" for his resignation - just an assertion in a newspaper.

On 18 June 2008 the Bank of England announced that Gieve would be stepping down in 2009, two years early, after his work to reform the Bank to take on formal and legal responsibility for financial stability was complete.^[4]

Gieve was educated at Charterhouse School in Godalming, Surrey and New College, Oxford, where he read first for a BA degree in Philosophy, Politics and Economics, and then for an MPhil degree in Philosophy. He joined the Civil Service in 1974, and has served in a number of departments. Privately, Gieve is known as a keen cyclist, footballer and golfer, and as a loyal supporter of Arsenal Football Club. He is married with two sons.

He was made a Companion of the Civil Division of the Order of the Bath (CB) in 1999, and promoted as Knight Commander of the same Division and Order (KCB) in the 2004 New Year Honours list.^[5]

John Gieve is also an Honorary Fellow of Regent's University London.