Whitlam was born in Prahran, a suburb of Melbourne, and was educated at a local state school before winning a scholarship to Wesley College, Melbourne. In 1900 he took first place in the Victorian Public Service clerical examination and joined the Department of Lands and Survey. After Federation he transferred to the Commonwealth Public Service, joining the Commonwealth Crown Solicitor's Office. In 1911 he moved to the land tax branch of the Treasury, where he employed the young John McEwen (later Prime Minister) as a clerk. He studied at the University of Melbourne, graduating in law in 1914.

Whitlam married Martha ('Mattie') Maddocks in September 1914.^[1] Their house at 'Ngara' 46 Rowland St. Kew in Melbourne was built for them by Edward Maddocks, Mattie's father. Future Prime Minister Gough Whitlam was born there in 1916, and Freda Whitlam in Sydney in 1920 – she became principal of Presbyterian Ladies' College, Sydney and later moderator of the New South Wales Synod of the Uniting Church.

In 1918 Whitlam transferred to the Sydney office of the Crown Solicitor's office, and in 1920 he was admitted as a barrister and solicitor of the High Court of Australia. He became deputy Crown Solicitor in 1921, assistant Crown Solicitor (based in Canberra) in 1927, and Crown Solicitor in December 1936. In this position he was senior legal adviser to the government for 12 years, and his views were respected and influential. Cameron Hazlehurst writes:

On the Lyons government's controversial national insurance initiative, for example, he drafted legislation for the National Insurance Commission, recommended the appointment of J. B. Brigden as chairman, and drew up the agreement between the commission and the Australian branch of the British Medical Association. He also briefed W. R. Dovey, his son's future father-in-law, as counsel assisting the subsequent royal commission. Closer in political sentiment to John Curtin and J. B. Chifley than to their predecessors, Whitlam was largely responsible for preparing the documentation for the 1944 referendum on Commonwealth powers and, with the solicitor-general, for advising H. V. Evatt during the bank nationalisation litigation (1947–49).^[2]

As a public servant, Whitlam had no formal involvement in politics, but he was active in civic and community affairs in Canberra, then a small and isolated town, and was also active in the local Presbyterian Church. In 1933 he led a campaign against Canberra residents being required to pay a hospital tax when they had no elected local government and no parliamentary representation. He was known to have pro-Labor views. In a 1973 interview, Gough Whitlam said that had his parents been British, they would have been Liberals. "In the Australian context they would vote Labor as the party of change and public responsibility – things being done by elected persons rather than by self-perpetuating directorates."^[3] Whitlam was also a friend of Evatt, who was Attorney-General in the 1941–49 Labor government and later Leader of the Opposition.

Whitlam bought one of the first blocks of land offered for public auction in Canberra in the Blandfordia auction that occurred on 12 December 1924. He paid $440 for Block 1, Section 2 now known as 70 Empire Circuit (formerly Australia Circuit) and lived there for more than 30 years.^[4]

Early biographers of Gough Whitlam were quick to detect his father's influence:

"The key to Fred Whitlam's character was tolerance – he loathed any form of prejudice on grounds of class, religion or race – and his overwhelming preoccupation was human rights... Related to his concern for fair treatment of minorities and individuals was a deep interest in foreign affairs. Fred Whitlam was the driving force in the Canberra branch of the Institute of International Affairs in its early years... All this rubbed off on his son."^[5]

Whitlam was a pioneer of international human rights law in Australia, and this was the area in which he exercised his most powerful influence over his son Gough Whitlam's career. As a member of the Australian delegation to the Paris Peace Conference in 1946, Whitlam argued Australia's case for a permanent international human-rights court, an idea whose time was yet to come. "Instructed by Evatt not to compromise, he reported to his wife that he had 'stiffened the sinews and summoned up the blood', but to no avail.".^[2] He contributed to drafts of the 1948 Universal Declaration of Human Rights.

Whitlam retired as Crown Solicitor in April 1949, but continued to be closely involved in United Nations matters as an adviser to the Department of External Affairs. He was an Australian representative at the United Nations Commission on Human Rights in 1950 and 1954. He died in Canberra in 1961, by which time his son was Deputy Leader of the Federal Labor Party. Graham Freudenberg writes of Fred Whitlam's influence on Gough Whitlam:

"Whitlam's family background [in Canberra] and his father's career had three crucial influences on his thinking: on the role and nature of the Federal Government, the role and nature of the public service, and the problems of urban life in a new suburb."^[6]

Paul Hasluck, a public servant before becoming a Liberal politician in 1949 (and no admirer of Gough Whitlam), wrote of Fred Whitlam:

"I came to know him as a public-spirited, meticulous and dutiful man with an inquiring but cautious mind, who was always very concerned to make sure that whatever was done was right, both in the sense of legally unexeptionable and soundly based on principle. He was a good churchman. He was certainly not intolerant or censorious, but his meticulous concern about what was right, though it would lead him often to differ from the accepted views, sometimes made it seem that he was rigidly orthodox. He was held in very high regard and respect. He was kindly and modest."^[7]