Born in Harrison, New York and raised in the South Bronx, Halley graduated from Townsend Harris High School at age 14, and was forced to wait until age 16 to enroll at Columbia University, from which he graduated with a Juris Doctor at age 20. After waiting until his twenty-first birthday to become eligible to pass the bar examination, he went into private practice. Fulton, Rowe & Hart LLP was formed in 1946 by Hugh Fulton, Rudolph Halley, and Henry G. Walter, Jr. Mr. Fulton was executive assistant to the U.S. attorney for the Southern District of New York from 1938 to 1941. Mr. Fulton and Mr. Halley were members of the 1941 U.S. Senate Special Truman Committee, created to investigate the national defense program (Mr. Fulton was its chief counsel), and the 1950 U.S. Senate Special Kefauver Committee, created to investigate crime in interstate commerce (Mr. Halley was its chief counsel). Mr. Halley was President of the New York City Council from 1951 until 1953, and in 1953 he ran for mayor of New York City, finishing in third place (Robert Wagner won the election). . During this time, he married and divorced twice. In 1941, he went to work for the U.S. Senate Special Committee to Investigate the National Defense Program, better known as the "Truman Committee" for its chairman, then-Senator Harry S. Truman, which investigated fraud and waste in defense contracting during World War II.

In 1950, Halley was named Chief Counsel to the Senate Special Committee to Investigate Crime in Interstate Commerce, better known as the Kefauver Committee, which was charged with investigating the influence of organized crime, particularly its involvement in gambling and political corruption. In his role as Chief Counsel, Halley personally questioned every witness called to testify before the Committee. The Committee's hearings, which were televised nationally, made Halley a celebrity. On May 1, 1951, shortly after the Committee concluded its hearings in New York City, Halley announced his resignation. Over the course of the next several months, he translated his celebrity into work in television, narrating the CBS crime drama Gang Busters and hosting the documentary program Crime Syndicated. He also wrote a short-lived column for Hearst Newspapers.

In the summer of 1951, Halley announced his candidacy for President of the New York City Council, a position that would later be replaced by that of New York City Public Advocate, in a special election held to replace Vincent R. Impellitteri, who had ascended to the mayor's office after the resignation of William O'Dwyer. He ran on an anti-corruption, anti-crime, anti-Tammany Hall platform and promised to "teach the political bosses a lesson." Although a lifelong Democrat, he did not seek the party's nomination, instead running as the nominee of the Liberal Party. He also appeared on the ballot as the nominee of the Fusion Party and the Independent Citizens Party. On Election Day, Halley unexpectedly triumphed, tallying 657,871 votes (39%).

As President of the City Council, Halley was best known for feuding with both Mayor Impellitteri and Governor Thomas E. Dewey over state funding for the City, which was necessary to balance the municipal budget. While Dewey demanded increases in property taxes and the subway fare in return for state aid, Halley favored leaving both taxes and fares alone and instead cutting government waste. Impellitterri opposed both plans. Things turned ugly when Halley accused Dewey of "ruthlessly playing politics" with the budget, and Dewey publicly called Halley "as stupid and ignorant as he is shallow and venomous."

In 1953, Halley declined to run for re-election and instead declared his candidacy for Mayor of New York. Running once again on the Liberal and Independent Citizens lines, he ran third with 467,104 votes (21%), behind Democratic Manhattan Borough President Robert F. Wagner, Jr. and Republican attorney Harold Riegelman. (Halley beat Riegelman for second place in the Bronx and nearly did so in Brooklyn.)

After the election, Halley retired from politics, becoming a name partner in the law firm of Fulton, Walter & Halley, headquartered at Rockefeller Center. Although there was speculation that he might run for Attorney General of New York in 1954, he never sought office again. On November 19, 1956, Halley died in Manhattan of natural causes at age 43. His death has been attributed to both pneumonia and pancreatitis. He was survived by his wife and four children (Marian, Henry, Peter and Michael).

• Gill, Brendan, The Talk of the Town, "Romantic," The New Yorker, February 27, 1954
• Moritz, Owen, "Rudolph Halley: Streak of Light," New York Daily News, June 24, 1999
• Paid notice, New York Times, November 19, 2006