Prior to the middle of the eighteenth century, the area was still undeveloped. Orange Street is first shown in a 1754 map as a two-block street running from the "High Road To Boston" (which later became Chatham St. and finally, Park Row), and ended at a small clearing where the later "bend" in the street would occur, which was at the time along the banks of the unnamed body of "fresh water" later known as the Collect Pond, with its marsh lying to the north. The future "Five Points" intersection was at this time a normal crossing of two streets, Orange and Cross, with Cross Street running from Mott Street (as it always would), to an unmarked "Little Water Street", and then ending at the banks of the pond. Anthony Street (which would complete the "fifth point" and was later renamed Worth Street) did not yet exist.^[2]

By 1797, the alignment north of the bend is shown, running all the way to Prince Street, but this section was called "Mary Street". A walkway had been built next to the street, along the pond and its marsh and running from the bend to almost Hester Street.^[3]

In the new century, the street (the whole length by now renamed "Orange Street", and the areas to the west of it built up) was shown beginning in a dead end north of Prince Street.^[4] An 1803 plan, however, showed it merging with Crosby Street at Houston Street.^[5] At Spring Street, Elm Street merged with Orange Street; and at Broome Street, Centre Street merged with Orange Street. The triangle formed by Broome, Orange, and Centre Streets was later the location of the original Centre Market.^{[citation needed]}

By 1850, the current alignment was set in place^[6] with Centre Market becoming a full block between Grand and Broome Streets, with the portions of the original street alignment north of Broome being connected only to Centre Street and renamed Marion Place (and is currently known as Cleveland Place, with Elm Street—now Lafayette Street—taking the alignment north of there and extending past the original dead end). The street on the east side of the Market, which was displaced a bit east of where Orange ends at Grand, also had taken on the name Centre Market Place.^{[citation needed]}

On the southern end, Orange Street always ended at Chatham Street. Past there, another street, slightly to the east, named Roosevelt Street, continued to the East River waterfront.^{[citation needed]}

The southern end of the street deteriorated into a slum, largely due to the infilling of the Collect Pond, which lowered property values, causing the middle class to move out, and poor immigrants and African Americans to move in.^[7] The area, particularly the street, eventually became known for gang violence.^{[citation needed]}

In between, the first bowling alleys also were opened on the street, behind the saloons at Nos. 51 and 63,^[8] and the tap dance was created by competing black and Irish dancers at a tavern at 67 Orange.^[9]

In 1854, to try to remove some of the stigma the area had taken on already, some of the primary streets were renamed,^[10] including Orange Street, which was then named after Charles Baxter, a state legislator who fought and died as a lieutenant colonel in the Mexican–American War.^[11][12]

Nonetheless, the area, and the street, would maintain its seedy reputation. Gangs included the Baxter Street Dudes, who ran the Grand Duke's Theatre from their headquarters on the street during the 1870s. When various artists and photographers (most notably, Jacob Riis) would capture the scenes of the Five Points intersection and the squalor of the area in the 1870s and 1880s, many Baxter Street scenes, including such residences as the "Dens of Death" would be seen.^[13]

When Baxter Street was part of Little Italy, it was home to many Italian immigrants from northern Italy, especially Genoa. After World War II, many of the Italians moved to Brooklyn, Staten Island, Long Island, Westchester and New Jersey.

Later, much of the Five Points area was cleared. The east side, the Mulberry Bend, was turned into the Columbus Park in 1895. The west side of the street, and the entirety of Baxter Street south of Worth Street, was demolished for the Manhattan Civic Center in the 20th century. South of Canal Street, Baxter Street's west side adjoins the rear of the New York City Criminal Court, which is lined with numerous law and bail bond offices; Baxter Street is heavily connected to police and the law, despite having a history rife with crime.^[14]

The street's past was portrayed in a play by New York playwright Barbara Kahn, The Ballad of Baxter Street, which premiered in 2005 at Theater for the New City.^[15] In the backdoor pilot of the Arthur spinoff Postcards From Buster, Buster Baxter and his father notice the street sign while walking in Chinatown and comment about how the Chinese characters spell their name.

• Five Points

• Anbinder, Tyler (2001). Five Points: The 19th Century New York City Neighborhood That Invented Tap Dance, Stole Elections, and Became the World's Most Notorious Slum. New York: The Free Press (Simon & Schuster). ISBN 0-684-85995-5.

• New York Songlines: Baxter Street
• Experiencing Baxter Street: Then and Now, Reality and Fiction