Fort George was one of a number of new forts (most built on the sites of earlier forts) housing coastal artillery built in the early and mid-Nineteenth Century within or satellite to St. George's Garrison. The heaviest concentration of coastal artillery batteries and fortifications in Bermuda had, and would continue to be, at the East End of the archipelago of Bermuda, where St. George's Harbour and Castle Harbour (with its own history of fortification) were the only harbours easily accessible from the open Atlantic due to the reefline surrounding Bermuda. After the American War of Independence, Bermuda had been selected as the only remaining British territory between Nova Scotia and the British West Indies, being also in a position (640 miles off North Carolina) from which to dominate the Atlantic seaboard of the new United States of America. The Royal Navy spent a dozen years surveying the reef, identifying a channel (originally called Hurd's Channel after then-Lieutenant Thomas Hannaford Hurd, the officer who had located it, but now referred to as The Narrows) suitable for ships-of-the-line to enter the Northern Lagoon, the Great Sound, and Hamilton Harbour before establishing a base at Bermuda in 1795. Although the Admiralty began buying and leasing land at Bermuda's West End, on the Great Sound, for the eventual construction of the Royal Naval Dockyard, the lack of development and infrastructure there meant the Navy used St. George's as its base pending the construction of the dockyard on Ireland Island.

Hurd's Channel led from Five Fathom Hole, the opening in the reefline through which vessels previously had already passed into the original main eastern channel (between Paget Island and Governor's Island, both of which held fortified coastal batteries from the early Seventeenth Centuries) into St. George's Harbour, around St. Catherine's Point at the north-eastern end of St. George's Island.^[5] A succession of forts named Fort St. Catherine's have existed on this point since 1612, and the fifth and last of these was built in the Nineteenth Century. On Retreat Hill to its rear, Fort Victoria and Fort Albert were built in the 1840s. Between them, the three guarded Hurd's Channel and the Northern Lagoon from navigation by an enemy.

To the rear of these three forts, a pair of identical forts were built in the 1650s on hilltops on either side of St. George's Town: Fort George was built at Mount Hill, on the western side of the town (the site of a succession of earlier defence works, beginning with a watchtower in 1612, rebuilt with the addition of a signal gun in 1619, to which the name Riche's Mount (or Rich's Mount) was originally applied). The first proper fort was built on the site in the 1790s, but this was replaced with the current fort; the Western Redoubt (originally intended to be named ‘’Fort William’’) was built on another hilltop on the northern side of the town, between Government Hill (from which Government House had moved to Mount Langton in Pembroke Parish with the move of the Parliament of Bermuda and the colonial capital from St. George's Town to the then Town of Hamilton in 1815) and Barrack Hill, the site of the Royal Barracks of St. George's Garrison.

Fort George and the Western Redoubt both overlook the harbour to the south, and to a greater or lesser degree the Northern Lagoon to the north of the island (the area of enclosed water immediately to the north of St. George's Island is called Murray's Anchorage, and had been the anchorage for the naval squadron of the North America Station when the naval base was at St. George's). The two forts therefore could operate also against enemy vessels entering the Northern Lagoon, but were primarily intended to fire on an enemy that entered the harbour and to provide rear defence to the forts on Retreat Hill and St. Catherine's Point.^[6]

Fort George was completed and armed, and later re-armed with more modern weapons, while Fort William was completed but never armed as it was deemed by then to be excess to need. Instead, named the Western Redoubt, it was converted into a magazine, which would have been subsidiary to the Ordnance depot on Ordnance Island.

Fort George was not one of those re-armed with modern muzzle loading rifles at the end of the Nineteenth Century. During the First World War, the it was used by the Pilot Service and the Royal Naval Examination Service, who operated a steamer from the harbour to meet arriving ships at Five Fathom Hole for inspection, before piloting them to harbour. Most of the coastal artillery batteries in Bermuda, excepting St. David's Battery, were either obsolete or reduced to a care and maintenance state that was not ready for war during post-First World War cutbacks to the British Army.^[7][8]

In 1957, a later phase of cutbacks resulted in the closure of the Bermuda Garrison, with all regular units and detachments, including the Command staff, withdrawn, leaving only the part-time Bermuda Militia Artillery and Bermuda Rifles. The dockyard was re-rated as a base, losing its ability to repair or refit its vessels, and this was to close in 1995. All Admiralty and War Office, including St. George's Garrison, land was transferred to the colonial government in 1958.

Today, all of the East End fortifications and batteries are parts of the UNESCO World Heritage Site the Historic Town of St George and Related Fortifications), though not all are open to the public.^[9]

The fort is square shaped, and the gunfloor originally had emplacements for four guns, one on each side behind an earthwork glacis. Inside of these was a higher keep, separated from the gunfloor by a dry moat, with a drawbridge. The original armament was four 64-Pounders on the gun floor, and four 24-Pounders atop the keep. In the 1680s, this was replaced with two RML 11 inch 25 ton gun (one facing the harbour and the other towards Murray's Anchorage) and two RML 64-pounder 58 cwt guns facing east and west, of which only the first pair remain, both on their original mounts.^[10] A Signal Station was also placed at the fort, with Semaphore signal mast. Others were placed at Mount Wyndham (Admiralty House), Mount Langton (Government House), Gibb's Hill Lighthouse, and the Royal Naval Dockyard. These provided a visual telegraph linking the various naval and military facilities and Government House until the introduction of an electric telegraph network in the 1860s, and were also used to communicate to vessels in visual range on the water. The signal mast no longer survives, though the one at Gibb's Hill Lighthouse does. The civil government built a wireless and radar station on top of the keep, utilised by Bermuda Harbour Radio and the Bermuda Maritime Operations Centre.^[11]