Fiskavaig was subject to Highland Clearances and this was begun by Dr. Lauchlan MacLean as part of the North Talisker estate clearance from 1818 to 1825 when Hugh MacAskill took over the estate in 1825. Fiskavaig was largely cleared c.1827. Along with Ardhoil (which no longer exists) this consisted of 10-12 families. A witness for the Napier Commission report in 1883 when asked what happened to the residents of Fiskavaig stated: "Some of them went to America, and others were scattered about the country."^[4]

A 'Deer Forests Commission' report of a survey of the Talisker sheep farm in May 1893 notes a "nondescript cottar fisherman who lived at the side of Fiskavaig Bay" as one of the few inhabitants of the area between Fernilea round to Talisker.

Fiskavaig was re-populated in the summer of 1923 (along with Portnalong and Fernilea) when the 1919 Land Settlement Act aimed to resettle populations following the end of the First World War, through the creation of smallholdings and crofts. The 1923 restoration of the North Talisker area was made possible when the Scottish Board of Agriculture acquired a 60,000 acre area of land belonging to Norman MacLeod of MacLeod, part of the North Talisker Sheep Farm. This land was compulsorily purchased for £58,609 and allowed the provision of 68 crofts to be offered to families, of which in 1924: "43 are tenanted by Harrismen, 20 by Lewismen and 5 by Skyemen. There is a total population of approximately 400."^[5]

Fiskavaig Bay sits on the southern shore of Loch Bracadale and is bordered on the west by Sgùrr nan Uan (Lamb Craig) and to the east by Coille Ghuail (Coal Grove).

A small river, Alt Ribhein (meaning 'Ribbon River') runs into the bay at its south-east while Fiskavaig Burn makes its way into the bay at its west.

The bay is shallow and exposes dark sand flats and tidal pools at low tide (along with the remains of two stone fish traps, one central and one on its eastern side) with high tide moving back to the rocks & rock pools at the foot of the bay.

The island of Wiay is 1.5 miles offshore into Loch Bracadale, as is the tidal island of Oronsay,. Also visible are Tarner Island & Harlosh Island.

Fiskavaig overlooks Healabhal Bheag and Healabhal Mhòr (Macleod's Tables) across Loch Bracadale on the Duirinish Peninsula, and Macleod's Maidens are visible looking west to the northernmost mouth of Loch Bracadale, where it opens to the Atlantic Ocean. The village of Carbost and the Talisker Distillery are 4 miles southeast.

Notable local hills are Cnoc Glas Heilla ^[6](116m), Arnaval (369m) and Dirivallan (208m). It is possible to reach Talisker Bay on foot from the north, along a rough but well defined track.

The name Fiskavaig (alternative spelling Fiscavaig) comes from the Old Norse, meaning "fish bay". Another example of this naming is Tarskavaig, meaning "cod bay", in the south of Skye.

An alternative theory that the name derives from the Gaelic "Fasgadh-Beag" meaning "little shelter"^[7] has been dismissed by the OS Record of Names as "A very bad case of finding a forced Celtic meaning & words for a name which is not Celtic. "Fasgadh beag" or Little shelter is entirely different from both the ordinary spelling and ordinary pronunciation of this name."^[8]

A Class I Pictish symbol stone (dated from the 6th to 8th Century) was found on the beach at the high water mark in 1921. As noted in a report:

A symbol-stone was noted at high-water mark on the shore of Fiscavaig Bay, an inlet of Loch Bracadale, about 1921. After being concealed for some time by shingle it was rediscovered in 1927 and presented to the National Museum of Antiquities of Scotland (NMS X.IB 213) (1).

The stone is a slab of schist, 0.89m high and 0.27m thick and tapering in width from 0.52m at the top to 0.43m above the foot, which is broken obliquely. In the upper part there is a double-disc-and-Z-rod symbol. The discs are unornamented and the Z-rod has a foliated head at the upper right terminal, but a plain fork at the other one. Below this there is a crescent-and-V-rod symbol. The crescent encloses in the left half two unequal arches contained by a larger one, but the right half is worn and only one small arch can be traced. The right terminal of the V-rod has also been obliterated, but the other retains most of its foliation.^[9]

Carved from a slab of schist, the stone bears double disc and Z-rod and crescent and V-rod symbols.^[10]

It is now on display at the Museum of Scotland in Edinburgh.

The Fiskavaig Rock Shelter is a late Iron Age (2AD to 560AD) rock shelter on the west coast between Talisker Bay in the south and Rubha nan Clach in the north (NG30283264) at Uamh an Eich Bhric (The cave of the speckled horse) which contains substantial occupation and midden deposits.

A programme of excavation was carried out between 2006 and 2010 uncovering ."..a wide range of small finds... including stone, iron, copper-alloy, bone and antler artefacts, ceramics and evidence for both iron and copper-alloy working (iron slag, crucible fragments, mould fragments, fragments of furnace lining and copper-alloy scrap) as well as leather-working and textile production.^"[11]

According to the archaeological reports "All of the material seems to have been deposited within a tight time frame, perhaps as little as 30 years and probably no more than 100 years."^[11]

No further excavations are planned due to the rate of erosion noted at the site.^[11]