Wither was the eldest son of Charles Wither of Oakley Hall, Hampshire and his wife Dorothy Smith, daughter of Sir William Smith, 1st Baronet of Redcliff, Buckinghamshire. The family had been seated at Oakley Hall since 1626 and since then had acquired further estates in the neighbourhood, to which Wither succeeded on the death of his father in 1697.^[1] He matriculated at Balliol College, Oxford on 7 May 1700, aged 15,^[2] and travelled abroad in the Netherlands from 1706 to 1707. He married Frances Wavell, daughter of Thomas Wavell of Winchester, Hampshire, on 17 July 1707.^[1]

In 1707, Wither investigated the possibility of standing for Stockbridge at the next general election and secured local Whig support. He was appointed High Sheriff of Hampshire in November 1707, but gave up the office soon after. He stood for Parliament at Whitchurch in a contested by-election on 17 January 1708, and was seated as a Whig Member of Parliament on petition on 17 February 1708. He did not stand at the 1708 British general election.^[1]

Wither was appointed surveyor-general of woods and forests in 1720, holding the position for the rest of his life. At the 1722 British general election, he stood unsuccessfully for Shaftesbury. He was returned as a government supporter for Christchurch at the 1727 British general election, He supported the Administration and voted with them on the Hessians in 1730.^[3]

Wither died on 20 November 1731, and was buried at Deane. He left three surviving daughters, a son and daughter having predeceased him. His wife Frances died in 1752, followed a day later by her eldest daughter Dorothy. Their second daughter Henrietta Maria.married Edmund Bramston and succeeded to Oakley Hall. They were the parents of Wither Bramston who rebuilt the Hall. Charles Wither's third daughter Ann married her cousin William Beech whereby Oakley Hall came into the Beech family in 1832.^[4]