The son of John Hervey, first Earl of Bristol; he was MP for Bury St. Edmunds and Groom of the Bedchamber to the Prince of Wales. There is no evidence to support the allegation that he was the father of Horace Walpole.