The son of Pierre Gaspard, comte d'Orsay (1748–1809); he served as a general in Napoleon's army (1813) and was the father of the dandy and painter Alfred, Count d'Orsay.