On the site of the present building, a Priest’s House is recorded as having been built in 1585 by Rev. Thomas Rockes. The present house was considerably rebuilt in about 1846 by the Vicar of Butleigh Rev. W. J. Neville, brother of the Squire, Ralph Neville Grenville. The house was known as the Vicarage until about 1919 when it was decided to build a new Vicarage at the entrance to Church Walk. The house was then called Butleigh House, owned by the Butleigh Court Estate, and, in 1919, Hugh Clark (grandson of James Clark who with his brother Cyrus were the founders of C & J Clark) lived there with his family until 1935. When the sale of the Butleigh Court Estate took place in 1947, the property was sold and has had various owners since. During its time as a boarding house for Millfield School, Ralph Douglas-Scott-Montagu (later, 4th Baron Montagu of Beaulieu) boarded here. Lord Montagu is the owner of the Beaulieu Estate, home of the National Motor Museum that cares for the Grenville Steam Carriage.