Ashley House was the most important great house close to the village of Walton-on-Thames. It stood in Walton-on-Thames for over 300 years. The house was built between 1602 and 1605 for Lady Berkley, wife of the seventh Lord Berkley. The house cost £3119.4s.7½d. It was a Jacobean House. The house passed through many hands throughout the years. These included the 1st Earl of Anglesey, Christopher Villiers, Benjamin Weston, Richard Boyle and Viscount Shannon Whose memorial stands in St Mary's Church.
In 1786 Sir Henry Fletcher acquired the house and its grounds. It remained property of the Fletcher family until the 1860's when it was sold to David Sasson. When the last male Sasson dies without an heir, the estate was broken up for housing development and the house was demolished. The lands in which Ashley house was situated were of considerable size, originally comprising of 190 Acres, when the common land of Walton was enclosed, the size of the estate rose to 440 Acres.
The main drive of Ashley house originally ran into the middle of Walton High street, roughly opposite to where WH Smiths now stands today. Ashley house, was demolished in 1930 and its estate was used to build houses on in the 1920's and 1930