The building was constructed in the period 1868–72, for an estimated cost of £50,000. The house was designed by Thomas Worthington, for the editor and proprietor of the Manchester Guardian, John Edward Taylor. The building was described by Pevsner as "grossly picturesque in red brick and red terra cotta."
The Towers was once the home of the notable engineer Daniel Adamson – whose idea for the canalisation of the Rivers Irwell and Mersey resulted in the creation of the Manchester Ship Canal project which made the rivers into Manchester navigable for sea-going ships. He invited representatives of several Lancashire towns, local businessmen and politicians, and two civil engineers, Hamilton Fulton and Edward Leader Williams. Fulton proposed a tidal canal, with no locks and a deepened channel into Manchester; Williams was in favour of a series of locks. Both engineers were invited to submit proposals, and Williams' plans were selected to form the basis of a bill submitted to Parliament in November 1882. Because of intense opposition by Liverpool and the railway companies, the necessary enabling Act of Parliament was not passed until 6 August 1885. Certain conditions were attached: £5 million had to be raised, and the ship canal company had to buy both the Bridgewater Canal and the Mersey & Irwell Navigation within two years.
In 1920 it became the base of the Shirley Institute of the British Cotton Industry Research Association as a research centre dedicated to cotton production technologies. A significant contribution to the purchase price of £10,000 was made by William Greenwood, the MP for Stockport, who asked that the building be named after his daughter, Shirley.
Italics denote building under construction