Barbara Bodichon was the extra-marital child of Anne Longden, a milliner from Alfreton, and the Whig politician Benjamin (Ben) Leigh Smith (1783–1860), the only son of the Radical abolitionist William Smith. Benjamin had four sisters. One, Frances (Fanny) Smith, married William Nightingale (né Shore) and produced a daughter, Florence, the nurse and statistician; another, Joanna Maria, married John Bonham-Carter (1788–1838) MP and founded the Bonham Carter family. Ben's father wanted him to marry Mary Shore, the sister of William Nightingale, now a relative by marriage.
Ben Smith's home was in Marylebone, London, but from 1816 he inherited and purchased property near Hastings: Brown's Farm near Robertsbridge, with an extant house built about 1700, and Crowham Manor, Westfield, which included 200 acres (0.81 km2). Although a member of the landed gentry, Smith held radical views. He was a Dissenter, a Unitarian, a supporter of free trade, and a benefactor to the poor. In 1826 he bore the cost of building a school for the inner city poor at Vincent Square, Westminster, and paid a penny a week towards the fees for each child, the same amount as paid by their parents.
On a visit to his sister in Derbyshire in 1826, Smith met Anne Longden. She became pregnant by Smith and he took her to the south of England, establishing her in a rented lodge at Whatlington, a small village near Battle, East Sussex. There she lived as "Mrs Leigh", the surname of Ben Smith's relations on the nearby Isle of Wight. Barbara's birth created a scandal because the couple did not marry; illegitimacy carried a heavy social stigma. Smith rode from Brown's Farm to visit them daily, and within eight weeks Anne was pregnant again. When their son Ben was born, the four of them went to America for two years, during which time another child was conceived.
On their return to Sussex they lived openly together at Brown's and had two more children. After their last child was born in 1833, Anne became ill with tuberculosis and Smith leased 9 Pelham Crescent, Hastings, which faced the sea; the healthy properties of sea air were highly regarded at the time. A local woman, Hannah Walker, was employed to look after the children. Anne did not recover, so Smith took her to Ryde, Isle of Wight, where she died in 1834.
Early on, Barbara showed a force of character and breadth of sympathies that would win her a prominent place among philanthropists and social workers. She and a group of London friends began to meet regularly in the 1850s to discuss women's rights, and became known as "The Ladies of Langham Place". This became one of the first organised women's movements in Britain. They pursued many causes vigorously, including their Married Women's Property Committee. In 1854, she published her Brief Summary of the Laws of England concerning Women, which was useful in promoting the passage of the Married Women's Property Act 1882. During this period she became close friends with the artist Anna Mary Howitt, for whom she sat on several occasions.