In succession, three early owners of Paxton House possessed two estates on the Caribbean island of Grenada.

Ninian Home (1732-95) acquired the estates of Waltham and Paraclete, which were worked by hundreds of enslaved African people. Ninian became the Governor of the island and he was killed in a French-inspired slave uprising. The estates then passed to George Home (1735-1820), followed by William Foreman Home (1782-1852), who received £6,226 (equivalent to over £800,000 today) in compensation for the emancipation of 247 slaves at Waltham in 1835. Ninian had bought Paxton House in 1773, and he purchased most of its Chippendale furniture.

Ninian spent much of his life in Grenada, so he saw slavery in action. There is some evidence that he was influenced by ideas from the Scottish Enlightenment – he wrote in a letter that slaves were; “so dependent upon the white people, and so much in their power, that wantonly to use that power is the height of cruelty…… They are human beings as well as ourselves. I will not keep any overseer that ill treats the negroes in any way.” So Ninian may have been less cruel than other slave owners, but he invested heavily in his sugar estates and so profited from the forced labour of enslaved people.