So turns my memory to that brilliant sight 

When wit and beauty held their festal night; 

When the thronged hall its glittering groups displayed 

Of nature’s loveliness, by art arrayed; 

—Henry Bowen Anthony (The Fancy Ball) 

Ships of the Sea Maritime Museum 

The S.S. Savannah was the first steamship to cross the Atlantic Ocean. One of its principal owners was William Scarbrough, who came to Savannah in 1802 at the age of twenty-six and rapidly, if only temporarily, transformed himself into an extraordinarily wealthy merchant. 

Scarbrough was soon the President of the Savannah Steamship Company and the enlarged social ambitions he shared with his wife naturally required a house to match his wealth and their importance.  

That house became known colloquially as The Castle. The William Scarbrough House, located at 41 Martin Luther King, Jr. Boulevard, was built in 1819. The architect for the house was William Jay, who also designed the Owens-Thomas House and Slave Quarters (PFS #13), The Savannah Theatre (PFS #41), The Mary Telfair House (PFS #34), and The Juliette Gordon Low House (PFS #32), among others. 

Even before the house was fully completed, President James Monroe was its guest. Soon after his inauguration in 1817, Monroe toured northern American states. In March 1819, he turned South. His purpose was to assess the war-readiness of the nation. The experience of 1812, had taught U.S. leaders the folly in their lack of preparedness. At the bequest of the Scarbrough power-couple, the Savannah City Council offered The Castle as the official Presidential residence during Monroe’s visit. 

But that wouldn’t be the most memorable reception held at the mansion. In 1837, an amateur poet and novice cotton broker named Henry Bowen Anthony attended a fancy-dress ball held at the Scarbrough House. By this time, the house was owned by Godfrey and Julia (Scarbrough) Barnsley. A wealthy cotton merchant, Barnsley made substantial improvements to the William Scarbrough House and its grounds. 

Henry Bowen Anthony, only 22 years old at that time, immediately wrote a poem to commemorate the party entitled The Fancy Ball: A Sketch. Having other skills and later becoming wealthy in his own right as a cotton trader, Anthony would go on to serve as both Governor and as a Senator of Rhode Island. 

At the time President Monroe was visiting Savannah in 1819, William Scarbrough was entangled in what economic historians would later call The Panic of 1819. Then, in 1820, one of Savannah’s many fires destroyed Scarbrough’s commercial properties and inventories. Scarbrough became incapable to meet the demands of his creditors and was briefly sent to jail. William Jay was among his many creditors. 

William’s sister, Lucy Scarbrough Isaac, had married well and bailed out her brother and his house. 

Much later, the house would again be saved by George Wymberley Jones Derenne, a decedent of Nobel Jones, an original founder of the Georgia Colony and developer of the Wormsloe Plantation (PFS #7), locating there the West Broad Street School for African Americans, which operated for nine decades. 

In 1994, the house came into the hands of Mills B. Lane IV — that wonderful family name so important to Savannah — who restored the Scarbrough House to serve as the Ships of the Sea Maritime Museum.