James travelled with a large retinue of courtiers, including John Churchill, later 1st Duke of Marlborough, and George Legge, Master of Ordnance and later 1st Baron Dartmouth, and Scottish nobles and politicians, such as the Lord Provost of Edinburgh, James Dick, and the Lord President of Scotland, James Graham, Marquis of Montrose. He also travelled with a squadron, which included the Ruby, Happy Return, Lark, Dartmouth, Pearl and the royal yachts the Mary, Katharine, Charlotte, and Kitchin. The Gloucester was evidently crowded with approximately 330 people onboard, and it took two hours to load James’s baggage and retinue, with passengers swapping ships at the last minute. Naval administrator and famous diarist Samuel Pepys had intended to travel on the Gloucester but moved to the Katharine ‘for room’s sake’. The fleet finally departed Margate Road on 4 May.

​

The next day there was a protracted argument between the Duke of York, the Gloucester’s pilot James Ayres, and several naval officers over the ship’s course, and a few hours later at 5.30am on 6 May, the ship struck the Leman and Ower sandbanks. Hoping the Gloucester could be saved, James only abandoned ship shortly before the Gloucester sank, transferring to an accompanying vessel the Mary to complete his voyage to Edinburgh, captained by Christopher Gunman. Hundreds of passengers and crew died, including Robert Kerr, Earl of Roxborough; Donogh O’Brian, Lord Ibracken; John Hope, Laird of Hoptoun; and Sir Joseph Douglas. James’s own family was affected: James Hyde second lieutenant on the Gloucester, the youngest brother of his first wife Anne, also drowned, as did many of his household.