Fort Charlotte was designed by John Mylne, Master Mason to King Charles II. Work began on the original fort in 1665-7, and it was subsequently rebuilt in 1781 after lying in disrepair for almost a century.