Ansley Callaway, Ledyard