The oceanfront compound could be demolished to build a more modern home.