You can destroy an LDAP server using the Server app (and still using slapconfig -destroyldapserver). To do so, open the Server app and click on Open Directory. Then click on the Open Directory server in the list of servers.
When prompted to destroy the LDAP Master, click on Next.
When asked if you’re sure, click Continue.
When asked if you’re really, really sure, click Destroy.
Wait.