After opening up the Edit Custom Genres via the Tools menu click on Add and a new entry called "[New Genre]" in Blue should be added. Where that is create your new Genre and then Click OK.

Your custom genre should now be added.

If you reopen the same Edit Custom Genres window and your newly added Custom Genres aren't listed then I would suggest doing a backup of your collection first and then doing a Database Repair and see it that fixes the problem.