I managed to reproduce the problem with the Membership Request web part. I'll look into it and put a fix in the next release.
The Password Recovery web part works fine for me in Anonymous mode. Make sure that you've turned on anonymous access in the authentication provider for the web application, as well as on the site. If you don't want to give the whole site Anonymous Access,
then you'll have to choose "Lists and Libraries" for the anonymous access level of the site, and then enable anonymous access on the library containing the page you want to give access to. If this is a publishing site, make sure the page is published and approved.