I've verified on my own SharePoint installation with a site and a sub site, that you can indeed have a user with permissions on the sub site, but no permissions on the parent site.
My guess as to what is happening is that there's some type of resource on the sub-site page that actually resides on the parent site, and as such requires requires permissions on the parent site in order to display the sub site page (The resource may be an
image, or perhaps the page accesses a list). As soon as you try to access a single resource you don't have permissions for, SharePoint will redirect you to the access denied or login page.
Maybe try creating a new blank empty team site to test the permissions with. If the resource that's getting accessed is accessed through the master page though (an image), you'll get the same issue. In that case you can run Fiddler, which should show you the
last resource accessed before you're redirected to the access denied or login page.
Also, just to clarify, although the FBA Pack allows you to manage the permissions for FBA Users, it's SharePoint itself that actually enforces the permissions.