Yeah, as long as you have both FBA and Claims configured on the same zone you'll get the prompt. The only out of the box way I can think of is to use 2 zones, which means 2 urls. Otherwise, I think you'll have to create your own custom login
page that will attempt to login the user via Windows Authentication, and if that fails present a Forms login page.
You may want to try posting this in the SharePoint 2010 forums to see if anybody else has a solution for this: