You should use the most popular one, phpBB, since it has so many mods. Although, even with an XHTML compliant theme on some pages the validator reports errors due to internal coding. I heard from people before that it would require a lot of changes to fix. That's why I don't develop themes for phpBB, at least not until a stable beta release of phpBB 3.
Simple Machines Forum is the best free forum by far, although I don't like the slightly restricted license. There's also not nearly as many mods for SMF, although there are still quite a few. And if you have a modded SMF they still provide support, unlike phpBB's response of "Sorry, but we only support vanilla phpBB."
phpBB requires a lot of mods just for basic features that should've been implimented by default. Not many themes are made using XHTML. The themes aren't very impressive either and all of them are really overused. Most people don't bother to make their own theme for phpBB.
The themes for SMF are certainly better in quality. SMF, by default, uses valid XHTML so themes are easier to develop without having to rewrite code. I like how the theme files are PHP files because it makes things a lot easier. When someone asked phpBB about this they said they don't allow PHP in template files for security reasons. That's understandable, but SMF does and it's still much more secure than phpBB.