Further to what mephis said, the simplest way to get PHP and MySQL working is to use a package. If you're on Windows, use a WAMP.
There are two I can personally recommend and still use, and can help you set up if needed.
If you are using Windows XP, give
Apache2Triad a go. In addition to PHP, it will also enable you to use Perl and Python, and several databases (including MySQL), with GUIs and control panels. It is also easy to use once it is set up. All you need to do is to go to
http://localhost and you will find your own personal web server (nobody else can see it though).
For any other version of Windows, you should try
EasyPHP. It's a lot simpler than Apache2Triad and as it's name implies, a lot easier to set up and get going from the start. It doesn't have Perl or Python and the only database is MySQL. But it does have GUIs.
If I remember rightly, it comes with PHP4, and you have to upgrade it to PHP5 yourself (which can get complicated if you're new to it). But like I said, if you need help then I'm familiar to it.
If you do choose the WAMP path, then uninstall Apache, PHP and MySQL first if you already tried installing them!