VPS stands for Virtual Private Server... essentially its a Virtual Machine and in most cases is fully exposed to the internet with a direct external ip address... Learn the bash shell well if you're going to get one...
No they're not free... I have one on linode.com thats 20 bux a month... the ones here on X10 are better priced...