4GB of RAM should be plenty for all of the games today. My friends don't have any more than 2gigs and they can run Crysis, World in Conflict and Call of Duty 4 without any lag/loading what so ever with all of the graphics and sound quality settings maxed out, and with a high frame rate above 50fps, and yet, they always have loads of apps open on their Vista machines. Also, you are running 64bit, right? I'm not sure what OS you have, but from what I know, Windows (I'm assuming you use this) will only be using 3.5GB out of the 4GB because of the way 32bit is designed.
If the game is lagging instead of loading constantly and you are sure that no other open apps/background tasks/processes are hogging the system, you might need a faster CPU and/or Graphics card. If it's loading constantly, close out some resource hogging apps (like Firefox with loads of tabs open, or if it's been open for weeks) or clean out some of your system temp files.