An index.html is the default page that is served (displayed to the browser) when no specific file in a directory is specified. For example, http://site.com/ will actually display http://site.com/index.html (usually)
You need it in public_html because that's where your publicly viewable pages are. Think of the other directories as system directories and don't mess with them.