If you cannot delete the files/directories or they seem to come back on their own...
Go to cPanel --> File Manager
Go up one level to the parent of public_html , which is /home/YourUserName
Note the permissions of public_html.
Change the permissions to 0777
Enter public_html and delete the files
Go back up a level and return public_html to its original permissions
Note: if you installed Joomla in a subdirectory, say /joomla, of public_html replace public_html with that directory in the above instructions.