php Upload Function help please

[freecrm]

My Site is a customer relationship management system, fully written from scratch.

Within it is a function to import existing contact details (.csv file) into the DB. However, this relies on the file_get_contents from a URL, which means that a user has to have their .csv uploaded to a server.... which many won't have a clue how to do.

What I would prefer is for them to just upload a file from their system and then the code loads the data from that temporary file.

I know this is probably possible but don't have clue where to start.

Any assistance is appreciated.

 

[misson]

This is where to start: "Handling file uploads".

 

[freecrm]

This is where to start: "Handling file uploads".

OK - thanks for the starting point. This page didn't exactly help, but it did lead me to a similar helpful one...

I now have a 3 page process.

1) upload file to tmp dir

<form action="datapreview.php" method="post" enctype="multipart/form-data" name="form1" id="form1">
    <label>
    <input type="hidden" name="MAX_FILE_SIZE" value="30000" />
    <input name="userfile" type="file" size="80" />
    </label>
    <p>
      <label>
      <input type="submit" name="button" id="button" value="Upload" />
      </label>
    </p>
  </form>

2) Check data and preview

<h2>File Information</h2>
<br />
File Name: <?php echo $_FILES['userfile']['name'];?>
  <br />
File Type: <?php echo $_FILES['userfile']['type'];?>
<br/>
File Size (bytes): <?php echo $_FILES['userfile']['size'];?>
<br/>
Temp Uploaded File Name: <?php echo $_FILES['userfile']['tmp_name'];?>
<br/>
Error Code: <?php echo $_FILES['userfile']['error'];?>
<br/>
<br/>
<h2>Preview Data to be imported</h2>
<?php
$filecontent = file_get_contents($_FILES['userfile']['tmp_name']);
echo "<br/>";
print_r($filecontent);
?>

3) import into DB

Don't think I'll have any problems here.

My questions are..

a) how do I verify that the file is .csv format?
b) on page 2, how do I get the data to preview in tabular format? Will this have to be on a loop and create rows based on exploding by /n?
c) what happens to the temporary file? Is it automatically deleted on browser closure?

 

[lemon-tree]

a) Look for the features that make it a .csv file and just verify. Firstly, check file extension. Secondly, is the format correct. Finally, use strip_tags() and make sure the file does not pose a security risk.
b) Just loop through, replacing the blanks
c) The server deletes it after the PHP execution completes

 

[misson]

OK - thanks for the starting point. This page didn't exactly help, but it did lead me to a similar helpful one...

Which is the point of the page. As the entry point for the PHP file upload documentation, it links to specific subject pages.

<form action="datapreview.php" method="post" enctype="multipart/form-data" name="form1" id="form1">
    <label>
    <input type="hidden" name="MAX_FILE_SIZE" value="30000" />
    <input name="userfile" type="file" size="80" />
    </label>

The <label> element is supposed to contain a label for a form control, not a form control itself. Why are you using them as <input> containers?
Example:

<form action="datapreview.php" method="post" enctype="multipart/form-data" name="form1" id="form1">
    <label for="userfile">File:</label>
    <input name="userfile" type="file" size="80" />
...

I also don't recommend using the "MAX_FILE_SIZE" form variable even though it shows up in the PHP documentation examples. Form variables shouldn't contain trusted data and the only way of validating that particular value is to store the same information server side.

c) what happens to the temporary file? Is it automatically deleted on browser closure?

Note that c) is answered immediately above the last example on the POST method uploads page.

 

[freecrm]

Thanks for the info - I'm working on this part by part.

First - getting the file extention. I've tried using pathinfo, but for some reason, the extention part of the array is not supported by the x10 version. Any ideas?

i.e. $pathinfo = pathinfo($_FILES['userfile']['tmp_name'], PATHINFO_EXTENSION);

I've also tried to split it off using $ext = $pathinfo['extension']; but got nowhere.

In fact, if you print_r the array, there is nothing there containing the extension information!

I have used strip_tags as suggested as this could be a problem (although if it remains in the /tmp/ directory, surely if this is not public access, it wouldn't cause a problem.

I've also pretty much done the loop.

Thanks for the help so far.

 

[lemon-tree]

I have used strip_tags as suggested as this could be a problem (although if it remains in the /tmp/ directory, surely if this is not public access, it wouldn't cause a problem.

Very true, but I just do it as a safety precaution, you shouldn't really have to do it.

To get the extension, I generally just split the string by the '.' and look at the last item in the array.

 

[misson]

First - getting the file extention. I've tried using pathinfo, but for some reason, the extention part of the array is not supported by the x10 version. Any ideas?

i.e. $pathinfo = pathinfo($_FILES['userfile']['tmp_name'], PATHINFO_EXTENSION);

The temporary name is pseudo-random and not based on the original file name. Use $_FILES['userfile']['name'] to get the original file name, including extension.

 

[freecrm]

That would explain a lot - thanks.

I have now completed the preview page but am struggling to get my head around how to validate the format.

I think I can also use a character count to establish a size restriction.

I'll get there I'm sure.

 

[misson]

I have now completed the preview page but am struggling to get my head around how to validate the format.

The standard approach would be to parse the file and throw an exception if the parse fails. For CSV files, use fgetcsv() to parse the next line & then check that the number of fields is correct. If fgetcsv() fails, make sure you use feof() to check whether you've reached EOF or had a parse error.

 

[freecrm]

Thanks

I love the fgetcsv function!

OK, my validation page now works nicely and warns the user if the format is wrong in any way.

In addition, I have used a data preview system to output the data into tabular format.

My next issue can probably be solved quite simply....

The first page is the file form page with a submit that takes you to page 2.

Page 2 is the validation/ preview page.

Page 3 will then insert into MySQL if the user accepts the preview page so my question is this...

On page 1, do I have 2 submit buttons (preview/ import) where the preview action is the preview/validation page (target="_blank")and the import is the actual import function?

or do I have to re-assign all the post fields on page 2 (43 of them!) so that it then submits again to page 3?

Is this clear :S

 

[misson]

On page 1, do I have 2 submit buttons (preview/ import) where the preview action is the preview/validation page (target="_blank")and the import is the actual import function?

or do I have to re-assign all the post fields on page 2 (43 of them!) so that it then submits again to page 3?

I'm not a fan of popups. Setting up the input fields isn't hard; just loop through $_REQUEST, adding a hidden field for each. Transferring the parsed CSV through the second submit the is slightly trickier, but that's what sessions are for.

However, there's a better way to do it. This is a great time for an IFRAME. Start with a hidden IFRAME (or dynamically add it to the doc), show it and display the preview in that. That way, the 1st form never goes away, there's no popup, and if the preview doesn't look good, the user doesn't have to refill the whole form.

 

[freecrm]

Thanks Mission

I can't get my head around the iframe idea.

If I have the iframe as the preview pane, surely I would still have to have a submit button to call the functions in that iframe. Presumably, the form action would be the preview file URL that is being included.

In addition, I would still have to have an "import" button if they accept it (not in the iframe), but the form action is in the form tag itself, not the button.

So would I need two forms, one with action preview and one with action import?

In which case, I might as well resort to the sessions idea because I would still be duplicating fields (hidden or not).
Edit:
Pretty much cracked this now but I have one pretty major problem.

I have used Mission's idea of passing the uploaded file to page 2 (Data Preview/ Validation) and then on to page 3 which inserts into database.

However, the uploaded file only remains for page 2. By the time page 3 is opened, the tmp file no longer exists.. and my file_get_contents cannot open the stream.

How can I get round this?

 

[misson]

If I have the iframe as the preview pane, surely I would still have to have a submit button to call the functions in that iframe. Presumably, the form action would be the preview file URL that is being included.

In addition, I would still have to have an "import" button if they accept it (not in the iframe), but the form action is in the form tag itself, not the button.

So would I need two forms, one with action preview and one with action import?

You can do it with one form: use JS to change the target attribute of the form to an IFRAME.

One thing about this approach: proper design principles dictate it should degrade when JS is disabled/not supported. Among other things, the preview page should include a form that's hidden/removed by JS so that a user can import from the preview page. Of course, at that point you're looking at the same problem you're currently dealing with. Then why do it? It's the most useable design.

Edit:
Pretty much cracked this now but I have one pretty major problem.

I have used Mission's

"Misson" -- only one "I".

idea of passing the uploaded file to page 2 (Data Preview/ Validation) and then on to page 3 which inserts into database.

However, the uploaded file only remains for page 2. By the time page 3 is opened, the tmp file no longer exists.. and my file_get_contents cannot open the stream.

How can I get round this?

The "POST method uploads" page has one solution: use move_uploaded_file to move the file, which will prevent deletion. An alternative is to save partially processed data (if any) in a temporary file. The second option has the added advantages of reducing network usage and server load (no need to process data twice).

Of course, in either case you'll have to clean up the temp files for when a use previews but neither cancels nor submits the form. You'd have to schedule a cleanup job to remove any temp files older than a certain amount (1 hour?). Try to figure out how many orphan files are likely to pile up in a given period in a worst-case scenario, how much space they'd take up and how much space you can spare. Use that to figure out the maximum allowable age for temp files. Cleanup at half that interval.

Hmm... sessions aren't sounding too bad.

 

[freecrm]

Thanks Misson.

I've finally cracked it.

I did use the move_uploaded_file technique, but didn't need to schedule a clean-up job.

The new filename is simply a re-name using session_id().".".$ext. In this way, if two users are doing this at the same time, it will create a different filename, regardless of $_FILES['originalfile']['name'].

In Page 3, after all rows of data have been inserted, I simply did an f_open or die(), followed by an f_close to check the file wasn't already open (shouldn't be) and then did an unlink to delete it.

Thanks for all your help on this one.

 

[misson]

In Page 3, after all rows of data have been inserted, I simply did an f_open or die(), followed by an f_close to check the file wasn't already open (shouldn't be) and then did an unlink to delete it.

What if a user never visits the final page?

 

[freecrm]

What if a user never visits the final page?

Aaarrgghhh!!!

OK, back to plan A - schedule!

 

[misson]

The cleanup job only needs to take care of orphaned temp files--the ones left behind when a user doesn't visit the 3rd page. This is what I was trying to say in an earlier post:

Of course, in either case you'll have to clean up the temp files for when a use previews but neither cancels nor submits the form.

On the plus side, there probably won't be too many orphaned files.

As you're using sessions to generate filenames, one other thing that can cause orphans is if the user waits awhile between previewing the data and importing it causing the session to expire. This one you need to detect and give an error message for in the 3rd page.

It might be easier to store the data in a session variable, even if there's a fair bit of it. That way you don't need to worry about orphans. I'm not sure if there'll be any performance impact from this approach, though I don't think there should be much of one.