Insert records from a local csv file

freecrm

New Member
Messages
629
Reaction score
0
Points
0
Using MySQL/php.

All I need to do is to allow a user to insert records into a MySQL table from a .csv file on his/her local computer...

Sounds simple..

Hmmm...:nuts:

I have looked into DATA LOAD LOCAL INFILE, but aparently, this only works from the server in which the code/data resides.

Do I have to upload a the file first?

Anyone done this before?
 

Scoochi2

New Member
Messages
185
Reaction score
0
Points
0
yeah, the csv file will need to be uploaded onto *a* server. Not necessarily your server, any will do so long as your script has the permissions needed to read it.
Once the script is online, you can do something along the lines of the following in order to convert the file into an array in your PHP script:

PHP:
$file = 'location of file';
$array = explode(',',file_get_contents($file));
Then, it's as simple as using a foreach to get all values of $array into your database :)

Alternatively, use a textarea form and get the list from $_POST
 

freecrm

New Member
Messages
629
Reaction score
0
Points
0
yeah, the csv file will need to be uploaded onto *a* server. Not necessarily your server, any will do so long as your script has the permissions needed to read it.
Once the script is online, you can do something along the lines of the following in order to convert the file into an array in your PHP script:

PHP:
$file = 'location of file';
$array = explode(',',file_get_contents($file));
Then, it's as simple as using a foreach to get all values of $array into your database :)

Alternatively, use a textarea form and get the list from $_POST

Cool - thanks - I'll get back to you when I've done some dev and testing.

P.S. - like the text area option - this will avoid many formatting issues.
 

Scoochi2

New Member
Messages
185
Reaction score
0
Points
0
Also remember to use the trim function if the list might include whitespace that you don't want to keep :)
 

freecrm

New Member
Messages
629
Reaction score
0
Points
0
OK - finally come back to this and I'm still struggling...

And to be honest, I don't know much about arrays or foreach!!!

I have put in the following:

PHP:
<?php
mysql_select_db($database_freecrm, $freecrm)
or die(mysql_error()); 

if ((isset($_POST["MM_insert"])) && ($_POST["MM_insert"] == "form1")) {

$file=$_POST['filepath'];
$arrfile = explode(',',file_get_contents($file)); 

		foreach($arrfile as $value){
 			echo $value.'<br />';
			$insert="INSERT INTO TEST (FIRSTNAME, LASTNAME, COMPANY) VALUES ('$value')";
			mysql_query($insert) OR die(mysql_error());
		}

}
?>

I'm getting an error code

"FIRSTNAME"
Column count doesn't match value count at row 1


One thing I noticed - this is surely just returning one value for each DB record, whereas the csv file could contain several columns (does this have to be specified?)

As you can see, I am trying (very trying) but I'm not even sure I'm going in the right direction.
 

mattura

Member
Messages
570
Reaction score
2
Points
18
well your insert columns and values need to match for a start:
PHP:
$fnam=?;
$lname=?;
$comp=?;
INSERT INTO `test` (`firstname`,`lastname`,`company`) VALUES ('$fnam','$lnam','$comp');
I'm not sure about the state of your array when you are in that loop, so I put question marks. It may be $value[0], $value[1], $value[2] for example.

Do the values have to be in separate columns? Sometimes I have a database column which is a text field and consists of comma separated values. I can then SELECT just one column, and make php do the hard work using explode() and implode().
It depends on what you are storing.

Edit:

Oh, show an example of your csv - I think you will have to explode() differently. Perhaps on line breaks, or every third record or something, then within the foreach loop another explode() on commas.

If you have:
Code:
joe,bloggs,amtex\n john,smith,acme\n james,black,comtech\n
you need to have
PHP:
$triplet=explode("\n",$);
foreach($triplet as $vals) {
 $fnam=$vals[0];
 $lnam=$vals[1];
 $comp=$vals[2];
}

if in doubt, use print_r($triplet) to find out what your array looks like
Edit:
Hey, I found this and thought of you:
http://docs.php.net/manual/en/function.str-getcsv.php
have fun!
 
Last edited:

freecrm

New Member
Messages
629
Reaction score
0
Points
0
Thanks Guys..

For starters, my file is a comma seperated values file as follows (Copied directly from notepad but with \n at new lines):

For ease of testing, this file is located at www.freecrm.x10hosting.com/testdata.csv

Code:
"FIRSTNAME","LASTNAME","COMPANY"\n
"Joe","Bloggs","Joe Bloggs Ltd"\n
"Jane","Bloggs","Joe Bloggs Ltd"\n
"John","Doe","John Doe Ltd"\n

My code now looks like this...

PHP:
//connect to db or show error
mysql_select_db($database_freecrm, $freecrm)
or die(mysql_error()); 
	//if form hidden field returns a value, execute the following script
	if ((isset($_POST["MM_insert"])) && ($_POST["MM_insert"] == "form1")) {
		//allocate posted file path to variable
		$file=$_POST['filepath'];
		//load file and split into array
		$arrfile = explode(',',file_get_contents($file)); 

		//loop through each line
		foreach($arrfile as $value){
		
			//define column values
			$firstname=value[1];
			$lastname=value[2];
			$company=value[3];
			
			//insert into db
			$insert="INSERT INTO TEST (FIRSTNAME, LASTNAME, COMPANY) VALUES ('$firstname','$lastname','$company')";
			mysql_query($insert) OR die(mysql_error());
		}

	}

I have tried several versions of $something=$value[1] but I can't get it right. The square brackets are returning an error.

Code:
Parse error: syntax error, unexpected '[' in /home/freecrm/public_html/crmimexport/contactimport.php

I have tried with (), and without brackets but no difference.

Just one thing to bear in mind, field (column) 1 is an autoincrement Integer ID and not specified in the csv file.

I checked out the page on the php site but it's all gobbledegook to me!!! I would prefer to understand what I'm doing rather than just copy lines and lines of strange code... :)
 

Scoochi2

New Member
Messages
185
Reaction score
0
Points
0
The following function takes a filename as the only parameter, and returns an array.
Use as follows:
$data = explode_by_lines('testdata.csv');

PHP:
<?php
function explode_by_lines($filename)
  {
  $data_1 = array();
  foreach (file($filename) as $array)
    {
    $data_1[] = explode(",",str_replace('"','',$array));
    }
  return $data_1;
  }
$data = explode_by_lines('testdata.csv');
print_r($data);
?>
The above will output
Code:
Array
(
    [0] => Array
        (
            [0] => FIRSTNAME
            [1] => LASTNAME
            [2] => COMPANY

        )

    [1] => Array
        (
            [0] => Joe
            [1] => Bloggs
            [2] => Joe Bloggs Ltd

        )

    [2] => Array
        (
            [0] => Jane
            [1] => Bloggs
            [2] => Joe Bloggs Ltd

        )

    [3] => Array
        (
            [0] => John
            [1] => Doe
            [2] => John Doe Ltd

        )

)
 
Last edited:

freecrm

New Member
Messages
629
Reaction score
0
Points
0
Thanks Scoochi

I have put this code in but am not getting anywhere with it.

In your example, where do i insert the file path from the form?

You have references to $filename (which is testdata.csv but could be any number of names) and also to the filename itself.

I have tried various combinations but I continue to get the same error.

Code:
Warning: file(filepath) [function.file]: failed to open stream: No such file or directory in /home/freecrm/public_html/crmimexport/contactimport.php on line 27 [foreach (file($filename) as $array)]

Is this becuase it only works for files on your own server?

I need it to work for any hosted file (i.e. for a user.)

I'm amazed there isn't a free script for this...
 

Scoochi2

New Member
Messages
185
Reaction score
0
Points
0
It should work for any file. Just make sure you use an absolute rather than a relative path.
That being said, it depends on how the sever the script is hosted on whether it will for for files hosted elsewhere.
I think on X10 you need the intermediate level PHP (at least).

You do not need to change the function at all. You specify the filename when you call the function. For example:
PHP:
$data = explode_by_lines('http://arandomdomain.com/directory/testdata.csv');
 

mattura

Member
Messages
570
Reaction score
2
Points
18
Thanks Guys..
My code now looks like this...

PHP:
//connect to db or show error
mysql_select_db($database_freecrm, $freecrm)
or die(mysql_error()); 
	//if form hidden field returns a value, execute the following script
	if ((isset($_POST["MM_insert"])) && ($_POST["MM_insert"] == "form1")) {
		//allocate posted file path to variable
		$file=$_POST['filepath'];
		//load file and split into array
		$arrfile = explode(',',file_get_contents($file)); 

		//loop through each line
		foreach($arrfile as $value){
		
			//define column values
			$firstname=value[1];
			$lastname=value[2];
			$company=value[3];
			
			//insert into db
			$insert="INSERT INTO TEST (FIRSTNAME, LASTNAME, COMPANY) VALUES ('$firstname','$lastname','$company')";
			mysql_query($insert) OR die(mysql_error());
		}

	}

I have tried several versions of $something=$value[1] but I can't get it right. The square brackets are returning an error.

Code:
Parse error: syntax error, unexpected '[' in /home/freecrm/public_html/crmimexport/contactimport.php

I have tried with (), and without brackets but no difference.

Just one thing to bear in mind, field (column) 1 is an autoincrement Integer ID and not specified in the csv file.

I checked out the page on the php site but it's all gobbledegook to me!!! I would prefer to understand what I'm doing rather than just copy lines and lines of strange code... :)

Try just above '//define column values':
$value=explode(",",$value);

and changing $arrfile=explode(",", ...
to $arrfile=explode("\n", ...

and it's worth trying to understand at least some of the php docs, at least after a little experience. It can take you far!
Good luck!
 

freecrm

New Member
Messages
629
Reaction score
0
Points
0
It should work for any file. Just make sure you use an absolute rather than a relative path.
That being said, it depends on how the sever the script is hosted on whether it will for for files hosted elsewhere.
I think on X10 you need the intermediate level PHP (at least).

You do not need to change the function at all. You specify the filename when you call the function. For example:
PHP:
$data = explode_by_lines('http://arandomdomain.com/directory/testdata.csv');

OK - have changed the code to the following:

PHP:
<?php
//connect to db or show error
mysql_select_db($database_freecrm, $freecrm)
or die(mysql_error()); 
 //if form hidden field returns a value, execute the following script
 if ((isset($_POST["MM_insert"])) && ($_POST["MM_insert"] == "form1")) {
 
 $filepath = $_POST['filepath'];
  
  //allocate form path to variable
  function explode_by_lines($filename)
    {
    $data_1 = array();
    foreach (file($filename) as $array)
     {
     $data_1[] = explode(",",str_replace('"','',$array));
     }
    return $data_1;
    }
  $data = explode_by_lines('$filepath');
  print_r($data);

 
   
   //insert into db
   //$insert="INSERT INTO TEST (FIRSTNAME, LASTNAME, COMPANY) VALUES ('$firstname','$lastname','$company')";
   //mysql_query($insert) OR die(mysql_error());
  
 }
?>

but still no joy... :(

The file path I used was

http://www.freecrm.x10hosting.com/testdata.csv

but I still get the same error, where it is looking for a file local to the script...

I do have intermediate level as I use putenv() functions.

not sure where to go with this now...
Edit:
Try just above '//define column values':
$value=explode(",",$value);

and changing $arrfile=explode(",", ...
to $arrfile=explode("\n", ...

and it's worth trying to understand at least some of the php docs, at least after a little experience. It can take you far!
Good luck!

Thanks Matt

Still getting the same error

Code:
[B]Parse error[/B]: syntax error, unexpected '[' in [B]/home/freecrm/public_html/crmimexport/contactimport.php[/B] on line [B]31[/B]

Line 31 is

PHP:
$firstname=value[1];

So still no joy although I see where you were going with that... :)
 
Last edited:

natsuki

New Member
Messages
112
Reaction score
0
Points
0
freecrm said:
Still getting the same error
Code:
[B]Parse error[/B]: syntax error, unexpected '[' in [B]/home/freecrm/public_html/crmimexport/contactimport.php[/B] on line [B]31[/B]
Line 31 is
PHP:
$firstname=value[1];
So still no joy although I see where you were going with that... :)
I think what is causing the error is value[1] instead of $value[1] you simply forgot the $ for variable.
I tried to check your code and added some stuffs so I could check it. And I put stuffs in functions so I could use the code again in the future.
But your code is still unchanged except for a few that I added. You can test it here if you like:
testing using your csv just input it in the box http://www.freecrm.x10hosting.com/testdata.csv (the last line is a newline with no data that's why you'll get a row with empty strings.

index.html - form I just made up blah
test.php - the one called by "post" method of form
functions.php - some stuffs

PHP:
<?php
/**
  * Some functions I just made up just for testing^^^
  * Put them on separate file and just use require.
  * file: functions.php
  * - natsuki^^
  */

// this function simply creates the table test needed
function create_table($host, $user, $pass, $db, $table)
{
    $link = mysql_connect_db($host, $user, $pass);
    if (!$link)
    {
        die('error!');
    }
    mysql_select_db($db, $link) or die(mysql_error());
    $sql = 'CREATE TABLE ' . $table . ' (
        id INT NOT NULL AUTO_INCREMENT,
        FIRSTNAME VARCHAR(40) NULL,
        LASTNAME VARCHAR(40) NULL,
        COMPANY VARCHAR(40) NULL,
        PRIMARY KEY(`id`)
        )';
    mysql_query($sql) or die(mysql_error());
    return true;
}

// simply puts every line in an array
function get_data_per_line($string)
{
    $string = explode("\n", $string);
    return $string;
}

// make string db-safe
function format_db_string($string)
{
    $string = trim($string);
    if (!get_magic_quotes_gpc())
    {
        $string = addslashes($string);
    }
    return $string;
}

// to connect to MySQL db
function mysql_connect_db($hostname = 'localhost', $user = '', $pass = '')
{
    $link = mysql_connect($hostname, $user, $pass);
    if (!$link)
    {
        die('Could not connect to database: ' . mysql_error());
    }
    return $link;
}

?>
PHP:
<?php
# ------------------------------------------- start addition -----------
// file: test.php

// require functions.php
require 'functions.php';

// just setting the test variables:
define('_USER_', 'natsuki'); // constant

$_host     = 'localhost';
$_db     = _USER_ . '_test';
$_user    = _USER_ . '_test';
$_pass    = 'test';

$database_freecrm = $_db;

// connect to db to get link
$freecrm = mysql_connect_db($_host, $_user, $_pass);
# ------------------------------------------- end addition --------------------
//connect to db or show error
mysql_select_db($database_freecrm, $freecrm)
or die(mysql_error()); 
    //if form hidden field returns a value, execute the following script
    if (isset($_POST['MM_insert']) && ($_POST['MM_insert'] == 'form1'))
    {
        //allocate posted file path to variable
        $file = $_POST['filepath'];
        
        // load file and split into array per newline "\n"
        $temp = file_get_contents($file);
        $temp = get_data_per_line($temp);
        
        // or just use $arrfile from the start. I used temp because it's shorter xD
        $arrfile = array(); // explicit declaration as array
        $arrfile = $temp;
        
        //$arrfile = explode(',',file_get_contents($file)); //

        //loop through each line
        foreach ($arrfile as $value)
        {
            // explicit array, to prevent $value from being an array itself
            $data = array(); 
            $data = explode(',', $value); // separate data per ','
            
            //define column value
            $firstname = str_replace('"', '', $data[0]); // just making safe
            $lastname = str_replace('"', '', $data[1]);
            $company = str_replace('"', '', $data[2]);
            
            $firstname = format_db_string($firstname); // format string for db use
            $lastname = format_db_string($lastname);
            $company = format_db_string($company);
            
            //insert into db
            $insert = "INSERT INTO TEST (FIRSTNAME, LASTNAME, COMPANY)
                VALUES ('$firstname', '$lastname', '$company');";
            mysql_query($insert) OR die(mysql_error());
        }
        echo "Success putting data from csv $file to database $database_freecrm
            and table TEST!";
    }
?>
Just remember that the table I made is this:
PHP:
$sql = 'CREATE TABLE ' . $table . ' (
        id INT NOT NULL AUTO_INCREMENT,
        FIRSTNAME VARCHAR(40) NULL,
        LASTNAME VARCHAR(40) NULL,
        COMPANY VARCHAR(40) NULL,
        PRIMARY KEY(`id`)
        )';
It's very important to have the primary key set to auto increment so you won't get duplicate key errors
^^such a long post^^
 

freecrm

New Member
Messages
629
Reaction score
0
Points
0
Natsuki - you are an absolute star!!!!!!!!! (And I'm very thankful for your time and am sending you 200 creds - not much I know but I don't have many!)

It works a treat and no problems. I tidied up the data file to get rid of the extra line...

Yay!!!!!!!!!!!!!!! :biggrin:

Just one quick question... how do I ignore the first line??
 
Last edited:

natsuki

New Member
Messages
112
Reaction score
0
Points
0
you can do it with the unset() function then reindex the array with array_values() just change the function get_data_per_line to this:

PHP:
// simply puts every line in an array and deliberately ignores 1st line of $string
function get_data_per_line($string)
{
	$string = explode("\n", $string);
	unset($string[0]);
	$string = array_values($string);
	return $string;
}
or you can add it in the main script, so you can still use get_data_per_line() somewhere else just change the line to this
PHP:
// load file and split into array per newline "\n"
        $temp = file_get_contents($file);
        $temp = get_data_per_line($temp);
        unset($temp[0]);
	  $temp = array_values($temp);
^^
 
Last edited:

freecrm

New Member
Messages
629
Reaction score
0
Points
0
Perfect - many thanks...
Edit:
Please close this thread.
 
Last edited:

freecrm

New Member
Messages
629
Reaction score
0
Points
0
Yep, well done natsuki for spotting that missing $.

Lol - I detect a note of sarcasm here! ;)

I am also thankful for your efforts and have given you 100 creds. He did write the entire script that worked though including some additional validation...
 

natsuki

New Member
Messages
112
Reaction score
0
Points
0
even if you put $ value wasn't an array anyway so you get no values for index xp

I plan on making a csv thingy class but it won't be as simple as this ^^
 

mattura

Member
Messages
570
Reaction score
2
Points
18
Lol - I detect a note of sarcasm here! ;)
haha! No, it's just I was away for a while and natsuki posted the reply!

It would be an array after making the suggested change:
PHP:
$value=explode(",",$value);

Well I'm very grateful for the credits, not sure I deserve them...
 
Top