The Script that Would Not Die

[fflandrath53]

Ok, when it comes to programming I'm living proof a little knowledge is dangerous. A php script to clean-up some hot links was launched, it appeared to process for a while then it crashed. The first strangeness, the server displayed a "web site temporarily unavailable" page instead of a standard php error message. At first I shrugged it off as just another bug somewhere in my code, the code that crashed and exited the system, flushed into electronic oblivion. Or so I thought.

I was messing around with php image functions, copying source images from remote hosts to a rollerposter directory, /bm_images/. For example:

case "JPEG":
  $im=@imagecreatefromjpeg($img0);
  if( !$im )
    {
      $img0="http://rollerposter.pcriot.com/images/bookmark.png";
      $img0_width=32;
      $img0_height=32;
   } else {
     $img0="http://rollerposter.pcriot.com/images/bm_images/$temp.jpg";
     $write_img=@imagejpeg($im, "./images/bm_images/$temp.jpg");
     if( !$write_img )
     {
        $img0="http://rollerposter.pcriot.com/images/bookmark.png";
        $img0_width=32;
        $img0_height=32;
     }
  }
break;

Next I checked the error log. Nothing. Then things got really strange. I looked in the target directory, which was emptied before running the script, (See example1.txt attachment), it was full of images. And it was continuing to add images every time I refreshed the display. The script was still running, doing exactly what I had intended except, it was invisible. It had crashed, I was certain of that. But here it was, steadily adding images.

For about an hour I flailed around like a fish out of water trying to make the problem go away. Nothing I did worked and instead crashed my browser trying to delete images from the ever-growing directory.

Frustrated I went over to the legacy file manager, it has a command to delete a folder and any files in it. That seemed to work, at least the directory was gone and so the images stopped. Or so I thought.

Well I re-created the directory /bm_images/ and made some code changes, then I noticed the strangest thing, remember I'm not running any script at this time, yet suddenly the /bm_images/ directory is once again full of images. And it's still growing.

So, back to the legacy fm and delete the directory, again. I tried one more time, same result, images just keep coming.

There are about 5300 entries in the table that are SELECTED into a standard mysql array ($row=['FIELD']) in a WHILE loop controlled by the array length.

If anyone can shed some light on this odd turn of events I would really appreciate it.

Thanks Floyd

btw, Starka seems to be running very well, and the new cpanel is much improved over the last one. Thanks for such a great free hosting service (and I'm very close to being able to purchase an upgrade).

 

[fflandrath53]

Ok!

It appears The Script that Would Not Die, had died. RIP...

Now back to the original problem:
Roller~Poster collects bookmarks. Each bookmark contains an image, user selected or a default. Up to now many of these images were hot linking and thus a departure from accepted best practices, Internet etiquette. It also resulted in loosing Associated Press images (used with permission) that have an automatic expiration date.

So, I came to the conclusion I needed to learn how to copy source images to my files and use them instead. I have since added a copy function if a bookmark is entered with a hotlinked image.

Of course that still leaves me with hot links in thousands of previously posted bookmark images that need to be copied and updated. That's when I inadvertently created The Script that Would Not Die. My script tried to copy everything in one fell swoop. I suspect that php stacks the image functions, executing them async in the order they were placed on the stack? Eventually memory runs out or there's a timeout called and the script is usually flushed, (or in this case, not).

If this is correct then a way to check a function's progress would be needed to throttle the execution and avoid running out of resources/time. Any suggestions? I got nothing.

Now I'm thinking I'll use javascript and xml to collect hot linked entries, copy or replace the image then update the table, one bookmark at a time.

To be continued... Roller~Poster

 

[fflandrath53]

So, to close out this thread, here's the final code I came up with to copy hot linked images to my x10 files...

HTML and Javascript​

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">

<head>
	<title>Roller~Poster ~> copy/update hot linked images</title>
	<meta http-equiv = "Content-Language" content = "en-us" />
	<meta http-equiv = "Content-Type" content = "text/html;charset = UTF-8" />
	<meta name = "author" content = "Floyd Ferris Landrath, Portland, Oregon" />
	<meta name = "robots" content = "noindex,nofollow" />
	<meta name="description" content="Roller~Poster is a content aggregator and online bookmark service; an experimental, not-for-profit application.  This feature is based on an AP developer "sandbox" interface." />
	<meta name="keywords" content="roller poster, aggregator, online bookmarks, not-for-profit, AP, sandbox" />
	<link rel = "stylesheet" type = "text/css" href = "/_roller.css" />
	<script language="JavaScript" type="text/javascript" src="http://rollerposter.pcriot.com/js/_roller.functionLib0.js"></script>

<script language = "JavaScript" type="text/javascript">
<!--
var xWin="", winInterval, winTimeout, tag1, tag2, bmMax, bmIndex;
var bmIndices = new Array();
var xmlObj=GetXmlHttpObject();

window.onload=function() 
{
	self.focus();
	var urlX="http://rollerposter.pcriot.com/img_fix.php?act=1";	//collect index for each hot linked image
	xmlObj=GetXmlHttpObject();
	xmlObj.open("GET",urlX,false);
	xmlObj.send(null);

	try { var xmlDoc = xmlObj.responseXML; }
	catch(err)
	{
		alert("Unable to retrieve xmlDoc, indicies.  Cancelled.  "+err);
		return;
	}
	try { tag1=xmlDoc.getElementsByTagName("tag1")[0].childNodes[0].nodeValue;  }
	catch(err) 
	{
		alert("Unable to retrieve tag1, index string.  Cancelled.  "+err);
		return;
	}
	bmIndices=tag1.split("\n");
//	alert(xmlObj.statusText +"  found: "+bmIndices.length+" targets ("+xmlObj.responseText.length+"bytes),  indices: "+tag1);
//	var answer=confirm(xmlObj.statusText +"  found: "+bmIndices.length+" targets.\nClick \"OK\" to proceed.");
	if( !bmIndices.length ) return;
	bmMax=bmIndices.length;
	bmIndex=0;
	resetIndex();
}

//function startJob() { winInterval=window.setInterval('resetIndex();',3000); }

function step2()
{
	if (xmlObj.readyState==4 && xmlObj.status==200)
	{
		try { var xmlDoc = xmlObj.responseXML; }
		catch(err)
		{
			document.writeln("<hr />Unable to retrieve xmlDoc index#: "+bmIndices[bmIndex]+", skipped.  "+err+"<hr />");
		}
		try { tag1=unescape(xmlDoc.getElementsByTagName("tag1")[0].childNodes[0].nodeValue);  }
		catch(err) 
		{
			document.writeln("<hr />Unable to retrieve tag1 for index#: "+bmIndices[bmIndex]+", skipped.  "+err+"<hr />");
		}
		document.writeln("<br />index: "+bmIndices[bmIndex]+" #"+(bmIndex+1)+"of"+bmIndices.length+"  "+tag1+"<br />");
	}
}
function resetIndex()
{
	window.scrollTop=999999;
	var urlX="http://rollerposter.pcriot.com/img_fix.php?act=2&bm_index="+bmIndices[bmIndex]+"&sid="+Math.random();
	xmlObj=GetXmlHttpObject();
	xmlObj.onreadystatechange=function()
	{
		if (xmlObj.readyState==4 && xmlObj.status==200)
		{
			try { var xmlDoc = xmlObj.responseXML; }
			catch(err)
			{
			document.writeln("<hr />Unable to retrieve xmlDoc index#: "+bmIndices[bmIndex]+", skipped.  "+err+"<hr />");
			}
			try { tag1=unescape(xmlDoc.getElementsByTagName("tag1")[0].childNodes[0].nodeValue);  }
			catch(err) 
			{
				document.writeln("<hr />Unable to retrieve tag1 for index#: "+bmIndices[bmIndex]+", skipped.  "+err+"<hr />");
			}
			document.writeln("<br />index: "+bmIndices[bmIndex]+" #"+bmMax+"of"+bmIndices.length+"  "+tag1+"<br />");
			bmIndex+=1;
			bmMax-=1;
			if( bmMax>0 ) window.location.reload(true);
//			if( bmMax==0 ) window.clearInterval(winInterval);
		}
	}
	xmlObj.open("GET",urlX,true);
	xmlObj.send(null);
}

//-->
</script>
</head>
<body>

<div style="top:0; left:0; height:100%; width:100%; position:absolute;">
	<h1>Test</h1>
	<a href="#nogo" onclick="resetIndex(); return false;">Start</a>
	<hr />
</div>

</body>
</html>

PHP​

<?php
	header("Content-type: text/xml");
//	require "_roller.session.php";
	session_save_path("/home/floyd/public_html/tmp");
	ini_set("session.gc_maxlifetime", 86400);
	ini_set("session.gc_probability", 1);
	ini_set("session.gc_divisor",200);
	ini_set("session.bug_compat_warn", off);
	ini_set("allow_url_fopen", on);
	ini_set ('user_agent', $_SERVER['HTTP_USER_AGENT']);
 	session_start();
//	require "_roller.vars.php";
	$connect = mysql_connect("localhost", "floyd_floyd", "xxxxxxxx") or die("dB Connection Error... " . mysql_error());
	$dB = mysql_select_db("floyd_rollerposter", $connect) or die("Unable to select dB..." . mysql_error());
	date_default_timezone_set("America/Los_Angeles");
	$file_base = "/home/floyd/public_html";
	foreach ($_SESSION as $key => $value)  { $_SESSION[$key] = mysql_real_escape_string(trim($value)); }
	if(get_magic_quotes_gpc()) { $_GET = array_map('stripslashes', $_GET); }
	$_GET = array_map('strip_tags', $_GET);
	foreach ($_GET as $key => $value)  { $_GET[$key] = mysql_real_escape_string(trim($value)); }

//	$_table = mysql_query("SELECT * FROM _public WHERE archiveIndex > 6516");

	$act=(isset($_GET['act'])) ? $_GET['act'] : "1";
	switch($act)
	{
		case "1":
			collect_indices();
		break;
		case "2":
			$bm_index=(isset($_GET['bm_index'])) ? $_GET['bm_index'] : false;
			if( !$bm_index ) 
			{ 
				echo '<?xml version = "1.0" encoding = "UTF-8"?><tag0>';
				echo "<tag1>Error, index missing or invalid.</tag1>";
				echo "</tag0>";
				exit();
			}
			img_copy($bm_index);
		break;
		default:
			collect_indices();
	}

function collect_indices()
{
	$_table = mysql_query("SELECT * FROM _public");
	$indices=( mysql_num_rows($_table) > 0 ) ? "" : "Error: no bookmarks selected.";
	while($row1 = mysql_fetch_array($_table)) 
	{
		$archive_index=$row1['archiveIndex'];
		$img0=$row1['img0'];
		if( parse_url(strtolower($img0), PHP_URL_HOST) !== strtolower($_SERVER['SERVER_NAME']) )
		{
			$indices.=$archive_index."\n";
		}
	}
	echo '<?xml version = "1.0" encoding = "UTF-8"?><tag0>';
	echo "<tag1>$indices</tag1>";
	echo "</tag0>";
	exit();
}

function img_copy($archive_index)
{
	$types = array(1 => "GIF", 2 => "JPEG", 3 => "PNG", 4 => "SWF", 5 => "PSD", 6 => "BMP", 7 => "TIFF", 8 => "TIFF");
	$bm_count=0;
	$indices="";
	$im=false;
	$write_img=false;

	$_table = mysql_query("SELECT * FROM _public WHERE archiveIndex='$archive_index' LIMIT 1");
	$row1=mysql_fetch_array($_table);
	$img0=$row1['img0'];
	$img0_old=$row1['img0'];
	$img0_height=$row1['img0_height'];
	$img0_width=$row1['img0_width'];
	$tStamp=$row1['tStamp'];
	$return_msg="?  ".$img0_old;

	$imagetype=@exif_imagetype($img0);
	if ( $imagetype && array_key_exists($imagetype, $types) ) 
	{
		$temp=img_file_name($archive_index);
		switch( $types[$imagetype] )
		{
			case "GIF":
				$im=@imagecreatefromgif($img0);
				if( $im )
				{
					header('Content-Type: image/gif');
					$img0="http://rollerposter.pcriot.com/images/bm_images/$temp.gif";
					$write_img=@imagegif($im, "./images/bm_images/$temp.gif");
				}
			break;
			case "JPEG":
				$im=@imagecreatefromjpeg($img0);
				if( $im )
				{
					header('Content-Type: image/jpeg');
					$img0="http://rollerposter.pcriot.com/images/bm_images/$temp.jpg";
					$write_img=@imagejpeg($im, "./images/bm_images/$temp.jpg");
				}
			break;
			case "PNG":
				$im=@imagecreatefrompng($img0);
				if( $im )
				{
					header('Content-Type: image/png');
					$img0="http://rollerposter.pcriot.com/images/bm_images/$temp.png";
					$write_img=@imagepng($im, "./images/bm_images/$temp.png");
				}
			break;
			case "BMP":
				$im=@imagecreatefromwbmp($img0);
				if( $im )
				{
					header("Content-type: image/vnd.wap.wbmp");
					$img0="http://rollerposter.pcriot.com/images/bm_images/$temp.bmp";
					$write_img=@imagewbmp($im, "./images/bm_images/$temp.bmp");
				}
			break;
			default:
				$im=false;
				$write_img=false;
		}
	}
	if( !$im || !$write_img )
	{
		$img0="http://rollerposter.pcriot.com/images/bookmark.png";
		$img0_width=32;
		$img0_height=32;
	}
	$return_msg="OK, replaced $img0_old with $img0";
	$_update = mysql_query("UPDATE _public SET  img0='$img0', img0_height='$img0_height', img0_width='$img0_width' WHERE archiveIndex = '$archive_index' LIMIT 1");
	if( !$_update ) { $return_msg="Error, unable to update _public image $img0_old with $img0"; }
//	sleep(1);
	echo '<?xml version = "1.0" encoding = "UTF-8"?><tag0>';
	echo "<tag1>".htmlentities($return_msg,ENT_QUOTES,'UTF-8')."</tag1>";
	echo "</tag0>";
	exit();
}

function img_file_name($indx) 
{ 
	$aZ09 = array_merge(range('A', 'Z'), range('a', 'z'),range(0, 9)); 
	$out=$indx."."; 
	for($c=0;$c < 5;$c++) { $out .= $aZ09[mt_rand(0,count($aZ09)-1)]; } 
	return $out; 
}

?>

So, it's not pretty but it worked. I also made some changes to bookmark input and edit code to detect hot link image and copy it from the get-go. I learned a lot doing this project, although I'm glad it's done.

 

[misson]

Thanks for posting follow-ups. Not many do that.

Depending on how $_SESSION is filled out, your code may quote values stored therein multiple times. You should prepare data to be passed to a layer (e.g. quoting) at the point you're about to pass the data to the layer, not before. Thirdly, the mysql extension is on its way to deprecation. Switch to PDO and use prepared statements. If you need a PDO tutorial, try "Writing MySQL Scripts with PHP and PDO".