spooks Posted August 31, 2009 Share Posted August 31, 2009 (edited) Remove Unused Images This is an old contrib, but had no official support thread, so here it is! Scenario: I have 9000 image files. 2400 of them are not being used. How to find them and remove them quickly and safely. This script does just that. Here is the full script, read it for more details. Just copy it and run it in the admin root What this script does: * Read the database and report back a list of installed images * Mark (in RED) any DB listed image which do not exist on the server * Read the server images and report back a list * Compare the lists for missing data * Offer you the ability to check or uncheck listed images controlling what gets moved * Move the USER checked listed images from myimage.jpg to UNUSED directory * Allow you to ftp and delete ALL moved images 1.4 by Spooks Added reading of category images Added reading of manufaucturer images Added option to check product description (default action) as many have html descriptions with images. Added reading for some system files to check for images (default is header.php, index.php & stylesheet.css) Added option to remove files from list by pattern match (ie thumbnails). Removed some invalid html code Modified to take image dir from configure.php Corrected links so session wont be lost (could have caused failure of operation before &/or chuck you from admin) Modified to move images to 'UNUSED' directory rather than renaming, means backup not needed and easier to revert. If files are in a sub-directory they will be moved to a sub-directory of 'UNUSED' with the same name. Started support thread as I couldn't find one. 1.3 by azer : removed by default the png and gif support switch bak to osc standart one image product query 1.2 by azer : added back some bacground on columns background bgcolor="#F5F5F5" commented gif reports since it give a lot of infos on image type few people use for products photos changed server variable to get them from configure.php variable folowed the osc php file layout and call the js + html code on the proper section changed the query to hide by default the product that are inactive TODO: have a switch to show inactive , and out of stock or all products TODO : have a switch to select wich extensions to show TODO : "The total number of images on the server are" is wrong if the images are in subfolders Contribution is found at http://addons.oscommerce.com/info/6346 Edited August 31, 2009 by spooks Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
MichaelB Posted August 31, 2009 Share Posted August 31, 2009 Sam, I'm having an issue with this .... hopefully you can give me some insight as to how I can resolve it. I keep getting the following error : Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 24 bytes) in /xxxxx/xxxxxxxx/xxxxxxx/catalog/admin/remove_unused_images.php on line 154 I use Ultrapics 2.07 so I had to modify the image array to $table_array = array("products_image", "products_image_med", "products_image_lrg", "products_image_sm_1", "products_image_xl_1", "products_image_sm_2", "products_image_xl_2", "products_image_sm_3", "products_image_xl_3", "products_image_sm_4", "products_image_xl_4", "products_image_sm_5", "products_image_xl_5", "products_image_sm_6", "products_image_xl_6"); I have almost 2300 items in my local test store, and almost 500 in my online catalog. Some items have multiple images. I also use images in some of my info pages. I have tested on both sites with same error. Any ideas? Quote Link to comment Share on other sites More sharing options...
spooks Posted August 31, 2009 Author Share Posted August 31, 2009 you have run out of memory while reading images from the dbase, try reducing the query. you could try increasing your limit too, ie ini_set('memory_limit', '48M'); though you'll likely need to speak to your host. Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
MichaelB Posted August 31, 2009 Share Posted August 31, 2009 you have run out of memory while reading images from the dbase, try reducing the query. you could try increasing your limit too, ie ini_set('memory_limit', '48M'); though you'll likely need to speak to your host. Thanks a lot for the help, Sam. Greatly appreciated. Of course, in order to get it to run locally, I had to do ini_set('memory_limit', '1200M'); . Michael Quote Link to comment Share on other sites More sharing options...
azer Posted September 1, 2009 Share Posted September 1, 2009 in local with 400 product i got Fatal error: Maximum execution time of 100 seconds exceeded in shop\admin\remove_unused_images.php on line 149 any ideas Quote MS2 Link to comment Share on other sites More sharing options...
spooks Posted September 1, 2009 Author Share Posted September 1, 2009 again thats just while reading product image data from the dbase, I'll have to take a closer look at that code & see if I cam make it more efficient. Do you have many images in your dbase & had u expanded the sql? Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
spooks Posted September 1, 2009 Author Share Posted September 1, 2009 Uploaded new version 1.41 Cleaned up and re-arranged the code to reduce query counts (and memory use) Added new array for extra tables to make it easy to add extra tables into the mix (see 80 - 84 of file) Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
Andreas2003 Posted September 2, 2009 Share Posted September 2, 2009 Hello Sam, I installed your version 1.41 today, which is working really fine, thanks for the improvements. I have installed UltraImage, so I added all image-fields to the array, which is also working. But I have a problem: I'm storing my images in many subfolders in the image-directory. The function, which is gathering the filenames is unfortunately not working with subfolders, meaning automatically. To work with it, I have to name every subdirectory I'm using. And if I add a new product (and new images), if have to take care, that the new subdir is named in the function too. Of course, this would work, but is not very efficient. Is it possible to change that, so that the function is gathering all filenames in any subdir too? Many thanks in advance for your input. Regards Andreas Quote Link to comment Share on other sites More sharing options...
spooks Posted September 2, 2009 Author Share Posted September 2, 2009 Is it possible to change that, so that the function is gathering all filenames in any subdir too? I`m not sure it would be wise to automatically check all folders, maybe an option to list all folders so those to be checked can be ticked. If I get time, and inclination, I'll look at it. :) Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
Andreas2003 Posted September 2, 2009 Share Posted September 2, 2009 (edited) I`m not sure it would be wise to automatically check all folders, maybe an option to list all folders so those to be checked can be ticked. Of course, I only meant the subfolders in the images directory :-) So to say, scan the directory recursive. If you have concerns about that the function is scanning too much, maybe an exception folder list maybe a solution. Thanks for any help, Regards Andreas Edited September 2, 2009 by Andreas2003 Quote Link to comment Share on other sites More sharing options...
bibleman Posted September 3, 2009 Share Posted September 3, 2009 Hi I have installed this wonderful contribution and everything seems to be working well but for one thing. I am getting this error. Warning: rename() [function.rename]: SAFE MODE Restriction in effect. The script whose uid is 10119 is not allowed to access /home/httpd/vhosts/mysite.com/httpdocs/images/UNUSED owned by uid 48 in /home/httpd/vhosts/mysite.com/httpdocs/admin/remove_unused_images.php on line 405 I have made the UNUSED directory myself in the images folder. = images/UNUSED But still not working. For my purposes I really don't need them in a UNUSED directory, the renaming the images with the prefix unused_ as it did before this GREAT update was working fine for me. So... Got any ideas how to fix this? Or what coded to take out that did the moving to a new UNUSED directory and what code to put back in to just let it do the renaming of the image file only. Any help would be appreciated. Thanks Quote I'm having a great Day - hope you are too! Leon Link to comment Share on other sites More sharing options...
spooks Posted September 4, 2009 Author Share Posted September 4, 2009 Slightly over zelous settings on your server!! Roll on php 6!! ;) The move uses rename as before, just in a different way, to revert: change (394) if ($checked_unused_images[$i]) { $hold_change = 1; //if image is in a folder create folder in UNUSED $pos = strpos($checked_unused_images[$i], '/'); if ($pos){$dir = substr($checked_unused_images[$i],0,$pos);if(!is_dir($root_images_dir . 'UNUSED/' . $dir)) mkdir($root_images_dir . 'UNUSED/' .$dir);} $rename_image_diff = 'UNUSED/' . $checked_unused_images[$i]; if (rename($root_images_dir . $checked_unused_images[$i], $root_images_dir . $rename_image_diff)){ $msg_s .= '<font color="Blue">MOVED </font>' . $root_images_dir . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . $rename_image_diff . '<br>'; }else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . ' did not get moved!<br />';} } to: if ($checked_unused_images[$i]) { $hold_change =1; $rename_image_diff = '#UNUSED_' . $checked_unused_images[$i]; //if image is in a folder add the prefix if (strpos($checked_unused_images[$i], '/')){ $rename_image_diff = str_replace('/', "/#UNUSED_", $checked_unused_images[$i]);//inserting the prefix after the / }else{$rename_image_diff = '#UNUSED_' . $checked_unused_images[$i];} if (rename($root_images_dir . '/' . $checked_unused_images[$i], $root_images_dir . '/' . $rename_image_diff)){ $msg_s .= '<font color="Blue">RENAMED </font>' . $root_images_dir . '/' . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . '/' . $rename_image_diff . '<br>'; }else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . 'did not get renamed';} } Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
bibleman Posted September 4, 2009 Share Posted September 4, 2009 Slightly over zelous settings on your server!! Roll on php 6!! ;) The move uses rename as before, just in a different way, to revert: change (394) if ($checked_unused_images[$i]) { $hold_change = 1; //if image is in a folder create folder in UNUSED $pos = strpos($checked_unused_images[$i], '/'); if ($pos){$dir = substr($checked_unused_images[$i],0,$pos);if(!is_dir($root_images_dir . 'UNUSED/' . $dir)) mkdir($root_images_dir . 'UNUSED/' .$dir);} $rename_image_diff = 'UNUSED/' . $checked_unused_images[$i]; if (rename($root_images_dir . $checked_unused_images[$i], $root_images_dir . $rename_image_diff)){ $msg_s .= '<font color="Blue">MOVED </font>' . $root_images_dir . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . $rename_image_diff . '<br>'; }else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . ' did not get moved!<br />';} } to: if ($checked_unused_images[$i]) { $hold_change =1; $rename_image_diff = '#UNUSED_' . $checked_unused_images[$i]; //if image is in a folder add the prefix if (strpos($checked_unused_images[$i], '/')){ $rename_image_diff = str_replace('/', "/#UNUSED_", $checked_unused_images[$i]);//inserting the prefix after the / }else{$rename_image_diff = '#UNUSED_' . $checked_unused_images[$i];} if (rename($root_images_dir . '/' . $checked_unused_images[$i], $root_images_dir . '/' . $rename_image_diff)){ $msg_s .= '<font color="Blue">RENAMED </font>' . $root_images_dir . '/' . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . '/' . $rename_image_diff . '<br>'; }else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . 'did not get renamed';} } THANKS - That did it. I have been wanting to remove the old images for a while and the fact that you fixed it to account for images that were part of product descriptions really helped. Thanks again. Leon Quote I'm having a great Day - hope you are too! Leon Link to comment Share on other sites More sharing options...
spooks Posted September 4, 2009 Author Share Posted September 4, 2009 THANKS - That did it. Your welcome, if anyone else needs this there is one other edit you need to do: find (384): if(!is_dir($root_images_dir . 'UNUSED')) mkdir($root_images_dir . 'UNUSED'); and delete. Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
spooks Posted September 4, 2009 Author Share Posted September 4, 2009 Uploaded new version V1.5 Modified, so rather than specify subfolders to include, you now specify an exluded list, otherwise all sub-folders of 'images' are checked (non-recursive). However the subfolder check is 'off' by default, enable on line 54 Added option for product links for 'not-found' images, this will not include images 'not-found' within product descriptions or any extra tables checked. This opion is 'off' by default, see line 51. Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
azer Posted September 4, 2009 Share Posted September 4, 2009 sub folder otion is really good but doenst show for me in lef i can see for database: Image/2009/products_img/lrg/lrg_adid-0190-1240-2-1.jpg that is in fact in : catalog/images/Image/plline/products_img/lrg/lrg_adid-0190-1240-2-1.jpg but in middle this sub-subfolder is not scanned byt the program , any solution ? Quote MS2 Link to comment Share on other sites More sharing options...
spooks Posted September 4, 2009 Author Share Posted September 4, 2009 sub folder otion is really good but doenst show for me in lef i can see for database: Image/2009/products_img/lrg/lrg_adid-0190-1240-2-1.jpg that is in fact in : catalog/images/Image/plline/products_img/lrg/lrg_adid-0190-1240-2-1.jpg but in middle this sub-subfolder is not scanned byt the program , any solution ? The programe is not designed to look any deeper than the first child level of 'images' you seem to have it reading a 4th level deep, did u input that url directly? or have u used a images dir starting deep? The directory scan I`ve used should only give the first child level of 'images'. Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
azer Posted September 5, 2009 Share Posted September 5, 2009 if you use fck editor or any wysiwyg editor, you will have a subfolder in images folder so :/Image/2009/products_img is not so much : 2009 is year and then image or products is there any way to make it look inside any subfolder, since there are many contribution who alloaws you to create subfolder while uploading ou product image i could say there is one contribution already that does this : http://addons.oscommerce.com/info/5962 so maybe idea and code could be taken from there Quote MS2 Link to comment Share on other sites More sharing options...
spooks Posted September 5, 2009 Author Share Posted September 5, 2009 fck does not need to do that, if you look at http://addons.oscommerce.com/info/2900 that I did, it just uses the 'images' dir In my experiance most only have 1 level of sub-dirs within images, I`ll look at modding it for a recursive mode though when I`ve time. Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
Andreas2003 Posted September 10, 2009 Share Posted September 10, 2009 I upgraded to v1.5, very cool, great enhancement. Thank you for the upgrade! But I have the same problem as Azer => 1 child is (unfortunately) not enough :huh: Quote Link to comment Share on other sites More sharing options...
stragami Posted September 25, 2009 Share Posted September 25, 2009 (edited) Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/htdocs/ks013/html/catalog/admin/remove_unused_images.php on line 233 what i can do to speedup? there are only 3000 images in the folder catalog/images for 900 products thx alfred Edited September 25, 2009 by stragami Quote Link to comment Share on other sites More sharing options...
spooks Posted September 25, 2009 Author Share Posted September 25, 2009 in remove__unused_images.php after: require('includes/application_top.php'); add: set_time_limit (0); Thiat change will not work if your server is working in 'safe mode' Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
stragami Posted September 25, 2009 Share Posted September 25, 2009 You are a STAR ! Quote Link to comment Share on other sites More sharing options...
Freeman Posted November 6, 2009 Share Posted November 6, 2009 Hi There! This looks a nice contrib. Scanning my image folder it "detects" as unused also images that are used :(( Also i've added another field called products_image_2 but this isn't considered too. Any idea? Tx Fab Quote Advice on forum are Free, Email or Pm to fix your site is work...which I charge for :) ------------------- Link to comment Share on other sites More sharing options...
spooks Posted November 6, 2009 Author Share Posted November 6, 2009 Hi There! This looks a nice contrib. Scanning my image folder it "detects" as unused also images that are used ( Also i've added another field called products_image_2 but this isn't considered too. Any idea? Tx Fab this will happen if you add extra fields and fail to reflect that in the checking sql, edit the file to suit. Quote Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al. Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.