Jump to content
spooks

Remove Unused Images [Support Thread]

Recommended Posts

Remove Unused Images

 

This is an old contrib, but had no official support thread, so here it is!

 

Scenario: I have 9000 image files. 2400 of them are not being used.

How to find them and remove them quickly and safely.

This script does just that.

 

Here is the full script, read it for more details.

Just copy it and run it in the admin root

 

What this script does:

* Read the database and report back a list of installed images

* Mark (in RED) any DB listed image which do not exist on the server

* Read the server images and report back a list

* Compare the lists for missing data

* Offer you the ability to check or uncheck listed images controlling what gets moved

* Move the USER checked listed images from myimage.jpg to UNUSED directory

* Allow you to ftp and delete ALL moved images

 

1.4 by Spooks

Added reading of category images

Added reading of manufaucturer images

Added option to check product description (default action) as many have html descriptions with images.

Added reading for some system files to check for images (default is header.php, index.php & stylesheet.css)

Added option to remove files from list by pattern match (ie thumbnails).

Removed some invalid html code

Modified to take image dir from configure.php

Corrected links so session wont be lost (could have caused failure of operation before &/or chuck you from admin)

Modified to move images to 'UNUSED' directory rather than renaming, means backup not needed and easier to revert. If files are in a sub-directory they will be moved to a sub-directory of 'UNUSED' with the same name.

Started support thread as I couldn't find one.

 

1.3 by azer :

removed by default the png and gif support

switch bak to osc standart one image product query

 

1.2 by azer :

added back some bacground on columns background bgcolor="#F5F5F5"

commented gif reports since it give a lot of infos on image type few people use for products photos

changed server variable to get them from configure.php variable

folowed the osc php file layout and call the js + html code on the proper section

changed the query to hide by default the product that are inactive

 

TODO: have a switch to show inactive , and out of stock or all products

TODO : have a switch to select wich extensions to show

TODO : "The total number of images on the server are" is wrong if the images are in subfolders

 

 

Contribution is found at http://addons.oscommerce.com/info/6346

Edited by spooks

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

Sam,

I'm having an issue with this .... hopefully you can give me some insight as to how I can resolve it. I keep getting the following error :

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 24 bytes) in /xxxxx/xxxxxxxx/xxxxxxx/catalog/admin/remove_unused_images.php on line 154

 

I use Ultrapics 2.07 so I had to modify the image array to

$table_array = array("products_image", "products_image_med", "products_image_lrg", "products_image_sm_1", "products_image_xl_1", "products_image_sm_2", "products_image_xl_2", "products_image_sm_3", "products_image_xl_3", "products_image_sm_4", "products_image_xl_4", "products_image_sm_5", "products_image_xl_5", "products_image_sm_6", "products_image_xl_6");

 

I have almost 2300 items in my local test store, and almost 500 in my online catalog. Some items have multiple images. I also use images in some of my info pages. I have tested on both sites with same error.

 

 

Any ideas?

Share this post


Link to post
Share on other sites

you have run out of memory while reading images from the dbase, try reducing the query.

 

you could try increasing your limit too, ie ini_set('memory_limit', '48M'); though you'll likely need to speak to your host.


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites
you have run out of memory while reading images from the dbase, try reducing the query.

 

you could try increasing your limit too, ie ini_set('memory_limit', '48M'); though you'll likely need to speak to your host.

 

 

Thanks a lot for the help, Sam. Greatly appreciated. Of course, in order to get it to run locally, I had to do ini_set('memory_limit', '1200M'); .

 

Michael

Share this post


Link to post
Share on other sites

in local with 400 product i got

 

Fatal error: Maximum execution time of 100 seconds exceeded in shop\admin\remove_unused_images.php on line 149

 

any ideas


MS2

Share this post


Link to post
Share on other sites

again thats just while reading product image data from the dbase, I'll have to take a closer look at that code & see if I cam make it more efficient.

 

Do you have many images in your dbase & had u expanded the sql?


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

Uploaded new version 1.41

 

 

Cleaned up and re-arranged the code to reduce query counts (and memory use)

Added new array for extra tables to make it easy to add extra tables into the mix (see 80 - 84 of file)


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

Hello Sam,

 

I installed your version 1.41 today, which is working really fine, thanks for the improvements.

I have installed UltraImage, so I added all image-fields to the array, which is also working.

 

But I have a problem:

I'm storing my images in many subfolders in the image-directory.

The function, which is gathering the filenames is unfortunately not working with subfolders, meaning automatically.

 

To work with it, I have to name every subdirectory I'm using.

And if I add a new product (and new images), if have to take care, that the new subdir is named in the function too.

Of course, this would work, but is not very efficient.

 

Is it possible to change that, so that the function is gathering all filenames in any subdir too?

 

Many thanks in advance for your input.

Regards

Andreas

Share this post


Link to post
Share on other sites
Is it possible to change that, so that the function is gathering all filenames in any subdir too?

 

I`m not sure it would be wise to automatically check all folders, maybe an option to list all folders so those to be checked can be ticked.

 

If I get time, and inclination, I'll look at it. :)


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites
I`m not sure it would be wise to automatically check all folders, maybe an option to list all folders so those to be checked can be ticked.

 

Of course, I only meant the subfolders in the images directory :-)

So to say, scan the directory recursive.

 

If you have concerns about that the function is scanning too much, maybe an exception folder list maybe a solution.

 

Thanks for any help,

Regards

Andreas

Edited by Andreas2003

Share this post


Link to post
Share on other sites

Hi

 

I have installed this wonderful contribution and everything seems to be working well but for one thing.

 

I am getting this error.

 

Warning: rename() [function.rename]: SAFE MODE Restriction in effect. The script whose uid is 10119 is not allowed to access /home/httpd/vhosts/mysite.com/httpdocs/images/UNUSED owned by uid 48 in /home/httpd/vhosts/mysite.com/httpdocs/admin/remove_unused_images.php on line 405

 

I have made the UNUSED directory myself in the images folder. = images/UNUSED

 

But still not working.

 

For my purposes I really don't need them in a UNUSED directory, the renaming the images with the prefix unused_ as it did before this GREAT update was working fine for me.

 

So...

 

Got any ideas how to fix this?

 

Or what coded to take out that did the moving to a new UNUSED directory and what code to put back in to just let it do the renaming of the image file only.

 

Any help would be appreciated.

 

Thanks


I'm having a great Day - hope you are too!

 

Leon

Share this post


Link to post
Share on other sites

Slightly over zelous settings on your server!! Roll on php 6!! ;)

 

The move uses rename as before, just in a different way, to revert:

 

change (394)

 

 

if ($checked_unused_images[$i])
{

  $hold_change = 1;

  //if image is in a folder create folder in UNUSED
  $pos = strpos($checked_unused_images[$i], '/');
  if ($pos){$dir = substr($checked_unused_images[$i],0,$pos);if(!is_dir($root_images_dir . 'UNUSED/' . $dir)) mkdir($root_images_dir . 'UNUSED/' .$dir);}

  $rename_image_diff = 'UNUSED/' . $checked_unused_images[$i];

  if (rename($root_images_dir . $checked_unused_images[$i], $root_images_dir . $rename_image_diff)){
  $msg_s .= '<font color="Blue">MOVED </font>' . $root_images_dir . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . $rename_image_diff . '<br>';
  }else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . ' did not get moved!<br />';}

}

 

to:

 

 

 

 

if ($checked_unused_images[$i])

{

$hold_change =1;

$rename_image_diff = '#UNUSED_' . $checked_unused_images[$i];

//if image is in a folder add the prefix

if (strpos($checked_unused_images[$i], '/')){

$rename_image_diff = str_replace('/', "/#UNUSED_", $checked_unused_images[$i]);//inserting the prefix after the /

}else{$rename_image_diff = '#UNUSED_' . $checked_unused_images[$i];}

if (rename($root_images_dir . '/' . $checked_unused_images[$i], $root_images_dir . '/' . $rename_image_diff)){

$msg_s .= '<font color="Blue">RENAMED </font>' . $root_images_dir . '/' . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . '/' . $rename_image_diff . '<br>';

}else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . 'did not get renamed';}

}


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites
Slightly over zelous settings on your server!! Roll on php 6!! ;)

 

The move uses rename as before, just in a different way, to revert:

 

change (394)

 

 

if ($checked_unused_images[$i])
{

  $hold_change = 1;

  //if image is in a folder create folder in UNUSED
  $pos = strpos($checked_unused_images[$i], '/');
  if ($pos){$dir = substr($checked_unused_images[$i],0,$pos);if(!is_dir($root_images_dir . 'UNUSED/' . $dir)) mkdir($root_images_dir . 'UNUSED/' .$dir);}

  $rename_image_diff = 'UNUSED/' . $checked_unused_images[$i];

  if (rename($root_images_dir . $checked_unused_images[$i], $root_images_dir . $rename_image_diff)){
  $msg_s .= '<font color="Blue">MOVED </font>' . $root_images_dir . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . $rename_image_diff . '<br>';
  }else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . ' did not get moved!<br />';}

}

 

to:

 

 

 

 

if ($checked_unused_images[$i])

{

$hold_change =1;

$rename_image_diff = '#UNUSED_' . $checked_unused_images[$i];

//if image is in a folder add the prefix

if (strpos($checked_unused_images[$i], '/')){

$rename_image_diff = str_replace('/', "/#UNUSED_", $checked_unused_images[$i]);//inserting the prefix after the /

}else{$rename_image_diff = '#UNUSED_' . $checked_unused_images[$i];}

if (rename($root_images_dir . '/' . $checked_unused_images[$i], $root_images_dir . '/' . $rename_image_diff)){

$msg_s .= '<font color="Blue">RENAMED </font>' . $root_images_dir . '/' . $checked_unused_images[$i] . '<font color="Blue"> TO </font>' . $root_images_dir . '/' . $rename_image_diff . '<br>';

}else{$msg_s .= 'UNKNOWN ERROR - The file ' . $checked_unused_images[$i] . 'did not get renamed';}

}

 

THANKS - That did it.

 

I have been wanting to remove the old images for a while and the fact that you fixed it to account for images that were part of product descriptions really helped.

 

Thanks again.

 

Leon


I'm having a great Day - hope you are too!

 

Leon

Share this post


Link to post
Share on other sites
THANKS - That did it.

 

Your welcome, if anyone else needs this there is one other edit you need to do:

 

find (384):

if(!is_dir($root_images_dir . 'UNUSED')) mkdir($root_images_dir . 'UNUSED');

and delete.


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

Uploaded new version V1.5

 

Modified, so rather than specify subfolders to include, you now specify an exluded list, otherwise all sub-folders of 'images' are checked

(non-recursive). However the subfolder check is 'off' by default, enable on line 54

 

Added option for product links for 'not-found' images, this will not include images 'not-found' within product descriptions or any extra tables checked. This opion is 'off' by default, see line 51.


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

sub folder otion is really good but doenst show for me

in lef i can see for database:

 

Image/2009/products_img/lrg/lrg_adid-0190-1240-2-1.jpg

 

that is in fact in : catalog/images/Image/plline/products_img/lrg/lrg_adid-0190-1240-2-1.jpg

 

but in middle this sub-subfolder is not scanned byt the program , any solution ?


MS2

Share this post


Link to post
Share on other sites
sub folder otion is really good but doenst show for me

in lef i can see for database:

 

Image/2009/products_img/lrg/lrg_adid-0190-1240-2-1.jpg

 

that is in fact in : catalog/images/Image/plline/products_img/lrg/lrg_adid-0190-1240-2-1.jpg

 

but in middle this sub-subfolder is not scanned byt the program , any solution ?

 

 

The programe is not designed to look any deeper than the first child level of 'images' you seem to have it reading a 4th level deep, did u input that url directly? or have u used a images dir starting deep?

 

The directory scan I`ve used should only give the first child level of 'images'.


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

if you use fck editor or any wysiwyg editor, you will have a subfolder in images folder so :/Image/2009/products_img is not so much : 2009 is year and then image or products

is there any way to make it look inside any subfolder, since there are many contribution who alloaws you to create subfolder while uploading ou product image

i could say there is one contribution already that does this : http://addons.oscommerce.com/info/5962

 

so maybe idea and code could be taken from there


MS2

Share this post


Link to post
Share on other sites

fck does not need to do that, if you look at http://addons.oscommerce.com/info/2900 that I did, it just uses the 'images' dir

 

In my experiance most only have 1 level of sub-dirs within images, I`ll look at modding it for a recursive mode though when I`ve time.


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

I upgraded to v1.5, very cool, great enhancement.

Thank you for the upgrade!

 

But I have the same problem as Azer => 1 child is (unfortunately) not enough :huh:

Share this post


Link to post
Share on other sites
Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/htdocs/ks013/html/catalog/admin/remove_unused_images.php on line 233

 

what i can do to speedup?

there are only 3000 images in the folder catalog/images for 900 products

thx

alfred

Edited by stragami

Share this post


Link to post
Share on other sites

 

in remove__unused_images.php

after:

require('includes/application_top.php');

add:

set_time_limit (0);

 

Thiat change will not work if your server is working in 'safe mode'


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

Hi There!

This looks a nice contrib.

Scanning my image folder it "detects" as unused also images that are used :((

Also i've added another field called products_image_2 but this isn't considered too.

Any idea?

Tx

Fab


Advice on forum are Free, Email or Pm to fix your site is work...which I charge for :)

-------------------

Share this post


Link to post
Share on other sites

Hi There!

This looks a nice contrib.

Scanning my image folder it "detects" as unused also images that are used sad.gif(

Also i've added another field called products_image_2 but this isn't considered too.

Any idea?

Tx

Fab

 

 

this will happen if you add extra fields and fail to reflect that in the checking sql, edit the file to suit.


Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×