81 replies to this topic

[spooks]

How many times have you modified your site but the mod went wrong and you wish you had backed up first, this makes creating that backup very easy with just a single click.

This contribution creates a zip file of your entire site (excluding the database) for immediate download to your computer so making creation of a site backup quick and easy with just a single click.

Your backup, cached and sessions folders will be excluded, others can be easily added.

Security:
The code will check the user is logged into admin at all stages, so only administrators are able to obtain and download the compressed site copy.

Two zipping methods are provided, File and Memory (streaming).

File Mode Zipping.
The file method first creates a temporary zipped copy of the website within your backup folder then immediately provides the download of that, the temporary zipped copy is then immediately deleted. This process uses very little memory so will always work as long as there is sufficient disk space to store the temporary file.

Memory Mode Zipping.
With this method the zipping all occurs within memory and at the same time the download is provided (streaming). Memory usage is still relatively low so should not prove an issue, however this requires at least PHP Version 5.1.2, if your server has less than that the memory mode zipping option will not appear.

Compatibility: For osC 2.3.x and below


Contribution Download Available at http://addons.oscommerce.com/info/8387

Edited by spooks, 25 March 2012 - 02:28 PM.

[mattjt83]

Hi Sam,

I installed this mod today and get this error:

Fatal error: Class 'ZipArchive' not found in blah blah blah........

I looked in the includes/classes/zipstream.php file and it has the class as  class ZipStream {

I tried changing the ZipArchive to ZipStream in backup_website.php and then I got undefined function open.

Everything seems so straighforward so I'm not really sure where it went wrong...  

[spooks]

Hi,

You didn't do anything wrong and you don't necessarily need to change anything!  

ZipArchive is a standard php class, I assumed it would be included as standard, obviously I was wrong in that

Ask your hosts to add the ZipArchive class to your php

ZipArchive is only used for file mode zipping, the ZipStream class being used for memory mode (streaming)

To prevent the error message, change line 83 of backup_website.php from

$mode = isset($_POST['mode']) ? $_POST['mode'] : '';

to

$mode = isset($_POST['mode']) ? $_POST['mode'] : '';
   if ($mode == 'file' && !class_exists('ZipArchive')) $mode = '';

Edited by spooks, 25 March 2012 - 03:56 PM.

[mattjt83]

Thanks for the quick reply Sam!  I run the site on a vps server so they will make me do the change myself.  I will probably have to skip this mod then if it won't work without that class.  

[spooks]

@mattjt83

What version of PHP are you using then? Is it too early for the streaming option?

I have been considering adding a gzip option which would be much less restrictive, though I was thinking people would be using upto date servers generally.

This works on all the servers my clients use but perhaps some hosts don't take that much trouble.

[mattjt83]

php version is 5.3

[spooks]

@mattjt83

So the memory (streaming) option should work fine for you, does it not? or is there a reason you only want the file mode?

[mr_absinthe]

Sam, after clicking the "here" button in both the File Mode Zipping and the Memory Mode Zipping I receive the following errors:

Notice: Constant PREVNEXT_BUTTON_PREV already defined in /hosting/www/xxx/www/admin/includes/languages/czech.php on line 388 Notice: Constant PREVNEXT_BUTTON_NEXT already defined in /hosting/www/...

.../includes/languages/add_ccgvdc_english.php on line 32 Notice: Constant TEXT_VALID_CATEGORIES_NAME already defined in /hosting/www/xxx/www/admin/includes/languages/add_ccgvdc_english.php on line 33 Fatal error: Call to undefined function tep_href_link() in /hosting/www/xxx/www/admin/includes/languages/czech.php on line 440

[spooks]

Hi,

When you click either button the code performs a limited startup routine which does not call application _top.php thats as the output is not to be a page but just the download

The code does have to call some standard osC routines as the users status must be checked and some variable settings are needed.

The code calls languages/$language.php to get date formatting, it would appear your site or your browser is set to czech and that in turn is calling add_ccgvdc_english.php which seems to be doing some very non-standard stuff including adding links with tep_href_link

I would suspect you always get the notices, you just see them here as there is no page display.

The call to tep_href_link fails as my code doesn't call  DIR_WS_FUNCTIONS . 'html_output.php' as nothing in that is needed

what is happening with your language files and what made them that way?

[mr_absinthe]

Not sure what is happening to the language files - the czech language is there as the shop runs in the Czech Republic and the whole language pack has been added as usual - no other errors in any other contributions has been detected or seen. The previous version Backup Store Files has been working fine as well. Same errors do appear if I switch admin language to English.

[spooks]

Well clearly there is something odd that language pack, you wouldn't see the error generally as most add-ons will call application-top always, so instantiate   DIR_WS_FUNCTIONS . 'html_output.php');

You could modify this to allow the vagaries of your files, I see no point in modifying the contribution itself.

To stop the error, in backup_website.php:

after(90):

require(DIR_WS_FUNCTIONS . 'general.php');

add:

require(DIR_WS_FUNCTIONS . 'html_output.php');

and remove that same line from line 170 (appears there purely of error display)

To stop the Notice Display caused by conflicts within your language files

after(10):

$guess = 0; // ...

add:

error_reporting(E_ALL & ~E_NOTICE);

[mr_absinthe]

The solution above worked, however none of the .zip files could be xtracted, receiving errors in Directory Opus as well as in WinRAR - !   C:\Users\User\Downloads\Website-Backup-26-03-2012 (1).zip: The archive is either in unknown format or damaged.

[mattjt83]

@spooks I tried the second method and it just froze my computer LOL!  Probably more to do with my computer than anything but I never could get this to work.  Not a huge deal as I can still use ftp for this.  Thanks for your help!

Matt

[spooks]

@mr_absinthe

Was that with the memory (streaming) method, it states in the doc that can happen if there size estimate is too low, try increasing the guess value. For the file method it uses 2% over final file size but I believe in rare cases that may be insufficient, the problem is with zipped content the download stops the moment content_length is reached, if thats a single byte short of the full file it will be seen as corrupt.

Alternatively you can take size out of the equation by removing from ZipStream.php the line (521):

'Content-Length'			=> $this->size

for streaming & for file mode from backup_website.php the line(161)

header("Content-Length: ".floor(filesize(DIR_FS_BACKUP . $filename . $filedate)*1.02));

Though that will make the download less friendly.

@mattjt83

Could you say what platform your using (windows/linux) and if you could take a look in your apache error log to see if it sheds any light on what went wrong.

I must admit I haven't tested on a windows server but I thought I had caught all the bugs in ZipStream.php

Edited by spooks, 26 March 2012 - 04:23 PM.

[mr_absinthe]

I've increased $guess = 1; and now receiving this error when I use both File Mode Zipping and Memory Mode Zipping:

Warning: array_map(): Argument #2 should be an array in /hosting/www/xxx/www/admin/backup_website.php on line 121 plus many headers already sent warnings.
When I changed it to $guess = 2; - file downloaded but unable to open - corrupted. (Estimated compressed file size: 18.22 MB - real size 12 MB - quite a difference)
When changed to  $guess = 3; - same as  $guess = 1;

Edited by mr_absinthe, 26 March 2012 - 07:20 PM.

[spooks]

@mr_absinthe

Is your backup folder empty by any chance? Sorry, not something I thought of

change the line to

@array_map( "unlink", glob( DIR_FS_BACKUP . $filename."*" ) );

i.e. just add the @ in front.

Your issue isn't the size, its content sent before the header, any warnings/notices etc will cause that, ensure you have the code I posted before to stop the notices

[mr_absinthe]

I've added the code as instructed:

   $exclude = 'DIR_FS_BACKUP, DIR_FS_CACHE, SESSION_WRITE_DIRECTORY, contest/'; // add excluded as required, include full path from catalog folder i.e. includes/languages/
   ini_set('memory_limit', '128M'); // set memory limit, increase if you receive an out of memory error, you may need to speak to your host!!
   $guess = 4; // increase if size estimation is too low
   error_reporting(E_ALL & ~E_NOTICE);

The backup folder is not empty, but I've added the @ anyway:

@array_map( "unlink", glob( DIR_FS_BACKUP . $filename."*" ) );

$guess = 1; - corrupted file
$guess = 2; - corrupted file
$guess = 3; - corrupted file
$guess = 4; - corrupted file

Tested with both backup modes.The good news is that there are no errors, at least visible

[spooks]

@mr_absinthe

Its probably something in the headers your server don't like (could you say if windows or Linux) could you check the file creation is good, comment out line 169
    

@unlink(DIR_FS_BACKUP . $filename . $filedate);

then run file mode, then take a look in your backup folder, you will find the zipped file still there, if that's OK then its something in the download process thats causing it.

You can't test in memory mode, as no file is stored.

[mr_absinthe]

It is Linux, php 5.3.10.
$guess = 1;
File on the server is OK and the file that was downloaded is corrupted.
Estimated compressed file size: 17.96 MB - real size 12MB. So I played with $guess just for fun and the most realistic estimated was achieved with value $guess = -20;.
I've made no changes to ZipStream.php BTW.

Edited by mr_absinthe, 26 March 2012 - 09:20 PM.

[modem2.0]

Hi Spooks,

I've just installed this great contribuiton (thanks for providing it), but I'm having some strange behaviors.
I get the following statistics:

Website size: 24.16 MB Estimated compressed file size: 16.43 MB Containing 257 folders with 6467 Files.

I tested with both zipping methods with the following results (I have added other excluded folders):

• Testing with File Mode Zipping
  
  • gives a 500 error in the server. Looking at the logs I see the error

    [/list]
    Premature end of script headers: backup_website.php
    

  • When I look at the backup folder, I was expecting to see there the Zip file, or a part of it at least, but nothing, I just have there the database backup files. Even while the backup is running, I look at the folder and I see nothing there. Is it expected?

• Testing with Memory Mode Zipping (same configuration as for the other test):
  
  • I'm able to download the ZIP file, but as the estimated compressed file is 16.43MB and the real file is 11.1MB, the download runs well until it reaches the real file size, then it seems to be stopped for some time (I did not count the time) and then suddenly it says the download was complete. The file is ok, and extracts correctly.
    
  • Then, regarding the statistics I see:
    
    • The website size is more or less correct
      
    • the estimated compressed size is way too big (the downloaded file is 11.1MB or 11,677,866 bytes)
      
    • the file count is completely wrong, I get 2450 files, pus the Backup info.txt you add. I've done a "manual" backup to compare with, by using a complete copy of the site and deleting the excluded folders and I got 2458 files: all files with 0 bytes are skipped. All empty folders are also skipped. Is it normal?

I'm running on a shared server, under linux with PHP 5.2.14, and the guess varialble is set to 0.