Jump to content



Photo
* * * * * 1 votes

Contribution: Auto Backup Database


  • Please log in to reply
181 replies to this topic

#1   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 25 May 2008 - 16:14

Auto Backup

This contribution automatically backs up your database at regular intervals whilst you are logged into admin.
It will create an archive backup at a longer intervals, you set the intervals in admin.
It can also zips the backups, if required, also set in admin.

The zip function uses the pear library to avoid problems with disabled system etc functions.

Contrib is at http://addons.oscommerce.com/info/2314
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#2   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 25 May 2008 - 16:38

It can also zips the backups, if required, also set in admin.

The zip function uses the pear library to avoid problems with disabled system etc functions.

I wonder why you bother because as far as I know gzip'ed files can be unzipped with Windows programs fine and gzip functions are much more widely supported in PHP (so no need to rely on external libraries).

#3   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 25 May 2008 - 18:54

I wonder why you bother because as far as I know gzip'ed files can be unzipped with Windows programs fine and gzip functions are much more widely supported in PHP (so no need to rely on external libraries).


I am not that familier with gzip, perhaps you are? I was not able to get gzip to reliably compress the backup without using system or shell_exec, & as I mentioned thier restricted, so not an option. Clearly as this is a backup it must be reliable.

If you have a better solution you would like to share I`d love to see it.

/smile.gif' class='bbc_emoticon' alt=':)' />
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#4   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 25 May 2008 - 19:15

If you have a better solution you would like to share I`d love to see it.

I'm not saying it is better, but I think it would be easier then geting your hosting provider to add PEAR (if you already have that available there is no problem of course).

I used the following in Database backup manager (addon #5769). To a new file for admin/includes/functions called db_backup.php the function osc_zip was added:
function osc_gzip($directory, $file_in, $delete_file = false, $level = 6) {
  $in_file = $directory . $file_in;
  $out_file = $directory . $file_in . '.gz';
  if (!file_exists ($in_file) || !is_readable ($in_file)) {
	return false;
  }
  if (file_exists($out_file)) {
	return false;
  }
  $fin_file = fopen($in_file, "rb");
  if (!$fout_file = gzopen($out_file, "wb".$level)) {
	return false;
  }

  while (!feof ($fin_file)) {
   $buffer = fread($fin_file, 8192); // 8 kB is maximum value
   gzwrite($fout_file, $buffer, 8192);
  }

  fclose($fin_file);
  gzclose($fout_file);
  if ($delete_file == true) {
	unlink($in_file);
  }
  return true;
}
Note that I didn't invented this, I "composed" it from various examples.
This is called in backup.php as follows (if the system gzip is not available, it looks if the function gzwrite is available):
switch ($_POST['compress']) {
			case 'gzip':
			  if (@file_exists(LOCAL_EXE_GZIP) && $safe_mode_setting == false) {
				exec(LOCAL_EXE_GZIP . ' ' . DIR_FS_BACKUP . $backup_file);
				unlink(DIR_FS_BACKUP . $backup_file);
			  } elseif (@function_exists('gzwrite')) {
				$gzip_result = osc_gzip (DIR_FS_BACKUP, $backup_file, true);
				  if (!$gzip_result) {
					$messageStack->add_session(ERROR_ON_GZIP, 'error');
				  } 
			  }
			  break;
Of course the function osc_gunzip was added to the functions file too in case the system gunzip is not available but gzopen gzgets are:
function osc_gunzip($directory, $file_in, $delete_file = false) {
  $in_file = $directory . $file_in;
  $out_file = substr($in_file, 0, -3);
  if (!file_exists ($in_file) || !is_readable ($in_file)) {
	return false;
  }
  if (file_exists($out_file)) {
	return false;
  }
  $fin_file = gzopen($in_file, "rb");
  if (!$fout_file = fopen($out_file, "wb")) {
	return false;
  }

  while (!gzeof ($fin_file)) {
   $buffer = gzgets($fin_file, 8192);
   fputs($fout_file, $buffer, 8192);
  }

  gzclose($fin_file);
  fclose($fout_file);
  if ($delete_file == true) {
	unlink($in_file);
  }
  return true;
}


#5   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 25 May 2008 - 19:54

Thanks for that, I`ll look at incorporating some of that in a future release as some may find it a hastle to ask their host for a install (or host may not be cooperative). I think the buffers you have set may have been my stumbling block.

The Pear solution is clearly much neater mind, only 2 lines of code needed!

I was glad to see you`ve released database_backup_manager as I had noticed bugs in the stock backup.php, I would see these two contribs as complimentary, my clients like the security of a automatic backup, but Auto Backup has no restore functions.

/cool.gif' class='bbc_emoticon' alt='B)' />
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#6   carryG

carryG
  • Members
  • 125 posts
  • Real Name:Carry G
  • Gender:Female
  • Location:USA

Posted 25 May 2008 - 20:39

Installed this and it worked the first time, auto created the sql file, but hasn't worked since....?

I did the install sql but see nothing in admin to adjust this mod, am I missing something? It seems it does not add it to the configuration menu as intended or...?

#7   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 25 May 2008 - 20:50

Installed this and it worked the first time, auto created the sql file, but hasn't worked since....?

I did the install sql but see nothing in admin to adjust this mod, am I missing something? It seems it does not add it to the configuration menu as intended or...?



The default settings are backups every 20mins & archives every 24Hrs, if you want to check its still working, just delete the last backup with the standard backup tool, it should make another immediatly.

As mentioned in the install controls are in module options, they seem to be hidden on most versions?? to alter install Total Configuration http://addons.oscommerce.com/info/5040 , its a quick, easy install.

/smile.gif' class='bbc_emoticon' alt=':)' />
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#8   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 25 May 2008 - 20:52

I would see these two contribs as complimentary

Absolutely true although Auto Backup Database might hit the same brick wall as the stock backup.php when you have a large database and/or a medium sized database on a slow server: time outs. The Database backup manager contribution can pause, reload and continue where it has stopped (included the ideas and [modified] code of another program for that: Xt-Dump).

#9   carryG

carryG
  • Members
  • 125 posts
  • Real Name:Carry G
  • Gender:Female
  • Location:USA

Posted 25 May 2008 - 21:13

Yes, I see that now, it did generate another one. Will it delete the sqls after awhile or just pile up hundreds and hundreds after a long period of time? Great mod btw.

#10   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 25 May 2008 - 21:21

Yes, I see that now, it did generate another one. Will it delete the sqls after awhile or just pile up hundreds and hundreds after a long period of time? Great mod btw.


No, thats what the original mod did, as I thought I made clear If a old backup less than maximum age does`nt exist, last backup is kept.
IE with 'Backup Save Interval' set to 12Hrs a archived backup is created every 12Hrs.

So with default settings every 24Hrs your backup list grows by one, but initially you will get two close together, but the time difference will increase upto the maximum then it repeats.

/cool.gif' class='bbc_emoticon' alt='B)' />
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#11   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 25 May 2008 - 22:12

Absolutely true although Auto Backup Database might hit the same brick wall as the stock backup.php when you have a large database and/or a medium sized database on a slow server: time outs. The Database backup manager contribution can pause, reload and continue where it has stopped (included the ideas and [modified] code of another program for that: Xt-Dump).



Good point, not seen that problem so far though. How did you slow down your server to test this? I don't have access to any slow servers (thankfully) so might have trouble reproducing any fault.
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#12   azer

azer
  • Members
  • 842 posts
  • Real Name:azer
  • Location:Paris, france

Posted 26 May 2008 - 10:38

Hi i need some help
i installed the 2.3 version with zip, i installed zip.php and checked pear.php is included well
when i go in my admin page i got this error :

Notice: strtotime() [function.strtotime]: Called with empty time parameter in v:\easyphp\www\ms2fr\shop\admin\includes\auto_backup_db.php on line 53

Notice: strtotime() [function.strtotime]: Called with empty time parameter in v:\easyphp\www\ms2fr\shop\admin\includes\auto_backup_db.php on line 54


code for those line is :
Ligne 52 : $at_current_date = strtotime(date('YmdHi'));

Ligne 53 : $at_dif_date = ($at_current_date - strtotime($at_entry))/60;

Ligne 54 : $at_last_date = ($at_current_date - strtotime($at_last))/60;

REQUEST: Being able to define the date format
by the way in the Language French 30 Nov 2004 version someone has witten some modifications to have the database name handled with european format : dmY

REQUEST 2 : being able to chose between system ZIP process or Pear process

/blush.gif' class='bbc_emoticon' alt=':blush:' />
MS2

#13   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 26 May 2008 - 11:06

You must have an old version of php on your server, no-one else has seen this error, its occured as you have no files yet, the file dates are extracted from the file name, and converted at this point, clearly your version of php cannot cope with the null value.

I`ll do a update later, but its not urgent as by now you will probably have 2 backup files, so no error anymore.

The date format stored in the filename is infact YmdHi as mentioned above this is integral to the function, so to change would mean more messing around on read. Since you also get the file date in your store format displayed on backup.php that seems rather unecessary.

This version use pear & zip, the pear library uses the zip function, so you must have both, so there is no choice.
As stated ealier in this forum I`ll look later at including gzip, but its a lot of extra code for a small file.

Hope that clears up your problem.

/cool.gif' class='bbc_emoticon' alt='B)' />
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#14   azer

azer
  • Members
  • 842 posts
  • Real Name:azer
  • Location:Paris, france

Posted 26 May 2008 - 13:26

for gzip , good exemple would be "Backup using GZIP" : http://www.oscommerc...tributions,3851

i hope it wouls save you some time :-)
MS2

#15   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 26 May 2008 - 13:45

I keep coming up with this, it uses exec, it might be the easy way, but that command is restricted, so its a no go.
I don't believe in producing anything that might require people to reduce their security to use it.

Cheers anyway.

/cool.gif' class='bbc_emoticon' alt='B)' />
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#16   azer

azer
  • Members
  • 842 posts
  • Real Name:azer
  • Location:Paris, france

Posted 26 May 2008 - 13:55

You must have an old version of php on your server, no-one else has seen this error, its occured as you have no files yet, the file dates are extracted from the file name, and converted at this point, clearly your version of php cannot cope with the null value.

I`ll do a update later, but its not urgent as by now you will probably have 2 backup files, so no error anymore.

The date format stored in the filename is infact YmdHi as mentioned above this is integral to the function, so to change would mean more messing around on read. Since you also get the file date in your store format displayed on backup.php that seems rather unecessary.

This version use pear & zip, the pear library uses the zip function, so you must have both, so there is no choice.
As stated ealier in this forum I`ll look later at including gzip, but its a lot of extra code for a small file.

Hope that clears up your problem.

/cool.gif' class='bbc_emoticon' alt='B)' />


For my versions : Apache 1.3.41 - PHP 4.4.7 - MySQL 4.1.22 - PhpMyAdmin 2.5.3 - gd 2.015
for now i use in local serveur and i got zip.php et pear.php working (tested)

thanks you for the time you take

for the date format it's more in the naming of the backup file than the functions used, but i think seing the changement done in the version "Language French 30 Nov 2004" would give you more infos.
MS2

#17   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 26 May 2008 - 15:45

My server versions are Apach 2.2.8 PHP 5.2.6 SQL 4.1.22 MyAdmin 2.11.4, my host obviously keeps thing up to date.

I took a quick look a the french version & the only difference I could see was it uses YmdHis, I removed the seconds as not needed
, I can assure you it would make the functions messy if you used dmYHi as you`d have to re-arrange that on read.

Your not making the same error supremecenterhosting made are you, thinking that the date in the file name was just cosmetic?
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#18   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 30 May 2008 - 21:11

Added gzip option, thanks to Jan Zonjee for the gzip function code.
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.

#19   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 30 May 2008 - 21:34

How did you slow down your server to test this? I don't have access to any slow servers (thankfully) so might have trouble reproducing any fault.

I didn't, I added a large table that has nothing to do with osCommerce to the database to make sure my computer (ok not really fast) took more than 30 seconds to backup to or restore from a backup file (several minutes for example).

#20   spooks

spooks
  • Members
  • 7,017 posts
  • Real Name:Sam
  • Gender:Male
  • Location:UK

Posted 30 May 2008 - 21:48

I didn't, I added a large table that has nothing to do with osCommerce to the database to make sure my computer (ok not really fast) took more than 30 seconds to backup to or restore from a backup file (several minutes for example).



Ok, would you mind saving me the trouble of creating that by PMing it to me so I can test.

Thanks
Sam

Remember, What you think I ment may not be what I thought I ment when I said it.

Contributions:


Auto Backup your Database, Easy way

Multi Images with Fancy Pop-ups, Easy way

Products in columns with multi buy etc etc

Disable any Category or Product, Easy way

Secure & Improve your account pages et al.