Jump to content



Photo
* * * * * 1 votes

[Contribution] Database Manager


  • Please log in to reply
110 replies to this topic

#1   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 24 February 2008 - 01:25

This contribution aims to get the database backup manager in osC 2.2 to work better with large databases especially on servers where safe_mode is set to on.

#2   xavkick

xavkick
  • Members
  • 66 posts
  • Real Name:xavier

Posted 25 February 2008 - 01:02

This contribution aims to get the database backup manager in osC 2.2 to work better with large databases especially on servers where safe_mode is set to on.



Hi, I have try your contribution, but i still have problem to backup my database.
When i launch you contrib, after 30 second, i have the backup but when i look in the backup i do not have all the table.

xav

#3   jumpwire

jumpwire
  • Members
  • 2 posts
  • Real Name:John
  • Gender:Male

Posted 29 February 2008 - 05:24

This contribution aims to get the database backup manager in osC 2.2 to work better with large databases especially on servers where safe_mode is set to on.


uploaded the files and i get this error:

Warning: array_multisort() [function.array-multisort]: Argument #1 is expected to be an array or a sort flag in /"myroot"/catalog/admin/backup.php on line 1112

any ideas?? thanks

john

#4   invasi0n

invasi0n
  • Members
  • 158 posts
  • Real Name:Andrew

Posted 09 March 2008 - 08:17

Hi,

I downloaded the contribution and i think everything works perfect. There's just one thing that I don't understand... You said that we have to let the script to do its job, and do not click on anything, but screenshot_restore_1.gif shows a Submit button. What is its use?

Thanks.

#5   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 09 March 2008 - 08:31

There's just one thing that I don't understand... You said that we have to let the script to do its job, and do not click on anything, but screenshot_restore_1.gif shows a Submit button. What is its use?

The script relies on a tiny bit of JavaScript to submit itself. In the unlikely event you can work in the admin with JavaScript turned off the script wouldn't work so therefore the Submit button (that I have to hide again if the submit is in progress otherwise you start two backup processes...). You can also hit the submit button if you can't wait the 4 seconds :)

#6   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 09 March 2008 - 08:39

Hi, I have try your contribution, but i still have problem to backup my database.
When i launch you contrib, after 30 second, i have the backup but when i look in the backup i do not have all the table.

Sorry guys, didn't notice your posts until today. Regarding your problem, this contribution was supposed to avoid that. Perhaps safe mode is off but still you cannot increase the time a script can run with tep_set_time_limit. Try "forcing" the script to avoid running any longer than 30 seconds please by commenting line 24 and uncommenting line 25 in admin/backup.php:

// $safe_mode_setting = (@ini_get('safe_mode') == 'On' || (@ini_get('safe_mode') === 1)) ? true : false;
$safe_mode_setting = true; // for testing
After 25 seconds or a little over it you should be presented with the intermediate screen like shown in screenshot_reload_backup.gif.

#7   invasi0n

invasi0n
  • Members
  • 158 posts
  • Real Name:Andrew

Posted 09 March 2008 - 08:41

The script relies on a tiny bit of JavaScript to submit itself. In the unlikely event you can work in the admin with JavaScript turned off the script wouldn't work so therefore the Submit button (that I have to hide again if the submit is in progress otherwise you start two backup processes...). You can also hit the submit button if you can't wait the 4 seconds /smile.gif' class='bbc_emoticon' alt=':)' />

I understand now. Thanks.

#8   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 09 March 2008 - 08:44

uploaded the files and i get this error:

Warning: array_multisort() [function.array-multisort]: Argument #1 is expected to be an array or a sort flag in /"myroot"/catalog/admin/backup.php on line 1112

any ideas??

Perhaps you don't have any backups yet? That's a scenario I didn't think of (and didn't test). Oops.

It does happen when there are no backups. If I add another two lines to check for that I don't get the warning anymore (admin/backup.php). Adding a line on line 1084 and the end for it about 35 lines further:
if (sizeof($contents) > 0) { // if no backups are there yet
// you would get an error Warning: array_multisort() [function.array-multisort]: Argument #1 is expected to be an array or a sort flag in backup.php on line 1112
	foreach ($contents as $key => $filedata) {
	  $contents_size[$key] = $filedata['size'];
	  $contents_file[$key] = $filedata['file'];
	  $contents_date[$key] = $filedata['date'];
	}
	
// sort files as requested, by name, file name, date ascending or descending
	switch ($listing) {
	  case "name-asc":
		array_multisort($contents_file, SORT_ASC, $contents);
	  break;
	  case "name-desc":
		array_multisort($contents_file, SORT_DESC, $contents);
	  break;
	  case "date-asc":
		array_multisort($contents_date, SORT_ASC, $contents);
	  break;
	  case "date-desc":
		array_multisort($contents_date, SORT_DESC, $contents);
	  break;
	  case "size-asc":
		array_multisort($contents_size, SORT_ASC, $contents);
	  break;
	  case "size-desc":
		array_multisort($contents_size, SORT_DESC, $contents);
	  break;
	  default:
// the old behaviour: new files last
		array_multisort($contents_date, SORT_ASC, $contents);
	  break;
	}
 } // end if (sizeof($contents) > 0)

Edited by Jan Zonjee, 09 March 2008 - 08:58.


#9   xavkick

xavkick
  • Members
  • 66 posts
  • Real Name:xavier

Posted 09 March 2008 - 10:21

Sorry guys, didn't notice your posts until today. Regarding your problem, this contribution was supposed to avoid that. Perhaps safe mode is off but still you cannot increase the time a script can run with tep_set_time_limit. Try "forcing" the script to avoid running any longer than 30 seconds please by commenting line 24 and uncommenting line 25 in admin/backup.php:

// $safe_mode_setting = (@ini_get('safe_mode') == 'On' || (@ini_get('safe_mode') === 1)) ? true : false;
$safe_mode_setting = true; // for testing
After 25 seconds or a little over it you should be presented with the intermediate screen like shown in screenshot_reload_backup.gif.



THANK YOU for your help, but i have still the same problem; i have switch the line 24 an 25 but but still have a blanck screen after 20 second.
But when i go back i have few tables but not all.
What should i do ?

#10   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 09 March 2008 - 11:44

THANK YOU for your help, but i have still the same problem; i have switch the line 24 an 25 but but still have a blanck screen after 20 second.
But when i go back i have few tables but not all.
What should i do ?

A blank screen?
You should be getting the HML that starts around line 291:
if ($resume_dump == true && $safe_mode_setting == true) { // in safe mode: no set_time_limit available
			fclose($fp);
// now show HTML that will resume backup
?>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html <?php echo HTML_PARAMS; ?>>
<head> (etcetera, etcetera to around line 416)
Nothing there if you do a view source of that page?
It looks like there is an error of some kind (is there an error log?) but for the life of me I wouldn't know what that could be. I tested this a thousand times (OK, I exagurate a bit but still a lot of times) without problem.

#11   Freeman

Freeman
  • Members
  • 82 posts
  • Real Name:fab
  • Gender:Male
  • Location:Italy

Posted 09 March 2008 - 11:54

Hi Jan,
i tried to backup the dband it went fine, but when trying to restore it this is the error msg i had :
" Error: File pointer for getting file offset cannot be read."
And it will stop /sad.gif' class='bbc_emoticon' alt=':(' />
Any idea? I see that it refers to the db_backup function but a bit lost with the code /tongue.gif' class='bbc_emoticon' alt=':P' />
Advice on forum are Free, Email or Pm to fix your site is work...which I charge for :)
-------------------

#12   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 09 March 2008 - 12:34

i tried to backup the db and it went fine, but when trying to restore it this is the error msg i had :
" Error: File pointer for getting file offset cannot be read."
And it will stop :(
Any idea? I see that it refers to the db_backup function but a bit lost with the code :P

To be honest I have no idea. From looking back at that function and where that error is generated the restore has already been underway for some time. At the time I even thought about throwing that out because the file pointer should be set and the checking for it superfluous (obviously not...):
// Get the current file position

  if ($error == false) {
	  $file_offset = ftell($fd);
	if (!$file_offset) {
	  return array('status' => 'error', 'description' => ERROR_CANNOT_READ_FILE_POINTER, 'file_offset' => $file_offset);
	  $error = true;
When does it occur? On the first "run" of the restore, does it come up immediately or after about 25 seconds (when the end of the "run" should be near)?

#13   Freeman

Freeman
  • Members
  • 82 posts
  • Real Name:fab
  • Gender:Male
  • Location:Italy

Posted 09 March 2008 - 14:32

at start...no gap
if you are online i'm quite sure we can troubleshoot it really fast since i'm a developer myself (not experienced like you probably)
read pvt msg /smile.gif' class='bbc_emoticon' alt=':)' />
Advice on forum are Free, Email or Pm to fix your site is work...which I charge for :)
-------------------

#14   shpy

shpy
  • Members
  • 86 posts
  • Real Name:Le Quoc Hung

Posted 12 March 2008 - 12:07

Hi, what is difference of original backup and this ?! I mean, the size of the backup files. If use the original, i have a 3MB file. After install, the size is ~2.2MB ! All are in SQL only (not compresed).

With the giude of this contribution, i change the Gzip path in admin/includes/applicationtop.php and Backup manager found GZipp function. But it cannt use it. When backup not download and use Gzip, it did not create any file. If Gzip and just download, it load a 611 byte *.gz file. If i don't change Gzipp path, the Gzip function disappeare. How's about that?
Paint for VietNamese :

#15   gerdus

gerdus
  • Members
  • 1 posts
  • Real Name:Gerdus Bronn

Posted 03 April 2008 - 09:58

Hi

I downloaded your contribution and made a back up. But now I am concerned because the backup is only 2.8 mb against the 7 mb of the previous backup. Why did this happen?

I am scared to restore the back up because it might not restore everything.

#16   shpy

shpy
  • Members
  • 86 posts
  • Real Name:Le Quoc Hung

Posted 06 April 2008 - 08:26

i found that this backup manager will not backup all tables and all content of these tables ! So, i must found the way to comeback the original backup manager !
Paint for VietNamese :

#17   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 06 April 2008 - 08:51

i found that this backup manager will not backup all tables and all content of these tables ! So, i must found the way to comeback the original backup manager !

Getting back the original backup manager is easy enough. Just replace the admin/backup.php with the original one (or download osC 2.2 RC2a and use the one from that). Only one original define is commented out in the language file for the contribution, probably you will never need that one.

Is there a pattern about certain tables/contents not being backed up? Some tables (sessions and whosonline) are not supposed to be backed up. They are also not backed up by more recent versions of backup.php.

The difference in size in backup (2.2 instead of 3) as mentioned in your earlier post (missed that, sorry) is due to more efficient storing of the values in the tables as explained in the package itself.

#18   shpy

shpy
  • Members
  • 86 posts
  • Real Name:Le Quoc Hung

Posted 06 April 2008 - 16:45

Thank you for reply. That's my fault ! I'm sorry, I did not read carefuly the install.txt .
Yes, the whos_online table was not backed up. My site have the html_cace contribution (which add some colums to this table), when restore the database to the new site, the struture of database miss the whos_online table's.

And the Admin Acess contribute have that issue too. There is any way for me backup all tables in my database ?

(Thanks again for explain why new backup file smaler than the older. Now i understand it)
Paint for VietNamese :

#19   Jan Zonjee

Jan Zonjee

    Governor of Peace

  • Core Team
  • 7,094 posts
  • Real Name:Jan Zonjee
  • Gender:Male
  • Location:the Netherlands

Posted 06 April 2008 - 20:50

Yes, the whos_online table was not backed up. My site have the html_cace contribution (which add some colums to this table), when restore the database to the new site, the struture of database miss the whos_online table's.

And the Admin Acess contribute have that issue too. There is any way for me backup all tables in my database ?

Thanks for your reply and feedback. I now realize I was not completely right about the original backup.php not backing up sessions and whos_online because it does backup the structure of the table, only not the contents.

With this contribution also the structure is not added (because of the way the code was put together with more flexibility of which tables you want to backup). I think I should go back and see if that can be added in again. It is not really a problem if you restore a backup over your present backup because then it will first get the structure of the table sessions and whos_online with a certain command, drop the table and then restore the backup without contents but with the changes you have made to the table.
However, this doesn't work if you take the backup and use it on an empty database. My fault, haven't thought about that.

For restoring the original tables sessions and whos_online you can use the queries from oscommerce.sql (the install sql file):
DROP TABLE IF EXISTS sessions;
CREATE TABLE sessions (
  sesskey varchar(32) NOT NULL,
  expiry int(11) unsigned NOT NULL,
  value text NOT NULL,
  PRIMARY KEY (sesskey)
);

DROP TABLE IF EXISTS whos_online;
CREATE TABLE whos_online (
  customer_id int,
  full_name varchar(64) NOT NULL,
  session_id varchar(128) NOT NULL,
  ip_address varchar(15) NOT NULL,
  time_entry varchar(14) NOT NULL,
  time_last_click varchar(14) NOT NULL,
  last_page_url text NOT NULL
);
If you then run the portion of the sql of the html_cache contribution that changes the table whos_online (or add it to the code above) you should be fine.

I'm not familiar with the Admin Access contribution but this database manager will backup all tables from the database, whether they are originally in osCommerce or not. Only not sessions and whos_online.

#20   invasi0n

invasi0n
  • Members
  • 158 posts
  • Real Name:Andrew

Posted 16 April 2008 - 13:39

Hello,

I just tried this contribution with safe_mode ON and I get this error:

Fatal error: Maximum execution time of 60 seconds exceeded in F:\xampp\htdocs\...

Database size is 1.8 MB.

I use firefox and JavaScript is enabled.

If I turn safe_mode Off, it works without problems.

Edited by invasi0n, 16 April 2008 - 13:42.