904 replies to this topic

[Jamsy]

Quote

Here is what my tep_href_link() looks like:

Where do i find this code?

Thanks

[RavenWulf]

Hey all,

In reading this topic I also saw someone refer to the config cache contribution.  I am trying to install this on my local testing machine (I have easyphp1-7 installed)

I think I have it all good, but the cache file is not being generated (I don't recieve any errors though)  The instructions say to put in the "absolute" path to where the cache file is located.  I think I may have this wrong and was wondering if any of you have an idea I how I can set it correctly.

The path to the folder is:

C:/EasyPHP1-7/www/config_cache

I have tried every combo of this I can think of and nothing works..does anyone have suggestions?

Sorry if this is misplaced, but I though to ask the gurus of optimization, and ya'll seem to be here

RW

[RavenWulf]

Oh one more request for the night  

Can someone post info in the Join command?  where to use it, what it does, etc..

THANKS!
RW

[Spaceboy]

Very nice work Bobby ! After having played around with the configuration settings, I also has a look at all those index.php queries that are HUGE! As you said, OSC is way to generic for all Live shops and a lot of stuff can be dropped if not needed (banners, taxes, etc).

I think that it's especial very useful in iterations where the same query is repeated over and over again (product listings, etc.)

My other optimisations done so far :

- gzip compression (level 7)
- configuration flat file (cache)
- tep_get_tax_rate() is exited at beginning if my variable USE_OF_TAX = 0
- drop of banner code
- merge of query for "specials" and products price query.

For me it's more as a "background" task when I'm tired after having added another nifty contribution (hehe).

I do not display my new products so I cannot help you with the extra JOIN.

Bye,

Didier.

[wizardsandwars]

Quote

Here is what my tep_href_link() looks like:
CODE

function tep_href_link($page = '', $parameters = '', $connection = 'NONSSL', $add_session_id = false, $search_engine_safe = true) {

Now, even if the session_id's are not appended to the URL they will still be created for guests uniquely. Once they log in they get another or resume a previous. Once they put something in their cart the session_id is recorded in session and that is good for the life of their visit. If they log in they will get a more permanent session_id.

This isn't really a valid solution. Essentially, all this is doing is setting the 'Force Cookie Use' to true. Customers that have browsers that do not allow cookies will not be able to purchase.

I think that it would be worth letting everyone know that the 'Page Cache' contribution doesn't work unless you force cookie use.

Edited by wizardsandwars, 02 November 2004 - 10:40 PM.

[wizardsandwars]

Or, to be more accurate, if you use page case without having 'force cookie usage' set to true, then you have a very serious security risk.

[gscreations]

Chemo great work, im starting to optimise my site, got this result on index.php it has a modified modules/new_products.php box on it which as you can see loops through to retreive data, any clues on where to start to reduce this down(other than a LIMIT:

Current Parse Time: 1.225 s with 73 queries
Array
(
    [QUERIES] => Array
        (
            [0] => select configuration_key as cfgKey, configuration_value as cfgValue from configuration
            [1] => select value from sessions where sesskey = '01a0e28161b759a9d2d59ebe7220f6ed' and expiry > '1099436796'
            [2] => select code, title, symbol_left, symbol_right, decimal_point, thousands_point, decimal_places, value from currencies
            [3] => delete from whos_online where time_last_click < '1099435896'
            [4] => select count(*) as count from whos_online where session_id = '01a0e28161b759a9d2d59ebe7220f6ed'
            [5] => update whos_online set customer_id = '0', full_name = 'Guest', ip_address = '62.254.0.34', time_last_click = '1099436796', last_page_url = '' where session_id = '01a0e28161b759a9d2d59ebe7220f6ed'
            [6] => select specials_id from specials where status = '1' and now() >= expires_date and expires_date > 0
            [7] => select count(p.products_id) as total  from products p left join specials s on p.products_id = s.products_id where products_status = '1'
            [8] => select p.products_id, p.products_image, p.products_tax_class_id, if(s.status, s.specials_new_products_price, p.products_price) as products_price from products p left join specials s on p.products_id = s.products_id where products_status = '1' order by p.products_date_added desc limit 0, 20
            [9] => select categories_id from products_to_categories where products_id = '72'
            [10] => select parent_id from categories where categories_id = '28'
            [11] => select products_name from products_description where products_id = '72'
            [12] => select categories_id from products_to_categories where products_id = '71'
            [13] => select parent_id from categories where categories_id = '28'
            [14] => select products_name from products_description where products_id = '71'
            [15] => select categories_id from products_to_categories where products_id = '70'
            [16] => select parent_id from categories where categories_id = '28'
            [17] => select products_name from products_description where products_id = '70'
            [18] => select categories_id from products_to_categories where products_id = '69'
            [19] => select parent_id from categories where categories_id = '28'
            [20] => select products_name from products_description where products_id = '69'
            [21] => select categories_id from products_to_categories where products_id = '68'
            [22] => select parent_id from categories where categories_id = '28'
            [23] => select products_name from products_description where products_id = '68'
            [24] => select categories_id from products_to_categories where products_id = '67'
            [25] => select parent_id from categories where categories_id = '28'
            [26] => select products_name from products_description where products_id = '67'
            [27] => select categories_id from products_to_categories where products_id = '66'
            [28] => select parent_id from categories where categories_id = '28'
            [29] => select products_name from products_description where products_id = '66'
            [30] => select categories_id from products_to_categories where products_id = '65'
            [31] => select parent_id from categories where categories_id = '28'
            [32] => select products_name from products_description where products_id = '65'
            [33] => select categories_id from products_to_categories where products_id = '64'
            [34] => select parent_id from categories where categories_id = '28'
            [35] => select products_name from products_description where products_id = '64'
            [36] => select categories_id from products_to_categories where products_id = '63'
            [37] => select parent_id from categories where categories_id = '28'
            [38] => select products_name from products_description where products_id = '63'
            [39] => select categories_id from products_to_categories where products_id = '59'
            [40] => select parent_id from categories where categories_id = '27'
            [41] => select products_name from products_description where products_id = '59'
            [42] => select categories_id from products_to_categories where products_id = '58'
            [43] => select parent_id from categories where categories_id = '27'
            [44] => select products_name from products_description where products_id = '58'
            [45] => select categories_id from products_to_categories where products_id = '57'
            [46] => select parent_id from categories where categories_id = '27'
            [47] => select products_name from products_description where products_id = '57'
            [48] => select categories_id from products_to_categories where products_id = '56'
            [49] => select parent_id from categories where categories_id = '27'
            [50] => select products_name from products_description where products_id = '56'
            [51] => select categories_id from products_to_categories where products_id = '55'
            [52] => select parent_id from categories where categories_id = '27'
            [53] => select products_name from products_description where products_id = '55'
            [54] => select categories_id from products_to_categories where products_id = '54'
            [55] => select parent_id from categories where categories_id = '27'
            [56] => select products_name from products_description where products_id = '54'
            [57] => select categories_id from products_to_categories where products_id = '53'
            [58] => select parent_id from categories where categories_id = '27'
            [59] => select products_name from products_description where products_id = '53'
            [60] => select categories_id from products_to_categories where products_id = '52'
            [61] => select parent_id from categories where categories_id = '30'
            [62] => select products_name from products_description where products_id = '52'
            [63] => select categories_id from products_to_categories where products_id = '51'
            [64] => select parent_id from categories where categories_id = '30'
            [65] => select products_name from products_description where products_id = '51'
            [66] => select categories_id from products_to_categories where products_id = '49'
            [67] => select parent_id from categories where categories_id = '27'
            [68] => select products_name from products_description where products_id = '49'
            [69] => select startdate, counter from counter
            [70] => update counter set counter = '558'
            [71] => select count(*) as total from sessions where sesskey = '01a0e28161b759a9d2d59ebe7220f6ed'
            [72] => update sessions set expiry = '1099438237', value = 'cart|O:12:\"shoppingcart\":5:{s:8:\"contents\";a:0:{}s:5:\"total\";i:0;s:6:\"weight\";i:0;s:6:\"cartID\";N;s:12:\"content_type\";b:0;}language|s:7:\"english\";languages_id|s:1:\"1\";currency|s:3:\"GBP\";navigation|O:17:\"navigationhistory\":2:{s:4:\"path\";a:2:{i:0;a:4:{s:4:\"page\";s:16:\"product_info.php\";s:4:\"mode\";s:6:\"NONSSL\";s:3:\"get\";a:2:{s:5:\"cPath\";s:2:\"22\";s:11:\"products_id\";s:2:\"29\";}s:4:\"post\";a:0:{}}i:1;a:4:{s:4:\"page\";s:9:\"index.php\";s:4:\"mode\";s:6:\"NONSSL\";s:3:\"get\";a:2:{s:6:\"output\";s:1:\"1\";s:6:\"osCsid\";s:32:\"01a0e28161b759a9d2d59ebe7220f6ed\";}s:4:\"post\";a:0:{}}}s:8:\"snapshot\";a:0:{}}' where sesskey = '01a0e28161b759a9d2d59ebe7220f6ed'
        )

)

[Konquest]

Chemo, first I would like to say thanks for all your help with the optimization process.  I was wondering if you tried the Page Cache + Config Cache mods with Simple Template System installed.  I managed to get the config cache working but when I turn on Page Cache I get this error:

Fatal error: Call to a member function on a non-object in /home/perfumeo/public_html/shop/includes/sts_display_output.php on line 200

[Chemo]

RavenWulf, on Nov 2 2004, 02:02 PM, said:

ok I feel really dumb, sorry...LOL I HAVE got to learn the difference between admin and catalog LOL!!!

THANKS!!!
RW

<{POST_SNAPBACK}>

No problem...

Jamsy, on Nov 2 2004, 03:03 PM, said:

Where do i find this code?

Thanks

<{POST_SNAPBACK}>

It is located in *catalog*/includes/functions/html_output.php

RavenWulf, on Nov 2 2004, 04:05 PM, said:

Oh one more request for the night 

Can someone post info in the Join command?  where to use it, what it does, etc..

THANKS!
RW

<{POST_SNAPBACK}>

The JOIN is different for each module so can't give specifics unless you give me a specific file (i.e. - new_products.php)

wizardsandwars, on Nov 2 2004, 05:46 PM, said:

Or, to be more accurate, if you use page case without having 'force cookie usage' set to true, then you have a very serious security risk.

<{POST_SNAPBACK}>

Let's do this...take a look at the code and see if there is a quick way to remove the session_id's from the URL's and cache file name.

In the case of my client who I developed it for he uses SEF URL's so this is to be expected on live shops installing (of course, I didn't see it or would have done something about it before releasing).

At any rate, I'll take a look and see what can be done.  It might be as simple as a str_replace() to get it out.

gscreations, on Nov 2 2004, 06:11 PM, said:

Chemo great work, im starting to optimise my site, got this result on index.php it has a modified modules/new_products.php box on it which as you can see loops through to retreive data, any clues on where to start to reduce this down(other than a LIMIT:

Current Parse Time: 1.225 s with 73 queries
< SNIP Lots of data >

<{POST_SNAPBACK}>

Send me your modified file and I'll send one back with a few JOINS to get those queries down.

Konquest, on Nov 2 2004, 07:04 PM, said:

Chemo, first I would like to say thanks for all your help with the optimization process.  I was wondering if you tried the Page Cache + Config Cache mods with Simple Template System installed.  I managed to get the config cache working but when I turn on Page Cache I get this error:

Fatal error: Call to a member function on a non-object in /home/perfumeo/public_html/shop/includes/sts_display_output.php on line 200

<{POST_SNAPBACK}>

I have not looked at the template systems available but this is the second report of non-compatibilitiy with page cache.  I need to put that disclaimer in there somewhere until we can get them working together!

[wizardsandwars]

It won't matter if the store owner has SEF enabled or not. If they do not have 'Force Cookie Usage' set to true, then the session will always be hardcoded into the cashe page, thereby allowing the session to be 'hi-jacked'.

The session might be able to be removed with a search and replace function, or we might be able to pass the section of the html_output file an indicator NOT to assign a SID, like we do for spiders. Abviously, we wouldn't be able to use the USER AGENT, but I bet we could come up wit hsome other sort of indicator that this is something that should not be assigned an SID.

[Terfexdotcom]

Spaceboy, on Nov 2 2004, 09:30 PM, said:

- configuration flat file (cache)
- tep_get_tax_rate() is exited at beginning if my variable USE_OF_TAX = 0

Didier.

Where did you get these two contribs from Didier?

Chemo, on Nov 3 2004, 04:48 PM, said:

The JOIN is different for each module so can't give specifics unless you give me a specific file (i.e. - new_products.php)

Send me your modified file and I'll send one back with a few JOINS to get those queries down.

Whats your email address, would love it if you could do this for me!

[Spaceboy]

Terfexdotcom, on Nov 3 2004, 09:40 PM, said:

Where did you get these two contribs from Didier?
Whats your email address, would love it if you could do this for me!

<{POST_SNAPBACK}>

1. configuration flat file Faster Page Loads, Less DB queries

2. it's my own development. Very simple : change general.php

////
// Returns the tax rate for a zone / class
// TABLES: tax_rates, zones_to_geo_zones
  function tep_get_tax_rate($class_id, $country_id = -1, $zone_id = -1) {
    global $customer_zone_id, $customer_country_id;

// DDB - 040804 - DO not use taxes if not told so
if (USE_OF_TAX == 0) return 0;
    if ( ($country_id == -1) && ($zone_id == -1) ) {

Just add a record in the configuration table for the parameter USE_OF_TAX (values 0-1).

Didier.

[wizardsandwars]

You know, for those of you following along, there is a seriously flawed problem with the 'page cash' contribution. I've talked this through with 3 or 4 of the best programmers I know, and if you do not have 'force cookie use' set to true, there really isn't a good solution to fix it.

However, after doing quite a bit of testing, we've determined that it's not really the reduced unumber of queries that helps the load time of the page. It's actually the little bit of code that takes out all of the white space from the HTML output that makes it load so fast.

Before installing the page cache contribution, my site did 29 queries, and loaded in about 39 seconds on a 56k.

After installing the contribution, it had 9 queries, and loaded in 19.02 seconds.

After finding the flaw in the page cash contribution, I disabled the write to the cash file, and deleted all of the previous cache files. It now has 27 queries, and loadin in 22.17 seconds.

So, although the page cache really can't be fixed if you want to allow customers that don't have cookies enabled to be able to purchase, it's still pretty easy to use the html compression part of this contribution to make your page load faster.

[Chemo]

wizardsandwars, on Nov 3 2004, 09:37 PM, said:

You know, for those of you following along, there is a seriously flawed problem with the 'page cash' contribution. I've talked this through with 3 or 4 of the best programmers I know, and if you do not have 'force cookie use' set to true, there really isn't a good solution to fix it.

However, after doing quite a bit of testing, we've determined that it's not really the reduced unumber of queries that helps the load time of the page. It's actually the little bit of code that takes out all of the white space from the HTML output that makes it load so fast.

Before installing the page cache contribution, my site did 29 queries, and loaded in about 39 seconds on a 56k.

After installing the contribution, it had 9 queries, and loaded in 19.02 seconds.

After finding the flaw in the page cash contribution, I disabled the write to the cash file, and deleted all of the previous cache files. It now has 27 queries, and loadin in 22.17 seconds.

So, although the page cache really can't be fixed if you want to allow customers that don't have cookies enabled to be able to purchase, it's still pretty easy to use the html compression part of this contribution to make your page load faster.

<{POST_SNAPBACK}>

I added the compression (as you noticed it is actually just removing the whitespace) and haven't even mentioned it.  It's nice to see someone that still looks at the code to see what's under the hood

I think there may be a simple solution to the session id problem.  It is the same thing I faced when trying to create a cached page everyone can use yet still keep a portion dynamic (the shopping cart).  The session id is set in application_top.php so why not do the same thing as with the shopping cart info?

On writing the cache file remove the session id and insert placeholder.  On cache page retrieval str_replace() the placeholder with the actual session id.  I'll get on it and see what I can come up with.

[Chemo]

I just uploaded v1.2 of the page cache contribution.  The session_id support is in there now

Here's how I did it:

Class constructor

global $SID
$this->customer_sid = $SID;  

cache_this_page() method

...
echo str_replace(array("<%CART_CACHE%>", "<osCsid>"), array($this->customer_cart, $this->customer_sid), file_get_contents($this->cache_file) );    
...

write_file() method

$buffer = preg_replace("/osCsid=.{32}/", "<osCsid>", $buffer);

So, on new load it outputs the osCsid like normal.  On cache file write it removes the osCid from all the URL's and replaces it with a placeholder.  On subsequent calls to the cache page it inserts the $SID in place.  

Now, how about having some people test it?

[Jamsy]

Quote

Before installing the page cache contribution, my site did 29 queries, and loaded in about 39 seconds on a 56k.

After installing the contribution, it had 9 queries, and loaded in 19.02 seconds.

After finding the flaw in the page cash contribution, I disabled the write to the cash file, and deleted all of the previous cache files. It now has 27 queries, and loadin in 22.17 seconds

I hear what you are saying but don't think that your own load time tests on a 56k modem and on one solitary site can be a sound basis for testing.

With so many different isp's, backbones and servers coupled with so many different client side computers - surely the important thing for all of us is the Parse Time. With this we then KNOW how fast or slow OScommerce is, irrespective of different internet connections and your own image sizes etc etc.

[wizardsandwars]

Quote

So, on new load it outputs the osCsid like normal. On cache file write it removes the osCid from all the URL's and replaces it with a placeholder. On subsequent calls to the cache page it inserts the $SID in place.

Now, how about having some people test it? 

That sounds like a pretty good solution. I wish I had thoguht of it.

I'll give it a whirl.

Quote

With so many different isp's, backbones and servers coupled with so many different client side computers - surely the important thing for all of us is the Parse Time. With this we then KNOW how fast or slow OScommerce is, irrespective of different internet connections and your own image sizes etc etc.

I guess that's true, but I saw less that on tenth of a second difference from before the contrib to after it. I think that testing load time on your own sever, being consistant on the tool you use is a much better guage of the improvements each owner is making.

[MikeMike]

I just installed this contribution and got a couple of questions:

This might be a stupid question, but will this affect my statistic?
I am using advlogger and have the cgi script called from each page through a line like this: a.src="/advlogger/logger.cgi?ref="+escape(refer)+"&doc="+escape(doc)+"&res="+x+"x"+y;
Will it still call the script from a cached page?


What lifetime is best to use?
It's preset to 5 minutes, but why not 1 or 2 hours hours or maybe more?


Regards
Mike

[MikeMike]

One more thing....
The shopping cart in left column doesn't show up before the customer is logged in.
When the customer is not logged in and order a product with "buy now' he can't see if it is actually added to the cart, unless he actually go to the cart and check it out.
Any way around that? Maybe some kind of pop-up confirmation when a item is placed in the cart....

[chardae]

Chemo, on Nov 2 2004, 11:41 AM, said:

Here are the options:
1. Disable default appending session_id's to URL's setting in tep_href_link()
2. Create code to strip the session id's from the cache code

Here is what my tep_href_link() looks like:

  function tep_href_link($page = '', $parameters = '', $connection = 'NONSSL', $add_session_id = false, $search_engine_safe = true) {

Now, even if the session_id's are not appended to the URL they will still be created for guests uniquely.  Once they log in they get another or resume a previous.  Once they put something in their cart the session_id is recorded in session and that is good for the life of their visit.  If they log in they will get a more permanent session_id.

<{POST_SNAPBACK}>

For some Reason when I make this change...

function tep_href_link($page = '', $parameters = '', $connection = 'NONSSL', $add_session_id = false, $search_engine_safe = true) {

I can not log in anymore. What do you think could be causing this?