502 replies to this topic

[popsel]

Hi!

I just installed OSC_SEC 2.5r2 on my nearly ready German shop.  

May you please change the  $GETcleanup filter, because it filters German umlauts, like ö Ö ä Ä Ü ü if set to $GETcleanup = 1

This is the same if the umlauts are html coded, eg. ü = ü  

Thank you in advance.


Regards

[Taipo]

Are the umlauts just on the vowels or are there more. What is the full set?

[popsel]

Taipo, on 20 May 2011 - 11:28 AM, said:

Are the umlauts just on the vowels or are there more. What is the full set?

Hi!

In Germany we have a ß character, too.
So all the special charcters we need here are:

ö Ö ä Ä ü Ü ß and the € = € sign.


It would be a good idea to add a way to add more special characters (some table or similar).
Other countries will need other special characters, too.

Regards

popsel

Edited by popsel, 20 May 2011 - 11:52 AM.

[Taipo]

Try this out for now.

Scroll to about line 720 in osc_sec.php and find the function below:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					  $nodes[ $key ] = preg_replace("/[^ a-zA-Z0-9@%:{}\/_.-]/i", "", urldecode( $value ) );
				}
		  }
	 } else {
		  $nodes = preg_replace("/[^ a-zA-Z0-9?=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
	}
	return $nodes;
  }

Replace with this:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					  $nodes = preg_replace("/[^ a-zA-Z0-9?äöüÄÖÜ€ß=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
				}
		  }
	 } else {
		  $nodes = preg_replace("/[^ a-zA-Z0-9?äöüÄÖÜ€ß=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
	}
	return $nodes;
  }

See if that works for you.

[popsel]

Taipo, on 20 May 2011 - 11:58 AM, said:

Try this out for now.

Scroll to about line 720 in osc_sec.php and find the function below:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					  $nodes[ $key ] = preg_replace("/[^ a-zA-Z0-9@%:{}\/_.-]/i", "", urldecode( $value ) );
				}
		  }
	 } else {
		  $nodes = preg_replace("/[^ a-zA-Z0-9?=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
	}
	return $nodes;
  }

Replace with this:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					  $nodes = preg_replace("/[^ a-zA-Z0-9?äöüÄÖÜ€ß=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
				}
		  }
	 } else {
		  $nodes = preg_replace("/[^ a-zA-Z0-9?äöüÄÖÜ€ß=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
	}
	return $nodes;
  }

See if that works for you.

Hi!

This works for input to forms like Order Editor V5.09, which is already a big success.
It is not working if the special chars are HTML coded. eg. ü is still filtered.

Thank you for very fast reply.

Regards

Popsel

[hostricity]

I've installed OSC SEC on a customer site, but I cannot figure out how to test it.

I'm using OSC 2.2 RC2.

Could someone suggest some url strings that would get me banned?

Note: I have the admin directory Apache password protected, so I'm talking about testing the catalog, not the admin.

Thanks,

Geoff

[hostricity]

hostricity, on 20 May 2011 - 07:12 PM, said:

I've installed OSC SEC on a customer site, but I cannot figure out how to test it.

...

Thanks,

Geoff

I did discover that OSC SEC gave an error message in Admin that it couldn't write to the htaccess file. I changed the permissions and the error message went away, but nothing was actually written to the .htaccess.

Is it correct that OSC SEC won't write anything to the .htaccess until it actually has an ip to ban?

Anyway, I haven't as yet figured out what url string I could use to cause my ip to be banned or to generate an email.

[Taipo]

Try something like:

http://www.yourdomain.com/catalog/admin/categories.php/login.php?cookies=1

[Taipo]

popsel, on 20 May 2011 - 12:30 PM, said:

Hi!

This works for input to forms like Order Editor V5.09, which is already a big success.
It is not working if the special chars are HTML coded. eg. ü is still filtered.

Thank you for very fast reply.

Regards

Popsel

I guess what I will need to test this is a scenario or addon that tries to put ü into a GET request. Keep in mind that ü and other html coded umlauts are not filtered in form variables (POST requests). At the moment I am hesitant to allow the '&' and ';' as whitelisted characters by themselves because they, well at least the ';' is a regularly used character in attempts to inject into the database.

If you can give me a scenario that I can test and simulate here I might be able to come up with a better method of allowing the German characters through the GET whitelist.

[popsel]

Taipo, on 20 May 2011 - 09:48 PM, said:

I guess what I will need to test this is a scenario or addon that tries to put ü into a GET request. Keep in mind that ü and other html coded umlauts are not filtered in form variables (POST requests). At the moment I am hesitant to allow the '&' and ';' as whitelisted characters by themselves because they, well at least the ';' is a regularly used character in attempts to inject into the database.

If you can give me a scenario that I can test and simulate here I might be able to come up with a better method of allowing the German characters through the GET whitelist.

Hi!

I noticed the missing umlaut characters when the Discount Coupon Codes V3.34 addon shows error messages.
After translating the text constants to German the html coded umlauts are truncated,
eg. input string is: ü
this is cut down to: uuml  (missing the first and last character).
The & and ; characters are filtered, so the browser shows uuml instead of ü

The message string is provided this way:

These are the German translations in catalog\includes\languages\german.php:

//kgt - discount coupons
define('ENTRY_DISCOUNT_COUPON_ERROR', 'Der eingegebene Gutscheincode ist ungültig.');
define('ENTRY_DISCOUNT_COUPON_AVAILABLE_ERROR', 'Der eingegebene Gutscheincode ist nicht mehr gültig.');
define('ENTRY_DISCOUNT_COUPON_USE_ERROR', 'Unsere Aufzeichnungen zeigen, dass Sie diesen Gutschein-Code bereits %s Mal benutzt haben.  Sie dürfen diesen Code nicht mehr als %s Mal verwenden.');
define('ENTRY_DISCOUNT_COUPON_MIN_PRICE_ERROR', 'Der Mindestbestellwert für diesen Gutschein beträgt %s und wurde noch nicht erreicht');
define('ENTRY_DISCOUNT_COUPON_MIN_QUANTITY_ERROR', 'Die minimale Anzahl der Produkte zur Benutzung dieses Gutscheines ist %s');
define('ENTRY_DISCOUNT_COUPON_EXCLUSION_ERROR', 'Einige oder alle der Produkte im Warenkorb sind von der Verrechnung mit diesem Gutschein ausgeschlossen.' );
define('ENTRY_DISCOUNT_COUPON', 'Gutschein-Code:');
define('ENTRY_DISCOUNT_COUPON_SHIPPING_CALC_ERROR', 'Die Ihnen berechneten Versandkosten wurden geändert.');
//end kgt - discount coupons
?>

If conditions are met eg. entering a wrong coupon code in frontend (checkout_payment.php) text constants are send from
catalog\includes\classes\discount_coupon.php in the function get_coupon()
example: look how the constant ENTRY_DISCOUNT_COUPON_ERROR is send here:

	function get_coupon( $code, $delivery ) {
	  global $customer_id; //needed for customer_exclusions
	  $check_code_query = tep_db_query( $sql = "SELECT dc.*
												FROM " . TABLE_DISCOUNT_COUPONS . " dc
												WHERE coupons_id = '".tep_db_input( $code )."'
												  AND ( coupons_date_start <= CURDATE() OR coupons_date_start IS NULL )
												  AND ( coupons_date_end >= CURDATE() OR coupons_date_end IS NULL )" );
	  if( tep_db_num_rows( $check_code_query ) != 1 ) { //if no rows are returned, then they haven't entered a valid code
		$this->message( ENTRY_DISCOUNT_COUPON_ERROR ); //display the error message
	  } else {
		//customer_exclusions
		$check_user_query = tep_db_query( $sql = 'SELECT dc2u.customers_id

So the output should be
Der eingegebene Gutscheincode ist ungültig
but it is:
Der eingegebene Gutscheincode ist unguumlltig.

So IMHO the fiter should block the single problematic chars, like & ; but bypass umlaut strings like ü if they are exactly written as a refrence pattern.


Regards

Popsel

[Taipo]

Ok thanks for that. That gives me something to work with.

[popsel]

Taipo, on 21 May 2011 - 09:33 AM, said:

Ok thanks for that. That gives me something to work with.

Hi!

I just noticed another problem with filtering:
In Germany we use comma as separator between Euro and cents. So the system is set according to it.
I set up a discount coupon to be used with a mimimal order value of 10 EURO.
Now, with $GETcleanup = 0, after adding an amount of 9 EURO to the cart and entering the coupon code I get an
error message
(translated to English) ... minimal order is 10,00 € ...
This is like it should be.
After setting $GETcleanup = 1; there is a funny message with the same conditions I get:
(translated to English) ... minimal order is 1000 ...  (the comma and the € sign is missing)
This can be really confusing to a customer, isn`t it ?

Regards

Popsel

Edited by popsel, 21 May 2011 - 12:39 PM.

[Taipo]

Try this out and let me know if it works for you.

Replace the code I gave you above with this:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					 $nodes = getCleaner( $nodes );
				}
		  }
	 } else {
		  $nodes = getCleaner( $nodes, TRUE );
		  $nodes = preg_replace("/[^ a-zA-Z0-9?,ßäöüÄÖÜ€=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
		  $nodes = getCleaner( $nodes, FALSE );
	}
	return $nodes;
  }

  /**
  * Called above, this will clean up
  * values but not interfere with umlauts
  */
  function getCleaner($string, $conv=1) {
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] . 
	 $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] . 
	 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] );
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€");
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,szlig,euro");
	for($wr=0;$wr<=count($tolist);$wr++) {
	   if ( $conv > 0 ) {
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string);
	   } else {
	   $string = str_replace($x.$fromlist[$wr], $tolist[$wr], $string);
	   }
	}
	return $string;
}

Edited by Taipo, 21 May 2011 - 02:20 PM.

[popsel]

Taipo, on 21 May 2011 - 02:16 PM, said:

Try this out and let me know if it works for you.

Replace the code I gave you above with this:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					 $nodes = getCleaner( $nodes );
				}
		  }
	 } else {
		  $nodes = getCleaner( $nodes, TRUE );
		  $nodes = preg_replace("/[^ a-zA-Z0-9?,ßäöüÄÖÜ€=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
		  $nodes = getCleaner( $nodes, FALSE );
	}
	return $nodes;
  }

  /**
  * Called above, this will clean up
  * values but not interfere with umlauts
  */
  function getCleaner($string, $conv=1) {
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] . 
	 $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] . 
	 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] );
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€");
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,szlig,euro");
	for($wr=0;$wr<=count($tolist);$wr++) {
	   if ( $conv > 0 ) {
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string);
	   } else {
	   $string = str_replace($x.$fromlist[$wr], $tolist[$wr], $string);
	   }
	}
	return $string;
}

Hi!

Thanks for your engagement.

After a quick check I get this display in the frontend with Firefox:
Der Mindestbestellwert für diesen Gutschein beträgt 10,00 und wurde noch nicht erreicht.

checking the generated html code with Firebug shows amp inside the umlaut strings ?!
<td class="headerError">Der Mindestbestellwert f&amp;uuml;r diesen Gutschein betr&amp;auml;gt 10,00 und wurde noch nicht erreicht.</td>

hmm.. but the comma inside the 10,00 is now visible  

Here is a realtime html editor, suitable to show how the strings are displayed from browser:
http://htmledit.squarefree.com/
To simulate it you may copy this original output string into the window to see the result:

<td class="headerError">Der Mindestbestellwert f&uuml;r diesen Gutschein betr&auml;gt  10,00  und wurde noch nicht erreicht.</td>

Edited by popsel, 21 May 2011 - 03:24 PM.

[popsel]

popsel, on 21 May 2011 - 03:10 PM, said:

To simulate it you may copy this original output string into the window to see the result:

<td class="headerError">Der Mindestbestellwert f&uuml;r diesen Gutschein betr&auml;gt  10,00  und wurde noch nicht erreicht.</td>

Sorry, the way I did copy was wrong. It should be:

Der Mindestbestellwert f&amp;uuml;r diesen Gutschein betr&amp;auml;gt  10,00  und wurde noch nicht erreicht.

[Taipo]

Try this. Replace this part of the code above:

  /** 
  * Called above, this will clean up 
  * values but not interfere with umlauts 
  */ 
  function getCleaner($string, $conv=1) { 
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] .  
		 $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] .  
		 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] ); 
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€"); 
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,szlig,euro"); 
	for($wr=0;$wr<=count($tolist);$wr++) { 
	   if ( $conv > 0 ) { 
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string); 
	   } else { 
		   $string = str_replace($x.$fromlist[$wr], $tolist[$wr], $string); 
	   } 
	} 
	return $string; 
}

With the following:

  /**
  * Called above, this will clean up
  * values but not interfere with umlauts
  */
  function getCleaner($string, $conv=1) {
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] . 
			  $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] . 
				 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] );
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€,ä,ä,ü,Ä,Ö,Ü,€,&szlig");
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,szlig,euro,ae,oe,ue,Ae,Oe,Ue,euro,szlig");
	$finlist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€,ä,ö,ü,Ä,Ö,Ü,€,ß");
	for($wr=0;$wr<=count($tolist);$wr++) {
	   if ( $conv > 0 ) {
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string);
	   } else {
	   $string = str_replace($x.$fromlist[$wr], $finlist[$wr], $string);
	   }
	}
	return $string;
}

This probably could be written in simpler terms, but in the end, its the result that matters.

[popsel]

Taipo, on 21 May 2011 - 09:18 PM, said:

Try this. Replace this part of the code above:

  /** 
  * Called above, this will clean up 
  * values but not interfere with umlauts 
  */ 
  function getCleaner($string, $conv=1) { 
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] .  
		 $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] .  
		 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] ); 
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€"); 
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,szlig,euro"); 
	for($wr=0;$wr<=count($tolist);$wr++) { 
	   if ( $conv > 0 ) { 
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string); 
	   } else { 
		   $string = str_replace($x.$fromlist[$wr], $tolist[$wr], $string); 
	   } 
	} 
	return $string; 
}

With the following:

  /**
  * Called above, this will clean up
  * values but not interfere with umlauts
  */
  function getCleaner($string, $conv=1) {
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] . 
			  $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] . 
				 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] );
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€,ä,ä,ü,Ä,Ö,Ü,€,&szlig");
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,szlig,euro,ae,oe,ue,Ae,Oe,Ue,euro,szlig");
	$finlist = explode(",", "ä,ö,ü,Ä,Ö,Ü,ß,€,ä,ö,ü,Ä,Ö,Ü,€,ß");
	for($wr=0;$wr<=count($tolist);$wr++) {
	   if ( $conv > 0 ) {
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string);
	   } else {
	   $string = str_replace($x.$fromlist[$wr], $finlist[$wr], $string);
	   }
	}
	return $string;
}

This probably could be written in simpler terms, but in the end, its the result that matters.

Hi!

I checked with the same conditions, but the result did not change  

Frontend output is this:
Der Mindestbestellwert f&uuml;r diesen Gutschein betr&auml;gt 10,00 und wurde noch nicht erreicht.

Firebug analysis is showing that Firefox is getting this string:
<td class="headerError">Der Mindestbestellwert f&amp;uuml;r diesen Gutschein betr&amp;auml;gt 10,00 und wurde noch nicht erreicht.</td>

That`s why Firefox does not display umlauts. The generated code f&amp;uuml;r provides a display of f&uuml;r instead of the desired output: für

So I was trying to find out the conditions to get the desired output.

If $nonGETPOSTReqs = 1 and umlauts are not coded like &uuml; but normal written as ü

define('ENTRY_DISCOUNT_COUPON_MIN_PRICE_ERROR', 'Der Mindestbestellwert für diesen Gutschein beträgt %s und wurde noch nicht erreicht');

and this line is disabled for testing

 /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
		  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
					  scrubster( $value );
				} else {
					 $nodes = getCleaner( $nodes );
				}
		  }
	 } else {
		  $nodes = getCleaner( $nodes, TRUE );   
		  // Next line disabled for testing:
		 // $nodes = preg_replace("/[^ a-zA-Z0-9?,ßäöüÄÖÜ€=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
		  $nodes = getCleaner( $nodes, FALSE );
	}
	return $nodes;
  }

then I receive this (perfect) frontend output:
Der Mindestbestellwert für diesen Gutschein beträgt 10,00 € und wurde noch nicht erreicht.

Looking into the code using Firebug shows this:
<td class="headerError">Der Mindestbestellwert für diesen Gutschein beträgt 10,00 € und wurde noch nicht erreicht.</td>

From this view I think there are 2 problems:
1. The filter function (scrubster) sometimes still filters standard written umlauts (ö ä ü Ö Ä Ü €)
   To show that, enabling the above disabled line, in function scrubster shows this result:
   Frontend ouput:
     Der Mindestbestellwert fr diesen Gutschein betrgt 10,00 und wurde noch nicht erreicht.
   Firebug analysis:
     <td class="headerError">Der Mindestbestellwert fr diesen Gutschein betrgt 10,00 und wurde noch nicht erreicht.</td>

2. If umlauts are html coded in the message string, they are not bypassed correctly. So the browser shows the code for umlauts but not the umlauts special character itself.

Hopefully this research can help you to fix this problem.

Thanks again for your patience.

[Taipo]

Ok that does help a bit.

Try this for size:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					  $nodes = getCleaner( $nodes, TRUE );
					  $nodes = preg_replace("/[^ a-zA-Z0-9?,=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
					  $nodes = getCleaner( $nodes, FALSE );
				}
		  }
	 } else {
		  $nodes = getCleaner( $nodes, TRUE );
		  $nodes = preg_replace("/[^ a-zA-Z0-9?,=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
		  $nodes = getCleaner( $nodes, FALSE );
	}
	return $nodes;
  }

  /**
  * Called above, this will clean up
  * values but not interfere with umlauts
  */
  function getCleaner($string, $conv=1) {
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] . 
			  $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] . 
				 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] );
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,€,ß,ä,ä,ü,Ä,Ö,Ü,€,&szlig,ä,ö,ü,Ä,Ö,Ü,€,ß");
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,euro,szlig,ae,oe,ue,Ae,Oe,Ue,euro,szlig,axe,oxe,uxe,Axe,Oxe,Uxe,euxro,szlxig");
	$finlist = explode(",", "ä,ö,ü,Ä,Ö,Ü,€,ß,ä,ö,ü,Ä,Ö,Ü,€,ß,ä,ö,ü,Ä,Ö,Ü,€,ß");
	for($wr=0;$wr<=count($tolist);$wr++) {
	   if ( $conv > 0 ) {
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string);
	   } else {
	   $string = str_replace($x.$fromlist[$wr], $finlist[$wr], $string);
	   }
	}
	return $string;
}

Hopefully this might work. It should work where umlauts are html coded and are in actual form. Where it probably will not work is where you want to display the html encoded umlaut. But that would be rare in osCommerce one would think.

Edited by Taipo, 22 May 2011 - 08:14 AM.

[popsel]

Taipo, on 22 May 2011 - 08:13 AM, said:

Ok that does help a bit.

Try this for size:

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	 if ( is_array( $nodes ) ) {
	  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
  			  scrubster( $value );
				} else {
					  $nodes = getCleaner( $nodes, TRUE );
					  $nodes = preg_replace("/[^ a-zA-Z0-9?,=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
					  $nodes = getCleaner( $nodes, FALSE );
				}
		  }
	 } else {
		  $nodes = getCleaner( $nodes, TRUE );
		  $nodes = preg_replace("/[^ a-zA-Z0-9?,=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
		  $nodes = getCleaner( $nodes, FALSE );
	}
	return $nodes;
  }

  /**
  * Called above, this will clean up
  * values but not interfere with umlauts
  */
  function getCleaner($string, $conv=1) {
	$x = md5( $_SERVER["REMOTE_ADDR"] . $_SERVER["HTTP_USER_AGENT"] . 
			  $_SERVER["HTTP_HOST"] . $_SERVER["DOCUMENT_ROOT"] . 
				 $_SERVER["SERVER_SOFTWARE"] . $_SERVER["PATH"] );
	$tolist = explode(",", "ä,ö,ü,Ä,Ö,Ü,€,ß,ä,ä,ü,Ä,Ö,Ü,€,&szlig,ä,ö,ü,Ä,Ö,Ü,€,ß");
	$fromlist = explode(",", "ae,oe,ue,Ae,Oe,Ue,euro,szlig,ae,oe,ue,Ae,Oe,Ue,euro,szlig,axe,oxe,uxe,Axe,Oxe,Uxe,euxro,szlxig");
	$finlist = explode(",", "ä,ö,ü,Ä,Ö,Ü,€,ß,ä,ö,ü,Ä,Ö,Ü,€,ß,ä,ö,ü,Ä,Ö,Ü,€,ß");
	for($wr=0;$wr<=count($tolist);$wr++) {
	   if ( $conv > 0 ) {
		   $string = str_replace($tolist[$wr], $x.$fromlist[$wr], $string);
	   } else {
	   $string = str_replace($x.$fromlist[$wr], $finlist[$wr], $string);
	   }
	}
	return $string;
}

Hopefully this might work. It should work where umlauts are html coded and are in actual form. Where it probably will not work is where you want to display the html encoded umlaut. But that would be rare in osCommerce one would think.

Ok, we have to see where the problem occurs. It still doesn't provide the right output.
With actual version I have:

Frontend showing:
Der Mindestbestellwert f&uuml;r diesen Gutschein betr&auml;gt 10,00 und wurde noch nicht erreicht.

Firebug showing:
<td class="headerError">Der Mindestbestellwert f&amp;uuml;r diesen Gutschein betr&amp;auml;gt 10,00 und wurde noch nicht erreicht.</td>

I added a logfile to see whats going in and out of function scrubster

  /**
  * Clean up GET request vars
  * as well as multidimensional arrays
  */
  function scrubster( $nodes ) {
	unlink("scrubster.txt");				  // Erase old logfile first
	$file = fopen("scrubster.txt", "a+");	 // Open debug file
	fwrite($file, 'in ='.$nodes.chr(10));	 // Write input to file

	 if ( is_array( $nodes ) ) {
		  foreach( $nodes as $key=>$value ) {
				if ( is_array( $value ) ) {
					  scrubster( $value );
				} else {
					  $nodes = getCleaner( $nodes, TRUE );
					  $nodes = preg_replace("/[^ a-zA-Z0-9?,=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
					  $nodes = getCleaner( $nodes, FALSE );
				}
		  }
	 } else {
		  $nodes = getCleaner( $nodes, TRUE );
		  $nodes = preg_replace("/[^ a-zA-Z0-9?,=@%:{}\/_.-]/i", "", urldecode( $nodes ) );
		  $nodes = getCleaner( $nodes, FALSE );
	}

	fwrite($file, 'out='.$nodes.chr(10).chr(13));			 // Write output to file and add new Line
	fclose($file);	 // Close file

	return $nodes;
  }

This is the result:

in =Der Mindestbestellwert für diesen Gutschein beträgt  10,00 € und wurde noch nicht erreicht.
out=Der Mindestbestellwert für diesen Gutschein beträgt  10,00  und wurde noch nicht erreicht.

So we see that nothing beside the € sign is changed. The changes must appear elsewhere.
(If I do the same test with normal umlauts they are filtered in the output.)

Is there any further prcosessing of these strings inside OSC_SEC beside the function scrubster ?

[Taipo]

Nothing else filters the GET requests. The two other actions that happen are 1/ back in the osc.php the setlocale( LC_TIME, "en_NZ" ); is where setlocale determines the time, so unless that has been changed to the wrong settings it should not interfere with the output, and the other place is where $_REQUEST is reset as a merge of $_GET and $_POST vars. see:  $_REQUEST = array_merge( $_GET, $_POST );

The difficulty I am having in trying to debug this is that it tests correct on my test system here. And by the looks of the output it is reporting correctly. The only other thing is the way the browser may be converting the output itself. What is the charset set to in the html head of your page, the DOCTYPE and what is the charset set to in your browser?

ps if you dont mind, can you PM me the url to your website so I can see it for myself?

Edited by Taipo, 22 May 2011 - 11:19 AM.