Jump to content



Issue Information

  • #000278

  • 0 - None Assigned

  • Duplicate

  • 3.0.1

  • -

Issue Confirmations

  • Yes (0)No (0)
Photo

Invalid UTF-8 sequence

Posted by Gergely on 18 April 2011 - 11:00

Hi Harald,

PHP Warning: json_encode(): Invalid UTF-8 sequence in argument in E:\xampp\htdocs\v3\osCommerce\OM\Core\Site\Admin\Application\ErrorLog\RPC\GetAll.php on line 32

XAMP fress install
Server Date:04/18/2011 12:58:15
Server OS:Windows NT 5.1
Server Host:
Server Up Time: HTTP
Server:Apache/2.2.17 (Win32) mod_ssl/2.2.17 OpenSSL/0.9.8o PHP/5.3.4 mod_perl/2.0.4 Perl/v5.10.1
PHP Version:PHP v5.3.5 / Zend v2.3.0
Database Host:localhost (127.0.0.1)
Database:MySQL v5.5.8
Database Date:04/18/2011 12:58:12
Database Up Time:0:17

regards,
Gergely

Hi Harald,

PHP Warning: json_encode(): Invalid UTF-8 sequence in argument in E:\xampp\htdocs\v3\osCommerce\OM\Core\Site\Admin\Application\ErrorLog\RPC\GetAll.php on line 32

XAMP fress install
Server Date:04/18/2011 12:58:15
Server OS:Windows NT 5.1
Server Host:
Server Up Time: HTTP
Server:Apache/2.2.17 (Win32) mod_ssl/2.2.17 OpenSSL/0.9.8o PHP/5.3.4 mod_perl/2.0.4 Perl/v5.10.1
PHP Version:PHP v5.3.5 / Zend v2.3.0
Database Host:localhost (127.0.0.1)
Database:MySQL v5.5.8
Database Date:04/18/2011 12:58:12
Database Up Time:0:17

regards,
Gergely


I have changed from latin_swedish to utf8_unicode_ci charset in database so it works correctly. /rolleyes.gif' class='bbc_emoticon' alt=':rolleyes:' />

I have changed from latin_swedish to utf8_unicode_ci charset in database so it works correctly. /rolleyes.gif' class='bbc_emoticon' alt=':rolleyes:' />


it would be checking before working in install sequences or take warning:

Warning: use utf8 database if you would like to install non ISO8859-1 based languages.


regards,
Gergely

Updating status to: Duplicate

Hi Gergely..

This is not due to MySQL but SQLite3 and JSON. Please see the following bug report:

http://forums.oscomm...ror-log-module/

Marking as duplicate from #259.

Hi Gergely..

In regards to MySQL and UTF-8 databases/tables, Setup creates the tables in utf8 with the following in create table queries:

ENGINE=MyISAM CHARACTER SET utf8 COLLATE utf8_general_ci;

Could you confirm where the latin_swedish setting came from?

Kind regards,

Could you confirm where the latin_swedish setting came from?


Hi Harald,

the default databases collate in Apach Friends XAMP is latin_swedish and not utf8 for hungarian users. I was surprised when i saw it.

http://www.apachefri...g/en/xampp.html


regards,
Gergely

Hi Gergely..

A table can be defined with a different charset other than the default defined for the database. That is why all CREATE TABLE queries performed during Setup contain "ENGINE=MyISAM CHARACTER SET utf8 COLLATE utf8_general_ci" and should be created in utf8.

Did this not occur for your tables?

Kind regards,

Hi Gergely..

A table can be defined with a different charset other than the default defined for the database. That is why all CREATE TABLE queries performed during Setup contain "ENGINE=MyISAM CHARACTER SET utf8 COLLATE utf8_general_ci" and should be created in utf8.

Did this not occur for your tables?

Kind regards,


Hi Harald,

the SQL install code for some reason was no influence.

My steps:

1. I installed Xamp
2. Make v3 database in phpMyadmin (the default Collate was latin_swedish_ci)
3. Setup V3

The main Collate was latin_swedish_ci and caused errors

Next steps:
4. Empty v3
5. Run SQL command ALTER DATABASE `v3` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci
6. Setup V3 again
and Works fine!


V3 (and v3alfa5) doesnt work correctly if database DEFAULT CHARSET is not utf8 because dropping errors and errors.
I draw a conclusion we must use default charset in utf8 for database.

At last summer i wrote utf8 pre-controller php file for future. Use for V3 "ad libitum".


regards,
Gergely

Attached Files



Hi Harald!

The same problem persits in osCommerce V2 and utf8 charset. (fixed on my github)

If using latin or single byte charsets in database the multibyte characters not sure to code on that codepage!

I focused on second part installation into exist database with prefixes.



I suggest that not allow to install V3 into non utf8 charset database. Are there any solution for it and I'm late or we need to do it first?

I think the installation process need conditions for charset.

Possible problems occured with other database engines. (tested on MySQL engine only)


Regards,
Gergely

Ok. I was late.
/blush.png' class='bbc_emoticon' alt=':blush:' />

V3 Create table rules:

CREATE TABLE PREFIX+TABLE_NAME (
FIELDS...
) ENGINE=MyISAM CHARACTER SET utf8 COLLATE utf8_general_ci;

When programmers want to build a table they need to set Charset and collate defenetly if Database not set the default charset UTF8 and Collate utf8_general_ci.

Example when need to set charset:
V3 core installed into the next DATABASE STUCTURE:

create database v3_test DEFAULT CHARACTER SET latin1 DEFAULT COLLATE latin1_swedish_ci;


CREATE TABLE PREFIX+TABLE_NAME (
FIELDS...
) ENGINE=MyISAM;

is fault.


All Right.

show table status;


All Right.

I think this is usable! Thanks!