Re: Fixed dynamic chat bug regarding special characters

  • From: "Bryan Garaventa" <bgaraventa11@xxxxxxxxxxxxxx>
  • To: <programmingblind@xxxxxxxxxxxxx>
  • Date: Sun, 6 Jul 2008 19:57:33 -0700

If everybody was happy with everything, we'd live in a perfect world...

----- Original Message ----- From: "Octavian Rasnita" <orasnita@xxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Sunday, July 06, 2008 12:05 PM
Subject: Re: Fixed dynamic chat bug regarding special characters


Well, I have tried it on your page, but it doesn't work.

If the page display the text using UTF-8, it also gets the input using UTF-8, so it will work for all users no matter that they use french, spanish, japanese or romanian special chars. For english-speaking users, the UTF-8 chars are the same as the ISO-8859-2 chars, so it is no difference.

But the sample chat on your site is not working well.

Octavian

----- Original Message ----- From: "Bryan Garaventa" <bgaraventa11@xxxxxxxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Sunday, July 06, 2008 9:51 PM
Subject: Re: Fixed dynamic chat bug regarding special characters


The script does support UTF8, that is what I was saying.

My point is, that I can't force anyone to use a UTF8 charset, so the script needs to be able to use any of them. So now it does.

If you want to use UTF8, you just enter 'UTF-8' in the $charSetString variable in the config file, and then you have UTF8 encoding for your page using the chat script. It can't get much simpler than that.

Also, this would theoretically enable Japanese, Chinese, and Russian charsets to work properly as well. So it is pretty versatile.

In short though, the chat app will work properly on any UTF8 page.

----- Original Message ----- From: "Octavian Rasnita" <orasnita@xxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Sunday, July 06, 2008 1:45 AM
Subject: Re: Fixed dynamic chat bug regarding special characters


Well, I don't understand how it was fixed if it doesn't work for everyone. It doesn't matter what single byte charset the site admin uses, because the visitors can be anyone, using any type of chars in any type of encoding.

A french site admin shouldn't consider that all the visitors of his page have only french names and they don't need to use other kind of chars. These things regarding charsets were not very important 10 years ago, but now that UTF-8 exists and is supported by many programming languages and programs, it should be use everywhere it is possible.
Well, it was just an advice, but you can do whatever you want.

Octavian

----- Original Message ----- From: "Bryan Garaventa" <bgaraventa11@xxxxxxxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Sunday, July 06, 2008 11:06 AM
Subject: Re: Fixed dynamic chat bug regarding special characters


The special char problem appears to be all fixed.

The script now has a new variable, $charSetString, which can be used to identify the charset string constant as specified at http://php.net/htmlentities

Since this setting is user specified, any person using the script in any language, should be able to insert the script in their page, specify the desired charset string that matches their page, and the message parser will automatically encode the message string accordingly at runtime.

The drawback though, which there is no answer for that I know of, is that if a user inputs a string of characters that is not supported by the page's charset, the message will likely not display those characters correctly. I don't see any solution to this, other than what I've done by allowing the script message encoder to exactly reflect the charset for the parent page.

The reason for my own pages being displayed in iso-8859-1 has to do with the html generator I use, which overwrites any modified pages with new ones whenever I publish updates to the site. This means that over 150 pages or so are updated if I manually change the charset for any of them. Stupid yes, but it would be too much difficulty to change to a new editor at this point... I get a headache just thinking about all that recoding...

----- Original Message ----- From: "Octavian Rasnita" <orasnita@xxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Friday, July 04, 2008 12:55 PM
Subject: Re: Fixed dynamic chat bug regarding special characters


If you use UTF-8, you won't need to do that, because each char for each language has a special code if it is encoded with UTF-8. The english chars are represented using a single byte, but the special chars in other languages can be represented using more than 1 byte for a character so there won't be any conflict with other chars from other languages.

But as I said, I don't know if this is very easy to do with PHP if you don't use a framework that does the encoding/decoding of UTF-8 chars automaticly.

Octavian

----- Original Message ----- From: "Bryan Garaventa" <bgaraventa11@xxxxxxxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Friday, July 04, 2008 10:22 PM
Subject: Re: Fixed dynamic chat bug regarding special characters


I know, the problem is that I can't be certain which char set the page will be for any user who is using the chat script, so it has to be able to handle this automatically.

This should do the trick at
http://php.net/htmlentities

For instance, making a setting in the config file to enter the desired char set string, then something like the following statement would convert the message at runtime
$msg = htmlentities($msg, ENT_NOQUOTES, $charSetString);

Will give this a go in a bit.

----- Original Message ----- From: "Octavian Rasnita" <orasnita@xxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Friday, July 04, 2008 3:02 AM
Subject: Re: Fixed dynamic chat bug regarding special characters


Well, when using UTF-8, you don't even need to use character entities. You can use simple UTF-8 chars and they display correctly.

Octavian

----- Original Message ----- From: "Bryan Garaventa" <bgaraventa11@xxxxxxxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Friday, July 04, 2008 10:52 AM
Subject: Re: Fixed dynamic chat bug regarding special characters


Thanks, I'll look into it. I think I've stumbled across the problem, looks like the characters are being saved correctly, and also retrieved correctly, but the html entities are not being converted to html equivalents.

I found the page at
http://www.prolifique.com/entities.php.txt
which may help fix this, I'll play around with it and see if I can implement something usable within the parser.

----- Original Message ----- From: "Octavian Rasnita" <orasnita@xxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Friday, July 04, 2008 12:16 AM
Subject: Re: Fixed dynamic chat bug regarding special characters


Those chars are in romanian language, but I don't think this is important, because UTF-8 should treat all unicode chars the same.

Oh, I've re-visited your page and I've seen:

<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

I don't think it is sane to use ISO-8859-1 these days.
I think that all the pages should use UTF-8 for beeing able to display correctly special chars in more languages.

I don't know how easy is to use UTF-8 when using PHP, but there are more PHP frameworks and I think most of them should be using UTF-8 as a default...

Octavian

----- Original Message ----- From: "Bryan Garaventa" <bgaraventa11@xxxxxxxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Friday, July 04, 2008 8:49 AM
Subject: Re: Fixed dynamic chat bug regarding special characters


Can you tell me which language set the characters belong to? This is turning out to be a bit difficult to track down. What I need to do is convert the special chars into a Unicode string, then convert them back without any data loss. Perhaps a combo of utf8 and base64 encoding might help, but I'm not sure what to test against. Knowing the language and chars that you are entering would help.

Thanks.


----- Original Message ----- From: "Octavian Rasnita" <orasnita@xxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Thursday, July 03, 2008 10:22 PM
Subject: Re: Fixed dynamic chat bug regarding special characters


Hi Brian,

Yes I've tested again and now with Firefox, but the special chars are still not printed correctly.
I guess those chars are send using UTF-8 encoding.

Octavian

----- Original Message ----- From: "Bryan Garaventa" <bgaraventa11@xxxxxxxxxxxxxx>
To: <programmingblind@xxxxxxxxxxxxx>
Sent: Friday, July 04, 2008 5:16 AM
Subject: Fixed dynamic chat bug regarding special characters


Octavian, regarding the issue that you mentioned earlier about the chat app not displaying special characters correctly, could you test this when you have a chance to see if the chat form is now accepting special characters? This should be fixed. I changed the save method from urlencode/urldecode to base64_encode/base64_decode, which should correct the problem. I think...
http://gutterstar.net/dynamic_live_chat.php
Thanks,
Bryan

__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind


__________
View the list's information and change your settings at //www.freelists.org/list/programmingblind

Other related posts: