Special characters not encoded properly

The Issue :

Arabic/Hebrew/non-latin characters,when extracted from database/dynamic source engine, are not properly rendered, instead are shown as '??????'

The Story (debug steps) :

Whenever a user saves something (let's say meta keywords as shown above) to database, it is saved normally with correct encoding, but after saving the characters are not rendered properly but instead as '?????' due to encoding problem.

All the files are UTF-8 Encoded, the headers are sent as UTF-8,the database collation is utf8_general_ci,meta charset is set to utf-8 and the connection to database is also utf-8.

Saving the keywords directly via MySQL console or PHPMyAdmin will render the same result, thus the issue is with showing the characters and not saving them.

Getting the keywords manually using mysql_query + mysql_fetch_array will solve the issue, thus the issue is caused by the function used to get the keywords (or the file containing this function).

The Solution :

Can be found Here

Best Regards

Answers


If you are using MySQL check if your tables and also columns are set to utf8. If you've created all database structure and changed the database coding to utf8 afterwards it does not change coding of existing tables and columns. It just sets the default coding for newly created relations.


I noticed your pages are XHTML, don't use htmlentities() (even with correct encoding) on XHTML pages. The only characters that should be converted to entity references in XHTML(or XML) documents are &, ", <, >

function xmlEntities( $s ) {
    return str_replace( array( '&', '"', '<', '>' ),
                    array( '&amp;', '&quot;', '&lt;', '&gt;' ), $s );
}

The Solution :

It seems like the editor used to save the files as UTF8 had a bug and did not save them properly encoded. Switching to Sublime Text 2, and saving all the files as UTF8 has solved the issue.

Best Regards


Need Your Help

How to avoid committing .gitignore file in Eclipse Egit

eclipse git

I have a file on my local branch that I have added to the .gitignore file that is shipped with Ecipse git(Egit) but each time I try to commit and push that branch to the remote branch, I see my .

What is the use of .exp and what is the difference between .lib and .dll?

c dll linker static-libraries

During compilation and linking, what is use of .exp? What is the difference between .lib and .dll? I know that .lib will be used, while linking and .dll will be used when running the program. But w...