Forum posts to 'Site Builders'

Re: Site Search - German Site - Problem with Umlaut

Ingo Schommer — Mon, 10 Nov 2008 04:43:27 +1300

Thanks for your patch and ideas, Jim! The problem is that we'd break special character searches in all other (unescaped) fields like "Title". There's two existing tickets with partial patches for the problem, we need somebody to write unit tests for the existing functionality before we can proceed with making the searchform i18n-compatible. Shouldn't be too hard, the groundwork is done in the patch. Anyone keen?

http://open.silverstripe.com/ticket/2267
http://open.silverstripe.com/ticket/2363

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut - Solution

Kolja Hanke — Mon, 10 Nov 2008 01:02:00 +1300

Any solution by now?

Generally the search works fine.
With umlaute the previously described "no results" occurs.
However, when you search for the word "möchten" but type "möchten" the search does get the results that contain the word.

I wonder how the data is stored in the database and if the query parameters need to convert characters to html entities before executing the query and also to convert back to correct HTML to get the umlaute issue solved?

Hint: I'm running SS 2.2.3

Executing the sql query "select id, content from SiteTree limit 10;" from shell (not phpMyAdmin) I get confirmed what was written before: the database contains plain HTML code:
<h4>Pflege ist Dienstleistung, </h4> <p>öde ist langweilig</p>

So I would need to convert the search term using htmlentities() before passing it to the query.

Here is what I did (and it's working):
changed line 76 in sapphire/search/SearchForm.php from

75 public function getResults($numPerPage = 10){
76 $keywords = $_REQUEST['Search'];

into

75 public function getResults($numPerPage = 10){
76 $keywords = htmlentities($_REQUEST['Search'], ENT_QUOTES, 'UTF-8');

For the results I used the following in Page_Results.ss template:

<p>$Content.LimitWordCount</p>

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut

Sun, 20 Jul 2008 05:44:00 +1200

I've got the same problem, so what can I do?

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut

Tue, 01 Jul 2008 15:07:58 +1200

Same problem, also by a searches for words with Umlaute fail.

How can I get this work?

Version 2.2.2

Help please!

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut

robert Koch — Thu, 05 Jun 2008 17:28:14 +1200

No no, content is stored escaped (with html_entities I think)!

Example: [code]ist immer ein Kompromiß aus Funktionalität, Wirtschaftlichkeit und Sicherheit.[/code]

I started to work with 2.2.0 and updated last week to version 2.2.2!

What can I do now?

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut

Ingo Schommer — Thu, 05 Jun 2008 09:47:56 +1200

hm database content is stored in utf8 (so don't be fooled by phpmyadmin giving you garbled output, its html encoding is mostly set to latin1).

can you try to get the sql-query thats generated (in SearchForm.php), append a COLLATE utf8 statement (see http://dev.mysql.com/doc/refman/5.1/en/charset-collate.html), and execute it manually? if it still doesn't work, can you post the query thats failing here? (-> http://pastie.caboo.se)

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut

robert Koch — Thu, 05 Jun 2008 07:43:34 +1200

Oke,
if I use [code]$Content.LimitWordCount[/code] instead of [code]$Content.LimitWordCountXML[/code] I get the correct Umlaute.

But in the database Content is stored escaped! Is this right?

...searches for words with Umlaute fail. How can I get this work?

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut

Ingo Schommer — Mon, 19 May 2008 22:18:26 +1200

so you've followed our search tutorial? http://doc.silverstripe.com/doku.php?id=tutorial:4-site-search

it outputs the content summary as $Content.LimitWordCountXML, which triggers Convert::raw2xml(). so far so good, no double htmlentities encoding. can you make sure the content is stored without being escaped into htmlentities?

as for the newsletter: we need more info, the source of the sent mails is helpful. i suspect you're sending unicode characters in a iso8859-1 (latin1) encoded mail.

Posted to: Site Search - German Site - Problem with Umlaut

Re: Site Search - German Site - Problem with Umlaut

Ralf Giese — Sat, 17 May 2008 21:07:43 +1200

same problem, also by a sent newsletter - thanks for help

Posted to: Site Search - German Site - Problem with Umlaut

Site Search - German Site - Problem with Umlaut

robert Koch — Fri, 16 May 2008 20:38:10 +1200

Hello,
I'm building a German site - in German we have "Umlaute" (special signs) such as "ä ö ü Ä Ö Ü ß".

I want to integrate a Site Search. I followed the intstructions on the Tutorial and implemented the search.

Now I have the problem, that the Umlaut "ä", written as entity "ä" in the content gets re-escaped to &auml; when the search results are displayed - how can I overcome this problem that all ampersands (still those in entities too) get "escaped"?!

Can anyone please help me?!

Posted to: Site Search - German Site - Problem with Umlaut