[SOLVED] Fixing Icedove character encoding

Sector11 · 2016-11-19 20:22:24

SOLVED - fast forward to the solution! And then the OOPS! I missed johnraff's post in my quest. KUDOS johnraff

EDIT: version added

  $ apt list -a icedove
Listing... Done
icedove/stable,now 1:45.4.0-1~deb8u1 amd64 [installed]
icedove/stable 1:45.2.0-1~deb8u1 amd64

  $ icedove -v
 Icedove 45.4.0

My wife received an email from a friend ... some of the characters were "funny" mostly related to smilies:

ðŸ•ðŸ¥estos no se mueren jajaja
ðŸŽ¶ðŸŽ¬finde de mÃºsica y buena pelÃcula yâ˜€

She, S12, was not happy so I tried fixing it and made it worse. I'm in deep mud here!!!!

I typed this is claws-mail:

 Só what do I dó now?
 
 Spanish accents are messed up.

  á â à
  ←↓↑→

left-down-up-right arrows

and sent it to my wife's account - she uses Icedove and it looks like this:

 SÃ³ what dÃ³ I do now?

 Spanish accents are messed up.  

  Ã¡ Ã© Ã²
 â† â†“ â†‘ â†’

left-down-up-right arrows

Anyone got any idea what did and how to fix it?

Toss out a life line will ay!!!!!

Sector11 · 2016-11-19 20:26:14

Must add: Old messages that were already here are displaying just fine.

ohnonot · 2016-11-20 09:34:22

clearly default encoding has changed from utf-8 to ascii presumably.
try these text encoding settings from settings (Prefs -> Display -> fonts&formatting -> advanced)
edit:
check settings both for S11 and S12.

Last edited by ohnonot (2021-07-02 09:51:33)

Head_on_a_Stick · 2016-11-20 13:49:08

Is this only with Icedove or does it happen elsewhere?

Which localisation variants do you have installed?

apt search icedove-l10n | grep installed

We should probably also check the output of:

localectl

Can you give any hints at all as to the nature of your attempted fixes?

Sector11 · 2016-11-20 14:48:29

ohnonot wrote:

edit:
check settings both for S11 and S12.

Well I made those settings identical in Icedove:

The only difference before making it identical was the choices of
Sans-serif: DejaVu Serif
Monospace: Fantasque Sans Mono
and
Incoming mail: Western (ISO-8859-1) - that was changed yesterday from "Unicode (UTF-8)" after reading about this on the net.

Then I opened claws-mail (used by Sector11 only, with mail in 'text mode') and sent this to Icedove (used by Sector12 in mail received in HTML mode):

This is a fáke tést with español characters

and or østûff like ½ € and £
← left
→ right
↑  up
↓  down

:)  :D  :lol:
-----
Decide to make today a good day.

and then opened Icedove to see:

This is a fÃ¡ke tÃ©st with espaÃ±ol characters

and or Ã¸stÃ»ff like Â½ â‚¬ and Â£
â† left
â†’ right
â†‘  up
â†“  down

  :D  :lol:

-----
Decide to make today a good day.

Sector11 · 2016-11-20 15:07:09

Head_on_a_Stick wrote:

Is this only with Icedove or does it happen elsewhere?

Nowhere else - to the best of my knowledge. Just tested in LO-Writer and it's perfect:

Head_on_a_Stick wrote:

Which localisation variants do you have installed?
apt search icedove-l10n | grep installed

 20 Nov 16 @ 12:01:36 ~
  $ apt search icedove-l10n | grep installed

WARNING: apt does not have a stable CLI interface yet. Use with caution in scripts.

 
 20 Nov 16 @ 12:01:39 ~
  $ ser icedove-l10n | grep installed
alias ser = aptitude search
 
 20 Nov 16 @ 12:01:55 ~
  $

Head_on_a_Stick wrote:

We should probably also check the output of:
localectl

 20 Nov 16 @ 12:03:00 ~
  $ localectl
   System Locale: LANG=en_CA.UTF-8
                  LANGUAGE=en_CA:en
       VC Keymap: n/a
      X11 Layout: latam
       X11 Model: microsoftpro
     X11 Options: terminate:ctrl_alt_bksp
 
 20 Nov 16 @ 12:03:04 ~
  $

That has been the same since my #! days.

Head_on_a_Stick wrote:

Can you give any hints at all as to the nature of your attempted fixes?

Aftre searching the net I saw something that said MS (outlook) interprets emotes differently and to try incoming mail as: Western (ISO-8859-1) which did not work. now back to Unicode (UTF-8) as seen in my previous post.

Head_on_a_Stick · 2016-11-20 15:09:41

Sector11 wrote:

alias ser = aptitude search

Sorry S11, that won't work -- `aptitude` doesn't use "installed" in the output

EDIT: oops, didn't read your post properly, disregard :8

Try this instead:

apt search icedove | grep installed

Sector11 · 2016-11-20 15:15:51

More info:
Copy and pasting the "test text" into Icedove and sending it to Sector11 in Claws-mail the results are perfect.

... and reading the /Sent folder in Icedove the text is just fine as well.

Sector11 · 2016-11-20 15:18:33

Head_on_a_Stick wrote:

Sorry S11, that won't work -- `aptitude` doesn't use "installed" in the output
Try this instead:
apt search icedove | grep installed

Or list is better:
 20 Nov 16 @ 12:16:49 ~
  $ apt search icedove | grep installed

WARNING: apt does not have a stable CLI interface yet. Use with caution in scripts.

icedove/stable,now 1:45.4.0-1~deb8u1 amd64 [installed]
 
 20 Nov 16 @ 12:16:53 ~
  $ list icedove
Alias for: apt list -a filename
Listing... Done
icedove/stable,now 1:45.4.0-1~deb8u1 amd64 [installed]
icedove/stable 1:45.2.0-1~deb8u1 amd64

 
 20 Nov 16 @ 12:17:13 ~
  $

At least I don't get the warning. But why 2 ??????

Head_on_a_Stick · 2016-11-20 15:20:32

Sector11 wrote:

list is better

Oh yes, so it is -- thanks for the tip

Well I'm a bit flummoxed here, sorry, hopefully others will be able to do better.

Sector11 · 2016-11-20 15:36:20

Well, after all the help you give everyone here it just made my day to give you a new tip to help others. Trust me - you are more than welcome.

Also "apt list -a" will give you a complete alphabetical listing of everything installed on your system - double spaced.

EDIT: hold that. Gives you *everything* - installed or not!!!!

And Icedove - (Thunderbird) I'd love to deep-six but S12 wants her HTML mail. {sigh}

It was working perfectly up until, quoting S12, "about four days ago" but she never said anything. I don't use it except to read "Exchange Rates" that we've been getting in "Thunderbird" since our Ubuntu days.

OT & AWESOME

apt list -a | grep installed

Last edited by Sector11 (2016-11-20 15:45:40)

BLizgreat! · 2016-11-20 15:59:32

Perhaps time to get dirty ? Don't have any real experience with thunderbird or icedove but should be a way to backup all her contacts and etc. Then perhaps try removing or purging the sucker and turn around and reinstall it ?

Only a thought. Of course you realize better than anyone, if ya in trouble now, lose her stuff and then we'll see. Errrrr more like we may never hear from you again.

Last edited by BLizgreat! (2016-11-20 16:00:21)

Sector11 · 2016-11-20 16:19:51

BLwillbegreat! wrote:

Only a thought. Of course you realize better than anyone, if ya in trouble now, lose her stuff and then we'll see. Errrrr more like we may never hear from you again.

And there lies the greater truth!

However, that has been done in the past and with, for example, fresh installs of a distro, backing up and restoring ~/.mozilla and or ~/.icedove has never been a problem

ohnonot · 2016-11-20 21:59:01

Sector11 wrote:

More info:
Copy and pasting the "test text" into Icedove and sending it to Sector11 in Claws-mail the results are perfect.

just to get that right, you are typing (why copy paste?) a message on S12's icedove, sending it to S11, and opening it in claws mail, all looks fine?
the previous preferences are all set to utf-8?
what if you open the same message in icedove?

where and how can you reliably reproduce the problem?

can we safely say this is an issue with S12's icedove, not yours, not apt/package related?
how did you troubleshoot? tried sending mail from your webmail directly, tried opening it in her webmail directly?
could it be a setting of her webmail, and not icedove?

it looks very much like utf-8 encoded text being decoded as ascii, i hope we can all agree on that?

you said she wants html viewing; maybe there's other settings in icedove, regarding html.
another look at my TB's settings did not unfortunately reveal anything.
it has an "about:config" like FF, and i can search that for utf.

but right now i suspect some of my other questions might lead us closer.

Sector11 · 2016-11-20 23:28:27

ohnonot wrote:

Sector11 wrote:
More info:
Copy and pasting the "test text" into Icedove and sending it to Sector11 in Claws-mail the results are perfect.
just to get that right, you are typing (why copy paste?) a message on S12's icedove, sending it to S11, and opening it in claws mail, all looks fine?

Yes ...

1- If I paste this into a compose new email in Icedove (S12) and send it to Claws-mail (S11) it looks fine in Icedove before sending it.

This is a fáke tést with español characters

and or østûff like ½ € and £
 ← left
 → right
 ↑  up
 ↓  down

   :D  :lol:
 -----
 Decide to make today a good day.

2. S11 (Claws mail) responds

Awesome I got it
Feliz día español

and my wife sees, in icedove, my text and the quoted original text:

Awesome I got it

Feliz día español

On Sun, 20 Nov 2016 20:08:26 -0300
xxxxxxxxxx <xxxxxxxxxx@xxxxxxx.xxx> wrote:

> This is a fáke tést with español characters
>
> and or østûff like ½ € and £
> ← left
> → right
> ↑  up
> ↓  down
>
>   :D  :lol:
> -----
> Decide to make today a good day.

-----
Decide to make today a good day.

HUH!!!!!!! WTFarmer!!!!!!!!!!!!!

I did nothing ... she has been getting scrambled mail all day!!!!! ?????

Like this:

Am I going mad ... ????
I give up. :8 :8 :8

ohnonot · 2016-11-24 04:13:15

so if i understand correctly, your wife receives your unicode characters and sees them sorrectly, then she receives some other mail and special charactersa are scrambled again?
it must be an encoding issue on your wife's side.
try checking all her icedove settings again, and also her webmail account settings!!!

and do more troubleshooting, like sending her the same weird text from a disposable emailaddress.

Last edited by ohnonot (2016-11-24 04:14:08)

Sector11 · 2016-11-24 16:45:11

Thank you for your continued interest, ohnonot.

ohnonot wrote:

so if i understand correctly, your wife receives your unicode characters and sees them correctly,

Correct!

then she receives some other mail and special characters are scrambled again?

Also correct.

Icedove sees " á, ñ, ü, ¡ and ¿" type characters sent from claws-mail (me) on the same machine just fine.

Claws-mail sends/receives "text" mail only to/from "everyone" ... I HATE HTML emails. (OK, that's an opinion and OT)
Icedove sends "text" mail and receives HTML messages with some problems: as seen here, but not in all cases:

país. ¿Te acordás? Cerrá los

The above are from two emails today.

it must be an encoding issue on your wife's side.
try checking all her icedove settings again,

Been doing this almost daily, sometime making it worse sometime better.

and also her webmail account settings!!!
and do more troubleshooting, like sending her the same weird text from a disposable emailaddress.

There is no "webmail" here. Only mails from our ISP.

Here's images of Icedove settings (at least the ones I think are pertinent) :
General
Display - Formatting -> [Advanced] = Fonts & Encodings
Composition General -> Send options: HTML and Plain Text Domain lists are empty.

While doing this she received another email. From our Cell Phone company ... one big image except for this, at the end, which is perfect:

Nota: Este correo se envía en forma personal y directa al propietario de esta casilla cumpliendo con la normativa vigente en la República Argentina, por lo que no podrá ser considerado Spam por contener una forma real de eliminación del listado. Desuscríbeme de esta lista de contactos

BLizgreat! · 2016-11-24 17:53:24

1. Maybe the problem is on the server side of things ? Not overly up on this subject myself. Don't think I've ever used an email client. Even though thunderbird is installed on this OS. Might be time to widen the net in search of a fix. See stuff like this.

2. Thunderbird and Icedove likely have support forums. Here's thunderbirds forum. Always best to seek out peeps with the most experience and knowledge on a topic eh. Anything that applies to thunderbird's going to apply to the rebranded Icedove too.

3. I GOT IT ( kidding this is tongue in cheek Sector) just send a comm to all her contacts, that from now on emails sent to her will have to be made in a photo editor so that the text doesn't get screwed up. IMMA GENIUS, LOOK MA ... NO HANDS.

Last edited by BLizgreat! (2016-11-24 17:54:18)

Sector11 · 2016-11-24 18:32:38

BLwillbegreat! wrote:

1. Maybe the problem is on the server side of things ? Not overly up on this subject myself. Don't think I've ever used an email client. Even though thunderbird is installed on this OS. Might be time to widen the net in search of a fix. See stuff like this.

Who's server? Our ISP or each individual server out there that's the starting point of each email sent her?

What my claws-mail and her icedove have in common: This BunsenLabs install (we use the same login), email server: Our ISP
So that eliminates problems between us (well as far as mail is concerned).

As to searches; been there, done that (added -windows), and still doing it.

2. Thunderbird and Icedove likely have support forums. Here's thunderbirds forum. Always best to seek out peeps with the most experience and knowledge on a topic eh. Anything that applies to thunderbird's going to apply to the rebranded Icedove too.

I will check that thank you. Strange thing that really frys by butt - NO not a flame "that high" - is we've been using thunderbird/icedove since 2007 and never had a problem. All of a sudden from out of nowhere - OOPS!

3. I GOT IT ( kidding this is tongue in cheek Sector) just send a comm to all her contacts, that from now on emails sent to her will have to be made in a photo editor so that the text doesn't get screwed up. IMMA GENIUS, LOOK MA ... NO HANDS.

Yea and I can just hear the "chuckles" to "outright laughter" now.

And tears of laughter as they yell: "HEY YOU! ... YEA, YOU! Take a tpying corse or get youre keyboard off your fingers!"

Sector11 · 2016-11-24 19:25:58

@BLwbg - et al

Done - posted in the Mozilla forums!

#1 2016-11-19 20:22:24

[SOLVED] Fixing Icedove character encoding

#2 2016-11-19 20:26:14

Re: [SOLVED] Fixing Icedove character encoding

#3 2016-11-20 09:34:22

Re: [SOLVED] Fixing Icedove character encoding

#4 2016-11-20 13:49:08

Re: [SOLVED] Fixing Icedove character encoding

#5 2016-11-20 14:48:29

Re: [SOLVED] Fixing Icedove character encoding

#6 2016-11-20 15:07:09

Re: [SOLVED] Fixing Icedove character encoding

#7 2016-11-20 15:09:41

Re: [SOLVED] Fixing Icedove character encoding

#8 2016-11-20 15:15:51

Re: [SOLVED] Fixing Icedove character encoding

#9 2016-11-20 15:18:33

Re: [SOLVED] Fixing Icedove character encoding

#10 2016-11-20 15:20:32

Re: [SOLVED] Fixing Icedove character encoding

#11 2016-11-20 15:36:20

Re: [SOLVED] Fixing Icedove character encoding

#12 2016-11-20 15:59:32

Re: [SOLVED] Fixing Icedove character encoding

#13 2016-11-20 16:19:51

Re: [SOLVED] Fixing Icedove character encoding

#14 2016-11-20 21:59:01

Re: [SOLVED] Fixing Icedove character encoding

#15 2016-11-20 23:28:27

Re: [SOLVED] Fixing Icedove character encoding

#16 2016-11-24 04:13:15

Re: [SOLVED] Fixing Icedove character encoding

#17 2016-11-24 16:45:11

Re: [SOLVED] Fixing Icedove character encoding

#18 2016-11-24 17:53:24

Re: [SOLVED] Fixing Icedove character encoding

#19 2016-11-24 18:32:38

Re: [SOLVED] Fixing Icedove character encoding

#20 2016-11-24 19:25:58

Re: [SOLVED] Fixing Icedove character encoding

Board footer