• UTF-8 messages

    From Al@21:4/106 to apam on Wed Oct 23 09:11:32 2019
    Hello Apam,

    There are times when message just seem to end at some random spot.

    The trouble seems to be with messages that contain UTF-8 characters.

    One message (two actually) begin with "Bj~Trn" (as seen in golded). When viewing those messages in Magicka I see "Bj" and that's it. Just a blank
    screen after that.

    I've seen this in Fido a few times but wasn't sure what to think.

    The Bj~Trn is what I see in golded when folks spell Bjorn with the two
    dots obove the o in UTF-8. I think ;)

    If you like I could zip up that jam base and you could have a look?

    Ttyl :-)
    Al

    --- MagickaBBS v0.13alpha (Linux/x86_64)
    * Origin: The Rusty MailBox - Penticton, BC Canada (21:4/106)
  • From Oli@21:1/151 to Al on Wed Oct 23 18:38:54 2019
    On Wed, 23 Oct 2019 05:11:32 -0700
    "Al -> apam" <0@106.4.21> wrote:

    Hello Apam,

    There are times when message just seem to end at some random spot.

    The trouble seems to be with messages that contain UTF-8 characters.
    [...]
    The Bj~Trn is what I see in golded when folks spell Bjorn with the two
    dots obove the o in UTF-8. I think ;)

    This should be a correctly encoded UTF-8 message with some umlauts

    Björn Björn Björn

    äöü
    ÄÖÜ

    and emojis

    🧙♀️
    🐉
    🔮
    🧙
    🧙♂️

    ---
    * Origin: (21:1/151)
  • From Al@21:4/106 to Oli on Wed Oct 23 10:08:54 2019
    This should be a correctly encoded UTF-8 message with some umlauts

    Björn Björn Björn

    I see your spelling of Bjorn just fine. :)

    äöü ÄÖÜ

    and emojis

    ?

    Your message seems to end here, no tearline, no origin although I see it
    fine in golded.

    Thanks!

    Ttyl :-)
    Al

    --- MagickaBBS v0.13alpha (Linux/x86_64)
    * Origin: The Rusty MailBox - Penticton, BC Canada (21:4/106)
  • From Oli@21:1/151 to Al on Wed Oct 23 20:25:36 2019
    This should be a correctly encoded UTF-8 message with some umlauts

    Bjrn Bjrn Bjrn

    I see your spelling of Bjorn just fine. :)



    and emojis

    ?

    Your message seems to end here, no tearline, no origin although I see
    it fine in golded.

    Not sure it's entirely Magicka's fault. Maybe JamNNTPd has some problems with UTF-8 too. My previous message was composed in Thunderbird. If I save this message and open it with a text editor, it complains about invalid characters.


    --- GoldED+/LNX 1.1.5-b20180707
    * Origin: this message has been mangled by Mystic BBS (21:1/151)
  • From apam@21:1/125 to Al on Thu Oct 24 12:31:22 2019
    There are times when message just seem to end at some random spot.

    The trouble seems to be with messages that contain UTF-8 characters.

    It's iconv bombing out due to invalid characters. It's supposed to
    replace invalid characters with a ? or something, but it's not.

    I've added IGNORE to the list, so now troublesome characters will be
    ignored instead, it will be available when I merge the wordwrap branch.

    Andrew

    --- MagickaBBS v0.13alpha (Linux/x86_64)
    * Origin: Nocturnal - nocturnal.hopto.org:2023 (21:1/125)
  • From apam@21:1/125 to Al on Thu Oct 24 12:49:17 2019
    There are times when message just seem to end at some random spot.

    The trouble seems to be with messages that contain UTF-8 characters.

    It's iconv bombing out due to invalid characters. It's supposed to
    replace invalid characters with a ? or something, but it's not.

    It seems to be that the messages you sent identified as UTF-8 but
    weren't. IE, the CP437 o with dots above it was used.

    I've made a few changes again, now if iconv bombs out, it will just
    display the raw text - you might get funny characters, but at least it
    won't be cut off.

    Andrew


    --- MagickaBBS v0.13alpha (Linux/x86_64)
    * Origin: Nocturnal - nocturnal.hopto.org:2023 (21:1/125)
  • From Al@21:4/106 to apam on Wed Oct 23 20:35:18 2019
    I've made a few changes again, now if iconv bombs out, it will just display the raw text - you might get funny characters, but at least it won't be cut off.

    Yep, I updated and revisited those messages.. Seems to be some off
    characters alright, but I can see the whole message now. So that's good progress.

    Ttyl :-)
    Al

    --- MagickaBBS v0.13alpha (Linux/x86_64)
    * Origin: The Rusty MailBox - Penticton, BC Canada (21:4/106)