NetTalk Central

Author Topic: Netwebmethod/JSON and international characters  (Read 1865 times)

AtoB

  • Jr. Member
  • **
  • Posts: 74
    • View Profile
    • Email
Netwebmethod/JSON and international characters
« on: May 02, 2023, 08:54:55 AM »
Hi all,

I'm receiving utf postdata via nettalk and probably need to convert somewhere in the process as I don't save the international characters correctly, escpecially the euro-sign (although special characters like ??? etc. work fine).

I receive the following (as a json string within quotes), which seem to me like correctly encoded:

"\u00e4\u00e0\u00e2\u00e9\u20ac"

The last one displays as "?" in my screens and should be the euro sign (but the first four get "translated" somehow correctly)

Currently I use following code to load the json (gtSt is effectively a stringtheory object) into my queues

  gtSt.Setvalue(SELF.rP_web.Getvalue('_postdata_'))
 
  json.start()
  json.SetTagCase(jf:CaseAsIs)
  json.Load(element, gtSt) ! Load From a StringTheory object   

Where (and with what method) should I modify this string.

TIA,

Ton

Bruce

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11220
    • View Profile
Re: Netwebmethod/JSON and international characters
« Reply #1 on: May 03, 2023, 01:24:26 AM »
Hi Ton,

Json is always unicode (utf-8) but also can include encode characters, as you are seeing;
"\u00e4\u00e0\u00e2\u00e9\u20ac"

There are a few ways to tackle this, but for simplicity, this is what I'd do;

gtSt.Setvalue(SELF.rP_web.Getvalue('_postdata_'))
gtSt.jsonDecode() ! this turns the \u into actual utf-8 characters
gtSt.ToAnsi(st:encodeutf8 , st:CP_WINDOWS_1252)

and then the JSON stuff.

[1] I'm suggesting st:CP_WINDOWS_1252, as your local code page, but that will differ depending on your country.
st:CP_WINDOWS_1252 is for most of Western and Scandinavian Europe though. I don't know if that's the right code page for you though.

Cheers
Bruce


AtoB

  • Jr. Member
  • **
  • Posts: 74
    • View Profile
    • Email
Re: Netwebmethod/JSON and international characters
« Reply #2 on: May 03, 2023, 05:39:09 AM »
Hi Bruce,

thanks, that worked and the codepage is correct too :-)

AtoB

  • Jr. Member
  • **
  • Posts: 74
    • View Profile
    • Email
Re: Netwebmethod/JSON and international characters
« Reply #3 on: May 04, 2023, 03:12:49 AM »
Hi Bruce,

two additional questions (both performance related):

1 - jFiles apparently does the translations of the "/u...." stuff somewhere when loading/parsing the data (but assumes another codepage?). Is this configurable? So can I set the target codepage prior to loading and all will be processed in one go?

2 - is the SELF.rP_web.Getvalue('_postdata_') somewhere accessable directly as a stringtheory object?

TIA,
Ton

Bruce

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11220
    • View Profile
Re: Netwebmethod/JSON and international characters
« Reply #4 on: May 08, 2023, 05:47:34 AM »
1.  yes
 it takes the code page from the system{prop:charset}
 so you may want to set that;
  system{PROP:CharSet} = CHARSET:ANSI
 should do it

You could also set it explicitly by deriving the jsonclass.PushStack method, and setting;

  Self.Stack.Buffer.CodePage = st:CP_WINDOWS_1252

2. no, you can assign it from that into a stringtheory object of your own.

Cheers
Bruce





AtoB

  • Jr. Member
  • **
  • Posts: 74
    • View Profile
    • Email
Re: Netwebmethod/JSON and international characters
« Reply #5 on: May 08, 2023, 09:22:07 AM »
Hi Bruce,

number 1 is het good answer  :)

(I looked in jfiles.clw to find something related to codepages or so, but I didn't realize it was all stringtheory behind the scenes). The prop:charset is a nice way to get all my stringtheory objects in the right codepage.

Thanks!