NetTalk Central

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - AtoB

Pages: 1 [2]
16
Hi All,

I'm trying to get my webservices secure, but can't get it to work in my test environment (locally, so I guess there is no firewall is involved ...)

I want all traffic with this server to be secure, so my documentation should only be visible when accessed over https and my methods too.

I created my CA certificate (see post last week: I did a OpenSSL (re-)install, it is now running version 1.1.0b, dated 26 sept 2016, but I don't think this is the culprit right now ...)

I added my CA-root certificate to both browsers (I use FF, but also tried IE), but both don't show my documentation, when I type "https://localhost:443/myservicename".

 - Firefox then shows the message something like "Unable to connect to localhost ... errorcode: SSL_ERROR_NO_CYPHER_OVERLAP"
- IE says I should activate TLS1.0 through 1.2 in my browser settings (which are activated ...)

I also tried calling a method from my (Clarion) webclient testing procedure (over port 443 and using the https "prefix"), but this gives "The error number was -53 which menas Open Timeout or Failure error - [SSL Error = 16].

The last error lead me to nettalk central (<g>) so I checked the netsimple code but it says "self.SSLMethod = NET:SSLMethodTLS" in the init method (and as far as I know, I'm not changing it ...)

the following is the code to activate ssl serverside (both files are present in the \certificates folder):

  ThisWebserver.SSL = 1 ! Use SSL to make a Secure Web Server
  ThisWebserver.SSLCertificateOptions.DontVerifyRemoteCertificateCommonName = 1
  ThisWebserver.SSLCertificateOptions.DontVerifyRemoteCertificateWithCARoot = 1
  ThisWebserver.SSLCertificateOptions.CertificateFile = 'certificates\webservice.crt'
  ThisWebserver.SSLCertificateOptions.PrivateKeyFile = 'certificates\webservice.key'
  ThisWebserver.SSLCertificateOptions.ServerName = 'www.tvdb.nl.crt'
  ThisWebserver.MoveFolder(clip('web') & '\certificates','certificates')

I'm not sure what the "ServerName" property should contain in my test environment (I also tried "webservice.crt" ...).

I'm out of ideas right now ... Is there a way to somehow trace where this stuff stops working. I don't see any request coming in at the NT server, but I don't now wether https request show up there  at all ?

Any help is really appreciated!

TIA,
Ton

17
Web Server - Ask For Help / creating ca certificate ...
« on: November 08, 2016, 04:36:41 AM »
Hi all,

in order to secure a webservice I thought I'd create a certificate. Step one creating the CA certificate doens't seem to work:

- I run CreateCACertificate.bat
- I type a password twice

then it seems to go wrong, the follwing is displayed:


Code: [Select]
[color=blue]--- Create Certificate using Private Key
(Please enter the same password you used earlier when asked to do so)

WARNING: can't open config file: /usr/local/ssl/openssl.cnf
Unable to load config info from /usr/local/ssl/openssl.cnf


--- Display Certificate

WARNING: can't open config file: /usr/local/ssl/openssl.cnf
Error opening Certificate .\YourCARoot\cacert\YourCA.crt
5612:error:02001002:system library:fopen:No such file or directory:.\crypto\bio\bss_file.c:391:fopen('.\YourCARoot\cacert\YourCA.crt','rb')
5612:error:20074002:BIO routines:FILE_CTRL:system lib:.\crypto\bio\bss_file.c:393:
unable to load certificate
[/color]

It looks like a "openssl.cnf" file is missing somehow, at least I cannot find it ... I'm not getting (for example) a country code to billed in ...

- is this file required?
- are there any other requirements for this batch to complete?

TIA,
Ton



18
Web Server - Ask For Help / jFiles : improving parsing speed?
« on: September 14, 2016, 12:22:18 AM »
Hi Bruce,

I'm using jFiles/nettalk webservices quite intensively and now I am into the speed "thingy". I have to deal with tremendous amounts of rather large requests and optimised my appliction and backend interaction as much as possible.

As jFiles ships with source I can look for improvements there too, (and you've probably guessed it :) so I did ... I've attached the modified 1.26 version, hope you will consider implementing these adjustments and suggestions below

some requests and/or suggestions (not applied to the attached source <g>):

1 - something I haven't tried, but only thought of: is it a good idea to create one toplevel object as a container for most of jsonclass properties in one one place? Then each created jsonclass object will need a reference to this toplevel object. But this would probably create less overhead than all the cloning and cascading (up/down) of the properties for each object. The objects themselves can be smaller, so construction them runs faster. Again I don't grasp the concept of jFiles completely and I'm only thinking out loud :-)

2 - the general loading/parsing of the json file contains quite a lot of calls to the .HandleChar procedure, but since version 1.25 (?) you've added a third parameter pPlace that is filled with a call to substr() for each call. I haven't changed these the attached version, but stripping them out improves the call to the LoadString method with another 4 to 5 percent speed improvement. Maybe passing the pToParse String by reference and only substr it when an error occurs (which should be rarely the case ...)?

3 - would you consider removing (or making it a configurable object property (with the downside of cascading ...)?) the calls to SELF.Trace in at least both the .FillStructure method and the .GetObject (when no object is found) method?. I prefer to ship all my code with debug option on, so these calls are effectively made in the production system and do slowdown the system (on the 5000 record example this saves me .3 secs per request)


Now for the applied improvement (see attached modified jFiles.clw/inc)

when loading json into a queue I noticed the fieldlabels are processed for each queue record/json object. I've created a "preprocessor" for this, with little or no overhead that reduces a total processing time of 5000 records from 2.24 secs to 1.95 secs (this is total "service time", so includes api validition, fetching against backend of 5000 records, and producing a 5000 record result queue and turning that into json too)

the following needs to be changed:

- in jfiles.inc : add the ColNameQType declaration
- in jfiles.inc : modify the prototype of the first Fillstructure item to : FillStructure                   procedure(*GROUP pGroup, <ColNameQType pColNameQ>),Long,Proc,VIRTUAL                         

- in files.clw : in the FillStructure (first occur.) method the following is added (see around line 1539):

      if ~OMITTED(pColNameQ)
        GET(pColNameQ, c)
        nIndex = SELF.Position(pColNameQ.label)
!?       self.trace(all(' ',indent) & 'Trying to fill ' & clip(pColNameQ.label) & ' nIndex = ' & nIndex  )               
      else
        PropertyName.SetValue(WHO(pGroup,c))
        self.AdjustFieldName(PropertyName,self.TagCase)
        nIndex = SELF.Position(PropertyName.GetValue())
!?       self.trace(all(' ',indent) & 'Trying to fill ' & clip(PropertyName.GetValue()) & ' nIndex = ' & nIndex  )
      end

- in jFiles.clw the method "JSONClass.load  Procedure(QUEUE pQueue)" is modified: this does the actual preprocessing and passes the local queue to the .FillStructure method


There is also another FillStructure method that's called with a queue as a parameter (second occurence), that I haven't touched (as I don't effectively use it), but that might also benefit from this!


Hope you have time to implement (at least some of it). Let me know if I can be of any help. I'm willing to invest some time in it too!

regards,
Ton

19
Hi Bruce,

I - think - I've spotted a tricky error in the parsing of the json.

Whenever a literal (null, true, false) or numeric gets processed as the last element of an array IN HUMAN READABLE form there is a linebreak (<13,10>) right after this value (before the close "}" of the record structure is processed added to these literals or numbers.

Below I've pasted an example that shows this misbehaviour. the array "orderLines" contains three elements and all "ending" fields ("qAcceleratedType", "label1", "label1") have values that are directly followed by a linebreak. When you modify the "HandleChar" method of the JSONCLASS to include the trace like below you see what's going on quite easily:

JSONClass.HandleChar              PROCEDURE(STRING pChar)
  CODE
  if SELF.Stack.StringStarted OR SELF.Stack.NumericStarted OR SELF.Stack.BooleanStarted OR SELF.Stack.NullStarted
    IF not SELF.Stack.Buffer &= NULL
      SELF.Stack.Buffer.Append(pChar)
      self.trace('pChar : ' & pChar & ' val(pChar) : ' & val(pChar))
    end
  end

I think the solution is maybe to add the following to the loadstring method to only add linebreak when within string values:

    of '<10>' orof '<13>'
      if SELF.Stack.StringStarted
        SELF.HandleChar(pToParse[c])
      end!if


This is the json

{
   "orders_response": {
      "apiVersion": "1.0",
      "orderLines": [
         {
            "fkSetLineId": 12056305,
            "qAcceleratedType": 30
         }
         {
            "fkSetLineId": 12056307,
            "qAcceleratedType": 30,
            "label1": true
         }
         {
            "fkSetLineId": 12056307,
            "qAcceleratedType": 30,
            "label1": null
         }
      ]
   }
}

20
Hi all,

(maybe this is more xFiles/jFiles related question ...)

whenever I import xml or json into a queue I nicely end up with all records in the Queue, but is there a way for me to detect wether an element was omitted or "null" when it imported into the queue?

a party is sending me an array of "objects" (records) I have to update, but it makes quite a difference wether some object properties are omitted/null or not (those fields need not be updated). I now end up with zeroes (for numeric fields) and empty strings, but I can't write these if the json didn't specify these fields ...

json is my top priority, but handling this via xml would be nice too

TIA,
Ton

21
Hi all,

I want to add a css file (or is there a better way) to the generated help pages for the webservice methods. I modified the prompts of the "global" netwebserver procedure to add a script and the generated code shows the addition (file is "broek.css"):

  IF(true)
    s_web._SitesQueue.Defaults.HtmlCommonStyles = |
      s_web.AddStyle(clip('shoestrap3') & '/theme.css',true) &|
      s_web.AddStyle('broek.css') &|
 ...

but the actual served page doesn't contain this (not even after F5 :-))

what am I doing wrong?

TIA,
Ton


22
Hi all,

so far I've created only methods with a couple of single variables as incoming parameters. Now I want to add the possibility to handle both these singular variables and some compound json (to be stored in a queue). I've added a queue to the incoming parameters and studied the genereted help file from the service, but apart from the soap structures I don't see the queue occuring in this help ...

with only a list of variables one can specify these as parameters in the url (&somevar=somevalue), but I guess the compound json should be put in the request body directly (with corresponding headers set to the right "content-type" so server knows how to handle), but where do my singular variable go now?

Singular var = "readable"
Compound json should go into the "SomeQueue"

This is part of the example request for the soap handling:

POST /wsSupply HTTP/1.1
Host: localhost:8088
Content-Type: application/soap+xml
Content-Length: length

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
  <soap:Body>
    <sets xmlns="">
      <readable>value</readable>
      <SomeQueue_list>
        <SomeQueue>
          <SQ_TEST>value</SQ_TEST>
        </SomeQueue>
        <SomeQueue>
          <SQ_TEST>value</SQ_TEST>
        </SomeQueue>
      </SomeQueue_list>
    </sets>
  </soap:Body>
</soap:Envelope>

And this is the HTTP POST for JSON:

POST /sets HTTP/1.1
Host: localhost:8088
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Content-Length: length

readable=value

But here there is nothing generated with regards to the "SomeQueue" structure ...

How should one compose such a (json) request?

TIA
Ton

23
Web Server - Ask For Help / parsing JSON with jFiles
« on: February 24, 2016, 05:38:55 AM »
Hi all,

I've the following json string :

{"FTP" : {"srv" : "test.test.nl","prt" : "","usr" : "username","pwd" : "wachtwoord","pas" : 1}}

which I want to read into the following group structure

FTPGroup                GROUP,PRE(FTP)
srv                       STRING(250)
prt                       STRING(20)
usr                       STRING(250)
pwd                       STRING(250)
pas                       BYTE
                        END


with the following code :

        lSt_JSON.SetValue(CLIP(EMESS:sendto))
        lJSON_FTP.TagCase = jF:CaseLower
        lJSON_FTP.Load(FTPGroup, lSt_JSON, 'FTP')


but the parser somehow treats the "test.test.nl" value as a numeric as it comes up with the following error :

ErrorTrap: "Unexpected literal (.) at position 24 v" : "test.test.nl"

this is the first time I'm using jFiles as a parser, so I might be doing something wrong of course, but I haven't got a clue !

24
Web Server - Ask For Help / POST and parameters
« on: December 19, 2015, 03:16:30 AM »
Hi all,

I can't get parameters working with a webservice via a POST. I'm posting three parameters (see below the request that arrives serverside): "tabbed=1&traditional=0&readable=1"

In the primefields routine I expect the parameters to be accesible. In this routine the _postdata_ is filled with this string, but I can only get the FIRST parameter with the p_Web.GetValue('') command ....

I've dived into the netweb.clw file a little and it looks like something goes wrong in the NetwebserverWorker._ParseSettings method: I expect the line to be split on the "&" characters but it only sees one parameter (the part after the first "=", goes into the value of the first parameter). But I'm clueless to what is causing this

Can somebody tell me what I'm doing wrong here or what to check for?

TIA,
Ton


This is the request (incoming at the server)

POST /wsSupply/setLines HTTP/1.0
Accept: application/json
Accept-Language: en
Accept-Encoding:  gzip
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16 ( .NET CLR 3.5.30729; .NET4.0C)
Host: localhost:8088
Content-Length: 33
Connection: Keep-Alive

tabbed=1&traditional=0&readable=1

25
Hi (Bruce),

I think there is an issue with a webmethods that proceduce (large sets of) json output on several threads concurrently.

I have a method that produces a rather large resultset (10,000+ items in a Queue). When I serve this data as JSON and activate this method multiple times (more or less concurrently) the server starts launching new threads (as it should), but then the server starts to crawl.

I've reproduced this by adding a new "method" to  the 77 example app that simply loads data from an xml file to a queue and than simply serves that queue as JSON. Serving a single set takes roughly 20 seconds, asking for more sets while the first is still being processed, gets the webserver on its knees (I think memory consumption is also too high ... and sometimes it crashes (and gpf reporter doesn't kick in) ...

Simply letting the method serve xml instead of JSON (very same Queue with the same data, but now only served as xml) and the problem doesn't appear!!! Asking for three sets "concurrently" takes a little over a minute and nicely ends in all threads being closed and server ready for other requests.

Now if can have that for json too, I'll be more than happy <g>

using NT 8.65 and jFiles 1.13/1.14

BTW, just starting with webservices, but really love the nettalk product very much, there's been some great work done in there!

26
Hi,

I think the field case settings for (at least) the webservice documentation is ignored. I'm using Queues as return type and have external names on the fields and set the "Fieldname Case" to "AsIs", but the documentation for the JSON output shows up as uppercase. (XML output shows mixed case, so that's fine). Using Nettalk 8.65 and jFiles 1.13

first a snippet for the generated XML (correct):
  <category>
        <id>value</id>
        <label>value</label>
      </category>

and this is for the JSON:

"category" : [
      {
        "ID" : value,
        "LABEL" : "value"
      }
    ]

Don't know what the actual output is like (need to test this ...)

Pages: 1 [2]