Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
Pages: [1]
1
The Rest - Ask For Help / Re: NetSimple Client sending multiple packets on a thread
« on: September 01, 2011, 04:18:07 AM »
MultiClient won't help in this one case as the client target is always the same for this thread - same IP and same port. Muklticlient isn't that needed because we don't internally do very much processing just mostly a "trucking company" moving data as freight. This one instance is a VLAN connection with both a server and a client. Others may be HTTPS or just TCPIP. This client sends requests and our sever on another thread gets the responses because the service can't be done in real time. It takes people to do a service and send the results. They do send status messages that are tracked with a unique ID.
So based on what you said I have a threaded queue of record numbers I want to send. This queue is populated via NOTIFY. A Timer will call the routine for making the next send request and when called will set a block flag so it won't be called again until the open is connected AND the send has been sent. The code after the send will clear the flag and the next EVENT:Timer will stage the next one. The open will perhaps have to wait if it is done sending the last one but the blocking flag will be set at the routine start so that any NOTIFY events will be added to the queue for later processing. Sooner or later the queue will be empty (i hope).
If a send fails I don't need to worry as the server thread won't get a return message. That thread will Set the sent bit in the data record on disk so we know it was sent and received. We still don't have the service results yet as they come back on the server thread when they get around to it and that might be a day or two.
I'm doing a lot of this type of tossing a package over the fence and sending it some place. Internally we send things inside our own network that may be located in more than one data center and with those transmissions we get a response with every send. We hope to have a half dozen providers so we can bundle these services as one service.
So based on what you said I have a threaded queue of record numbers I want to send. This queue is populated via NOTIFY. A Timer will call the routine for making the next send request and when called will set a block flag so it won't be called again until the open is connected AND the send has been sent. The code after the send will clear the flag and the next EVENT:Timer will stage the next one. The open will perhaps have to wait if it is done sending the last one but the blocking flag will be set at the routine start so that any NOTIFY events will be added to the queue for later processing. Sooner or later the queue will be empty (i hope).
If a send fails I don't need to worry as the server thread won't get a return message. That thread will Set the sent bit in the data record on disk so we know it was sent and received. We still don't have the service results yet as they come back on the server thread when they get around to it and that might be a day or two.
I'm doing a lot of this type of tossing a package over the fence and sending it some place. Internally we send things inside our own network that may be located in more than one data center and with those transmissions we get a response with every send. We hope to have a half dozen providers so we can bundle these services as one service.
2
The Rest - Ask For Help / NetSimple Client sending multiple packets on a thread
« on: August 31, 2011, 03:50:38 PM »
I have a NetSimple client on a thread that gets a Notify to send a SysyID of a data record. Once I have done the Send how can I know when I can open a new connection and send another record.
It comes down to will a Notify come in during the period after I open the previous connection but before the send. I can set a blocking flag between the open method and the send method and buffer the requests to a queue. So I am fine between the open and send. Can I open a new connection right after I do the Send or do I need to watch for something else on a timer to know when the next open can be called.
During this period between open and send will Notify events be able to post? I assume while waiting for the connection to open they can but once I trap the open connection and start the send it can't. This leaves the post send period in question.
Since the IP and Port are the same it seems launching a new thread for each transmission won't solve the problem but would stack up the port opens. Would that be true?
It comes down to will a Notify come in during the period after I open the previous connection but before the send. I can set a blocking flag between the open method and the send method and buffer the requests to a queue. So I am fine between the open and send. Can I open a new connection right after I do the Send or do I need to watch for something else on a timer to know when the next open can be called.
During this period between open and send will Notify events be able to post? I assume while waiting for the connection to open they can but once I trap the open connection and start the send it can't. This leaves the post send period in question.
Since the IP and Port are the same it seems launching a new thread for each transmission won't solve the problem but would stack up the port opens. Would that be true?
3
The Rest - Share Knowledge / Re: Getting length from a non Clarion NetSimple packet
« on: August 30, 2011, 04:07:08 AM »
Sounds exactly as required Bruce. Thanks for the feature!
4
The Rest - Ask For Help / Re: NetSimple responsding to Telnet
« on: August 23, 2011, 07:12:47 PM »
This is a service provider connection to a client that connects to the NetTalk Server to send back replies that get made. They require a telnet test that replies "OK" We made it work but it was a tad convoluted.
The Server is a NetTalk NetSimple not using WholePacket (see shared knowledge post from today on BIg Endian length). What we do is detect the open connection and just send an OK, then go about the process of catching the packet data and waiting for the final byte. We have assumed that this connection to send is "stuff" will open the connection blast it at us and close the connection without waiting for a thank you very much. We have to send them a thank youi note with our own NetSimple client in a wrapper formatted "yes we got it" so they can bill us. We don't open the package we get just the wrapper.Based on that we send it back then proceed along our merry way with the package we want to process.
This Telnet requirement is over a VPN so the security is not that big an issue. I can only assume they like to do a telnet to make sure the connection is alive from time to time. Should the connection fail as they push us data I suppose some Network technician tries a telnet to "poke us with a stick". I think they also want ping too. If it wasn't as VPN it would be suspect but then again they do a whole lot of this stuff.
Fortunately using the Big Endian length header works and they always send data in a documented XML wrapper that contains a nationally standard package of data used in the pharmacy industry. Yes, the clients are drug dealers - the real ones.
The Server is a NetTalk NetSimple not using WholePacket (see shared knowledge post from today on BIg Endian length). What we do is detect the open connection and just send an OK, then go about the process of catching the packet data and waiting for the final byte. We have assumed that this connection to send is "stuff" will open the connection blast it at us and close the connection without waiting for a thank you very much. We have to send them a thank youi note with our own NetSimple client in a wrapper formatted "yes we got it" so they can bill us. We don't open the package we get just the wrapper.Based on that we send it back then proceed along our merry way with the package we want to process.
This Telnet requirement is over a VPN so the security is not that big an issue. I can only assume they like to do a telnet to make sure the connection is alive from time to time. Should the connection fail as they push us data I suppose some Network technician tries a telnet to "poke us with a stick". I think they also want ping too. If it wasn't as VPN it would be suspect but then again they do a whole lot of this stuff.
Fortunately using the Big Endian length header works and they always send data in a documented XML wrapper that contains a nationally standard package of data used in the pharmacy industry. Yes, the clients are drug dealers - the real ones.
5
The Rest - Share Knowledge / Getting length from a non Clarion NetSimple packet
« on: August 23, 2011, 06:53:58 PM »
The WholePacket NetSimple Methodis great for NetTalk to NetTalk objects since it sends the length as a 4 byte header and can assemble the packets for you saving some code. (See the manual boundary example).
There is a standard way that many non NetTalk TCP applications use. They send the length as a 4 bytes header but it is no Clarion LONG you can distinguish.
The Bytes are sent as a Big Endian ULONG Integer. This is great until you find out Clarion is not Big Endian So we pretend it's a STRING instead and it's all better. You just need a little funny math.
Should Capesoft add an option to WholePacket to use Big Endian longs they could make WholePacket work with other players as well as Clarion and it wouldn't break anyones code since we compute length as a Clarion LONG, but NetTalk could use Big Endian as an option and we wouldn't care since it is doing all the work.
The following functions will convert properly (we tested it today with a an Online service)
ToBigEndian PROCEDURE (LONG RawVal),STRING
ByteArray BYTE,DIM(4)
OutputVal STRING(4),OVER(ByteArray)
CODE
ByteArray[1] = INT(RawVal/256^3)
ByteArray[2] = INT((RawVal % 256^3) / 256^2)
ByteArray[3] = INT((RawVal % 256^2) / 256)
ByteArray[4] = INT(RawVal % 256)
RETURN OutputVal
And to go the other way:
FromBEndian PROCEDURE (STRING RawVal), LONG
ByteArray BYTE,DIM(4)
InputVal STRING(4),OVER(ByteArray)
Result LONG
CODE
InputVal = RawVal
Result = (INT(ByteArray[1]) * 256^3 + |
INT(ByteArray[2]) * 256^2 + |
INT(ByteArray[3]) * 256 + |
INT(ByteArray[4])) ! The packet includes the 4 byte header
RETURN Result
Special thanks to Marc Walgren of MItten Software for being the tester that would have asked me to fix it if it didn't work..
There is a standard way that many non NetTalk TCP applications use. They send the length as a 4 bytes header but it is no Clarion LONG you can distinguish.
The Bytes are sent as a Big Endian ULONG Integer. This is great until you find out Clarion is not Big Endian So we pretend it's a STRING instead and it's all better. You just need a little funny math.
Should Capesoft add an option to WholePacket to use Big Endian longs they could make WholePacket work with other players as well as Clarion and it wouldn't break anyones code since we compute length as a Clarion LONG, but NetTalk could use Big Endian as an option and we wouldn't care since it is doing all the work.
The following functions will convert properly (we tested it today with a an Online service)
ToBigEndian PROCEDURE (LONG RawVal),STRING
ByteArray BYTE,DIM(4)
OutputVal STRING(4),OVER(ByteArray)
CODE
ByteArray[1] = INT(RawVal/256^3)
ByteArray[2] = INT((RawVal % 256^3) / 256^2)
ByteArray[3] = INT((RawVal % 256^2) / 256)
ByteArray[4] = INT(RawVal % 256)
RETURN OutputVal
And to go the other way:
FromBEndian PROCEDURE (STRING RawVal), LONG
ByteArray BYTE,DIM(4)
InputVal STRING(4),OVER(ByteArray)
Result LONG
CODE
InputVal = RawVal
Result = (INT(ByteArray[1]) * 256^3 + |
INT(ByteArray[2]) * 256^2 + |
INT(ByteArray[3]) * 256 + |
INT(ByteArray[4])) ! The packet includes the 4 byte header
RETURN Result
Special thanks to Marc Walgren of MItten Software for being the tester that would have asked me to fix it if it didn't work..
6
The Rest - Ask For Help / NetSimple responsding to Telnet
« on: August 18, 2011, 06:10:57 PM »
We are connecting to a service provider via NetSimple Server / client. We will submit requests and they will respond (not real time) via a NetSimple Server. They want to be able to do a test with Telnet. This is a VPN connection so it's secure and we can restrict access to the server. The question is How can we respond to a telnet connection using NetTalk? Ideally it's the only thing this NetTalk server thread would do.
7
The Rest - Ask For Help / Re: NetSimple Server
« on: August 16, 2011, 02:52:59 AM »
I actually got a little lucky (it's only a moment in time). I can look for a </Message> that marks the end BUT I also will receive a 4 byte length signed integer in Big Endian format. Does this code to convert to a Clarion LONG look right:
FromBEndian PROCEDURE (STRING RawVal)!, LONG
ByteArray BYTE,DIM(4)
InputVal STRING(4),OVER(ByteArray)
Result LONG
CODE
InputVal = RawVal
Result = (INT(ByteArray[1]) * 256^3 + |
INT(ByteArray[2]) * 256^2 + |
INT(ByteArray[3]) * 256 + |
INT(ByteArray[4])) - 4
RETURN Result
Looking at the docs (late yesterday) this looks like a popular method for passing the length. It might be worth an option to support "big endian" format as a passed length for WholePacket
With the leading length a </Message> closing it should not be that hard.
If the server closes the connection before I give up what event will fire on my end? I know I have to probably do this a few more times with a few more clients and a few servers too. Some may be on a VPN but I think some others may be not. This one will be nice since I don't have to run my own client to poll for responses from prior requests but I figure that is coming too. This connection I just send over the request and then need to have a server wait a good long time for a response. They have to perform a phone call verification with a doctor so it's not very "real time". Mostly it just is matching up my requests with supplied responses and sending them back where they came from. This is basically just an "internet trucking" application delivering and receiving loads.
Are you aware of any standard response that is used from a server. I get the impression these folks don't want one.
FromBEndian PROCEDURE (STRING RawVal)!, LONG
ByteArray BYTE,DIM(4)
InputVal STRING(4),OVER(ByteArray)
Result LONG
CODE
InputVal = RawVal
Result = (INT(ByteArray[1]) * 256^3 + |
INT(ByteArray[2]) * 256^2 + |
INT(ByteArray[3]) * 256 + |
INT(ByteArray[4])) - 4
RETURN Result
Looking at the docs (late yesterday) this looks like a popular method for passing the length. It might be worth an option to support "big endian" format as a passed length for WholePacket
With the leading length a </Message> closing it should not be that hard.
If the server closes the connection before I give up what event will fire on my end? I know I have to probably do this a few more times with a few more clients and a few servers too. Some may be on a VPN but I think some others may be not. This one will be nice since I don't have to run my own client to poll for responses from prior requests but I figure that is coming too. This connection I just send over the request and then need to have a server wait a good long time for a response. They have to perform a phone call verification with a doctor so it's not very "real time". Mostly it just is matching up my requests with supplied responses and sending them back where they came from. This is basically just an "internet trucking" application delivering and receiving loads.
Are you aware of any standard response that is used from a server. I get the impression these folks don't want one.
8
The Rest - Ask For Help / NetSimple Server
« on: August 15, 2011, 10:42:38 AM »
I need a NetSimple server to receive a packet that will be an XML data file that I know how to read. This connection will come through a VPN and of course the one client calling is the only client that can call over the VPN but it won't be a Clarion application that can use NET:SimpleWholeDataPacket.
The XML structure is standard but the contents are variable length but will be well under 16KB more like 2KB. When I receive the partial packet how do I know when the whole packet has been received? Unlike when I use the NET:SimpleWholeDataPacket I won't really know. Given the dedicated link it should all be there very quick but how quick is pretty quick:)
The XML structure is standard but the contents are variable length but will be well under 16KB more like 2KB. When I receive the partial packet how do I know when the whole packet has been received? Unlike when I use the NET:SimpleWholeDataPacket I won't really know. Given the dedicated link it should all be there very quick but how quick is pretty quick:)
Pages: [1]