[pskmail] Fwd: Some confusion about Stream IDs

  • From: William <va3udp@xxxxxx>
  • To: pskmail@xxxxxxxxxxxxx
  • Date: Wed, 12 Nov 2008 09:53:55 +0100

The response from the author of the ARQ2 spec. The bottom line seems
to be that pskmail is doing things differently, so ARQ2.pdf is not
a good reference to know what pskmail is doing. The difference is
enough that coding from ARQ2.pdf would give something incompatible
with pskmail.

I don't know why pskmail treats stream IDs differently from what the
spec says, as this doesn't seem like filling in gaps but rather taking
away things. I would be rather interested to know this.

In any case the only accurate documentation we have is some confusing
(to me) perl code. Let's not waste too much time on the current version
that will be replaced anyways, but can we work on documenting the
new DAMA protocol on the wiki?

As I figure out what the current pskmail is doing I will also try to
document it.


Début du message réexpédié :

De : "Paul L Schmidt, K9PS"
Date : 11 novembre 2008 17:03:03 GMT+01:00
À : William <va3udp@xxxxxx>
Cc : pa0r <rein@xxxxxxxxxxxx>
Objet : Rép : Some confusion about Stream IDs

William wrote:
Hi Paul,
I'm working on a python implementation of ARQ2 with the intention
of interoperating with the PSKMail network. This has led to some
discussion on the PSKMail mailing list [1] about the correct use
of stream IDs. I've copied Rein PA0R the developer of PSKMail on
this message.

Sounds like a fun project - although for interconnection with the
PSKMail network, the authoritative answers would come from Rein.

A few years ago, I put together a couple of drafts for a spec for
sound card ARQ, intended to be discussion starters and hopefully the
basis for a sound card based mode to pretty much take the place
of Amtor, Pactor, and Packet modes on HF.  Unfortunately, there wasn't
a lot of discussion resulting.

At the time I put the drafts together, I was finishing up my Masters'
Degree, and thought that (how naive!) after finishing that, I would
actually have some more time to work on fun stuff.  I should have
known better - being in my mid-40's, with wife, 4 kids, church, and
other responsibilities - that as soon as I finished the degree, everybody
who'd been put off for the last couple of years (due to working on
the degree) would be lined up for a chunk of my newly freed-up time.
They were.

Rein was about the only person at that time who took much of an interest
in the draft.  The world was anxiously awaiting the release of SCAMP,
so the rest of the interest in reliable sound card modems was focused
on that.

Rein's PSKMail implemented the draft protocol, and filled in a lot of
the gaps -- timing issues, etc. -- that the draft hadn't defined.

As time went on, W1HKJ also implemented the spec with his FLARQ program, again filling in some of the gaps. Since these were independent efforts,
the gaps were likely filled in differently.

I wonder if you could comment on my understanding of how a connection
is supposed to work. An example session,
1 --> <soh>00cVA3UDP:1023 PA0R:23 1 6EDC0<eot>
2 <-- <soh>01kPA0R:23 VA3UDP:1023 5 66CD0<eot>
3 --> <soh>05 some data4E04<soh>05!some more data7182<eot>
4 <-- <soh>01s !FB66<eot>
5 --> <soh>05d"D7B5<eot>
6 <-- <soh>01s "FA26<eot>
Particularly at issue is the choice of sequence numbers. As I read
the spec, I think that the client initiating the connection specifies
the streamnumber that the server accepting the connection is to use
for transmitting (1 in this case) and the server specifies the stream
number that the client should use for transmitting (5 in this case).
An exception is the connection request that is sent to stream 0 because
no number can be yet assigned. Is this correct?

When you say sequence numbers here, I'm thinking you may be referring to the stream numbers. Stream numbers are simply a short-hand notation for part of the header block. The idea here was that it would not be good to limit the
communications between stations to a single stream of data, so more
advanced protocols could be built on top of this layer. In the example
above, the link emulates a TCP/IP telnet connection, with VA3UDP
originating a connection from port 1023 to the telnet port (23) of PA0R. Rather than negotiating a mutually-agreeable designation for the shorthand, each station just tells the other what to call the link. For each packet, the transmitting station already knows what stream the packet came from; the short-hand just lets the receiving station know what to do with it. This makes protocols like FTP possible, without having to put full headers
on each packet.

Designation of the stream number then becomes entirely the choice of the receiving station, is set to whatever value will provide the receiver with a unique number for look-up into the "real" connection information (e.g.
the VA3UDP:1023->PA0R:23 link)

In the above example:

Line 1: VA3UDP wants to (c)onnect his port 1023 to port 23 at PA0R. The ID for the stream at VA3UDP's end will be 1, and there will be 64 bytes
per packet.  The stream ID for this particular packet is 0, since PA0R
doesn't have a stream set up for it yet.

Line 2: PA0R ac(k)nowledges the request for his port 23 to be connected
to VA3UDP's port 1023, and packets in the VA3UDP->PA0R direction will
be referred to as stream 5.

Line 3: VA3UDP is sending some data to PA0R's stream 5 (which from Line 2, we know is data from VA3UDP port 1023 to PA0R port 23, with the stream
5 designation being shorthand notation for VA3UDP:1023->PA0R:23).  The
space indicates this is the first data packet.
The second packet in line 3, with the sequence ID of "!" is more data for
that stream, with sequence 1.

What PSKMail does is use the second byte from line 2, the server's
idea of stream ID for communication in both directions and does not
include the stream ID in the payload of connection requests and
acknowledgement, that it it would send just,
7 --> <soh>00cVA3UDP:1023 PA0R:23 6D962<eot>
8 <-- <soh>01kPA0R:23 VA3UDP:1023 6265E<eot>
9 --> <soh>01 some data7E11<eot>
   ... etc...

It appears that the stream ID's are omitted from the connection request
and acknowledgment lines, and stream 1 is assumed for both directions.

To me it seems like this would work in a case where one client always
talks to one server at a time and there is never any other traffic on
the channel, and it seems to be different behaviour from the

Other traffic on the channel shouldn't bother things as long as the
station ID packets are used at the beginning of each group of packets
(see the top of page 2 of the ARQ spec version 0.2).

Incidentally It would seem a good idea for both ends to chose stream
IDs randomly rather than sequentially to minimize collision in
environments where there may be several connections in progress on
the same channel (time division multiplexing with an arbitrating
service to assign timeslots which I gather Rein is working on).

The idea in the spec was that the ID packets would allow the receiving
station to sort between packets sent to him and packets for another
station. Randomly selected stream numbers would provide some additional
disambiguation, but shouldn't be a requirement.  Particularly in cases
where only one station in a group can be heard, there is no way to
know for sure that any stream number you select is unique.  Use of the
ID packet assures that the combination of station ID and stream number
will be unique.

Also, as I was making the above example sessions, I did come across an edge case unrelated to stream numbers -- the first block on lines 4 and
6. At those points the server acknowledges data from the client but
hasn't yet sent any data itself. So the 0x20 in those blocks is wrong, it should actually cause the client to start requesting the nonexistent
block 0x20. A  workaround is perhaps to always send some data at the
beginning of the session.

The "connection request" and "connection acknowledged" are by definition
packet number zero (the "space" character). The actual error is in the
first data packet being sequenced as zero - it should be sequenced as
#1.  That's not to say that the spec is the best way to do it -- this
is exactly where group development of the spec would have really paid off.

Unfortunately, as mentioned earlier, most of the eyes were on SCAMP when this spec was put up for comment, and it received very little review/ comment.

Would it be ok to post any of your reply to the mailing list?

Sure.  Caveat: It's been almost 4 years since I put the paper up for
comment. If this were 20 years ago, I might remember what I did 4 years before then a bit better than I remember 4 years ago now. The spec was incomplete in December of 2004, and there's no guarantee that I am correctly
remembering what I was thinking then.

Many thanks for any light that you can shed on this matter.
73 de William EA5/VA3UDP
va3udp /@/ rac.ca
[1] //www.freelists.org/archives/pskmail/11-2008/msg00046.html

73 de William EA5/VA3UDP
va3udp /@/ rac.ca

Other related posts:

  • » [pskmail] Fwd: Some confusion about Stream IDs - William