[pskmail] Re: jPskmail 2: THOR and RSID benchmarking against Fldigi

Hi Rein,

 

I can confirm that in my test THOR11 and THOR8 work very well. 

 

I use the noise source in the TXing Fldigi on one PC then feed both an Fldigi and jPskmail2 on the other PC so I can do a side by side comparison.

 

Results are:

 

-15dB for THOR11 both OK in Fldigi and jPskmail. Deviation in Hz before errors appear at -13dB: +/-80 Hz for Fldigi and +/-60Hz for jPskmail modem. I can't explain the difference but the range in jPskmail is quite usable. 

-15dB for both in THOR8. at -16dB the jPskmail modem decodes with less than 1/2 the errors in Fldigi. Deviation before errors at -13dB: Fldigi +/-150Hz, jPskmail +/-80Hz.

 

So I can't see anything wrong with these modems. Sorry.

 

Could it be a timing issue in the sense that it takes longer to synchronize. Is it the start or the end or middle of blocks that don't decode?

 

I tested my server with THOR11 in unproto (several times) and connected modes (once) and it worked well.

 

So I am not sure where the issue is coming from.

 

All the best,

 

73, John

 

 

On Mon, Dec 5, 2011 at 7:37 PM, Rein Couperus <rein@xxxxxxxxxxxx> wrote:

Hi John, I have been testing a lot this weekend... unfortunately the central heating 
produced a fault and I had to take care of that fist (it is mod-winter here).

I am testing on air with a K3 with a damaged receiver. It got too much HF during the contest and 
probably one of the switching diodes got roasted. So every server is S0 now, there is no AGC and 
everyone is deep in the noise. Excellent test bed :).

I have problems with THOR11 and THOR8. TX is working fine but RX does not decode properly.
THOR22 works fine, and I am trying to find the cause, I suspect it is the sync or the signal level.
Did you test with the latest jpskmail-2.0.3?

Your test figures look really excellent, better than could be hoped for.

I am looking at the PTT timing, which needs some tuning. Also found some problems with mode switching 
on the server which can be easily corected.

And then of course there is the manual to do...

Rein

Rein,

 

Did some benchmarking between Fldigi and the Java modems in THOR22 and THOR11. I was running both Fldigi and the Java client 2 on the same machine so the data they were receiving from the sound card was the same.

 

Thor 22 decoded down to -12 dB ( one or two errors per 100 characters). Same results on both with a very very slight advantage to Fldigi due to better re-sampling I believe, but really marginal and visible only at -13dB in white noise.

 

Thor 11 decoded down to -15dB with the same error rate as above. Same difference between Fldigi and Java: sometimes the Java modem will decode blocks properly at -16dB and sometimes Fldigi will, so it is really the same from a practical point of view.

 

RSID detection comparing stock Fldigi (original hamming distance = 1 max) and Java modem with hamming distance = 5 max:

 

@ -15dB white noise: Fldigi 1 out of 10 transmissions, Java modem 7 out of 10.

 

@ -14dB white noise: Fldigi  4 out of 10 transmissions, Java modem 9 out of 10.

 

@ -13dB white noise: Fldigi 9 out of 10 transmissions, Java modem 8 out of 10.

 

@ -12dB white noise: Fldigi  8 out of 10 transmissions, Java modem 9 out of 10.

 

@ -11dB white noise: Fldigi 9  out of 10 transmissions, Java modem 9 out of 10.

 

No false positives in either Java or Fldigi (expected in the later one). So with the modifications to the Hamming Distance we are at or better detection level than stock Fldigi.

 

Reviewing the weekend log of the server with a modified Fldigi with hamming distance of 5, I had 5 false positives. So I decided to try with a value of 4 maximum. That is running now.

 

Testing in the same context as above between Fldigi (HD = 4 max) vs the Java modem (HD = 5 max) I get the following results:

 

@ -16dB in white noise Fldigi gets 8 out of 10 and Java modem 7 out of 10. The better re-sampling is visible there. 

 

So in summary for RSID: Both Fldigi and Java modems benefit from the relaxed matching criteria without side-effects so there is real benefit to change both to a max level of 4 (Fldigi)  and 5 (Java modems due to smaller set of RSIDs possible). Exact values to be validated by field trials.

 

73, John