[pskmail] Re: jPskmail 2: THOR and RSID benchmarking against Fldigi

  • From: "Rein Couperus" <rein@xxxxxxxxxxxx>
  • To: pskmail@xxxxxxxxxxxxx
  • Date: Mon, 5 Dec 2011 09:37:24 +0100 (CET)

Hi John, I have been testing a lot this weekend... unfortunately the central heating 
produced a fault and I had to take care of that fist (it is mod-winter here).

I am testing on air with a K3 with a damaged receiver. It got too much HF during the contest and 
probably one of the switching diodes got roasted. So every server is S0 now, there is no AGC and 
everyone is deep in the noise. Excellent test bed :).

I have problems with THOR11 and THOR8. TX is working fine but RX does not decode properly.
THOR22 works fine, and I am trying to find the cause, I suspect it is the sync or the signal level.
Did you test with the latest jpskmail-2.0.3?

Your test figures look really excellent, better than could be hoped for.

I am looking at the PTT timing, which needs some tuning. Also found some problems with mode switching 
on the server which can be easily corected.

And then of course there is the manual to do...

Rein


Rein,
 
Did some benchmarking between Fldigi and the Java modems in THOR22 and THOR11. I was running both Fldigi and the Java client 2 on the same machine so the data they were receiving from the sound card was the same.
 
Thor 22 decoded down to -12 dB ( one or two errors per 100 characters). Same results on both with a very very slight advantage to Fldigi due to better re-sampling I believe, but really marginal and visible only at -13dB in white noise.
 
Thor 11 decoded down to -15dB with the same error rate as above. Same difference between Fldigi and Java: sometimes the Java modem will decode blocks properly at -16dB and sometimes Fldigi will, so it is really the same from a practical point of view.
 
RSID detection comparing stock Fldigi (original hamming distance = 1 max) and Java modem with hamming distance = 5 max:
 
@ -15dB white noise: Fldigi 1 out of 10 transmissions, Java modem 7 out of 10.
 
@ -14dB white noise: Fldigi  4 out of 10 transmissions, Java modem 9 out of 10.
 
@ -13dB white noise: Fldigi 9 out of 10 transmissions, Java modem 8 out of 10.
 
@ -12dB white noise: Fldigi  8 out of 10 transmissions, Java modem 9 out of 10.
 
@ -11dB white noise: Fldigi 9  out of 10 transmissions, Java modem 9 out of 10.
 
No false positives in either Java or Fldigi (expected in the later one). So with the modifications to the Hamming Distance we are at or better detection level than stock Fldigi.
 
Reviewing the weekend log of the server with a modified Fldigi with hamming distance of 5, I had 5 false positives. So I decided to try with a value of 4 maximum. That is running now.
 
Testing in the same context as above between Fldigi (HD = 4 max) vs the Java modem (HD = 5 max) I get the following results:
 
@ -16dB in white noise Fldigi gets 8 out of 10 and Java modem 7 out of 10. The better re-sampling is visible there. 
 
So in summary for RSID: Both Fldigi and Java modems benefit from the relaxed matching criteria without side-effects so there is real benefit to change both to a max level of 4 (Fldigi)  and 5 (Java modems due to smaller set of RSIDs possible). Exact values to be validated by field trials.
 
73, John

Other related posts:

  1. Home
  2. pskmail
  3. Archive
  4. 12-2011