19 April, 2013

Overmodulated JT65 on HF?

Sometimes it is crowded on JT65 on HF due to too little bandwidth. When only 2 kHz is available and each signal needs 175 Hz that's understandable. But then others seem to complain that some overmodulate their transmitters so that they occupy more than the 175 Hz, making it even harder to fit an extra signal in the band.

As I have been running a lot of JT65 lately on HF, I also have seen this phenomenon and it piqued my interest to try to understand what was going on. The image below shows such a strong station to the very left, at about -1000 Hz where the red marker is located. After some seconds I turned on the attenuator of my K3, so the signal was attenuated by 10 dB (press image for zoom).

What one can see is that what appears initially (at the bottom of the waterfall) as a splattering signal, becomes quite fine when the attenuator is turned on. Then it spills into neighboring frequencies again as the attenuator is turned off again.

It appears then that it is the JT65 decoder software which is too sensitive to strong signals. Now, I cannot really say that I understand all of the decoder code, but I think that it has to do with the way the power spectrum is estimated. The FORTRAN code for ps.f is listed below. It comes from the BerliOS repository for WSJT which has the same code for this routine as JT65-HF-Comfort:

subroutine ps(dat,nfft,s)

      parameter (NMAX=16384+2)
      parameter (NHMAX=NMAX/2-1)
      real dat(nfft)
      real s(NHMAX)
      real x(NMAX)
      complex c(0:NHMAX)
      equivalence (x,c)

      do i=1,nfft
         x(i)=dat(i)/128.0       !### Why 128 ??

      call xfft(x,nfft)
      do i=1,nh
         s(i)=fac*(real(c(i))**2 + aimag(c(i))**2)


What is apparent here is that the raw data, dat, is just put directly into the Fast Fourier Transform routine, xfft, after scaling. There is no windowing function. A window function tapers down the beginning and end of the data set. Window functions in spectral analysis are somewhat involved and I refer to the Wikipedia article for details. But when there is no window function (= rectangular window), the first sidelobe is only 13 dB down from the mainlobe. So that could be why a 10 dB attenuator is enough to remove most of the spillover into adjacent frequencies above. 

This could be remedied by using a smooth window function. There are many to chose from, but let's take a Hamming window as an example, with its first sidelobe 43 dB down. This means that if the data had been multiplied by this taper, the dynamic range would have been in the order of 30 dB higher. But as data is lost by the tapering, the downside is that the bin width increases. For this particular window the noise bandwidth goes up by a factor of 1.36 so sensitivity would suffer by 10log(1.36) or about 1.3 dB (ref: Harris, "On the use of Windows for Harmonic Analysis with the Discrete Fourier Transform," Proc. IEEE, 1978)There could potentially be other negative side effects on decoding also which I cannot foresee from the limited  time I have used in trying to understand the algorithms. 

My first impression from using the new JT9 mode is that the problem is much smaller there than for JT65, so maybe something like what I am discussing here has been done in the decoder software. But as far as I know, the source code has not been releasted into the public domain yet by K1JT, so I cannot verify it now.

But it seems clear to me that what looks like splatter has much less to do with overdriving and overmodulating transmitters than one may think, and more to do with the particular way that the spectral estimate is found in the JT65 decoder software. Combined with the variable propagation which is an intrinsic feature of HF and which may create a highly variable signal strength, this is what seems to create the spillover.


  1. That clears up something that has bothered me on JT65. Many times someone will complain about someone overdriving, but the signal appeared to be OK to me. It happens frequently. Now I think the "complainer" is probably receiving the station much more strongly than I am and it is overdriving his JT65 software. The signal is weaker for me and my software is not being overdriven. The transmitted signal is indeed clean.

    Thanks for the insight.


    1. Hi Eric. Thanks for your comment. I have learnt that JT65 was designed as a weak signal VHF mode. Now that it is used on HF as well, it looks as if the large variation in signal level is far larger than it was designed for, hence the problems with overdrive. But this would of course be interesting to get a confirmation for from someone who knows the software very well.

  2. The code doing the FFT display and the actual JT65 code may use completely separate codebases. What may seem as an overdrive in the FFT display may not actually be overdriving the JT65 demod. If however the JT65 decoded data has more errors in it when the signal is stronger (no attenuator) then there may be overload problems in the demod as well. Your point about the lack of windowing function may be part of the answer (to the display problem).

    It appears that the Fortran code: x(i)=dat(i)/128.0 is scaling the data down to give more headroom.

    The REAL datatype is often 4 bytes long in Fortran 77 and it is able to take values from 1.175494E-38 to 3.402823E+38 with 7 digits of precision. Sign bit makes the range double.


    1. What I am talking about is not overdrive, i.e. going over the allowable range of the data variable, but spectral leakage. Such leakage is seen in energy that spills over into neighboring frequencies, while overdrive is primarily seen as harmonic and intermodulation distortion. Leakage will occur despite the data being inside the word format.

      To get around it, one needs an FFT code with multiplication with a smooth window function, like the Hamming window, i.e. something like this:

      do i=1,nfft
      x(i)=hamming(i) * dat(i)/128.0
      call xfft(x,nfft)

      The window function has its maximum at the center, i.e. around samples 64-65 and its minimum value at the ends, i.e indices 1 and 64, and it is symmetric.