Overmodulated JT65 on HF?

Sometimes it is crowded on JT65 on HF due to too little bandwidth. When only 2 kHz is available and each signal needs 175 Hz that's understandable. But then others seem to complain that some overmodulate their transmitters so that they occupy more than the 175 Hz, making it even harder to fit an extra signal in the band.

As I have been running a lot of JT65 lately on HF, I also have seen this phenomenon and it piqued my interest to try to understand what was going on. The image below shows such a strong station to the very left, at about -1000 Hz where the red marker is located. After some seconds I turned on the attenuator of my K3, so the signal was attenuated by 10 dB (press image for zoom).

What one can see is that what appears initially (at the bottom of the waterfall) as a splattering signal, becomes quite fine when the attenuator is turned on. Then it spills into neighboring frequencies again as the attenuator is turned off again.

It appears then that it is the JT65 decoder software which is too sensitive to strong signals. Now, I cannot really say that I understand all of the decoder code, but I think that it has to do with the way the power spectrum is estimated. The FORTRAN code for ps.f is listed below. It comes from the BerliOS repository for WSJT which has the same code for this routine as JT65-HF-Comfort:

subroutine ps(dat,nfft,s)

      parameter (NMAX=16384+2)

      parameter (NHMAX=NMAX/2-1)

      real dat(nfft)

      real s(NHMAX)

      real x(NMAX)

      complex c(0:NHMAX)

      equivalence (x,c)

      nh=nfft/2

      do i=1,nfft

         x(i)=dat(i)/128.0       !### Why 128 ??

      enddo

      call xfft(x,nfft)

      fac=1.0/nfft

      do i=1,nh

         s(i)=fac*(real(c(i))**2 + aimag(c(i))**2)

      enddo

      return

      end

What is apparent here is that the raw data, dat, is just put directly into the Fast Fourier Transform routine, xfft, after scaling. There is no windowing function. A window function tapers down the beginning and end of the data set. Window functions in spectral analysis are somewhat involved and I refer to the Wikipedia article for details. But when there is no window function (= rectangular window), the first sidelobe is only 13 dB down from the mainlobe. So that could be why a 10 dB attenuator is enough to remove most of the spillover into adjacent frequencies above. 

This could be remedied by using a smooth window function. There are many to chose from, but let's take a Hamming window as an example, with its first sidelobe 43 dB down. This means that if the data had been multiplied by this taper, the dynamic range would have been in the order of 30 dB higher. But as data is lost by the tapering, the downside is that the bin width increases. For this particular window the noise bandwidth goes up by a factor of 1.36 so sensitivity would suffer by 10log(1.36) or about 1.3 dB (ref: Harris, "On the use of Windows for Harmonic Analysis with the Discrete Fourier Transform," Proc. IEEE, 1978). There could potentially be other negative side effects on decoding also which I cannot foresee from the limited  time I have used in trying to understand the algorithms. 

My first impression from using the new JT9 mode is that the problem is much smaller there than for JT65, so maybe something like what I am discussing here has been done in the decoder software. But as far as I know, the source code has not been releasted into the public domain yet by K1JT, so I cannot verify it now.

But it seems clear to me that what looks like splatter has much less to do with overdriving and overmodulating transmitters than one may think, and more to do with the particular way that the spectral estimate is found in the JT65 decoder software. Combined with the variable propagation which is an intrinsic feature of HF and which may create a highly variable signal strength, this is what seems to create the spillover.