The basic principal to follow is to keep your equipment within its linear range of operation.
I found that the linear range of operation of my sound card was essentially the middle half of its dynamic range. When signals with sample values in the upper quarter, or lowest quarter of my sound card's dynamic range were generated by my sound card, they showed evidence of non-linear operation. Since then, I have scaled signals to only use the middle half of my sound card's dynamic range.
If the line input, or microphone input, to your sound card is connected to an audio source, then that audio will be present in the audio output of your sound card, unless those unwanted sources are muted while your sound card is supplying audio input to your transmitter.
The audio level going into the radio has been the most difficult setting to make on the transmit side. A two tone test signal was developed to aid this process. It is the file named "2TONE-12.WAV".
The two tone test signal consists of the sum of an 1180 Hz sinusoid and a 1520 Hz sinusoid. Note that these frequencies straddle the 1200 Hz and 1500 Hz lines on the MMSSTV spectrum display. During the first 2 seconds of the two tone signal, a constant amplitude is maintained. At the 2 second point in the signal, the amplitude steps down, such that the power drops by 3 db. This level is maintained until the 4 second point in the signal is reached. At the 4 second point, the amplitude drops again, so that the power drops by another 3 db. This level is maintained until the 6 second point in the signal. At the 6 second point, the amplitude steps back to its original level, resulting in a 6 db increase in power. The last 6 seconds of the two tone signal are a repetition of the first 6 seconds.
For the case of a SSB transmitter, if you can monitor the output power of your transmissions, then you can set the level properly by adjusting it until you see the -3db, -3db, +6db, -3db, -3db sequence in your output power while transmitting the two tone test signal. This is the most reliable way to set the level for a SSB transmitter.
If you can't monitor your output power, then a receiver can be used to monitor your transmissions. However, if the receiver is not adjusted properly, it can be the source of non-linearity, rather than the transmitter.
When the person monitoring your transmission of the two tone signal can hear the -3db, -3db, +6db, -3db, -3db pattern, then you know that both the transmitter and receiver are adjusted properly. An alternative to listening to the audio of the two tone signal is to monitor it with a spectrum display program. The MMSSTV program has been used for this purpose. When the levels are set up properly on both the transmitter and the receiver, there will be only two frequencies present in the spectrum display of the received two tone signal. When the level is too large, at either the transmitter, or the receiver, or both, then the spectrum display will show energy present at more than just two frequencies. The spurious energy is generated by non-linear operation.
Note that the non-linearity may be in the sound card at the receiver. It is possible for the two tone signal coming out of the receiver to contain energy at only two frequencies, but the spectrum display shows energy at more than two frequencies. The cure for this situation is to reduce the level going into the sound card. Unfortunately, evidence of non-linear operation on a spectrum display does not tell you where, in the chain of equipment the non-linearity is happening. It could be in:
Once you get everything into the linear range of operation, you can adjust one thing at a time to determine what it takes to begin non-linear operation of the various pieces of equipment in the chain.
One series of tests demonstrated that one transmitter operating in FM mode generated many more spurious emissions when over driven, than it did when operating in SSB mode and was over driven. Thus, you need to be much more careful in setting the audio level going into the radio when operating in FM mode than you do when operating in SSB mode.
Below is the spectrum of part of the leader section of a transmission from Australia to the United States. The energy of this signal should be concentrated at 12 different frequencies, with no significant energy being present at any other frequencies. This is the case for the spectrum below, which shows there was no significant non-linear operation of any equipment in the chain.
The spectrum above shows that the frequency response of this channel was not very flat, during the leader portion of the signal. The two highest frequency subcarriers (nominally at 1955 Hz and 2185 Hz) are unmodulated during the leader, and start out at the same amplitude. The spectrum above shows that they differ in amplitude by about 10 db, with the one at 1955 Hz being weaker, at this receiving location.
Below is the spectrum of part of the leader section of the same transmission from Australia to the United States. The receiver was different for the spectrum below and the spectrum above. The transmitter was the same for these two cases. Looking at the plot below, you can see three spikes above 2200 Hz generated by non-linear operation by equipment at this receiving location.
The spectrum just above shows that the frequency response of this channel was also not very flat, during the leader portion of the signal. The nominally 1955 Hz and 2185 Hz subcarriers are reversed in relative strength at this receiving location, with respect to their relative strength at the first receiving location. These two receiving locations are about 80 air miles apart, and about 9500 air miles from the transmitter (in Australia).
Since the subcarriers are 230 Hz apart, some spurious energy may show up at integer multiples of 230 Hz, due to intermodulation distortion, when there is non-linear operation. Also, non-linear operation can create harmonics of frequencies actually present. Harmonics of the lower frequencies may fall in the desired passband of the signal. Harmonics of the higher frequencies may be aliased, due to the sampling process, so that their aliases fall in the desired passband of the signal.
Even though equipment at the second receiving location was operated in the non-linear mode, it was not severe enough to prevent all of the errors from being corrected.