Signal Optimization Metrics Questions for the Experts

[AA5H]

Some Questions on Signal Optimization Metrics for the System Experts!

This is going to be a long and verbose post, so apologies in advance.

I realize there are dozens of posts on how to optimize; most take a "do this" and "do that" shotgun approach by eliminating common impairments. As a former military signals analyst, I prefer to take a systems approach based on baseline measurements of the circuit's known voltage and noise levels.
 My challenge is determining the correlation between the system's electrical metrics and the optimized signal received by the servers.
From my perspective, these factors impact system optimization:

Home Receiver Unit baseline metrics:

1. Noise floor: Drives the signal-to-noise ratio (SNR).
   
2. Pre-amplifier gain: Typically provides a stronger received signal, but can also reduce the SNR depending on the receiver's ability to reject or filter out-of-band signals.
   
3. Filters: Dampen or suppress out-of-band signals.
   
4. OSI Wireline and Network settings: Duplex mode and packet size (to prevent packet fragmentation in routers with DF/DNF settings).
   
5. A/D serialization Settings: Left at Defaults
   
   

Server-side metrics:

1. Timestamp quality: The accuracy of timestamps on server-received packets compared to the server's PPS Stratum 1 source.
   
2. Packet latency: Measured in jitter.
   
3. Packet distortion: Noise resulting in random or out-of-sequence characters and CRC errors.
   
4. Surplus packets: Packets received by the server error-free, but rejected as surplus in areas with high receiver counts.
   
5. Factor "X": (More on that below).
   
   

Now, to the meat and potatoes: When it comes to system optimization, what server-reported metrics actually determine success?

1. Packet Count: Not accurate; it can include routed packets with broken UDP or higher-layer structures.
   
2. Packets Valid/Involved: Partially accurate, but valid/involved counts may only reflect the integrity of the UDP structure, not the quality of the payload.
   
3. Packets Used: What I consider true success—packets received at the server with an intact payload, timestamps within the system threshold, and a payload needed by the server to complete a calculation.
   
4. Packet "Efficiency": The ratio of "Valid" to "Used" packets (just my guess).
   

And now for "Factor X"...
I've systematically eliminated as many variables as possible from my setup. I use a field strength meter to identify RFI hotspots and mitigate them either by shutting off the source or using ferrite rings and clamps optimized to the source frequency. I've built three separate antenna types, tried various grounding scenarios, and added external RFI suppression to the antenna leads (1mF–2mF poly caps).

After a week of trial and error, my "Packets Used" count rose steadily into the 2,000–4,000 range—placing it in the top tier compared to similar receivers. 

Success! Or so I thought; read on

Unfortunately, my high "Packets Used" count isn't sustainable over time. It regularly declines to the 400–500 range for hours; it sometimes recovers to 1,000–1,500, and occasionally spikes back up to 2,000–4,000. My "Packets Used" count fluctuates wildly without any configuration changes, antenna changes,  environmental changes (including local weather), or additional RFI sources. I live 1 km from any neighbor, switching station, or other external RFI source. These fluctuations occur without any GPS errors, identified interference events, or filtering changes.

My only explanation is "Factor X" on the server side. Is it normal for "Packets Used" to vary wildly throughout the day as atmospheric or server conditions change, or is this shift caused by something on "my end"?

Again, apologies for such a detailed and long-winded post.

Current Setup:

• System Blue-Mini
  
• Three Coils of Times Microwave LMR-600 (2x each, with a 90-degree polarization shift)
  

Thanks

Brian (AA5H)

   
   

[VA_LM_Blue]

Greetings,

Just a few thoughts, I'm sure others will chime in!

Speed reading thru most everything you wrote and then looking at your attached plots at the bottom of your very nice systems description I wonder first why you are using DSP filters? The signal plot shows what appears to be a near delta function spike. The frequency spectrum is flat and full of "signal (noise)" from the delta function spike.

Do you live near any electrified fencing? Do you have another impulse noise source problem?

From a simple point-of-view and thinking about what parts of the system you have control over lead to the fact that you should spend all your time on finding a quiet, 0 to 300-kHz, place on your property to place the antennas. Once you do that there are some basic tweeks you can do in the controller firmware that might be useful to do once you find your quiet place... but even those aren't really required.

Make sure you have a stable internet connection with out local contention from other users on your local net. Once those packets go out of your house there is nothing more you can do really.

Check on your part of the system now and then to be sure it is running, producing useful data, etc.

Hope your system doesn't get hit by lightning, downburst or a tornado.

[cutty]

Out-thinking this, I believe.

If by “Packets” you mean Signals... yep, those numbers can and will change hourly, daily, seasonally, and sometimes dramatically without you changing a thing.

First question is always:

Where’s the lightning?

When looking at any station in near-real-time, the location and amount of lightning matters enormously, as does the population of other stations participating in those strokes. That is one reason the numbers can move around so much even when your receiver itself has not changed.

If by “packets” you literally mean UDP data traveling from the controller to the server, that is a different issue. Yes, packets can be lost, although normally that is relatively rare. Delay getting there is normal; delay large enough to materially affect things is much less common.

We recently had a very useful example because we happened to have two co-located Blitzortung stations to compare directly:

https://forum.blitzortung.org/mybb/showt...3#pid28113

We found an apparent difference of roughly 4 milliseconds between them. Four milliseconds is an eternity in lightning-location language. In that case the evidence pointed toward something in the local network path.

If something like that is occurring somewhere beyond your own network, of course, there is essentially nothing you can do about it.

But before chasing a server-side “Factor X,” I think there is another important distinction:

Used is not really a station-success meter, nor is it a measure of successful packet delivery.

Signals and Valid occur earlier in the chain. Valid signals are then considered by the processing system; Involved are signals associated with actual detected strokes, and Used are the Involved signals actually selected for the location calculation.

So a Used count changing from 3,000 to 500 does not, by itself, mean that anything has deteriorated at your station.

The Selected Region statistics can vary dramatically with lightning activity, station location, the processing region, and which other stations are available at the moment.

As for optimizing the receiving system, most of what an operator really needs is in the two attached PDFs... more or less. I’m still refining some of the presentation, but the basic concepts are there. Which is why some 'instructions' might seem shotgun, do this do that... station environments are unique... take the two documents, keep it simple, realize that once it leaves your network, you have zero control.

The BlitzView notes are probably the document most directly relevant to the statistics you are looking at. They also explain why stability matters so much, including gain, timing, trigger level, sieve behavior, and the danger of trying to optimize while the station itself is constantly changing settings automatically.

The other PDF is more background on why I tend not to approach lightning reception like conventional narrow-band RF “tuning.” Lightning is a transient impulse phenomenon; the receiver is sampling an impulse-energy/time signature rather than hunting for some nice repeating carrier frequency.

As a one-time Air Force Electronic Warfare Shop technician, followed by about 30 years in consumer electronics, radio and television, I’d gently suggest that some alternate thinking becomes very valuable when getting acquainted with this lightning-location hobby.

I came into it wanting to measure, isolate, adjust and solve everything too.

That approach is useful — but only up to a point.

A tremendous amount of what you see in Blitzortung statistics is controlled by lightning location, propagation, network geometry, the other stations available at that moment, and what the servers need from them. Those are things completely outside your station.

The sooner we admit how much of this is outside our control, the sooner we can quit beating the tar out of ourselves trying to “fix” every statistical change.