Efficiency

[micha.d]

I would like to bring the attention back to the title: Efficiency
I have one more very very general observation that I would like to present. Having a value in % in my eyes must mean that the value is in the range 0 - 100. This is the case for the "Effectivity" on http://de.blitzortung.org. It is not the case for the "Efficiency" on https://www.lightningmaps.org. Values like "< -999%" are obviously a bit "special". Formulas like a/b - c/d or a/(b-c) cannot generate a % value.

--> So either use % and stick to the range 0 - 100 or please just name it differently.

[allsorts]

I would like to change the measure for the efficiency of the stations.

I quite like Cutty's:

Efficiency = Strokes Detected / Signals Sent 
Effectivity = Strokes Detected / Signals Sent - Strokes Detected

Being based purely on the numbers from a given station it removes the influence of the total region count which generally has the affect of pushing a station that sends few but high quality signals down the list.

Not sure how to combine those with a distance. "signals sent" doesn't have any distance information but does that matter? I don't think it does.

Actually, it does... in a 'thought experiment'...

Hmm, whilst I agree that a stations enviroment and location affect the settings (threshold/gain etc) of that station my gut feeling about the above formula is that distance "cancels out".

ARGH! I hate web based editors that drop your carefully crafted message in the bit bucket if you accidentaly load another page in the same tab/window. I'd typed in a load of thought experiment showing that distance doesn't come into the formulas. It also showed that gain governed range and that threshold is best set after the gain/range value to the point where local noise only rarely triggers a signal. Note this was a thought experiment in a nice friendly world not the hostile, real, one. Finding a setting for the threshold with variable local noise levels might be tricky.

The range a station is optimised for is a choice for the operator but for most benefit to the network ought to follow that relatively isolated stations go for range and ones with nearby neighbours restrict their range.

[allsorts]

Cutty:

Efficiency = Strokes Detected / Signals Sent
Effectivity = Strokes Detected / Signals Sent - Strokes Detected

Has anyone used the above on real numbers?
Recent grab from the stations list of my station (1627) and the three closest others:

Sta Sent Used L Eff Eft
1690 1240 184 14% 15% 17%
1627 274 71 11% 26% 35%
1645 1714 180 10% 11% 12%
1620 1395 140 10% 10% 11%

L : Current Effectivity L (S and M where zero).
Eff : Used/Sent
Eft : Used/(Sent - Used)

Hey, I like this new measure. B-)

[readbueno]

(https://www.lightningmaps.org/blitzortun...n_id=14731),
....

EDIT: Before someone complains. I mixed up some numbers. The link above is not my station. But I stick to 3 antennas in the next live. Back to efficiency ...

ATTENTION!!!
Hi All, Whoever the above Station does belong to. The gain settings are wrong, there appears to be an extra 0 on the 10 making it 16*100 on both channels shown!
My views on high gains are well enough known. 
Whilst this is obviously a mistake, I think this station is still trying to use far too much gain!
I have yet to see a situation where anything over 10 was useful, except for generating problems and extra signals that are not lightning.
If you are one of these stations that insists on High gains and low thresholds, Please, Please try out very reduced gains, these boards are extremely sensitive and already well below the ambient noise floor in most situations, therefore increased gain only equals reduced signal to noise ratio, splatter caused by overload mixing products and other problems.
Spread the word! Make someone your Buddy, you help them and they help you, and when not much is happening you can "chat" via email to someone that understand what you are talking about.
Regards,

Brian.

[readbueno]

Cutty:

Efficiency = Strokes Detected / Signals Sent
Effectivity = Strokes Detected / Signals Sent - Strokes Detected

Has anyone used the above on real numbers?
Recent grab from the stations list of my station (1627) and the three closest others:

Sta   Sent  Used   L       Eff     Eft
1690  1240  184  14%  15%  17%
1627    274   71  11%  26%  35%
1645  1714  180  10%  11%  12%
1620  1395  140  10%  10%  11%

L : Current Effectivity L (S and M where zero).
Eff : Used/Sent
Eft : Used/(Sent - Used)

Hey, I like this new measure. B-)

Hi All, I have yet to see any figures for Effectivity "S", ever, at my station and rarely get figures for Effectivity "M," even though some of the strokes are listed in the archives at only 25  km away and many more in the 100 to 300 km range.
I would like to see these numbers extended over a much greater time period, maybe several time options, 1 hr, 4 hrs, 12 hrs, one day, one week, one month, Total of all!
Some people have not yet realised, that if their station is sending many signals to the server and if there is no lightning in their vicinity, it is a sure sign that your station has a problem.
Yes, we all probably take more attention, when there is a lot of action on the screen, but it is also important to see what your station is doing during the "quiet" times.
Just another idea, may be we should not be looking so much at Individual station Efficiency/Effectivity, but at Overall Network Efficiency.
What are the effects of varying the number of stations needed to validate a stroke, on the description it says the more the better, but how many strokes are not recorded, because there were too few station to validate them, maybe by quite a small margin, i.e, because only ten, or even less, stations registered the stroke!
Is all this data lost?
Why are so few stations actually contributing to this forum. This is an important question for us all and I am sure there are many people brighter than me that would have perfectly valid ideas on this subject, if only they would state them, instead of keeping them to themselves.
Maybe, we need a "Banner Headline" Page, that appears every time we open the maps in our browsers, that gives a summary of the days events and the present situation of the network?
Regards,

Brian.  

[Alanpenwith]

Hi All
I'm very new to this but it seems that there are probably very few stations with the same characteristics which makes a comparison difficult if not impossible. 
The only thing constant for all stations is a particular stroke.
So what if we could analyse by looking at who is watching what. In other words use only the signals from the same cluster of storms and then further analyse by range bands from the storms. If only one (or clustered) storm is used the distortion of data from those stations able to pick up signals from two or more storms would be eliminated. I suggest a formula something like:- Stations in range of storm A  0 - 500K  "My detections / Mean of all detections" and the same for increasing range bands. It doesn't eliminate false detections but at least it tries to demonstrate how a station is performing in relation  to the same events compared to its peers.  Stations below the mean should perhaps adjust whilst those very much above the mean could expect false detections. 
I'm no statistician so maybe this is too simplistic approach but I hope it is thinking out of the box.
Regards
Alan

[cutty]

Let me bring the general 'Cutty' suggestion back into focus, since it was generally outlined in two posts buried up above....

"

Hi Folks,
I would like to change the measure for the efficiency of the stations. It may not be appropriate to calculate only one numerical value, but rather to introduce different possibly even competing measures for the efficiency of the systems.
The first value could indicate how many of the transmitted signals are involved in the calculated strikes. The second value would be distance dependent and indicates how many impacts with a certain distance to the own detector the station was involved.
The current combination of these values (thate what we are doing now) seems to be not usefull for adjusting a detektor.
Any ideas for further efficiency measures?
/Egon

Just brainstorming:  
Suppose we explore this, for each unique station, as opposed to, or in addition to, station's relative "Network / Region" performance. This suggestion is for 'Typical" installations.  It's my belief that a second group might be considered, for example, those stat1ons that are 'purposefully' employed across regions or oceans. However they, I believe, should be
"qualified" on Efficiency and Effectivity overall before being placed into that group. And 'disqualified' if performance degrades. If makes little sense to me to have a station with a noisy environment attempting to serve 'cross region' or ocean, for example.
We define:
Goal: detect maximum number of strokes accurately with minimum number of signals
Efficiency: Ability to accomplish something with the least amount of time and effort.
Effectivity: Actual production of the intended result.

Efficiency = Strokes Detected / Signals Sent
Effectivity = Strokes Detected / Signals Sent - Strokes Detected

examples:
Station:  200 strokes with 1000 signals sent:
Efficiency = 200/1000=20%
Effectivity = 200 / (1000-200) 800 = 25%
Station: 200 stokes with 500 signals sent:
Efficiency = 200/500=40%
Effectivity = 200 / (500-200) 300=67%
Station: 800 stokes with 2500 signals sent:
Efficiency = 800/2500=32%
Effectivity = 800 / (2500-800) 1700=47%

That is, a station which sends a lot of signals, but few strikes is not very efficient.
For Effectivity; some of the 'signals' sent ARE strikes, and they are removed from the Total Signals, then strikes compared to 'remaining' (unused" signals.
The thought experiment runs sort of like this:
Briefly, divide the stations according to 'intent'... say, for example, three groups.
Group ONE > operators who haven't bothered to define or optimize well.  Anything 'in'.,..  Anything 'Out'.
Group TWO >  Stations that have established a 'relatively' quiet operational invironment, normally function with a 'High" "Strokes to signals", and have optimized for 'distance'... would include many of the 'dual' region / fringe stations perhaps. Since they already operate with 'low' 'excess' signals. They can operate for example as the 'group greater than 1200 km'. Their signals would also include the signals of a nearby 'group three', of course.
Group THREE> Stations optimized for <1200 km... those that run many excess signals when using high gains and / or low thresholds, or with frequent 'sporadics' that could  be eliminated, or minimized with lower gains / and or higher thresholds.  They now optimize for reduced distance. Optimizing in this fashon also lessens the typical "excess signals' relative to 'detections within 1200 km'.
And you could have perhaps a smaller group, perhaps operating E field only, or C Horizontal only, operating at distances <600km or any other subdivision similar...
The decrease in 'out of distance' signals, with an optimizing within distance should allow the same similar formulas too be used... though a group 2 station might show a 'higher effieciency' or 'lower efficiency' than a station in group 3, or vice versa, the rules are the same, and the 'comparisons' similar, within the group.  not necessarily outside the group.
Both groups should be much more efficient and effective than MOST of the 'group ones'... and my 'gut feeling' is that watching numbers between groups TWO and THREE would actually show similar efficiency / effectivity numbers... IF the group TWOs had strokes <1200 km subtracted from both signals and strokes ,,,
Again, an operator shouldn't compare 'outside his Group" , ...
But... Especially on group TWOs... a station should establish, over time, that it BELONGS in group TWO !  To do so, they'd have to operate as a GROUP one until the "Higher Effeciency / Effectivity" numbers qualified them as a 'group TWO'... if a station degrades over some time frame, than they should be 'demoted' to group ONE, unless they opted to go for Group 3.
All stations would begin as group ONE, and then optimize for, and qualify for group TWO or THREE.  Actually, I think the server can do that automatically, since it already is capable of breaking strokes down by distance.d A 'quiet' group TWO station might not have as many 'signals' at high latitudes seasonally, but the 'noise signals' may remain essentially the same.... for a Group THREE, some adjustmen might need to be made seasonally, since 'theoretically' their 'local noise' soruces would remain at similar levels year round, while signal numbers rise and fall.
In those cases a comparison with 'degradation' over time of total 'signals' might be applied.

[readbueno]

Hi Alan, I am not so sure about "false strokes" maybe it would be better to call them mis-classified, as we measure mostly CG lightning, but often record CC lightning as much less powerful strokes, especially in near storms.
If our detection was better, as I am sure it will be, we would perhaps have better discrimination between different types of lightning, or interference that mimics lightning, from the point of view of our stations.
I am sure that most people running a station are more aware of how lightning behaves in its various forms, but here are a few basic sites were lightning is explained:-

https://en.wikipedia.org/wiki/Lightning

https://www.nationalgeographic.com/envir...lightning/

http://www.nssl.noaa.gov/education/svrwx101/lightning/

I know there are many more, some very complicated, but these do give some basic facts.

Alan, As storms are a moving phenomena I am not sure how your cluster theory would work in practice.
Could you be more definite in describing how individual and groups of stations would organise their data, to make more usable.
With over 1.4 billion strokes a year world-wide, we are still missing a lot of data in the strokes that we do archive, and even for the data that we do collect, I am not sure what analysis or research is being done, in relation to classifying strokes, propagation characteristics, even diurnal variation.
Is there even a good way that this huge amount of data could be indexed and worked on, unless we have access to super computing or maybe distributed computing, like "SETI" or "Folding @ Home", or even something like "Zooniverse" where people organise vast amounts of data as a volunteer service and pastime?
Certainly even organising the data from my own station appears to be beyond my meagre capabilities, and I would welcome ideas and programs from those that are more expert in these matters.
Sorry, I am probably off topic again, but these ideas do come to mind whilst reading the forum and may add to the overall mill of information.
Regards,

Brian.

[cutty]

Let me bring the general 'Cutty' suggestion back into focus, since it was generally outlined in two posts buried up above....

"

Hi Folks,
I would like to change the measure for the efficiency of the stations. It may not be appropriate to calculate only one numerical value, but rather to introduce different possibly even competing measures for the efficiency of the systems.
The first value could indicate how many of the transmitted signals are involved in the calculated strikes. The second value would be distance dependent and indicates how many impacts with a certain distance to the own detector the station was involved.
The current combination of these values (thate what we are doing now) seems to be not usefull for adjusting a detektor.
Any ideas for further efficiency measures?
/Egon

Just brainstorming:  
Suppose we explore this, for each unique station, as opposed to, or in addition to, station's relative "Network / Region" performance. This suggestion is for 'Typical" installations.  It's my belief that a second group might be considered, for example, those stat1ons that are 'purposefully' employed across regions or oceans. However they, I believe, should be
"qualified" on Efficiency and Effectivity overall before being placed into that group. And 'disqualified' if performance degrades. If makes little sense to me to have a station with a noisy environment attempting to serve 'cross region' or ocean, for example.

We define:
Goal: detect maximum number of strokes accurately with minimum number of signals
Efficiency: Ability to accomplish something with the least amount of time and effort.
Effectivity: Actual production of the intended result.

Efficiency = Strokes Detected / Signals Sent
Effectivity = Strokes Detected / Signals Sent - Strokes Detected

examples:
Station:  200 strokes with 1000 signals sent:
Efficiency = 200/1000=20%
Effectivity = 200 / (1000-200) 800 = 25%
Station: 200 stokes with 500 signals sent:
Efficiency = 200/500=40%
Effectivity = 200 / (500-200) 300=67%
Station: 800 stokes with 2500 signals sent:
Efficiency = 800/2500=32%
Effectivity = 800 / (2500-800) 1700=47%

That is, a station which sends a lot of signals, but few strikes is not very efficient.

For Effectivity; some of the 'signals' sent ARE strikes, and they are removed from the Total Signals, then strikes compared to 'remaining' (unused" signals.

The thought experiment runs sort of like this:
Briefly, divide the stations according to 'intent'... say, for example, three groups.

Group ONE > operators who haven't bothered to define or optimize well.  Anything 'in'.,..  Anything 'Out'.
Group TWO >  Stations that have established a 'relatively' quiet operational invironment, normally function with a 'High" "Strokes to signals", and have optimized for 'distance'... would include many of the 'dual' region / fringe stations perhaps. Since they already operate with 'low' 'excess' signals. They can operate for example as the 'group greater than 1200 km'. Their signals would also include the signals of a nearby 'group three', of course.
Group THREE> Stations optimized for <1200 km... those that run many excess signals when using high gains and / or low thresholds, or with frequent 'sporadics' that could  be eliminated, or minimized with lower gains / and or higher thresholds.  They now optimize for reduced distance. Optimizing in this fashon also lessens the typical "excess signals' relative to 'detections within 1200 km'.

And you could have perhaps a smaller group, perhaps operating E field only, or C Horizontal only, operating at distances <600km or any other subdivision similar...

The decrease in 'out of distance' signals, with an optimizing within distance should allow the same similar formulas too be used... though a group 2 station might show a 'higher effieciency' or 'lower efficiency' than a station in group 3, or vice versa, the rules are the same, and the 'comparisons' similar, within the group.  not necessarily outside the group.
Both groups should be much more efficient and effective than MOST of the 'group ones'... and my 'gut feeling' is that watching numbers between groups TWO and THREE would actually show similar efficiency / effectivity numbers... IF the group TWOs had strokes <1200 km subtracted from both signals and strokes ,,,
Again, an operator shouldn't compare 'outside his Group" , ...

But... Especially on group TWOs... a station should establish, over time, that it BELONGS in group TWO !  To do so, they'd have to operate as a GROUP one until the "Higher Effeciency / Effectivity" numbers qualified them as a 'group TWO'... if a station degrades over some time frame, than they should be 'demoted' to group ONE, unless they opted to go for Group 3.

All stations would begin as group ONE, and then optimize for, and qualify for group TWO or THREE.  Actually, I think the server can do that automatically, since it already is capable of breaking strokes down by distance.d A 'quiet' group TWO station might not have as many 'signals' at high latitudes seasonally, but the 'noise signals' may remain essentially the same.... for a Group THREE, some adjustmen might need to be made seasonally, since 'theoretically' their 'local noise' soruces would remain at similar levels year round, while signal numbers rise and fall.

In those cases a comparison with 'degradation' over time of total 'signals' might be applied.

[Alanpenwith]

Hi All
I realise that storms are moving phenomena but the movement is relatively slow and my thought was that there might be a simple way of tuning a detector. Well tuned detectors must benefit the network as a whole. Not everyone would have the time to analyse a lot of historical data and, like me, many have probably not learnt enough to use it effectively.

Maybe something along the lines I suggest would be a sort of 'Quick Start Guide' for those with limited time and/or knowledge run in parallel with the 'proper tools' if the server capacity would allow it. Brian has pointed me to the Strikes versus Signals graphic which gives a guide as to what I am doing but not what I should be achieving in comparison to stations which should be showing similar results.

As recently we have has storm clusters distant from each other I have been able to improve my performance somewhat by watching other stations fairly close to me. What I am suggesting is a formalisation of that.

I see the point of Egon's grading system but the spread of the system must mean there are some who 'don't have the time or knowledge/lost interest' who be be stuck in group one but might be motivated by a simple method of improving.
Regards
Alan