+- Blitzortung.org Forum (https://forum.blitzortung.org/mybb)
+-- Forum: Public Forums (https://forum.blitzortung.org/mybb/forumdisplay.php?fid=29)
+--- Forum: General Discussion (https://forum.blitzortung.org/mybb/forumdisplay.php?fid=31)
+--- Thread: Efficiency (/showthread.php?tid=1261)
RE: Efficiency - readbueno - 2017-11-03
(2017-11-03, 21:48)pasense Wrote: readbueno,
since I started the station here there was no thunderstorm on the Atlantic nearer than about 1000 km, and some scattered thunderstorms over North Africa, so there was simply no local lightning at all. This may change when we get lows from the west, with lightning and the sorely missed rains, but at the moment nothing like this is in the forecasts.
Fair enough!
pasense_me.zip (Size: 226.16 KB / Downloads: 6)
RE: Efficiency - allsorts - 2017-11-04
Hi'yall,
I can see this discussion going round and round in circles and not getting any where if we aren't careful. I think it's time for a summary and way forward:
1) There isn't a set of guides for new participants. Guides on the subjects of:
Overview - What a station should aim to do, taking into account its location, to provide the network with high quality data.
Controller - Construction, description of how it works, description of each firmware setting, option, mode, etc.
Power - Noisy SMPSU's, cures, PoE, Linear supplies.
Antennas - Construction, mounting, connection, location, use of their chracteristics.
Enviroment - Common interference/noise sources, tools and methods to trace noise sources.
Settings - Ways to interpret the station information available from blitzorgtung, LMO, MyBlitzortung. The effects and side
effects of hardware pararmeters.
All stuff for the FAQ, much of it is in the forum(s), if it can be found.
2) The current station measures (Effectivity S, M and L) are not particulary useful aids to setting up or optimising a station. Only one alternative(*) has been suggested that has had some disscussion. Namely Cutty's:
Efficiency = Strokes Detected / Signals Sent
Effectivity = Strokes Detected / Signals Sent - Strokes Detected
Probably time to apply to data sets from a range of real stations to check that it does return useful and meaningful values.
(*) If there have been others they have been lost in the noise by this reader.
RE: Efficiency - pasense - 2017-11-04
@allsorts,
thanks for the fine summary; only one thing I would like to add: better tools for analysis like waterfall and signal strength histogram.
If it is the consensus that we need a FAQ with this content, I will wait until Sunday evening UTC and then contact Egon directly.
RE: Efficiency - readbueno - 2017-11-04
Hi Dave, et al, There was also Alan's suggestion of Geographical clusters, Cutty's "Three group system" and my suggestion that time and some form of averaging/weighting should be included in the calculation.
(RMS/Time? x strokes + signals sent /100) or something?Yes, I think we have enough basic and not so basic questions to actually start the FAQ and maybe that would be the way to handle it, a very basic public and possible open FAQ, "Ten things you want to know about Lightning Detection!" type, and a more detailed and restricted FAQ for stations with a matching forum for discussion of new or debatable questions, your 7 or 8 categories seem to cover most things and a perhaps a catch-all miscellaneous section as well.
When does it open for business?
Brian.
RE: Efficiency - allsorts - 2017-11-05
For the "station measure" the KISS Principle needs to be applied (Keep It Simple Stupid). The current blitzortung station list Effectivity isn't very fair, take these numbers for my and the next closest three stations pulled from the station list earlier:
Cutty's
Sigs Parti Ef L Effc Efct
4306 1814 16% 0.42 0.73
1229 619 1% 0.50 1.01
1016 330 0% 0.32 0.48
494 341 0% 0.69 2.23
4216 1788 16% 0.42 0.74
1229 633 1% 0.52 1.06
1001 337 0% 0.34 0.51
502 360 0% 0.72 2.54
As these numbers come from the station list they cover the previous 60 mins and up to 5000 km. The vast majority of those strikes are happening over Itally, 1000 miles (1600 km) away. There does need to be a time period over which the signals and particpated strikes are counted. An hour seems about right for a quick "how is my station doing" and if updated every minute ought to reflect any settings changes fairly rapidly and totaly after an hour, which isn't too long to wait. I think there should also be a longer period, 24 or 48 hours perhaps, that would give an indication of how the station is doing over the longer term. Storms would generally come and go over that time scale.
If we do adopt Cutty's suggestion there does need to be at least a divide by zero trap and/or some cap on the value of "Effectivity". A cap of 10 seems reasonable, to get a Effectivity of 10, 91% of the signals sent would have to be used. Also this value is exponential but I think this helps us as the sensitivity of the value increases with the percentage of signals used. That is 50% of signals used gives a value of 1, 60% 1.5, 70% 2.33, 80% 4, 90% 9. A target of getting over 66% (ie a value greater than 2) of signals used ought to be achievable by most stations. The 2.23 and 2.54 above represent 69% and 72% used.
Distance, geographic clusters, etc aren't really relevant to a stations "performance measure". We want that measure to be a useful aid in adjusting a stations settings (mainly gain and threshold) to achieve the goal of maximum participation with minimum signals sent. The station operator decides on the range of their station following the recomended guidelines of isolated - longer range - higher gain, lots of neighbours - shorter range - lower gain. The threshold is then adjusted to a level where local interference only rarely triggers the sending of a signal. This may mean that an isolated station with a noisy local enviroment doesn't have the range that the guidelines suggest it ought to have to give maximum benefit the network. That is solved by reducing the local noise, if possible, if not "it is, what it is".
RE: Efficiency - readbueno - 2017-11-05
Hi Dave, I think that does not sound too bad. The actual cap/weighting and the number of signals used data, should be calculated using data that is available from the server over various periods of time, and IMO the longer periods do not need to be "real-time," i.e. they could be calculated and stored only once over that time period.
I still think that geographical clusters might serve some useful purpose, but maybe they aren't so important.
For several days the results in the station lists do not appear to be updating for several stations and in many cases near strokes are not registered at all? This could be a software bug.
Although, I seem to have very low "ping" rates and over the last few days Traceroute has shown a lot of hops within the internet network, with very long delays, so this might be part of the problem.
At the height of the storm over southern Spain and Italy, the archives did not seem to produce any graphics, even for strokes that were several hours old, was this data lost?
These things would also matter if you were adjusting the station, just using the raw data.
The, more or less, ten percent of stations on the far right and, especially the ones at the bottom right, of the meteomelin graph (the stations sending 10,000+ signals per hour, for very few strokes.) should have a closer look at how their station is functioning and make the appropriate adjustments and mitigate their sources of interference as much as is possible.
Maybe these stations should be excluded from the server during busy periods automatically.
It is not all noise that triggers interference mode and maybe the criteria for this needs to be reassessed.
These stations could be jamming up the server, when others are actually trying to send real data.
My 2 cents.
Brian.
RE: Efficiency - allsorts - 2017-11-05
(2017-11-05, 17:07)readbueno Wrote: Hi Dave, I think that does not sound too bad. The actual cap/weighting and the number of signals used data, should be calculated using data that is available from the server over various periods of time, and IMO the longer periods do not need to be "real-time," i.e. they could be calculated and stored only once over that time period.
Well what ever is adopted will be handled by the server as the current system is. I agree though that for evaluation the data ought to come from a carefully crafted URL request to the server. I did check that, for my station, the "Participated" count in the station list agreed with what MyBlitzortung said, which they more or less did.
I think the longer periods should still be a rolling period, ie the last 24 hours, updated every 5 minutes. If you had 24 hours updated every 24 hours the first data could be nearly 48 hours old: Update at T0, first data is T-24 hours old. Just before the next update at, T+24, the first data is 24+24 = 48 hours old.
(2017-11-05, 17:07)readbueno Wrote: I still think that geographical clusters might serve some useful purpose, but maybe they aren't so important.
Geography probably has a place to check how your station is doing in a general way. If you assume that sferics are going to be fairly similar within, say, 50 km of the station. Large differences in Signals and Participated would indicate a problem of some sort. Relatively high level of local noise, forcing the threshold up, a poorly setup station, causes being on either yours or the other station(s).
(2017-11-05, 17:07)readbueno Wrote: Maybe these stations should be excluded from the server during busy periods automatically.
(Stations sending 10's of thousands of signals/hour but only participating for very few of those signals).
Excluding the data from these stations doesn't help with them "clogging up" the link(s) into the server. Unless they are told to shut up the packets will still be sent and the server do something with them even if it's just drop them based on IP address. If we can come up with a good "performance measure" that could be used via the regular "phone home" system to tell a station to shut up. The hard part is how do you inform the station operator that their station has been told to shut up? email I guess, if a valid email is known. A "message waiting" flag in the controller web interface might not be seen for weeks.
RE: Efficiency - readbueno - 2017-11-05
(2017-11-05, 18:57)allsorts Wrote: Excluding the data from these stations doesn't help with them "clogging up" the link(s) into the server. Unless they are told to shut up the packets will still be sent and the server do something with them even if it's just drop them based on IP address. If we can come up with a good "performance measure" that could be used via the regular "phone home" system to tell a station to shut up. The hard part is how do you inform the station operator that their station has been told to shut up? email I guess, if a valid email is known. A "message waiting" flag in the controller web interface might not be seen for weeks.There are the Alerts, if these are set up and people checked them, they could be used?
There is also the considerable "waste" of data.
On a suggestion, this evening I tried my station on full Auto, in that three hours, my station sent more signals than in the previous 96 hours and the data rate shot up from less than 15 Mbytes per day to over 1Gbyte per day.
All this extra data for a very marginal return in a few extra strokes recorded?
I have also discovered that UDP does not have a complete communication "handshake," like TCP does and that if data suffers packet loss the station just continues sending data regardless of any acknowledgement, and the server is not even aware that large chunks of data may in fact be missing. With TCP each block of data has to be received correctly before the next set of packets is sent, In UDP this is not the case and there is no redundancy.
While this important fact improves gameplaying by not repeating data requests, thus speeding up the game and filling in from previous data, the lost data just being exactly that LOST!
In a lightning detection and recording system, however, the network could in fact be losing vast amounts of data, due to overload or other reasons and this is not so acceptable?
There is probably, as in most systems, an optimal acceptable rate for data sent and data lost, however by overstressing the system by sending tens or even in some cases hundreds of thousands of extra signals per hour, when this is mainly useless noise, is not an efficient nor effective way to run the network.
Internet Protocols are designed for different types of usage and if we are to have complete archives and mapping then this would also need to be addressed.
Brian.
RE: Efficiency - dagnazza - 2017-11-06
(2017-11-05, 19:04)readbueno Wrote:(2017-11-05, 18:57)allsorts Wrote: Excluding the data from these stations doesn't help with them "clogging up" the link(s) into the server. Unless they are told to shut up the packets will still be sent and the server do something with them even if it's just drop them based on IP address. If we can come up with a good "performance measure" that could be used via the regular "phone home" system to tell a station to shut up. The hard part is how do you inform the station operator that their station has been told to shut up? email I guess, if a valid email is known. A "message waiting" flag in the controller web interface might not be seen for weeks.There are the Alerts, if these are set up and people checked them, they could be used?
There is also the considerable "waste" of data.
On a suggestion, this evening I tried my station on full Auto, in that three hours, my station sent more signals than in the previous 96 hours and the data rate shot up from less than 15 Mbytes per day to over 1Gbyte per day.
All this extra data for a very marginal return in a few extra strokes recorded?
I have also discovered that UDP does not have a complete communication "handshake," like TCP does and that if data suffers packet loss the station just continues sending data regardless of any acknowledgement, and the server is not even aware that large chunks of data may in fact be missing. With TCP each block of data has to be received correctly before the next set of packets is sent, In UDP this is not the case and there is no redundancy.
While this important fact improves gameplaying by not repeating data requests, thus speeding up the game and filling in from previous data, the lost data just being exactly that LOST!
In a lightning detection and recording system, however, the network could in fact be losing vast amounts of data, due to overload or other reasons and this is not so acceptable?
There is probably, as in most systems, an optimal acceptable rate for data sent and data lost, however by overstressing the system by sending tens or even in some cases hundreds of thousands of extra signals per hour, when this is mainly useless noise, is not an efficient nor effective way to run the network.
Internet Protocols are designed for different types of usage and if we are to have complete archives and mapping then this would also need to be addressed.
Brian.
@Brian, TCP demands network discipline all the way and NAT specific policies.
By the way, lot of setting to the Blue system can be done remotely by the server: all amp levels, thresholds, fine tune of filters, etc. However this will require rethinking all security policy, introduction of TLS, etc. And servers must become little smarter, some of the AI required to adapt the network of sensors per given sferics conditions / regions. I'm sure @Egon is playing with some code already
RE: Efficiency - allsorts - 2017-11-07
Alerts: Not looked into them very much, if they send an email to the operator that would work. Relying on the operator looking at the right part of the web interafce or the LEDs isn't good enough. I sometimes don't loook at the web interface for weeks and the controller is up in a loft out of eye and ear sight...
Yes, UDP is "send and forget", however I think all the data for a single signal fits into a single packet of 1.5 k bytes ish. So you don't need the overhead of the TCP error checked/retransmission system to ensure that all the data of a signal arrives. It all does, or doesn't if the packet gets lost. Assuming any packet loss is random acros stations, losing packets isn't going to have a great impact on the network. Say 50 stations report a stroke and 15 are needed to located it, 35 can get lost...