[arm-allstar] logging ideas.. was: update has an error

David McGough kb4fxc at inttek.net
Fri Jul 28 15:02:34 EST 2017 

Interesting idea! ....I hadn't considered using journald as a general 
purpose logging facility.


On Fri, 28 Jul 2017, "Doug Crompton via arm-allstar" wrote:

> Dave,
> 
> I think dumping anything to the journal would be the way to go. It takes
> care of size  (currently set to 50M) and there are all kinds of system
> parameters that can be set. Yes it is all in one file but getting specific
> data out is easy with the available commands.
> 
> https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs
> 
> Here is the journal from my hub showing (no) warnings and worse after
> running 17 days -
> 
>  15:38:11 up 17 days,  1:10,  1 user,  load average: 0.87, 0.78, 0.57
> [root at 27225-HUB etc]# journalctl -p warning -b
> -- No entries --
> 
> There are a lot of journal commands. We could write scripts to extract
> different data.
> 
> 
> *73 Doug*
> 
> *WA3DSP*
> 
> *http://www.crompton.com/hamradio <http://www.crompton.com/hamradio>*
> 
> 
> 
> On Fri, Jul 28, 2017 at 12:59 PM, "David McGough via arm-allstar" <
> arm-allstar at hamvoip.org> wrote:
> 
> >
> > Hi Doug,
> >
> > I started this thread about logging methods. My main concerns were to
> > significantly increase the amount of log data available, without have it
> > get out of hand or be forever increasingly complex to manage.
> >
> >
> > 73, David KB4FXC
> >
> >
> >
> > On Fri, 28 Jul 2017, Doug Crompton wrote:
> >
> > > Tom,
> > >
> > >  OK understood but the journal does that now for system logging and any
> > > other logs can just simply be truncated using a script like I do it. No
> > > need to get fancy. Specify the maximum size and let it trim it daily or
> > > more often if desired. If you are concerned about it being oversized you
> > > can create a tmpfs directory and limit the size in fstab like we do with
> > > /var/log now. I run a 24/7 hub here and I never have to worry about
> > > oversize logs using those scripts. There is a lot of ram space and logs
> > > require very little compared to other things.
> > >
> > > tmpfs             472180     284    471896   1% /tmp
> > > tmpfs              51200    1484     49716   3% /var/log
> > >
> > >
> > > *73 Doug*
> > >
> > > *WA3DSP*
> > >
> > > *http://www.crompton.com/hamradio <http://www.crompton.com/hamradio>*
> > >
> > >
> > >
> > >
> > > On Fri, Jul 28, 2017 at 12:14 PM, Tom Hayward <tom at tomh.us> wrote:
> > >
> > > > Doug,
> > > >
> > > > You must have missed the beginning of the thread. We're not talking
> > > > about logging to an SD card, but rather creating a circular buffer of
> > > > stderr messages in memory. One way to do this is with a syslog
> > > > application, tmpfs (ramdisk), and log rotation. There is no need to
> > > > keep the data long term, just long enough so that the admin menu can
> > > > display whatever error was returned after one of its child apps
> > > > crashes.
> > > >
> > > > Tom KD7LXL
> > > >
> > > > On Fri, Jul 28, 2017 at 9:09 AM, Doug Crompton <wa3dsp at gmail.com>
> > wrote:
> > > > > Please keep in mind that while logging is nice any logging that would
> > > > > survive a reboot has to write to the SD card. Currently all logging
> > is
> > > > done
> > > > > to RAM and not the SD card. Any method to log to SD should be
> > something
> > > > that
> > > > > could be turned on when necessary and the rest of the time turned
> > off.
> > > > Most
> > > > > would not need theses logs under normal circumstances. I am against
> > > > anything
> > > > > that continually writes to the SD card, it is bad pratice.
> > > > >
> > > > > I run a hub that is 24/7 and I have implemented two methods to
> > truncate
> > > > > logs. The scripts are attached if you want to see how I do it. Logs
> > could
> > > > > also be rsync'ed to another system before truncating using a script
> > like
> > > > > this.
> > > > >
> > > > > Also a user could install a USB stick and direct logging to it or use
> > > > remote
> > > > > logging to another system.
> > > > >
> > > > > 73 Doug
> > > > > WA3DSP
> > > > > http://www.crompton.com/hamradio
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jul 28, 2017 at 11:34 AM, "David McGough via arm-allstar"
> > > > > <arm-allstar at hamvoip.org> wrote:
> > > > >>
> > > > >>
> > > > >> All the syslog utilities that I'm readily familiar with aren't as
> > smart
> > > > as
> > > > >> I'd like;  these common tools will rotate all log entries into the
> > waste
> > > > >> basket after a period of time and leave the administrator looking at
> > > > empty
> > > > >> files....Ideally, the mechanism would always keep at least the
> > amount of
> > > > >> log data specified by the low water mark setting...
> > > > >>
> > > > >> Anyhow, we'll figure out some strategy!
> > > > >>
> > > > >> 73, David KB4FXC
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Fri, 28 Jul 2017, Tom Hayward wrote:
> > > > >>
> > > > >> > I definitely had something more simple in mind, like checking the
> > > > >> > return code of processes started by the menu.
> > > > >> >
> > > > >> > Most syslog utilities support log rotation on both size and time.
> > I
> > > > >> > haven't looked at which one you use in Hamvoip, but it could
> > probably
> > > > >> > be configured with the buffer capability you describe.
> > > > >> >
> > > > >> > Tom KD7LXL
> > > > >> >
> > > > >> > On Fri, Jul 28, 2017 at 7:52 AM, David McGough <kb4fxc at inttek.net
> > >
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > Hi Tom,
> > > > >> > >
> > > > >> > > I agree that in certain scenarios additional debugging info or
> > logs
> > > > >> > > would
> > > > >> > > be very useful, as you suggest. As we all know, logs can also
> > be a
> > > > >> > > problem, if allowed to grow without maintenance....I recently
> > had a
> > > > >> > > low
> > > > >> > > memory issue due to logs on a busy RPi3 that had been up
> > > > continuously
> > > > >> > > for
> > > > >> > > 2+ months.
> > > > >> > >
> > > > >> > > Ideally, application messages and those generated from Admin
> > Menu
> > > > >> > > operations would get stored in a circular buffer, which could
> > then
> > > > be
> > > > >> > > queried from a menu option. There are already several tools
> > which
> > > > >> > > could be
> > > > >> > > used, but, I'd like to have a little more flexibility---perhaps
> > a
> > > > >> > > ring-buffer log that has a low and high water mark, but also
> > > > includes
> > > > >> > > a
> > > > >> > > timestamp. So, the plan would be to weight the size of the
> > buffer,
> > > > >> > > between
> > > > >> > > the water marks, based on the age of the log entries. The log
> > would
> > > > be
> > > > >> > > allowed to grow up to the high water mark in bursts, but then
> > would
> > > > >> > > shrink
> > > > >> > > back to the low mark after a period of time.
> > > > >> > >
> > > > >> > > Do you (or anyone?) know of an ready-to-use tool that'll do
> > this???
> > > > >> > >
> > > > >> > >
> > > > >> > > 73, David KB4FXC
> > > > >> > >
> > > > >> > >
> > > > >> > > On Thu, 27 Jul 2017, "Tom Hayward via arm-allstar" wrote:
> > > > >> > >
> > > > >> > >> On Thu, Jul 27, 2017 at 9:24 AM, "C B via arm-allstar"
> > > > >> > >> <arm-allstar at hamvoip.org> wrote:
> > > > >> > >> >
> > > > >> > >> > Thanks
> > > > >> > >> > would you consider having the error messages stay on the
> > screen
> > > > >> > >> > until a OK button is pressed?
> > > > >> > >> > this may be helpful.
> > > > >> > >> > Chris
> > > > >> > >>
> > > > >> > >> That would be really helpful for any admin menu command that
> > > > returned
> > > > >> > >> an error. The other day when I was affected by the buffer
> > overflow
> > > > >> > >> bug
> > > > >> > >> for long hostnames, the observed behavior was that clicking
> > > > >> > >> simpleusb-tune-menu in the menu would immediately go back to
> > the
> > > > >> > >> menu.
> > > > >> > >> I had to run the command from a bash shell before I could see
> > that
> > > > >> > >> there was an error message and non-zero return code. Luckily I
> > was
> > > > >> > >> the
> > > > >> > >> one working on it because the repeater owner I was working with
> > > > isn't
> > > > >> > >> as familiar with Linux and probably wouldn't have thought to
> > try
> > > > >> > >> that.
> > > > >> > >> The admin menu hides the complexity of bash, but in its current
> > > > form
> > > > >> > >> it also hides all of the error detection and reporting
> > > > functionality
> > > > >> > >> we have in bash.
> > > > >> > >>
> > > > >> > >> Tom KD7LXL
> > > > >> > >> _______________________________________________
> > > > >> > >>
> > > > >> > >> arm-allstar mailing list
> > > > >> > >> arm-allstar at hamvoip.org
> > > > >> > >> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> > > > >> > >>
> > > > >> > >> Visit the BBB and RPi2/3 web page - http://hamvoip.org
> > > > >> > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >> _______________________________________________
> > > > >>
> > > > >> arm-allstar mailing list
> > > > >> arm-allstar at hamvoip.org
> > > > >> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> > > > >>
> > > > >> Visit the BBB and RPi2/3 web page - http://hamvoip.org
> > > > >>
> > > > >
> > > >
> > > >
> > >
> >
> > _______________________________________________
> >
> > arm-allstar mailing list
> > arm-allstar at hamvoip.org
> > http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> >
> > Visit the BBB and RPi2/3 web page - http://hamvoip.org
> >
> >
> _______________________________________________
> 
> arm-allstar mailing list
> arm-allstar at hamvoip.org
> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> 
> Visit the BBB and RPi2/3 web page - http://hamvoip.org
>

More information about the arm-allstar mailing list