[arm-allstar] logging ideas.. was: update has an error

Doug Crompton wa3dsp at gmail.com
Fri Jul 28 14:41:44 EST 2017 

Dave,

I think dumping anything to the journal would be the way to go. It takes
care of size  (currently set to 50M) and there are all kinds of system
parameters that can be set. Yes it is all in one file but getting specific
data out is easy with the available commands.

https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs

Here is the journal from my hub showing (no) warnings and worse after
running 17 days -

 15:38:11 up 17 days,  1:10,  1 user,  load average: 0.87, 0.78, 0.57
[root at 27225-HUB etc]# journalctl -p warning -b
-- No entries --

There are a lot of journal commands. We could write scripts to extract
different data.


*73 Doug*

*WA3DSP*

*http://www.crompton.com/hamradio <http://www.crompton.com/hamradio>*



On Fri, Jul 28, 2017 at 12:59 PM, "David McGough via arm-allstar" <
arm-allstar at hamvoip.org> wrote:

>
> Hi Doug,
>
> I started this thread about logging methods. My main concerns were to
> significantly increase the amount of log data available, without have it
> get out of hand or be forever increasingly complex to manage.
>
>
> 73, David KB4FXC
>
>
>
> On Fri, 28 Jul 2017, Doug Crompton wrote:
>
> > Tom,
> >
> >  OK understood but the journal does that now for system logging and any
> > other logs can just simply be truncated using a script like I do it. No
> > need to get fancy. Specify the maximum size and let it trim it daily or
> > more often if desired. If you are concerned about it being oversized you
> > can create a tmpfs directory and limit the size in fstab like we do with
> > /var/log now. I run a 24/7 hub here and I never have to worry about
> > oversize logs using those scripts. There is a lot of ram space and logs
> > require very little compared to other things.
> >
> > tmpfs             472180     284    471896   1% /tmp
> > tmpfs              51200    1484     49716   3% /var/log
> >
> >
> > *73 Doug*
> >
> > *WA3DSP*
> >
> > *http://www.crompton.com/hamradio <http://www.crompton.com/hamradio>*
> >
> >
> >
> >
> > On Fri, Jul 28, 2017 at 12:14 PM, Tom Hayward <tom at tomh.us> wrote:
> >
> > > Doug,
> > >
> > > You must have missed the beginning of the thread. We're not talking
> > > about logging to an SD card, but rather creating a circular buffer of
> > > stderr messages in memory. One way to do this is with a syslog
> > > application, tmpfs (ramdisk), and log rotation. There is no need to
> > > keep the data long term, just long enough so that the admin menu can
> > > display whatever error was returned after one of its child apps
> > > crashes.
> > >
> > > Tom KD7LXL
> > >
> > > On Fri, Jul 28, 2017 at 9:09 AM, Doug Crompton <wa3dsp at gmail.com>
> wrote:
> > > > Please keep in mind that while logging is nice any logging that would
> > > > survive a reboot has to write to the SD card. Currently all logging
> is
> > > done
> > > > to RAM and not the SD card. Any method to log to SD should be
> something
> > > that
> > > > could be turned on when necessary and the rest of the time turned
> off.
> > > Most
> > > > would not need theses logs under normal circumstances. I am against
> > > anything
> > > > that continually writes to the SD card, it is bad pratice.
> > > >
> > > > I run a hub that is 24/7 and I have implemented two methods to
> truncate
> > > > logs. The scripts are attached if you want to see how I do it. Logs
> could
> > > > also be rsync'ed to another system before truncating using a script
> like
> > > > this.
> > > >
> > > > Also a user could install a USB stick and direct logging to it or use
> > > remote
> > > > logging to another system.
> > > >
> > > > 73 Doug
> > > > WA3DSP
> > > > http://www.crompton.com/hamradio
> > > >
> > > >
> > > >
> > > > On Fri, Jul 28, 2017 at 11:34 AM, "David McGough via arm-allstar"
> > > > <arm-allstar at hamvoip.org> wrote:
> > > >>
> > > >>
> > > >> All the syslog utilities that I'm readily familiar with aren't as
> smart
> > > as
> > > >> I'd like;  these common tools will rotate all log entries into the
> waste
> > > >> basket after a period of time and leave the administrator looking at
> > > empty
> > > >> files....Ideally, the mechanism would always keep at least the
> amount of
> > > >> log data specified by the low water mark setting...
> > > >>
> > > >> Anyhow, we'll figure out some strategy!
> > > >>
> > > >> 73, David KB4FXC
> > > >>
> > > >>
> > > >>
> > > >> On Fri, 28 Jul 2017, Tom Hayward wrote:
> > > >>
> > > >> > I definitely had something more simple in mind, like checking the
> > > >> > return code of processes started by the menu.
> > > >> >
> > > >> > Most syslog utilities support log rotation on both size and time.
> I
> > > >> > haven't looked at which one you use in Hamvoip, but it could
> probably
> > > >> > be configured with the buffer capability you describe.
> > > >> >
> > > >> > Tom KD7LXL
> > > >> >
> > > >> > On Fri, Jul 28, 2017 at 7:52 AM, David McGough <kb4fxc at inttek.net
> >
> > > >> > wrote:
> > > >> > >
> > > >> > > Hi Tom,
> > > >> > >
> > > >> > > I agree that in certain scenarios additional debugging info or
> logs
> > > >> > > would
> > > >> > > be very useful, as you suggest. As we all know, logs can also
> be a
> > > >> > > problem, if allowed to grow without maintenance....I recently
> had a
> > > >> > > low
> > > >> > > memory issue due to logs on a busy RPi3 that had been up
> > > continuously
> > > >> > > for
> > > >> > > 2+ months.
> > > >> > >
> > > >> > > Ideally, application messages and those generated from Admin
> Menu
> > > >> > > operations would get stored in a circular buffer, which could
> then
> > > be
> > > >> > > queried from a menu option. There are already several tools
> which
> > > >> > > could be
> > > >> > > used, but, I'd like to have a little more flexibility---perhaps
> a
> > > >> > > ring-buffer log that has a low and high water mark, but also
> > > includes
> > > >> > > a
> > > >> > > timestamp. So, the plan would be to weight the size of the
> buffer,
> > > >> > > between
> > > >> > > the water marks, based on the age of the log entries. The log
> would
> > > be
> > > >> > > allowed to grow up to the high water mark in bursts, but then
> would
> > > >> > > shrink
> > > >> > > back to the low mark after a period of time.
> > > >> > >
> > > >> > > Do you (or anyone?) know of an ready-to-use tool that'll do
> this???
> > > >> > >
> > > >> > >
> > > >> > > 73, David KB4FXC
> > > >> > >
> > > >> > >
> > > >> > > On Thu, 27 Jul 2017, "Tom Hayward via arm-allstar" wrote:
> > > >> > >
> > > >> > >> On Thu, Jul 27, 2017 at 9:24 AM, "C B via arm-allstar"
> > > >> > >> <arm-allstar at hamvoip.org> wrote:
> > > >> > >> >
> > > >> > >> > Thanks
> > > >> > >> > would you consider having the error messages stay on the
> screen
> > > >> > >> > until a OK button is pressed?
> > > >> > >> > this may be helpful.
> > > >> > >> > Chris
> > > >> > >>
> > > >> > >> That would be really helpful for any admin menu command that
> > > returned
> > > >> > >> an error. The other day when I was affected by the buffer
> overflow
> > > >> > >> bug
> > > >> > >> for long hostnames, the observed behavior was that clicking
> > > >> > >> simpleusb-tune-menu in the menu would immediately go back to
> the
> > > >> > >> menu.
> > > >> > >> I had to run the command from a bash shell before I could see
> that
> > > >> > >> there was an error message and non-zero return code. Luckily I
> was
> > > >> > >> the
> > > >> > >> one working on it because the repeater owner I was working with
> > > isn't
> > > >> > >> as familiar with Linux and probably wouldn't have thought to
> try
> > > >> > >> that.
> > > >> > >> The admin menu hides the complexity of bash, but in its current
> > > form
> > > >> > >> it also hides all of the error detection and reporting
> > > functionality
> > > >> > >> we have in bash.
> > > >> > >>
> > > >> > >> Tom KD7LXL
> > > >> > >> _______________________________________________
> > > >> > >>
> > > >> > >> arm-allstar mailing list
> > > >> > >> arm-allstar at hamvoip.org
> > > >> > >> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> > > >> > >>
> > > >> > >> Visit the BBB and RPi2/3 web page - http://hamvoip.org
> > > >> > >>
> > > >> > >
> > > >> >
> > > >>
> > > >> _______________________________________________
> > > >>
> > > >> arm-allstar mailing list
> > > >> arm-allstar at hamvoip.org
> > > >> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> > > >>
> > > >> Visit the BBB and RPi2/3 web page - http://hamvoip.org
> > > >>
> > > >
> > >
> > >
> >
>
> _______________________________________________
>
> arm-allstar mailing list
> arm-allstar at hamvoip.org
> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
>
> Visit the BBB and RPi2/3 web page - http://hamvoip.org
>
>

More information about the arm-allstar mailing list