[arm-allstar] Echolink, Asterisk & one-way audio issues

Mark Szlaga mszlaga at szlaga.net
Sun Mar 4 21:45:29 EST 2018 

David,

Thank you for the reply.  I thought I was losing my mind!!!!!  :D

The GMARC repeater is heavily used.  We have nets almost nightly both via
Echolink and Allstar at present.

I can certainly work with you if you want to monitor my node and do
debugging, as it is a known problematic system.

Please reach out directly if you wish to do this.

Anything I can do to help get this troubleshot and build us a better
product in the end is TOTALLY worth it.

Keep up the good work.  Even with all the implementation issues we have
been running in to, folks still speak VERY highly of the new Allstar system.

Regards,
Mark - AC8LT

On Sun, Mar 4, 2018 at 9:35 PM, David McGough <kb4fxc at inttek.net> wrote:

>
> Hi Mark,
>
> Thanks for your AllStar and echolink experience feedback.
>
> You and a few others have reported regular crashes when echolink is
> enabled AND BEING HEAVILY USED. I manage a couple AllStar/echolink systems
> down here and haven't really noticed any problems, but echolink doesn't
> get heavily used. Since the failure seems to be a seg-fault in the
> chan_echolink thread, I'm hopeful the conditions which generate this crash
> won't be too hard to locate/duplicate. I'm actively debugging this now and
> am suspecting the cause is sporadic packet loss causing an improperly
> handled error condition. We will squash this problem ASAP!
>
> Note that there have been no major chan_echolink changes in over a year
> now, so I can't explain why crashes would have gotten worse with recent
> software updates??  My suspicion is you may have had a change in your
> echolink usage patterns (e.g.: increased usage?) that coincided over the
> same timeframe??  ....Anyhow, as I said, I'm looking for a solution.
>
> For a historical footnote, the original ("stock") chan_echolink driver did
> have some crash-me bugs which I also regularly butted heads with back in
> yesteryear (my "ACID" days). Those issues got resolved back around
> August 2016 and were related mostly to echolink nodelist updates going
> haywire.
>
> The one-way audio issue is a different beast. When it was originally
> reported, I figured it was due to a bug I introduced; I have VERY heavily
> updated the app_rpt code. However, after reviewing several user's systems
> where they were experiencing one-way audio, I know it affects AllStar
> versions at least as far back as the stock 0.325 version (originally
> ohnosec.org SVN release 1512, 10/19/2014)---even old "ACID" systems, too.
> So, one-way audio isn't a recently introduced problem.
>
> I originally thought one-way audio was only related to IAX2 connections
> between nodes. However, it has recently been reported with echolink, too.
> Again, probably indicating an increase in chan_echolink usage. But, this
> also means this problem isn't IAX2 specific...So, on a positive note, that
> narrows the places to look for this problem!
>
> For me, one-way audio is fleetingly sporadic.  Until a couple weeks ago, I
> had NEVER been able to locally duplicate this failure mode.  Then, after a
> network hiccup, I had a test node go one-way and I was able to perform a
> lot of diagnostics and learned a lot...Now, I've again modified my debug
> setup and am again waiting....And, waiting.... We'll get this one fixed,
> too. Eventually.
>
>
> 73, David KB4FXC
>
>
>
>
>
>
> On Sun, 4 Mar 2018, "Mark Szlaga via arm-allstar" wrote:
>
> > Doug,
> >
> > Thank you for your responses.  While I realize Echolink is not the
> > preferred protocol, the goal is to move everything where possible to
> > Allstar while combining various VoIP protocols in to a single point
> > directly connected to the repeater itself.
> >
> > Regarding Echolink.  I realize it is a mostly stateless connection, but I
> > find it odd that the control channel is still showing as connected, but
> no
> > audio.  Is this a known failure mode with Echolink and Allstar?  We have
> > not been seeing it with the previous SVXlink node.  If it is we can work
> > around it.  I just need to set expectations for the repeater trustee.
> >
> > Regarding Crashing.  This is the error message I am seeing from asterisk.
> > Note, this issue is across two different SD cards, and two different
> > RasPi3s.  All devices are known good and solid.
> >
> > safe_asterisk: line 125: 23632 Segmentation fault
> > (core dumped) nice -n $PRIORITY ${ASTBINDIR}/asterisk -f ${CLIARGS}
> > ${ASTARGS} &> /dev/${TTY}
> > Asterisk ended with exit status 139
> > Asterisk exited on signal 11
> > Automatically restarting Asterisk.
> > mpg123:: no process found
> >
> > Upon restart it is fine for a few minutes/hours until it restarts.
> > Thankfully every time it restarts and comes back.
> >
> > This has gotten worse since the update from the original image
> installation
> > to the latest version:
> > RPi2-3 Version 1.5rc4-15 Allstar - February 15, 2018 - KB4FXC, WA3DSP
> >
> > Regarding RX Delay.  I need help here.  1.5 seconds is a lifetime in this
> > industry.  WAY too long.  Any pointers on things to start investigating?
> >
> > Note: during normal local nets I set the RXOnDelay to 10 and RXAudioDelay
> > to 7 with no issues.  Works like a champ.
> > The ping-pong and pulsing only occurs during connected nets that we are
> not
> > hosting.  This issue is both on Allstar as well as Echolink.
> >
> > Thoughts?  Comments/questions?
> >
> > Regards,
> > Mark - AC8LT
> >
> > On Fri, Mar 2, 2018 at 3:29 PM, "Doug Crompton via arm-allstar" <
> > arm-allstar at hamvoip.org> wrote:
> >
> > > Mark,
> > >
> > >   I might also add to my previous remarks that Echolink unlike Allstar
> has
> > > no permanent connect mode. If a connection is flaky you are going to
> lose
> > > it at some point and it won't come back unless you reconnect. You
> should
> > > never depend on an Echolink connection. It is what it is.
> > >
> > >
> > > *73 Doug*
> > >
> > > *WA3DSP*
> > >
> > > *http://www.crompton.com/hamradio <http://www.crompton.com/hamradio>*
> > >
> > >
> > > On Thu, Mar 1, 2018 at 11:42 PM, "Mark Szlaga via arm-allstar" <
> > > arm-allstar at hamvoip.org> wrote:
> > >
> > > > We are seeing three issues on the WW8GM node (47185) that I have not
> been
> > > > able to find a straight solution for when searching.
> > > >
> > > > 1) Echolink randomly drops audio carrier entirely.
> > > >
> > > > While I realize the Echolink module is meant for compatibility, we
> are
> > > > using the module to replace an existing SVXlink node and expand in to
> > > > Allstar, with the goal of moving as many nets to utilize Allstar
> where
> > > > possible.
> > > >
> > > > That said, there is one net held on the ALARA conference node that is
> > > still
> > > > Echolink that we connect in to.
> > > >
> > > > Tonight and on previous occasions we have had instances where the
> audio
> > > > drops.  The connection still shows in Allstar and Echolink, I see the
> > > node
> > > > updating names as the participants transmit, but no audio.
> > > >
> > > > Every time a drop and reconnect fixes the issue.
> > > >
> > > > The question is, what causes this?  Is this a "known issue" with the
> > > > module, or are there workarounds.
> > > >
> > > > If this is new, what debugging can I run to capture the necessary
> > > > information to troubleshoot?
> > > >
> > > > 2) Ping-pong, bumping, pulsing...  I'v heard it called a number of
> > > > different names, but the result is the same...
> > > >
> > > > We have had some issues tuning both the Allstar and Echolink sides
> when
> > > > connected to nets.
> > > >
> > > > NOTE: This issue is ONLY seen during nets with other nodes, not
> locally
> > > or
> > > > when we are connected to one node.
> > > >
> > > > Configuration:
> > > > Hardware: Motorola Radius 1225
> > > > Software: Raspberry Pi3 running the HamVoIP image, updated to latest
> last
> > > > night.
> > > > Audio interface: RB_RIM_RC210 from repeater-builder.com
> > > > Carrier config: CoS is set to TSQL and repeater only transmits Tone
> > > before
> > > > courtesy tone during nets.
> > > >
> > > > Why this interface, because the node will be connected to the RC210
> at
> > > the
> > > > main repeater site once the bugs are worked out.
> > > >
> > > > But...  At this point we have RXOnDelay set to 75, with RXAudioDelay
> set
> > > to
> > > > 7.  This is enough that we aren't having bumps or bouncing, but 1.5
> > > seconds
> > > > is a lifetime!
> > > >
> > > > Any thoughts on things to try to reduce this and get a more
> manageable
> > > > delay timing?
> > > >
> > > > 3) Last but not least, we do have occasional Asterisk restarts when
> > > > connected to Echolink.
> > > >
> > > > We are seeing Asterisk restarts when connected to Echolink nodes.
> > > Allstar
> > > > and normal usage does not cause restarts.
> > > >
> > > > What can I do to the system to log these restarts to see why it is
> doing
> > > > this?  I have been letting a console run with the following
> > > > debugging/logging:
> > > >
> > > > core set verbose 99
> > > > core set debug 99
> > > > rpt debug level 7
> > > > echolink debug level 7
> > > >
> > > > Unfortunately it seems that when the connection is dropped, no error
> is
> > > > shown on the screen other than the disconnect.
> > > >
> > > > Thoughts?
> > > >
> > > > Thanks to the team.  You have made a great product and the GMARC
> club is
> > > > loving it.  I just want to get the last of the gremlins squashed so
> they
> > > > will love this more!
> > > >
> > > > Regards and 73,
> > > > Mark - AC8LT
> > > > _______________________________________________
> > > >
> > > > arm-allstar mailing list
> > > > arm-allstar at hamvoip.org
> > > > http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> > > >
> > > > Visit the BBB and RPi2/3 web page - http://hamvoip.org
> > > >
> > > _______________________________________________
> > >
> > > arm-allstar mailing list
> > > arm-allstar at hamvoip.org
> > > http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> > >
> > > Visit the BBB and RPi2/3 web page - http://hamvoip.org
> > >
> > _______________________________________________
> >
> > arm-allstar mailing list
> > arm-allstar at hamvoip.org
> > http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
> >
> > Visit the BBB and RPi2/3 web page - http://hamvoip.org
> >
>
>

More information about the arm-allstar mailing list