Strange Problem With PRI On 64-bit?
I have some more investigation to do on this, but I wanted to see if anyone here had any insight into the issue I’ve run into.
The hardware is a HP DL360 G6 with a TE420 gen 5 4-port T1 PRI card. It is one of several systems that have been running without issue since 2010/2011. They have all been running CentOS 4 32-bit with Zaptel 1.4.12.1 (with patch for gen
5 card), libpri 1.2.8 and asterisk 1.2.32.
Having taken this particular system out of production, I updated it to CentOS
6.9 32-bit, with DAHDI 2.11.1, LibPRI 1.6.0 and Asterisk 11.25.3 (this version of Asterisk is required at the moment due to custom modifications). This appears to work fine.
In order to reduce the number of different versions we support, I reinstalled the OS using the 64-bit version of CentOS 6.9 instead, and rebuilt, using the same versions as above.
However, for reasons I don’t understand, the 64-bit version was logging frequent PRI errors every few minutes:
[Apr 1 03:40:52] VERBOSE[8989] chan_dahdi.c: PRI Span: 2 TEI=0 MDL-ERROR (A): Got supervisory frame with F=1 in state 7(Multi-frame established)
[Apr 1 03:40:58] VERBOSE[8988] chan_dahdi.c: PRI Span: 1 TEI=0 MDL-ERROR (A): Got supervisory frame with F=1 in state 7(Multi-frame established)
[Apr 1 03:44:06] VERBOSE[8990] chan_dahdi.c: PRI Span: 3 TEI=0 MDL-ERROR (A): Got supervisory frame with F=1 in state 7(Multi-frame established)
[Apr 1 03:46:38] VERBOSE[8990] chan_dahdi.c: PRI Span: 3 TEI=0 MDL-ERROR (A): Got supervisory frame with F=1 in state 7(Multi-frame established)
[Apr 1 03:47:20] VERBOSE[8988] chan_dahdi.c: PRI Span: 1 TEI=0 MDL-ERROR (A): Got supervisory frame with F=1 in state 7(Multi-frame established)
[Apr 1 03:47:24] VERBOSE[8989] chan_dahdi.c: PRI Span: 2 TEI=0 MDL-ERROR (A): Got supervisory frame with F=1 in state 7(Multi-frame established)
This left the PRIs in strange states – trying to make a call failed with cause 101.
So I re-installed the 32-bit OS again, and rebuilt, and the above MDL-ERRORs were no longer present, and the system operated normally again.
So my question is: does anyone have any clues why there would be a difference in PRI behaviour between 32-bit and 64-bit builds? Has anyone else run into anything similar?
Cheers Tony
10 thoughts on - Strange Problem With PRI On 64-bit?
That does seem quite odd. If I remember right, those messages would come up if it looked like the other end hadn’t received a message when it thought it should have. I can’t think of anything that would particularly impact 64 bit systems versus 32 bit systems in that domain (ISDN real time message timing, etc). Are you sure there’s nothing else different (kernel version or something else like that)?
Maybe also run a patlooptest on the spans in question to make sure that they’re running cleanly.
Matthew Fredrickson
In article, Matt Fredrickson wrote:
Hi Matt, thanks for the reply.
Both the 32-bit and 64-bit were fresh installs of the latest CentOS 6.9 from online repositories using a kickstart build. I’m going to try installing the
64-bit version again tomorrow to see if the problem re-appears, just to be certain it wasn’t anything transient. I don’t think there is anything unclean about the spans, because they were running fine on CentOS 4 with the versions I mentioned, and are now running well again with 32-bit CentOS 6.
Cheers Tony
Hey Tony,
The paylooptest recommendation wasn’t necessarily about screening hardware problems but weeding out that there weren’t any potential driver issues on the 64bit install.
The libpri makefile doesn’t install things for 64 bit systems in the right place [1]
without your help. You’ll need to specify where to install the library on the command line for your system:
sudo make install libdir=/usr/lib64
Richard
[1] https://issues.asterisk.org/jira/browse/PRI-100
Isn’t this handled by the CentOS package manager?
Antony.
In article, Matt Fredrickson wrote:
Ok, sure. I assumed that DAHDI would have had a lot of use on 64-bit by now, especially since RHEL 7 is 64-bit only (although I believe CentOS 7 has been made available also in 32-bit).
I’ve just done a 64-bit rebuild again, so will do some tests.
Thanks Tony
In article, Richard Mudgett wrote:
Ah, thanks. I did in fact discover the following 64-bit libraries were installed into /usr/lib instead of /usr/lib64:
1. From DAHDI, libtonezone.so
2. From LibPRI, libpri.so
3. From Asterisk, libasteriskssl.so
I found that running “ldconfig” caused them all to be discovered:
[root@bridge05 ~]# ldd /usr/sbin/asterisk
linux-vdso.so.1 => (0x00007ffc77ff9000)
libasteriskssl.so.1 => /usr/lib/libasteriskssl.so.1 (0x00007efeae1d4000)
libc.so.6 => /lib64/libc.so.6 (0x00007efeade40000)
libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00007efeadaed000)
libz.so.1 => /lib64/libz.so.1 (0x00007efead8d7000)
libm.so.6 => /lib64/libm.so.6 (0x00007efead653000)
libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x00007efead3c4000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007efead158000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007efeacd73000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007efeacb6f000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007efeac952000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007efeac731000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007efeac517000)
/lib64/ld-linux-x86-64.so.2 (0x00007efeae3d6000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007efeac2d3000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007efeabfec000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007efeabde8000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007efeabbbc000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007efeab9b1000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007efeab7ae000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007efeab58f000)
[root@bridge05 ~]# ldd /usr/sbin/dahdi_cfg
linux-vdso.so.1 => (0x00007fff6cbaa000)
libtonezone.so.2 => /usr/lib/libtonezone.so.2 (0x00007f862f74a000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f862f52d000)
libm.so.6 => /lib64/libm.so.6 (0x00007f862f2a9000)
libc.so.6 => /lib64/libc.so.6 (0x00007f862ef15000)
/lib64/ld-linux-x86-64.so.2 (0x00007f862f97e000)
[root@bridge05 ~]# ldd /usr/lib/asterisk/modules/chan_dahdi.so
linux-vdso.so.1 => (0x00007ffe8b1df000)
libtonezone.so.2 => /usr/lib/libtonezone.so.2 (0x00007f54adde4000)
libpri.so.1.4 => /usr/lib/libpri.so.1.4 (0x00007f54adb68000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f54ad94b000)
libc.so.6 => /lib64/libc.so.6 (0x00007f54ad5b7000)
libm.so.6 => /lib64/libm.so.6 (0x00007f54ad333000)
/lib64/ld-linux-x86-64.so.2 (0x00007f54ae2d3000)
[root@bridge05 ~]#
So I assumed that all should be ok, otherwise the executables would fail to run
(I initially discovered this when dahdi_cfg couldn’t find libtonezone).
Would there be any subtle issues with the 64-bit libraries being loaded from /usr/lib instead of /usr/lib64?
Should Asterisk and DAHDI builds also be updated to use /usr/lib64 when building on a 64-bit OS? Or the build instructions?
Regards Tony
In article, Tony Mountifield wrote:
… And this time, I am not getting the MDL errors that I was getting last time. So it looks like 64-bit is ok (which is what I would expect!), the trunks must have got into an unexpected state, and I didn’t try hard enough to get them to reset. I have now made several test calls in and out without problems.
So apologies for the false alarm, and I appreciate the feedback!
Cheers Tony
dahdi-tools 2.11 now uses autoconf. It still installs to /usr/lib or is it an older version?
dahdi-tools: not AFAIK.
In article <20180404133024.kpidrkuiyjoqd3ox@xorcom.com>, Tzafrir Cohen wrote:
I compiled dahdi-linux-complete-2.11.1+2.11.1, by doing:
make make install make config make -C tools install-config
In fact I did all the above with a DESTDIR=$DESTDIR appended to each line, as I was building a binary bundle for system building.
Before doing so I also did:
mkdir -p $DESTDIR/etc/udev/rules.d $DESTDIR/etc/rc.d/init.d $DESTDIR/etc/sysconfig/network-scripts
Maybe that defeated autoconf, and made it default to /usr/lib?
If I had also done a mkdir $DESTDIR/usr/lib64 before building, maybe autoconf would have found it?
But in any case, running ldconfig made everything get found:
[root@bridge05 ~]# ldconfig -p | fgrep -v lib64
552 libs found in cache `/etc/ld.so.cache’
libtonezone.so.2 (libc6,x86-64) => /usr/lib/libtonezone.so.2
libtonezone.so (libc6,x86-64) => /usr/lib/libtonezone.so
libpri.so.1.4 (libc6,x86-64) => /usr/lib/libpri.so.1.4
libpri.so (libc6,x86-64) => /usr/lib/libpri.so
libasteriskssl.so.1 (libc6,x86-64) => /usr/lib/libasteriskssl.so.1
libasteriskssl.so (libc6,x86-64) => /usr/lib/libasteriskssl.so
[root@bridge05 ~]#
Cheers Tony