Asterisk 16.14.0 Pjsip Transport-tls Cert Parsing Error
Hi,
I’m experimenting with Asterisk-16.14.0 on a CentOS7 box, and run into problems loading the SSL certificate to establish transport-tls. Tried self-signed certificate generated with ast_tls_cert under contrib/scripts and the one issued by Letsencrypt, both would bomb out with a parsing error:
[Dec 3 15:47:50] ERROR[11233] res_pjsip/config_transport.c: Transport:
transport-tls: cert_file /home/asterisk/certs/asterisk.crt is either missing or not readable
[Dec 3 15:47:50] ERROR[11233] config_options.c: Error parsing cert_file=/home/asterisk/certs/asterisk.crt at line 24 of
What’s interesting is that the self-signed asterisk.crt only has 20 lines. For letsencrypt certificate (both cert.pem and fullchain.pem), it’d bomb out at line 22.
Here’s the transport section of my /etc/asterisk/pjsip.conf:
[transport-udp]
type = transport
protocol = udp
bind = 0.0.0.0
[transport-tls]
type = transport
protocol = tls
bind = 0.0.0.0
;cert_file = /home/asterisk/certs/cert.pem
;cert_file = /home/asterisk/certs/fullchain.pem
;priv_key_file = /home/asterisk/certs/privkey.pem
cert_file = /home/asterisk/certs/asterisk.crt
priv_key_file = /home/asterisk/certs/asterisk.key
allow_reload = true
And a full listing of /home/asterisk/certs:
-rw-r—–. 1 asterisk asterisk 1212 Dec 2 17:19 asterisk.crt
-rw-r—–. 1 asterisk asterisk 578 Dec 2 17:18 asterisk.csr
-rw-r—–. 1 asterisk asterisk 891 Dec 2 17:18 asterisk.key
-rw-r—–. 1 asterisk asterisk 2103 Dec 2 17:19 asterisk.pem
-rw-r—–. 1 asterisk asterisk 1749 Dec 2 17:18 ca.crt
-rw-r—–. 1 asterisk asterisk 3311 Dec 2 17:18 ca.key
-rw-r—–. 1 asterisk asterisk 1923 Nov 13 16:29 cert.pem
-rw-r—–. 1 asterisk asterisk 3570 Nov 13 15:11 fullchain.pem
-rw-r—–. 1 asterisk asterisk 1704 Nov 13 15:12 privkey.pem
The self-sign asterisk.crt:
—–BEGIN CERTIFICATE—
13 thoughts on - Asterisk 16.14.0 Pjsip Transport-tls Cert Parsing Error
Hi,
I’m experimenting with Asterisk-16.14.0 on a CentOS7 box, and run into problems loading the SSL certificate to establish transport-tls. Tried self-signed certificate generated with ast_tls_cert under contrib/scripts and the one issued by Letsencrypt, both would bomb out with a parsing error:
[Dec 3 15:47:50] ERROR[11233] res_pjsip/config_transport.c: Transport:
transport-tls: cert_file /home/asterisk/certs/asterisk.crt is either missing or not readable
[Dec 3 15:47:50] ERROR[11233] config_options.c: Error parsing cert_file=/home/asterisk/certs/asterisk.crt at line 24 of
What’s interesting is that the self-signed asterisk.crt only has 20 lines. For letsencrypt certificate (both cert.pem and fullchain.pem), it’d bomb out at line 22.
Here’s the transport section of my /etc/asterisk/pjsip.conf:
[transport-udp]
type = transport
protocol = udp
bind = 0.0.0.0
[transport-tls]
type = transport
protocol = tls
bind = 0.0.0.0
;cert_file = /home/asterisk/certs/cert.pem
;cert_file = /home/asterisk/certs/fullchain.pem
;priv_key_file = /home/asterisk/certs/privkey.pem
cert_file = /home/asterisk/certs/asterisk.crt
priv_key_file = /home/asterisk/certs/asterisk.key
allow_reload = true
And a full listing of /home/asterisk/certs:
-rw-r—–. 1 asterisk asterisk 1212 Dec 2 17:19 asterisk.crt
-rw-r—–. 1 asterisk asterisk 578 Dec 2 17:18 asterisk.csr
-rw-r—–. 1 asterisk asterisk 891 Dec 2 17:18 asterisk.key
-rw-r—–. 1 asterisk asterisk 2103 Dec 2 17:19 asterisk.pem
-rw-r—–. 1 asterisk asterisk 1749 Dec 2 17:18 ca.crt
-rw-r—–. 1 asterisk asterisk 3311 Dec 2 17:18 ca.key
-rw-r—–. 1 asterisk asterisk 1923 Nov 13 16:29 cert.pem
-rw-r—–. 1 asterisk asterisk 3570 Nov 13 15:11 fullchain.pem
-rw-r—–. 1 asterisk asterisk 1704 Nov 13 15:12 privkey.pem
The self-sign asterisk.crt:
—–BEGIN CERTIFICATE—
It’s missing or not readable! Take care, that the file access rights of the file and the complete path are ok. Do a strace to verify, if the file is really loaded at all.
Michael
—
Thanks Michael for the suggestion! I’ve installed strace and assigned one of the endpoints (SOFTPHONE_B) to use transport-tls. Then run strace (as user asterisk):
[asterisk@voip1 ~]$ strace asterisk -rx “module reload res_pjsip.so”
execve(“/usr/sbin/asterisk”, [“asterisk”, “-rx”, “module reload res_pjsip.so”], 0x7fff2ec172a0 /* 18 vars */) = 0
brk(NULL) = 0x1a7e000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) 0x7f4802805000
access(“/etc/ld.so.preload”, R_OK) = -1 ENOENT (No such file or directory)
open(“/etc/ld.so.cache”, O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_sizeY200, …}) = 0
mmap(NULL, 59200, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f48027f6000
close(3) = 0
open(“/lib/libasteriskssl.so.1”, O_RDONLY|O_CLOEXEC) = 3
read(3,
“\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\n\0\0\0\0\0\0″…, 832)
= 832
fstat(3, {st_mode=S_IFREG|0755, st_sizey9664, …}) = 0
mmap(NULL, 2105504, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) 0x7f48023e2000
mprotect(0x7f48023e4000, 2093056, PROT_NONE) = 0
mmap(0x7f48025e3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f48025e3000
close(3) = 0
open(“/lib/libasteriskpj.so.2”, O_RDONLY|O_CLOEXEC) = 3
read(3,
“\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300Y\3\0\0\0\0\0″…, 832)
= 832
fstat(3, {st_mode=S_IFREG|0755, st_size
You should use strace like this as root and from the very beginning of the start of asterisk:
strace -f -o /tmp/strace.log asterisk -vvv -mqf -C /etc/asterisk/asterisk.conf
-f means, to follow even forked processes, … (see man page)
-o writes all the output to a file. You can search afterwards pretty easily for the file (or the open call).
You shouldn’t do this in production but in the test environment!
You have to run it as long as the error has happened.
Thanks Michael
—
Sorry – my wrong – not necessarily as root – it should be started the same way and in the same context as it runs normally.
—
I saved that file in “x.crt”.
openssl x509 -in x.crt -noout -text
….
RSA Public-Key: (1024 bit)
….
I saved that file in “y.crt”.
openssl x509 -in y.crt -noout -enddate notAfter=Jan 29 01:24:25 2021 GMT
That makes no sense.
Which version of openssl ist used on that CentOS7 box ?
In “/etc/ssl/openssl.cnf” you find something like this:
MinProtocol = TLSv1.2
CipherString = DEFAULT@SECLEVEL=2
You could set the level to “1” or even to “0” and restart Asterisk.
—
Stefan Tichy
—
Thanks Stefan for the pointer.
There isn’t a /etc/ssl/openssl.cnf on the CentOS7 box. There is a
/etc/pki/tls/openssl.cnf, but there’s no MinProtocol or CipherString defined there. I installed corebot (for Letsencrypt auto renewal) thru snap. The openssl.cnf that comes with snap (under
/var/lib/snapd/snap/core/current/etc/ssl) is pretty similar to the one under /etc/pki/tls, in both lacking MinProtocol and CipherString definitions.
[root@voip1 ~]# openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017
if it helps with anything.
Thanks,
–Ruisheng
Thanks for the detailed explanation Michael.
I stop the current asterisk process (started by systemd), and restart it as asterisk:
[asterisk@voip1 ~]$ strace -f -o /home/asterisk/strace.log asterisk -fmq
-vvv -C /etc/asterisk/asterisk.conf
from the log there was no attempt to even open the cert file. I edited
/etc/asterisk/pjsip.conf to add a “method = tlsv1” line to the transport-tls section. Rerun the strace command, and here the part re cert files:
8189 stat(“/home/asterisk/certs/asterisk.crt”, {st_mode=S_IFREG|0640, st_size=1
212, …}) = 0
8189 geteuid() = 1002
8189 getegid() = 1002
8189 getuid() = 1002
8189 getgid() = 1002
8189 access(“/home/asterisk/certs/asterisk.crt”, R_OK) = 0
8189 stat(“/home/asterisk/certs/asterisk.key”, {st_mode=S_IFREG|0640, st_size=8
91, …}) = 0
8189 geteuid() = 1002
8189 getegid() = 1002
8189 getuid() = 1002
8189 getgid() = 1002
8189 access(“/home/asterisk/certs/asterisk.key”, R_OK) = 0
8189 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
8189 setsockopt(16, SOL_SOCKET, 0xffff /* SO_??? */, [1], 4) = -1
ENOPROTOOPT (
Protocol not available)
8189 setsockopt(16, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
8189 setsockopt(16, SOL_TCP, TCP_NODELAY, [1], 4) = 0
The tls transport is not established in the end.
beating around bushes, and finally seem to stomp on something that worked!
Simply move the cert file locations from /home/asterisk/certs to
/etc/asterisk/keys
[root@voip1 asterisk]# ls -l keys
total 36
-rw-r—–. 1 asterisk asterisk 1212 Jan 29 14:18 asterisk.crt
-rw-r—–. 1 asterisk asterisk 578 Jan 29 14:18 asterisk.csr
-rw-r—–. 1 asterisk asterisk 891 Jan 29 14:18 asterisk.key
-rw-r—–. 1 asterisk asterisk 2103 Jan 29 14:18 asterisk.pem
-rw-r—–. 1 asterisk asterisk 1749 Jan 29 14:18 ca.crt
-rw-r—–. 1 asterisk asterisk 3311 Jan 29 14:18 ca.key
-rw-r—–. 1 asterisk asterisk 1923 Jan 29 14:18 cert.pem
-rw-r—–. 1 asterisk asterisk 3570 Jan 29 14:18 fullchain.pem
-rw-r—–. 1 asterisk asterisk 1704 Jan 29 14:18 privkey.pem
and tls was established. With self-sign cert, I’d need to add ca_list_file in the transport-tls section in /etc/pjsip.conf for it to fly.
[transport-tls]
type = transport
protocol = tls
bind = 0.0.0.0:5061
; ca_list_file = /etc/asterisk/keys/ca.crt
; cert_file = /etc/asterisk/keys/asterisk.crt
; priv_key_file = /etc/asterisk/keys/asterisk.key
cert_file = /etc/asterisk/keys/fullchain.pem
priv_key_file = /etc/asterisk/keys/privkey.pem
method = tlsv1_2
allow_reload = true
Not sure what was the nature of the problem. Maybe Selinux? There was no complaint from that department though.
Thanks for the help and suggestions,
–Ruisheng
I’m missing the “open” (or “openat”) and the following “read” call – weren’t there any or didn’t you post them? These are the important calls! They will show, if the file is used at all or not (and possibly the reason, why it is not used – EACCESS
e.g.).
Thanks Michael
—
Hi,
This error means that the file either does not exist or that Asterisk is not able to open it for reading. In your case it looks like the file exists so the Asterisk process was not able to read the file (this could be permissions or SELinux or whatever other reason). It never gets to actually trying to parse it as a certificate.
The subsequent message mentioning “at line 24 of” is just a bug in the configuration framework, it is not referring to line 24 of the certificate file.
Kind regards, Sean
Michael,
There weren’t any open or openat actions on the cert files (located under
/home/asterisk/certs). The same is true for cert files located under
/etc/asterisk/keys:
24138 stat(“/etc/asterisk/keys/fullchain.pem”, {st_mode=S_IFREG|0640, st_size4
44, …}) = 0
24138 geteuid() = 1002
24138 getegid() = 1002
24138 getuid() = 1002
24138 getgid() = 1002
24138 access(“/etc/asterisk/keys/fullchain.pem”, R_OK) = 0
24138 stat(“/etc/asterisk/keys/privkey.pem”, {st_mode=S_IFREG|0640, st_size04
, …}) = 0
24138 geteuid() = 1002
24138 getegid() = 1002
24138 getuid() = 1002
24138 getgid() = 1002
24138 access(“/etc/asterisk/keys/privkey.pem”, R_OK) = 0
24138 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 16
24138 setsockopt(16, SOL_SOCKET, 0xffff /* SO_??? */, [1], 4) = -1
ENOPROTOOPT (
Protocol not available)
24138 setsockopt(16, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
24138 setsockopt(16, SOL_TCP, TCP_NODELAY, [1], 4) = 0
24138 bind(16, {sa_family
Thanks Sean for the note. It does look Selinux might have a hand in the pot. I did try with selinux permission set to permissive and it made no difference though. Keeping configuration related stuff under /etc/asterisk seems to help.
–Ruisheng