better TLS/SSL certificate management

[joe]

https://github.com/jjssoftware/asustor-certbot

[joe]

Awesome, Thanks for sharing

No problem, hope it works for you. The hardest thing might be getting pip to install cryptography and certbot without an error because it compiles code as part of the install. From memory I think it basically needs gcc and some other bits installing first. Once that hurdle is overcome it should be fine.

[joe]

In stark contrast to the Asustor certificate renewal process which failed silently without warning or any detailed reasonable explanation from Asustor support:

viewtopic.php?f=23&t=6576&p=30216&hilit=renewal#p30216

..on March 21st 2018, my Let's Encrypt certificate renewed flawlessly via certbot. Daily renewal attempts up to the 21st were logged by certbot as "Cert not yet due for renewal" and the actual successful renewal on the 21st was also fully logged. Following certificate renewal, the certificate was successfully installed on my AS202-TE according to the custom rules that I have configured in my certbot renewal-hooks/deploy script.

Take note Asustor, this is how certificate renewal should be handled. Improve your own implementation or drop it in favour of official integrated support for certbot.

[core]

https://github.com/jjssoftware/asustor-certbot

Nice. I need to try this. It seems like there is missing explanation of where to place scripts and how to set paths in the scripts.

[joe]

https://github.com/jjssoftware/asustor-certbot

Nice. I need to try this. It seems like there is missing explanation of where to place scripts and how to set paths in the scripts.

I didn't include certbot path specifics in the README.md because I had a suspicion (rightly or wrongly) that certbot / letsencrpyt paths might be variable due to supported number of HDDs and the Asustor Linux "special" file system layout differing between the various Asustor NAS box models.

Suggested paths are included within the sample shell scripts in the repo.

I am able to say that following initial letsencrypt certificate creation via certbot on my AS202-TE NAS, I ended up with a letsencrypt file system directory structure being created here:

Code: Select all

/volume0/usr/builtin/etc/letsencrypt

..that looks like this:

Screenshot 2020-06-13.10.59.07.png (40.75 KiB) Viewed 985 times

[core]

Oh I see. Thanks for the extra explanation.

[bbbaton]

From memory I think it basically needs gcc and some other bits installing first. Once that hurdle is overcome it should be fine.

I'm late to the party, but could you please elaborate a bit on this?
I can't seem to "find" any suitable compilers to install.
Where do you even start looking for what you need?

I seem to have been able to install cryptography, but certbot is a whole other story.

admin@NYNAS:/volume1/home/admin $ pip install cryptography --upgrade
Requirement already up-to-date: cryptography in /volume1/.@plugins/AppCentral/python/lib/python2.7/site-packages
Cleaning up...

admin@NYNAS:/volume1/home/admin $ pip install certbot
Downloading/unpacking certbot
Downloading certbot-1.9.0.tar.gz (382kB): 382kB downloaded
Running setup.py egg_info for package certbot
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.py[cod]' found anywhere in distribution
Downloading/unpacking acme>=1.8.0 (from certbot)
Downloading acme-1.9.0.tar.gz (91kB): 91kB downloaded
Running setup.py egg_info for package acme
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.py[cod]' found anywhere in distribution
Downloading/unpacking ConfigArgParse>=0.9.3 (from certbot)
Downloading ConfigArgParse-1.2.3.tar.gz (42kB): 42kB downloaded
Running setup.py egg_info for package ConfigArgParse
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

Downloading/unpacking configobj (from certbot)
Downloading configobj-5.0.6.tar.gz
Running setup.py egg_info for package configobj

Requirement already satisfied (use --upgrade to upgrade): cryptography>=1.2.3 in /volume1/.@plugins/AppCentral/python/lib/python2.7/site-packages (from certbot)
Downloading/unpacking distro>=1.0.1 (from certbot)
Downloading distro-1.5.0.tar.gz (54kB): 54kB downloaded
Running setup.py egg_info for package distro
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)

warning: no files found matching 'CHANGES'
warning: no previously-included files matching '*.py[co]' found anywhere in distribution
Downloading/unpacking josepy>=1.1.0 (from certbot)
Downloading josepy-1.5.0.tar.gz (53kB): 53kB downloaded
Running setup.py egg_info for package josepy
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

Downloading/unpacking parsedatetime>=1.3 (from certbot)
Downloading parsedatetime-2.6.tar.gz (60kB): 60kB downloaded
Running setup.py egg_info for package parsedatetime

no previously-included directories found matching '.DS_Store'
Downloading/unpacking pyrfc3339 (from certbot)
Downloading pyRFC3339-1.1.tar.gz
Running setup.py egg_info for package pyrfc3339

Downloading/unpacking pytz (from certbot)
Downloading pytz-2020.4.tar.gz (310kB): 310kB downloaded
Running setup.py egg_info for package pytz

Requirement already satisfied (use --upgrade to upgrade): setuptools in /volume1/.@plugins/AppCentral/python/lib/python2.7/site-packages/setuptools-2.0.1-py2.7.egg (from certbot)
Downloading/unpacking zope.component (from certbot)
Downloading zope.component-4.6.2.tar.gz (90kB): 90kB downloaded
Running setup.py egg_info for package zope.component

warning: no previously-included files matching '*.dll' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching 'coverage.xml' found anywhere in distribution
no previously-included directories found matching 'docs/_build'
Downloading/unpacking zope.interface (from certbot)
Downloading zope.interface-5.2.0.tar.gz (227kB): 227kB downloaded
Running setup.py egg_info for package zope.interface
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

warning: no previously-included files matching '*.dll' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching 'coverage.xml' found anywhere in distribution
warning: no previously-included files matching 'appveyor.yml' found anywhere in distribution
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'benchmarks'
Downloading/unpacking mock (from certbot)
Downloading mock-4.0.2.tar.gz (71kB): 71kB downloaded
Running setup.py egg_info for package mock

Downloading/unpacking PyOpenSSL>=0.15.1 (from acme>=1.8.0->certbot)
Downloading pyOpenSSL-19.1.0.tar.gz (160kB): 160kB downloaded
Running setup.py egg_info for package PyOpenSSL

warning: no previously-included files found matching 'leakcheck'
warning: no previously-included files found matching 'codecov.yml'
warning: no previously-included files matching '*.py' found under directory 'leakcheck'
warning: no previously-included files matching '*.pem' found under directory 'leakcheck'
no previously-included directories found matching 'doc/_build'
no previously-included directories found matching '.travis'
Downloading/unpacking requests[security]>=2.6.0 (from acme>=1.8.0->certbot)
Downloading requests-2.25.0.tar.gz (101kB): 101kB downloaded
Running setup.py egg_info for package requests
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
warnings.warn(msg)
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)

Installing extra requirements: 'security'
Downloading/unpacking requests-toolbelt>=0.3.0 (from acme>=1.8.0->certbot)
Downloading requests-toolbelt-0.9.1.tar.gz (207kB): 207kB downloaded
Running setup.py egg_info for package requests-toolbelt

no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching '*.py[cdo]' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
Downloading/unpacking six>=1.9.0 (from acme>=1.8.0->certbot)
Downloading six-1.15.0.tar.gz
Running setup.py egg_info for package six
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

no previously-included directories found matching 'documentation/_build'
Downloading/unpacking zope.deferredimport>=4.2.1 (from zope.component->certbot)
Downloading zope.deferredimport-4.3.1.tar.gz
Running setup.py egg_info for package zope.deferredimport

warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking zope.deprecation>=4.3.0 (from zope.component->certbot)
Downloading zope.deprecation-4.4.0.tar.gz
Running setup.py egg_info for package zope.deprecation

warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking zope.event (from zope.component->certbot)
Downloading zope.event-4.5.0.tar.gz
Running setup.py egg_info for package zope.event

warning: no previously-included files matching '*.dll' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.class' found anywhere in distribution
warning: no previously-included files matching '*' found under directory 'docs/_build'
Downloading/unpacking zope.hookable>=4.2.0 (from zope.component->certbot)
Downloading zope.hookable-5.0.1.tar.gz
Running setup.py egg_info for package zope.hookable
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking chardet>=3.0.2,<4 (from requests[security]>=2.6.0->acme>=1.8.0->certbot)
Downloading chardet-3.0.4.tar.gz (1.9MB): 1.9MB downloaded
Running setup.py egg_info for package chardet

warning: no files found matching 'requirements.txt'
Downloading/unpacking idna>=2.5,<3 (from requests[security]>=2.6.0->acme>=1.8.0->certbot)
Downloading idna-2.10.tar.gz (175kB): 175kB downloaded
Running setup.py egg_info for package idna
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)

warning: no previously-included files matching '*.pyc' found under directory 'tools'
warning: no previously-included files matching '*.pyc' found under directory 'tests'
Downloading/unpacking urllib3>=1.21.1,<1.27 (from requests[security]>=2.6.0->acme>=1.8.0->certbot)
Downloading urllib3-1.26.2.tar.gz (286kB): 286kB downloaded
Running setup.py egg_info for package urllib3
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
warnings.warn(msg)
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)
error in urllib3 setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.
Complete output from command python setup.py egg_info:
/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'

warnings.warn(msg)

/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'

warnings.warn(msg)

/usr/local/AppCentral/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'

warnings.warn(msg)

error in urllib3 setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_admin/urllib3
Storing complete log in /home/admin/.pip/pip.log
admin@NYNAS:/volume1/home/admin $

(Newbie here, with some linux experience. I start to remember why I never felt too comfortable with the system...)

[joe]

Hi there,

I'm not going to walk you through all of the problems and hiccups that you might encounter along the way of getting this installed because there can be a fair few things that need to be worked through to get Certbot installed.

One positive thing to say is that this 100% is doable and I am able to say that with confidence because I recently (about 4 months ago) upgraded to Python v3.6 on my AS202-TE and also got the latest version of Certbot successfully installed and working at that time.

I have added a few more details and pointers in the repo that might help you make progress: https://github.com/jjssoftware/asustor- ... llation.md

It would be nice if Asustor officially supported Certbot on their special flavour of linux. I suggested that they do this back in 2018 in a direct support issue but it would appear that they have no interest in doing it. We are where we are as they say.

Good luck

[bbbaton]

After spending several hours, I conceded and decided its not gonna happen.
It feels like running hurdles. Every step, every command, every option is 20 minutes of Google and forums and github posts to get a grip on. And even when you know what to type, it still doesn't work because every single Linux install is apparently unique.

A week or so ago I managed to get that elusive green padlock working with ASUSTOR's built in certificate manager, along with some manual editing of domains in a few config files. Unfortunately newer macs don't approve of this certificates, and a few friends just couldn't connect at all.

But to get my own certificates installed and configured... It's just not worth the head ache.

Unless I could create the certificates on a windows machine on the same network, and somehow get it onto the nas?

(perhaps asking that question has revealed my absolute ineptitude on this matter)

[joe]

There are no silly questions and your idea could work to be fair.

One thing that comes to mind with the idea of delegating LetsEncrypt certificate renewal via Certbot to some machine other than the NAS is that: 1) as you identified you would need to get the renewed cert onto the NAS at cert renewal time and 2) in addition you would need to restart services on the NAS that use the cert so that those services effectively "see" and serve the renewed cert.

On my NAS box I have a few things that need to be restarted when certs renew i.e. Asustor ADM, Shellinabox, NodeRed, Plex and this gets handled by the letsencrypt renewal-hook script which is just a regular simple shell script that gets called when the cert successfully renews.

Cert renewal is generally a thing that people like to have happen in an automated way and there certainly could be a way to automate this when cert renewal is performed off the NAS.