You are not logged in.

#1 2020-04-14 19:40:26

Route99
Member
Registered: 2020-04-13
Posts: 10

[Tutorial] Making your cloud WebDAV server on BunsenLabs

Making your Cloud on BunsenLabs Linux Helium using the WebDAV protocol

I'd like to share some latest experiments to create the titled setup, that was put in a tutorial.
I've "developed" this procedure for Debian 10 and used for several Debian-based distros, how you can read below.

Your feedback is welcome.

English is not my native language, but my pro beta background in chemistry helps....
I will update this text when needed: your feedback, typing/copy-paste issues, updates after experiments.

The situation
Since 2011 I’m running a Debian "cloud" WebDAV-based server on our systems (*) using the WebDAV protocol. The main user is my son being a professional independent documentary photographer who had a range of external backup hard disks and a very small NAS to store his work.
My Linux experience, learned in an auto-didactic manner, just buy a Linux book with a CD  those days and get some old PC (486DX type) started in the middle of the nineties to make an own server/router setup  sharing the cable internet connection with the neighbors in those days. In 2011 I picked up Linux again and built for my son 2 cloud servers based upon the WebDAV protocol, world-wide accessible via port 443 using a WebDAV client and mount the data disks as local disks (easy on MacOS/Linux, Netdrive V1.3 last free version, for many people the easiest method in Windows). For obvious reasons the 2 servers are physically on different addresses.

The challenge?
The hardware is now nearly 9 years young, but still running pretty OK.. first on Debian 6 (Squeeze), and later reasonable on Debian 8 (Jesse). We are almost at the end of the LTS of Debian 8 (June 2020) and so it is desirable to secure new installations in time. Will Debian 10 (Buster) reasonable in performance as Jesse now or? So it may be interesting to test also some lighter Debian based distributions and use Debian 9 or 10 as a reference. We need a small desktop to be able to run in some cases a raw photo viewer which is done by RawTherapee with satisfactory results.

In order to speed up testing, everything was tested first in a virtual machine (Oracle Virtual 6.0 Box ), mainly to test the software first, especially the configuration of Apache and WebDAV  as going from Squeeze to Jesse, changes where also present. In a later stage the firewall using nftables will also be configured.

So far I’ve tested several Debian based/derived distros:
• Debian 10 (Buster)
• MX-Linux-19
• Anti-X 9
• BunsenLabs Linux Helium

Condensed results so far:
Debian 10, MX-Linux-19 and BunsenLabs Linux Helium, could be setup quite easy according the procedure below that was first established with Debian 10 only as a reference.  Anti-X 9 needs some pre-work to use the same procedure. This pre-work will be left out here on purpose as this is a BunsenLabs forum. On request the info can be added.  I started with Buster, it would be more correct to compare Bunsenlaps with Stretch. My initial plan was “going 10-based”, but I was impressed by BunsenLabs. We are very much used to Mate, so that was installed afterwards for some more ease for my son.

To-do:
Finish the firewall work. My current nftables config seems to work OK under Debian 10 and BunsenLabs Linux Helium and yet not under the other distro’s. Would be nice to finish that so I can later do a full comparison on the target system.
And: The proof of the pudding is in the eating...., so test it on our target system instead of the VM's(*). 

The used procedure status 14-04-2020
For the installation the terminal was used. For changing the configuration file(s) I preferred Gedit, but use what you like yourself. I recall that Gedit needed to be installed in the usual way. For using Gedit: Open a second terminal and you can call that handy tool with root rights as:

sudo gedit 

There was a moment, need to repeat it for better reporting …, one command using sudo could not be effectuated. Then it was tried as root with “su”, but the root account is locked in  BunsenLabs. It worked after unlocking:

sudo passwd root

1. User setup for later WebDAV  configuration
Of course, the (extra?) user that will use WebDAV must exist or use only the current user, so first

adduser user

Add the user to the www-data group, the group holding the WebDAV space on your disk:

usermod -a -G www-data user

2. Installing Apache & SSL
Install Apache     

sudo apt-get update
sudo apt-get install apache2

Install SSL key/certificates

sudo openssl genrsa -des3 -out server.key 1024

Read the messages in the terminal carefully and fill out where necessary!         

sudo openssl req -new -key server.key -out server.csr

The key is valid for 365 days: Change “365” to your needs.

sudo openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

Copy to the file location

sudo cp server.crt /etc/ssl/certs
sudo cp server.key /etc/ssl/private

Activate SSL               

sudo a2enmod ssl
sudo a2ensite default-ssl

It is a kind of mandatory restart to check Apache before to proceed with the WebDAV installation:   

sudo /etc/init.d/apache2 restart

Test the webserver: Login as the intended user with your password

https://127.0.0.1

Notes:
Instead of a key based on 1024 Bit it is safer to use 2048 or 4096 Bit (the 1024 test was faster).
Now you have your webserver ready to continue for the installation of WebDAV.
Apache webserver works


3. Installing WebDAV with Digest Authentication
In the past “Basic” instead of  "Digest Authentication" was used. Some extra safety measures now.

Note: By default, WebDAV is installed at this location:

/var/www/webdav

We already had a data folder in the root of the disk:

/data

So we created the Webdav folder there on:

/data/webdav

Activate the Webdav modules

sudo a2enmod dav_fs
sudo a2enmod dav

Go to the root of the file system, need to use "su", so you maybe need to unlock the root:

cd /

Create the desired file location for Webdav:

mkdir data
mkdir data/webdav

Set the rights for the web server for the WebDAV user:

chown www-data:user /data
chown www-data:user /data/webdav
chown www-data /data/webdav

Note: Check whether your user is a member of the www-data group, the  holder of the WebDAV space on your disk, so add the user to this group if it is not done/known:

usermod -a -G www-data user

For the WebDAV folder: Check the owner of the group and user

ls -dl /data/webdav

The output will look like for www-data being the group owner and me  being the "user":

drwxr-xr 2 www-data route99 4096 mrt 28 19:26 /data/webdav

Activate the “Digest Authentication” method and add a password:

sudo a2enmod auth_digest

Create folder where the password is stored:

sudo mkdir /etc/password

Create password with AuthName WebDav for user:

sudo htdigest -c /etc/password/digest-password WebDav user

Note: The “AuthName WebDav” is used in the default-ssl file discussed below. Create a strong password, especially if you access the webdav space from outside your local network by forwarding the 433 port to the IP-address of this local WebDAV server (How? See the manual of your modem/router).

Adapt the default-ssl file

/etc/apache2/sites-enabled/default-ssl

Look up this text in the default-ssl file

CustomLog $ {APACHE_LOG_DIR} /access.log combined

AFTER this line, after first creating a new line with the ENTER key, add this text:

Alias /webdav /data/webdav
#
<Directory /data/webdav/>
 Options Indexes MultiViews
 AllowOverride None
 Order allow,deny
 allow from all
</Directory>
#
<Location /webdav>
 DAV On
 AuthType Digest
 AuthName "WebDav"
 AuthUserFile /etc/password/digest-password
 Require valid-user
		</Location>

And save the default-ssl file!
Note: If you use the original  WebDAV location

/home/user/webdav

Then use this modification of the default-ssl file:

Alias /webdav /home/user/webdav
#
<Directory /home/user/webdav/>
Options Indexes MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
#
<Location /webdav>
DAV On
AuthType Digest
AuthName "WebDav"
AuthUserFile /etc/password/digest-password
Require valid-user
</Location>

Restart Apache to test the WebDAV server:         

sudo /etc/init.d/apache2 restart

Test the WebDAV server: Login as the intended user with your password

https://127.0.0.1/webdav

When you see something like the picture: Congratulations! It is working.
WebDAV server working

(*) Our 2 server systems, the main components:
• Motherboard: Asus E35M1-I (onboard CPU AMD Fusion E-350 en 6x SATA)
• RAM 8GB DDR3: Kingston 2x4GB, 1066MHz, DDR3, Non-ECC, CL7, DIMM
• 10-12 TB HD’s: 2TB Samsung Spinpoint HD204UI  & HD203WI (5400 rpm) + a 3TB Toshiba DT01ACA300

References:
• Used iso’s:
Debian-live-10.3.0-amd64-mate.iso   
MX-19.1_x64.iso               
AntiX-19.1_x64-base.iso   
Helium-5-amd64.hybrid.iso
• Homepage Debian:             https://www.debian.org/
• Homepage MX-Linux            https://mxlinux.org/
• Homepage Anti-X              https://antixlinux.com/
• Homepage BunsenLabs          https://www.bunsenlabs.org/
• Inspired by several WebDAV reviews, setup pages, forums 
• Inspired for my first Debian setup:  http://www.bernaerts-nicolas.fr/linux/7 … bdav-share

Last edited by Route99 (2020-04-15 20:27:58)

Offline

#2 2020-04-15 05:57:34

ohnonot
...again
Registered: 2015-09-29
Posts: 4,753
Website

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

WOW!  yikes

A few general questions about WebDAV (which I'm not familiar with):

  • The last picture looks just like serving the files through Apache. I’m guessing the whole point of WebDAV is to have also write/create(=upload) access? How?

  • Does it have to be accessed from the web browser?

  • I’m guessing it needs to be accessible from non-*nix machines? Otherwise I’d just choose NFS or sshfs?


BL quote proposals to this thread please.
how to ask smart questions | my repos / my repos | my blog
---
Thank you for posting direct image links!

Offline

#3 2020-04-15 06:43:43

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

ohnonot wrote:

WOW!  yikes

Well it took me from 2011 till now to get these results... wink
Thank you smile

The last picture looks just like serving the files through Apache. I’m guessing the whole point of WebDAV is to have also write/create(=upload) access? How?

* Does it have to be accessed from the web browser?

Apache is the server environment you need. The WebDAV protocol, working only on top of Apache, is a much safer method than using (S)FTP or the old SMB (SAMBA) sharing.
So I also choose for HTTPS port 443 and not for HTPP port 80.
Extra safety. So far we never had any safety issue starting from 2011.  Ok, not an absolute guarantee, but I've chosen based upon was published, the best version. For this reason I've chosen to update  WebDAV with Digest Authentication.

When you setup WebDAV as described you can up- and download data in several ways, a few:
* Mount the webdav server as a local disk using a WebDAV program like Netdrive.
* Mount it through command-line
* Use system tools as present in MacOS
 
For downloading only: You may use the browser, that was my picture.
 
On mobile phones: I use a simple app under iOS: WebDav Nav, the free version. There is also a paid version with more options.
https://apps.apple.com/nl/app/webdav-na … d382551345
Also free or paid available for Android https://play.google.com/store/apps/deta … vnav&hl=nl

*I’m guessing it needs to be accessible from non-*nix machines? Otherwise I’d just choose NFS or sshfs

Yes, you can use it on non-*nix machines.
https://en.wikipedia.org/wiki/WebDAV

Client support

  • Git supports writing to HTTP remotes, although the

  • Linux via GVfs, including GNOME Files and via KIO, including Konqueror and Dolphin

  • macOS, including native support for CalDAV and CardDAV, the design of which is based on WebDAV

  • Microsoft Windows, including native support in Explorer

  • Microsoft Office

The native support under Windows using a command-line approach is sometimes difficult.
So far the 3rd party Netdrive V1.3 tool works great here. But often this must be done:Under Control Panel \ All Control Panel Items \ Internet Options under LAN Settings uncheck “Automatically detect settings”

Under Windows the Netdrive result will look like in de picture below. Netdrive V1.3 is the last free version, I use it myself too. PB me if you need a Wetransfer link for this V1.3.
netdrive2.jpg

Last edited by Route99 (2020-04-15 08:25:56)

Offline

#4 2020-04-15 08:35:56

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,892

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

Looks good to me, but

Install SSL key/certificates

sudo openssl genrsa -des3 -out server.key 1024

DES is weak crypto; you should be using something similar to this

openssl genrsa -aes256 -out server.key 2048
openssl genrsa -aes128 -out server.key 2048

Note that you can also do the self-signed cert generation process in one command:

openssl req -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365

with sufficiently recent versions of OpenSSL.


Per aspera ad astra.

Offline

#5 2020-04-15 08:53:13

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

Thanks for your feedback! Good points. cool
I will test and update it asap.

Last edited by Route99 (2020-04-15 10:32:21)

Offline

#6 2020-04-15 10:33:57

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,892

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

Route99 wrote:

Thanks for your feedback! Good points.:cool:
I will test and update it asap.

I almost forgot. In order to get a self-signed cert out of the box in Debian, it is sufficient to

sudo apt install ssl-cert

Then you will find pre-generated best-practice SSL certs ready to include in the web server in /etc/ssl/certs/ssl-cert-snakeoil.pem for the cert and /etc/ssl/private/ssl-cert-snakeoil.key for the secret key.


Per aspera ad astra.

Offline

#7 2020-04-15 11:11:48

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

Ok, but  I did not know they where good enough te be used, so in 2010 I "followed "my inspiration" weblink, last reference.

So I can skip this part:

Install SSL key/certificates

sudo openssl genrsa -des3 -out server.key 1024

Read the messages in the terminal carefully and fill out where necessary!         

sudo openssl req -new -key server.key -out server.csr

The key is valid for 365 days: Change “365” to your needs.

sudo openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

Copy to the file location

sudo cp server.crt /etc/ssl/certs
sudo cp server.key /etc/ssl/private

and just use your proposal code:

twoion wrote:
sudo apt install ssl-cert

and then those defaults are already on the intended file location.

Offline

#8 2020-04-15 11:32:59

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

@ohnonot]
THE ease of mounting a WebDAv space on your computer, whatever your OS is:

You can do on the locally WebDAV-mounted space nearly everything similar to what you are used on a normal local physical disk/partition.

So in principle nearly all of the right mouse options, the copy/paste behavior, etc.

Using SFTP/FTP is quite limited in such options and are much less secure, including also SMB (SAMBA), which I never considered to use.

An important addition to the mobile apps:
In principle I avoid the use of our external IP-address by using a VPN tunnel to my home. 
So the WebDAV Nav app contains only the internal ip address where it should connect to. I can only approach the webDAV server when my VPN is "on" on my mobile phone.
The VPN is working on a Raspberry 3B+ which also runs PiHole to filter "wrong DNS" addresses.

Of course I've tested the access from outside our network to the WebDAV server and that works too. It is a personal choice to make it work like it is now.

Last edited by Route99 (2020-04-15 11:50:17)

Offline

#9 2020-04-15 11:59:59

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 767

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

Route99 wrote:

So in principle nearly all of the right mouse options, the copy/paste behavior, etc.

Using SFTP/FTP is quite limited in such options and are much less secure, including also SMB (SAMBA), which I never considered to use.

Ohnononononono is talking about sshfs (which is based on ssh; or one could say on sftp). It has full support for all mouse stuff you are talking about (for example in Thunar). Actually, I'm using it in 'ranger' file manager (CLI file manager), and it works as expected. And sshfs is way way easier to install and setup; for example, sshfs requires only ssh and fuse on client side, and ssh on server side, while WebDAV requires working Apache server ... quite a difference, as ssh is usually ubiquitous on linux/unix/bsd, and also ssh-server can be easily installed on MSWin OS. (Been there, done that ...)

In this respect, I would like you to comment on your statement

Route99 wrote:

...
The WebDAV protocol, working only on top of Apache, is a much safer method than using (S)FTP or the old SMB (SAMBA) sharing.
...

I agree that FTP is not safe, (don't really know about SMB), but I was under impression that SFTP (i.e. ssh) is as safe as it gets (of course provided everything is configured properly). On what fact do you base your statement? I'm genuinely interested, as I have several setup based on ssh/sftp, and I was under impression they are reasonable secure.

However: what I see is that WebDAV and for example sshfs are not meant to be for the same audience i.e. workflow ... therefore, this post is not intended as bashing of OP and/or WebDAV wink ... In this respect, I thank you Route99 for your guide, I'm sure people will find it useful.


Postpone all your duties; if you die, you won't have to do them ..

Offline

#10 2020-04-15 15:52:43

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

iMBeCil wrote:

I agree that FTP is not safe, (don't really know about SMB), but I was under impression that SFTP (i.e. ssh) is as safe as it gets (of course provided everything is configured properly).

I see your point.
I guess the SFTP is for a different target group as developers and other hard core IT-ers, to do the exact configuration, whilst I'm more an advanced user, helped by my beta-background but no IT-er. So my view is not that of a professional IT-er, my real background was in the OP.

I will re-check my text in the guide on such aspects, thanks for your feedback and your compliments.thumb-up

Offline

#11 2020-04-15 21:04:42

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

Update:
I cloned the VM and continued:
* Removed Apache2 completely to be sure (I forgot to clone the basic VM...)
To delete configuration and data files of Apache2 and it's dependencies:

sudo apt-get purge --auto-remove apache2

* Removed ssl-cert completely (else you get a warning you have to do everything manually):

sudo apt-get purge --auto-remove ssl-cert

Now follow @twoion sugesstion to do al at once:
And start again installing apache2 and ssl-cert

sudo apt-get update
sudo apt-get install apache2
sudo apt-get install ssl-cert

Check on the certificate in  /etc/ssl/certs and  /etc/ssl/private: The default *.pem and *.key were present (*pem in "certs-sub-folder and *.key in the private folder, both with the right time stamp/rights.

Ok, done.... and continued in the tutorial at this point:
Activate SSL           

sudo a2enmod ssl
sudo a2ensite default-ssl

... .et cetera...
and finished with the webserver test:

https://127.0.0.1

And... ?
@twoion It works (as you proposed)! (same screen:"Apache2 Debian Default Page")

Well that's OK for tonight I think, I'm tired, did quite some pre-work in our house tonight as my son will paint our house in corona time...he lost his work nearly completely because of corona.

Will do the webdav section asap, as now it is not configured in the right way.
When all the work is done I will update the Tutorial from above.

To be continued soon.

Offline

#12 2020-04-16 05:49:03

ohnonot
...again
Registered: 2015-09-29
Posts: 4,753
Website

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

I guess it all comes down to supporting non-GNU/Linux operating systems.
Otherwise I don't see why ssh-based solutions aren't just as safe. Or NFS, which I use unencrypted locally, but can be made safe, too, I understand.
Filesystem integration, that's a given with all these methods. Thanks for clarifying.

Route99 wrote:

Apache is the server environment you need.

Are you saying another web server like NginX wouldn't work? I'm pretty sure that's not true.


BL quote proposals to this thread please.
how to ask smart questions | my repos / my repos | my blog
---
Thank you for posting direct image links!

Offline

#13 2020-04-16 06:59:58

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

@ohnonot
I did not investigate that, I hope you understand that. 
I will change soon that to "Apache is the server environment you need in the chosen apache/webdav case for my tutorial".
I did not investigate in depth other routes towards other "file sharing/cloud" software as I've made a choice for the apache/webdav combi, which is as such a proven/safe concept.

Others as FreeNAS.org also use webdav (since V9.x), https://www.freenas.org/about/features/. As we need a small desktop FreeNAS is not an option in our case and V9.x with their 1st webdav version,  was not available when I started in 2011.
So I'd like to concentrate on that, with respect for completely different solutions, but those are outside the scope of my tutorial.

@To all (potential) forum contributors:
Summarized: It feels better to me if everyone who wants to contribute does so for the proven/safe concept I'd like to pursue. Your input for that concept is still very welcome to me and I am very happy with the constructive feedback that I already received.

And: I challenge others to share their experience/knowledge to display other routes in a kind of a tutorial so that users have a choice between different routes towards this goal. So many other people already did that, my contribution is only one of the latest that were added.

This is also good/very positive for the forum, active members that are willing to share such a tutorial or others information. And of course the forum is also the platform for the many other subjects I did not mention, everybody knows for himself/herself what subjects that may be.

Last edited by Route99 (2020-04-16 13:37:53)

Offline

#14 2020-04-16 20:04:28

Route99
Member
Registered: 2020-04-13
Posts: 10

Re: [Tutorial] Making your cloud WebDAV server on BunsenLabs

After sanding several doors from our house this evening...;)  just finished the webdav configuration .. and it works. cool
See the picture.


To do:

  • Will publish the code + comment asap as there are small but essential changes for the webdav-install/config section to be made in the tutorial.

  • Test nftables firewall again

  • Test Debian with all these changes too

  • And try to get MX-Linux et al also get done for these changes.

  • When the house painting is ready do the real installation on our server (have spare disks here....).


webdav-16-04-2020.jpg

Last edited by Route99 (2020-04-16 20:09:37)

Offline

Board footer

Powered by FluxBB