ProxMox Updates

linux-category-inverted

As mentioned in my previous article, ProxMox uses a subscription model to pay for development, however they do support home/lab usage via nagware.

However, by default, the base install assumes your going to have a subscription and that makes for a recurring error each time you try and update the server with “apt-get”:

W: The repository 'https://enterprise.proxmox.com/debian/pve stretch Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch https://enterprise.proxmox.com/debian/pve/dists/stretch/pve-enterprise/binary-amd64/Packages 401 Unauthorized
E: Some index files failed to download. They have been ignored, or old ones used instead.

This is caused by the fact that without a subscription, the standard update repo for ProxMox is unavailable.

Fortunately they do have a public repo which they give instructions on this support page.

The basic steps are as follows:

  1. Open a shell on your ProxMox Server.
  2. Go in to “/etc/apt/sources.list.d”.
  3. Copy “pve-enterprise.list” to “pve-enterprise-no-sub.list”
  4. Edit “pve-enterprise-no-sub.list” and replace the contents with the lines from the support page.
  5. Edit the “pve-enterprise.list” file and comment out the repo line (use a #), alternatively you can just delete this file if you want as well.

Then re-run “apt-get update && apt-get upgrade” and your off to the races.

ProxMox Mail Config

linux-category-inverted

After getting ProxMox all setup, there was one last thing to investigate, mail was not flowing from the ProxMox host.  Taking a quick look, two changes were required.

  1. The Postfix aliases database wasn’t initialized, so it was throwing an error, simply running “newaliases” on both nodes resolved this error.
  2. As port 25 is blocked by my host, all mail has to go through my ISP’s mail servers.  Since I already have a Postfix server configured to handle this, setting the “relayhost” value in “/etc/postfix/main.cf” resolved this issues easily enough.

Moving VM’s to ProxMox

linux-category-inverted

In my previous post I installed ProxMox as my HyperVisor, but I didn’t mention anything about moving VM’s from Hyper-V over to it.  Well, here’s how I went about the process.

First, I tracked down this article, which has a pretty good detailed guide of the process.  Basically it comes down to the following:

  • Export the VM from Hyper-V to get a clean snapshot.
  • Transfer the .vmdk files up to the ProxMox host.
  • Create a new VM in ProxMox with the same CPU/Memory/Disks.
  • Convert the .vmdk files to the KVM format.
  • Start the VM and make any other changes, like IP, DNS, etc. that are required if the OS doesn’t detect the changed hardware properly.

There are a few caveats:

  • While exporting from Hyper-V, the completion percentage is displayed in the “Status” column of the Hyper-V manager.  However that column is not always visible depending on the resolution of the display and how large the window is, so make sure to expand the Hyper-V manager until you can see it.
  • In the article, the .vmdk is converted to a .qcow2 file, however for me the default format of disks is actually raw, so replace “-O qcow2” with “-O raw”.
  • Similarly, the location of the disks in the article is incorrect, it’s actually in /dev/pve, but that’s actually a link back to /dev/dm-? but you can use either.

That makes the conversion command something like:

qemu-img convert -O raw ~/BootDisk.vhd /dev/pve/vm-102-disk-1

Overall the migration process is quite easy, if a little slow between the transfer of the .vmdk files as well as converting them over to the new format.

 

Replacing Hyper-V aka ProxMox

linux-category-inverted

As I have been moving away from Microsoft solutions, one of the outstanding items on my TODO list is replacing Hyper-V as my virtualization system.

Previously I had configured two Windows 2012 servers with Hyper-V and used them as my physical hosts, adding VM’s for each of the various functions I needed.  That included Exchange, Web, VPN, Domain Controllers, etc.

I’ve replaced Exchange with a combination of NextCloud (for contacts and calendar) and DoveCot (for mail), which really limits my requirement for the Domain Controllers or relying on Hyper-V.

So I started to look around for a hyper-visor to replace Hyper-V and after looking at a few, ProxMox, an open source “bare metal” hyper-visor.  I’ve put bare metal in quotes here as it’s really just running Debian underneath a pretty slick web UI.

Installing ProxMox is pretty straight forward, download the ISO, burn it to a CD (or there are USB stick instructions as well) and let it go to town.

Once it is done, you connect to it through the web interface (note that you get a certificate warning as by default it uses a self signed certificate) and you can go to town installing VMs.

As my first VM, I spun up an Ubuntu 16.04 image to act as the replacement file server.  The first challenge was getting the ISO of Ubuntu on to ProxMox, it’s well documented that the web interface has a hard limit for upload size that most ISO’s exceed.  It’s a well documented issue and a quick Internet search brought to light the solution, using SFTP instead.

In my old configuration I had two Windows 2012 file servers, one on each VM host, using DFSR to replicate the data between them and DFS to present them as a single share.  My main data volume is a 6TB drive, which contains just a little over 3TB at the moment (and growing of course).

I’d decided recently to dump the DFS-R replication in preparation for moving to Linux (it was really more trouble than it was worth, the number of times replication broke for any number of reasons was kind of a pain) and so after Ubuntu installed (no issues there), the next issue was attaching the 6TB physical disk to the VM.

The web interface is of little help here, but after some more digging around the web, I found a helpful article on how to achieve it.

 

Once both the VM and additional storage were in place, configuring Samba was pretty straight forward and copying the data across, while time-consuming, was easy enough as well.

My intention, when I’ve moved my other VM host over to ProxMox, is to similarly attach the second 6TB drive to my NextCloud install and sync the data between them, allowing me to expose my file share through the NextCloud interface as an external storage provider as a local directory instead of though an SMB share as I’m doing it now.

So what about ProxMox?

Overall it’s been good so far, there is one other annoyance I will mention.  Each time you logon to the admin interface you get a pop-up about you not having “No Valid Subscription”.

This, in effect, makes ProxMox NagWare instead of Open Source as there is no requirement to get a subscription unless you require support.

Fortunately all of the source code is available and so you can remove the pop-up pretty easily.  I used these instructions on how to do it.  Though the JavaScript file had moved since it was written, but was easy enough to find.

The web interface seems quite good, with console access to the VM’s as well as the ProxMox host, you can do pretty much everything from it.

There are also lots of advanced options, like replication and failover.  I haven’t tried these yet, but as I decommission the last of my Windows Servers, I’ll likely start poking at them some more.

The only other slight annoyance (and it’s not so much ProxMox’s fault) is that when your using the console, sometimes the input focus can switch from the console and before you know what’s happen your browsers has done something like go to the previous page because you hit the backspace key to correct a command.

However, since after initial setup, your unlikely to use the web console very much, it seems like a minor thing overall.

NextCloud

linux-category-inverted

As I mentioned a while ago when I reviewed ZorinOS, I’ve been thinking more and more about moving away from big commercial software.  It started with my move away from Windows Phone (or more accurately, Microsoft abandoning the platform and moving away from their users) and continued with the details of how much data Windows 10 collects.

I had known for a while that I was going to have to eventually do something with my Exchange server, 2010 was old and getting older all the time.  I did have a newer version of Exchange, but for the few mailboxes I host, it always seemed like overkill.

The big issue with moving from Exchange had never been mail per say, IMAP servers and clients are plentiful and fully function, but contacts and calendar had always been another story.

I knew CalDav and CardDav had come along to try and solve the issue so I decided to look around and see what state things were in.  It turns out, much better than I had expected.

There were several good CardDav/CalDav server implementations around but the one that really stood out to me was NextCloud.

Not only is NextCloud a full CardDav/CalDav server, it’s a full file sync (ala OneDrive) server as well.  That ticked off another check box for me as the convince of OneDrive is huge.

Of course installing such a complicated piece of software can be daunting, fortunately, there’s an easy solution, they have a VM image with everything setup for you (they even have a step by step walk through on how to set it up).

Once the VM was up and running, logging in to the web interface is easy enough (after you of course accept the self sign cert of course).

I’ll go in to how to resolve the certificate issue in a future post, but obviously Let’s Encrypt will be involved.

Features

NextCloud has a host of features, both the ones that come standard, and those that are found in the store.

The ones that I have installed are:

  • Files, Contacts and Calendar (obviously)
  • Mail
  • Notes
  • News
  • Phone Sync
  • Bookmarks
  • Tasks
  • External Storage

At first I connected Mail to my existing Exchange server, but in another article I’ll go in to the details of how I setup DoveCot on the NextCloud server and decommissioned my Exchange server completely.

News

This is another feature that really caught my attention, for many years I have been using a perl script to pull RSS feeds from various sources and add them to a folder on my Exchange server via IMAP.  That way, once an article had been read, it synced across all of my mail clients.

Outlook has always worked well with this system, but other clients have always struggled and I suspected to replace Exchange, I’d have to find another way to read my RSS feeds.

News solved the problem.

It is an RSS feed aggregator and has support for updating the read/unread status across clients that support it.

Phone Sync

This was an unexpected surprise, an SMS sync client for Android.  It’s a little buggy, but better than nothing.

Bookmarks

I’ve always hated being tied to one browser, I tend to move across devices and browsers pretty fluidly and so I seldom use the built-in bookmark features of a browser.  Instead I created a bookmark web app several years ago that manages all my bookmarks for me and I use it as my home page.

That app uses a MySQL database to store the bookmarks but NextCloud’s UI is better and so I’ve started to look at porting my app over to NextCloud as there are plugin’s for every major browser available for NextCloud’s bookmark feature.

Tasks

I installed this to sync up my phone’s tasks, I don’t use them often, but it’s kind of nice to have them available online and backed up.

External Storage

This was another big feature that I like quite a lot.  As I’ve mentioned before, I have a large library of software on my Windows file server, but that means I have to be connected to my local network to get access to them.

With NextCloud and the External Storage plugin, I can expose that file share as part of the NextCloud web interface and get to my library from anywhere.

NextCloud Clients

One of the things that really drew me to NextCloud was the availability of clients for just about every OS.  Windows, Linux, Android, iOS and Mac.

This broad support really makes the case for NextCloud as a replacement for OneDrive or any other cloud storage provider.

About that VM…

There is one issue with the supplied VM image that I didn’t touch on earlier, the default image only has a 32g partition.

You can download other images that have 500g or 1tb, but those aren’t very granular choices.

Fortunately all of the VM’s come with LVM partitions so adding additional space to the VM is easy… just kidding, it’s a real pain in the butt if your not familiar with LVM.

Unfortunately finding a good tutorial isn’t easy, but this one was the best I found and solved the problem by adding an additional 80g to my VM.

Final Thoughts

I’ve been very happy with NextCloud, it was an eye-opening find that really solved a lot of issues I knew I was going to have by moving away from Exchange and other services that I had always taken for granted in the Microsoft ecosystem.

Without it my move away from Exchange may not have even been possible and it certainly has made it possible to go beyond what I thought was possible.

At this point, I’m now convinced that moving 100% off of big commercial software and entirely on to open source.

That’s something that just a few months ago I would have said was impossible.