Replacing Hyper-V aka ProxMox

linux-category-inverted

As I have been moving away from Microsoft solutions, one of the outstanding items on my TODO list is replacing Hyper-V as my virtualization system.

Previously I had configured two Windows 2012 servers with Hyper-V and used them as my physical hosts, adding VM’s for each of the various functions I needed.  That included Exchange, Web, VPN, Domain Controllers, etc.

I’ve replaced Exchange with a combination of NextCloud (for contacts and calendar) and DoveCot (for mail), which really limits my requirement for the Domain Controllers or relying on Hyper-V.

So I started to look around for a hyper-visor to replace Hyper-V and after looking at a few, ProxMox, an open source “bare metal” hyper-visor.  I’ve put bare metal in quotes here as it’s really just running Debian underneath a pretty slick web UI.

Installing ProxMox is pretty straight forward, download the ISO, burn it to a CD (or there are USB stick instructions as well) and let it go to town.

Once it is done, you connect to it through the web interface (note that you get a certificate warning as by default it uses a self signed certificate) and you can go to town installing VMs.

As my first VM, I spun up an Ubuntu 16.04 image to act as the replacement file server.  The first challenge was getting the ISO of Ubuntu on to ProxMox, it’s well documented that the web interface has a hard limit for upload size that most ISO’s exceed.  It’s a well documented issue and a quick Internet search brought to light the solution, using SFTP instead.

In my old configuration I had two Windows 2012 file servers, one on each VM host, using DFSR to replicate the data between them and DFS to present them as a single share.  My main data volume is a 6TB drive, which contains just a little over 3TB at the moment (and growing of course).

I’d decided recently to dump the DFS-R replication in preparation for moving to Linux (it was really more trouble than it was worth, the number of times replication broke for any number of reasons was kind of a pain) and so after Ubuntu installed (no issues there), the next issue was attaching the 6TB physical disk to the VM.

The web interface is of little help here, but after some more digging around the web, I found a helpful article on how to achieve it.

 

Once both the VM and additional storage were in place, configuring Samba was pretty straight forward and copying the data across, while time-consuming, was easy enough as well.

My intention, when I’ve moved my other VM host over to ProxMox, is to similarly attach the second 6TB drive to my NextCloud install and sync the data between them, allowing me to expose my file share through the NextCloud interface as an external storage provider as a local directory instead of though an SMB share as I’m doing it now.

So what about ProxMox?

Overall it’s been good so far, there is one other annoyance I will mention.  Each time you logon to the admin interface you get a pop-up about you not having “No Valid Subscription”.

This, in effect, makes ProxMox NagWare instead of Open Source as there is no requirement to get a subscription unless you require support.

Fortunately all of the source code is available and so you can remove the pop-up pretty easily.  I used these instructions on how to do it.  Though the JavaScript file had moved since it was written, but was easy enough to find.

The web interface seems quite good, with console access to the VM’s as well as the ProxMox host, you can do pretty much everything from it.

There are also lots of advanced options, like replication and failover.  I haven’t tried these yet, but as I decommission the last of my Windows Servers, I’ll likely start poking at them some more.

The only other slight annoyance (and it’s not so much ProxMox’s fault) is that when your using the console, sometimes the input focus can switch from the console and before you know what’s happen your browsers has done something like go to the previous page because you hit the backspace key to correct a command.

However, since after initial setup, your unlikely to use the web console very much, it seems like a minor thing overall.

NextCloud

linux-category-inverted

As I mentioned a while ago when I reviewed ZorinOS, I’ve been thinking more and more about moving away from big commercial software.  It started with my move away from Windows Phone (or more accurately, Microsoft abandoning the platform and moving away from their users) and continued with the details of how much data Windows 10 collects.

I had known for a while that I was going to have to eventually do something with my Exchange server, 2010 was old and getting older all the time.  I did have a newer version of Exchange, but for the few mailboxes I host, it always seemed like overkill.

The big issue with moving from Exchange had never been mail per say, IMAP servers and clients are plentiful and fully function, but contacts and calendar had always been another story.

I knew CalDav and CardDav had come along to try and solve the issue so I decided to look around and see what state things were in.  It turns out, much better than I had expected.

There were several good CardDav/CalDav server implementations around but the one that really stood out to me was NextCloud.

Not only is NextCloud a full CardDav/CalDav server, it’s a full file sync (ala OneDrive) server as well.  That ticked off another check box for me as the convince of OneDrive is huge.

Of course installing such a complicated piece of software can be daunting, fortunately, there’s an easy solution, they have a VM image with everything setup for you (they even have a step by step walk through on how to set it up).

Once the VM was up and running, logging in to the web interface is easy enough (after you of course accept the self sign cert of course).

I’ll go in to how to resolve the certificate issue in a future post, but obviously Let’s Encrypt will be involved.

Features

NextCloud has a host of features, both the ones that come standard, and those that are found in the store.

The ones that I have installed are:

  • Files, Contacts and Calendar (obviously)
  • Mail
  • Notes
  • News
  • Phone Sync
  • Bookmarks
  • Tasks
  • External Storage

At first I connected Mail to my existing Exchange server, but in another article I’ll go in to the details of how I setup DoveCot on the NextCloud server and decommissioned my Exchange server completely.

News

This is another feature that really caught my attention, for many years I have been using a perl script to pull RSS feeds from various sources and add them to a folder on my Exchange server via IMAP.  That way, once an article had been read, it synced across all of my mail clients.

Outlook has always worked well with this system, but other clients have always struggled and I suspected to replace Exchange, I’d have to find another way to read my RSS feeds.

News solved the problem.

It is an RSS feed aggregator and has support for updating the read/unread status across clients that support it.

Phone Sync

This was an unexpected surprise, an SMS sync client for Android.  It’s a little buggy, but better than nothing.

Bookmarks

I’ve always hated being tied to one browser, I tend to move across devices and browsers pretty fluidly and so I seldom use the built-in bookmark features of a browser.  Instead I created a bookmark web app several years ago that manages all my bookmarks for me and I use it as my home page.

That app uses a MySQL database to store the bookmarks but NextCloud’s UI is better and so I’ve started to look at porting my app over to NextCloud as there are plugin’s for every major browser available for NextCloud’s bookmark feature.

Tasks

I installed this to sync up my phone’s tasks, I don’t use them often, but it’s kind of nice to have them available online and backed up.

External Storage

This was another big feature that I like quite a lot.  As I’ve mentioned before, I have a large library of software on my Windows file server, but that means I have to be connected to my local network to get access to them.

With NextCloud and the External Storage plugin, I can expose that file share as part of the NextCloud web interface and get to my library from anywhere.

NextCloud Clients

One of the things that really drew me to NextCloud was the availability of clients for just about every OS.  Windows, Linux, Android, iOS and Mac.

This broad support really makes the case for NextCloud as a replacement for OneDrive or any other cloud storage provider.

About that VM…

There is one issue with the supplied VM image that I didn’t touch on earlier, the default image only has a 32g partition.

You can download other images that have 500g or 1tb, but those aren’t very granular choices.

Fortunately all of the VM’s come with LVM partitions so adding additional space to the VM is easy… just kidding, it’s a real pain in the butt if your not familiar with LVM.

Unfortunately finding a good tutorial isn’t easy, but this one was the best I found and solved the problem by adding an additional 80g to my VM.

Final Thoughts

I’ve been very happy with NextCloud, it was an eye-opening find that really solved a lot of issues I knew I was going to have by moving away from Exchange and other services that I had always taken for granted in the Microsoft ecosystem.

Without it my move away from Exchange may not have even been possible and it certainly has made it possible to go beyond what I thought was possible.

At this point, I’m now convinced that moving 100% off of big commercial software and entirely on to open source.

That’s something that just a few months ago I would have said was impossible.

 

 

 

 

 

 

 

 

 

 

Linux Desktop Apps

linux-category-inverted

So switching from Windows to Linux involves more than just the OS, not all of those familiar apps are going to have versions that run on Linux and so you have to find new ones to replace them.

Of course in a pinch you can run Wine, but that’s no guaranteed solution.

Here are some of the most common apps I use and what I’ve moved to for Linux versions.

Web Browsers

My primary browser on Windows has be Edge since it came out, with Firefox being my secondary browser.  Recently though, with an eye to moving to Linux, I’d started to switch from Edge to Vivaldi.

The good news is that there are lots of browsers for Linux, Vivaldi and Firefox are both supported and so I’m using them, in that order.

Mail/Calendar

In an upcoming post I’ll go in to more detail about my switch from Exchange, but here I’ll simply say that my main mail program is now Thunderbird, with Lightening as my calendar app.

There are several other mail/calendar apps on Linux, but after playing with them I found Thunderbird to be a better solution.

News Reader

Something I hadn’t used on Windows, as I’d been feeding my new feeds directly in to Exchange, was a news reader.

After looking at the options, I found FeedReader, which supported NextCloud and looked far better than any other desktop news reader I found.

Password Safe

I’ve used Keepass for years and it has great cross platform support, including running as a Mono app on Linux.

A simple:

apt-get install keepass2

did the job.

Text Editor

On Windows I’ve been using NotePad++ for years and this has been the hardest app to replace, at the moment it looks like Sublime Text is going to win out over the other options.

Office Suite

Of course MS Office is king on Windows, but not available for Linux.  The next best thing is LibreOffice.

I’ve used LibreOffice a little bit over the years and never been really all that impressed with it, but recent versions have really started to come along and it’s been working well so far.

Comparison Tool

When I’m programming it’s often useful to be able to compare files against each other and I’ve used BeyondCompare for this, fortunately they have a Linux version as well so it was a no brainier.

GIT Client

GIT has become an important part of development these days and a good GUI really simplifies the task of using it.   On Windows I had been using SourceTree, but there is no Linux version so I’ve been using GitKraken.

It’s has a lot of nice features but is missing a few things that SourceTree did, but it’s not a huge difference and the GIT CLI is always available.

Twitter

The one desktop app I’m still missing is Twitter, there are a couple of Linux clients around, but I haven’t found one I really like, so at the moment I’m still using the web interface.

Windows 10 in Tablet Mode

windows-category-inverted

I’ve been using a Microsoft Surface 3 for a couple of years now as my main “at home consumption device”.  Basically it’s for browsing the net while watching TV, or reading a book, or watching some “TV” in bed.

It’s a great piece of hardware and with Windows 10, Microsoft replaced the old Windows 8.1 tablet code with the new “Tablet Mode”.

Tablet mode works well in general but there are two things that really bother me about it:

  1. Edge Background Downloads
  2. Close Window Behaviour

Edit Background Downloads

Microsoft Edge is really the only browser that supports a proper touch mode and so it gets used as my primary browser on the Surface.  This in general is fine and Microsoft has done a good job with it, however there is one thing they’ve really messed up in tablet mode.

If you start a download and then switch away from Edge, most of the time the download will stop.  This does not happen in “regular” mode, only tablet mode.

I assume this is a battery saving feature, stopping tabs from consuming power, but it’s very annoying if you have started a big download and want to go check your e-mail or something else.

There are two ways around it, either disable tablet mode temporarily while the download runs, or start the download in another browser.

Neither are particularly good and I haven’t found any way to tell Edge to keep “alive” when it’s not in the foreground in tablet mode.

Close Window Behaviour

The other annoyance is the behaviour of Windows when you close a task.  Say you download a zip file in Edge, when it’s done you select “Run” and Windows dutifully opens up your desired zip file manager.

You extract your files somewhere and then close the zip manager.  Where would you expect Windows to take you back to?  Edge?

That would seem reasonable as it was the last app that was open and used.

Nope, right back to the start screen.

The same is true for any time you close an app like this.  Maybe your browsing files in Explorer and then decide to open a picture in Photos.  Close Photo’s and instead of going back to Explorere, your back at the start menu again.

I’m not sure what Microsoft’s logic here is, but it seems strange to say the least.

Ubuntu on Windows

windows-category-inverted

In the Anniversery Update, Microsoft added support for “Bash on Ubuntu on Windows”, a full linux subsystem, allowing you to run virtually any console level Linux application, including things like Apache, MySQL, PHP and other standard offerings, without any modification.

Of course, these are available for Windows as well, but having a full LAMP (Linux Apache MySQL PHP) stack on your local system without having to run a full VM is kind of useful.

When I picked up my new Ultrabook, I decided to try out Ubuntu on Windows instead of using Wampserver for my local PHP development.

Installing Ubuntu for Windows was easy enough, enable the feature in Windows and away you go.  Once finished, you can then run Bash and get a full Linux command line available.

Installing packages is exactly the same as with Ubuntu, for example to install Apache, simply do a “sudo apt-get install apache2”.

MySQL, PHP, Apache, phpMyAdmin, etc. all installed without issue and I had a functional LAMP stack soon afterwards.

It’s quite an impressive feature really, but there are a few drawbacks:

  • Apache started automatically with the system, but for some reason that I haven’t tracked down yet, MySQL doesn’t.
  • File system access (see more below)

Overall, it’s a useful tool if you need Linux but don’t want to install a full VM.

File System Access

Here’s the show stopper though.  The files system inside of the Ubuntu install is a real Linux file system, everything you expect is there; etc, mnt, var, …

To accomplish that, Microsoft has created “root” directory inside the current users home directory.

In Windows you can browse to it via:

C:\Users\[username]\AppData\Local\lxss

Inside the Ubuntu instance, Microsoft has mounted your permeant drive letters under the /mnt directory.

This let’s you move files back and forth between Ubuntu and Windows.

That sounds like a great idea until you notice that to accomplish this trick Microsoft has created different file properties for Windows and Ubuntu.  So for example if you edit a file in Linux, the last modified date is update in Linux but not Windows.

Likewise if you create a file through Windows in the Linux file system, it won’t show up in Linux as it doesn’t have the “right” file properties associated with it.

This makes trying to do development in this mixed environment virtually impossible.

For example, if your working on a PHP app and editing the file in your favorite Windows editor and save it, PHP will never see the file modified and not reload it in to the cache so you will always get the unedited code running on the server.

There are a few ways around this, like disabling opcache in PHP, or “touching” the files after editing, but these are hacks at best.

In addition, I found some files simply disappeared from Linux after being edited in a Windows editor and the only way to get them back was to remove them and copy them from /mnt/c.

Conclusion

It’s an interesting option to have, but it’s too immature at this point to use in any serious fashion.

I’ll keep an eye on it, but I’m removing the LAMP stack and going back to Wampserver for the time being.