Clusterducks

Introduction

What is it?

clusterducks is an open-source web application that allows network administrators to manage hundreds (or thousands) of netboot devices that share common golden images.

The name is a play on words, and we're apologizing in advance for any mishaps with its pronunciation wink

The platform is written in PHP and has a few components:

  • Web interface (panel) to manage configuration for multiple networks
    • Runs in Docker container or standard virtual / physical machine
    • Low requirements - runs on a Raspberry Pi
    • PHP + MySQL + OpenSSH
    • Communicates with server agents via tunnel servers
    • SSL Certificate Authority
  • Tunnel service
    • Broker HTTP API requests from the control panel to remote agents
    • Must be completely accessible via TCP by web panel as a DMZ (via LAN or IPSEC with external firewalls)
    • Typically requires a public hostname for agents to connect to
    • Simple appliance - zero configuration required, just start a container or run the installer
    • Users and keys are automatically managed; provides an authentication wrapper around OpenSSH using AuthorizedKeysCommand + MySQL
    • Integrated with iptables to further isolate connections from one another and secure the tunnel endpoint from untrustworthy operators
  • Server agent
    • Provides HTTP API and iPXE menu rendering for netboot devices
    • HTTPD / MySQL / DHCP / TFTP / iPXE + ZFS + iSCSI/NFS
    • dnsmasq configuration automatically managed
    • Libvirt for virtualization support (KVM)
    • Multiple active slaves receive updates from a single master server
    • Connects to the tunnel service for NAT bypass to receive API requests
    • Servers can be designated as storage only, compute only, or both
  • Netbooted devices
    • Can be virtual or bare metal
    • Windows (7, 8.1, 10, 2012R2), Linux (dracut)
    • No disk imaging required - no hard drives required, either
    • Must be able to receive DHCP broadcasts from boot server(s); must support iPXE
    • Uses iSCSI (Windows/Linux) for OS storage
    • Custom "vDisks" store persistent data and increase redundancy
    • Overlay filesystems allow transparent separation of OS and instance data for Linux devices using our initrd module
    • Folder Redirection allows transparent separation of OS and user data for Windows users

Why go through all this trouble?

Shared storage makes things more manageable. Virtual machines have had many benefits because of their shared architecture; by grafting those capabilities onto bare metal, we have the manageability of virtualization with the performance of true hardware.

Cost is a factor; If devices no longer need a hard drive to boot, this may reduce operating & replacement costs and associated lost data / downtime.

Netboot does not limit a device to operating only from the network; extra drives can still be connected for faster local storage access.

I'm not convinced. Why the web interface?

Clusterducks has a secure installation by default - the control panel serves the role of Certificate Authority, issuing and revoking certificates for devices. The CA root chain can be embedded in any device that will have a role in your network.

Managing a production network without appropriate scripts can be a very dangerous process. Managing several production networks is more stressful. This web interface does only a few things, but it does them very well.

All of the common repetitive tasks involved in diskless network maintenance are rolled into an easy-to-use point-and-click package, with checks in place to ensure the operator never misses a step. When networks are easier to maintain, people without the System Administrator badge can do it.

Because of the platform's position as a fundamental network service, it is able to manage a lot of tedious tasks for your administrators;

  • Multiple distinct networks can be managed from one interface thanks to the segregated client-server architecture
    • Permissions delegation allows administrators to ensure technicians only see networks they are assigned to
  • Tiered storage architecture allows efficient image storage and replication of cloned snapshots
  • By default, devices are non-persistent for security reasons (OS storage is reprovisioned upon boot); If a user is infected with malware, just reboot and the infection is gone
  • Device IP addresses are statically assigned from the web interface; DHCP/BOOTP is abandoned early in boot process
  • vDisks may be assigned to a device for persistent data storage. Because they are never reprovisioned, it is safe for use with volatile data (changes frequently)
    • Integrated support for overlayfs (in Linux) for true separation of OS and instance data

Isn't it complicated? I don't want to dedicate my life to learning this stuff.

Netboot maintenance is tedious and dangerous if you do it the old way, writing config files by hand and sending signals to daemons to reload them.

clusterducks intuitive web interface makes managing networks easy and fun. It's free to use with a permissive license (AGPL-3.0) that even allows commercial operations.

Open source ensures this software will never go away or become vaporware because it is owned by the public. The main limitation is that the source must remain as the original copyright holder intended it to be; open. That's not so bad, right?