Clusterducks

Getting Started

clusterducks is simple to deploy and use, but learning the concepts may initially take some time.

To help you get started, a series of alerts are visible on your control panel dashboard after first install or creation of new networks. Each alert indicates a problem that must be resolved before the network is operational.

Panel Settings

When SMTP is configured, the panel will send an email notification whenever your account is accessed. In addition, it will also notify all users when their servers have become unreachable (or recover).

Clients, Networks and Servers

Most items in clusterducks are tied to one object or another, as a mark of ownership:

  • Clients are the highest level of ownership
  • Users and Networks are owned by Clients
  • Servers, Devices, Containers, Menus (and all filesystems, because of replication) are owned by Networks

When clusterducks is first configured, a client is automatically created and associated with the admin user. No networks are created for you.

Add a Client (UI)

  1. Use the Clients page (building icon)
  2. Click New Client
  3. Enter details, click Save

Add a Client (API via cURL)

  1. Retrieve your API token from the account settings page

  2. Create the client:

    export API_TOKEN="..."
curl -sH "Authorization: Token token=\"${API_TOKEN}\"" -H 'Content-Type: application/json' \
    -X POST "https://panel.example.com/api/v0.9.2/clients.json" \
    -d '{
        "Name": "New Client"
    }'

Add a Network (UI)

Before a server or any devices can be configured, you must have a network:

  1. Go to the Networks page
  2. Click New Network
  3. Enter the new network name
  4. (Optional) Configure any network limits (MaxNetworks, MaxServers, etc)
  5. Select which client the network will be associated with
  6. Click Add
  7. Click the Edit icon for the new network
  8. Ensure that the appropriate tunnel server is selected

If you have non-superadmin users, access may be delegated using the "pilot" icon.

Add a Network (API via cURL)

  1. Retrieve your API token from the account settings page

  2. Create a network, using the newly created Client ID:

    export API_TOKEN="..."
curl -sH "Authorization: Token token=\"${API_TOKEN}\"" -H 'Content-Type: application/json' \
    -X POST "https://panel.example.com/api/v0.9.2/networks.json" \
    -d '{
        "Name": "New Network",
        "Client": 2
    }'

Add a Tunnel Server (UI)

Tunnel servers are used to proxy HTTP API commands to remote nodes. This masquerades the actual remote server from the Internet.

  1. Go to the Tunnel Servers page
  2. Click the New Tunnel Server button
  3. Alias is a human-readable name for the tunnel server, it can be whatever you want.
  4. IP is a routable address that must be reachable by the panel webserver

You may need to click the "refresh" icon to force the table to display with the new record.

Add a Tunnel Server (API via cURL)

  1. Retrieve your API token from the account settings page

  2. Create a tunnel server, using the newly created network ID:

    export API_TOKEN="..."
curl -sH "Authorization: Token token=\"${API_TOKEN}\"" -H 'Content-Type: application/json' \
    -X POST "https://panel.example.com/api/v0.9.2/tunnels.json" \
    -d '{
        "Alias": "New Tunnel Service",
        "network": 1,
        "IP": "4.2.2.1"
    }'

Register the Master Server

The first server that is configured is the master server. It can be converted to a slave node later, if desired.

  1. Go to the Servers page
  2. Click the New Server button
  3. Enter the new server name; this is used for aesthetics only. The server name does not need to be a resolveable hostname.
  4. Enter the server iSCSI IP; this is used for network traffic between devices and storage nodes. It must be reachable by devices that the server will provide storage services for.
  5. Enter the server Replication IP; this must be reachable by any slave servers. If left blank, the iSCSI IP will be used instead. It is strongly recommended to use a dedicated network for replication traffic on medium to large networks.
  6. ZFS Pool Name is the name of a manually created ZFS pool. Any needed filesystems are created on the pool automatically:
    • poolname/iscsi/{devices,containers,ctemplates,groups,volumes}
      • Live filesystems
    • poolname/vdisks
      • Persistent data storage
    • poolname/erator
      • "Trash" data to be eliminated at administrator's leisure
  7. Tunnel Port is a port forward that will be opened on the network's selected tunnel server
  8. Click Add
  9. Go to the Networks page
  10. Click the Edit icon for the current network
  11. Ensure that your newly-added server is selected as the master server
  12. Save network settings
  13. Configure your new server's networking using the values specified earlier.
  14. Create a network bridge for VMs.
  15. Execute install.sh -h "panel.hostname" -k "server.key" -n "bridge.name", once finished, reboot the server.
  16. After the server starts, verify that "Panel to server OK" shows in the server Status (stats field contains data, via https://{panel.hostname}/api/v0.9.2/servers/{name}/stats.json)

Note: Because this installer will bootstrap a fresh server from beginning to end (including SSL certificates, httpd, SQL and its credentials) it is recommended to start with a minimal OS (only networking configured).

Add a Server (API via cURL)

  1. Retrieve your API token from the account settings page

  2. Create a server, using the newly created network ID:

    export API_TOKEN="..."
curl -sH "Authorization: Token token=\"${API_TOKEN}\"" -H 'Content-Type: application/json' \
    -X POST "https://panel.example.com/api/v0.9.2/servers.json" \
    -d '{
        "Name": "masterServer",
        "network": 1,
        "IP": "192.168.1.10",
        "Pool": "tank"
    }'

Create a Volume (UI)

  1. Go to the Volumes page
  2. Click New Volume
  3. Fill the required fields; a volume can be as large as your master pool allows. Volumes are thin provisioned, but this can really set you up to shoot yourself in the foot. Start small and grow the volumes later as needed.
  4. Click Add; if the tunnel connection and server pool are ok, it will be created. If not, check your server stats and ensure they're available and healthy before trying again.

Create a Volume (API via cURL)

  1. Retrieve your user API token from the Account Settings page

  2. Execute the following from a shell prompt:

    export API_TOKEN="..."
curl -sH "Authorization: Token token=\"${API_TOKEN}\"" -H 'Content-Type: application/json' \
    -X POST "https://panel.example.com/api/v0.9.2/volumes.json" \
    -d '{
        "network": 1,
        "Alias": "New Volume 1",
        "vol_size": "42"
    }'

Create a Group (UI)

Groups are clones of volumes, where actual OS installation and maintenance occurs.

  1. Go to the Groups page
  2. Click the New Group button
  3. Fill out the group name and ensure it is based on the volume we selected earlier.
  4. Click Add and the panel will send a request via the tunnel service for the master to clone the volume into a new group image.
  5. Click the properties icon and select whether you'd like to use Strict Versioning and Lazy Locks.

Strict Versioning will ensure all devices across all servers boot to the assigned version only. If a server does not have the version, the device will attempt another server, until it finds an available image. If it doesn't find the image on any servers (as in a satellite site with long replication times) it will remain in a "reboot loop" until it does. If no image becomes available, the device will not boot.

If Strict Versioning is not enabled, the server will attempt to provision the device image from the latest available local snapshot. This may be indesirable from a security perspective for certain updates, but Strict Versioning can be enabled or disabled as needed. It may be desirable to boot devices to older snapshots if site uptime is absolutely critical, or replication times are longer than acceptable downtime, etc.

Lazy Locks are a companion property to Strict Versioning. With Lazy Locks, unlocked devices are not subjected to reprovisioning under Strict Versioning conditions described above.

Without Lazy Locks, it is possible to lose data stored in unlocked devices. However, if unlocked devices store no critical data, disabling Lazy Locks will enforce image version compliance across the entire group and simplify administration.

  1. Go to the Network settings for your current network
  2. Assign your new group as the default, allowing netbooting devices to register and join that group to install your new OS

Create a Group (API via cURL)

Execute the following; replace VolumeID with the volume created in the previous task:

export API_TOKEN="..."
curl -sH "Authorization: Token token=\"${API_TOKEN}\"" -H 'Content-Type: application/json' \
    -X POST "https://panel.example.com/api/v0.9.2/groups.json" \
    -d '{
        "network": 1,
        "Name": "New Volume 1",
        "VolumeID": 1
    }'