1. Sharing Storage

Objective: Setup an NFS shared file system

While distributed filesytems (such as Lustre, GPFS, and others) are commonly used in clustered computing environments, NFS (Network File System) is still used for file-sharing between systems within a network. Your objective is to create commonly used NFS shares to be used by all of the nodes within our cluster.

A few things you will need to know:

  • we’ll setup NFS twice, first as one would normally do it, then as required by Warewulf, our node provisioner. While following along the normal way will work temporarily, it’s not recommended as it’s not needed nor will it persist across Warewulf configuration changes.

  • There are two interactive users:

    • admin - this will be the user you will do administrative tasks with - this user has sudo access.

    • user - this will be an unprivileged user

    • both users have the same password - tuxcluster

  • While your cluster does have access to the internet. All packages you will need will be included on the file-system.

  • Your cluster will be accessible using a private IP range.

  • The nodes will be assigned names and IP addresses as follows:

    • boxocluster-node-[1-4] - 10.0.0.[11-14]

The head node boxocluster-node-1 is accessible via [ssh](https://linux.die.net/man/1/ssh by issuing the following command using admin:tuxcluster as the username:password.

ssh -p2222 admin@localhost

All nodes are accessible via ssh using admin:tuxcluster as the username:password.

Setting up NFS shares normally (Optional)

Note: Performing the following steps is not necessary or permanent. The instructions exist only as a teaching aid.

resources: dpkg, systemctl, exportfs

While typically packages on a RHEL-based system would be installed using dnf or yum, we are working within an environment that does not have access to the network. All necessary packages are available under /apps/pkgs.

The first step will be to install the NFS server package.

rpm --install --verbose /apps/pkgs/nfs-kernel-server*.rpm

Now, create the export entries in /etc/exports using vim

/home           10.0.0.0/24(rw,async,root_squash)
/mnt/apps       10.0.0.0/24(ro,async,root_squash)
/mnt/shared     10.0.0.0/24(rw,async,root_squash)

This will allow any computer in the 10.0.0.0 subnet (10.0.0.1 - 10.0.0.1154) to mount the exported directories. /mnt/apps is exported as read only so that this path is preserved from accidental writing from computers that mount these directories.

Now, start the NFS server:

sudo systemctl start nfs-server
sudo systemctl status nfs-server
sudo exportfs -a

You should get an active (exited) status. At this point, /home, /mnt/apps, and /mnt/shared should be available to mount. Note: it seems that on these builds that showmounts -e will cause it to hang.

If you want to make a change to /etc/exports after the NFS server has been started, you can set the new exports running exportfs -a as root.

Setting up NFS using Warewulf (Required)

Warewulf is heavily container oriented, so it uses many containerized practices for both the compute nodes and the head. One such practice is the use of overlays for configurable files such as /etc/exports. We can use Warewulf’s configuration file to automatically do everything we’ve just explained.

Edit /opt/warewulf/etc/warewulf/warewulf.conf to look like the following:

...
nfs:
    enabled: true
    export paths:
        - path: /home
          export options: rw,sync
          mount options: defaults
          mount: true
        - path: /apps
          export options: ro,sync,no_root_squash
          mount options: defaults
          mount: true
        - path: /shared
          export options: ro,sync,no_root_squash
          mount options: defaults
          mount: true
    systemd name: nfs-server
...

Now run the following command as root to populate the changes to the host system:

wwctl configure nfs

That’s it. Warewulf handles the rest.