fluxcoil wiki

Setting up 2 linux systems in a simple server/client configuration, to use nfsv4 exports with kerberos. IPA is used as KDC.

yum -y install nfs-utils

# create /etc/gssapi_mech.conf :
-------------------------------------------
# library                              initialization function
# ================================    ==========================
# The MIT K5 gssapi library, use special function for initialization.
libgssapi_krb5.so.2          mechglue_internal_krb5_init
#
-------------------------------------------

# ensure the host is already member in the IPA domain,
# and create additionally the nfs principal
kinit admin
ipa service-add nfs/rhel6u3b.fluxcoil.net
ipa-getkeytab -s rhel6u3b.fluxcoil.net -p nfs/rhel6u3b.fluxcoil.net -k /etc/krb5.keytab
klist -ekt /etc/krb5.keytab

# now prepare the nfs export.
# create a nfsv4root and bindmount the real data there
mkdir -m 1777 /mnt/nfsv4root
mkdir /mnt/nfsv4root/store /realpath
mount -n --bind /realpath /mnt/nfsv4root/store

# ensure SECURE_NFS="yes" is set in /etc/sysconfig/nfs
grep SECURE_NFS /etc/sysconfig/nfs

# create /etc/idmapd.conf

# configure the export
cat >/etc/exports<<EOT
/mnt/nfsv4root       gss/krb5(sync,rw,fsid=0,insecure,no_subtree_check,anonuid=65534,anongid=65534)
/mnt/nfsv4root/store gss/krb5(sync,rw,nohide,insecure,no_subtree_check,anonuid=65534,anongid=65534)
EOT

service rpcbind restart
service rpcsvcgssd restart
service nfs restart

tail -f /var/log/messages &

yum -y install ipa-client ipa-admintools nfs-utils
ipa-client-install

# yum -y install krb5-workstation krb5-libs pam_krb5 cyrus-sasl-gssapi

# now open firewall ports, copy /etc/krb5.conf to the client
scp rhel6u3b:/etc/krb5.conf .

# and create additionally the nfs principal
kinit admin
ipa service-add nfs/rhel6u3a.fluxcoil.net
ipa-getkeytab -s rhel6u3b.fluxcoil.net -p nfs/rhel6u3a.fluxcoil.net -k /etc/krb5.keytab
klist -ekt /etc/krb5.keytab

vi /etc/sysconfig/nfs
# make sure that RPCGSSDARGS="-vvv" is set for debugging - disable this later
# ensure SECURE_NFS="yes" is set in /etc/sysconfig/nfs

# now start the daemons
service rpcbind restart
service rpcidmapd start
service rpcgssd start

tail -f /var/log/messages &

# now mounting should work:
mkdir -p /mnt/tmp
mount -t nfs4 -o sec=krb5 rhel6u3b.fluxcoil.net:/ /mnt/tmp

Logentries of a successful mount:

kernel: Slow work thread pool: Starting up
kernel: Slow work thread pool: Ready
kernel: FS-Cache: Loaded
kernel: Registering the id_resolver key type
kernel: FS-Cache: Netfs 'nfs' registered for caching
rpc.gssd[21508]: dir_notify_handler: sig 37 si 0x7fff8fc7ffb0 data 0x7fff8fc7fe80
rpc.gssd[21508]: dir_notify_handler: sig 37 si 0x7fff8fc7ffb0 data 0x7fff8fc7fe80
rpc.gssd[21508]: dir_notify_handler: sig 37 si 0x7fff8fc7ffb0 data 0x7fff8fc7fe80
rpc.gssd[21508]: dir_notify_handler: sig 37 si 0x7fff8fc7feb0 data 0x7fff8fc7fd80
rpc.gssd[21508]: handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt0)
rpc.gssd[21508]: handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
rpc.gssd[21508]: handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt0)
rpc.gssd[21508]: process_krb5_upcall: service is '<null>'
rpc.gssd[21508]: Full hostname for 'rhel6u3b.fluxcoil.net' is 'rhel6u3b.fluxcoil.net'
rpc.gssd[21508]: Full hostname for 'rhel6u3a.fluxcoil.net' is 'rhel6u3a.fluxcoil.net'
rpc.gssd[21508]: No key table entry found for RHEL6U3A.FLUXCOIL.NET$@FLUXCOIL.NET while getting keytab entry for 'RHEL6U3A.FLUXCOIL.NET$@FLUXCOIL.NET'
rpc.gssd[21508]: No key table entry found for root/rhel6u3a.fluxcoil.net@FLUXCOIL.NET while getting keytab entry for 'root/rhel6u3a.fluxcoil.net@FLUXCOIL.NET'
rpc.gssd[21508]: Success getting keytab entry for 'nfs/rhel6u3a.fluxcoil.net@FLUXCOIL.NET'
rpc.gssd[21508]: Successfully obtained machine credentials for principal 'nfs/rhel6u3a.fluxcoil.net@FLUXCOIL.NET' stored in ccache 'FILE:/tmp/krb5cc_machine_FLUXCOIL.NET'
rpc.gssd[21508]: INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_FLUXCOIL.NET' are good until 1354349335
rpc.gssd[21508]: using FILE:/tmp/krb5cc_machine_FLUXCOIL.NET as credentials cache for machine creds
rpc.gssd[21508]: using environment variable to select krb5 ccache FILE:/tmp/krb5cc_machine_FLUXCOIL.NET
rpc.gssd[21508]: creating context using fsuid 0 (save_uid 0)
rpc.gssd[21508]: creating tcp client for server rhel6u3b.fluxcoil.net
rpc.gssd[21508]: DEBUG: port already set to 2049
rpc.gssd[21508]: creating context with server nfs@rhel6u3b.fluxcoil.net
rpc.gssd[21508]: DEBUG: serialize_krb5_ctx: lucid version!
rpc.gssd[21508]: prepare_krb5_rfc4121_buffer: protocol 1
rpc.gssd[21508]: prepare_krb5_rfc4121_buffer: serializing key with enctype 18 and size 32
rpc.gssd[21508]: doing downcall
kernel: alg: No test for __aes-aesni (__driver-aes-aesni)
kernel: alg: No test for __ecb-aes-aesni (__driver-ecb-aes-aesni)
kernel: alg: No test for __cbc-aes-aesni (__driver-cbc-aes-aesni)
kernel: alg: No test for __ecb-aes-aesni (cryptd(__driver-ecb-aes-aesni))
kernel: padlock: VIA PadLock not detected.

message: rpc.gssd: rpcsec_gss: gss_init_sec_context: (major) Unspecified GSS failure. Minor code may provide more information - (minor) Server not found in Kerberos database
explanation: Stop the rpcgss service and run it in debugmode to see more informations on the issue, i.e. 'rpc.gssd -f -vvvv'.

message: WARNING: KDC has no support for encryption type while getting initial ticket for principal 'nfs/rhel5u8b.fluxcoil.net@FLUXCOIL.NET' from keytab 'FILE:/etc/krb5.keytab'
message: rpc.svcgssd[..]: ERROR: GSS-API: error in handle_nullreq: gss_accept_sec_context(): GSS_S_FAILURE (Unspecified GSS failure. Minor code may provide more information) - Encryption type DES cbc mode with CRC-32 not permitted
explanation: Your client has only a principal with a key of type DES available. While some clients can only deal with this type, this is very insecure and by default not allowed/supported by newer kerberos KDC. To enable this nontheless add 'allow_weak_crypto = true' to the [libdefaults] section of file /etc/krb5.conf on the KDC system.

message: WARNING: Failed to create krb5 context for user with uid 0 for server rhel6u2c.fluxcoil.net explanation: The rpc.gssd on the client could not create the krb5 context. One possible reason is that no principal 'nfs/rhel6u2c.fluxcoil.net' has been created in the KDC and stored in the KDC's keytab.

message: mount.nfs4: Invalid argument explanation: Several possible causes. One: are the required kernel modules loaded? On RHEL5 this has to be done manually: 'rpcsec_gss_krb5'.

message: rpc.svcgssd[123]: ERROR: GSS-API: error in handle_nullreq: gss_accept_sec_context(): GSS_S_FAILURE (Unspecified GSS failure. Minor code may provide more information) - Wrong principal in request
explanation: When changing the principals of client and server the rpc.svcgssd might still have a previous principal cached that is no longer in use. Restart the service to solve the problem.

message: mount.nfs4: Permission denied (on the client, at mount attempt)
explanation: Multiple potential causes. Maybe principals have changed and the rpc.svcgssd daemon on the server has to be restart.

# have all daemons been restarted?

# errors/warnings in /var/log/messages?

# Is rpc.gssd running in debugmode on nfs-client? Activate in /etc/sysconfig/nfs.

# Can the export be seen from the client via 'showmount'?

# Firewalls?

# nfs debugging can be
# activated
echo 32767 > /proc/sys/sunrpc/nfs_debug
# deactivated
echo 0 > /proc/sys/sunrpc/nfs_debug

klist -ke    # which principals are in /etc/krb5.keytab ?