Lessons Learned: Kerberos Debugging in Java

Working with Kerberos can easily cause a lot of trouble. Troubleshooting can take several hours.
In this blog I'll show you what will help you best when using Kerberos with Java for example to secure a Hadoop cluster.

When Kerberos is not working as expected it is important to understand why. Enabling Kerberos debug logging is a very valuable resource to understand what is happening.
To enable Kerberos debugging you need to set the following JVM property:

-Dsun.security.krb5.debug=true

Now read your log file very carefully. This will help you to understand what is missing.

Usually you will define your Kerberos configuration within your C:\Windows\krb5.ini or /etc/krb5.conf file. Make sure that your hostname mapping to your Kerberos realm is correct in here.
There are also a few other JVM properties that are usually not required, but can be useful to override/define your configuration at application startup:

-Djava.security.krb5.kdc=hostname.of-your.kerberos.server
-Djava.security.krb5.realm=YOUR.KERBEROS.REALM
-Djava.security.auth.login.config=file:/C:/Programme/Tomcat-IDP/conf/kerberos.jaas

Kerberos is very sensitive to DNS configuration.

Here are some more shell commands that are very helpful to test if Kerberos is working in general (outside of your Java application):

# Login with a specific keytab file
kinit -k -t /path/to/your/keytab

# List all local available tokens. After kinit there should be at least your tgt token.
klist

# Request a ticket for a specific service. Check if the service is registered correctly at your Kerberos server.
kvno service/hostname@domain

https://web.mit.edu/kerberos/krb5-1.12/doc/user/user_commands/kvno.html

Here is a sample configuration for your krb5.ini file:

[libdefaults]
default_realm = HORTONWORKSHA.COM
dns_lookup_kdc = false
dns_lookup_realm = false
ticket_lifetime = 86400
renew_lifetime = 604800
forwardable = true
default_tgs_enctypes = rc4-hmac
default_tkt_enctypes = rc4-hmac
permitted_enctypes = rc4-hmac
udp_preference_limit = 1
kdc_timeout = 3000

[realms]
CLOUDERAHA.COM = {
  kdc = talend.cloudera57
  admin_server = talend.cloudera57
}
HORTONWORKSHA.COM = {
  kdc = hdp24masternode
  admin_server = hdp24masternode
}

[domain_realm]
.hortonworksha.com = HORTONWORKSHA.COM
hortonworksha.com = HORTONWORKSHA.COM
.clouderaha.com = CLOUDERAHA.COM
clouderaha.com = CLOUDERAHA.COM

And here is another example for a JAAS configuration file (first for a normal user and second for a technical service account):

alice { 
    com.sun.security.auth.module.Krb5LoginModule required
    refreshKrb5Config=true
    debug=true
    principal="alice";
};

sts { 
    com.sun.security.auth.module.Krb5LoginModule required
    debug=true
    refreshKrb5Config=true
    useKeyTab=true
    storeKey=true
    keyTab="file:c:/Users/jbernhardt/workspace/kerberos/sts.keytab"
    principal="sts/tal-csg01";
};

If you want to define the location for your cached tickets, set the following system property accordingly:

# Windows Style
set KRB5CCNAME="C:\Users\jbernhardt\krb5cc_jbernhardt"

# Linux Style
export KRB5CCNAME="/home/jbernhardt/krb5cc_jbernhardt"

Sample Java Code:

package test;

import java.io.IOException;
import java.net.URL;
import java.security.Principal;
import java.util.Set;

import javax.security.auth.Subject;
import javax.security.auth.callback.Callback;
import javax.security.auth.callback.CallbackHandler;
import javax.security.auth.callback.NameCallback;
import javax.security.auth.callback.PasswordCallback;
import javax.security.auth.callback.UnsupportedCallbackException;
import javax.security.auth.login.LoginContext;
import javax.security.auth.login.LoginException;

public class JaasLoginTest {

    public static void main(String argv[]) {
        URL conf = JaasLoginTest.class.getClassLoader().getResource("jaas.conf");
        System.setProperty("java.security.auth.login.config", conf.toString());
        System.setProperty("java.security.krb5.realm", "TEST.TALEND.DE");
        System.setProperty("java.security.krb5.kdc", "192.168.0.100");
        System.setProperty("sun.security.krb5.debug", "true");
        
        // Only needed when not using the ticket cache
        CallbackHandler callbackHandler = new CallbackHandler() {
            @Override
            public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
                for (Callback callback : callbacks) {                
                    if (callback instanceof NameCallback) {
                        ((NameCallback)callback).setName("alice");
                    }
                    if (callback instanceof PasswordCallback) {
                        ((PasswordCallback)callback).setPassword("password".toCharArray());
                    }
                }
                
            }
        };

        try {
            LoginContext lc = new LoginContext("alice", callbackHandler);
//            LoginContext lc = new LoginContext("sts", callbackHandler);
            lc.login();
            Subject subject = lc.getSubject();
            Set<Principal> principals = subject.getPrincipals();
            Set<Object> credentials = subject.getPrivateCredentials();
            System.out.println("OK: " + principals);
            System.out.println("OK: " + credentials);
        } catch (LoginException e) {
            e.printStackTrace();
        } 
    }
}

Lessons Learned

06 February 2017

Kerberos Debugging in Java

Useful Links

1 comment: