How to diagnose AD integration problems?

Issue

  • You are trying to activate the authentication to an AD service and you don’t succeed.
  • Authentication via AD is set in Jenkins, but certain users cannot log in (“UserX”).

Environment

  • Jenkins
  • CloudBees Jenkins Enterprise (CJE)
  • CloudBees Jenkins Operation Center (CJOC)
  • Active Directory (AD) plugin
  • Active Directory server

Resolution

  • Note: If this is the first time you are integrating Jenkins with an AD server and there are not security restrictions on the instance, you might want to configure Security as Security Realm : Active Directory, keeping Authorisations to Anyone can administrate Jenkins. This will allow you to update the Security configuration and try to login without loosing your administrator access

I. Jenkins should be using the AD Global Catalog

Microsoft Domains can expose the LDAP and/or the Global Catalog.

The most typical integration issue between Jenkins and AD servers comes when using the LDAP catalog (ports 389/TLS:636) instead of the Global Catalog (ports 3268/TLS:3269). When using the LDAP catalog a Domain Controller could redirect you to a different server to look for a specific user/group. However, when Domain Controllers expose the Global Catalog store a full copy of all objects in the directory for its host domain and a partial, read-only copy of all objects for all other domains in the forest. Basically, this means that you will not go out from this server when looking for users/groups. This makes the login process more reliable, faster and less error prone.

To check if your server is exposing or not a global catalog you can run the command below from Jenkins server machine. This will provide you the list of all the servers which are exposing the global catalog. If there is not any server, then you should convince your AD administrator to set it up.

nslookup -q=SRV _gc._tcp.<DOMAIN_NAME>

On the other hand, if you want to check if the Domain is correctly exposing the LDAP catalog, you can use.

nslookup -q=SRV _ldap._tcp.<DOMAIN_NAME>

If you are not specifying any Domain Controller at plugin level (Domain Controller field is blank), the ad-plugin will always try to use the Global Catalog by default, but in case this one is not exposed, then it will fall back into the LDAP catalog.

II. Domain Controllers health

The second most common source of login failures comes when you are not using any Domain Controller at plugin level and there are broken servers exposed by the domain.

If your logins are not reliable, you should not rely only in the Domain - until the issue is caught- and set-up one or more Domain Controllers in the plugin separate by comma. i.e domain-controller-1.example.com:3268,domain-controller-2.example.com:3268.

For the moment, at plugin level there is not any automatic mechanism to provide you the list of failed servers. One simple thing you can do is go through the list of all the servers provided by the command nslookup -q=SRV _gc._tcp.<DOMAIN_NAME> or nslookup -q=SRV _ldap._tcp.<DOMAIN_NAME> - depends on the catalog you are using - and check if the login is or not success. For this, you can use the Test Connection button with a single Domain Controller and a bindDn. Once you did the test, then you can pass to the next Domain Controller to check if the connection is successful or not. So the idea is to check per each Domain Controller if the the connection is successful with the Test connection button.

III. DNS resolution health

DNS resolution of the Domain Name should work and must be correct.

nslookup <DOMAIN_NAME>
  • Note: In case we are specifying Domain Controllers at plugin level, the plugin will work even if the resolution of DNS is not correct.

IV. Ensure the binddn is correct

For this task, graphical tools such as Active Directory Administrative Center might be helpful to understand the forest of your AD service. Click on the User to use as Manager DN > Attributed Editor Tab > distinguishedName field.

V. Create a logger to understand why the login is failing

Create/Update a dedicated Logs recorder (AD) as explained in Jenkins Logging including:

  • hudson.plugins.active_directory = ALL

Having done that, first test your AD settings and then try to log in with a AD user (“UserX”).

VI. Use ldapsearch command

With the help of ldapsearch command you can validate your AD settings in Jenkins, which makes use those LDAP attributes by the Java Naming and Directory InterfaceTM (JNDI) for accessing to naming and directory services.

On Linux you can use ldapsearch by installing the package openldap-clients. On the other hand, on Windows you will need to install a LDAP client like Open Ldap.

Searching for a certain user

Note: The following commands works for under ldap:// connections (not ldaps://)

Method 1
ldapsearch -LLL -H ldap://<DOMAIN_NAME> -s subtree -b "<searchbase>" -D "<bindn>" -w "<passwd>" "(& (userPrincipalName=<userid>@<DOMAIN_NAME>)(objectCategory=user))"

Note: By default, userPrincipalName is equivalent to the userid unless the AD admin changes it.

Method 2 (in case Method 1 fails)
ldapsearch -LLL -H ldap://<DOMAIN_NAME> -b "<searchbase>" -D "<binddn>" -w "<passwd>" "(& (sAMAccountName=<userid>)(objectCategory=user))"

Note: By default, sAMAccountName is equivalent to the userid unless the AD admin changes it.

Searching for a certain group
ldapsearch -LLL -H ldap://<DOMAIN_NAME> -s subtree -M -b "<searchbase>" -D "<binddn>" -w "<passwd>" "(& (cn=<groupid>)(objectCategory=group))"

Mapping ldapsearch - Jenkins AD plugin configuration: The above ldapsearch fields should match with the following fields in the below AD plugin configuration:

Under $JenkinsURL/configureSecurity/ > Security Realm > AD setting

  • <DOMAIN_NAME> -> Domain Name: support-cloudbees.com

  • <binddn> -> Bind DN. In the above image, CN=felix, OU=Support, DC=example, DC=com

  • <passwd> -> Bind Password

Note that <searchbase> is not a field for AD plugin configuration. It is the organization unit we want to look into and it could be the root DN (DC=support-cloudbees, DC=com) or another one from a specific branch, OU=Support, DC=support-cloudbees, DC=com

On the other hand, in $JenkinsURL/login/, <userid> is the user you would like to authenticated in Jenkins. It can be the managerDN itself or for a different user on the tree.

Notes:

  1. Execute the commands above in the Jenkins Server side, ldapsearch is included in ldap-utils for Linux based System. Please, for testing purposes, install ldap-utils in the same server as Jenkins.
  2. Take care to escape special character with \ in case it is necessary.
  3. Use ldaps just for TLS (SSL) end-points (ldaps://<DOMAIN_NAME>).
  4. Regarding TCP and UDP ports by default ldap is on 389, or on port 636 for ldaps. Microsoft Global Catalog is available by default on ports 3268, and 3269 for ldaps.
  5. For the following commands, in case you want to avoid your password to get discovered, -w "<passwd>" can be replaced by:
    • -W, which it will ask you for the password.
    • -y ./pass.txt, so /pass.txt contains your credentials.

EXAMPLE/SCENARIO

Check I: Check if Jenkins is using the AD Global Catalog

Input:

nslookup -q=SRV _gc._tcp.example.net

Expected output:

Server:		127.0.1.1
Address:	127.0.1.1#53

Non-authoritative answer:
_gc._tcp.example.net	service = 0 100 389 dcpr2.example.net.

Authoritative answers can be found from:
dcpr2.example.net	internet address = 192.168.1.81

Check III: Check that the DNS resolution of the Domain Name is correct

Input:

nslookup example.net

Expected output:

Server:		127.0.1.1
Address:	127.0.1.1#53

Name:	example.net
Address: 192.168.1.81

Check IV: Ensure the binddn is correct

In this case, we are looking for the distinguishedName of the user “fari” : CN=fari,OU=users,OU=support1,DC=example,DC=com.

Check V: Create a logger to understand why the login is failing

Logging with a user

After logging successfully with a user (“exampleUser”), a similar output like this would be expected:

oct 11, 2016 3:15:47 PM FINNEST hudson.plugins.active_directory.ActiveDirectorySecurityRealm$DescriptorImpl bind

Binding as CN=exampleUser,OU=users,OU=support1,DC=example,DC=com to ldap://192.168.1.80:3268/

oct 11, 2016 3:15:47 PM FINNEST hudson.plugins.active_directory.ActiveDirectorySecurityRealm$DescriptorImpl bind

Bound to 192.168.1.80:3268

oct 11, 2016 3:15:47 PM  FINNEST hudson.plugins.active_directory.LDAPSearchBuilder search

searching (& (userPrincipalName={0})(objectCategory=user))[exampleUser@example.com] in DC=example,DC=com using {java.naming.referral=follow, java.naming.ldap.version=3, java.naming.security.principal=CN=exampleUser,OU=users,OU=support1,DC=example,DC=com, java.naming.ldap.attributes.binary=tokenGroups objectSid, java.naming.provider.url=ldap://192.168.1.80:3268/, com.sun.jndi.ldap.read.timeout=60000, java.naming.security.credentials=…} with scope 1 returning null

oct 11, 2016 3:15:47 PM  FINNEST hudson.plugins.active_directory.LDAPSearchBuilder searchOne

no result

oct 11, 2016 3:15:47 PM  FINNEST hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider resolveGroups

Looking up group of CN=exampleUser,OU=users,OU=support1,DC=example,DC=com

oct 11, 2016 3:15:47 PM  FINNEST hudson.plugins.active_directory.LDAPSearchBuilder search

searching (|(objectSid={0})(objectSid={1}))[[B@46cf1e4e, [B@515a1738] in DC=example,DC=com using {java.naming.referral=follow, java.naming.ldap.version=3, java.naming.security.principal=CN=fari,OU=users, OU=support1,DC=example,DC=com, java.naming.ldap.attributes.binary=tokenGroups objectSid, java.naming.provider.url=ldap://192.168.1.80:3268/, com.sun.jndi.ldap.read.timeout=60000, java.naming.security.credentials=…} with scope 2 returning [cn]

oct 11, 2016 3:15:47 PM FINNEST hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider parseMembers

CN=exampleUser,OU=users,OU=support1,DC=example,DC=com is a member of cn: Users

oct 11, 2016 3:15:47 PM FINNEST hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider parseMembers

CN=exampleUser,OU=users,OU=support1,DC=example,DC=com is a member of cn: Domain Users

oct 11, 2016 3:15:47 PM FINNEST hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider resolveGroups

Stage 2: looking up via memberOf

oct 11, 2016 3:15:47 PM  FINNEST hudson.plugins.active_directory.LDAPSearchBuilder search

searching (member:1.2.840.113556.1.4.1941:={0})[CN=exampleUser,OU=users,OU=support1,DC=example,DC=com] in DC=example,DC=com using {java.naming.referral=follow, java.naming.ldap.version=3, java.naming.security.principal=CN=fari,OU=users, OU=support1,DC=example,DC=com, java.naming.ldap.attributes.binary=tokenGroups objectSid, java.naming.provider.url=ldap://192.168.1.80:3268/, com.sun.jndi.ldap.read.timeout=60000, java.naming.security.credentials=…} with scope 2 returning [cn]

oct 11, 2016 3:15:47 PM ADVERTENCIA hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider resolveGroups

Group lookup via Active Directory's 'LDAP_MATCHING_RULE_IN_CHAIN' extension failed. Falling back to recursive group lookup strategy for this and future queries

oct 11, 2016 3:15:47 PM  FINNEST hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider recursiveGroupLookup

Looking up group of {tokengroups=tokenGroups: [B@46cf1e4e, [B@515a1738, cn=cn: exampleUser}

oct 11, 2016 3:15:47 PM FINNEST org.acegisecurity.ui.AbstractProcessingFilter successfulAuthentication

Authentication success: org.acegisecurity.providers.UsernamePasswordAuthenticationToken@b9e0c74e: Username: hudson.plugins.active_directory.ActiveDirectoryUserDetail@0: Username: exampleUser; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: authenticated, Domain Users, Users: Username: exampleUser; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: authenticated, Domain Users, Users; Password: [PROTECTED]; Authenticated: true; Details: org.acegisecurity.ui.WebAuthenticationDetails@fffdaa08: RemoteIpAddress: 127.0.0.1; SessionId: 10sp08o3cn7yl1lciqqf60341p; Granted Authorities: authenticated, Domain Users, Users

oct 11, 2016 3:15:47 PM FINNEST org.acegisecurity.ui.AbstractProcessingFilter successfulAuthentication

Updated SecurityContextHolder to contain the following Authentication: 'org.acegisecurity.providers.UsernamePasswordAuthenticationToken@b9e0c74e: Username: hudson.plugins.active_directory.ActiveDirectoryUserDetail@0: Username: exampleUser; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: authenticated, Domain Users, Users: Username: exampleUser; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: authenticated, Domain Users, Users; Password: [PROTECTED]; Authenticated: true; Details: org.acegisecurity.ui.WebAuthenticationDetails@fffdaa08: RemoteIpAddress: 127.0.0.1; SessionId: 10sp08o3cn7yl1lciqqf60341p; Granted Authorities: authenticated, Domain Users, Users'

oct 11, 2016 3:15:47 PM FINNEST org.acegisecurity.ui.AbstractProcessingFilter successfulAuthentication

Redirecting to target URL from HTTP Session (or default): /manage

oct 11, 2016 3:15:47 PM FINNEST jenkins.security.SecurityListener fireLoggedIn

logged in: exampleUser

Check VI: Use ldapsearch command

Searching for a certain user

In this case: “carlos”.

Method 1

Input:

ldapsearch -LLL -H ldap://example.com -s subtree -b "DC=example,DC=com" -D "CN=carlos,OU=users,OU=support1,DC=example,DC=com" -w "Casa*1234" "(& (userPrincipalName=carlos@example.com)(objectCategory=user))"

Method 2

Input:

ldapsearch -LLL -H ldap://example.com -b "DC=example,DC=com" -D "CN=carlos,OU=users,OU=support1,DC=example,DC=com" -w "Casa*1234" "(& (sAMAccountName=carlos)(objectCategory=user))"
Method 1 or 2 outputs

Same expected output are expected from 1 or 2

dn: CN=carlos,OU=users,OU=support1,DC=example,DC=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: carlos
givenName: carlos
distinguishedName: CN=carlos,OU=users,OU=support1,DC=example,DC=com
instanceType: 4
whenCreated: 20160712213231.0Z
whenChanged: 20160804013914.0Z
displayName: carlos
uSNCreated: 12776
memberOf: CN=secGroup1,OU=groups,OU=support1,DC=example,DC=com
uSNChanged: 36927
name: carlos
objectGUID:: eSbpnL0Hr0CzY11yV0oQEQ==
userAccountControl: 512
badPwdCount: 0
codePage: 0
countryCode: 0
badPasswordTime: 131147485555082170
lastLogoff: 0
lastLogon: 131147486006966352
pwdLastSet: 131147483547580111
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAA5A3hZHM0MKDZmdb+UgQAAA==
accountExpires: 9223372036854775807
logonCount: 0
sAMAccountName: carlos
sAMAccountType: 805306368
userPrincipalName: carlos@example.com
lockoutTime: 0
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=example,DC=com
dSCorePropagationData: 20160728202804.0Z
dSCorePropagationData: 16010101000000.0Z
lastLogonTimestamp: 131142167128858572

# refldap://ForestDnsZones.example.com/DC=ForestDnsZones,DC=example,DC=com

# refldap://DomainDnsZones.example.com/DC=DomainDnsZones,DC=example,DC=com

# refldap://example.com/CN=Configuration,DC=example,DC=com

Searching for a certain group

In this case: “secGroup1”.

ldapsearch -LLL -H ldap://example.com -s subtree -M -b "DC=example,DC=com" -D "CN=carlos,OU=users,OU=support1,DC=example,DC=com" -w "Casa*1234" "(& (cn=secGroup1)(objectCategory=group))"

Expected output:

dn: CN=secGroup1,OU=groups,OU=support1,DC=example,DC=com
objectClass: top
objectClass: group
cn: secGroup1
member: CN=carlos,OU=users,OU=support1,DC=example,DC=com
member: CN=joony,OU=users,OU=support1,DC=example,DC=com
distinguishedName: CN=secGroup1,OU=groups,OU=support1,DC=example,DC=com
instanceType: 4
whenCreated: 20160728202832.0Z
whenChanged: 20160728202927.0Z
uSNCreated: 20505
uSNChanged: 20512
name: secGroup1
objectGUID:: bdeiKJHV70u2Xw/ClB5z5A==
objectSid:: AQUAAAAAAAUVAAAA5A3hZHM0MKDZmdb+UwQAAA==
sAMAccountName: secGroup1
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=example,DC=com
dSCorePropagationData: 16010101000000.0Z

# refldap://ForestDnsZones.example.com/DC=ForestDnsZones,DC=example,DC=com

# refldap://DomainDnsZones.example.com/DC=DomainDnsZones,DC=example,DC=com

# refldap://example.com/CN=Configuration,DC=example,DC=com

References

Was this article helpful?
1 out of 1 found this helpful

Have more questions?

0 Comments

Please sign in to leave a comment.