Unable to connect to LDAP over SSL

[Gayam]

The "User Directory" configuration in Jira works with the regular LDAP on port 389, but the LDAP over SSL on port 636 doesn't work. I tried with JVM Arguments like using TLSV1 only or "plain ssl" as suggested in the community, but couldn't connect to AD over SSL. I am able to connect using Softerra LDAP Client so I know the AD SSL setup is good. The firewall ports are open.

2018-05-22 14:59:30,590 http-nio-8080-exec-16 ERROR O094429 899x404x1 1pcqtfv 10.232.122.46,172.28.128.209 /plugins/servlet/embedded-crowd/configure/ldap/ [c.a.c.embedded.admin.ConfigurationController] Configuration test failed for user directory: [ My Domain Active Directory], type: [ CONNECTOR ]
com.atlassian.crowd.exception.runtime.OperationFailedException: <LDAP HostName>:636; nested exception is javax.naming.CommunicationException:
zq.msds.kp.org:636 [Root exception is java.net.SocketException: Connection reset]
at com.atlassian.crowd.embedded.core.CrowdDirectoryServiceImpl.testConnection(CrowdDirectoryServiceImpl.java:78)
at sun.reflect.GeneratedMethodAccessor665.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
at com.sun.proxy.$Proxy55.testConnection(Unknown Source)

[Gayam · Accepted solution]

Thank you all for you suggestions. The issue was that our firewall was blocking the LDAP SSL traffic on port 636. The simple "telnet <host> <port>" works, but when the application tries to send ldaps traffic, the firewall was blocking it from the server network. It was allowed from our corporate network so we were able to connect to AD over LDAPS from our desktops. Please check with your firewall team to see if they allow LDAPs traffic before troubleshooting further

[Andy Heinzer]

I think from your description, you have already seen the KB JIRA Connection reset error when synchronising with Active Directory 2012r2

 because you have mentioned trying to set the JVM startup parameter of

-Djdk.tls.client.protocols=TLSv1

 Does Jira have the SSL certificate for this address in its keystore?   If not, then it might be possible for other applications to connect to this service using SSL from the same machine, but it could explain why Jira cannot.

I'd recommend walking through Connecting to SSL services if you have not already.

If you have already done that and you still see this same error, then I'd want to see if you can use the SSLPoke utility on Unable to Connect to SSL Services due to PKIX Path Building Failed.  This utility can act as a separate Java application that can use the same $JAVA_HOME as your system.  This way it can be a helpful test to understand if the problem is with the certificate, or perhaps there is something more specific to Jira and the protocol being used.

[Gayam]

Thank you for you reply Andrew. I did try the SSLPoke utility and I get the same result if I have the Root certificates in my cacerts or with a new keystore. The openssl command also doesn't give me any certificates. I am guessing that there is a different way to connect to the LDAP over SSL. The LDAP clients are able to connect but I am not getting the certificates with the Java program or the openssl command. I am trying to setup a new AD in the local environment and try. Please let me know if there are any other methods that I can try.

[Andy Heinzer]

You said you get the same results, but what is the exact response you get with the SSLPoke utility?   Successfully connected?  Or does it show some other connection error?

[Gayam]

Same "Connection Reset"

java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:747)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:138)
at SSLPoke.main(SSLPoke.java:31)

[Philipp Sendek]

Hi @Gayam,

we just encounter the same issue at a client and I was wondering if you were able to solve it.

Would appreciate your feedback.

Regards,
Philipp

[Philipp Sendek]

Hello @Andy Heinzer,

we have the exact same issue at a client. 
We have cloned their production Confluence and upgraded it to 6.13 which results in the SocketException: Connection reset whenever C tries to establish a connection with LDAP.

Using openssl just gives this (also no cert):

openssl s_client -connect decos005p.de.xxxxx.com:636
CONNECTED(00000004)
write:errno=131
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 295 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
 Protocol : TLSv1.2
 Cipher : 0000
 Session-ID:
 Session-ID-ctx:
 Master-Key:
 Key-Arg : None
 PSK identity: None
 PSK identity hint: None
 SRP username: None
 Start Time: 1547558089
 Timeout : 300 (sec)
 Verify return code: 0 (ok)

Using SSLPoke gives the same error, even with adding the jdk.tls.client.protocols=TLSv1 parameter:

 ./java -Djavax.net.ssl.trustStore=../lib/security/cacerts -Djdk.tls.client.protocols=TLSv1 SSLPoke decos005p.de.xxxxxx.com 636
java.net.SocketException: Connection reset
 at java.net.SocketInputStream.read(SocketInputStream.java:210)
 at java.net.SocketInputStream.read(SocketInputStream.java:141)
 at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
 at sun.security.ssl.InputRecord.read(InputRecord.java:503)
 at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
 at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
 at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:750)
 at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
 at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:138)
 at SSLPoke.main(SSLPoke.java:31)

As long as this is an issue, users cannot test the new version of Confluence and we won't be able to upgrade the production system.

I'll appreciate your feedback.

Regards,
Philipp Sendek

[Andy Heinzer]

Hi @Gayam

Sorry I didn't reply to this thread sooner.   Given your error in Jira, and the similar error in the SSLPoke tool, I'm not convinced this is a certificate problem just yet as I had initially anticipated.  If it was we would see a PKIX path building error.  But instead it sounds like the remote server (the LDAP/AD server) is resetting the connection on Jira way before that could even happen.  Why it's doing that, I can't say just yet.

Also, given that you have tried to use the TLS 1.0 and the 'plain ssl' JVM startup configurations and they have not helped, it's likely that your environment is not actually affected by the exact scenarios that we currently have KB articles for. 

That said, I think it would be helpful if we were to follow the diagnostic steps from one of these KBs: Connecting JIRA to Active Directory over LDAPS fails with "Connection reset"

Diagnostic Steps

• Analyzing a tcpdump generated during the synchronization attempt ...

We might not get the exact same results as the KB is talking about, but I think at this point, taking a closer look at a tcpdump at the time that Jira tries to connect to this server using SSL would prove insightful at least to let us know more about this problem.  Could you also let me know what version of Jira you're using, and What version of Active Directory (Windows Server) you are using here?

@Philipp Sendek _brainbits_ I would also be interested to see if we can learn more about your environment, such as AD version, and if we can take a closer look at a tcpdump when Confluence attempts to establish this SSL connection.

[Philipp Sendek]

 Hi @Gayam and @Andy Heinzer,

After some more digging, I share Andrew's suspicion that the LDAP Server seems to be killing the connection before anything happens.

The client has different VMs and the LDAP connection works without any additional tasks on all except this one.
To get some more info, I ran openssl and SSLPoke on another VM and it worked without issues. There I get the full output including certificate and its detailled information.

@Gayam: So while all your LDAP clients work on your network, maybe you (or the LDAP team) can verify, that there is no whitelisting missing that could block Confluence from correctly communicating with LDAP *inside* of the LDAP server? It can of course not be a simple network whitelist as the network connection can be established, it just doesn't go forward from there.

Unfortunately, I'm not deeply familiar with what's possible in MS AD in these regards, but my tests lead me to the conclusion that it's neither a Confluence nor really a Java problem (since openssl's request is also not complete).

I hope that helps. I'll keep you up to date if we made any progress on our issue.

Regards,
Philipp

[Gayam]

No, this still remains unsolved. Hopefully someone from Atlassian can validate and answer :(