HTTPS configuration no longer works post-Jira upgrade

[Tim Menke]

Currently running Jira Software 7.11.2 on Server 2016 with HTTPS.  Everything works great.

Earlier we tried upgrading to 7.13.1.   The upgrade goes fine, it loads with plain HTTP.  We didn't realize it would wipe out the HTTPS configuration so we roll it back.

A while later, we upgrade again,  again, upgrade succeeds, we copy the HTTPS configuration back.  Our java keystore was located in the install directory so it got deleted as part of the upgrade,  we restore the file to it's original location, however we still can't get HTTPS to load.    After learning that the entire installation gets blown away during an upgrade, we try moving the JKS to a folder outside of the installation directory.  We update server.xml with the new path and restart jira, but HTTPS will still not load.    Ultimately, we've rolled everything back to pre-upgrade again.  oh and we did restore the security constraint to web.xml too so we didn't miss that.  

I'm getting a clone of our Jira server set up so that we can get this worked out before we try upgrading production again.   

But I'm kinda stumped as to what the problem is.  we restore the HTTPS configuration back to server.xml,  we add the security constraint back to web.xml, we restore the JKS and move it to a folder outside of the installation directory (any special permissions required? but it didn't like it when we restored it to it's original directory either), but otherwise it's the same JKS with the same password.   

The only other discrepancy I think of is that there was a cacerts file that got deleted as part of the upgrade, but I don't know why that file was necessary since the server.xml was pointing to the JKS which is where the keys should be.  

So pre-upgrade, HTTPS works, but post-upgrade after restoring the configuration to server.xml, web.xml and updating the path to the jks outside of the installation directory, HTTPS no longer functions.    Only other thing I changed was to have it use TLS1.2 only instead of TLS1.0, 1.1 and 1.2

What else should I be checking when I get this clone going?  Thanks!

[Tim Menke · Accepted solution]

Sigh...I'm such a bonehead.   

Got it up and running on the clone.   performed the upgrade.  Upgrade was successful and loads in plain HTTP.  (2nd time though that the service failed to shutdown properly and I had to manually kill the process)

- edit service.xml and add the HTTPS connector back in, update the redirect port back to 443

- edit web.xml and put the security constraint back in.  

- take a copy of the jks and put it in a folder outside of the installation directory. and update the keystore path in server.xml

- start the service back up and this time I can actually take a look at the catalina logs

"Alias name [jira] does not identify a key entry"

That's all it was, when I rebuilt the HTTPS connector, I used the default alias used in the guides.  but I never noticed that the keystore didn't use that alias and used something different.   Updated the server.xml with the correct key alias and viola, it loads in HTTPS with no problem.  

sigh, can't believe I missed that. 

[Marty]

I'm glad you got to the bottom of it!

[Tim Menke]

I'll be doing the upgrade to the production server tomorrow.  

[Tim Menke]

Upgrade was a success last night!   Still ran into some weird issues, but nothing showstopping.

After updating server.xml and web.xml Jira would not load.  The logs claimed I had a greater than character in my keystore path.   I looked again, but I could not find anything wrong with the path.   Thought I was going to have to roll back again, but on a whim, I changed the backslashes in the path to forward slashes, re-saved, and then it loaded successfully.   Not sure why that mattered.  Backslashes were used before the upgrade, Backslashes worked in the clone upgrade.  

I also had to upgrade my JVM initial pool memory, took a little while but I got that upgraded.

The upgrade report then claimed that the gadget health check failed and that some of my add-ons were incompatible and were disabled, but we had made sure we had updated everything prior to the upgrade to be compatible.   Despite this, when I went to app management, nothing was disabled and all the health checks show as green, so not sure what happened there.  

But overall it all went well, and we've got some experience under our belts for the inevitable upgrade to 8.  

Thanks for the assistance!

[Alexis Robert]

Hi @Tim Menke , 

when you say that "HTTPS doesn't work after the upgrade", do you mean that you get an error when trying to access your instance with https://jira.company.com or do you get an error in the logs ? 

This is the first step to be able to help you so can you provide more details on the error message or logs that you get ?

Otherwise, one thing I can think of that would make your upgrade fail: the cacert file might have been used to update the root certificates to trust an authority that wasn't trusted by default, I've had this happen once with a Lets Encrypt cert or with a self-signed (by my AD server) cert.

Let me know if this helps, 

--Alexis

[Tim Menke]

Sorry, I meant HTTPS times out when trying to load the page post upgrade.    Since we had a short upgrade window, we didn't have a chance to look at any of the error logs.  Which is why we're doing the clone now so that we can have an extended period of time for troubleshooting.  

as for the cacerts file,  one thing I did have time to try was restoring the cacerts file to its original location, but it still didn't load after restarting the service.  but I was trying to move very fast so I might have missed something.  

[Alexis Robert]

When you get your clone up and running, you can also try some debug from the command line as explained here, I find this very helpful as it helps narrow down the problem when dealing with certificates. 

[Marty]

Hi Tim,

I'd recommend raising a support request at support.atlassian.com.

Thanks,

Marty

[Tim Menke]

I thought I read somewhere that Atlassian didn't support HTTPS-related issues since that's more of a tomcat thing.