Nginx + Bitbucket + SSL

[Robert McMonagle]

I'm trying to get Bitbucket working with a SSL and no port number however it just doesn't seem to want to work. I'm on version 7.1 and I've modified the properties file and the nginx config file for default as instructed and I just cant get it to work. Here is my code

Nginx conf: 

server {

listen 80;

server_name git.example.com;

rewrite ^ https://git.example.com$request_uri? permanent;

}




server {
listen 80;
server_name git.example.com;
root /var/www/bitbucket;

location / {

return 301 https://$host$request_uri;
}

}

server {
listen 443 ssl;
server_name git.example.com;
root /var/www/bitbucket;

ssl on;
ssl_certificate /etc/letsencrypt/live/git.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/git.example.com/privkey.pem;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;

location /secure/ForgotLoginDetails.jspa {
return 301 https://git.example.com;
}

location /bitbucket {
proxy_pass http://localhost:7990/;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_redirect off;
}

}

/var/atlassian/application-data/bitbucket/shared/bitbucket.properties:

:server.port=7990
server.secure=true
server.scheme=https
server.proxy-port=443
server.proxy-name=git.example.com
server.context-path=/bitbucket

 
Right now all I get is  

403 Forbidden

Please help! Been trying for two days to make this work. Thank you!!

[Robert McMonagle]

Ok I completely re-installed everything. I had to do this because I needed to change the database... but regardless I re-installed everything. Setup based on your precise code and the result is unfortunately not working.

If I type in https://git.domain.com 

it takes me to: http://localhost:7990/dashboard 

which is not https and not even the domain it actually inserts localhost and says this site cannot be reached. 

However if I manually type in http://git.domain.com:7990 it will load the site however not be secure just as lastime. Perhaps something is wrong in the proxy you typed? 

Thanks for the help! 

[Michael Walker]

If you use port 7990, it will always be insecure as this implies that you are bypassing the proxy as the proxy is only listening for requests on port 80 and 443. This will be the case even if the proxy server and Bitbucket are configured correctly.

Additionally, I would like to confirm that after updating the bitbucket.properties config, that is is still owned by the default bitbucket user "atlbitbucket". If this is not the case, then Bitbucket will not be able to read the file with the given "proxy-port" and "proxy-name" flags that you will have needed to have added beforehand. Once those flags are in place and assuming the permissions are still correct, Bitbucket will need to be fully restarted so that Bitbucket can read in those arguments during startup.

You will also need to update the base URL of Bitbucket by navigating in the web UI to the admin panel > "server settings". Once saved, Bitbucket will attempt to redirect you to "https://git.domain.com" when interacting with many internal links.

[Michael Walker]

I would also like to add that if you continue to have issues, removing the SSL configuration and simply ensuring that normal HTTP requests can be proxied from Nginx to Bitbucket would be the next step, reducing the complexity of the issue until we are able to identify why we are not receiving the expected behavior.

This would involve a very simple Nginx block with just the use of port 80 to see if http://git.domain.com can be used to proxy Bitbucket.

server { 
    listen 80;
    server_name git.domain.com;
    location / {
        proxy_pass http://127.0.0.1:7990/
    }
}

[Robert McMonagle]

If I remove 7990 from the url link in the server settings and save it then go to just the domain name without putting in 7990 it redirects me to the url : localhost:7990/dashboard  and obviously nothing is there because it has the word localhost and not my domain name or ip.. 

If i try the test you provided with the server block just listening on port  80 the site loads fine however if you attempt to login it results in a HTTP 500 Status error. 

[Robert McMonagle]

I might just give up on this and use it unsecure which is disappointing as this is our first experience with bitbucket and gitlab which we previously used in the past was secure able instantly with out any issues. But it is just taking up too much time to just have this secure. 

[Robert McMonagle]

Do you possibly know the correct number of connections that Bitbucket should be set to when using a connection pool? I had it at 3 then set it to 6. 

and added: db.pool.idle.testInterval=1 

to my properties but still seems like it gets 500 error at various times 

[Robert McMonagle]

I had to delete Bitbucket AGAIN! It just seems like the Atlassian programs want to use up too many Database connections! as well as have issues with SSL. I really ant to use the software so I'm going to start all over again without SSL this time and a new database. I'm concerned its going to use up too many database connections again however. 

I'm attempting to install bitbucket, jira, trello, and confluence to test them so hopefully this wont burn all the connections. However so far I'm just not happy with my experience with the software. 

[Michael Walker]

I'm sorry to hear that you keep having to reinstalling the application, however, with only just the bitbucket.properties file changing, there shouldn't be any reason or need to do so. You can simply comment out the offending customization lines in the config to return the application to it's original configuration.

As for the Database, Bitbucket uses a connection pool of up to 80 concurrent connections by default. Limiting the pool down to such a small number would be severely detrimental to the normal operation of Bitbucket so we recommend against reducing the number so greatly.

Lastly, I have confirmed the syntax and spelling of the configurations and can confirm they work under normal environments as I am running the exact config that I provided to you in one of my test environments. With this in mind, I definitely recommend switching to a non-SSL test scenario where you simply configure Nginx to use plain text HTTP to confirm if we can reach a scenario with expected behavior.

I should note that if not using SSL, you should be able to connect to Bitbucket through a reverse proxy without any configuration changes to Bitbucket or any Atlassian application what so ever if the proxy is configured correctly. Production environments will still require some tweaks but simply accessing the application and logging in should be completely possible without any changes to the application. This should be true of nearly all web applications. With that said, if you have any other customizations in your reverse proxy, I would suggest commenting them out during your initial testing/setup.

[Michael Walker]

Hi Robert,

Looking at your config, I see an issue around your context path configuration. Currently, you are using the line "proxy_pass http://localhost:7990" which is normally correct, however, because you updated Bitbucket's configuration with the line "server.context-path=/bitbucket" the following is happening:

Client:
GET https://git.example.com/bitbucket/login
Nginx proxy_passes the request:
GET http://localhost:7990/login (stripping out the "bitbucket" part due to the location block)
Bitbucket:
receives GET http://localhost:7990/login but drops it because all requests need to start with "/bitbucket" due to the application-level configuration (server.context-path=/bitbucket)

Because Bitbucket itself is listening to/expecting all URIs to prefix with "/bitbucket" your client would actually need to enter "...:443/bitbucket/bitbucket/login" to reach "/bitbucket/login" the login page as expected.

To fix this, simply update your proxy_pass line to match the below and reload/restart Nginx.

proxy_pass http://localhost:7990/bitbucket

Cheers!

[Michael Walker]

Side Note, sorry for the messy example, I tried to write it 3 different ways and it never came out quite how I wanted it but I hope it makes sense. If not, give me a shout!

[Robert McMonagle]

That didn't seem to work. I don't mind using the domain with the port and no /bitbucket

however the main issue is if you goto https://git.domain.com:7990 it won't load however if you goto http://git.domain.com:7990 without the https it will load.

I've removed the /bitbucket for now and just have location set to /. 

[Michael Walker]

Robert,

Since we know that bitbucket does work with http://git.domain.com:7990 that means that Bitbucket itself is working as expected. In order to switch to https, you would need to either ensure the route passes through your Nginx (which is listening on port 80 and 443 by the looks of it) or you would have to provide the ssl configuration to Bitbucket itself on the tomcat layer. Continuing to use Nginx should be the easier solution though so let me provide what I believe should be your expected config (based on the info you've already given us) for both Bitbucket and Nginx going forward. 

Bitbucket.properties

server.port=7990
server.secure=true
server.scheme=https
server.proxy-port=443
server.proxy-name=git.example.com

Please note that you must restart Bitbucket after a change to the properties config as it is only read during startup.

Nginx

server {
    listen 80;
    server_name git.example.com;

    location / {
        return 301 https://$host$request_uri;
    }
}
server {
    listen 443 ssl;
    server_name git.example.com;

    ssl on;
    ssl_certificate /etc/letsencrypt/live/git.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/git.example.com/privkey.pem;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    location / {
        proxy_pass http://localhost:7990/;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_redirect off;
    }
}

The above is pretty basic but should work out of the box for you. From there, just have a client try https://git.example.com and it should work. You're welcome to then re-introduce the other customizations (like your extra location blocks) one at a time to better understand their changes.

Give the above a try and let me know if you have any luck accessing the Bitbucket then.

[Robert McMonagle]

Thank you I will try this however I have had issues stopping and starting Bitbucket as well. If I run the command 

bin/stop-bitbucket.sh

from /var/www/bitbucket where I have it installed the server does not even seem to actually stop. I was able to kill the process once however this resulted in me not being able to start the service again and had to re-install.  

[Michael Walker]

Robert,

This is typically due to a permissions issue. When Bitbucket is installed, if you choose to add the service user, it will install everything under the OS  user "atlbitbucket". Please ensure that you use this user account when starting/stopping the application otherwise some file permissions can get off and prevent future attempts to start.

Do note, that if the above does happen, you can always reset the permissions back to "atlbitbucket" by running the following:

sudo chown -R atlbitbucket:atlbitbucket /var/atlassian/application-data/bitbucket /opt/atlassian/bitbucket

Going forward, to make sure you use "atlbitbucket" you can run the below:

# Use the service as it automatically associates the ownership correctly
sudo service atlbitbucket start/stop/restart
# Or utilize the atlbitbucket when calling the start/stop manually
sudo su atlbitbucket -c "/opt/atlassian/bitbucket/<version>/bin/start-bitbucket.sh (or stop-bitbucket.sh)

You shouldn't have to ever worry about reinstalling, a quick permission reset should do the trick

[Michael Walker]

Oh, I should explain the paths I included in my last post.

/var/atlassian/application-data/bitbucket is the default for the $BITBUCKET_HOME (or data path)

/opt/atlassian/bitbucket is the default base folder for the <Bitbucket-install> (or binaries path)

If you specified custom paths, just update them in the commands I provided in my last reply

[Robert McMonagle]

Ok I completely re-installed everything. I had to do this because I needed to change the database... but regardless I re-installed everything. Setup based on your precise code and the result is unfortunately not working.

If I type in https://git.domain.com 

it takes me to: http://localhost:7990/dashboard 

which is not https and not even the domain it actually inserts localhost and says this site cannot be reached. 

However if I manually type in http://git.domain.com:7990 it will load the site however not be secure just as lastime. Perhaps something is wrong in the proxy you typed? 

Thanks for the help! 

[Robert McMonagle]

Ok I completely re-installed everything. I had to do this because I needed to change the database... but regardless I re-installed everything. Setup based on your precise code and the result is unfortunately not working.

If I type in https://git.domain.com 

it takes me to: http://localhost:7990/dashboard 

which is not https and not even the domain it actually inserts localhost and says this site cannot be reached. 

However if I manually type in http://git.domain.com:7990 it will load the site however not be secure just as lastime. Perhaps something is wrong in the proxy you typed? 

Thanks for the help! 

[Robert McMonagle]

Ok I completely re-installed everything. I had to do this because I needed to change the database... but regardless I re-installed everything. Setup based on your precise code and the result is unfortunately not working.

If I type in https://git.domain.com

it takes me to: http://localhost:7990/dashboard

which is not https and not even the domain it actually inserts localhost and says this site cannot be reached.

However if I manually type in http://git.domain.com:7990 it will load the site however not be secure just as lastime. Perhaps something is wrong in the proxy you typed?

Thanks for the help!

[Robert McMonagle]

Ok I completely re-installed everything. I had to do this because I needed to change the database... but regardless I re-installed everything. Setup based on your precise code and the result is unfortunately not working.

If I type in https://git.domain.com

it takes me to: http://localhost:7990/dashboard

which is not https and not even the domain it actually inserts localhost and says this site cannot be reached.

However if I manually type in http://git.domain.com:7990 it will load the site however not be secure just as lastime. Perhaps something is wrong in the proxy you typed?

Thanks for the help!