BitBucket API throttling concurrent connections in addition to rate limiting

[Michal Mazurek]

I'm fetching some information concurrently for multiple different BitBucket Teams at the same time. Sometimes I get connection timeouts. What does the concurrent connections throttling look like? Is it only based on my IP?

[David V_ Corbin]

When cloning larger repos (10's of GB) the download will often pause and slow down, this is especially true if I am doing this multiple times in a short interval.... could this be a case of rate limiting?

[Stephen Sifers]

Hello Michal,

This is a great question about Bitbucket rate limiting. Here is how rate limiting with BItbucket cloud is currently configured:

How calls are measured

Tracking the number of calls is measured based upon if they are authenticated or unauthenticated calls as follows:

• Unauthenticated calls: measured against a specific IP address.
• Authenticated calls: measured against the user id.

Rate limits are divided into one-hour windows for both authenticated and unauthenticated requests.

Request limits types with limits

Git web (HTTPS://) requests => 60,000 requests per hour

hg web (HTTPS://) requests => 30,000 hgweb requests per hour

SSH requests (all SSH requests are rate limited) => 30,000 SSH requests per hour

Access to raw files                

Raw file requests: the number of times a file is actively downloaded, including files in the download section of the site.  => 5,000 raw file requests per hour

This is not a limit on actions such as:

• Pushing or pulling commits
• Running a fetch command to get recent changes      
• Pushing or pulling a branch (unless that branch contains more than 1000 new files)

Archive files: .zip, .gz, and other compressed archives of your repository. => 5,000 files per hour

List a user or team's repositories (/2.0/repositories/{username}) => 1,000 per hour

List all repositories on Bitbucket (/2.0/repositories) => 1,000 per hour

List a repository's commits (/2.0/repositories/{username}/{slug}/commits) => 1,000 per hour

Source documentation Rate limits

I hope this clarifies the rate limits within Bitbucket cloud.

Regards,
Stephen Sifers

[Michal Mazurek]

Yes, thank you, the server will return a 429 when it's rate limiting me. However it will also cause a connection to timeout sometimes. I suspect that happens when I try to establish too many connections concurrently. What's the concurrent connection limit?

[Stephen Sifers]

Hello Michal,

If you’re under your limits on the rate limits, then you should be able to connect without issue. However, some types of connections may take longer than 60 seconds causing a response of 429.

With this said, why type of action is being performed which is resulting in a response of 429?

Regards,
Stephen Sifers

[Michal Mazurek]

Don't you mean a 555? Yes, I'm also getting 555's on the `diffstat` API endpoint. I'm getting 429 when I send too many requests and go over the rate limit - that's fine.

Thank you for your reply, if you don't limit incoming connections on the firewall then that eliminates a variable for me and I must look for the reason of my timeouts elsewhere.

Could you help me with these issues as well? I got no reply, and currently I'm just ignoring the PRs that cause these errors, which might not be ideal for my users:

https://bitbucket.org/site/master/issues/17740/undocumented-404-to-20-repositories

https://bitbucket.org/site/master/issues/17738/undocumented-403-to-20-repositories

[Stephen Sifers]

Hello Michal,

I understood your question as you’re receiving a 429 response when performing an action. Thank you for the clarification of the 555 responses.

For the bug requests, we would encourage you to review Atlassian Cloud Bug Fix Policy. Which covers the following:

How we approach bug fixing

We update the product components in Atlassian Cloud as soon as possible, to take advantage of bug and security fixes. Developers responsible for bug fixing also monitor comments on existing and new bugs, so you can comment to provide feedback if you need to. We give high priority to security issues.

Please let us know if we can be of any more assistance.

Regards,
Stephen Sifers