Rest API to search a string in all the repositories in Bitbucket?

[amit raj]

Hello Friends,

I would like to search a string in all the Bitbucket server repositories using Rest API.

I can search the code from the Bitbucket Server "Search Box" but it is not possible if there are multiple strings to search and generate a report.

a common use case would be- search all repositories where a common password is exposed.

Is there any Rest API available or could you suggest any workaround?

Thank you

[GTDevelopmentPlatformLicense GTDevelopmentPlatformLicense · Accepted solution]

Hi @amit raj 

After upgrade Bitbucket data-center 7.6.0 and upgrade elasticsearch 6.8.6, I'm getting error like below.

Do you have any idea regarding this problem?

{
"error": {
"root_cause": [ {
"type": "index_not_found_exception",
"reason": "no such index",
"resource.type": "index_or_alias",
"resource.id": "bitbucket-search-v1",
"index_uuid": "_na_",
"index": "bitbucket-search-v1"
}],
"type": "index_not_found_exception",
"reason": "no such index",
"resource.type": "index_or_alias",
"resource.id": "bitbucket-search-v1",
"index_uuid": "_na_",
"index": "bitbucket-search-v1"
},
"status": 404
}

[amit raj · Accepted solution]

Hi All,

 I was able to search a string using ELS Rest api. 

this is the the endpoint-

http://<ELSEARCHURL>:9200/bitbucket-search-v1/_search?size=1000&from=0&q=<string-to-search>

eg. if your Bitbucket ELS is hosted on 192.168.10.1 at port 9200 and you would like to search the string "Test"

http://192.168.10.1:9200/bitbucket-search-v1/_search?size=1000&from=0&q=test

please note that it is huge search you need to iterate the loop until search does not find a string.

I wrote a small python program to do that.

https://github.com/inboxamitraj/Bitbucket-Python-Scripts/blob/master/bbsearch.py

Thank you

[Mary Slocum]

Hi Amit,

Nice script! 

One problem is that it returns repositoryId and projectId, but not the repo/project name. Is there a way to look up repo/project name by id? Also, is there any way to tell which branch in the repo that the search string was found (as a repo could have a lot of branches)?

Thanks,

Mary

[amit raj]

I think we can do something using elastic search Rest endpoints as Bitbucket usage elastic search behind the scene, working on it.

[MiniBurgula]

Hi @amit raj , have you any luck ? I am also looking for same feature in bitbucket api.

[amit raj]

Hi @MiniBurgula 

yes. I was able to search a string using ELS Rest api. 

this is the the endpoint-

http://<ELSEARCHURL>:9200/bitbucket-search-v1/_search?size=1000&from=0&q=<string-to-search>

eg. if your Bitbucket ELS is hosted on 192.168.10.1 at port 9200 and you would like to search the string "Test"

http://192.168.10.1:9200/bitbucket-search-v1/_search?size=1000&from=0&q=test

please note that it is huge search you need to iterate the loop until search does not find a string.

I wrote a small python program to do that.

Thank you

[amit raj]

python code-

https://github.com/inboxamitraj/Bitbucket-Python-Scripts/blob/master/bbsearch.py

[MiniBurgula]

Thanks for the information.

[Namita K]

Hi amitraj,

I cannot find your script - bbsearch.py . Is it possible to paste it here? 

[Namita K]

Hi @amit raj

I cannot find your script - bbsearch.py . Is it possible to paste it here? 

[Namita K]

Hi @amit raj  can you help me with the code pls

[sdeals]

Code lives here now: https://github.com/xeroceph/bbsearch/blob/63a6c4dd603ba47b923cd57f228f7f94956cc6c8/bbsearch.py

[Edwin Kyalangalilwa]

Hi @amit raj,

This link will be helpful. Bitbucket code search API is now available

[amit raj]

Thank you Edwin , I think this is only for Bitbucket Cloud, doesn't work for Bitbucket Server.

[prajakta chavan]

Hi,

Can we use this for Bitbucket server. https://<domain-name>/bitbucket-search-v1/_search?size=1000&from=0&q=test..

because I am getting error as dead link as 404 error.. Or I am fetching with wrong url.I am not able get what is url and hostname?is it Bitbucket domain name or something as ELS server(different than Bitbucket server)? Can anyone please reply?