Pipeline triggered from different repos

[Domenico Gagliano]

Hi all,

I have a pipeline in repo_A that is triggered when changes are pushed in repo_B and/or repo_C. 

When it is triggered by just one repo (B or C) it's all fine and everything works.

When I push change in the two repos whining short time (e.g. I push changes in repo_C two minute after repo_B while the pipeline in repo_A is already running) things goes wrong.

Is there a way for example to stop the first running once the second is triggered and run only the second? Or create a kind of queue?

Hope my question make sense.

Thanks for your help.

[Theodora Boudale · Accepted solution]

Hi @Domenico Gagliano,

I'm afraid that it is not possible to stop the first one running and then run the second one. There is a way to work around this issue, I am sharing more details below:

You could make use of the deployment keyword in the bitbucket-pipelines.yml file of repo A. In case the pipeline in repo A has multiple steps, you could combine them into one step and then add the deployment keyword to that step with one of the environments defined in the repo A Repository settings > Deployments. You can find more details here.

If a deployment step is already running on repo A when another one is triggered, then the second one will be automatically paused. Please keep in mind though that it is not possible to resume the paused ones automatically. A paused deployment can be manually resumed once the in-progress deployment completes.

In case you trigger the builds in repo A by using the atlassian/trigger-pipeline in repo B and repo C, and you use the variable WAIT in the pipe with value true, please keep in mind that a build in repo B and repo C will fail if the deployment step that it triggers in repo A gets paused (with an error message that the build in repo A is halted).

Is this something that would work for you? Please feel free to let me know if you have any questions.

I also wanted to let you know that we have feature requests for automatically resuming paused deployments as well as building concurrency control for non-deployment steps.

Kind regards,
Theodora

[Domenico Gagliano]

Hi @Theodora Boudale

thanks for your reply.
I solved the problem by 

making the pipeline in repo_b and repo_c like this:

pipelines:

  default:

    - step:

      script:

      - pipe: atlassian/trigger-pipeline:5.0.1

        variables:

          BITBUCKET_USERNAME: $BITBUCKET_USERNAME

          BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD

          REF_TYPE: "branch"

          REF_NAME: <repo_A_branch_name>

          DEBUG: "TRUE"

          REPOSITORY: <repo_A>

then I used this python script (the app in repo_A in a python app) that uses the bitbucket api and that I adapted from the one on this post:

https://community.atlassian.com/t5/Bitbucket-questions/Can-a-second-pipeline-deploy-automatically-cancel-a-previous-one/qaq-p/658074

import requests

import os

BITBUCKET_USER = os.environ.get("BITBUCKET_USER")

BITBUCKET_KEY = os.environ.get("BITBUCKET_KEY")

BITBUCKET_REPO_OWNER = os.environ.get("BITBUCKET_REPO_OWNER")

BITBUCKET_REPO_SLUG = os.environ.get("BITBUCKET_REPO_SLUG")

BITBUCKET_BRANCH = os.environ.get("BITBUCKET_BRANCH")

BITBUCKET_COMMIT = os.environ.get("BITBUCKET_COMMIT")

BITBUCKET_WORKSPACE = os.environ.get("BITBUCKET_WORKSPACE")

BITBUCKET_BUILD_NUMBER = os.environ.get("BITBUCKET_BUILD_NUMBER")

request_url = "https://api.bitbucket.org/2.0/repositories/%s/%s/pipelines/?sort=-created_on" % (BITBUCKET_REPO_OWNER, BITBUCKET_REPO_SLUG)

r = requests.get(request_url, auth=(BITBUCKET_USER, BITBUCKET_KEY))

result = r.json()

uuids = []

for value in result["values"]:

    target = value.get("target")

    if target.get("ref_name") == BITBUCKET_BRANCH:

        type = value.get("state").get("type")

        if type == "pipeline_state_in_progress" or type == "pipeline_state_pending":

          print("build number: "+ str(value.get("build_number")))

          test=int(value.get("build_number"))==int(BITBUCKET_BUILD_NUMBER)

          if not(test):

            uuids.append(value.get("uuid"))

for uuid in uuids:

    request_url = "https://api.bitbucket.org/2.0/repositories/%s/%s/pipelines/%s/stopPipeline" % (BITBUCKET_REPO_OWNER, BITBUCKET_REPO_SLUG, uuid)

    print("stopped "+uuid)

    response = requests.post(request_url, auth=(BITBUCKET_USER, BITBUCKET_KEY))

    print(response)

Basically this will stop any old pipelines in repo_A and keep only the most recent one (Which is what I wanted).

and I called it as first step in repo_A pipeline:

pipelines:

    default:

    - step:

         script:

             - pip install requests

             - python3 stop_pipeline.py

    - step:

         services: [docker]

         script:

               - ...

               - ... rest of the pipeline

               - ....

I hope this can help anyone who has this necessity.

Thanks!

[Domenico Gagliano]

Hi @Theodora Boudale

thanks for your reply.
I solved the problem by 

making the pipeline in repo_b and repo_c like this:

pipelines:

  default:

    - step:

      script:

      - pipe: atlassian/trigger-pipeline:5.0.1

        variables:

          BITBUCKET_USERNAME: $BITBUCKET_USERNAME

          BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD

          REF_TYPE: "branch"

          REF_NAME: <repo_A_branch_name>

          DEBUG: "TRUE"

          REPOSITORY: <repo_A>

then I used this python script (the app in repo_A in a python app) that uses the bitbucket api and that I adapted from the one on this post:

https://community.atlassian.com/t5/Bitbucket-questions/Can-a-second-pipeline-deploy-automatically-cancel-a-previous-one/qaq-p/658074

import requests

import os

BITBUCKET_USER = os.environ.get("BITBUCKET_USER")

BITBUCKET_KEY = os.environ.get("BITBUCKET_KEY")

BITBUCKET_REPO_OWNER = os.environ.get("BITBUCKET_REPO_OWNER")

BITBUCKET_REPO_SLUG = os.environ.get("BITBUCKET_REPO_SLUG")

BITBUCKET_BRANCH = os.environ.get("BITBUCKET_BRANCH")

BITBUCKET_COMMIT = os.environ.get("BITBUCKET_COMMIT")

BITBUCKET_WORKSPACE = os.environ.get("BITBUCKET_WORKSPACE")

BITBUCKET_BUILD_NUMBER = os.environ.get("BITBUCKET_BUILD_NUMBER")

request_url = "https://api.bitbucket.org/2.0/repositories/%s/%s/pipelines/?sort=-created_on" % (BITBUCKET_REPO_OWNER, BITBUCKET_REPO_SLUG)

r = requests.get(request_url, auth=(BITBUCKET_USER, BITBUCKET_KEY))

result = r.json()

uuids = []

for value in result["values"]:

    target = value.get("target")

    if target.get("ref_name") == BITBUCKET_BRANCH:

        type = value.get("state").get("type")

        if type == "pipeline_state_in_progress" or type == "pipeline_state_pending":

          print("build number: "+ str(value.get("build_number")))

          test=int(value.get("build_number"))==int(BITBUCKET_BUILD_NUMBER)

          if not(test):

            uuids.append(value.get("uuid"))

for uuid in uuids:

    request_url = "https://api.bitbucket.org/2.0/repositories/%s/%s/pipelines/%s/stopPipeline" % (BITBUCKET_REPO_OWNER, BITBUCKET_REPO_SLUG, uuid)

    print("stopped "+uuid)

    response = requests.post(request_url, auth=(BITBUCKET_USER, BITBUCKET_KEY))

    print(response)

Basically this will stop any old pipelines in repo_A and keep only the most recent one (Which is what I wanted).

and I called it as first step in repo_A pipeline:

pipelines:

    default:

    - step:

         script:

             - pip install requests

             - python3 stop_pipeline.py

    - step:

         services: [docker]

         script:

               - ...

               - ... rest of the pipeline

               - ....

I hope this can help anyone who has this necessity.

Thanks!

[Theodora Boudale]

Hi Domenico,

Thank you for the update and for sharing your script here, it's good to hear that this works well for you!