how to self drain bitbucket selfhost docker runner

[Arunkumar Gaddam]

In my case bitbucket APIs are not so useful to deregister runners.

I have a repo and multiple steps in repo are using selfhost docker runner label and running simultaneosly in different vms using vm azure scale set.

vm get spinned up only when job runs and should get deleted once job completes and docker runner self drains.

Since docker runner does not self exits it is bit hard to watch out pipelines and go to scaleset and shut down vms

Kindly assist

Regards,

Arun

[Arunkumar Gaddam]

@Mark C 

I am getting error as 400 bad request while registering docker runner dynamically from cmd line

The question, is it possible to create docker runner from cmd line without registering on bitbucket UI ?

I am using the below cmd

RUNNER_UUID="{$(uuidgen)}"

CLEAN_UUID=$(echo "$RUNNER_UUID" | tr -d '{}')

docker container run -d -v /tmp:/tmp -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/containers:/var/lib/docker/containers:ro -e ACCOUNT_UUID={xxxxxxxx} -e BITBUCKET_RUNNER_LABELS="test" -e RUNNER_UUID=$RUNNER_UUID -e RUNTIME_PREREQUISITES_ENABLED=true -e OAUTH_CLIENT_ID="xxxxxxxxxxx" -e OAUTH_CLIENT_SECRET="xxxxxxxxx" -e WORKING_DIRECTORY=/tmp --name runner-$CLEAN_UUID docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner

The docker container gets created but it failed to go online due to 400 bad request

I have created  OAUTH CLIENT ID and OAUTH CLIENT SECRET from bitbucket portal --> workspace settings --> OAUTH Consumers

is that right way to get OAUTH_CLIENT_ID & OAUTH_CLIENT_SECRET ?

If not, can you please advise where do we create them ?

[Theodora Boudale]

Hi,

Please allow me to step in as Mark is out of office at the moment.

You asked in an earlier reply the following:

Can I use same runner which is running in multiple VMs with same runner uuid and with same label

This is not possible, you cannot start the same runner from multiple VMs at the same time.

Regarding your latest question:

is it possible to create docker runner from cmd line without registering on bitbucket UI ?

This is not possible. Your Docker command fails because the runner hasn't been created on Bitbucket.

The other option, if you don't want to create runners manually, is to use the runner autoscaler, as Mark suggested.

Kind regards,
Theodora

[Mark C]

Hi @Arunkumar Gaddam

Welcome to the community.

Could you confirm if you're using Bitbucket Cloud Pipelines autoscaler?

If not yet, Pipelines autoscaler has an autoscaler cleaner where it automatically clean (delete) unhealthy runners and linked jobs.

If you're not using it, you will have to manually remove the stale runners and/or write your own script that will do the cleanup in your specific environment.

Regards,
Mark C

[Arunkumar Gaddam]

Thank you for your response @Mark C 

I found a way to clean up

I am facing different issue now

Can I use same runner which is running in multiple VMs with same runner uuid and with same label

so can I run it in different steps of pipeline or different repo pipelines ?

I would like run all those steps or repo pipelines simultaneously without being queued

when I did so it throws an error

[Mark C]

Hi @Arunkumar Gaddam

Could you provide the error you're getting?

Regards,
Mark C