Build-test-deploy of a docker container in a bitbucket pipeline

[plussier]

Hi all,

I'm trying to set up a bitbucket pipeline for a docker container. Like any software project, I want the basic pipeline process to be

On feature branches:

- Build the container

- Test the container (use it in a step?)

And on merge to the develop branch:

- Build the container (unless there's a better way!)

- Test

- Tag and push

For release and master branches, I'd like to be able to simply promote the most recent versions from the previous level; i.e. develop-> release, and release -> master.

However, I'm wondering if there's also a need to at some point delete entire images and/or tags as well. And if so, where does that fit in and what is the best way to accomplish that? Custom scripts around the Docker registry API ?

I'm very new to building/maintaining docker images and repositories, and the Docker Hub docs are not overly helpful with how to set up a proper pipeline process.

Any help/advice/pointers would be greatly appreciated!

Additionally:

How do you build a container in one step, and then use that container in the next step?

This seems pretty straight-forward, until names and tags on container images a built using dynamic variables...  For example, the following can't possibly work, because the image name I want to pull is dependent upon evaluating variables set in a script which hasn't been called yet.

definitions:
  script: &setVars
    export RELEASE="1.2.3";
    export PACKAGE="my-docker";
    if [[ ${BITBUCKET_BRANCH} =~ ^develop ]]; then REPO_MATURITY=dev; fi;
    if [[ ${BITBUCKET_BRANCH} =~ ^release ]]; then REPO_MATURITY=rc; fi;
    if [[ ${BITBUCKET_TAG} =~ ^[0-9]\.[0-9]\.[0-9] ]]; then REPO_MATURITY=released; fi;
    export REPO_MATURITY;
    export DOCKER_REG="myreg"
    export DOCKER_REPO="${DOCKER_REG}/${PACKAGE}"
    export DOCKER_TAG="${DOCKER_REPO}:${RELEASE}-${REPO_MATURITY}"
    export AWS_CLI_PROFILE=etq-development;

  script: &docker_login
    docker login ${DOCKER_USER} -p ${DOCKER_TOKEN}

  steps:
    - step: &build_container
        name: "Build container"
        caches:
          - docker
        services:
          - docker
        script:
          - *setVars
          - *docker_login
          - docker build -f ${PACKAGE}.dockerfile -t ${DOCKER_TAG}-${BITBUCKET_BUILD_NUMBER} .
          - docker push ${DOCKER_TAG}-${BITBUCKET_BUILD_NUMBER}

    - step: &test_container
        name: "Test container"
        caches:
          - docker
        image:
          name: ${DOCKER_TAG}-${BITBUCKET_BUILD_NUMBER}
          username: $DOCKER_USER
          password: $DOCKER_TOKEN
          email: $DOCKER_EMAIL
        script:
          - *setVars
          - run_test_script.sh

Now, if Bitbucket pipelines had the ability to set RUN TIME GLOBAL variables in the bitbucket-pipeline.yml file, this would work perfectly.  Other than attempting to set some of these variables in the repo-specific variables area, I'm stumped on how I can pull an image I just pushed in a previous step.

In case it's not obvious, I am indeed trying to build a container I want to use to actually run my pipelines in. And I want to be able to build and test this newly built container by actually running it in this pipeline.

Running `docker save` to save the image as an artifact which gets `docker load`ed in a the next doesn't seem to be the right thing either.

Why can't BB just have run-time global variables?!

Thanks.

[Ankit Gupta]

Hi Plussier,

I have solved a similar problem. You could define a `<branch>-latest` tag against the images for respective branches. Your deploy & test step then pulls this `latest` tag always.

This is something that works for me. You can push the desired release numbers to Docker Hub but also mark the `<branch>-latest` tag so as to reference that in the deploy & test pipelines.

[plussier]

Hi @Ankit Gupta

Thank you for that suggestion. It had crossed my mind to do something like this. One of the things I was stuck on was, what if the branch name isn't consistent (i.e. feature branches), and then how to I delete these images or tags.

But your solution seems better. Create a particular tag for all dynamic branches (i.e. my `default` pipeline!) and append the build number and/or `latest` tags. Then in the test step, pull `myimage:latest`.

I'll give that a go if my current approach doesn't work (which is attempting to use a custom pipeline triggered from itself and passing in the dynamic tag!).

Thanks.

-Paul