Docker compose build skips cache

[Christian Sciberras]

Problem

At some point, we started using docker compose in bitbucket like so:

- step:
    name: Build & Run
    image: docker:26
    services:
      - docker
    caches:
      - docker
    script:
      # Needed because custom docker image somehow enables buildkit which,
      # on the other hand, does not work with docker-compose (it requires
      # privileged mode, which is disabled in bitbucket+buildkit)
      - export DOCKER_BUILDKIT=0
      
      # Download(pull) 3rd-party docker images
      - docker compose pull
      
      # Build our custom images
      - docker compose build
      
      # Run something in the docker-compose stack (e.g. tests)
      - docker compose run -T app run-tests

The reason we split docker compose into 3 lines is so that the output of the run command (which we are mostly interested in) shows up in its own section in the bitbucket output. It looked like it worked perfectly..

Except that apparently `docker compose build` will somehow ignore the build cache.

Solution

It took us some time to figure this out...the solution is to avoid using `docker compose build`.

...
   script:
      # Needed because custom docker image somehow enables buildkit which,
      # on the other hand, does not work with docker-compose (it requires
      # privileged mode, which is disabled in bitbucket+buildkit)
      - export DOCKER_BUILDKIT=0
      
      # Pull/build & run something in the docker-compose stack (e.g. tests)
      - docker compose run -T app run-tests

To be clear, `docker compose build` should not ignore the cache:

1. we tried that locally and we observed that the cache is not ignored
2. the --no-cache option exists specifically for this behaviour

So this behaviour looks like some sort of bug in bitbucket.

[Patrik S]

Hey @Christian Sciberras ,

thank you for reaching out to Community!

By the description of your issue, it seems you're indeed being affected by a bug in Pipelines where cached for docker specifically and not being used in subsequent builds. We have the following bug report with our development team to fix this issue : 

• https://jira.atlassian.com/browse/BCLOUD-23193

I would suggest you to add your vote there, since this helps both developers and product managers to understand the impact. Also, make sure you add yourself as a watcher in case you want to receive first-hand updates from that ticket. Please note that all bug fixes are implemented with this policy in mind.

Thank you, @Christian Sciberras !

Patrik S