How do I use MySQL in Bitbucket Pipelines

[SebC]

How do I get a mysql database to run with by pipeline?

[SebC · Accepted solution]

You'll want to take advantage of services in bitbucket pipelines.

For the simple uses, you can create an empty database named 'pipelines' and use the 'root' user with password 'let_me_in' to connect on localhost:3306

pipelines:
  default:
    - step:
        image: node
        script:
          - npm install
          - npm test
        services:
          - mysql
 
definitions:
  services:
    mysql:
      image: mysql
      environment:
        MYSQL_DATABASE: pipelines
        MYSQL_ROOT_PASSWORD: let_me_in

For more complex situations, you can create a test user to help validate database permissions and other priviledged settings.

The below bitbucket-pipelines.yml will create an empty database named 'pipelines' and user 'test_user' with password 'let_me_in' to connect on localhost:3306. The root user will have a random password created for it and not be accessible.

pipelines:
  default:
    - step:
        image: node
        script:
          - npm install
          - npm test
        services:
          - mysql
 
definitions:
  services:
    mysql:
      image: mysql
      environment:
        MYSQL_DATABASE: pipelines
        MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
        MYSQL_USERNAME: test_user
        MYSQL_PASSWORD: test_user_password

[Daniele Piaggesi]

I tried with your configuration above:

MYSQL_DATABASE: pipelines
MYSQL_RANDOM_ROOT_PASSWORD: yes
MYSQL_USERNAME: test_user
MYSQL_PASSWORD: test_user_password

but Pipeline said to me that:

The 'environment' section in your bitbucket-pipelines.yml file must be a map.

Obviously it was a map. I remove this environment and put original ones, taken from mysql docker hub page. It worked.

Probably this configuration is broken and you should update your documentation.

[Yuriy Znatokov]

Having the same problem, obviously the configuration posted above and the one in the docs are broken.

How did you manage to solve it?

[caiquecssilva]

I had the same issue and i changed to MYSQL_DATABASE: 'pipelines' for example

[Yuriy Znatokov]

Got it working with the following config:

environment:
MYSQL_DATABASE: pipelines
MYSQL_ROOT_PASSWORD: root_user_password
MYSQL_USER: test_user
MYSQL_PASSWORD: test_user_password

[SebC]

Thanks for the feedback! Turns out there was an issue where values like "yes" or "true" are being converted to booleans, which is causing some problems when they're loaded into the service container (yes becomes true for example).

Quoting these values will force them to be treated as strings - I'll be discussing this within the Pipelines team to try and improve the experience.

[kenkinder]

So, this is still broken, and the documentation is still giving you examples that don't work. Is a fix coming any time?

[Ulrich Jahnke]

@SebCPlease change documentation https://confluence.atlassian.com/bitbucket/how-to-run-common-databases-in-bitbucket-pipelines-891130454.html to use

MYSQL_USER

in MySQL - test user section.

[Dheeraj Kumar]

+ mysql -u root -p root -h 127.0.0.1 < test.sql

bash: mysql: command not found

while running it i got this error

[maciejgut]

Try to install something that provides mysql command. For example on Debian image (in my case "php:7.1.29") this installs the standard mysql client:

- apt-get update && apt-get install -y mysql-client

[Jonas Lomholdt]

I just can't make this work. I get a `Exception: SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Name or service not known` no matter what I do. I have tried a regular user and root user. I also tried using the hostname `mysql`. Nothing works. Can anyone confirm that this is actually a working thing?

I also get a `mysqld: Shutdown complete` at the end of the database service log. Why?

[mtumaini]

Just wondering if anyone got a solution to this issue.

I am experiencing same issue with similar mysql configuration. My pipeline reports php_network_getaddresses failure. 

[Reece Alexander]

Ensure your connecting to the DB with the hostname 127.0.0.1 and not "localhost" as the latter means nothing in docker.

[AS]

I eventually got this working. But how can I change the mysql cnf file and then reboot the mysql server/service?
Really struggling with this bit, if anyone has any ideas?

[Richard Quadling]

At the moment, it seems using BitBucket's services does not allow you to inject a config. I need this as I'm testing a legacy app that is using the older auth mechanism and that needs to be enabled in MySQL 8.0.

I've managed to create the container and inject our config:

# Load Official MySQL Image
# -------------------------
#
# OMI10. Pull down required official MySQL Image
# OMI20. Start MySQL with our configuration
# OMI30. Wait for MySQL to be ready
load_mysql: &load_mysql |-
  echo 'Load Official MySQL Image'

  echo 'OMI10. Pull down required official MySQL Image'
  docker pull mysql:8.0 \
    > pipeline/${PIPELINE_LOG_PREFIX}_OMI10_pull_mysql.log \
    2> pipeline/${PIPELINE_LOG_PREFIX}_OMI10_pull_mysql_errors.log;

  echo 'OMI15. Set permissions on the mysql80.cnf file'
  chmod 0444 $BITBUCKET_CLONE_DIR/config/mysql/mysql80.cnf

  echo 'OMI20. Start MySQL with our configuration'
  docker run -d \
    --name mysql80 \
    -e MYSQL_DATABASE=digitickets \
    -e MYSQL_ROOT_PASSWORD=pass \
    -e TZ=Europe/Dublin \
    --health-cmd="mysqladmin ping -h localhost -u root -ppass" \
    --health-interval=10s \
    --health-timeout=10s \
    --health-retries=10 \
    --health-start-period=60s \
    -v $BITBUCKET_CLONE_DIR/config/mysql/mysql80.cnf:/etc/mysql/conf.d/mysql80.cnf:ro \
    -p 127.0.0.1:3306:3306 \
    mysql:8.0 \
      --default-authentication-plugin=mysql_native_password \
      > pipeline/${PIPELINE_LOG_PREFIX}_OMI20_run_mysql.log \
      2> pipeline/${PIPELINE_LOG_PREFIX}_OMI20_run_mysql_errors.log;

  echo 'OMI30. Wait for MySQL to be ready'
  for i in {60..0}; do
    docker exec mysql80 mysqladmin ping status -uroot -ppass \
     >> pipeline/${PIPELINE_LOG_PREFIX}_OMI30_wait_for_mysql.log \
     2>> pipeline/${PIPELINE_LOG_PREFIX}_OMI30_wait_for_mysql_errors.log \
  && break
    echo 'Waiting for MySQL...' >> pipeline/${PIPELINE_LOG_PREFIX}_OMI30_waiting_for_mysql.log \
    sleep 1
  done

  docker ps | tee pipeline/${PIPELINE_LOG_PREFIX}_docker_ps.log
  docker logs mysql80 | tee pipeline/${PIPELINE_LOG_PREFIX}_docker_logs.log

and this seems to be working fine.

But I'm not able to re-run the mysqladmin check externally.

echo 'D10. Wait until the DB is ready to be communicated with.'
(mysqladmin --host=127.0.0.1 --port=3306 --user=root --password=pass status ping || true) | tee pipeline/${PIPELINE_LOG_PREFIX}_D10_check_status.log
docker ps | tee pipeline/${PIPELINE_LOG_PREFIX}_docker_ps_v2.log
docker logs mysql80 | tee pipeline/${PIPELINE_LOG_PREFIX}_docker_logs_v2.log

That results in an error, but Docker is saying all is well.

D10. Wait until the DB is ready to be communicated with.
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
mysqladmin: connect to server at '127.0.0.1' failed
error: 'Lost connection to MySQL server at 'reading initial communication packet', system error: 0'
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2aba47c637a9 mysql:8.0 "docker-entrypoint.s…" 11 seconds ago Up 10 seconds (healthy) 127.0.0.1:3306->3306/tcp, 33060/tcp mysql80

[Alfred Bez]

Somehow this does not work for me, here's my bitbucket-pipelines.yml:

image: debian:stretch

pipelines:
  pull-requests:
    '*':
      - step:
        script:
          - apt-get update && apt-get install -y mysql-client
          - mysql -h 127.0.0.1 -u root -ptest_user_password -e "SHOW DATABASES"
definitions:
  services:
    mysql:
      image: mysql:5.7
      variables:
        MYSQL_DATABASE: 'pipelines'
        MYSQL_ROOT_PASSWORD: 'test_user_password'

 Output:

+ mysql -h 127.0.0.1 -u root -ptest_user_password -e "SHOW DATABASES"
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111 "Connection refused")

 

[Alfred Bez]

I forgot to add the service to the pull-requests pipeline:

image: debian:stretch

pipelines:
  pull-requests:
    '*':
      - step:
        script:
          - apt-get update && apt-get install -y mysql-client
          - mysql -h 127.0.0.1 -u root -ptest_user_password -e "SHOW DATABASES"
        services:
          - mysql
definitions:
  services:
    mysql:
      image: mysql:5.7
      variables:
        MYSQL_DATABASE: 'pipelines'
        MYSQL_ROOT_PASSWORD: 'test_user_password'

[Usama Habib]

Hi @Alfred_Bez did you experience this error after mentioning services tag in you yml file as well or was it gone? since my pipelines file is through me same error but not everytime time only sometimes i do have this *services* tag but its very unusual sometimes pipeline throws this error and sometime it passes build successfully for the same file. 

Any help will be appreciated if you know anything.

Thanks

[Alfred Bez]

Hello @Usama Habib , it works now for me, I don't see the error at all.