Simple Echo step doesnt run on self-hosted runner

[Sergei Haller]

I installed a simple pipeline, based on the starter template, which basically just has two steps, each of them an echo => one of them has the self-hosted label, the other doesn't.

image: atlassian/default-image:2

pipelines:
  default:
    - parallel:
      - step:
          name: "Simple Echo on Self-Hosted"
          runs-on: self.hosted
          script:
            - echo "Your build and test goes here..."

      - step:
          name: "Simple Echo (without runs-on)"
          script:
            - echo "Your linting goes here..."

(I hope I didnt mix up spaces when copy-pasting the code into message editor)

The above pipeline runs, step 2 is executed successfully. Step 1 fails (while setting up)

on the UI, I only see tis in setup block:

+ umask 000

+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="xxx" --depth 50 https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...

and this is it.

in the logs of the runner, I see this: 

[2021-10-05 17:43:52,488] Updating runner state to "ONLINE".
[2021-10-05 17:44:02,485] Setting runner state to executing step.
[2021-10-05 17:44:02,488] Getting step StepId{accountUuid={152d4ae4-e3cb-45b1-97d6-7f64fe55688c}, repositoryUuid={c3803e18-8eec-4c92-8d1b-9186a7e07ca8}, pipelineUuid={40fd2ae3-625d-43d9-a3e2-58c191a616ac}, stepUuid={b1fa6711-ab69-404f-af5d-ff81a635f0e5}}.
[2021-10-05 17:44:02,490] Getting oauth token for step.
[2021-10-05 17:44:02,490] Getting environment variables for step.
[2021-10-05 17:44:02,932] Getting all artifacts for step.
[2021-10-05 17:44:02,933] Getting SSH private key.
[2021-10-05 17:44:02,933] Getting known hosts.
[2021-10-05 17:44:03,145] SSH private key not found
[2021-10-05 17:44:03,158] Setting up directories.
[2021-10-05 17:44:03,158] Starting log uploader.
[2021-10-05 17:44:03,159] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_clone
[2021-10-05 17:44:03,160] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_build
[2021-10-05 17:44:03,161] Setting up step timeout: PT2H
[2021-10-05 17:44:03,161] Starting websocket listening to STEP_COMPLETED events.
[2021-10-05 17:44:03,162] Checking for step completion every PT30S seconds.
[2021-10-05 17:44:03,372] Updating step progress to PULLING_IMAGES.
[2021-10-05 17:44:03,623] Pulling image docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-dvcs-tools:prod-stable.
[2021-10-05 17:44:04,161] Appending log line to main log.
[2021-10-05 17:44:04,376] Pulling image atlassian/default-image:2.
[2021-10-05 17:44:05,820] Pulling image docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-auth-proxy:prod-stable.
[2021-10-05 17:44:06,481] Pulling image docker-hub.packages.atlassian.com/google/pause:latest.
[2021-10-05 17:44:07,424] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_pause
[2021-10-05 17:44:07,429] Updating step progress to CLONING.
[2021-10-05 17:44:07,429] Creating container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_pause.
[2021-10-05 17:44:07,482] Starting container.
[2021-10-05 17:44:07,666] Generating clone script.
[2021-10-05 17:44:07,667] Creating container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_clone.
[2021-10-05 17:44:07,667] Executing clone script in clone container.
[2021-10-05 17:44:07,713] Starting container.
[2021-10-05 17:44:07,839] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_system_auth-proxy
[2021-10-05 17:44:07,841] Creating container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_system_auth-proxy.
[2021-10-05 17:44:07,885] Starting container.
[2021-10-05 17:44:08,017] Adding container log: /var/lib/docker/containers/bc52e26967b50accb5a1e44870338702d5a86886293b3796cfd2548970035b32/bc52e26967b50accb5a1e44870338702d5a86886293b3796cfd2548970035b32-json.log
[2021-10-05 17:44:08,018] Waiting on container to exit.
[2021-10-05 17:44:08,018] Adding container log: /var/lib/docker/containers/9f1d2927fdf455a24eab5018521db5ece763a36cf8b90f88ce562532cf5a9122/9f1d2927fdf455a24eab5018521db5ece763a36cf8b90f88ce562532cf5a9122-json.log
[2021-10-05 17:44:08,018] Waiting on container to exit.
[2021-10-05 17:44:08,020] Creating exec into container.
[2021-10-05 17:44:08,024] Starting exec into container and waiting for exec to exit.
[2021-10-05 17:44:08,162] Appending log line to log: {2a46a637-d0d1-4c8f-91ba-1ff62a17cc76}.
[2021-10-05 17:44:08,164] Appending log line to main log.
[2021-10-05 17:44:08,179] Container has state (exitCode: Some(1), OOMKilled Some(false))
[2021-10-05 17:44:08,182] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_build
[2021-10-05 17:44:08,186] Not uploading caches. (numberOfCaches: 0, resultOrError: FAILED)
[2021-10-05 17:44:08,187] Not uploading artifacts. (numberOfArtifacts: 0, resultOrError: FAILED)
[2021-10-05 17:44:08,188] Updating step progress to PARSING_TEST_RESULTS.
[2021-10-05 17:44:08,918] Test report processing complete.
[2021-10-05 17:44:08,918] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_clone
[2021-10-05 17:44:09,161] Appending log line to main log.
[2021-10-05 17:44:09,161] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_clone
[2021-10-05 17:44:09,166] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_build
[2021-10-05 17:44:09,170] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_system_auth-proxy
[2021-10-05 17:44:09,184] Removing container efb3381e-fc6a-5e38-8d53-5087a834d8c0_b1fa6711-ab69-404f-af5d-ff81a635f0e5_pause
[2021-10-05 17:44:09,347] Updating step progress to COMPLETING_LOGS.
[2021-10-05 17:44:09,591] Shutting down log uploader.
[2021-10-05 17:44:09,939] Tearing down directories.
[2021-10-05 17:44:09,944] Cancelling timeout
[2021-10-05 17:44:09,945] Completing step with result Result{status=FAILED, error=None}.
[2021-10-05 17:44:10,204] Setting runner state to not executing step.
[2021-10-05 17:44:10,204] Waiting for next step.
[2021-10-05 17:44:22,487] Updating runner state to "ONLINE".

which doesnt help either, as it doesn't show any errors or anything. Where can I look for hints? 

The runner is apparently online and receives the execution request. The "container logs" referred to in the runner log are not available immediatly after execution (1s), so I wasn't able to get a hand on that, but not sure either, if there is more to expect...

any help appreciated