Deploy a single directory to FTP

[Deleted user]

Hey guys, 

I am very new to this topic and I want to deploy a single directory '/dist' that is built first in my bitbucket pipeline. I already expose it as an artifact but when I try to refer to the dist folder as a LOCAL_PATH, it cannot be found.
In the first step "building" I try to output the whole directory of the project and I can't see the '/dist' folder which was generated before with the command 'npm run generate'.

Here is my bitbucket-pipelines.yml: 

image: node:10.16.0
pipelines: 

default: 

- step: 

name: building 

caches: - node 

script: 

- echo "create fresh new project..." 

- echo "--- install dependencies ---" 

- npm install 

- echo "--- generate static sites of nuxt ---" 

- npm run generate 

- echo "$(ls -la dist/)" 

- ls 

artifacts: - dist/** 

- step: 

name: deploy 

deployment: staging 

caches: 

- node 

script: 

echo "$(ls -la ${BITBUCKET_CLONE_DIR}/)" 

# Modify the commands below to build your repository. 

- pipe: atlassian/ftp-deploy:0.2.1 

variables: 

USER: '' 

PASSWORD: '' 

SERVER: '' 

REMOTE_PATH: '/web/pipelineTest' 

LOCAL_PATH: '${BITBUCKET_CLONE_DIR}/'

[Tim Brugman]

I just started playing around with pipelines tonight, but I think I ran into the same problem and have the solution:

Every `- step` is a new docker build. So before your "deploy" build begins, your "building" machine gets shut down, and you lose the changes it generated. To get around this, I combined both steps into one step.

To the Atlassian folks: If my assumption here is wrong, and steps can build upon files generated by previous steps, please let me know.

EDIT - I hadn't learned about artifacts yet. You can ignore the above, but it may still be an alternate solution.

@[deleted]

At the bottom of your snippet, you've given us your FTP credentials. And they work. You should get new creds and never ever use that password again. :)

[Peter Plewa]

Hi @[deleted]

Your YML looks okay (assuming it's formatted properly) could you add a variable

DEBUG: 'true'

to the pipe configuration and send the log output?

Thanks,

Peter Plewa

[Deleted user]

Hi @Peter Plewa

I added the debug option to my .yml file. The strange thing in general is that in the build step the .nuxt directory builded. But the /dist directory in root is still missing or ignored. 

Screenshot: 

npm run build (no error and no warning is thrown in build step)

but ls -la is shown that there is no build created to upload to server:

[Peter Plewa]

Could you also upload raw logs that you can access clicking on the following button 

[Deleted user]

Sorry, for sure. 

+ umask 000

+ GIT_LFS_SKIP_SMUDGE=1 git clone --branch="bitbucket-staging-pipeline" --depth 50 https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...

+ git reset --hard b69e030d534c282cee124f0ad81eb74047fb2c8f
HEAD is now at b69e030 check files for artifacts in pipeline

+ git config user.name bitbucket-pipelines

+ git config user.email commits-noreply@bitbucket.org

+ git config push.default current

+ git config http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy http://localhost:29418/

+ git remote set-url origin http://bitbucket.org/$BITBUCKET_REPO_FULL_NAME

+ echo ".bitbucket/pipelines/generated" >> .git/info/exclude

+ chmod 777 $BUILD_DIR

Cache "node": Downloading
Cache "node": Downloaded 44.2 MiB in 1 seconds
Cache "node": Extracting
Cache "node": Extracted in 3 seconds

+ echo "create fresh new project..."
create fresh new project...

+ echo "--- install dependencies ---"
--- install dependencies ---

+ npm install
npm WARN bootstrap@4.3.1 requires a peer of jquery@1.9.1 - 3 but none is installed. You must install peer dependencies yourself.
npm WARN eslint-config-airbnb@17.1.1 requires a peer of eslint-plugin-react@^7.14.2 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.0.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.0.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/watchpack/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/nodemon/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/jest-haste-map/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 895141 packages in 9.903s
found 1 critical severity vulnerability
run `npm audit fix` to fix them, or `npm audit` for details

+ echo "--- generate node build of nuxt ---"
--- generate node build of nuxt ---

+ npm run build

> trusted-experts@1.0.0 build /opt/atlassian/pipelines/agent/build
> nuxt build


+ echo "--- generate static sites of nuxt ---"
--- generate static sites of nuxt ---

+ npm run generate

> trusted-experts@1.0.0 generate /opt/atlassian/pipelines/agent/build
> nuxt generate


+ echo "$(ls -la)"
total 760
drwxrwxrwx. 16 root root 4096 Aug 30 19:28 .
drwxr-xr-x. 8 root root 4096 Aug 30 19:27 ..
-rw-rw-rw-. 1 root root 212 Aug 30 19:28 .babelrc
-rw-rw-rw-. 1 root root 12 Aug 30 19:28 .dockerignore
-rw-rw-rw-. 1 root root 207 Aug 30 19:28 .editorconfig
-rw-rw-rw-. 1 root root 146 Aug 30 19:28 .env.default
-rw-rw-rw-. 1 root root 49 Aug 30 19:28 .eslintignore
-rw-rw-rw-. 1 root root 1483 Aug 30 19:28 .eslintrc.js
drwxrwxrwx. 8 root root 4096 Aug 30 19:28 .git
-rw-rw-rw-. 1 root root 38 Aug 30 19:28 .git-ftp-include
-rw-rw-rw-. 1 root root 1200 Aug 30 19:28 .gitignore
-rw-rw-rw-. 1 root root 122 Aug 30 19:28 .npmrc
drwxrwxrwx. 5 root root 4096 Aug 30 19:28 .nuxt
-rw-rw-rw-. 1 root root 2541 Aug 30 19:28 README.md
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 api
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 assets
-rw-rw-rw-. 1 root root 1534 Aug 30 19:28 bitbucket-pipelines.yml
drwxrwxrwx. 5 root root 4096 Aug 30 19:28 components
-rw-rw-rw-. 1 root root 439 Aug 30 19:28 jest.config.js
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 layouts
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 locales
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 middleware
drwxr-xr-x. 1070 root root 36864 Aug 30 19:28 node_modules
-rw-rw-rw-. 1 root root 10189 Aug 30 19:28 nuxt.config.js
-rw-rw-rw-. 1 root root 611825 Aug 30 19:28 package-lock.json
-rw-rw-rw-. 1 root root 2219 Aug 30 19:28 package.json
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 pages
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 plugins
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 static
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 store
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 test

+ ls -la
total 760
drwxrwxrwx. 16 root root 4096 Aug 30 19:28 .
drwxr-xr-x. 8 root root 4096 Aug 30 19:27 ..
-rw-rw-rw-. 1 root root 212 Aug 30 19:28 .babelrc
-rw-rw-rw-. 1 root root 12 Aug 30 19:28 .dockerignore
-rw-rw-rw-. 1 root root 207 Aug 30 19:28 .editorconfig
-rw-rw-rw-. 1 root root 146 Aug 30 19:28 .env.default
-rw-rw-rw-. 1 root root 49 Aug 30 19:28 .eslintignore
-rw-rw-rw-. 1 root root 1483 Aug 30 19:28 .eslintrc.js
drwxrwxrwx. 8 root root 4096 Aug 30 19:28 .git
-rw-rw-rw-. 1 root root 38 Aug 30 19:28 .git-ftp-include
-rw-rw-rw-. 1 root root 1200 Aug 30 19:28 .gitignore
-rw-rw-rw-. 1 root root 122 Aug 30 19:28 .npmrc
drwxrwxrwx. 5 root root 4096 Aug 30 19:28 .nuxt
-rw-rw-rw-. 1 root root 2541 Aug 30 19:28 README.md
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 api
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 assets
-rw-rw-rw-. 1 root root 1534 Aug 30 19:28 bitbucket-pipelines.yml
drwxrwxrwx. 5 root root 4096 Aug 30 19:28 components
-rw-rw-rw-. 1 root root 439 Aug 30 19:28 jest.config.js
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 layouts
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 locales
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 middleware
drwxr-xr-x. 1070 root root 36864 Aug 30 19:28 node_modules
-rw-rw-rw-. 1 root root 10189 Aug 30 19:28 nuxt.config.js
-rw-rw-rw-. 1 root root 611825 Aug 30 19:28 package-lock.json
-rw-rw-rw-. 1 root root 2219 Aug 30 19:28 package.json
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 pages
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 plugins
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 static
drwxrwxrwx. 2 root root 4096 Aug 30 19:28 store
drwxrwxrwx. 4 root root 4096 Aug 30 19:28 test
You already have a 'node' cache so we won't create it again
Searching for files matching artifact pattern .nuxt/**
Artifact pattern .nuxt/** matched 24 files with a total size of 92.4 KiB
Compressed files matching artifact pattern .nuxt/** to 25 KiB in 0 seconds
Uploading artifact of 25 KiB
Successfully uploaded artifact in 1 seconds
Searching for files matching artifact pattern .dist/**
Searching for files matching artifact pattern dist/**

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

[Peter Plewa]

It looks like the /dist directory is not being generated at all. It looks specific to your project configuration and there are number reasons this may happen. Does it work when you run the same set of commands in your local environment and when you try to run it in Docker?

[Deleted user]

The /dist directory is built locally. That is the strange thing... But I didn't try to run it with docker locally at all, because the local set up is not in need of docker. 

But I will try it. Thanks for your time!

[dynamiteReady]

@[deleted] @Peter Plewa Can confirm the exact same issue here. Builds in other environments (from Windows 8.1, to Ubuntu 1604, with NodeJS 10.4.*), but not on any Docker image over Bitbucket Pipelines.

I suspect any fix found here, will work for me.

My project details are:

- Nuxt 2.*.*
- Node 10.4.*

Just a question for any technician re-examining this issue... Are there any limits on files sizes used with Pipelines that people should be aware of?