Bitbucket Pipelines

yaatcicd

(Yet Another Approach To Continuous Integration & Continuous Delivery)

Introduction

  • Tim Bailey-Jones
  • Tampa, Florida
  • Over 25+ years in Linux/Unix
    Development/Automation
  • Amatuer Brass Player
    (trumpets/euphoniums/tubas)

Pipelines

  • An on-demand based build service triggered by repository pushes (git or mercurial)
  • Replaces the cost/overhead of a separate, dedicated CI server like Jenkins or Bamboo
  • Great for small projects without a lot of change
  • Fresh Builds occur in temporary/ephemeral Docker containers
  • Complete isolation from any side-effects from previous builds
BitBucket Pipelines supports lots of languages.

Bitbucket Pipelines

To activate Pipelines, navigate to the
BitBucket repository page.

Select "Pipelines NEW" from the left-hand menu.
Now Pipelines needs a file called bitbucketpipelines.yml from the root of your repository.

Fortunately, BitBucket gives you a great starting point. You just need to pick what kind of project it is:
bitbucket-pipelines.yml template for .NET Core:

# You can specify a custom docker image from Docker Hub as your build environment.
image: microsoft/dotnet:onbuild
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - export PROJECT_NAME=yourProjectName
          - export TEST_NAME=yourTestName
          - dotnet restore
          - dotnet build $PROJECT_NAME
          - dotnet test $TEST_NAME
                    
                
bitbucket-pipelines.yml template for C++ - Make
          
image: gcc:6.1
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - ./configure
          - make
          - make test
                    
                
bitbucket-pipelines.yml template for Clojure
                    
image: clojure:lein-2.7.1
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - lein test
                    
                
bitbucket-pipelines.yml template for Docker
                    
options:
  docker: true
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          # Set $DOCKER_HUB_USERNAME and $DOCKER_HUB_PASSWORD as environment variables in repository settings
          - export IMAGE_NAME=your-Dockerhub-account/your-docker-image-name:$BITBUCKET_COMMIT
          # build the Docker image (this will use the Dockerfile in the root of the repo)
          - docker build -t $IMAGE_NAME .
          # authenticate with the Docker Hub registry
          - docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
          # push the new Docker image to the Docker registry
          - docker push $IMAGE_NAME
                    
                
bitbucket-pipelines.yml template for Golang
                    
image: golang:1.7
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - PACKAGE_PATH="${GOPATH}/src/bitbucket.org/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}"
          - mkdir -pv "${PACKAGE_PATH}"
          - tar -cO --exclude-vcs --exclude=bitbucket-pipelines.yml . | tar -xv -C "${PACKAGE_PATH}"
          - cd "${PACKAGE_PATH}"
          - go get -v
          - go build -v
          - go test -v
                    
                
bitbucket-pipelines.yml template for Haskell
                    
image: haskell:7.10
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - stack test
                    
                
bitbucket-pipelines.yml template for Gradle/Java
                    
image: java:8
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          # You must commit the Gradle wrapper to your repository
          # https://docs.gradle.org/current/userguide/gradle_wrapper.html
          - bash ./gradlew build
                    
                
bitbucket-pipelines.yml template for Maven/Java
                    
image: maven:3.3.9
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - mvn -B verify # -B batch mode makes Maven less verbose
                    
                
bitbucket-pipelines.yml template for NodeJS
                    
image: node:6.9.4
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - npm install
          - npm test
                    
                
bitbucket-pipelines.yml template for PHP
                    
image: php:7.1.1
pipelines:
  default:
    - step:
        script:
          - apt-get update && apt-get install -y unzip
          - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
          - composer install
          - vendor/bin/phpunit
                    
                
bitbucket-pipelines.yml template for Ruby
                    
image: ruby:2.4.0
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - bundle install
                    
                
bitbucket-pipelines.yml template for Scala
                    
image: bitbucketpipelines/scala-sbt:scala-2.12
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - sbt test
                    
                
Save your template, and BitBucket creates and commits it to your repo.
Save your template, and BitBucket creates and commits it to your repo.
Save your template, and BitBucket creates and commits it to your repo.
Save your template, and BitBucket creates and commits it to your repo.
BitBucket immediately starts your build, because it is triggered by new commits.
In fact, it already failed!

Yes, let's find out why... Click on the red Failed! link.

npm failed because no package.json file exists yet.
Typical package.json file create, add and push.
Another Pipeline ran. It failed, because the package.json defined no tests.
You can just about anything you want in the steps section of the bitbucket-pipelines.yml file. (e2e tests, unit tests, code analysis, chat/email notifications, deploying artifacts, and the like)
So, I changed the exit 1 to exit 0 *, committed and pushed again.
* NOTE: Yes, turning off tests is usually not a Good Thing, but this is a demo.
A third pipeline ran, and was successful. Let's go ahead and see the successful results.
By default, steps without errors are not expanded.

Pipeline Environment Variables

Environment variables are a good place to
store certain kinds of information.
(machine names, IP addresses, usernames, passwords, API tokens, etc)

BitBucket has a hierarchy of environment variable sets.

  • Team-level variables (collections of Accounts)
  • Account-level variables
  • Repository-level variables
  • Pipeline-level variables (NEW!)

Any variable in one set can be overridden by a variable of the same name in a more-specific set.
(i.e. top-to-bottom)

Setting sensitive values in the BitBucket GUI.

Modify the bitbucket-pipelines.yml file to access the variables

edit, git add, git commit, git push

Accessing sensitive values in the pipeline environment.


BitBucket runs the pipeline faster than you can Alt-Tab.

Accessing sensitive values in the pipeline environment.


You can see the values echoed correctly, even though they are not in the repository source tree.

Deployment

The builds happen in ephemeral Docker containers, which are destroyed upon completion.
So, to be useful, deploy your build someplace else.
  • Hosting service
    (AWS, Azure, Google Cloud, OpenStack, Rackspace, Digital Ocean, etc)
  • Artifact repository
    (hub.docker.com, deb/rpm repository, JFrog Artifactory, etc)

Pipelines has many integrations to help out.

Deployment Integrations

  • Bitbucket - to the Downloads section of your repo
  • Amazon AWS (Elastic Beanstalk, S3, CodeDeploy)
  • Google Cloud (App Engine & Container Engine)
  • Microsoft Azure
  • Heroku
  • Docker Hub
  • Kubernetes
  • Read the Docs
  • NPM
  • Maven
  • Ansible Tower
  • Anything else?
    • Any arbitrary Webhook via curl or wget