From d8c353e2daccb907d6bc856d5ec99e8284759f62 Mon Sep 17 00:00:00 2001 From: Khosrow Moossavi <372575+khos2ow@users.noreply.github.com> Date: Fri, 22 Jun 2018 14:17:51 -0400 Subject: [PATCH 1/5] Update Readme --- README.md | 151 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 150 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a5085a3..41b2d53 100644 --- a/README.md +++ b/README.md @@ -1 +1,150 @@ -# cloudstack-systemvm-builder +# CloudStack SystemVM Template builder using Docker + +[![Build Status](https://travis-ci.com/khos2ow/cloudstack-systemvm-builder.svg?branch=master)](https://travis-ci.com/khos2ow/cloudstack-systemvm-builder) +[![Docker Automated build](https://img.shields.io/docker/automated/khos2ow/cloudstack-systemvm-builder.svg)](https://hub.docker.com/r/khos2ow/cloudstack-systemvm-builder/) +[![Docker Build Status](https://img.shields.io/docker/build/khos2ow/cloudstack-systemvm-builder.svg)](https://hub.docker.com/r/khos2ow/cloudstack-systemvm-builder/builds/) +[![license](https://img.shields.io/github/license/khos2ow/cloudstack-systemvm-builder.svg)](https://github.com/khos2ow/cloudstack-systemvm-builder/blob/master/LICENSE) + +Docker images for building Apache CloudStack SystemVM Templates. + +This will give portable, immutable and reproducable mechanism to build templates for releases. A very good candidate to be used by the Jenkins slaves of the project. + +## Table of Contents + +- [Supported tags and respective `Dockerfile` links](#supported-tags-and-respective-dockerfile-links) +- [Packges installed in conatiner](#packges-installed-in-conatiner) +- [Build templates](#build-templates) + - [Pull Docker images](#pull-docker-images) + - [Build local repository](#build-local-repository) + - [Clone Apache CloudStack source code](#clone-apache-cloudstack-source-code) + - [Build templates of local repository](#build-templates-of-local-repository) + - [Build remote repository](#build-remote-repository) + - [Build templates of remote repository](#build-templates-of-remote-repository) +- [Building tips](#building-tips) + - [Maven cache](#maven-cache) + - [Adjust host owner permission](#adjust-host-owner-permission) +- [Builder help](#builder-help) +- [License](#license) + +## Supported tags and respective `Dockerfile` links + +- [`packer` (packer/Dockerfile)](https://github.com/khos2ow/cloudstack-systemvm-builder/blob/master/packer/Dockerfile) +- [`veewee` (veewee/Dockerfile)](https://github.com/khos2ow/cloudstack-systemvm-builder/blob/master/veewee/Dockerfile) + +## Packges installed in conatiner + +List of available packages inside the container: + +- rpm-build +- yum-utils +- createrepo +- mkisofs +- git +- java 1.8 +- maven 3.5 +- tomcat +- python +- locate +- which + +## Build templates + +Building SystemVM templates with the Docker container is rather easy, a few steps are required: + +### Pull Docker images + +Let's assume we want to build templates based on Packer. We pull that image first: + + docker pull khos2ow/cloudstack-systemvm-builder:packer + +You can replace `packer` tag by `veewee` if you are building templates for CloudStack before 4.11.x. + +### Build local repository + +You can clone the CloudStack source code from repository locally on your machine and build templates against that. + +#### Clone Apache CloudStack source code + +The first step required is to clone the CloudStack source code somewhere on the filesystem, in `/tmp` for example: + + git clone https://github.com/apache/cloudstack.git /tmp/cloudstack + +Now that you have done so we can continue. + +#### Build templates of local repository + +Now that we have cloned the CloudStack source code locally, we can build templates by mapping `/tmp` into `/mnt/build` in the container. (Note that the container always expects the `cloudstack` code exists in `/mnt/build` path.) + + docker run \ + -v /tmp:/mnt/build \ + khos2ow/cloudstack-systemvm-builder:packer TODO [ARGS...] + +Or if your local cloudstack folder has other name, you need to map it to `/mnt/build/cloudstack`. + + docker run \ + -v /tmp/cloudstack-custom-name:/mnt/build/cloudstack \ + khos2ow/cloudstack-systemvm-builder:packer TODO [ARGS...] + +After the build has finished the *various formats* packages are available in */tmp/cloudstack/tools/appliance/dist* on the host system. + +### Build remote repository + +Also you can build templates of any remote repository without the need to manually clone it first. You only need to specify git remote and git ref you intend to build from. + +#### Build templates of remote repository + +Now let's assume we want to build templates of `HEAD` of `master` branch from https://github.com/apache/cloudstack repository, we build templates by mapping `/tmp` into `/mnt/build` in the container. The container will clone the repository (defined by `--git-remote` flag) and check out the REF (defined by `--git-ref` flag) in `/mnt/build/cloudstack` inside the container and can be accessed from `/tmp/cloudstack` from the host machine. + + docker run \ + -v /tmp:/mnt/build \ + khos2ow/cloudstack-systemvm-builder:packer \ + --git-remote https://github.com/apache/cloudstack.git \ + --git-ref master \ + TODO [ARGS...] + +Note that any valid git Refspec is acceptable, such as: + +- `refs/heads/` to build specified Branch +- `` short version of build specified Branch +- `refs/pull//head` to build specified GitHub Pull Request +- `refs/merge-requests//head` to build specified GitLab Merge Request +- `refs/tags/` to build specified Tag + +After the build has finished the *various formats* packages are available in */tmp/cloudstack/tools/appliance/dist* on the host system. + +## Building tips + +Check the following tips when using the builder: + +### Maven cache + +You can provide Maven cache folder (`~/.m2`) as a volume to the container to make it run faster. + + docker run \ + -v /tmp:/mnt/build \ + -v ~/.m2:/root/.m2 \ + khos2ow/cloudstack-systemvm-builder:packer TODO [ARGS...] + +### Adjust host owner permission + +**TODO** Builder container in some cases (e.g. using `--use-timestamp` flag) may change the file and directory owner shared from host to container (through volume) and it will create `dist` directory which holds the final artifacts. You can provide `USER_ID` (mandatory) and/or `USER_GID` (optional) from host to adjust the owner from whitin the container. + +This is specially useful if you want to use this image in Jenkins job and want to clean up the workspace afterward. By adjusting the owner, you won't need to give your Jenkins' user `sudo` privilege to clean up. + + docker run \ + -v /tmp:/mnt/build \ + -e "USER_ID=$(id -u)" \ + -e "USER_GID=$(id -g)" \ + khos2ow/cloudstack-systemvm-builder:packer TODO [ARGS...] + +## Builder help + +To see all the available options you can pass to `docker run ...` command: + + docker run \ + -v /tmp:/mnt/build \ + khos2ow/cloudstack-systemvm-builder:packer --help + +## License + +Licensed under [Apache License version 2.0](http://www.apache.org/licenses/LICENSE-2.0). Please see the [LICENSE](https://github.com/khos2ow/cloudstack-systemvm-builder/blob/master/LICENSE) file included in the root directory of the source tree for extended license details. From c18c2f301d1ef86ecca4d02cc464220a6fd84d77 Mon Sep 17 00:00:00 2001 From: Khosrow Moossavi <372575+khos2ow@users.noreply.github.com> Date: Fri, 22 Jun 2018 14:18:14 -0400 Subject: [PATCH 2/5] Add Makefile --- Makefile | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 Makefile diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..aba8824 --- /dev/null +++ b/Makefile @@ -0,0 +1,34 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +.PHONY: all veewee packer + +# Build docker tag based on provided info +# +# $1: tag_name +# $2: directory_name +define build_tag + docker build -t khos2ow/cloudstack-systemvm-builder:$(1) $(2) +endef + +all: veewee packer + +veewee: + $(call build_tag,veewee,veewee) + +packer: + $(call build_tag,packer,packer) From afb8fc3bac80aa2d18e11d46cfd29c394d2ed9eb Mon Sep 17 00:00:00 2001 From: Khosrow Moossavi <372575+khos2ow@users.noreply.github.com> Date: Fri, 22 Jun 2018 14:18:44 -0400 Subject: [PATCH 3/5] Add Jekyll config file --- _config.yml | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 _config.yml diff --git a/_config.yml b/_config.yml new file mode 100644 index 0000000..8cf0d11 --- /dev/null +++ b/_config.yml @@ -0,0 +1,3 @@ +title: CloudStack SystemVM Template builder +description: Docker images for building Apache CloudStack SystemVM Templates +theme: jekyll-theme-cayman \ No newline at end of file From d87e75bd91164ccd959ba7ec06f02f34f7e44485 Mon Sep 17 00:00:00 2001 From: Khosrow Moossavi <372575+khos2ow@users.noreply.github.com> Date: Fri, 22 Jun 2018 14:19:43 -0400 Subject: [PATCH 4/5] Add Travis CI config --- .travis.yml | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 .travis.yml diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..645879a --- /dev/null +++ b/.travis.yml @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +sudo: required +dist: trusty +group: edge + +language: generic +services: + - docker + +cache: + directories: + - $HOME/.m2 + timeout: 500 + +notifications: + email: false + +env: + global: + - IMAGE="khos2ow/cloudstack-systemvm-builder" + matrix: + - TAG="veewee" PARAMS="" + - TAG="packer" PARAMS="" + +install: make ${TAG} + +before_script: + - mkdir -p ${TRAVIS_BUILD_DIR}/tmp + - mkdir -p $HOME/.m2 + +script: | + docker run \ + --volume ${TRAVIS_BUILD_DIR}/tmp:/mnt/build \ + --volume $HOME/.m2:/root/.m2 \ + --rm \ + ${IMAGE}:${TAG} \ + --git-remote https://github.com/apache/cloudstack.git \ + --git-ref refs/heads/master \ + --remove-first \ + ${PARAMS} From aad1af2485ee4e732459e613cd8dd2326c210a83 Mon Sep 17 00:00:00 2001 From: Khosrow Moossavi <372575+khos2ow@users.noreply.github.com> Date: Wed, 27 Jun 2018 13:32:29 -0400 Subject: [PATCH 5/5] Update Readme --- README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 41b2d53..1b2fcf0 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,7 @@ This will give portable, immutable and reproducable mechanism to build templates List of available packages inside the container: +- **TODO** - rpm-build - yum-utils - createrepo @@ -57,7 +58,7 @@ Let's assume we want to build templates based on Packer. We pull that image firs docker pull khos2ow/cloudstack-systemvm-builder:packer -You can replace `packer` tag by `veewee` if you are building templates for CloudStack before 4.11.x. +You need to replace `packer` tag by `veewee` if you are building templates for CloudStack before `4.11.x.y`. ### Build local repository @@ -85,7 +86,7 @@ Or if your local cloudstack folder has other name, you need to map it to `/mnt/b -v /tmp/cloudstack-custom-name:/mnt/build/cloudstack \ khos2ow/cloudstack-systemvm-builder:packer TODO [ARGS...] -After the build has finished the *various formats* packages are available in */tmp/cloudstack/tools/appliance/dist* on the host system. +After the build has finished the *various formats* templates are available in */tmp/cloudstack/tools/appliance/dist* on the host system. ### Build remote repository @@ -110,7 +111,7 @@ Note that any valid git Refspec is acceptable, such as: - `refs/merge-requests//head` to build specified GitLab Merge Request - `refs/tags/` to build specified Tag -After the build has finished the *various formats* packages are available in */tmp/cloudstack/tools/appliance/dist* on the host system. +After the build has finished the *various formats* templates are available in */tmp/cloudstack/tools/appliance/dist* on the host system. ## Building tips