diff --git a/docker/.gitignore b/docker/.gitignore index bb538ed89a..7a18508fad 100644 --- a/docker/.gitignore +++ b/docker/.gitignore @@ -1,3 +1,3 @@ -logs/*.log +logs/**/*.log packages/ -unstable/*/bootstrap.ps1 +nightly/*/bootstrap.ps1 diff --git a/docker/README.md b/docker/README.md index 27dbac8212..467c0ad23b 100644 --- a/docker/README.md +++ b/docker/README.md @@ -2,7 +2,7 @@ Docker ====== These Dockerfiles enable building and running PowerShell in a container for each Linux distribution we support. -There are two sets: stable and unstable. +There are two sets: release and nightly. This requires an up-to-date version of Docker, such as 1.12. It also expects you to be able to run Docker without `sudo`. @@ -13,33 +13,41 @@ Please follow [Docker's official instructions][install] to install `docker` corr [install]: https://docs.docker.com/engine/installation/ -Stable ------- +Release +------- -The stable containers derive from the official distribution image, +The release containers derive from the official distribution image, such as `centos:7`, then install dependencies, and finally install the PowerShell package. +These containers live at [hub.docker.com/r/microsoft/powershell][docker-release]. + At about 440 megabytes, they are decently minimal, with their size being the sum of the base image (200 megabytes) plus the uncompressed package (120 megabytes), and about 120 megabytes of .NET Core and bootstrapping dependencies. -Unstable --------- +[docker-release]: https://hub.docker.com/r/microsoft/powershell/ -The unstable containers derive from their respective stable images, -such as `powershell/powershell:stable-centos7`, +Nightly +------- + +The nightly containers derive from their respective release images, +such as `powershell/powershell:centos7`, then run the `bootstrap.ps1` script which clones the repository, runs `Start-PSBootstrap -Package` to install building and packaging tools, runs `Start-PSBuild -Crossgen` to build PowerShell with native-image DLLs, runs `Start-PSPackage` to generate the platform's package, and finally installs the generated package. +These containers live at [hub.docker.com/r/microsoft/powershell-nightly][docker-nightly]. + At about 4 gigabytes, these images are *much* larger due to having all the tools installed. This is intended so that the containers are useful for reproducing packages. +[docker-nightly]: https://hub.docker.com/r/microsoft/powershell-nightly/ + Automation ========== @@ -47,15 +55,15 @@ A script, `launch.sh`, exists to automatically build all the respective images. It has a few configuration options, passed in as environment variables: -* `FORK`: the fork to clone in unstable builds (default: `PowerShell`) +* `FORK`: the fork to clone in nightly builds (default: `PowerShell`) * Set `FORK=andschwa` to clone https://github.com/andschwa/PowerShell -* `BRANCH`: the branch to checkout in unstable builds (default: `master`) +* `BRANCH`: the branch to checkout in nightly builds (default: `master`) * Set `BRANCH=feature-A` to checkout the `feature-A` branch -* `BUILDS`: the type of builds to run (default: `stable unstable`) - * This is order dependent! Stable images must exist for unstable images to be built - * Set `BUILDS=stable` to run the script for just the stable images +* `BUILDS`: the type of builds to run (default: `release nightly`) + * This is order dependent! Release images must exist for nightly images to be built + * Set `BUILDS=release` to run the script for just the release images * `DISTROS`: the distributions of Linux to run (default: `ubuntu14.04 ubuntu16.04 centos7`) * Set `DISTROS=centos7` to build just CentOS 7 images @@ -70,10 +78,10 @@ passed in as environment variables: * `RUNARGS`: additional arguments to be passed to `docker run` during testing * Set `RUNARGS="--rm"` to automatically delete the test container when finished -For each build type (stable and unstable), +For each build type (release and nightly), the selected distributions will run in parallel. The output is sent to log files in `docker/logs`. -For unstable builds, the generated packages are copied to `docker/packages`. +For nightly builds, the generated packages are copied to `docker/packages`. This script is very new, and there may be bugs. Use `set -x` to see exactly what commands it is running. @@ -84,13 +92,13 @@ Examples To build a CentOS container with the latest released package: ```sh -$ BUILDS=stable DISTROS=centos7 ./launch.sh -Logging to docker/logs/stable-centos7.log -Waiting for stable containers to finish; tail the logs for more information. +$ BUILDS=release DISTROS=centos7 ./launch.sh +Logging to docker/logs/release/centos7.log +Waiting for release containers to finish; tail the logs for more information. $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE -powershell/powershell stable-centos7 f0a11a8009b7 20 minutes ago 438.3 MB +powershell/powershell centos7 f0a11a8009b7 20 minutes ago 438.3 MB ... ``` @@ -98,15 +106,15 @@ To generate an Ubuntu 16.04 package from andschwa's `docker` branch: ```sh $ DISTROS=ubuntu16.04 FORK=andschwa BRANCH=docker ./launch.sh -Logging to docker/logs/stable-ubuntu16.04.log -Waiting for stable containers to finish; tail the logs for more information. -Logging to docker/logs/unstable-ubuntu16.04.log -Waiting for unstable containers to finish; tail the logs for more information. +Logging to docker/logs/release/ubuntu16.04.log +Waiting for release containers to finish; tail the logs for more information. +Logging to docker/logs/nightly/ubuntu16.04.log +Waiting for nightly containers to finish; tail the logs for more information. $ ls packages powershell_6.0.0-alpha.10-310-g5ded651-1ubuntu1.16.04.1_amd64.deb -$ tail logs/unstable-ubuntu16.04.log +$ tail logs/nightly/ubuntu16.04.log Step 5 : COPY bootstrap.ps1 / ---> Using cache ---> d18a28ff1e3d diff --git a/docker/launch.sh b/docker/launch.sh index 118492652b..4d1f9895e0 100755 --- a/docker/launch.sh +++ b/docker/launch.sh @@ -24,7 +24,7 @@ fi # Build both sets by default if [[ -z "$BUILDS" ]]; then - BUILDS="stable unstable" + BUILDS="release nightly" fi # Build specified distributions @@ -33,21 +33,28 @@ if [[ -z $DISTROS ]]; then fi for build in $BUILDS; do - # each distro can be done in parallel; but stable must be done before unstable + # each distro can be done in parallel; but release must be done before nightly for distro in $DISTROS; do - logfile="$build-$distro.log" + logfile="$distro.log" if [[ "$TEST" -eq 1 ]]; then logfile="test-$logfile"; fi - mkdir -p logs - logfile="logs/$logfile" + mkdir -p "logs/$build" + logfile="logs/$build/$logfile" echo "Logging to docker/$logfile" + + if [[ "$build" = release ]]; then + repo="powershell/powershell" + else + repo="powershell/powershell-nightly" + fi + ( - image="powershell/powershell:$build-$distro" + image="$repo:$distro" pushd "$build" if [[ "$TEST" -eq 1 ]]; then echo "LOG: testing $image" command="cd PowerShell; Import-Module ./build.psm1; Install-Dotnet -NoSudo; Start-PSPester -powershell powershell -Pester ./src/Modules/Shared/Pester" - # clone repo for stable images because it's not already done - if [[ "$build" = stable ]]; then + # clone repo for release images because it's not already done + if [[ "$build" = release ]]; then command="git clone --recursive https://github.com/$FORK/PowerShell -b $BRANCH; $command" fi # run Pester tests inside container @@ -56,14 +63,14 @@ for build in $BUILDS; do else echo "LOG: building $image" # copy the common script because it lives outside the docker build context - if [[ "$build" = unstable ]]; then + if [[ "$build" = nightly ]]; then cp bootstrap.ps1 "$distro" buildargs="--build-arg fork=$FORK --build-arg branch=$BRANCH" fi # build and tag the image so they can be derived from # BUILDARGS can be set in the environment docker build $buildargs $BUILDARGS -t "$image" "$distro" - if [[ "$build" = unstable ]]; then + if [[ "$build" = nightly ]]; then echo "LOG: Saving package to docker/packages" popd mkdir -p packages diff --git a/docker/unstable/bootstrap.ps1 b/docker/nightly/bootstrap.ps1 similarity index 100% rename from docker/unstable/bootstrap.ps1 rename to docker/nightly/bootstrap.ps1 diff --git a/docker/unstable/centos7/Dockerfile b/docker/nightly/centos7/Dockerfile similarity index 100% rename from docker/unstable/centos7/Dockerfile rename to docker/nightly/centos7/Dockerfile diff --git a/docker/unstable/ubuntu14.04/Dockerfile b/docker/nightly/ubuntu14.04/Dockerfile similarity index 100% rename from docker/unstable/ubuntu14.04/Dockerfile rename to docker/nightly/ubuntu14.04/Dockerfile diff --git a/docker/unstable/ubuntu16.04/Dockerfile b/docker/nightly/ubuntu16.04/Dockerfile similarity index 100% rename from docker/unstable/ubuntu16.04/Dockerfile rename to docker/nightly/ubuntu16.04/Dockerfile diff --git a/docker/stable/centos7/Dockerfile b/docker/release/centos7/Dockerfile similarity index 100% rename from docker/stable/centos7/Dockerfile rename to docker/release/centos7/Dockerfile diff --git a/docker/stable/ubuntu14.04/Dockerfile b/docker/release/ubuntu14.04/Dockerfile similarity index 100% rename from docker/stable/ubuntu14.04/Dockerfile rename to docker/release/ubuntu14.04/Dockerfile diff --git a/docker/stable/ubuntu16.04/Dockerfile b/docker/release/ubuntu16.04/Dockerfile similarity index 100% rename from docker/stable/ubuntu16.04/Dockerfile rename to docker/release/ubuntu16.04/Dockerfile