From 56e9600e030f3522d02c956cdb227f16d9518350 Mon Sep 17 00:00:00 2001 From: Zachary Folwick Date: Wed, 30 Sep 2015 12:28:45 -0700 Subject: [PATCH 01/12] initial commit --- docs/Testing.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 docs/Testing.md diff --git a/docs/Testing.md b/docs/Testing.md new file mode 100644 index 0000000000..49ba0ac14c --- /dev/null +++ b/docs/Testing.md @@ -0,0 +1,32 @@ +# PowerShell for Linux + +This readme is targeted at PowerShell for Linux users looking to write test suites to ensure quality of PowerShell products. + +## Getting started + +These instructions assume Ubuntu 14.04 LTS. It is assumed that PowerShell for Linux is currently installed on the system. + +### Obtain PowerShell + +PowerShell is required to enable and run the test suites. + +### Testing technology + +-------|------ +Pester | default cmdlet test framework +xUnit | default C# test framework +cppUnit | default C/C++ testing framework + +### Running the test suite + +Tests can be run from the `scripts` folder. If you are currently in `monad-linux/scripts`, then run `./build.sh make test` to run the complete tests suite. The table below shows the commands to run for the various test products bundled with Powershell for Linux (the table assumes the current working directory is `monad-linux/scripts`). + +It is strongly recommended that before major changes are tested, that users run `./build.sh make clean cleanall prepare` to ensure the environment is completely clean. + +------|--------- +Pester | ./build.sh make pester-tests +xUnit | ./build.sh make xunit-tests +cppunit | ./build.sh make native-tests +hashbang tests | ./build.sh make hashbang-tests + +within the `scripts` directory, you may also wish to run Pester tests on a single file. This can be done easily using `./build.sh make {path/from/scripts/to/pester/test} (E.g, `./build.sh make ../src/pester-test/Test-TESTFILE.Test.ps1`). From 690ae54b2c484e48281ff9a3379e64c25c251f2e Mon Sep 17 00:00:00 2001 From: Zachary Folwick Date: Wed, 30 Sep 2015 12:35:38 -0700 Subject: [PATCH 02/12] fixed markdown --- docs/Testing.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/Testing.md b/docs/Testing.md index 49ba0ac14c..f3a2685d17 100644 --- a/docs/Testing.md +++ b/docs/Testing.md @@ -12,6 +12,7 @@ PowerShell is required to enable and run the test suites. ### Testing technology +Technology | Purpose -------|------ Pester | default cmdlet test framework xUnit | default C# test framework @@ -23,6 +24,7 @@ Tests can be run from the `scripts` folder. If you are currently in `monad-linux It is strongly recommended that before major changes are tested, that users run `./build.sh make clean cleanall prepare` to ensure the environment is completely clean. +Technology | Run Method ------|--------- Pester | ./build.sh make pester-tests xUnit | ./build.sh make xunit-tests From cf78fa523c8baeb9a56f10a29436428baa0b7bff Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:35:58 +0000 Subject: [PATCH 03/12] Updated Testing.md --- docs/Testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Testing.md b/docs/Testing.md index f3a2685d17..4ca2d6ce36 100644 --- a/docs/Testing.md +++ b/docs/Testing.md @@ -31,4 +31,4 @@ xUnit | ./build.sh make xunit-tests cppunit | ./build.sh make native-tests hashbang tests | ./build.sh make hashbang-tests -within the `scripts` directory, you may also wish to run Pester tests on a single file. This can be done easily using `./build.sh make {path/from/scripts/to/pester/test} (E.g, `./build.sh make ../src/pester-test/Test-TESTFILE.Test.ps1`). +within the `scripts` directory, you may also wish to run Pester tests on a single file. This can be done easily using `./build.sh make {path/from/scripts/to/pester/test}` (E.g, `./build.sh make ../src/pester-test/Test-TESTFILE.Test.ps1`). From 7fa684d54bdf382a9b76d603cd95dcc28d1fbaa1 Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:37:37 +0000 Subject: [PATCH 04/12] Added file Pester Tests.md --- docs/Pester Tests.md | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 docs/Pester Tests.md diff --git a/docs/Pester Tests.md b/docs/Pester Tests.md new file mode 100644 index 0000000000..7d7a500147 --- /dev/null +++ b/docs/Pester Tests.md @@ -0,0 +1,2 @@ +#Pester Testing Test Guide + From c0c0ad5f33aa35665a2780822b16ce357d5be4db Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:38:09 +0000 Subject: [PATCH 05/12] Added file xUnit Tests --- docs/xUnit Tests | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/xUnit Tests diff --git a/docs/xUnit Tests b/docs/xUnit Tests new file mode 100644 index 0000000000..0ed9533df9 --- /dev/null +++ b/docs/xUnit Tests @@ -0,0 +1 @@ +# xUnit Tests Guide \ No newline at end of file From 2ad1f8fba7d28a444a167d9b215ba998a87a71e6 Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:38:17 +0000 Subject: [PATCH 06/12] Updated xUnit Tests --- docs/xUnit Tests | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/xUnit Tests b/docs/xUnit Tests index 0ed9533df9..8a2b09f150 100644 --- a/docs/xUnit Tests +++ b/docs/xUnit Tests @@ -1 +1 @@ -# xUnit Tests Guide \ No newline at end of file +#xUnit Tests Guide \ No newline at end of file From 89e84b0765dc99f4f3cd686ee1edf0a47cc7752c Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:38:41 +0000 Subject: [PATCH 07/12] Deleted /docs/xUnit Tests --- docs/xUnit Tests | 1 - 1 file changed, 1 deletion(-) delete mode 100644 docs/xUnit Tests diff --git a/docs/xUnit Tests b/docs/xUnit Tests deleted file mode 100644 index 8a2b09f150..0000000000 --- a/docs/xUnit Tests +++ /dev/null @@ -1 +0,0 @@ -#xUnit Tests Guide \ No newline at end of file From 568e3ca6c51762c0e2ae98612afbce3dcfe5e3a1 Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:39:09 +0000 Subject: [PATCH 08/12] Added file xUnit Tests.md --- docs/xUnit Tests.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/xUnit Tests.md diff --git a/docs/xUnit Tests.md b/docs/xUnit Tests.md new file mode 100644 index 0000000000..aab9d866d6 --- /dev/null +++ b/docs/xUnit Tests.md @@ -0,0 +1 @@ +#xUnit Testing Guide \ No newline at end of file From a41de25a12a06507ea27ee1c083bb8eff7ee9e83 Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:40:07 +0000 Subject: [PATCH 09/12] Added file Native Tests.md --- docs/Native Tests.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/Native Tests.md diff --git a/docs/Native Tests.md b/docs/Native Tests.md new file mode 100644 index 0000000000..b1261c48ba --- /dev/null +++ b/docs/Native Tests.md @@ -0,0 +1 @@ +# Native Code Testing Guide \ No newline at end of file From ba665f52c9b0ec559e5cb403748eea76ff58a2b1 Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 21:43:18 +0000 Subject: [PATCH 10/12] Updated Pester Tests.md --- docs/Pester Tests.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/Pester Tests.md b/docs/Pester Tests.md index 7d7a500147..3f8e0d61e5 100644 --- a/docs/Pester Tests.md +++ b/docs/Pester Tests.md @@ -1,2 +1,9 @@ #Pester Testing Test Guide +## Who this is for + +Cmdlet behavior is validated using the Pester testing framework. The purpose of this document is to create a single standard to maximize unit test coverage while minimizing confusion on expectations. What follows is a working document intended to guide those writing Pester unit tests for PSL. + +Test suites need to be created and many cmdlets added and unit-tested. The following list is to be used to guide the thought process of the developer in writing a suite in minimal time, while enhancing quality. + +Test suites should proceed as functional and system tests of the cmdlets, and the code treated as a black box for the purpose of test suite design. \ No newline at end of file From 6208416418f7a27d2d5ca873171f2290d2c3a82e Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 22:19:55 +0000 Subject: [PATCH 11/12] Updated Pester Tests.md --- docs/Pester Tests.md | 88 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 85 insertions(+), 3 deletions(-) diff --git a/docs/Pester Tests.md b/docs/Pester Tests.md index 3f8e0d61e5..0cb788f5b2 100644 --- a/docs/Pester Tests.md +++ b/docs/Pester Tests.md @@ -2,8 +2,90 @@ ## Who this is for -Cmdlet behavior is validated using the Pester testing framework. The purpose of this document is to create a single standard to maximize unit test coverage while minimizing confusion on expectations. What follows is a working document intended to guide those writing Pester unit tests for PSL. +Cmdlet behavior is validated using the Pester testing framework. The purpose of this document is to create a single standard to maximize unit test coverage while minimizing confusion on expectations. What follows is a working document intended to guide those writing Pester unit tests for PSL. + +Unit testing is done not only to validate that the block of code works as expected, but also to assist the developer to know precisely where in the code to look; in some cases, seeing the source code may inspire better unit tests. In many cases, a unit test *is* the only documented specification. Fortunately, the MSDN is a great source of information about Cmdlets. + +Test suites need to be created and many cmdlets added and unit-tested. The following list is to be used to guide the thought process of the developer in writing a suite in minimal time, while enhancing quality. + +Test suites should proceed as functional and system tests of the cmdlets, and the code treated as a black box for the purpose of test suite design. + + +### Use of Mocks +It is often necessary for the code to interact with the system or other components. When possible, use Mock objects to facilitate this in order to minimize external dependencies. Note: creating a Mock in PSL causes PowerShell to look at the Mock, never actually hitting any C# code. Cmdlets cannot be tested using Mocks. + +### Aliases + Each cmdlet with an alias must be tested with all of its aliases at least once to verify the code path calls the original function. + +## Testing Standards +### Readability +Every effort should be made to maximize readability of code. Code is written for the developer in the future to debug- not for the developer writing the code. + +1) When assertions are on consecutive lines, the pipes should line up: + +```sh +MyFirstCondition | Should Be 0 +MySecondCondition | Should Be 1 +``` + +This is less readable than: +```sh +MyFirstCondition | Should Be 0 +MySecondCondition | Should Be 1 +``` + +So the second section of code should instead be used. + +2) Use readable and meaningful variable name when assigning variables. + +3) Do not make large functions. Tests should be simple: define -> manipulate -> assert + +4) Remove the first 3 auto-generated lines of each .Tests.ps1 file. This is created automatically by Pester and is unnecessary. Each .Test.ps1 file should begin with a Describe block. + +5) Discard the auto-generated function file that is generated in tandem with the .Tests.ps1 file + +6) Name the test file "Test- when you create a new test fixture. + +7) Each test describes a behavior- use the word "Should" at the beginning of each test description- so it reads "It 'Should..." + +### Basic Unit Tests + +The following table should suffice to inspire in the developer sufficient content to create a suite of tests. + +test # | test name | entry criteria/setup | exit criteria/assertion +-------|-----------|----------------------|------------------------ +01 | Should be able to be called | without params (if applicable) | no throw +02 | Should be able to be called | minimal required params | no throw, expected output +03 | Should be able to use the X alias | minimal required params | no throw, expected output +04 | Should return the proper data type | required params | no throw, proper data type +05 | Should be able to accept piped input | piped input | expected output +06 | Should be able to call using the X parameter | use X parameter | no throw, expected output +07 | Should be able to call using the Y parameter | use Y parameter | no throw, expected output +08 | Should be able to call using the Z parameter | use Z parameter | no throw, expected output +09 | Should throw under condition X | create condition X | Throw error x +10 | Should throw under condition Y | create condition Y | Throw error y +11 | Should throw under condition Z | create condition Z | Throw error z + +These are the **basic** unit tests required to verify the functionality of any Cmdlet. If the above questions cannot be answered for each Cmdlet, then they cannot be verified to work. + +Look at the existing suites of pester tests located within `monad-linux/src/pester-test/` and use that as inspiration. + + +##Running Pester Tests +Pester tests may be run from outside of PowerShell via the command line. Build PowerShell and Pester using (assuming you're in the build folder) `./build.sh make ../src/pester-test/` or `./build.sh make pester-tests` + + + + + + + + + + + + + + -Test suites need to be created and many cmdlets added and unit-tested. The following list is to be used to guide the thought process of the developer in writing a suite in minimal time, while enhancing quality. -Test suites should proceed as functional and system tests of the cmdlets, and the code treated as a black box for the purpose of test suite design. \ No newline at end of file From 762913fb31a726d54b42baae9aba8c848a981518 Mon Sep 17 00:00:00 2001 From: Zach Folwick Date: Wed, 30 Sep 2015 23:51:36 +0000 Subject: [PATCH 12/12] Updated Pester Tests.md --- docs/Pester Tests.md | 35 ++++++++++++++++++++++++++++++----- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/docs/Pester Tests.md b/docs/Pester Tests.md index 0cb788f5b2..9a9e40fbbb 100644 --- a/docs/Pester Tests.md +++ b/docs/Pester Tests.md @@ -34,19 +34,44 @@ MyFirstCondition | Should Be 0 MySecondCondition | Should Be 1 ``` -So the second section of code should instead be used. +So the second section of code should instead be used. The same style should be followed for assignments of variables on consecutive lines: + +```sh +$var1 = +$variable2 = +$var3 = +$typeCollection1 = +$object1 = +... etc +``` + +is much less readable than +```sh +$var1 = +$variable2 = +$var3 = +$typeCollection1 = +$object1 = +... etc +``` + +So all assignment statements must be aligned. + +Other style standards are no less important to readability of the code: 2) Use readable and meaningful variable name when assigning variables. 3) Do not make large functions. Tests should be simple: define -> manipulate -> assert -4) Remove the first 3 auto-generated lines of each .Tests.ps1 file. This is created automatically by Pester and is unnecessary. Each .Test.ps1 file should begin with a Describe block. +4) Do not use tabs. Tabs are rendered differently depending upon the machine. This greatly affects readability. -5) Discard the auto-generated function file that is generated in tandem with the .Tests.ps1 file +5) Remove the first 3 auto-generated lines of each .Tests.ps1 file. This is created automatically by Pester and is unnecessary. Each .Test.ps1 file should begin with a Describe block. -6) Name the test file "Test- when you create a new test fixture. +6) Discard the auto-generated function file that is generated in tandem with the .Tests.ps1 file -7) Each test describes a behavior- use the word "Should" at the beginning of each test description- so it reads "It 'Should..." +7) Name the test file "Test- when you create a new test fixture. + +8) Each test describes a behavior- use the word "Should" at the beginning of each test description- so it reads "It 'Should..." ### Basic Unit Tests