From 4c967d13963bbfcc274baa80c4f849e4910d1500 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Tue, 5 Apr 2016 18:12:10 -0700 Subject: [PATCH 01/10] Update commiting.md : src/monad submodule is gone Adding an Update section with explanation that the whole complicated process is rarely needed after #656 fix [skip ci] --- docs/git/committing.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/git/committing.md b/docs/git/committing.md index 3625801aa6..305e64484a 100644 --- a/docs/git/committing.md +++ b/docs/git/committing.md @@ -1,5 +1,10 @@ #Commit Dance -Most of the time, you're going to be working in a submodule (say src/monad/monad/engine). + +**Update:** commit dance became much simpler after [removing psl-monad submodule](https://github.com/PowerShell/PowerShell/issues/656). +Meahwhile, there are still few submodules. If you need to touch their content, this doc provides the overview of the process. +Remember that it's written against `src/monad` submodule, which doesn't exist anymore. + +Sometimes, you need to do the work in a submodule (i.e. you added a new string in `.resx` file). The submodule has a relationship to the SuperProject (PowerShell), but in order to be sure that CI is notified about changes in a submodule, you need be sure that this is reflected as a pull request in the SuperProject. From ba2da7fcb3e0b59e833879b8d130309ef134e2dc Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Tue, 5 Apr 2016 18:38:36 -0700 Subject: [PATCH 02/10] Add workflow/resource.md [skip ci] Workflow for changes that touches .resx files - editing - updating - add/remove - start-resgen --- docs/workflow/resources.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 docs/workflow/resources.md diff --git a/docs/workflow/resources.md b/docs/workflow/resources.md new file mode 100644 index 0000000000..4a1749e0bd --- /dev/null +++ b/docs/workflow/resources.md @@ -0,0 +1,31 @@ +# Resources + +Resources are `.resx` files with string values that we use for error messages and such. +They live in `src\\resources` folders. + +At the moment `dotnet cli` doesn't support generating C# bindings (strongly typed resource files). + +We are using `src\windows-build\gen` folder in [src\windows-build](https://github.com/PowerShell/psl-windows-build) +with pre-generated `.cs` files to work-around it. +See [issue 756](https://github.com/PowerShell/PowerShell/issues/746) for details. + +## Editing resx files + +**Don't edit** resx files from Visual Studio. +It will try to create `.cs` files for you and you will get whole bunch of hard-to-understand errors. + +To edit resource file, use any **plain text editor**. +Resource file is a simple xml, and it's easy to edit. + +### Updating string + +If you just updated the string value, that's all you need to do: no need to re-generate `.cs` files + +### Adding or removing string + +When you adding or removing string, `.cs` file need to be changed. + +1. Run `Start-ResGen` function from `PowerShellGitHubDev.psm1` +1. Make sure your code is building with newly generated resources (run `Start-PSBuild`). +1. Go to submodule (`cd src\windows-build`) and perform the [submodule commit dance](docs/git/committing.md). +Follow working with [submodule rules](CONTRIBUTING.md#submodules) From bbee138aa1a1ac5913ce681acc33ab6ec881a926 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Tue, 5 Apr 2016 19:00:32 -0700 Subject: [PATCH 03/10] Create workflow/branches.md document [skip ci] - explain 3 long living branches and their relations. - describe a simple bugfix workflow --- docs/workflow/branches.md | 49 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 docs/workflow/branches.md diff --git a/docs/workflow/branches.md b/docs/workflow/branches.md new file mode 100644 index 0000000000..db1f9b0f35 --- /dev/null +++ b/docs/workflow/branches.md @@ -0,0 +1,49 @@ +# Branches + +PowerShell has a number of [long-living](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows#Long-Running-Branches) branches + +* **master** -- current development, upstream. It's based on source-depot, but has changes for Linux/OS X. +It also has changes for the latest version of .NET Core, i.e. rc3. +* **source-depot** -- an exact mirror of dev branch in Source Depot. +Here code flow in "sd -> github" case. +It should be treated mostly as **read-only** (unless you are integrating changes "sd->github"). +* **server2016** -- branch with the code that we want to ship in Server 2016. We use it for "github -> sd" integration. + +### Example bugfix + +Create a [feature-branch](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows#Topic-Branches) from **server2016**. + +``` +> # switch branch to server2016 to create feature-branch from the right commit +> git checkout server2016 +Branch server2016 set up to track remote branch server2016 from origin. +Switched to a new branch 'server2016' + +# create feature branch +> git checkout -b vors/encoding +``` + +**Note** how we use `alias/feature-name` pattern in the example above. + +Then we develop the changes in the feature branch. +We can push it to the [`origin`](https://github.com/PowerShell/PowerShell) to kick-in CI build or share work-in-progress. + +``` +> git push origin vors/encoding +``` + +Eventually we merge feature branch back to **server2016** via a Pull Request with a codereview from my peers. + +After merging feature branch we should bring changes into **master**, so we have bugfixes in the upstream branch as well. + +``` +> git checkout master +> git checkout -b vors/master +> # look-up the sha1 for relevant commits (i.e. with gitk --all) +> git cherry-pick .. +> git push origin vors/master +``` + +**Note** the first commit is not included in the diaposone, so the interval is open on the left side. + +Then I can create a Pull Requst from `vors/master` to `master` via GitHub web interface. From 48e2f5081ad11b4c4cb3c6281e6d80778a2df817 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Tue, 5 Apr 2016 19:05:04 -0700 Subject: [PATCH 04/10] Update references in resources.md [skip ci] --- docs/workflow/resources.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/workflow/resources.md b/docs/workflow/resources.md index 4a1749e0bd..64e93b9e42 100644 --- a/docs/workflow/resources.md +++ b/docs/workflow/resources.md @@ -27,5 +27,5 @@ When you adding or removing string, `.cs` file need to be changed. 1. Run `Start-ResGen` function from `PowerShellGitHubDev.psm1` 1. Make sure your code is building with newly generated resources (run `Start-PSBuild`). -1. Go to submodule (`cd src\windows-build`) and perform the [submodule commit dance](docs/git/committing.md). -Follow working with [submodule rules](CONTRIBUTING.md#submodules) +1. Go to submodule (`cd src\windows-build`) and perform the [submodule commit dance](../git/committing.md). +Follow working with [submodule rules](../../CONTRIBUTING.md#submodules) From de51b2d7ab0f1e24ccaf117e20b5d3ff53012731 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Tue, 5 Apr 2016 19:20:26 -0700 Subject: [PATCH 05/10] Add workflow/mapping.md to docs [skip ci] - explain mapping concept - describes mapping.json format - overview function to work wiht mapping.json files from PowerShellGitHubDev.psm1 --- docs/workflow/mapping.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 docs/workflow/mapping.md diff --git a/docs/workflow/mapping.md b/docs/workflow/mapping.md new file mode 100644 index 0000000000..27a9b8e533 --- /dev/null +++ b/docs/workflow/mapping.md @@ -0,0 +1,37 @@ +# Mapping + +PowerShell/PowerShell utilizes `dotnet cli` project model. +Source code for a library (executable) is located under `src/`. +I.e. System.Management.Automation.dll sources are located under `src/System.Management.Automation` + +In the windows source tree, this files located differently. +That's why we have `mapping.json` in the root of PowerShell/PowerShell repo. +This file is a simple json hashtable that describes mapping between files in source depot and GitHub. + +Keys are relative file paths from `src\monad` submodule (that has the same layout as admin sd enlistment). +Values are relative file paths in PowerShell/PowerShell GitHub project. + +**Note**: this "src\monad" prefix appears in keys for historical reasons. +We used to have a submodule at this path. +If you replace this `src\monad` with path to the **admin** enlistment, you will get the mapping to source depot. + +### PowerShellGitHubDev.psm1 + +Our dev module contains a number of functions to work that can be used to work with this mapping file. + +* `Copy-SubmoduleFiles` -- is used for "sd -> github" integration +* `New-MappingFile` -- was used to create the first version on mapping.json +* `Send-GitDiffToSd` -- the most interesting function for us: +it applies patch from git to **admin** enslistment with respect to `mapping.json`. +It supports `-WhatIf` switch. + +``` +> Send-GitDiffToSd -diffArg1 45555786714d656bd31cbce67dbccb89c433b9cb -diffArg2 45555786714d656bd31cbce67dbccb89c433b9cb~1 -pathToAdmin d:\e\ps_dev\admin +> cd d:\e\ps_dev\admin +> sd online ... +> # move files to new change list (i.e. with sdb) +> sd submit -c + +``` + + From d1908d40a85d11a32ba3e0cc72ecab49ae33d062 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Tue, 5 Apr 2016 19:29:33 -0700 Subject: [PATCH 06/10] Include links to docs\workflow\ in CONTRIBUTING.md - branches.md - mapping.md - resource.md [skip ci] --- CONTRIBUTING.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 63d74f905a..9112b5e615 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -65,7 +65,7 @@ Particularly: [MS-OSS-Hub]: https://opensourcehub.microsoft.com/articles/how-to-join-microsoft-github-org-self-service [2 factor authentication]: https://github.com/blog/1614-two-factor-authentication -Branches +[Branches](docs/workflow/branches.md) -------- * Checkout a new local branch for every change you want to make (bugfix, feature). @@ -109,6 +109,7 @@ updates, ensure no other changes are in the same commit. Submodule bumps may be included in feature branches for ease of work, but the update must be independently approved before merging into master. + Recommended Git configurations ------------------------------ @@ -139,3 +140,14 @@ git config --global rerere.enabled true git config --global rerere.autoUpdate true git config --global am.threeWay true ``` + +[Mapping](docs/workflow/mapping.md) +-------- + +Learn about new files locations in PowerShell/PowerShell. + +[Resources](docs/workflow/resources.md) +-------- + +Learn how to work with string resources in `.resx` files. + From 2bc6f6985fffa5335fa927385c9103108ae50850 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Tue, 5 Apr 2016 19:40:59 -0700 Subject: [PATCH 07/10] Add section about update in mapping.md [skip ci] --- docs/workflow/mapping.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/workflow/mapping.md b/docs/workflow/mapping.md index 27a9b8e533..c0189b0715 100644 --- a/docs/workflow/mapping.md +++ b/docs/workflow/mapping.md @@ -34,4 +34,23 @@ It supports `-WhatIf` switch. ``` +## Updating `mapping.json` + +If you are bringing new (that are not yet included) files from source-depot, you need to update `mapping.json` to include them. +This way, we can keep track of changes and have ability to integrate changes back to Source Depot. +We will use term **integrate** for that kind of new files. + +* Make a separate commit with update for `mapping.json`. Separate commit will help to manage this change in other branches. + +* You can use `Copy-SubmoduleFiles` function to copy files on disk. + +* Make a separate commit for integrated files. +Use `--author="PowerShell Team "` switch to indicate that it's a collective work. + +``` +git commit --author="PowerShell Team " +``` + +Use this approach for **test files** as well. +You can add them under `test` directory and include in CI test run, but keep the notion of integration in `mapping.json`. From 27eb8ecd12d4be5f019c33a858b2e3fd65f1e267 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Wed, 6 Apr 2016 11:32:10 -0700 Subject: [PATCH 08/10] Remove server2016 from the build status matrix [skip ci] --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 6a4731c7fa..c0e7396183 100644 --- a/README.md +++ b/README.md @@ -16,10 +16,10 @@ flavor of PowerShell. It is a collaborative effort among many teams: Build Status ------------ -| Platform | `master` | `server2016` | -|--------------|----------|--------------| -| Ubuntu 14.04 | [![Build Status](https://travis-ci.com/PowerShell/PowerShell.svg?token=31YifM4jfyVpBmEGitCm&branch=master)](https://travis-ci.com/PowerShell/PowerShell) | [![Build Status](https://travis-ci.com/PowerShell/PowerShell.svg?token=31YifM4jfyVpBmEGitCm&branch=server2016)](https://travis-ci.com/PowerShell/PowerShell) | -| Windows | [![Build status](https://ci.appveyor.com/api/projects/status/wb0a0apbn4aiccp1/branch/master?svg=true)](https://ci.appveyor.com/project/PowerShell/powershell-linux/branch/master) | [![Build status](https://ci.appveyor.com/api/projects/status/wb0a0apbn4aiccp1/branch/server2016?svg=true)](https://ci.appveyor.com/project/PowerShell/powershell-linux/branch/server2016) | +| Platform | `master` | +|--------------|----------| +| Ubuntu 14.04 | [![Build Status](https://travis-ci.com/PowerShell/PowerShell.svg?token=31YifM4jfyVpBmEGitCm&branch=master)](https://travis-ci.com/PowerShell/PowerShell) | +| Windows | [![Build status](https://ci.appveyor.com/api/projects/status/wb0a0apbn4aiccp1/branch/master?svg=true)](https://ci.appveyor.com/project/PowerShell/powershell-linux/branch/master) | Get PowerShell -------------- From bb8ed24b86e5b8c244dbb9dbbb342948e8a01073 Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Wed, 6 Apr 2016 11:35:32 -0700 Subject: [PATCH 09/10] Remove server2016 branch from branches.md [skip ci] --- docs/workflow/branches.md | 41 +-------------------------------------- 1 file changed, 1 insertion(+), 40 deletions(-) diff --git a/docs/workflow/branches.md b/docs/workflow/branches.md index db1f9b0f35..8144df2bc7 100644 --- a/docs/workflow/branches.md +++ b/docs/workflow/branches.md @@ -4,46 +4,7 @@ PowerShell has a number of [long-living](https://git-scm.com/book/en/v2/Git-Bran * **master** -- current development, upstream. It's based on source-depot, but has changes for Linux/OS X. It also has changes for the latest version of .NET Core, i.e. rc3. -* **source-depot** -- an exact mirror of dev branch in Source Depot. +* **source-depot** -- an exact mirror of dev branch in Source Depot for all [mapped](./mapping.md) files. Here code flow in "sd -> github" case. It should be treated mostly as **read-only** (unless you are integrating changes "sd->github"). -* **server2016** -- branch with the code that we want to ship in Server 2016. We use it for "github -> sd" integration. -### Example bugfix - -Create a [feature-branch](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows#Topic-Branches) from **server2016**. - -``` -> # switch branch to server2016 to create feature-branch from the right commit -> git checkout server2016 -Branch server2016 set up to track remote branch server2016 from origin. -Switched to a new branch 'server2016' - -# create feature branch -> git checkout -b vors/encoding -``` - -**Note** how we use `alias/feature-name` pattern in the example above. - -Then we develop the changes in the feature branch. -We can push it to the [`origin`](https://github.com/PowerShell/PowerShell) to kick-in CI build or share work-in-progress. - -``` -> git push origin vors/encoding -``` - -Eventually we merge feature branch back to **server2016** via a Pull Request with a codereview from my peers. - -After merging feature branch we should bring changes into **master**, so we have bugfixes in the upstream branch as well. - -``` -> git checkout master -> git checkout -b vors/master -> # look-up the sha1 for relevant commits (i.e. with gitk --all) -> git cherry-pick .. -> git push origin vors/master -``` - -**Note** the first commit is not included in the diaposone, so the interval is open on the left side. - -Then I can create a Pull Requst from `vors/master` to `master` via GitHub web interface. From 1560809147bda709018fe20f6b3791cc97bc5e2c Mon Sep 17 00:00:00 2001 From: Sergei Vorobev Date: Wed, 6 Apr 2016 11:37:39 -0700 Subject: [PATCH 10/10] Update mapping.md Add a note to use source-depot branch for initial adding [skip ci] --- docs/workflow/mapping.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/workflow/mapping.md b/docs/workflow/mapping.md index c0189b0715..98bf4bd245 100644 --- a/docs/workflow/mapping.md +++ b/docs/workflow/mapping.md @@ -40,6 +40,8 @@ If you are bringing new (that are not yet included) files from source-depot, you This way, we can keep track of changes and have ability to integrate changes back to Source Depot. We will use term **integrate** for that kind of new files. +* Use `source-depot` branch to initially add files. + * Make a separate commit with update for `mapping.json`. Separate commit will help to manage this change in other branches. * You can use `Copy-SubmoduleFiles` function to copy files on disk. @@ -51,6 +53,8 @@ Use `--author="PowerShell Team "` switch to indicate git commit --author="PowerShell Team " ``` +* Merge changes to `master` + Use this approach for **test files** as well. You can add them under `test` directory and include in CI test run, but keep the notion of integration in `mapping.json`.