C# Compiler Incremental Compilation

This topic is based on a Microsoft developer design document that describes specific stages of the implementation of the C# compiler.  This topic is included here as a potential guide to understanding portions of the C# source code.

The following is an outline of the stages of COMPILER::CompileAll().

Read Incremental Information

• Reads incremental file.
• Checks file list and command line arguments.
  

Declare

• ParseTopLevel.
• Create NSDECLSYMs.
• Create AGGSYMs.
• Changed sources.
• Sets access and kind (isClass, isInterface, isEnum, isStruct, isDelegate).
• If type list changed, do full rebuild.
• Unchanged sources and metadata.
• Sets access and approximates kind (cannot set isEnum, isStruct, or isDelegate until base is resolved).

All types in the compilation are known; however note the following:

• The kind (class, struct, and so on) is not necessarily correct.
• Types have no members or bases.

Check for change in type list. A change to the following will cause a full rebuild:

• Number of types.
• File a type was declared in.
• Accessibility of a type.
• Addition or removal of a type.
  

Resolve Inheritance

• Resolves namespace using clauses.
• Sets isResolvingBaseClasses.
• Binds to base class and implemented interfaces and resolve recursively.
• Sets kind for types imported from metadata and declared in unchanged sources.
• Sets hasResolvedBaseClasses.
• If base has changed, sets full rebuild.

All types have:

• Base class and interface lists are set.
• Kind (isEnum or isStruct) is set.
• Resolved using clauses for namespaces.

It is now legal to look up types in any context.

• Check for changes in the inheritance hierarchy.
• If any change is detected, stop resolving the inheritance hierarchy before the change is made and recover to a full rebuild.
  

Define

• Special classes (Object, AttributeUsage, marked as conditional or obsolete) have members defined.
• Declares global (module and assembly) attributes.
• Sets isDefined bit on AGGSYMs.
• Defines aggregate members non-recursively:
  • Does not define property accessors.
  • Does not check explicit interface implementations.
  • Uses 'name' attribute for indexers.
• Sets special compile time attributes:
  • Obsolete attribute.
  • Attribute-usage attribute.
  • Conditional attribute.
• Evaluates compile time constants:
  • Might cause other types to be defined on demand.
  • Might cause other constants to be evaluated on demand.
  • Causes files to be marked as changed if they are required to be in a defined state to evaluate a constant.

Changed files have all public members declared (except property accessors). 
Public (non-private, to be precise) interface is compared against metadata. 
If a type has changed, all dependant files are added to the changed list. 
If the changed threshold is reached (50% of files have been read from source), recover to a full rebuild.

At this point, the final decision is made for either an incremental or a full rebuild.

For an incremental rebuild, read member definitions for all unchanged files from metadata.

Prepare

• Prepare base classes and implemented interfaces.
  • For metadata and unchanged sources reads members from metaimport.
• Set isPrepared.
• Build abstract method list.
• Build implemented interface list.
• Set isOverride bit on imported methods.
• For changed sources.
  • Check definitions against inherited members (new and override).
  • Define property accessors.
  • Check explicit interface implementations.
  • Resolve layout for structs.

Common Language Specification

Common language specification checking on all members performed.

Emit (EmitTokens and Compile)

• Initializes emitter iface.
• Tokens for all members are emitted.
• Assembly options emitted to alink.
• Members compiled.
• Attributes emitted.
• Method bodies emitted.
• Finish output file.
• Do resources.
• Finish assembly emit.

Write Incremental Information

Writes incremental information file.

Marking a Source Input File as Changed

Things that cause a file to be marked as changed:

• The file time stamp has changed since the last rebuild.
• A file dependancy had an interface change.
• A changed file requires to evaluate one of its constants that is defined in the current file.
• Backing out to a full rebuild.

State a file can be in:

• Undeclared.
  • The file is fresh, we simply declare as normal.
• Declared.
  • The text contents of the file are unchanged.
  • Declarations for the types in the file have been read in from old metadata.
  • The list of types (and their namespaces) contained in this file can not have changed.
  • Parse file.
  • Declare file:
    • Creates new NSDECLSYMs for the file.
    • Matches the new parse trees to the existing AGGSYM declarations.
    • Re-parents the AGGGSYMs to the new NSDECLSYMs.
    • Leaves orphan NSDECLSYMs from the metadata import.
• Bases resolved.
  • Same as declared except:
    • The type list has not changed, so the base list for this type cannot have changed.
    • Never import the member definitions for a source file from metadata until you know you are never going to back out to a full rebuild.

Backing Out to a Full Rebuild

To back out to a full rebuild, all remaining unchanged files are marked as changed.  This forces them to be brought up to the current stage.

Emitting Metadata and Incremental Update

• Stages of metadata:
  • Has type list changed?
    • New types.
    • Deleted types.
    • Accessibility.
    • Kind (enum, struct, class, delegate).
    • Base class.
    • Implemented interfaces.
  • Has the interface type changed?
    • For each public member token in metadata:
      • Find matching symbol (except any property accessors, as they are not created yet).
      • Compare metadata and symbol.
    • Count public members from metadata.
    • Count public members from symbols (add the uncreated property accessors to count).
    • Compare counts of public members.
  • Match existing tokens to symbols.
  • Reemit tokens in changed files.
    • Delete tokens that are nolonger present.
    • Add new tokens.
    • Update tokens that have not changed.
      

Types

Interface Changed

• Check to see if type list has changed during ResolveBase.
• Do not add or delete types.
• Check custom attributes:
  • Attribute-Usage.
  • Obsolete.
  • CLS-Compliant Attribute.

Reemit

• Delete related tokens:
  • Custom Attributes.
  • Permissions tdHasSecurity.
  • Class Layout.
  • MethodImpls.
• Set flags with SetTypeDefProps.
• Base, interfaces remain unchanged.
• Emit Custom Attributes:
  • User custom attributes.
  • Pseudo-custom attributes:
    • StructLayout             tdLayoutMask
    • Guid
    • ComImport                 tdImport
    • InterfaceType
    • ClassInterface
    • Serializable                 tdSerializable
  • Compiler Generated:
    • DefaultMember.
    • StructLayout on structs.

Parameters

• Check by method for HasChanged.
• Don't add or delete parameter tokens.
• All parameter tokens must have the 'name' member set.
• Return value tokens are optional - only emitted if they have custom attributes.
  

Reemit

• Delete related tokens:
  • Custom Attributes.
  • Field Marshal.
• DefineParamProps - set flags and clear PCAs.
• Emit Custom Attributes:
  • User custom attributes.
  • Pseudo-custom attributes.
    • In                             pdIn.
    • Out                           pdOut.
    • Optional                     pdOptional.
    • Marshal                      pdHasFieldMarshal.
  • Compiler generated CAs.
    • ParamArray
    • Out

Methods

Emit

• EmitMethodDef:
  • Sets flags, signature, name, and parent.
  • Emits MethodImpls if required.
• EmitMethodInfo
  • Sets MethodImpl flags.
• Emit Custom Attributes.
• Emit Method RVA.

HasChanged

• Find existing symbol based on:
  • Name
  • Return type
  • Parameter types
• Check match on:
  • Accessibility
  • Paramarray
  • Varargs
  • Static
  • Virtual/override/abstract/new/sealed
  • Operator/implicit/explicit
  • Custom Attributes:
    • Conditional
    • Obsolete
    • CLS

Reemit

• Find existing symbol based on:
  • Name
  • Return type
  • Parameter types
  • Is static
  • Is varargs
• Delete related tokens:
  • Custom Attributes
  • PInvoke Map
  • Permissions
• Set flags with SetMethodProps
  • Note: cannot reset the signature.
• Add MethodImpl tokens.
• Emit Custom Attributes:
• User custom attributes
  • Pseudo-custom attributes
    • DllImport                     mdPinvokeImpl
    • MethodImpl                 miPreserveSig, miInternalCall, miSynchronized, miNoInlining
    • PreserveSig                 miPreserveSig
• Compiler-generated custom attributes.
  

Method Impls Structure

• Done during method emit.
• Deleted as related tokens of the type.
• Added again during method reemit.

Fields

Emit

• EmitMembVarDef
  • Name, flags, signature, parent, and constant value
• Emit custom attributes:
  • User custom attributes
  • Pseudo-custom attributes:
    • NonSerialized                 fdNotSerialized
    • Marshal                        fdHasFieldMarshal
    • FieldOffset
  • Compiler Generated custom attributes.
    • Decimal Constant
      

HasChanged

• Find existing symbol based on:
  • Name
  • Type
• Check match on:
  • Accessibility
  • Static/const/readonly
  • Constant value
  • Volatile
  • Custom attributes:
    • Obsolete
    • CLS-compliant

Reemit

• Find existing symbol based on:
  • Name
  • Type
  • Static
  • Volatile
• Delete related tokens:
  • Custom attributes.
• Set flags and constant value.
• Emit custom attributes.
  
  

Properties

Emit

• Nothing emitted for explicit property implementations.
  • Method impls structure only
• EmitPropertyDef:
  • Parent, name, get, set, and signature.
• Emit custom attributes:
  • User custom attributes .

HasChanged

• Find existing symbol based on:
  • Name
  • Argument types
• Check match on:
  • Not checked - checked by accessors:
    • Accessibility
    • Return type
    • Varargs
    • Param array
  • Indexer name.
  • Accessor count.
  • Custom attributes:
    • Obsolete
    • CLS

Reemit

• Find existing symbold based on:
  • Name
  • Indexer name
  • Argument types
  • Return types
  • Is static
• Delete related tokens:
  • Custom attributes.
• Reset:
  • Flags
  • Get/set tokens
• Emit custom attributes.
  
  

Events

Emit

• Nothing emitted for explicit event implementations.
• EmitEventDef:
  • Parent, name, type, add, and remove
• Emit custom attributes:
  • User custom attributes.

HasChanged

• Find existing symbol based on:
  • Name
• Check match on:
  • Type
  • Access
  • Custom attributes:
    • Obsolete
    • CLS

Reemit

• Find existing symbol based on:
  • Name
• Delete related tokens:
  • Custom attributes.
• Reset:
  • Flags
  • Type
  • Accessor tokens
• Emit custom attributes:
  • User custom attributes
    

Assembly

Emit

• Emit Custom attributes:
  • User custom attributes.
  • Pseudo-custom attributes.
  • Compiler-generated custom attributes:
    • Security Permission
    • Debuggable

HasChanged

• Check for match on CLS-compliant attribute:
  • Force full build on change.

Reemit

• Delete related tokens:
  • Custom Attributes
  • Security Permissions
• Emit custom attributes.

Module

Emit

• Emit custom attributes:
  • User custom attributes.
• Compiler-generated custom attributes:
  • UnverifiableCode

HasChanged

• Check for match on CLS-compliant attribute:
  • Force full build on change.

Reemit

• Delete related tokens:
  • Custom attributes.
• Emit custom attributes.
  
  

Private Implementation Details

Emit

• Track tokens by input file.

Reemit

• Delete all tokens for all changed input files.
  

Copyright (c) 2002 Microsoft Corporation. All rights reserved.