nBuildKit release - V0.2.4

Monday, July 21, 2014 | Posted in nBuildKit

Version V0.2.4 of the nBuildKit build library has been released.

This release introduces NuGet packages that allow Visual Studio C# and WiX projects to share a common configuration including code analysis and strong naming capabilities and to generate one or more source files before building the project.

Both packages provide the ability to share a configuration between multiple projects in a Visual Studio solution. The shared configuration can contain items like Debug and Release settings, targeted .NET framework, code analysis settings and many other options.

Beyond that the C# NuGet package provides capabilities to:

  • Generate AssemblyInfo.XXXX.cs files for:
    • AssemblyInfo.VersionNumber.cs contains the version numbers for the project as determined by nBuildKit from the versioning strategies provided. Currently supported are either a version number provided through an MsBuild project file or through GitHubFlowVersion. By allowing nBuildKit to generate the AssemblyInfo.VersionNumber.cs file it is possible to automatically version the binaries the same way as all other artifacts, e.g. NuGet packages, installers, documentation etc. etc.. The version number is currently provided through the AssemblyVersionAttribute as Major.Minor.0.0; through the AssemblyFileVersionAttribute as Major.Minor.Patch.Build; and through the AssemblyInformationalVersionAttribute as the full semantic version.
    • AssemblyInfo.BuildInformation.cs contains information about the current build of the binaries. This includes the configuration, e.g. Release; the date and time that the binary was compiled and information describing the build number and the commit number that were used to generate the binaries.
    • AssemblyInfo.InternalsVisibleTo.cs contains the InternalsVisibleToAttribute values for any assemblies that should have access to the internals of the current assembly for purposes of unit testing.
    • AssemblyInfo.Company.cs contains the information describing the 'company' that created the binaries. The information includes the AssemblyCompanyAttribute and the AssemblyCopyrightAttribute.
  • Generate a CompanyInformation.cs source file that contains an internal static class providing constants for the company name and the company URL.
  • Generate a ProductInformation.cs source file that contains an internal static class providing constants for the name of the product.
  • Generate an app.manifest file with the current project version number.

And the WiX NuGet package provides capabilities to

  • Generate a VersionNumber.wxi WiX include file that contains the version numbers for the application and the installer.
  • Generate a CompanyInformation.wxi WiX include file that contains the name and URL of the company that produces the product.
  • Generate a ProductInformation.wxi WiX include file that contains the name and description of the product.

The generation of all of these files can be enabled through a setting in the settings.props file that is also used by the nBuildKit.MsBuild NuGet package

The documentation for this library can be found on the nBuildKit wiki

nBuildKit release - V0.1.5

Monday, June 16, 2014 | Posted in nBuildKit

Version V0.1.5 of the nBuildKit build library has been released.

This first release introduces the MsBuild NuGet package. This package contains build scripts that provide the ability to perform a complete build consisting of workspace preparation, compilation of binaries, execution of unit tests, analysis of source code and binaries and finally packing of the binaries as NuGet packages or ZIP archives. The nBuildKit.MsBuild NuGet package also provides scripts that can be used to tag a release in the version control system (VCS) and deploy the artifacts to a file server, NuGet feed or a GitHub release.

The documentation for this library can be found on the nBuildKit wiki

Nuclei release - V0.8.0

Wednesday, May 28, 2014 | Posted in Nuclei

Version V0.8.0 of the Nuclei library has been released.

This release introduces a few large features to Nuclei.Communication library and adds some minor updates to the base Nuclei library.

The main focus of this release was adding version tolerance (#3) to the different layers of the communication stack:

  • The discovery layer - Provides ways to discover remote endpoints either automatically (via WCF discovery) or manually.
  • The protocol layer - Provides the means to send messages to one or more remote endpoints and handling the responses to those messages if they are expected.
  • The interaction layer - Provides an abstraction over the protocol layer in the form of an proxy objects that provide user-defined methods which can be invoked on a remote endpoint.

In version 0.8.0 of the Nuclei.Communication library each of these layers now supports the ability to negotiate with a remote endpoint to determine which communication version will be used to exchange data between the endpoints. The communication version which will be used is the highest (i.e. most recent) version that both endpoints support. If the endpoint do not support the same versions then communication will not be enabled between the endpoints.

The second focus was to improve the robustness of all the network activity. The main changes here were:

  • #7 - Detection of messages that have not received their response within a given time-out.
  • #11, #17, #18 - Method(s) to detect if a remote endpoint is still available and discard endpoint information if it is not.
  • #19, #23 - Automatically rebuilding the communication channel if it faults during message sending and then resend the message that caused the faulting.

The final focus was on decoupling the interaction interfaces from their implementations. With these changes an endpoint does not need to provide concrete implementations of either the command (#6) and notification (#31) interfaces but can map the members on those interfaces to equivalent members on any given object.

Additionally it is now possible for some parameters on a command interface method (ie. the method on a command interface) and a command object method (i.e .the concrete instance method that is mapped to the given command) to contain parameters that have a special meaning. The available options are:

  • For command interface methods (#23)
    • Provision of a time-out indicating the maximum amount of time the endpoint should wait for a response to the command invocation request.
    • Provision of a retry count indicating the maximum number of times the endpoint should send the command invocation request should any errors occur during invocation.
  • For command instance methods (#28)
    • Provision of the ID of the endpoint requesting the invocation of the command method.
    • Provision of the ID of the message that was send to request the invocation of the command method.

Finally the Nuclei library versioning was switched to use semantic versioning.

Improvements to the site

Friday, May 23, 2014 | Posted in Blog

Over the last few days some changes were made to the structure of this website. Of these changes the first change that was made is in the move from the standard GitHub pages URL to my own custom domain provided by the amazingly helpful people at dnsimple. While the URL for this website has changed it is still being hosted by GitHub pages and so the old URL will continue working just fine.

The only slight catch is that the project pages for my projects now suddenly redirect through my new domain as well. Apparently this is due to the way GitHub handles custom domains. For now I will live with this situation but later on my personal projects may be moved to separate organisations.

The second change is the addition of pagination for the landing page. As I continued to write more blog posts this page was getting rather long and slow to load. The new landing page only contains the last five blog posts and provides a way to navigate to the previous posts at the bottom of the page. This should make the page much quicker to load. I will share the implementation of the pagination in a future blog post but for those who are interested the changes necessary to implement pagination in DocPad are pretty minor and can be seen in a single commit.

Besides setting up the pagination of the landing page I also created an archive page which, logically, shows all the blog posts in chronological order. Again a future blog post will describe the necessary changes to include the archive page.

Last but not least the site has gotten a new favicon which is a little more vortex-y than the last icon.

Embedding a symbol stream in a PDB file

Monday, May 19, 2014 | Posted in Source server PDB Symbol stream SRCSRV nAnicitus UNC

The nAnicitus application processes NuGet symbol packages to push the symbols and sources up to their respective location for the symbol and source servers to work. In order to have a symbol server nothing special needs to be done, just push the symbols through the SymStore application and a nice directory with indexed symbols is created. However in order to allow debuggers to obtain the source files related to a given PDB some manipulation of the PDB files is necessary. Specifically the SRCSRV stream in the PDB file needs to be modified.

The documentation gives a decent overview of how the source indexing works but it does not actually provide the information necessary to determine what should be written to the SRCSRV stream for a given PDB file. In fact if you want to know what information should be written to the SRCSRV stream if you want to store the indexed source files in a directory instead of getting them from your source control system then even the almighty google is rather quiet.

In order to write to the SRCSRV stream one can use the PDBStr utility. However that still leaves the question of what to write to the stream. The source server documentation provides examples for VERSION 1 streams, i.e. the kind that point the debugger to a source control (VCS) command. If however you want to store symbols and sources in a UNC path then you need VERSION 2 streams. With some help from this blog post, some digging at MSDN and lots of trial-and-error it seems that the SRCSRV stream should look like:

SRCSRV: ini ------------------------------------------------
VERSION=2
INDEXVERSION=2
VERCTRL=http
SRCSRV: variables ------------------------------------------
SRCSRVVERCTRL=http
UNCROOT= <UNC_SOURCE_PATH>
HTTP_EXTRACT_TARGET=%UNCROOT%\\ + <UNC_SOURCE_PATH>
SRCSRVTRG=%http_extract_target%
SRCSRVCMD=
SRCSRV: source files ---------------------------------------

In this stream the variables mean:

  • Version = 2 - Indicates the version of the SRCSRV stream
  • VerCtrl = http - 'Version control' is done through HTTP. This variable is potentially optional.
  • SRCSRVVERCTRL = http - Specifies the VCS in use. In this case that's UNC, potentially over http.
  • UNCROOT - 'Local variable' indicating what the UNC root path is.
  • HTTP_EXTRACT_TARGET - 'Local variable' indicating how to determine the path of a source file on the server given it's embedded path and SRCSRV information.
  • SRCSRVTG - The template used by the debugger to determine the path of the source files based on their embedded path and SRCSRV information.
  • SRCSRVCMD - The command for the VCS to extract the source files. For UNC this is not required.

When nAnicitus processes a PDB file it generates a SRCSRV file that looks similar to this:

SRCSRV: ini ------------------------------------------------
VERSION=2
INDEXVERSION=2
VERCTRL=http
SRCSRV: variables ------------------------------------------
SRCSRVVERCTRL=http
UNCROOT=\\MyServer\sources
HTTP_EXTRACT_TARGET=%UNCROOT%\%var2%\%var3%\%var4%
SRCSRVTRG=%http_extract_target%
SRCSRVCMD=
SRCSRV: source files ---------------------------------------
c:\source\MyProject\MyClass.cs*MyProject*1.2.3.4*MyProject\MyClass.cs
SRCSRV: end------------------------------------------------

The redirection of the source paths is handled as <FILE>*<PROJECT>*<VERSION>*<RELATIVE_FILE> which means that the c:\source\MyProject\MyClass.cs file will be found on the source server at \\MyServer\sources\MyProject\1.2.3.4\MyProject\MyClass.cs

The process of pushing the symbols and sources up to their respective locations with nAnicitus is done via the following steps:

  1. The user drops the NuGet symbol package containing the binaries, PDB files and source files into the designated upload folder for nAnicitus.
  2. nAnicitus detects the new NuGet package and pushes the file path onto the queue for processing by the indexing thread.
  3. The indexing thread pulls the file path from the queue and unzips the NuGet symbol package in a temporary location.
  4. For each PDB the source paths are extracted from the PDB with scrtool.exe.
  5. For each source path the matching source file is located by looking at all source files and seeing source file path matches the 'best', i.e. using the longest common substring approach, starting from the end of the path in order to ensure a match on the file name.
  6. Once the source file is located the relative file path for the source file on the source server is calculated.
  7. Once all the source files from the PDB are processed the SRCSRV stream file is created and embedded into the PDB with pdbstr.exe.
  8. Once all PDBs have been indexed they are pushed through the symstore.exe tool to the symbol server.
  9. The source files are copied to the desired directory on the source server.
  10. Finally the original NuGet symbol package is moved to the directory containing all the processed symbol packages.

Note that the path to the source server directory is embedded in the PDB. If the location of the source server changes then the information in the PDB files will no longer be correct. While there is a way to redirect the embedded paths nAnicitus also stores the processed NuGet symbol packages in a designated directory. This makes it possible to re-process the packages should the need ever arise.