views:

230

answers:

2

Here's a strange one relating to a combination of the 'define' and 'include' functionality that the CC.NET preprocessor exposes. We're running CCNet 1.4.4.83, and our ccnet.config file is structured and split to make best use of common element blocks stored in subfiles that are included by the main config file; we also split out the core project definitions into their own include files too, leaving ccnet.config as essentially a series of includes thus:

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">
  <!-- Configuration root folder - defined so we can use it as a symbol instead of using slightly confusing relative paths -->
  <cb:define ccConfigRootFolder="C:\CruiseControl.NET\Config"/>

<!-- Globals - standard or shared build elements common to all projects -->
<cb:include href="$(ccConfigRootFolder)\Globals\globals.xml" xmlns:cb="urn:ccnet.config.builder"/>

<!-- CruiseControl.NET Configuration - refresh configuration if changed -->
  <cb:include href="$(ccConfigRootFolder)\CCNet Configuration\ccnet_configuration.xml" xmlns:cb="urn:ccnet.config.builder"/>

<!-- Project #1 -->
  <cb:include href="$(ccConfigRootFolder)\Project1\project1.xml" xmlns:cb="urn:ccnet.config.builder"/>

<!-- Project #2 -->
  <cb:include href="$(ccConfigRootFolder)\Project2\project2.xml" xmlns:cb="urn:ccnet.config.builder"/>
</cruisecontrol>

This works a treat - the preprocessor correctly includes and parses the <define> elements in globals.xml (and recursively parses further files included from globals.xml too) and the projects included afterwards (which contain references to those defined elements) are parsed correctly.

To further refine ccnet.config in an attempt to reduce the possibility of mistakes breaking the build process, we altered it to look like this:

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">
    <!-- Configuration root folder -->
    <cb:define ccConfigRootFolder="C:\CruiseControl.NET\Config"/>

    <!-- Project 'include' element definition -->
    <cb:define name="ProjectInclude">
        <cb:include href="$(ccConfigRootFolder)$(ccIncludePath)" xmlns:cb="urn:ccnet.config.builder"/>
    </cb:define>

    <!-- Include common gobal elements -->
    <cb:ProjectInclude ccIncludePath="\Globals\globals.xml"/>

    <!-- Project #1 -->
    <cb:ProjectInclude ccIncludePath="\Project1\project1.xml"/>

    <!-- Project #2 -->
    <cb:ProjectInclude ccIncludePath="\Project2\project2.xml"/>
</cruisecontrol>

As you can see, we embedded the common, repeated, part of the 'include' definition in it's own defined block, and then use that to effect each include using the path as a parameter - the idea being that future modifiers of the file won't have the opportunity to accidentally forget something on their new included project lines (such as the preprocessor URN); as long as their xml file exists and they get the path to it right, the rest is taken care of in the common include definition.

The only problem is that this doesn't work - for some reason, it looks like the globals.xml file isn't being parsed properly (or maybe even included) because the projects included after it complain of not having elements defined; that is, references to elements defined in the globals file don't appear to have been 'registered', because the projects don't recognise them.

We've tried taking out the nested includes from globals.xml and including them directly at the top level, to no avail. Commenting-out the first troublesome element reference in the project just causes Validator to complain about the next one, with message "Preprocessing failed loading the XML: Reference to unknown symbol XXXXX". If we embed the body of globals.xml into ccnet.config, however, this works. Bizarre though it may sound, it's as though the preprocessor is utterly failing to parse globals.xml but then happily chewing through the project files, only to then fail because the global references aren't defined.

Validator is failing silently if this is the case, however. And of course because it can't parse the project XML properly, we get nothing in the 'Original' or 'Processed' tabs either. The CruiseControl.NET Service itself fails to start with an unhelpful exception:

Service cannot be started. System.Runtime.Serialization.SerializationException: Type 'ThoughtWorks.CruiseControl.Core.Config.Preprocessor.EvaluationException' in Assembly 'ThoughtWorks.CruiseControl.Core, Version=1.4.4.83, Culture=neutral, PublicKeyToken=null' is not marked as serializable. Server stack trace: at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter) at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.Serialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Ser...

All the documentation says this should work, and there's no mention of any incompatibility or inconsistency when using 'include' inside a 'define'. So I'm stumped, and any insight or advice would, at this stage, be highly regarded.

+1  A: 

Hi

we're working on solving this issue for the 1.5 release (hopefully end this month) http://jira.public.thoughtworks.org/browse/CCNET-1865

with kind regards Ruben Willems

Williams
Hmm - this issue (CCNET-1865) relates to CCNet 1.5.0 RC1, specifically a recent 1.5.x build (i.e. it wasn't an issue prior to that recent build). So does the solution for CCNET-1865 address a much deeper issue that has existed since 1.4.4.83 (the version I'm running) and that has recently been exacerbated in a 1.5.x build? More importantly, are you stating that in order for the 'include' mechanism to work as expected, I'll need to upgrade my 1.4.4.83 install to a bleeding-edge 1.5 RC? There will be no regression patch for the 1.4.4.83 release?
Jonners
HiThe 1.4.4 has been discontinued, so there will be no patches done.Can you send me your config files, and I'll check if the 1.5 parses it correctly. Why do you not want to upgrade to 1.5?it does have a lot of new features,and in the current trunk (v1.6) we have a different engine for the pre-processor (based on LINQ), so updates the the 1.4.4 branch willeven be less likely.
Williams
Um, why is 1.4.4 discontinued?? It's the latest 'production' release - 1.5 only has CTP and RC1 releases, and is therefore not at its final 'RTM' release yet! Which is precisely why I'm not upgrading my 1.4.4 servers to 1.5, because I have around 50 or so projects building on 1.4.4 and a bunch of developers who would hang me by my toes out of a top floor window if I just plopped a CTP or RC1 build of CC.Net on their C.I. servers! The config that fails is exactly what I posted in the question - an 'include' inside a 'define' is parsed anomalously.
Jonners
I'll see what I can do, no promisses though ;-)I'm also still running the 1.4.4 in production, did not have the time for an update. The 1.5 final release is scheduled for end this month.
Williams
OK, I can live with that - a patch for 1.4.4, or an official 1.5 RTM gets me a result I can use to justify a maintenance slot on the C.I. VMs. I realise this isn't a 'critical' issue, but I've spent a lot of time educating the crew about splitting ccnet.config into smaller manageable chunks, and this problem makes it harder to justify 'simplicity' and 'reusability' as arguments. ;)
Jonners
A: 

CCnet 1.5 Final has been released, so this gives you the excuse to test it:-) If the problem still persists, you can also contact me via the ccnet lists. http://groups.google.com.ag/group/ccnet-user or http://groups.google.com.ag/group/ccnet-devel

Williams
I'll give it a go later this week on a test VM, and throw any feedback to the Google Group. :)
Jonners