Selecting unique records in XSLT/XPath

Question

I have to select only unique records from an XML document, in the context of an <xsl:for-each> loop. I am limited by Visual Studio to using XSL 1.0.

    <availList>
        <item>
          <schDate>2010-06-24</schDate>              
          <schFrmTime>10:00:00</schFrmTime>
          <schToTime>13:00:00</schToTime>
          <variousOtherElements></variousOtherElements>
        </item>
        <item>
          <schDate>2010-06-24</schDate>              
          <schFrmTime>10:00:00</schFrmTime>
          <schToTime>13:00:00</schToTime>
          <variousOtherElements></variousOtherElements>
        </item>
        <item>
          <schDate>2010-06-25</schDate>              
          <schFrmTime>10:00:00</schFrmTime>
          <schToTime>12:00:00</schToTime>
          <variousOtherElements></variousOtherElements>
        </item>
        <item>
          <schDate>2010-06-26</schDate>              
          <schFrmTime>13:00:00</schFrmTime>
          <schToTime>14:00:00</schToTime>
          <variousOtherElements></variousOtherElements>
        </item>
        <item>
          <schDate>2010-06-26</schDate>              
          <schFrmTime>10:00:00</schFrmTime>
          <schToTime>12:00:00</schToTime>
          <variousOtherElements></variousOtherElements>
        </item>
    </availList>

The uniqueness must be based on the value of the three child elements: schDate, schFrmTime and schToTime. If two item elements have the same values for all three child elements, they are duplicates. In the above XML, items one and two are duplicates. The rest are unique. As indicated above, each item contains other elements that we do not wish to include in the comparison. 'Uniqueness' should be a factor of those three elements, and those alone.

I have attempted to accomplish this through the following:

availList/item[not(schDate = preceding:: schDate and schFrmTime = preceding:: schFrmTime and schToTime = preceding:: schToTime)]

The idea behind this is to select records where there is no preceding element with the same schDate, schFrmTime and schToTime. However, its output is missing the last item. This is because my XPath is actually excluding items where all of the child element values are matched within the entire preceding document. No single item matches all of the last item's child elements - but because each element's value is individually present in another item, the last item gets excluded.

I could get the correct result by comparing all child values as a concatenated string to the same concatenated values for each preceding item. Does anybody know of a way I could do this?

Answer 1

The technique I've seen is to do this in two passes: sort the items by all three key fields, and then compare each item to its preceding item (instead of all preceding items).

Is it practical for you to run two separate transformations? It makes the problem much easier.

I saw the technique in an older edition of Michael Kay's XSLT book. You might find it in some of his sample code there.

Answer 2

I. As a single XPath expression:

/*/item[normalize-space() and not(. = preceding-sibling::item)]

II. More efficient (XSLT) implementation, using keys:

<xsl:stylesheet version="1.0"
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform"&gt;
 <xsl:output omit-xml-declaration="yes" indent="yes"/>
 <xsl:strip-space elements="*"/>

 <xsl:key name="kItemByVal" match="item" use="."/>

 <xsl:template match="/">
  <xsl:copy-of select=
   "*/item[generate-id() = generate-id(key('kItemByVal', .))]
   "/>
 </xsl:template>
</xsl:stylesheet>

Both I and II, when applied on the provided XML document correctly select/copy the following nodes:

<item><schDate>2010-06-24</schDate><schFrmTime>10:00:00</schFrmTime><schToTime>13:00:00</schToTime></item>
<item><schDate>2010-06-25</schDate><schFrmTime>10:00:00</schFrmTime><schToTime>12:00:00</schToTime></item>
<item><schDate>2010-06-26</schDate><schFrmTime>13:00:00</schFrmTime><schToTime>14:00:00</schToTime></item>
<item><schDate>2010-06-26</schDate><schFrmTime>10:00:00</schFrmTime><schToTime>12:00:00</schToTime></item>

Update: In case <item> has other children, then this transformation:

<xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"&gt;
    <xsl:output omit-xml-declaration="yes" indent="yes"/>

    <xsl:key name="kItemBy3Children" match="item"
     use="concat(schDate, '+', schFrmTime, '+', schToTime)"/>

 <xsl:template match="/">
       <xsl:copy-of select=
        "*/item[generate-id()
              = generate-id(key('kItemBy3Children',
                                concat(schDate,
                                       '+', schFrmTime,
                                       '+', schToTime)
                               )
                            )
               ]
        "/>
 </xsl:template>
</xsl:stylesheet>

produces the wanted result.