Merge two XML streams using XSLT matching on key values

Question

Hello,

I'm new to XSLT and I would like to know how to merge 2 XML streams before finally transforming it again.

The 2 streams are

Input 1

<Response>
    <Instrument>
        <Date value="2010-09-02">
            <Quantity>10</Quantity>
        </Date>
        <DXLID>1</DXLID>
    </Instrument>
    <Instrument TICKER="APPL" />
    <SF></SF>
    <Instrument>
        <Date value="2010-09-02">
            <Quantity>20</Quantity>
        </Date>
        <DXLID>2</DXLID>
    </Instrument>
    <Instrument TICKER="APPL" />
    <SF></SF>
</Response>

Input 2

<Response>
    <IM>
        <Instrument>
            <Date value="2010-09-02">
                <SAF>1</SAF>
                <SAR>2</SAR>
            </Date>
            <DXLID>1</DXLID>
        </Instrument>
        <Instrument>
            <Date value="2010-09-02">
                <SAF>1</SAF>
                <SAR>2</SAR>
            </Date>
            <DXLID>3</DXLID>
        </Instrument>
    </IM>
</Response>

Desired Output

<Response>
    <All>
        <Instrument>
            <Date value="2010-09-02">
                <SAF>1</SAF>
                <SAR>2</SAR>
                <Quantity>10</Quantity>
            </Date>
            <DXLID>1</DXLID>
        </Instrument>
        <Instrument>
            <Date value="2010-09-02">
                <Quantity>20</Quantity>
            </Date>
            <DXLID>2</DXLID>
        </Instrument>
        <Instrument>
            <Date value="2010-09-02">
                <SAF>1</SAF>
                <SAR>2</SAR>
            </Date>
            <DXLID>3</DXLID>
        </Instrument>
    </All>
</Response>

The merge needs to be based on a match between the DXLID node value and the value attribute of the Date node.

Note also that the merge needs to be a merge both ways.

Thanks in advance

Neil

Answer 1

This stylesheet:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"&gt;
    <xsl:key name="kInstrumentByDateAndDXLID" match="Instrument"
             use="concat(Date/@value,'++',DXLID)"/>
    <xsl:variable name="vSource1"
               select="document('Doc1.xml')/Response/Instrument[Date][DXLID]"/>
    <xsl:variable name="vSource2"
               select="document('Doc2.xml')/Response/IM/Instrument"/>
    <xsl:template match="node()|@*" name="identity">
        <xsl:copy>
            <xsl:apply-templates select="node()|@*"/>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="/">
        <Response>
            <All>
                <xsl:apply-templates select="$vSource1|$vSource2">
                    <xsl:sort select="DXLID"/>
                </xsl:apply-templates>
            </All>
        </Response>
    </xsl:template>
    <xsl:template match="Date/*[last()]">
        <xsl:call-template name="identity"/>
        <xsl:if test="count(../..|$vSource1)=count($vSource1)">
            <xsl:variable name="vKey"
                          select="concat(../@value,'++',../../DXLID)"/>
            <xsl:for-each select="$vSource2[last()]">
                <xsl:apply-templates
                 select="key('kInstrumentByDateAndDXLID',$vKey)/Date/*"/>
            </xsl:for-each>
        </xsl:if>
    </xsl:template>
    <xsl:template match="Instrument">
        <xsl:if test="count(.|$vSource1)=count($vSource1) or
                      not($vSource1[key('kInstrumentByDateAndDXLID',
                                        concat(current()/Date/@value,'++',
                                               current()/DXLID))])">
            <xsl:call-template name="identity"/>
        </xsl:if>
    </xsl:template>
</xsl:stylesheet>

Output:

<Response>
    <All>
        <Instrument>
            <Date value="2010-09-02">
                <Quantity>10</Quantity>
                <SAF>1</SAF>
                <SAR>2</SAR>
            </Date>
            <DXLID>1</DXLID>
        </Instrument>
        <Instrument>
            <Date value="2010-09-02">
                <Quantity>20</Quantity>
            </Date>
            <DXLID>2</DXLID>
        </Instrument>
        <Instrument>
            <Date value="2010-09-02">
                <SAF>1</SAF>
                <SAR>2</SAR>
            </Date>
            <DXLID>3</DXLID>
        </Instrument>
    </All>
</Response>

Note: The use of apply-templates allows to run the merge and second step transformation at once. Also, the use of fn:document for multiple input source, and XPath test for inclusion: count($node|$node-set)=count($node-set)

Edit: the same with keys. It looks like MSXSL4 has a bug, that's why I'm ussing $vSource2[last()] instead of $vSource2[1]

Answer 2

This transformation (includes the entire second document -- just for convenience):

<xsl:stylesheet version="1.0"
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform"&gt;
 <xsl:output omit-xml-declaration="yes" indent="yes"/>
 <xsl:strip-space elements="*"/>

 <xsl:key name="kDateByValAndId" match="Date"
  use="concat(@value, '+', ../DXLID)"/>

 <xsl:variable name="vDoc1" select="/"/>

 <xsl:variable name="vrtfDoc2">
    <Response>
        <IM>
            <Instrument>
                <Date value="2010-09-02">
                    <SAF>1</SAF>
                    <SAR>2</SAR>
                </Date>
                <DXLID>1</DXLID>
            </Instrument>
            <Instrument>
                <Date value="2010-09-02">
                    <SAF>1</SAF>
                    <SAR>2</SAR>
                </Date>
                <DXLID>3</DXLID>
            </Instrument>
        </IM>
    </Response>
 </xsl:variable>

 <xsl:variable name="vDoc2" select=
  "document('')/*/xsl:variable[@name='vrtfDoc2']"/>

 <xsl:template match="/">
  <Response>
   <All>
    <xsl:apply-templates select="/*/node()"/>
    <xsl:apply-templates mode="doc2" select=
     "$vDoc2/*/*/Instrument[Date and DXLID]" />
   </All>
  </Response>
 </xsl:template>

 <xsl:template match="node()|@*">
     <xsl:copy>
       <xsl:apply-templates select="node()|@*"/>
     </xsl:copy>
 </xsl:template>

 <xsl:template match="Date">
  <xsl:variable name="vkeyVal" select=
  "concat(@value, '+', ../DXLID)"/>

  <xsl:copy>
   <xsl:apply-templates select="node()|@*"/>
   <xsl:for-each select="$vDoc2">
    <xsl:apply-templates select="key('kDateByValAndId', $vkeyVal)/node()"/>
   </xsl:for-each>
  </xsl:copy>
 </xsl:template>
 <xsl:template match="SF|Instrument[@TICKER='APPL']"/>

 <xsl:template match="Instrument" mode="doc2">
  <xsl:variable name="vkeyVal" select=
  "concat(Date/@value, '+', DXLID)"/>

  <xsl:variable name="vcur" select="."/>

  <xsl:for-each select="$vDoc1">
   <xsl:if test="not(key('kDateByValAndId', $vkeyVal))">
    <xsl:copy-of select="$vcur"/>
   </xsl:if>
  </xsl:for-each>

 </xsl:template>
</xsl:stylesheet>

when applied on the original 1st document:

<Response>
    <Instrument>
        <Date value="2010-09-02">
            <Quantity>10</Quantity>
        </Date>
        <DXLID>1</DXLID>
    </Instrument>
    <Instrument TICKER="APPL" />
    <SF></SF>
    <Instrument>
        <Date value="2010-09-02">
            <Quantity>20</Quantity>
        </Date>
        <DXLID>2</DXLID>
    </Instrument>
    <Instrument TICKER="APPL" />
    <SF></SF>
</Response>

produces the wanted, correct result:

<Response>
   <All>
      <Instrument>
         <Date value="2010-09-02">
            <Quantity>10</Quantity>
            <SAF xmlns:xsl="http://www.w3.org/1999/XSL/Transform"&gt;1&lt;/SAF&gt;
            <SAR xmlns:xsl="http://www.w3.org/1999/XSL/Transform"&gt;2&lt;/SAR&gt;
         </Date>
         <DXLID>1</DXLID>
      </Instrument>
      <Instrument>
         <Date value="2010-09-02">
            <Quantity>20</Quantity>
         </Date>
         <DXLID>2</DXLID>
      </Instrument>
      <Instrument xmlns:xsl="http://www.w3.org/1999/XSL/Transform"&gt;
         <Date value="2010-09-02">
            <SAF>1</SAF>
            <SAR>2</SAR>
         </Date>
         <DXLID>3</DXLID>
      </Instrument>
   </All>
</Response>

Do note:

1. The namespace nodes will not appear if the second document is in its own file -- it is included in the current transformation just for convenience.

2. All nodes of the first document are procesed first (if some of them have matching nodes in the second document, the merge is performed).

3. Finally, all <Instrument> nodes from the second document that do not have corresponding <Instrument> nodes from the 1st documents, are copied into the output.

4. Matching Date nodes from the two documents are identified using keys.