Tuesday, March 27, 2012

Bulk load doesn't preserve element order?

Hi,
I have an interesing issue with sqlxml bulk load. It's about the order in which bulk load inserts rows in the database. I assumed that the rows should be inserted in the same order as the elements appear in the xml file, but looks like it is not always the case.
We use SQL server 2000 and SQLXML 3.0 SP 3. The files are large(2GB) and look something like this:
<Betalingskrav>
<Krav>
<BetalingsKravStatus><![CDATA[1]]></BetalingsKravStatus>
<FakturaDato><![CDATA[04.02.2006]]></FakturaDato>
<CustomerAgreementReference>
<![CDATA[12345689]]></CustomerAgreementReference>
....
<Samtaler>
<Samtale_Gruppeheader>
<SamtaleType><![CDATA[Mobilsamtaler]]></SamtaleType>
<Benevning><![CDATA[Abonnentnr]]></Benevning>
<Telefonnr><![CDATA[12 34 56 78]]></Telefonnr>
<Periode><![CDATA[Periode03.01.06-02.02.06]]></Periode>
<Etikett_Dest><![CDATA[Destinasjon/Operator]]></Etikett_Dest>
<Etikett_Oppr_nr><![CDATA[Oppringt nummer]]></Etikett_Oppr_nr>
<Etikett_SamtDato><![CDATA[Dato]]></Etikett_SamtDato>
<Etikett_Samtstart><![CDATA[Start]]></Etikett_Samtstart>
<Etikett_lngd><![CDATA[Samtalelengde]]></Etikett_lngd>
<Etikett_Kost><![CDATA[Kostnad]]></Etikett_Kost>
</Samtale_Gruppeheader>
<Samtale_Data>
<Dta_Dest><![CDATA[Tele2 GSM]]></Dta_Dest>
<Dta_Oppr_nr><![CDATA[12345678]]></Dta_Oppr_nr>
<Dta_SamtDato><![CDATA[02.01.06]]></Dta_SamtDato>
<Dta_Samtsstart><![CDATA[11:13]]></Dta_Samtsstart>
<Dta_lngd><![CDATA[ 0:00:37]]></Dta_lngd>
<Dta_Kost><![CDATA[0.81]]></Dta_Kost>
</Samtale_Data>
<Samtale_Data>
....
</Samtale_Data>
...
</Samtaler>
</Krav>
...
</Betalingskrav>

The schema (i list a shortened version):
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn:schemas-microsoft-com:mapping-schema">
<xsd:annotation>
<xsd:appinfo>
<sql:relationship name="Krav_T2KravLinje"
parent="Betalingskrav"
parent-key="BetalingskravID"
child="T2BetalingskravLinje"
child-key="BetalingskravID" />
</xsd:appinfo>
</xsd:annotation>
<xsd:complexType name="KravType">
<xsd:all>
<xsd:element name="BetalerID" default="9999" />
<xsd:element name="FakturaTypeID" default="0" />
<xsd:element name="MalID" default="0" />
<xsd:element name="BuntID" default="0" />
<xsd:element name="UtstederID" default="28" />
...
<xsd:element name="Samtaler" sql:is-constant="1" >
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Samtale_Gruppeheader" type="Samtale_GruppeheaderType"
sql:relation="T2BetalingskravLinje" sql:relationship="Krav_T2KravLinje" />
<xsd:element name="Samtale_Data" type="Samtale_DataType"
sql:relation="T2BetalingskravLinje" sql:relationship="Krav_T2KravLinje"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="MVATekst1" type="xsd:string" />
<xsd:element name="MVATekst2" type="xsd:string" />
...
</xsd:all>
</xsd:complexType>
<xsd:complexType name="Samtale_GruppeheaderType">
<xsd:sequence>
<xsd:element name="SamtaleType" type="xsd:string" />
<xsd:element name="Benevning" type="xsd:string" />
<xsd:element name="Telefonnr" type="xsd:string" />
<xsd:element name="Periode" type="xsd:string" />
<xsd:element name="Etikett_Dest" type="xsd:string" />
<xsd:element name="Etikett_Oppr_nr" type="xsd:string" />
<xsd:element name="Etikett_Samtant" type="xsd:string" />
<xsd:element name="Etikett_SamtDato" type="xsd:string" />
<xsd:element name="Etikett_Samtstart" type="xsd:string" />
<xsd:element name="Etikett_lngd" type="xsd:string" />
<xsd:element name="Etikett_Kost" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="Samtale_DataType">
<xsd:sequence>
<xsd:element name="Dta_Dest" type="xsd:string" />
<xsd:element name="Dta_Oppr_nr" type="xsd:string" />
<xsd:element name="Dta_Samtant" type="xsd:string" />
<xsd:element name="Dta_SamtDato" type="xsd:string" />
<xsd:element name="Dta_Samtsstart" type="xsd:string" />
<xsd:element name="Dta_lngd" type="xsd:string" />
<xsd:element name="Dta_Kost" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
<xsd:element name="Betalingskrav" sql:is-constant="1">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Krav" type="KravType" sql:relation="Betalingskrav" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
XML data represents telephone bills with a listing of all calls, sms's, data services and so on. Samtale_Gruppheader elements define headers, and Samtale_Data elements contain the items that go below the headings. The point is that the sequence of elements inside <Samtaler> must be preserved in the database via identity column to provide correct formatting of bills. Not a good approach, I understand, but we are dealing with a legacy system that is not easy to modify...
The shema above maps xml data to two tables in the database that have primary-foreign relationship. The foreign-key table(T2BetalingskravLinje) contains both contents of Samtale_Gruppeheader and Samtale_Data elements. When the bills are rendered, rows in this table are shown in the sequence given by the identity column, in other words in the order they were inserted into the table.
KeepIdentity property is set to false on the bulk load component, the SQL server generates identity keys itself. This works fine, foreign keys are propagated correctly. But the order of rows is not the same as the order of elements in <Samtaler>! Not all the times, but in about 5 to 10% of the cases and I don't see any pattern here. I extracted a <krav> element that had this problem from a real file, and tried to bulk load the resulting short file. The effect disappered, rows were in correct order...
So the question is: Is this behaiviour normal for bulk load component and the order of row insertion into foreign-key tables is not guaranteed to be the same as the order of elements in the xml file? Is it a bug or a feature? :-) Is there any way to enable order preservation?
Thanks in advance for any ideas or suggestions!

The bulkloading component on which the SQLXML bulkloader is built does not guarantee order. So the behaviour is normal.

Best regards

Michael

sql

No comments:

Post a Comment