|
Home > Archive > MS SQL Server DTS > January 2006 > SSIS - Sorting an XML Source & Memory Consumption
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
SSIS - Sorting an XML Source & Memory Consumption
|
|
|
| Hi there,
I have a large XML document (100Mg) that I want to process through a data
flow process.
Part of the process is to Sort the various relational tables created by the
XML Source - then merge join them.
The Sorts consume all the available memory and the process grinds to a halt.
However, if I source the same data from SQL tables, the sorts don't consume
anywhere near as much memory.
So a couple of questions:
1. Is there a difference between the way SSIS sorts XML Source Data and the
way it sorts OLE DB Source Data?
2. Are there any techniques/best practices when dealing with large XML Files?
Thanks,
Dave.
| |
| fischer@sofika.de 2006-01-28, 11:23 am |
| Hi Dave,
the product <xml>cmp can very performantly sort, merge and compare
large xml-files with low memory-consumption (http://www.xmlcmp.com).
Klaus
|
|
|
|
|