| Author |
XML bulk load can't load a huge file
|
|
| techy 2005-07-21, 11:24 am |
| We are trying to load XML file that is more than 500 MB.
Bulk load seems to work fine for files less then 500 MB but exits with non
specific error for anything bigger then 500 MB.
Did anybody experience the same behavior? I can't find any information on
XML source file size limitation for bulk load.
| |
| Michael Rys [MSFT] 2005-07-21, 8:24 pm |
| What options do you have specified?
Thanks
Michael
"techy" <techy@discussions.microsoft.com> wrote in message
news:CD0AA7A7-F5A2-4AAF-895B- 7B9682F59E06@microso
ft.com...
> We are trying to load XML file that is more than 500 MB.
>
> Bulk load seems to work fine for files less then 500 MB but exits with non
> specific error for anything bigger then 500 MB.
>
> Did anybody experience the same behavior? I can't find any information on
> XML source file size limitation for bulk load.
>
| |
|
| We are setting couple of parameters. You can see what we set from the actual
code below:
‘Instance
Set objBL = CreateObject(“SQLX
MLBulkLoad.SQLXMLBulkLoad.3.0”)
‘Parameters
objBL.ConnectionString =
“Provider=SQLOLEDB
;Server=(local);data
base=ADS_CONTROL;Tru
sted_Connection=yes;
Integrated Security=SSPI”
objBL.ErrorLogFile = ERROR_LOG_FILE_NAME ‘(“F:\ADSFeed\AD
S_Error.log”)
objBL.XMLFragment = True
‘Execution
objBL.Execute XSD_FILE_NAME, strXMLFileName
‘Cleanup
set objBL = Nothing
"Michael Rys [MSFT]" wrote:
> What options do you have specified?
>
> Thanks
> Michael
>
> "techy" <techy@discussions.microsoft.com> wrote in message
> news:CD0AA7A7-F5A2-4AAF-895B- 7B9682F59E06@microso
ft.com...
>
>
>
| |
| Bertan ARI [MSFT] 2005-07-25, 8:29 pm |
| Are you using something earlier than SqlXml3 Sp3? Before SqlXml3 SP3, there
was a bug in Bulkload prevents loading huge (>500 MB) Xml documents. It was
causing memory exhaustion and this was fixed in SqlXml3 SP3. This could be
the problem you are encountering.
--
Bertan ARI
This posting is provided "AS IS" with no warranties, and confers no rights.
"techy" <techy@discussions.microsoft.com> wrote in message
news:FA39A18D-80B7-4BBF-8C60- 3E92D17B3C65@microso
ft.com...[color=darkred]
> We are setting couple of parameters. You can see what we set from the
> actual
> code below:
>
> 'Instance
>
> Set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkLoad.3.0")
>
>
> 'Parameters
>
> objBL.ConnectionString =
> " Provider=SQLOLEDB;Se
rver=(local);databas
e=ADS_CONTROL;Truste
d_Connection=yes;Int
egrated
> Security=SSPI"
>
> objBL.ErrorLogFile = ERROR_LOG_FILE_NAME '(" F:\ADSFeed\ADS_Error
.log")
>
> objBL.XMLFragment = True
>
>
> 'Execution
>
> objBL.Execute XSD_FILE_NAME, strXMLFileName
>
>
> 'Cleanup
>
> set objBL = Nothing
>
>
> "Michael Rys [MSFT]" wrote:
>
| |
|
| We are using SQLXML3.0 SP3 (which I believe is the latest version) and we
have this problem.
Could you please reffer me to any posting related to this problem in any
version of SQLXML?
Did you try to load files more that 500 MB with SQLXML3.0 SP3?
Thanks
"Bertan ARI [MSFT]" wrote:
> Are you using something earlier than SqlXml3 Sp3? Before SqlXml3 SP3, there
> was a bug in Bulkload prevents loading huge (>500 MB) Xml documents. It was
> causing memory exhaustion and this was fixed in SqlXml3 SP3. This could be
> the problem you are encountering.
>
> --
> Bertan ARI
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
>
>
> "techy" <techy@discussions.microsoft.com> wrote in message
> news:FA39A18D-80B7-4BBF-8C60- 3E92D17B3C65@microso
ft.com...
>
>
>
| |
| Bertan ARI [MSFT] 2005-07-26, 1:24 pm |
| Yes, we tried bulkloading files up to 2 GB so 500MB should work. I don't
know any posting related to this issue and I am not sure if there is any.
Perhaps, we are dealing with a different issue. Could you provide me the
following information?
- What is the error message if there is any?
- Could you track the memory usage by using task manager and verify that if
it is constant during the execution?
- Are there any rows inserted in the database?
Thanks.
--
Bertan ARI
This posting is provided "AS IS" with no warranties, and confers no rights.
"techy" <techy@discussions.microsoft.com> wrote in message
news:24A2A353-861B-48EF-9949- B6EF87FFCF28@microso
ft.com...[color=darkred]
> We are using SQLXML3.0 SP3 (which I believe is the latest version) and we
> have this problem.
> Could you please reffer me to any posting related to this problem in any
> version of SQLXML?
>
> Did you try to load files more that 500 MB with SQLXML3.0 SP3?
>
> Thanks
>
> "Bertan ARI [MSFT]" wrote:
>
| |
|
| The error message is logged into error file and is as follows:
<?xml version="1.0"?>
<Result
State="FAILED">
<Error><HResult>0x8007000E</HResult>
<Description>General operational error</Description>
<Source><![CDATA[]]></Source><Type>FATAL</Type>
</Error>
</Result>
VB script comes with the message :"General Operational Error 0x8007000E"
The memory grows up all the time during the run. It reaches approximately
2GB before load exits with an error. We have 8GB of RAM on this server so I
don't think memory is an issue.
When bulk load exits there is nothing inserted into the target table because
it rolls back. With successful load we have rows inserted.
Thank for you reply. Since you are using bulk load for 2GB files we will
keep investigatiing on our problem.
Please let us know if you have any ideas on this issue.
"Bertan ARI [MSFT]" wrote:
> Yes, we tried bulkloading files up to 2 GB so 500MB should work. I don't
> know any posting related to this issue and I am not sure if there is any.
>
> Perhaps, we are dealing with a different issue. Could you provide me the
> following information?
> - What is the error message if there is any?
> - Could you track the memory usage by using task manager and verify that if
> it is constant during the execution?
> - Are there any rows inserted in the database?
>
> Thanks.
> --
> Bertan ARI
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
>
>
> "techy" <techy@discussions.microsoft.com> wrote in message
> news:24A2A353-861B-48EF-9949- B6EF87FFCF28@microso
ft.com...
>
>
>
| |
| Bertan ARI [MSFT] 2005-07-26, 8:24 pm |
| 2 GB memory usage is too much for Bulkload. Is it possible for me to work
with you for your investigation?
If I can get your data, xml and relational schema, I can debug the process
here. Let me know if this is possible.
--
Bertan ARI
This posting is provided "AS IS" with no warranties, and confers no rights.
"techy" <techy@discussions.microsoft.com> wrote in message
news:E5D24F12-3D3E-4888-A487- C80B153F94BE@microso
ft.com...[color=darkred]
> The error message is logged into error file and is as follows:
> <?xml version="1.0"?>
> <Result
> State="FAILED">
> <Error><HResult>0x8007000E</HResult>
> <Description>General operational error</Description>
> <Source><![CDATA[]]></Source><Type>FATAL</Type>
> </Error>
> </Result>
>
> VB script comes with the message :"General Operational Error 0x8007000E"
>
> The memory grows up all the time during the run. It reaches approximately
> 2GB before load exits with an error. We have 8GB of RAM on this server so
> I
> don't think memory is an issue.
>
> When bulk load exits there is nothing inserted into the target table
> because
> it rolls back. With successful load we have rows inserted.
>
> Thank for you reply. Since you are using bulk load for 2GB files we will
> keep investigatiing on our problem.
>
> Please let us know if you have any ideas on this issue.
>
>
> "Bertan ARI [MSFT]" wrote:
>
|
|
|
|