SQL Anywhere database - Storage Card Issues

Author

Storage Card Issues - Duplicate database

Donna Osburn

2005-08-17, 8:23 pm

We run our DB on a storage card or secure digital card on
mobile
computers. if we remove the storage card from the unit and
put
in a different one, at time (not consistently) it creates a
duplicate
of the database and then the user cannot log in with a
"Request to
start/stop the database failed". Does anybody have any
experience with this? I know the users are not shutting
down the
application before removing the card. Its almost like
ianywhere
writes what it had in memory back to storage card, thereby
making duplicates of the data.

I am open to suggestions!

Thanks

Donna

John Smirnios

2005-08-18, 9:40 am

What would you like to have happen when the user removes the device
containing a file that is actively in use by the database server?

Certainly, we shouldn't (and can't possibly as far as I can tell) create
a "duplicate" of the database on the second card. Of course, there can
be no accounting for what the OS does under our feet.

The server does attempt to "reopen" database files on media that has
been ejected & reinserted (because the OS often closes the files
automatically) but it looks only for an existing file then verifies that
the volume & file serial numbers match the previous file that was opened.

If the server cannot reopen the file, a message box will eventually be
displayed (about 10 seconds after the first IO attempt after the card
was removed) telling the user to reinsert the card. If the user does not
reinsert the card (ie. hits 'cancel' on the message box), the server
will go down with an assertion failure reporting an IO failure. You just
can't run a database without the database file...

-john.

--
John Smirnios
Senior Software Developer
iAnywhere Solutions Engineering

Whitepapers, TechDocs, bug fixes are all available through the iAnywhere
Developer Community at http://www.ianywhere.com/developer

Donna Osburn wrote:
> We run our DB on a storage card or secure digital card on
> mobile
> computers. if we remove the storage card from the unit and
> put
> in a different one, at time (not consistently) it creates a
> duplicate
> of the database and then the user cannot log in with a
> "Request to
> start/stop the database failed". Does anybody have any
> experience with this? I know the users are not shutting
> down the
> application before removing the card. Its almost like
> ianywhere
> writes what it had in memory back to storage card, thereby
> making duplicates of the data.
>
> I am open to suggestions!
>
> Thanks
>
> Donna

Donna Osburn

2005-08-18, 9:40 am

As to what i would expect to happen, the unit was powered
down in order to remove the media (because it's under the
battery inside the unit). So the power down functionality
that has always worked is simply leave database alone.
when the unit is powered back on and warm booted - what has
always happened is database is restarted by applicaiton and
moves on.
For some reason - when we warm boot the device - we now have
two physical copies of the database on the storage card -
one being 0 bytes in length.

I find this odd. I'm trying to determine if this is normal
behavior by ianywhere to fix itself. If it is - is there a
way to turn it off.
The good answer would be for us to have user shut down the
database before removing the card. Its not that simple.
They are locked to only our application and cannot exit it
without master user level passwords.

Donna
[color=darkred]
> What would you like to have happen when the user removes
> the device containing a file that is actively in use by
> the database server?
>
> Certainly, we shouldn't (and can't possibly as far as I
> can tell) create a "duplicate" of the database on the
> second card. Of course, there can be no accounting for
> what the OS does under our feet.
>
> The server does attempt to "reopen" database files on
> media that has been ejected & reinserted (because the OS
> often closes the files automatically) but it looks only
> for an existing file then verifies that the volume & file
> serial numbers match the previous file that was opened.
>
> If the server cannot reopen the file, a message box will
> eventually be displayed (about 10 seconds after the first
> IO attempt after the card was removed) telling the user
> to reinsert the card. If the user does not reinsert the
> card (ie. hits 'cancel' on the message box), the server
> will go down with an assertion failure reporting an IO
> failure. You just can't run a database without the
> database file...
>
> -john.
>
> --
> John Smirnios
> Senior Software Developer
> iAnywhere Solutions Engineering
>
> Whitepapers, TechDocs, bug fixes are all available through
> the iAnywhere Developer Community at
> http://www.ianywhere.com/developer
>
> Donna Osburn wrote:

Donna Osburn

2005-08-18, 8:23 pm

Its also important to note this only happens on the Windows
CE devices. We don't have the same issue on the Pocket PC
Devices. IAnywhere has had issues with the CE OS in the
past. I would assume this is another case of them not
handling something correctly under the hood on the CE OS.
Donna[color=darkred]

> What would you like to have happen when the user removes
> the device containing a file that is actively in use by
> the database server?
>
> Certainly, we shouldn't (and can't possibly as far as I
> can tell) create a "duplicate" of the database on the
> second card. Of course, there can be no accounting for
> what the OS does under our feet.
>
> The server does attempt to "reopen" database files on
> media that has been ejected & reinserted (because the OS
> often closes the files automatically) but it looks only
> for an existing file then verifies that the volume & file
> serial numbers match the previous file that was opened.
>
> If the server cannot reopen the file, a message box will
> eventually be displayed (about 10 seconds after the first
> IO attempt after the card was removed) telling the user
> to reinsert the card. If the user does not reinsert the
> card (ie. hits 'cancel' on the message box), the server
> will go down with an assertion failure reporting an IO
> failure. You just can't run a database without the
> database file...
>
> -john.
>
> --
> John Smirnios
> Senior Software Developer
> iAnywhere Solutions Engineering
>
> Whitepapers, TechDocs, bug fixes are all available through
> the iAnywhere Developer Community at
> http://www.ianywhere.com/developer
>
> Donna Osburn wrote:

John Smirnios

2005-08-22, 8:23 pm

Sadly, CE & Pocket PCs do not handle ejection of disks gracefully. When
you turn the power off, the system suspends immediately. Cleanup
actually takes place when power is restored. The OS has bugs where IOs
pending (even by the filesystem itself) at the time the power is turned
off can be written to the card when power is restored -- before the OS
even realizes that there is a different card in there!

The OS logically ejects & reinserts (unmounts & mounts) removable
filesystems when power is restored.

It is very possible for the following sequence to occur:

1. Applications & the filesystem (e.g. FAT) are writing to the flash card
2. Power is turned off
3. The card is swapped
4. Power is restored
5. IOs started by applications & the OS in #1 go out to the new card.
Even new IOs can go out to the card successfully!
6. The OS logically ejects the card
7. IOs start failing at this point (and it is where ASA automatically
does retries).
8. The OS logically reinserts the card
9. IOs start to work again. If the card has changed or if a certain time
has elapsed, file handles are closed to prevent IOs to the new card. ASA
will attempt to reopen the file.

#5 is a gaping whole in the OS. MS is working on plugging that whole and
numerous others but they exist. The only thing that can be done is to
shut down anything that may be accessing the cards before powering off.

#5 is made worse because of another CE OS bug. If a file on a card ever
grows while it is open, CE stops respecting the FILE_FLAG_WRITE_THRO
UGH
flag and FlushFileBuffers() no longer works either. That means there is
no way for an application to force data out to a disk when necessary. A
workaround will be implemented in ASA shortly (we will close & reopen
the file every time we grow it). In the meantime, try to avoid file
growth by pre-growing you database files with "ALTER DBSPACE ADD".

-john.

--
John Smirnios
Senior Software Developer
iAnywhere Solutions Engineering

Whitepapers, TechDocs, bug fixes are all available through the iAnywhere
Developer Community at http://www.ianywhere.com/developer

Donna Osburn wrote:[color=darkred
]
> As to what i would expect to happen, the unit was powered
> down in order to remove the media (because it's under the
> battery inside the unit). So the power down functionality
> that has always worked is simply leave database alone.
> when the unit is powered back on and warm booted - what has
> always happened is database is restarted by applicaiton and
> moves on.
> For some reason - when we warm boot the device - we now have
> two physical copies of the database on the storage card -
> one being 0 bytes in length.
>
> I find this odd. I'm trying to determine if this is normal
> behavior by ianywhere to fix itself. If it is - is there a
> way to turn it off.
> The good answer would be for us to have user shut down the
> database before removing the card. Its not that simple.
> They are locked to only our application and cannot exit it
> without master user level passwords.
>
> Donna
>
>

Donna Osburn

2005-08-23, 11:24 am

Thanks for the update. I'll watch for the ebf.
Donna[color=darkred]

> Sadly, CE & Pocket PCs do not handle ejection of disks
> gracefully. When you turn the power off, the system
> suspends immediately. Cleanup actually takes place when
> power is restored. The OS has bugs where IOs pending
> (even by the filesystem itself) at the time the power is
> turned off can be written to the card when power is
> restored -- before the OS even realizes that there is a
> different card in there!
>
> The OS logically ejects & reinserts (unmounts & mounts)
> removable filesystems when power is restored.
>
> It is very possible for the following sequence to occur:
>
> 1. Applications & the filesystem (e.g. FAT) are writing to
> the flash card 2. Power is turned off
> 3. The card is swapped
> 4. Power is restored
> 5. IOs started by applications & the OS in #1 go out to
> the new card. Even new IOs can go out to the card
> successfully! 6. The OS logically ejects the card
> 7. IOs start failing at this point (and it is where ASA
> automatically does retries).
> 8. The OS logically reinserts the card
> 9. IOs start to work again. If the card has changed or if
> a certain time has elapsed, file handles are closed to
> prevent IOs to the new card. ASA will attempt to reopen
> the file.
>
> #5 is a gaping whole in the OS. MS is working on plugging
> that whole and numerous others but they exist. The only
> thing that can be done is to shut down anything that may
> be accessing the cards before powering off.
>
> #5 is made worse because of another CE OS bug. If a file
> on a card ever grows while it is open, CE stops
> respecting the FILE_FLAG_WRITE_THRO
UGH flag and
> FlushFileBuffers() no longer works either. That means
> there is no way for an application to force data out to a
> disk when necessary. A workaround will be implemented in
> ASA shortly (we will close & reopen the file every time
> we grow it). In the meantime, try to avoid file growth by
> pre-growing you database files with "ALTER DBSPACE ADD".
>
> -john.
>
> --
> John Smirnios
> Senior Software Developer
> iAnywhere Solutions Engineering
>
> Whitepapers, TechDocs, bug fixes are all available through
> the iAnywhere Developer Community at
> http://www.ianywhere.com/developer
>
> Donna Osburn wrote:
> OS >>often closes the files automatically) but it looks
> only >>for an existing file then verifies that the volume
> & file >>serial numbers match the previous file that was
> opened. >>
> first >>IO attempt after the card was removed) telling
> the user >>to reinsert the card. If the user does not
> reinsert the >>card (ie. hits 'cancel' on the message box)
> , the server >>will go down with an assertion failure
> reporting an IO >>failure. You just can't run a database
> without the >>database file...
> through >>the iAnywhere Developer Community at
> unit >>>and put
> shutting >>>down the

Sponsored Links