Programming with dBASE - Replace cr/lf with evaltags

Author

Replace cr/lf with evaltags 

Ruud van den Bosch

2005-12-13, 3:23 am

I've imported some memo fields and in these fields are cr/lf but not the
evaltags dbase uses (like etc)

I need to combine the contents of these memo fields with some other
memofields out of a table. However in this table the memofields are saved
with evaltags.

When I combine the fields, the CR/LF will disappear. This is not what I
want.

How can I replace the CR/LF with evaltags ????

*Lysander*

2005-12-13, 7:23 am

In article <gatiYU8$FHA.1228@news-server>, ruud@derckx.nl says...

> How can I replace the CR/LF with evaltags ????

evaltags is the name of the property, and I think it stands for=20
"evaluate tags".

I don't know even, which are the correct tags.
I suppose for linebreak and for paragraph.

In general, memos are big text fields (long textfields).
So you could scan them character by character for chr(10)+chr(13), which=20
is the representation of CR/LF.

Look in the string functions of dBL how to replace a substring inmidst=20
of any string with another substring. I cannot be of too big help here,=20
because I am not using it this way.

--=20
ciao,
Andr=E9

Mervyn Bick

2005-12-13, 9:23 am

On Tue, 13 Dec 2005 13:08:36 +0200, *Lysander* <nobody@nowhere.com> wrote:

> In article <gatiYU8$FHA.1228@news-server>, ruud@derckx.nl says...
>
>
> evaltags is the name of the property, and I think it stands for
> "evaluate tags".
>
> I don't know even, which are the correct tags.
> I suppose for linebreak and for paragraph.
>
> In general, memos are big text fields (long textfields).
> So you could scan them character by character for chr(10)+chr(13), which
> is the representation of CR/LF.

CR/LF is actually chr(13)+chr(10) and if one looks at a file with a hex
editor such as XVI32 they show up as OD OA in hex.

 and are both HTML tags that can normally safely be used without a
corresponding or tag. I haven't tested this exhaustively but
displaying a memofield on a form with evaltags set true doesn't always
give the expected results with embedded and tags, especially the
 tag. CR'LFs embedded in the memo seem to be translated to simply
show 2 spaces on the screen.

> Look in the string functions of dBL how to replace a substring inmidst
> of any string with another substring. I cannot be of too big help here,
> because I am not using it this way.

Looping through the memofield in memory with substr() doesn't seem to do
the job. It looks as if one would have to do the changes straight into
the disk file using the read() and write() methods of the file class. The
problem is that one needs to replace 2 bytes with 3 (CR/LF with ie ODh
OAh with 3Ch 70h 3Eh) which will upset the memofield length which is
stored in the first byte of the memofield block so this will need to be
looked at as well. Bear in mind that if a memofield is longer than 511
bytes (the first byte of a 512byte block is used as a header) the memo
rolls over into the next block. If the number of tags causes this to
happen then the entire memofield will have to be rewritten and the pointer
in the .dbf file will have to be patched as well as fixing the pointer to
the next empty block in the .dbt file. Not impossible but not something
to be tackled lightly! If no memofield extends over an existing block
boundary (even if the memofield occupies more than one block) after the
tags have been substituted then the whole exercise will be fairly simple.

The format of version 7 .dbt files seems (by a rather cursory inspection)
to still be the same as that for dBase IV. The specs dBase IV .dbt files
can be found at
http://www.clicketyclick.dk/databases/xbase/format/

Mervyn

Michael Nuwer [dBVIPS]

2005-12-13, 11:23 am

The following code will replace CR/LF with "". The code creates a
sample file to work with.

// Create file for Demo
f = new file()
f.create("testFile.txt")
c = replicate("abc ", 10) + chr(13)+ chr(10)
c = replicate( c , 4)
c += chr(13) + chr(10) + chr(13) +chr(10)
c = replicate( c , 2)
c = left(c,c.length-6)
f.write( c )
f.close()
run(true, "notepad.exe testFile.txt")

***** Operative part ********************
******
f.open("testFile.txt")
cStr = f.read(f.size("testFile.txt"))
oRegExp = new OleAutoClient("VBScript.RegExp")
oRegExp.global = true
oRegExp.pattern = "\r\n"
cStr = oRegExp.replace(cStr, "")
********************
********************
********

// View result:
f.close()
f.create("copy.txt")
f.write( cStr )
f.close()
run(true, "notepad.exe copy.txt")

David Kerber

2005-12-13, 11:23 am

In article <op. s1p80fu4380wm4@mervy
nbick>, invalid@invalid.invalid
says...

....

Yes.
[color=darkred]
>
> CR/LF is actually chr(13)+chr(10) and if one looks at a file with a hex
> editor such as XVI32 they show up as OD OA in hex.
>
> and are both HTML tags that can normally safely be used without a
> corresponding or tag. I haven't tested this exhaustively but
> displaying a memofield on a form with evaltags set true doesn't always
> give the expected results with embedded and tags, especially the
> tag. CR'LFs embedded in the memo seem to be translated to simply
> show 2 spaces on the screen.

 doesn't even have a corresponding tag, because it simply
means "Line break", which is equivalent to a hard return (CR/LF) code.
 indicates a paragraph, which puts breaks both before and after the
indicated text. It does have a match, and many browsers will
render things slightly differently depending on whether or not the 
is there. In particular, if you put a without a , or before
the , you may get a different amount of blank space than you get
with the after the .

.....

--
Remove the ns_ from if replying by e-mail (but keep posts in the
newsgroups if possible).

Mervyn Bick

2005-12-13, 8:23 pm

On Tue, 13 Dec 2005 18:12:34 +0200, Michael Nuwer [dBVIPS]
<nuwermj@no.spam.yahoo.com> wrote:

>
> The following code will replace CR/LF with "". The code creates a
> sample file to work with.

As you point out, the code to replace all CR/LF instances with "" in a
text file using the read() and write() methods of the file class is quite
straight forward and is not exactly rocket science.

The problem is that it looks as if Ruud will have to tackle a .dbt file
and not a simple text file. This involves reading and writing the first
512byte block without change and then reading the next block, extracting
the memo length, which may indicate that further blocks need to be read,
and then keeping track of just how many substitutions have been made
before adjusting the length bytes and writing the revised data back to the
output file. All this after checking that the extended data will not
cross a 512 block boundary. Again, this is also not particularly
difficult to code although one must keep in mind that the output file must
have the same name as the .dbf file so the input file will have to be
renamed to something else first.

The hairy part comes when the extended data crosses a block boundary.
Unless it happened to be the very last record it would corrupt tne next
512byte block so then the data has to be written at the end of a file that
hasn't yet been written out. This means one would probably have to run
through the entire input file first checking for entries that would cross
boundaries, keep track of them, write the others and then go back for the
records with these extended fields. This would in turn involve patching
the header of the new output file to show the next vacant block. Having
done that, all that remains is the interesting bit which is to tackle the
..dbf to change the pointers to the .dbt entries for these particular
records. As the .dbt file has no back reference to the actual record in
the .dbf file one would have to scan the .dbf file looking for the old
block number to know where to write the new block number. Still not
rocket science but we're getting there! <g>

Mervyn

Michael Nuwer [dBVIPS]

2005-12-13, 8:23 pm

Could he use something like the following?

use :dbaseSamples:fish
copy to :dbaseSamples:fishTe
st
use

d = new database()
d.databasename = "dbasesamples"
d.active = true
q = new query()
q.database = d
q.sql = 'select * from fishTest'
q.active = true

oRegExp = new OleAutoClient("VBScript.RegExp")
oRegExp.ignoreCase = true
oRegExp.global = true
oRegExp.pattern = "\r\n"

q.rowset.first()
do
cStr = q.rowset.fields['Description'].value
cStr = oRegExp.replace(cStr, "")
q.rowset.fields['Description'].value = cStr
q.rowset.save()
until not q.rowset.next()

q.rowset.first()
do
? q.rowset.fields['Description'].value
until not q.rowset.next()

Mervyn Bick wrote:
> On Tue, 13 Dec 2005 18:12:34 +0200, Michael Nuwer [dBVIPS]
> <nuwermj@no.spam.yahoo.com> wrote:
>
>
>
>
> As you point out, the code to replace all CR/LF instances with "" in
> a text file using the read() and write() methods of the file class is
> quite straight forward and is not exactly rocket science.
>
> The problem is that it looks as if Ruud will have to tackle a .dbt file
> and not a simple text file. This involves reading and writing the
> first 512byte block without change and then reading the next block,
> extracting the memo length, which may indicate that further blocks need
> to be read, and then keeping track of just how many substitutions have
> been made before adjusting the length bytes and writing the revised
> data back to the output file. All this after checking that the
> extended data will not cross a 512 block boundary. Again, this is also
> not particularly difficult to code although one must keep in mind that
> the output file must have the same name as the .dbf file so the input
> file will have to be renamed to something else first.
>
> The hairy part comes when the extended data crosses a block boundary.
> Unless it happened to be the very last record it would corrupt tne next
> 512byte block so then the data has to be written at the end of a file
> that hasn't yet been written out. This means one would probably have
> to run through the entire input file first checking for entries that
> would cross boundaries, keep track of them, write the others and then
> go back for the records with these extended fields. This would in turn
> involve patching the header of the new output file to show the next
> vacant block. Having done that, all that remains is the interesting
> bit which is to tackle the .dbf to change the pointers to the .dbt
> entries for these particular records. As the .dbt file has no back
> reference to the actual record in the .dbf file one would have to scan
> the .dbf file looking for the old block number to know where to write
> the new block number. Still not rocket science but we're getting
> there! <g>
>
> Mervyn

Mervyn Bick

2005-12-14, 3:23 am

On Wed, 14 Dec 2005 00:01:00 +0200, Michael Nuwer [dBVIPS]
<nuwermj@no.spam.yahoo.com> wrote:

>
> Could he use something like the following?

If I could change the background of this message to a light rose pink to
denote my blush, I would.

Why do I always look for the hard way to do anything? Your way, dBase
does all the hard work as it should. <g>

That said, I think Ruud still has some work to do as dBase doesn't do what
I would expect with tags if a memo field is dragged straight onto a
form. I will post a screenprint of "Before and after" the running of your
demo in the Binaries NG.

Although I haven't made a screenprint, using ... around a line
doesn't work properly either.

If the combined memofields are to be displayed in dBase, Ruud might be
better served by converting the tags to CR/LF CR/LF pairs instead of
the other way round.

Mervyn.

Michael Nuwer [dBVIPS]

2005-12-14, 7:23 am

Mervyn Bick wrote:

> That said, I think Ruud still has some work to do as dBase doesn't do
> what I would expect with tags if a memo field is dragged straight
> onto a form. I will post a screenprint of "Before and after" the
> running of your demo in the Binaries NG.
>
> Although I haven't made a screenprint, using ... around a line
> doesn't work properly either.

Ah, yes, that is a dBASE bug. It should be resolved by adding a space
after the HTML tag ( " " ) so that it renders like this:

 Edibility

As I recall this problem happens when the tag begins in column one and
is followed by text. So, in this case, when there are two "p" tags in a
row, , the first causes the second to be in the first column. This
same outcome happens with other tags too.

Marko Mihorko [dBVIPS]

2005-12-14, 7:23 am

Hello Mervyn!

"Mervyn Bick" je napisal v sporocilo...

> Why do I always look for the hard way to do anything? Your way, dBase does
> all the hard work as it should. <g>

Hm, not all, becauseVBScript does its part as well, doesn't it? ;-)

> That said, I think Ruud still has some work to do as dBase doesn't do what I
> would expect with tags if a memo field is dragged straight onto a form.
> I will post a screenprint of "Before and after" the running of your demo in
> the Binaries NG.

Well, it is shown in the example below that _all_ the hard work could be done
truly in dBASE _only_ (and it looks like successfully, too. ;-)

As usual mark/copy/paste/save everything below to a file named f.e.
<mervyn.wfm> and later on doubleclick on it under the <Forms> tab of the
<Navigator> window.

First click (once!) on the pushbutton "Show evalTags" in order to see that
there are none present at the moment. After that click on the pushbutton
"ReplaceCRLF" and the evalTags should show up. Afterwards click on the
pushbutton "Hide evalTags" and everything should look still as expected.
Finally you can perhaps navigate to an another record and test further...

Marko Mihorko [dBVIPS]

set database to; close tables
if file('fish_M.dbf'); drop table fish_M
endif
use :dbaseSamples:fish
copy to fish_M
use
** END HEADER -- do not remove this line
//
// Generated on 14.12.2005
//
parameter bModal
local f
f = new MervynForm()
if (bModal)
f.mdi = false // ensure not MDI
f.readModal()
else
f.open()
endif

class MervynForm of FORM
with (this)
text = "Mervyn - Replace cr/lf with evaltags "
endwith

this.FISH1 = new QUERY()
this.FISH1.parent = this
with (this.FISH1)
sql = 'select * from "Fish_M.dbf"'
active = true
endwith

this.EDITOR1 = new EDITOR(this)
with (this.EDITOR1)
height = 13.5
left = 1.0
top = 0.5
width = 38.0
dataLink = form.fish1.rowset.fields["description"]
endwith

this.PUSHBUTTON1 = new PUSHBUTTON(this)
with (this.PUSHBUTTON1)
onClick = class::PUSHBUTTON1_O
NCLICK
left = 2.0
top = 14.5
text = "Replace CRLF"
endwith

this.PUSHBUTTON2 = new PUSHBUTTON(this)
with (this.PUSHBUTTON2)
onClick = class::PUSHBUTTON2_O
NCLICK
left = 23.0
top = 14.5
text = "Show evalTags"
endwith

this.rowset = this.fish1.rowset

function PUSHBUTTON1_onClick
oDesc = form.fish1.rowset.fields["description"]
form.cFR(oDesc.value, chr(13)+chr(10), "")
form.fish1.rowset.save()
return

function PUSHBUTTON2_onClick
form.editor1.evalTags = iif(form.editor1. evalTags,false,true)

this.text = iif(form.editor1.evalTags,;
"Show evalTags","Hide evalTags")
return

function cFR(cS,cF,cR)
if cS.indexOf(cF) > -1
do
cS = cS.stuff(cS.indexOf(cF),cF.length,cR)
until cS.indexOf(cF) = -1
endif
return cS
endclass

Ken Mayer [dBVIPS]

2005-12-14, 9:23 am

Mervyn Bick wrote:
> On Wed, 14 Dec 2005 00:01:00 +0200, Michael Nuwer [dBVIPS]
> <nuwermj@no.spam.yahoo.com> wrote:
>
>
>
>
> If I could change the background of this message to a light rose pink
> to denote my blush, I would.
>
> Why do I always look for the hard way to do anything? Your way, dBase
> does all the hard work as it should. <g>
>
> That said, I think Ruud still has some work to do as dBase doesn't do
> what I would expect with tags if a memo field is dragged straight
> onto a form. I will post a screenprint of "Before and after" the
> running of your demo in the Binaries NG.

FWIW, please don't post screen shots unless requested. Some of us are
still stuck using dialup connections, and it hampers us a lot to have to
wait for large downloads ...

Ken

--
/(Opinions expressed are purely my own, not those of dataBased
Intelligence, Inc.)/

*Ken Mayer* [dBVIPS]
/Golden Stag Productions/
dBASE at goldenstag dot net
http://www.goldenstag.net/dbase/dBASEBook.htm
http://www.goldenstag.net/GSP
http://www.goldenstag.net/dbase

Mervyn Bick

2005-12-14, 9:23 am

On Wed, 14 Dec 2005 14:12:24 +0200, Michael Nuwer [dBVIPS]
<nuwermj@nospam.please.yahoo.com> wrote:

> Ah, yes, that is a dBASE bug. It should be resolved by adding a space
> after the HTML tag ( " " ) so that it renders like this:

Replacing CR/LF with " " solves the problem as does Marko's solution of
replacing CR/LF with ""

Mervyn

Mervyn Bick

2005-12-14, 9:23 am

On Wed, 14 Dec 2005 14:31:03 +0200, Marko Mihorko [dBVIPS] <xyz@xyz.com>
wrote:

> Well, it is shown in the example below that _all_ the hard work could
> be done truly in dBASE _only_ (and it looks like successfully, too. ;-)

You are being too modest! I don't recall one instance where one of your
example programs hasn't worked exactly as it was supposed to.

Mervyn

Ruud van den Bosch

2005-12-14, 11:23 am

Thank you all.

The problem is solved !!

Ruud

"Ruud van den Bosch" <ruud@derckx.nl> schreef in bericht
news:gatiYU8$FHA.1228@news-server...
> I've imported some memo fields and in these fields are cr/lf but not the
> evaltags dbase uses (like etc)
>
> I need to combine the contents of these memo fields with some other
> memofields out of a table. However in this table the memofields are saved
> with evaltags.
>
> When I combine the fields, the CR/LF will disappear. This is not what I
> want.
>
> How can I replace the CR/LF with evaltags ????
>
>
>

Michael Nuwer [dBVIPS]

2005-12-14, 1:23 pm

Marko Mihorko [dBVIPS] wrote:

Hello Marko,

Thanks for your comments.

Best wishes to you and your family for this Christmas Season!!

Michael.

Michael Nuwer [dBVIPS]

2005-12-14, 1:23 pm

Mervyn Bick wrote:

>
> Replacing CR/LF with " " solves the problem as does Marko's solution
> of replacing CR/LF with ""

Thanks for this report Mervyn. In case there's a lurker or two with an
interest in "regular expressions" in dBASE, the code below will wrap
each paragraph in a tag. I'm posting this only as a leaning
example, not as a critic of anything in this thread.

set database to; close tables
if file('fishTest.dbf'); drop table fishTest
endif
use :dbaseSamples:fish
copy to fishTest
use

q = new query()
q.sql = 'select * from fishTest'
q.active = true

oRegExp = new OleAutoClient("VBScript.RegExp")
oRegExp.global = true
/// note this line
oRegExp.pattern = "(.*)\r\n"

q.rowset.first()
do
cStr = q.rowset.fields['Description'].value
/// note this line:
cStr = oRegExp.replace(cStr, "$1")
q.rowset.fields['Description'].value = cStr
q.rowset.save()
until not q.rowset.next()

Mervyn Bick

2005-12-14, 8:23 pm

On Wed, 14 Dec 2005 21:02:20 +0200, Michael Nuwer [dBVIPS]
<nuwermj@no.spam.yahoo.com> wrote:

> Thanks for this report Mervyn. In case there's a lurker or two with an
> interest in "regular expressions" in dBASE, the code below will wrap
> each paragraph in a tag. I'm posting this only as a leaning
> example, not as a critic of anything in this thread.

Your example works a treat and dBase doesn't choke when the memo field is
displayed on a form.

I read your paper on regular expressions in the dBulletin earlier this
year but I must admit that at the time I didn't give it the attention it
deserves as I no longer actively use dBase. After you posted your first
demo program a few days ago I reviewed the paper to get a better
understanding of what was happening in the code. This is certainly a very
sharp arrow to have in one's programming quiver. It gives a very powerful
alternative to the traditional DBL string handling functions and methods
for "find and replace" tasks.

Mervyn

Marko Mihorko [dBVIPS]

2005-12-15, 8:23 pm

Hello Mervyn!

"Mervyn Bick" je napisal v sporoèilo...

> You are being too modest! I don't recall one instance where one of your
> example programs hasn't worked exactly as it was supposed to.

Hm, well, <blush>. ;-)

Anyhow, I must admit that I surely do some... ;-)

Marko Mihorko [dBVIPS]

Marko Mihorko [dBVIPS]

2005-12-15, 8:23 pm

Hello Michael!

"Michael Nuwer [dBVIPS]" je napisal v sporoèilo...

> Thanks for your comments.

You are welcome.

And I guess that you noticed my emoticons which I always add quite abundantly
in all messages of mine. ;-)

I am doing it in perhaps vain hopes that my remarks or even roguish joke
attempts wouldn't be understood wrongly and/or too seriosly in whatever case.

Namely, I am well aware that a slight shade could make quite a difference in
the comprehension of the meaning as well as of the real intention in a language
which, after all, remains a non-domestic one for me.

> Best wishes to you and your family for this Christmas Season!!

Thank you and I wish the same to you and your ones...

Marko Mihorko [dBVIPS]

Michael Nuwer [dBVIPS]

2005-12-16, 7:23 am

Mervyn Bick wrote:
>
> Your example works a treat and dBase doesn't choke when the memo field
> is displayed on a form.
>
> I read your paper on regular expressions in the dBulletin earlier this
> year but I must admit that at the time I didn't give it the attention
> it deserves as I no longer actively use dBase. After you posted your
> first demo program a few days ago I reviewed the paper to get a better
> understanding of what was happening in the code. This is certainly a
> very sharp arrow to have in one's programming quiver. It gives a very
> powerful alternative to the traditional DBL string handling functions
> and methods for "find and replace" tasks.

Thanks Mervyn.

Happy Holidays

Michael Nuwer [dBVIPS]

2005-12-16, 7:23 am

Marko Mihorko [dBVIPS] wrote:
>
> And I guess that you noticed my emoticons which I always add quite abundantly
> in all messages of mine. ;-)
>
> I am doing it in perhaps vain hopes that my remarks or even roguish joke
> attempts wouldn't be understood wrongly and/or too seriosly in whatever case.
>
> Namely, I am well aware that a slight shade could make quite a difference in
> the comprehension of the meaning as well as of the real intention in a language
> which, after all, remains a non-domestic one for me.
>

Hello Marko,

I have never found your comments anything but helpful and alway polite.
I don't think you have any worry about being wrongly understood. :-)

Marko Mihorko [dBVIPS]

2005-12-17, 11:23 am

Hello Michael!

"Michael Nuwer [dBVIPS]" je napisal v sporocilo...

> I have never found your comments anything but helpful and alway polite.

Thank you.

> I don't think you have any worry about being wrongly understood. :-)

Well, I was really a bit worried since you wrote somewhere in this thread
(indeed, not as a reply to my first mesage but right after I posted it ;-) that
you are posting something but "not as a critic of anything in this thread".

However, I feel relieved now... ;-)

Marko Mihorko [dBVIPS]

Sponsored Links