How does dir /s and xcopy and zip
manage to descend into subdirectories and
come out and continue?

Do they make a private copy of the DTA
structure and restore it?

That contains all the necessary information?

muta,

> Do they make a private copy of the DTA
> structure and restore it?

That is one way to do it. Another is to create a new DTA record and use
INT 21h, AH=1Ah to make it the active one

You only need to save the part upto (not included) the (returned)
file-attribute field (at offset 0x15) though.

> That contains all the necessary information?

Yes, AFAIK it does.

By the way, I most always use a function recursion to create the folder
recursion : when the function encounters a folder I call the function
itself, but with the path to the just-found folder.

Regards,
Rudy Wieser

On Tue, 16 Aug 2022, muta...@gmail.com wrote:

> How does dir /s and xcopy and zip
> manage to descend into subdirectories and
> come out and continue?
>
> Do they make a private copy of the DTA
> structure and restore it?
>
> That contains all the necessary information?

They move the DTA pointer, generally. That's how my implementation does
it:

https://github.com/buricco/doslite/blob/main/cmd/attrib/attrib.c

-uso.

On Tuesday, August 16, 2022 at 9:08:21 PM UTC+8, R.Wieser wrote:
> muta,
> > Do they make a private copy of the DTA
> > structure and restore it?
> That is one way to do it. Another is to create a new DTA record and use
> INT 21h, AH=1Ah to make it the active one
>
> You only need to save the part upto (not included) the (returned)
> file-attribute field (at offset 0x15) though.
> > That contains all the necessary information?
> Yes, AFAIK it does.
>
> By the way, I most always use a function recursion to create the folder
> recursion : when the function encounters a folder I call the function
> itself, but with the path to the just-found folder.
>
> Regards,
> Rudy Wieser

Thanks guys for your responses.

But I don't see how 1a can work.

That requires you to know how big the DTA is, right?

But the DTA could be extended at any time by
a dos vendor to e.g. include a field for long filename.

Which is exactly what was done in pdos.

Or is there another call to retrieve the length of a DTA?

Saving up to 15h seems more robust.

But what is this?

--DOS 2.x and most 3.x---
0Dh WORD entry count within directory
0Fh DWORD pointer to DTA???
13h WORD cluster number of start of parent directory
---PC-DOS 4.01, MS-DOS 3.2/3.3/5.0---
0Dh WORD entry count within directory
0Fh WORD cluster number of start of parent directory
11h 4 BYTEs reserved
--

What is start of parent directory? The directory that contains this file?

Does that mean that msdos keeps going back to the first
directory cluster and then retraverses until it gets
to the required directory entry number?

I assume that is some absolute number
rather than logical, taking into account long filenames.

And with fat32, cluster numbers are 32 bit.

The reserved 4 bytes could have been used for that,
but I don't see any mention of that here:

http://www.ctyme.com/intr/rb-2977.htm

Any thoughts?

Thanks.

Muta,

> But I don't see how 1a can work.
>
> That requires you to know how big the DTA is, right?

Nope. It just sets the pointer to the new DTA

> That requires you to know how big the DTA is, right?

The INT itself knows nothing about sizes. But ask yourself, even if you
would not use the INT, wouldn't you than not still need to know about the
layout (and thus size) of the DTA to be able to use it ? How else would
you be able to grab, for instance, the returned file attributes ?

> Saving up to 15h seems more robust.

Whut ? You've just told us that the DTA can change (in contents and size),
but now you're going to use some fixed size for copying ? :-)

Yes, the DTA /could/ change. But if it does its an *OS* related change.
Well documented..

> But the DTA could be extended at any time by
> a dos vendor to e.g. include a field for long filename.
>
> Which is exactly what was done in pdos.

And that means you have to check the DOS version your program is running
under, and either tell the user that your program only works under version
X, Y and Z, or you need to write the apropriate code for all those versions
and branch to them according to the detected one.

> But what is this?

Do you think we are mindreaders ? I'm from a planet called Earth, and none
of us here can.

Next time tell us where you got the information from please. (you got lucky
that I could match a string to something in "Ralf Browns Interrupt list" and
by it found what you where talking about)

> What is start of parent directory? The directory that contains this file?

Thats data used by the OS to keep track of what it is doing. You can
touch. or (trying to) use it on your own peril. Consider yourself warned.
(tongue in cheek, but I'm serious)

But yes, that sounds about right.

> Does that mean that msdos keeps going back to the first
> directory cluster and then retraverses until it gets
> to the required directory entry number?

Why should it ? All entry records in a directory are of the same size.
IOW, multiply the index of the currently found entry by that size, add the
start sector and you have the last-found entry. No iteration needed. (a bit
simplified, but good enough for the job)

> I assume that is some absolute number
> rather than logical, taking into account long filenames.

Under DOS long filenames are cut up into parts and hidden away into regular
entry records (just with a special marker, similar to how deleted files are
marked), one after the other. The OS just grabs a few entries, takes the
apropriate fields and stich the long filename back together.

> And with fat32, cluster numbers are 32 bit.
>
> The reserved 4 bytes could have been used for that,
> but I don't see any mention of that here:

Than can be because Ralf Brown didn't go beyond the 16-bit DOS versions.
You would need to find yourself the specs for a/that particular 32-bit "DOS"
version to find out.

Regards,
Rudy Wieser

On Wednesday, August 17, 2022 at 2:36:01 PM UTC+8, R.Wieser wrote:
> Muta,
> > But I don't see how 1a can work.
> >
> > That requires you to know how big the DTA is, right?
> Nope. It just sets the pointer to the new DTA
> > That requires you to know how big the DTA is, right?
> The INT itself knows nothing about sizes. But ask yourself, even if you
> would not use the INT, wouldn't you than not still need to know about the
> layout (and thus size) of the DTA to be able to use it ? How else would
> you be able to grab, for instance, the returned file attributes ?

I would need to know the layout as it was at
a particular dos version. Not any new fields, like lfn, that
had been subsequently added.

> > Saving up to 15h seems more robust.
> Whut ? You've just told us that the DTA can change (in contents and size),
> but now you're going to use some fixed size for copying ? :-)

Yes, because existing things wouldn't change.

> Yes, the DTA /could/ change. But if it does its an *OS* related change.
> Well documented..

And as an os vendor taking over where
Microsoft left decades ago, I have documented
an extension.

> > But the DTA could be extended at any time by
> > a dos vendor to e.g. include a field for long filename.
> >
> > Which is exactly what was done in pdos.
> And that means you have to check the DOS version your program is running
> under, and either tell the user that your program only works under version
> X, Y and Z, or you need to write the apropriate code for all those versions
> and branch to them according to the detected one.

The application program can only code what was known
at the time, not what Microsoft or some other os vendor
added as an extension since then.

> > But what is this?
>
> Do you think we are mindreaders ? I'm from a planet called Earth, and none
> of us here can.
>
> Next time tell us where you got the information from please. (you got lucky
> that I could match a string to something in "Ralf Browns Interrupt list" and
> by it found what you where talking about)

I provided the link further down, but yes,
I should have provided it first.

> > What is start of parent directory? The directory that contains this file?
> Thats data used by the OS to keep track of what it is doing. You can
> touch. or (trying to) use it on your own peril. Consider yourself warned.
> (tongue in cheek, but I'm serious)

I am an os vendor. I need to fill in that data.

> But yes, that sounds about right.
> > Does that mean that msdos keeps going back to the first
> > directory cluster and then retraverses until it gets
> > to the required directory entry number?
> Why should it ? All entry records in a directory are of the same size.
> IOW, multiply the index of the currently found entry by that size, add the
> start sector and you have the last-found entry. No iteration needed. (a bit
> simplified, but good enough for the job)

I at least need to chain through the clusters.

> > I assume that is some absolute number
> > rather than logical, taking into account long filenames.
> Under DOS long filenames are cut up into parts and hidden away into regular
> entry records (just with a special marker, similar to how deleted files are
> marked), one after the other. The OS just grabs a few entries, takes the
> apropriate fields and stich the long filename back together.
> > And with fat32, cluster numbers are 32 bit.
> >
> > The reserved 4 bytes could have been used for that,
> > but I don't see any mention of that here:
> Than can be because Ralf Brown didn't go beyond the 16-bit DOS versions.
> You would need to find yourself the specs for a/that particular 32-bit "DOS"
> version to find out.
>
> Regards,
> Rudy Wieser

Muta,

> I would need to know the layout as it was at a particular dos version.

Exactly. And that is the info that Ralf Browns list tries to offer you.

> Not any new fields, like lfn, that had been subsequently added.

:-) How would you know if the version of DOS you're programming under
isn't one who actually has such extra fields ?

> Yes, because existing things wouldn't change.

I would not bet on that if I where you.

For the record, when you say "added" it normally means that extra fields can
be *inserted* (and/or obsolete fields removed). And that ofcourse means
that the offsets to the fields after it change.

What I mostly do is to first create a test program, and simply try to match
the data/record returned by the tested INT/procedure/etc. with the
information (as in the RBIL) at hand.

>> Yes, the DTA /could/ change. But if it does its an *OS* related change.
>> Well documented..
>
> And as an os vendor taking over where Microsoft left decades ago, I have
> documented an extension.

And that extention is only valid for your particular DOS version. It
could also easily clash with an extension written by someone else ...

> The application program can only code what was known
> at the time, not what Microsoft or some other os vendor
> added as an extension since then.

Quite so. And that means that when you run a program using "what Microsoft
or some other os vendor added as an extension since then" on a "what was
known at the time" version it wil crash-and-burn. Which is why you need
to do version checking.

> I am an os vendor. I need to fill in that data.

*You* need to fill in that data ? In that case its fully up to you to
decide what your DTA is going to look like / what you're going to store in
there. If you want to store the number of the first cluster of the
containing directory in there you can do that. If you don't want to that
you simply do not include that field in your DTA.

Its all upto you, as that part of it is, as I tried to make clear, an
"dragons be here, stay out" area.

On the other hand, if you're just trying to supplement an already existing
framework you should not be looking at the RBIL, but find yourself the specs
for the version of the OS you are trying to, quite literally, add some stuff
to.

On yet another hand, if you're just extending, isn't at least that "parent
cluster" field all filled in yet ?

IOW, its not at all clear to me which level you are operating on.

> I at least need to chain through the clusters.

Yes you would. But every time you want to grab a new entry from that
directory ? Why ?

I could easily imagine just storing the the sector/cluster the currently
returned entry is in. What good would it do me to know where the whole
directory started ? It just creates extra work.

Regards,
Rudy Wieser

On Thursday, August 18, 2022 at 6:07:04 PM UTC+8, R.Wieser wrote:
> Muta,
> > I would need to know the layout as it was at a particular dos version.
> Exactly. And that is the info that Ralf Browns list tries to offer you.

Which is useless if there is a newer version of
MSDOS since my program was written, which
operates on a larger DTA.

> > Not any new fields, like lfn, that had been subsequently added.
> :-) How would you know if the version of DOS you're programming under
> isn't one who actually has such extra fields ?

I don't care if it is the os providing a larger DTA.
It is only when I try to second guess MSDOS future
by providing a DTA, as an app, that there is an issue.
I think it should be invalid to provide your own DTA.
You should instead just use what the OS gives you.

> > Yes, because existing things wouldn't change.
> I would not bet on that if I where you.
>
> For the record, when you say "added" it normally means that extra fields can
> be *inserted* (and/or obsolete fields removed). And that ofcourse means
> that the offsets to the fields after it change.
>
> What I mostly do is to first create a test program, and simply try to match
> the data/record returned by the tested INT/procedure/etc. with the
> information (as in the RBIL) at hand.
> >> Yes, the DTA /could/ change. But if it does its an *OS* related change.
> >> Well documented..
> >
> > And as an os vendor taking over where Microsoft left decades ago, I have
> > documented an extension.
> And that extention is only valid for your particular DOS version. It
> could also easily clash with an extension written by someone else ...

Sure. But a standard MSDOS program, that simply
accepts the DTA from MSDOS, will work on either.

> > The application program can only code what was known
> > at the time, not what Microsoft or some other os vendor
> > added as an extension since then.
> Quite so. And that means that when you run a program using "what Microsoft
> or some other os vendor added as an extension since then" on a "what was
> known at the time" version it wil crash-and-burn. Which is why you need
> to do version checking.

You can't do version checking on this as it would
require refusing to run on higher versions in case
Microsoft had extended the DTA.

> > I am an os vendor. I need to fill in that data.
> *You* need to fill in that data ? In that case its fully up to you to
> decide what your DTA is going to look like / what you're going to store in
> there. If you want to store the number of the first cluster of the
> containing directory in there you can do that. If you don't want to that
> you simply do not include that field in your DTA.
>
> Its all upto you, as that part of it is, as I tried to make clear, an
> "dragons be here, stay out" area.
>
> On the other hand, if you're just trying to supplement an already existing
> framework you should not be looking at the RBIL, but find yourself the specs
> for the version of the OS you are trying to, quite literally, add some stuff
> to.
>
> On yet another hand, if you're just extending, isn't at least that "parent
> cluster" field all filled in yet ?

Good question. I had to check what was currently happening.
None of those fields are used currently. Only one active
traversal is supported at the moment and the directory
file handle is stored internally.

I could support multiple traversals by storing the
file handle in the DTA.

> IOW, its not at all clear to me which level you are operating on.

Both levels. Sensible rules for both os and app.

> > I at least need to chain through the clusters.
> Yes you would. But every time you want to grab a new entry from that
> directory ? Why ?
>
> I could easily imagine just storing the the sector/cluster the currently
> returned entry is in. What good would it do me to know where the whole
> directory started ? It just creates extra work.
>
> Regards,
> Rudy Wieser

muta,

>> > I would need to know the layout as it was at a particular dos version.
>> Exactly. And that is the info that Ralf Browns list tries to offer you.
>
> Which is useless if there is a newer version of
> MSDOS since my program was written, which
> operates on a larger DTA.

Your above sentence makes no sense. Somehow you seem to blame Ralf Brown
for not knowing about the DTAs of *future versions* of DOS.

As I already tried to make clear to you, if you run your program on *ANY*
version of DOS, future or past(!), which doesn't use the exact same DTA as
you are using now your program will most likely crash-and-burn. Its as
simple as that.

Hence the need for your program to check for the DOS version its trying to
run on, and abort (with an appropriate message) if you are not sure that the
detected version uses the same DTA as the one you are now writing your
program on.

> I don't care if it is the os providing a larger DTA.
> It is only when I try to second guess MSDOS future
> by providing a DTA, as an app, that there is an issue.

And the above is why you should *not* be trying to second guess it.
Unless you have no problem with having your program crash-and-burn
ofcourse - or worse, as I see you ask for fields in that DTA you should
*not* be touching, possibly trash the data on their drives.

> Sure. But a standard MSDOS program, that simply
> accepts the DTA from MSDOS, will work on either.

As you have still not mentioned anything about what your extension is
supposed to be doing there is no way for me to agree with it.

> You can't do version checking on this as it would
> require refusing to run on higher versions in case
> Microsoft had extended the DTA.

That sentence makes no sense either.

> Good question. I had to check what was currently happening.
> None of those fields are used currently.

How do you know ? Did you already try to just zero those fields out
(and/or put some other values in them) between two requests for the next
directory entry. If not ...

> Only one active traversal is supported at the moment and the
> directory file handle is stored internally.

"internally" ? You mean somewhere else than in the DTA ? Or are you
talking about that "first cluster of the current directory" field you where
talking about earlier ? If so, thats not a handle.

If it *is* a handle and its stored somewhere outside of the DTA, ask
yourself how you can iterate thru two (or more) directories at the same
time.

> I could support multiple traversals by storing the file handle in the DTA.

I've written programs which traversed two directories at a time, it just
used that INT 21h, AH=1Ah a lot. IOW, *all* the data needed to fetch a next
directory entry was-and-is already present in a standard DTA.

> Both levels. Sensible rules for both os and app.

Thats too bad, as I mentioned three ...

Regards,
Rudy Wieser

On Friday, August 19, 2022 at 7:09:00 PM UTC+8, R.Wieser wrote:
> muta,
>
> >> > I would need to know the layout as it was at a particular dos version.
> >> Exactly. And that is the info that Ralf Browns list tries to offer you.
> >
> > Which is useless if there is a newer version of
> > MSDOS since my program was written, which
> > operates on a larger DTA.
> Your above sentence makes no sense. Somehow you seem to blame Ralf Brown
> for not knowing about the DTAs of *future versions* of DOS.
>
> As I already tried to make clear to you, if you run your program on *ANY*
> version of DOS, future or past(!), which doesn't use the exact same DTA as
> you are using now your program will most likely crash-and-burn. Its as
> simple as that.

That's not the way to design things.
It should be upwardly compatible.

The os should be free to add new fields.
So long as it doesn't disturb the existing fields,
it should work.

> Hence the need for your program to check for the DOS version its trying to
> run on, and abort (with an appropriate message) if you are not sure that the
> detected version uses the same DTA as the one you are now writing your
> program on.
> > I don't care if it is the os providing a larger DTA.
> > It is only when I try to second guess MSDOS future
> > by providing a DTA, as an app, that there is an issue.
> And the above is why you should *not* be trying to second guess it.
> Unless you have no problem with having your program crash-and-burn
> ofcourse - or worse, as I see you ask for fields in that DTA you should
> *not* be touching, possibly trash the data on their drives.
> > Sure. But a standard MSDOS program, that simply
> > accepts the DTA from MSDOS, will work on either.
> As you have still not mentioned anything about what your extension is
> supposed to be doing there is no way for me to agree with it.

The extension is to return lfn.

> > You can't do version checking on this as it would
> > require refusing to run on higher versions in case
> > Microsoft had extended the DTA.
> That sentence makes no sense either.
> > Good question. I had to check what was currently happening.
> > None of those fields are used currently.
> How do you know ? Did you already try to just zero those fields out
> (and/or put some other values in them) between two requests for the next
> directory entry. If not ...
> > Only one active traversal is supported at the moment and the
> > directory file handle is stored internally.
> "internally" ? You mean somewhere else than in the DTA ? Or are you
> talking about that "first cluster of the current directory" field you where
> talking about earlier ? If so, thats not a handle.
>
> If it *is* a handle and its stored somewhere outside of the DTA, ask
> yourself how you can iterate thru two (or more) directories at the same
> time.

Currently I can't. That's why I posted my original question.
I was looking for a sensible design.

> > I could support multiple traversals by storing the file handle in the DTA.
> I've written programs which traversed two directories at a time, it just
> used that INT 21h, AH=1Ah a lot. IOW, *all* the data needed to fetch a next
> directory entry was-and-is already present in a standard DTA.
> > Both levels. Sensible rules for both os and app.
> Thats too bad, as I mentioned three ...
>
> Regards,
> Rudy Wieser

Anyway, the sensible rules I am thinking
of at the moment are that the app is
required to get the DTA from the os,
it can't provide its own because it
can't know the length of future DTAs.

If an app wants to do a new traversal
it is required to preserve up to 0x15.
The area before 0x15 is at the discretion
of the os.

I use a file handle, subject to change at any
time. MSDOS does whatever it wants
to support fat32 too.