The CP/M filesystem continues to delight and terrify me.

As I contemplate the CiderPress II implementation, two questions spring to mind:

(1) Did the CP/M v3.1 filesystem format extensions make it to the Apple II? Specifically, the "every 4th directory entry is actually a place to hold dates". I found "CPM3.1Z80_Softcard.zip" on asimov, but none of the disks seem to use this feature.

(2) Would it make sense to treat user numbers as directories? That seems like a natural way to handle them, but it does mean the tools would be referring to things as "0:FILE.TXT" instead of "FILE.TXT". I don't know how other tools handle this, absent a "user" command to set the current state. Each disk gets 16 (or is it 31?) pre-defined directories that can hold files but can't be modified.

An alternative would be to make it an editable file attribute, and just let people struggle when more than one file has the same name. (Not an issue in the GUI, problematic in the CLI.)

A better option would be to include it in the filename, but only when nonzero. So you have "FILE.TXT" and "1:FILE.TXT". Or maybe "1,FILE.TXT" so it doesn't get confused as a directory name. Or "FILE.TXT,1".

This would be easier if I'd ever used CP/M, and had some idea about the conventions. :-)

fadden wrote:
> The CP/M filesystem continues to delight and terrify me.
>

If computer file systems had been designed by persons with secretarial
experience, perhaps they'd be less brain-damaged.

Even now it's an arbitrary limitation against more than one file having the
same file name in a directory, even when there are plenty of practical use
cases for having multiple files with the same name in a directory with
differing modification or creation dates. An intelligent user interface
would notice this plurality and would allow differentiation based on date or
whatever other distinguishing attribute.

--
]DF$
The New Apple II User's Guide:
https://macgui.com/newa2guide/

I added my current thoughts on user numbers to the filesystem doc: https://github.com/fadden/CiderPress2/blob/main/DiskArc/FS/CPM-notes.md#appendix-user-numbers-and-ciderpress

On 8/10/23 21:54, fadden wrote:
> The CP/M filesystem continues to delight and terrify me.
>
> As I contemplate the CiderPress II implementation, two questions spring to
> mind:
>
> (1) Did the CP/M v3.1 filesystem format extensions make it to the Apple II?
> Specifically, the "every 4th directory entry is actually a place to hold
> dates". I found "CPM3.1Z80_Softcard.zip" on asimov, but none of the disks
> seem to use this feature.

Depends upon the vendor's implementation. CP/M-3.x timestamping was always an
optional feature that required initialization of the directory for those extra
entries and BIOS implementation to read/write them. A system that doesn't
support them simply ignores the entries since they exist in an "impossible"
user area. I'm reasonably sure that the stamping feature was supported by at
least one or two of the Apple ports. Off the top of my head, Circom, DRI
(Gold Card) and ALS all had CP/M-3.x ports for Apple. There were also a few
home-rolled ports for Applicard and Microsoft Softcard.

The so-called 'Datestamper' scheme was actually far more common in the later
years of community development. I added datestamper support to cpmtools a few
years back, so it now supports both schemes.

> (2) Would it make sense to treat user numbers as directories? That seems
> like a natural way to handle them, but it does mean the tools would be
> referring to things as "0:FILE.TXT" instead of "FILE.TXT". I don't know
> how other tools handle this, absent a "user" command to set the current
> state. Each disk gets 16 (or is it 31?) pre-defined directories that can
> hold files but can't be modified.

Referring to user area locations with the N: prefix is quite standard in the
enhanced CP/M world, e.g. Z-System. That's exactly how image manipulation
utilities like cpmtools manage them. Only enhanced CP/M implementations
support user areas > 15. Anything greater won't be damaged by file I/O, but
simply won't be accessible by generic CP/M. Placing the CCP as 31:ccp.sys
(lowercase) was commonplace among OEM implementations.

It would be helpful to provide a 'move' function to change user area for all
extents of a given file or set of files.

> An alternative would be to make it an editable file attribute, and just let
> people struggle when more than one file has the same name. (Not an issue
> in the GUI, problematic in the CLI.)
>
> A better option would be to include it in the filename, but only when
> nonzero. So you have "FILE.TXT" and "1:FILE.TXT". Or maybe "1,FILE.TXT"
> so it doesn't get confused as a directory name. Or "FILE.TXT,1".

I would vote for always including the user prefix when listing a directory
while inferring '0' when none is provided.

I'd strongly suggest taking a look at cpmtools for ideas and concepts about
mapping between CP/M and current filesystems.

On Sunday, August 13, 2023 at 8:35:57 AM UTC-7, Steven Hirsch wrote:
> > (1) Did the CP/M v3.1 filesystem format extensions make it to the Apple II?
> Depends upon the vendor's implementation. CP/M-3.x timestamping was always an
> optional feature that required initialization of the directory for those extra
> entries and BIOS implementation to read/write them. A system that doesn't
> support them simply ignores the entries since they exist in an "impossible"
> user area.

That's what I was hoping for. If they had to be generated on the fly things would be more complicated.

> Referring to user area locations with the N: prefix is quite standard in the
> enhanced CP/M world, e.g. Z-System. That's exactly how image manipulation
> utilities like cpmtools manage them. Only enhanced CP/M implementations
> support user areas > 15. Anything greater won't be damaged by file I/O, but
> simply won't be accessible by generic CP/M. Placing the CCP as 31:ccp.sys
> (lowercase) was commonplace among OEM implementations.

I found the 31:cp/m.sys and 31:cp/am.sys files on some of the disks from Asimov. They have allocation block numbers >= 128, on a disk that only has 128 blocks. For reasons I can't recall, CiderPress displays them and actually mods the block number, so the file is readable.

What purpose does this entry serve? That wasn't clear to me.

cpm.fsck from cpmtools flags the extent as erroneous ("bad status") and offers to clear the entry.

> It would be helpful to provide a 'move' function to change user area for all
> extents of a given file or set of files.
[...]
> I would vote for always including the user prefix when listing a directory
> while inferring '0' when none is provided.

My current inclination is to go with the "it's part of the filename" approach, rather than trying to create virtual directories. However, this can't use ":" as the delimiter, because that's interpreted as a directory separator by the tools (thanks to HFS), which is why I was contemplating ',' instead. Assuming user 0 when none is specified is part of the various plans.

The cp2 command-line tool accepts wildcards, so with the "virtual directory" approach, moving files between user areas is equivalent to moving files between directories. Something similar could be done for the filename-modification approach with a bit of trickery in the command-line tool, but you couldn't drag&drop in the GUI tool because there'd be nowhere to drop into.

> I'd strongly suggest taking a look at cpmtools for ideas and concepts about
> mapping between CP/M and current filesystems.

I have been; the documentation is a little thin (except for the CP/M filesystem description in section 5, which is wonderful).

It could also use a diskdefs entry for Apple II 3.5" disks. This worked with the (one and only) disk image I found:

# Apple II CP/AM on an 800KB 3.5" disk
diskdef apple-800
seclen 512
tracks 100
sectrk 16
blocksize 2048
maxdir 256
boottrk 2
os 2.2
end

On 8/11/23 8:56 AM, D Finnigan wrote:
> fadden wrote:
>> The CP/M filesystem continues to delight and terrify me.
>>
>
> If computer file systems had been designed by persons with secretarial
> experience, perhaps they'd be less brain-damaged.
>
> Even now it's an arbitrary limitation against more than one file having the
> same file name in a directory, even when there are plenty of practical use
> cases for having multiple files with the same name in a directory with
> differing modification or creation dates.

Heh. What would happen is secretaries would have (root) directories
full of files that all have the same name but differ /only/ by the date.

Or... my favorite. Do you know what HFS allows? TRAILING WHITESPACE.
Do you know what's practically impossible to see? TRAILING WHITESPACE.
Allowing arbitrary crap like spaces, backslashes, whitespace, or
whatever else the secretary barfed on the keyboard is great only if
there is a deterministic way to know a file by another (saner,
automatable, deterministic) handle. Lots of dedicated word processing
systems did exactly that... you have a file with a slug and a date, and
you can barf all you want into the metadata section.

On Sunday, August 13, 2023 at 1:51:20 PM UTC-7, David Schmidt wrote:
> Or... my favorite. Do you know what HFS allows? TRAILING WHITESPACE.

It also allows embedded '\0' bytes. Literally anything but ':'. That's the power of mouse-based file selection. :-)

UNIX allows trailing spaces... but not even the mighty "find -print0 | xargs -0" can handle HFS.

I've learned to love https://en.wikipedia.org/wiki/Control_Pictures . Maybe I should generate \u2423 for trailing spaces. (Those, at least, are much less trouble for computers than they are for humans.)

David Schmidt wrote:
> On 8/11/23 8:56 AM, D Finnigan wrote:
>> fadden wrote:
>>> The CP/M filesystem continues to delight and terrify me.
>>>
>>
>> If computer file systems had been designed by persons with secretarial
>> experience, perhaps they'd be less brain-damaged.
>>
>> Even now it's an arbitrary limitation against more than one file having
>> the
>> same file name in a directory, even when there are plenty of practical
>> use
>> cases for having multiple files with the same name in a directory with
>> differing modification or creation dates.
>
> Heh. What would happen is secretaries would have (root) directories
> full of files that all have the same name but differ /only/ by the date.

Sure why not? Let's say you have a directory of annual reports from 1980 to
1989. They ought to all have the same file name "Annual Report" and
differing dates for each year.

>
> Or... my favorite. Do you know what HFS allows? TRAILING WHITESPACE.
> Do you know what's practically impossible to see? TRAILING WHITESPACE.
> Allowing arbitrary crap like spaces, backslashes, whitespace, or
> whatever else the secretary barfed on the keyboard is great only if
> there is a deterministic way to know a file by another (saner,
> automatable, deterministic) handle. Lots of dedicated word processing
> systems did exactly that... you have a file with a slug and a date, and
> you can barf all you want into the metadata section.
>

A lot of what he have today is self-inflicted damage from choice of
command-line user interface. The classic example being spaces in filenames
when using a CLI with argument switches. The user interface could have been
designed a lot differently.

--
]DF$
The New Apple II User's Guide:
https://macgui.com/newa2guide/

On Sunday, August 13, 2023 at 9:23:15 AM UTC-7, fadden wrote:
> I found the 31:cp/m.sys and 31:cp/am.sys files on some of the disks from Asimov. They have allocation block numbers >= 128, on a disk that only has 128 blocks. For reasons I can't recall, CiderPress displays them and actually mods the block number, so the file is readable.
>
> What purpose does this entry serve? That wasn't clear to me.

I figured it out (data-only disk).

Ugh.

On 8/13/23 5:41 PM, D Finnigan wrote:
[...]
> A lot of what he have today is self-inflicted damage from choice of
> command-line user interface. The classic example being spaces in filenames
> when using a CLI with argument switches. The user interface could have been
> designed a lot differently.

Oh, absolutely. And in some cases, it was. Like Macintosh with
resource and data forks. HPFS with Extended Attributes. "Keep the
metadata with the file," they said. "It will be fun," they said. Your
actual fun may vary.

We eventually reach a point where data needs to be exchanged on things
beyond a gooey screen. Really messy filenames (and metadata stashing
schemes we don't all agree on) don't help.

On 8/13/23 20:02, fadden wrote:
> On Sunday, August 13, 2023 at 9:23:15 AM UTC-7, fadden wrote:
>> I found the 31:cp/m.sys and 31:cp/am.sys files on some of the disks from Asimov. They have allocation block numbers >= 128, on a disk that only has 128 blocks. For reasons I can't recall, CiderPress displays them and actually mods the block number, so the file is readable.
>>
>> What purpose does this entry serve? That wasn't clear to me.
>
> I figured it out (data-only disk).

I'm guessing this was a Premium Softcard IIe "data" diskette?

On Sunday, August 13, 2023 at 7:06:52 PM UTC-7, Steven Hirsch wrote:
> > I figured it out (data-only disk).
> I'm guessing this was a Premium Softcard IIe "data" diskette?

CP/AM seems to follow the same approach. I'm going to assume that all 140KB disks wrap around, and use the directory contents to determine whether or not the first 3 tracks are available.

The CiderPress II disk formatter takes a "make bootable" flag. For CP/M I think that'll mean whether or not to add the magic extent at the start of the directory. I'm undecided on whether to output an OS image to make the disk actually bootable, the way I do for DOS 3.3, because every version of the OS is just a little different. Chapter 3 in the CP/AM manual explains how to put your choice of OS on a floppy disk or other device (four different installers, "COPY /S" from an existing disk, etc.), so I don't think leaving the boot tracks blank will slow anybody down.

Seen in the boot block: "Update to Ver R5.1.1S by Steven Hirsch in 1986" .... anybody we know? :-)

On 8/13/23 23:36, fadden wrote:
> On Sunday, August 13, 2023 at 7:06:52 PM UTC-7, Steven Hirsch wrote:
>>> I figured it out (data-only disk).
>> I'm guessing this was a Premium Softcard IIe "data" diskette?
>
> CP/AM seems to follow the same approach. I'm going to assume that all
> 140KB disks wrap around, and use the directory contents to determine
> whether or not the first 3 tracks are available.

Yup. I wrote most of the CP/AM 5.x operating system and ended up copying that
idea from Microsoft.

> The CiderPress II disk formatter takes a "make bootable" flag. For CP/M I
> think that'll mean whether or not to add the magic extent at the start of
> the directory. I'm undecided on whether to output an OS image to make the
> disk actually bootable, the way I do for DOS 3.3, because every version of
> the OS is just a little different. Chapter 3 in the CP/AM manual explains
> how to put your choice of OS on a floppy disk or other device (four
> different installers, "COPY /S" from an existing disk, etc.), so I don't
> think leaving the boot tracks blank will slow anybody down.

That's probably the best choice. There are too many ways for things to go
wrong if you try to install system tracks without knowledge of the user's
specific hardware environment.

> Seen in the boot block: "Update to Ver R5.1.1S by Steven Hirsch in 1986"
> ... anybody we know? :-)

It was a good gig while it lasted.