So far I have no fix for the issue, but I've figured and tested an effective workaround: pasting a text file from Minicom to an ED session on CP/M, i.e. sending the contents of the file as if it would be typed in, then saving the file from ED. This requires setting at least a 5 ms character transmit delay in Minicom's terminal settings (^A T F). It's slower than XMODEM, but still acceptable and sufficiently practical.

The workaround lets me send HEX files for transferring CP/M programs, uuencoded (or similar) files for other binary, non-excutable files.

On 6/27/22 2:23 PM, Paolo Amoroso wrote:
> So far I have no fix for the issue, but I've figured and tested an effective workaround: pasting a text file from Minicom to an ED session on CP/M, i.e. sending the contents of the file as if it would be typed in, then saving the file from ED. This requires setting at least a 5 ms character transmit delay in Minicom's terminal settings (^A T F). It's slower than XMODEM, but still acceptable and sufficiently practical.
>
> The workaround lets me send HEX files for transferring CP/M programs, uuencoded (or similar) files for other binary, non-excutable files.

Since you can successfully send a very short file with XMODEM but not a
longer one makes me suspect an inability to keep up with incoming data.
So a lower bit rate would be the first thing to try.

Looking at the link to the blog which talks about XMODEM support this
would seem to fit into that category of can't receive. It also mentions
that besides getting the new version of software you need to follow a
special procedure to increase the receive buffer to 128 bytes.

I suspect even that would fail if the XMODEM-1K variant were used.

--
http://davesrocketworks.com
David Schultz

On Monday, June 27, 2022 at 10:23:26 PM UTC+2, David Schultz wrote:
> So a lower bit rate would be the first thing to try.

As far as I know the Z80-MBC2 supports only 115200 bps. When I tried 9600 bps I got garbage in Minicom, but it's worth trying again.

> Looking at the link to the blog which talks about XMODEM support this
> would seem to fit into that category of can't receive. It also mentions
> that besides getting the new version of software you need to follow a
> special procedure to increase the receive buffer to 128 bytes.

My understanding is that special procedure is part of rebuilding the firmware and no further action is required. The board's developed assured me my unit has the latest, fixed XMODEM setup as confirmed by the boot message "IOS: Found extended serial Rx buffer".

> Since you can successfully send a very short file with XMODEM but not a
> longer one makes me suspect an inability to keep up with incoming data.
> So a lower bit rate would be the first thing to try.

Except that it seems not even the first packet gets through. If it was
related to keeping up, it ought to move at least one packet before
timing out.

De

On Monday, June 27, 2022 at 10:23:26 PM UTC+2, David Schultz wrote:
> So a lower bit rate would be the first thing to try.

I checked the board's documentation and did a few tests. My Z80-MBC2 unit, which runs the IOS firmware, works only at 115200 bps. Any other bit rate generates garbage in the terminal and the firmware has no settings or other ways of changing the bit rate. Another build of the firmware, IOS Lite, is designed for Z80-MBC2 boards without the optional components (e.g. a microSD card reader) and supports 9600 bps.

On 6/27/22 10:07 PM, Dennis Boone wrote:
> > Since you can successfully send a very short file with XMODEM but not a
> > longer one makes me suspect an inability to keep up with incoming data.
> > So a lower bit rate would be the first thing to try.
>
> Except that it seems not even the first packet gets through. If it was
> related to keeping up, it ought to move at least one packet before
> timing out.
>
> De
No. With the XMODEM protocol, the transmitter waits for an ACK/NACK
after each packet. So any failure to keep up with incoming data will
manifest itself within a packet.

I dug up some version of the firmware which has comments about a fix to
allow XMODEM to work. This makes it mostly clear what is going on.

The ATMEGA is receiving the serial data and buffering it using the basic
code from the Arduino library. I hope that uses interrupts because if it
doesn't there is no hope.

The main loop is just the ATMEGA fielding I/O requests from the Z80. It
appears to have the ability to stop the Z80 while that happens. So if
the serial receive isn't interrupt driven, it could drop characters
while this is going on.

With at least 128 bytes allocated for the serial buffer there should be
no problems with XMODEM. Assuming the Arduino firmware can walk and chew
gum.

Another complication would be on the XMODEM receive side. It is critical
that this code perform any disk I/O after it validates an incoming
packet but before it sends an ACK. Reverse that order and the sender
begins sending the next packet while the disk I/O is going on. Since
that is also simulated using the ATMEGA, that would cause trouble.
Arduino code isn't exactly real time friendly.

--
http://davesrocketworks.com
David Schultz

Concerning the ability of the Z80-MBC2 to keep up with file transfers, the developer noted he can reliably transfer large files to CP/M via XMODEM, for example sending 200-500 Kb from Linux and Windows.

> > Except that it seems not even the first packet gets through. If it was
> > related to keeping up, it ought to move at least one packet before
> > timing out.

> No. With the XMODEM protocol, the transmitter waits for an ACK/NACK
> after each packet. So any failure to keep up with incoming data will
> manifest itself within a packet.

Paolo has been able to move small files, and the board is known to
be able do large file transfers successfully. There's something else
going on here. Data-dependent weirdness? Something about Paolo's
USB serial dongle or Chromebook environment?

De

On Thursday, June 30, 2022 at 4:03:52 PM UTC+2, Dennis Boone wrote:
> Paolo has been able to move small files, and the board is known to
> be able do large file transfers successfully. There's something else
> going on here. Data-dependent weirdness? Something about Paolo's
> USB serial dongle or Chromebook environment?

It's quite possible the issue is at the chromeOS/Crostini end, for example local buffering problems.

Call me crazy but at this point I would be working on repacking what to transfer into packets that work as is.

A lot of work into what sounds like a minor problem.

USB to serial adapters can be bought for just a few bucks from Amazon/ePay etc. Try a couple different adapters, I would also consider finding a different communication package that will work.

Try package from Chromebook to PC and PC to CPM so you have a general idea where the problem is.

Randy

On Thursday, June 30, 2022 at 5:06:19 PM UTC+2, Randy McLaughlin wrote:
> USB to serial adapters can be bought for just a few bucks from Amazon/ePay etc. Try a couple different adapters, I would also consider finding a different communication package that will work.

Good idea, thanks.

On 6/30/22 9:03 AM, Dennis Boone wrote:
> > > Except that it seems not even the first packet gets through. If it was
> > > related to keeping up, it ought to move at least one packet before
> > > timing out.
>
> > No. With the XMODEM protocol, the transmitter waits for an ACK/NACK
> > after each packet. So any failure to keep up with incoming data will
> > manifest itself within a packet.
>
> Paolo has been able to move small files, and the board is known to
> be able do large file transfers successfully. There's something else
> going on here. Data-dependent weirdness? Something about Paolo's
> USB serial dongle or Chromebook environment?
>
> De
I looked at the source code for sz to find out that the default timeout
period used is 60 seconds. So there should be no problem at all with it
waiting long enough for a reply from the target.

So the problem would appear to be within the target. Somehow it is
getting confused and not responding. The simplest way for that to happen
is for it to drop characters.

Hopefully the channel is 8 bit clean or close enough to it. sz escapes
some characters including problematic things like XON and XOFF. Options
are provided to do more of course. "-e" escapes all control characters.

The command "sz -TT" is supposed to send all 256 8 bit codes as an aid
in figuring out which ones aren't making it through.

--
http://davesrocketworks.com
David Schultz

On Thursday, June 30, 2022 at 5:06:19 PM UTC+2, Randy McLaughlin wrote:
> USB to serial adapters can be bought for just a few bucks from Amazon/ePay etc. Try a couple different adapters, I would also consider finding a different communication package that will work.

I tried another terminal emulator, Picocom, but sending files still times out. However, both Minicom and Picocom run the external sx program for file transfers, so I should get a different XMODEM implementation.

On Thursday, June 30, 2022 at 6:27:20 PM UTC+2, David Schultz wrote:
> Hopefully the channel is 8 bit clean or close enough to it. sz escapes
> some characters including problematic things like XON and XOFF. Options
> are provided to do more of course. "-e" escapes all control characters.
>
> The command "sz -TT" is supposed to send all 256 8 bit codes as an aid
> in figuring out which ones aren't making it through.

I tried running sx with the options -e and -TT, still timeouts.

On 6/30/22 1:07 PM, Paolo Amoroso wrote:
> On Thursday, June 30, 2022 at 5:06:19 PM UTC+2, Randy McLaughlin wrote:
>> USB to serial adapters can be bought for just a few bucks from Amazon/ePay etc. Try a couple different adapters, I would also consider finding a different communication package that will work.
>
> I tried another terminal emulator, Picocom, but sending files still times out. However, both Minicom and Picocom run the external sx program for file transfers, so I should get a different XMODEM implementation.
It is unlikely to be the XMODEM implementation on the sending side. What
have you done to check the receiver?

Since the sender is timing out waiting for an ACK, using a very long
timer, something is preventing the receiver from even sending a NACK.
What is that?

--
http://davesrocketworks.com
David Schultz

On Thursday, June 30, 2022 at 10:40:44 PM UTC+2, David Schultz wrote:
> It is unlikely to be the XMODEM implementation on the sending side. What
> have you done to check the receiver?

On CP/M I'm using version 2.7 of Martin Eberhard's XMODEM implementation patched[1] for the Z80-MBC2. I'll try the unpatched version 2.9, which also comes with a program for testing the serial link.

[1] https://groups.google.com/g/comp.os.cpm/c/oFsZ6PcsCkM/m/egABUI4-AgAJ

On Friday, July 1, 2022 at 9:06:16 AM UTC+2, Paolo Amoroso wrote:
> On CP/M I'm using version 2.7 of Martin Eberhard's XMODEM implementation patched[1] for the Z80-MBC2. I'll try the unpatched version 2.9, which also comes with a program for testing the serial link.

I tried both programs and XMODEM timed out. The first time I run the serial link testing program XMTERM, some interesting log messages whizzed by but I was unable to capture them. Subsequent runs displayed less useful and partly garbled messages such as:

B>xmterm greet.hex /r

========================
XMTERM 1.1 By M.Eberhard
========================

/In 8080 codet^G6##w######rI

Or like these:

B>xmterm greet.hex /r

========================
XMTERM 1.1 By M.Eberhard
========================

/H

Back to the drawing board, I'll run the program again and see what I come up with.

On 7/1/22 1:59 PM, Paolo Amoroso wrote:
> On Friday, July 1, 2022 at 9:06:16 AM UTC+2, Paolo Amoroso wrote:
>> On CP/M I'm using version 2.7 of Martin Eberhard's XMODEM implementation patched[1] for the Z80-MBC2. I'll try the unpatched version 2.9, which also comes with a program for testing the serial link.
>
> I tried both programs and XMODEM timed out. The first time I run the serial link testing program XMTERM, some interesting log messages whizzed by but I was unable to capture them. Subsequent runs displayed less useful and partly garbled messages such as:
>
> B>xmterm greet.hex /r
>
> ========================
> XMTERM 1.1 By M.Eberhard
> ========================
>
> /In 8080 codet^G6##w######rI
>
I looked at the source for XMTERM and that is part of a usage message
printed just before exit. So it didn't like something. Perhaps in your
configuration file.

Or maybe that you provided a file name that is useless to this program.
Which just echos characters. Which is going to be less than useful when
the XMODEM port is the same as the console.

--
http://davesrocketworks.com
David Schultz

I run XMTERM with various options but didn't get much output, for example:

B>xmterm test.hex /x1 /e /r

========================
XMTERM 1.1 By M.Eberhard
========================

But I got something interesting when running XMODEM like this on CP/M with greet.hex sent from Linux (I named it test.hex on CP/M):

B>xmodem test.hex /r

After the usual timeouts, in the Minicom file transfer dialog I got several lines of quickly scrolling messages and managed to capture only the last few ones:

+----------------[xmodem upload - Press CTRL-C to quit]----------------+
|Retry 0: Got 32 for sector ACK |
|Retry 0: NAK on sector |
|Retry 0: Retry Count Exceeded |
| |
|Transfer incomplete |
| |
| READY: press any key to continue...

I pressed a key and got this in the terminal window, which is likely leftover garbage:

743C13901011075637D560?
B>0901CD0500C94772656574696E67732066726F6D205375697465383038CC
0901CD0500C94772656574696E67732066726F6D205375697465383038CC?

On 7/2/22 8:29 AM, Paolo Amoroso wrote:
> I run XMTERM with various options but didn't get much output, for example:

And you never will unless you use different serial ports for the console
and xmodem. The xmterm output is going back to sz. Which eats it looking
for a NAK or ACK. That and I am not sure it sends the XMODEM start
character that sz is waiting for. But I am not diving back into the
source code to find out.

I see that there is an add on serial card which you probably need to
get. The alternative is to write a special purpose xmodem receive
program that logs all of its messages somewhere besides the console
port. A disk file perhaps or maybe RAM.

As near as I can tell the xmodem program you are using is optimized for
space (as little as possible) and speed. It uses all of the TPA it can
for a buffer to minimize disk accesses. Useful for floppy drives, less
so for hard drives, and not at all for anything that doesn't have long
start up or seek times.

--
http://davesrocketworks.com
David Schultz

On 7/2/22 8:29 AM, Paolo Amoroso wrote:
> I run XMTERM with various options but didn't get much output, for example:
>

XMTERM ignores a lot of options that make sense for XMODEM but it does
parse your XMODEM configuration file. Instead of trying to use sz to
send stuff to XMTERM, where sz will just eat the reply without you ever
seeing it, just use minicom.

As in start XMTERM (no options!) and then type stuff into minicom and
see what XMTERM echoes. Sending the termination character when you are done.

--
http://davesrocketworks.com
David Schultz

On Saturday, July 2, 2022 at 5:26:33 PM UTC+2, David Schultz wrote:
> And you never will unless you use different serial ports for the console
> and xmodem. The xmterm output is going back to sz. Which eats it looking

Isn't the /X1 option I run XMTERM with supposed to take care of this?

> I see that there is an add on serial card which you probably need to
> get. The alternative is to write a special purpose xmodem receive

Do you mean the uCom[1] board? I'd probably also need some way of connecting the RS-232 port to the USB port of my Chromebox, but this would complicate the simplicity of the Z80-MBC2 setup and there may not be any guarantee XMODEM would work. If I can't fix XMODEM, the workaround I described is acceptable for me and doesn't need additional hardware.

[1] https://hackaday.io/project/159973-z80-mbc2-a-4-ics-homebrew-z80-computer/log/164037-ucom-is-out

On 7/2/22 11:53 AM, Paolo Amoroso wrote:
> On Saturday, July 2, 2022 at 5:26:33 PM UTC+2, David Schultz wrote:
>> And you never will unless you use different serial ports for the console
>> and xmodem. The xmterm output is going back to sz. Which eats it looking
>
> Isn't the /X1 option I run XMTERM with supposed to take care of this?
>
You still have just the one serial port and sz is on the other end.

--
http://davesrocketworks.com
David Schultz

On 6/30/22 1:08 PM, Paolo Amoroso wrote:

> I tried running sx with the options -e and -TT, still timeouts.

Have you ever used the quiet mode xmodem option? (/Q)

This prevents it from printing a progress character to the console and
looking for a ^C.

Not a good thing to mix in with the xmodem traffic.

--
http://davesrocketworks.com
David Schultz

On Saturday, July 2, 2022 at 6:52:40 PM UTC+2, David Schultz wrote:
> As in start XMTERM (no options!) and then type stuff into minicom and
> see what XMTERM echoes. Sending the termination character when you are done.

Thanks, it looks like I misunderstood how XMTERM works. I thought it was an instrumented version of XMODEM that does file transfers and displays additional diagnostic info. Instead, I seem to understand it's a terminal-like program that essentially echoes the input typed interactively.

Running XMTERM on CP/M with no arguments and no XMODEM.CFG displays only the signon message:

========================
XMTERM 1.1 By M.Eberhard
========================

When I type, nothing happens. Pressing ^Z or ^C doesn't do anything either. With XMODEM.CFG containing /X1 /E /Q, running XMTERM produces multiple copies of the signon message and, again, typing or pressing ^Z/^C does nothing:

========================
XMTERM 1.1 By M.Eberhard
========================

========================
XMTERM 1.1 By M.Eberhard
========================

========================
XMTERM 1.1 By M.Eberhard
========================

========================
XMTERM 1.1 By M.Eberhard
========================

========================
XMTERM 1.1 By M.Eberhard
========================

========================
XMTERM 1.1 By M.Eberhard
========================

In another post you mentioned quiet mode. I tried it at both ends of a file transfer, XMODEM /X1 /Q /R on CP/M and sx -q on Linux, but still got a timeout.