I continued experimenting with the Z80-MBC2 and XMODEM.

In addition to Crostini Linux, recent versions of chromeOS support accessing USB serial devices also from Android apps[1], but file transfers freeze like on Linux with the only such terminal emulator that supports XMODEM. On chromeOS it's now possible to control serial USB devices also from web apps[2], but I haven't found any web terminal emulators with an XMODEM transfer feature.

Interestingly, the direct upload to the Z80-MBC2 of executable files in Intel HEX format does work, but requires tweaking the communication parameters[3].

[1] https://journal.paoloamoroso.com/controlling-the-z80-mbc2-from-android-on-chromeos
[2] https://journal.paoloamoroso.com/controlling-the-z80-mbc2-from-the-web
[3] https://journal.paoloamoroso.com/testing-hex-file-upload-to-the-z80-mbc2

Hard to do xmodem file transfer at 115200 with 8MHz Z80 without hardware handshake. So do you have hardware handshake enabled?
Bill

Bill, the Z80-MBC2 developer recommends not enabling hardware handshake as the board presumably doesn't support it, but it's worth giving it a shot.

Bare minimal CP/M BIOS is needed to support XMODEM at 115200 without handshake with 8MHz Z80 so I was curious how Z80-MBC2 does it. According to this post in Hackaday, it is done with bigger receive buffer in Arduino MightyCore. So I don't think the problem is in CP/M XMODEM software, but in the ATMega serial emulation.
https://hackaday.io/project/159973-z80-mbc2-a-4-ics-homebrew-z80-computer/log/160375-new-ios-for-xmodem-support

Bill

On 9/5/22 7:25 PM, Bill Shen wrote:
> Hard to do xmodem file transfer at 115200 with 8MHz Z80 without hardware handshake. So do you have hardware handshake enabled?
> Bill

That board uses an ATMEGA part to do a lot of things including emulating
hardware devices like the serial port. It has a buffer for received data
and its size can be selected at compile time. I don't think it even
tries for flow control in its firmware.

It has been a while since this thread was active but as I recall the
buffer size for this particular version is large enough to hold an
XMODEM packet. The data, not any overhead like escaped characters. (128
bytes) Assuming it can dribble out data to the Z80 (over a parallel
interface) while receiving it, that shouldn't be a problem.

If it can manage that, there should be no more data arriving until the
receiver sends an ACK.

But there is that little detail of using a single serial link for both
control and data.

--
http://davesrocketworks.com
David Schultz

On Tuesday, September 6, 2022 at 2:25:14 AM UTC+2, Bill Shen wrote:
> Hard to do xmodem file transfer at 115200 with 8MHz Z80 without hardware handshake. So do you have hardware handshake enabled?

Enabling hardware handshake hangs the terminal session and software handshake changes nothing in XMODEM transfers. The Z80-MBC2 developer recommends no handshake because, as he notes, the board's Arduino-derived chipset doesn't support hardware flow control.

On Sunday, September 11, 2022 at 9:18:39 AM UTC-4, paolo....@gmail.com wrote:
> On Tuesday, September 6, 2022 at 2:25:14 AM UTC+2, Bill Shen wrote:
> > Hard to do xmodem file transfer at 115200 with 8MHz Z80 without hardware handshake. So do you have hardware handshake enabled?
> Enabling hardware handshake hangs the terminal session and software handshake changes nothing in XMODEM transfers. The Z80-MBC2 developer recommends no handshake because, as he notes, the board's Arduino-derived chipset doesn't support hardware flow control.

I'm having a similar problem on a different target. XModem/CRC works great from Windows using TeraTerm but fails immediately with sx on linux. An interesting thing that may point in the right direction is that at 9600bps, the transfer proceeds although very slowly due to a retry between each packet (Retry 0: NAK on sector)...however the file does transfer. As you note, minicom and most other linux serial comms programs just call sx to do the transfer. Unfortunately, it looks like sx (lrzsz) is broken for xmodem and has been for a very long time. Oddly there doesn't seem to be a replacement...it might be worth writing one.

On Saturday, January 14, 2023 at 6:03:09 AM UTC+1, David Albert wrote:
> I'm having a similar problem on a different target. XModem/CRC works great from Windows using TeraTerm but fails immediately with sx on linux. An interesting thing that may point in the right direction is that at 9600bps, the transfer proceeds although very slowly due to a retry between each packet (Retry 0: NAK on sector)...however the file does transfer. As you note, minicom and most other linux serial comms programs just call sx to do the transfer. Unfortunately, it looks like sx (lrzsz) is broken for xmodem and has been for a very long time. Oddly there doesn't seem to be a replacement....it might be worth writing one.

What device are you trying to transfer files to? A Z80-MBC2?

On 1/13/23 11:03 PM, David Albert wrote:
> Unfortunately, it
> looks like sx (lrzsz) is broken for xmodem and has been for a very
> long time. Oddly there doesn't seem to be a replacement...it might
> be worth writing one.
>
From what I can see it hasn't been updated for a very long time yet
there is still a package for Fedora.

Before writing a replacement, it might be a better idea to figure out
what is wrong with the current version and fix it. I dug up something
resembling the Linux source (https://github.com/UweOhse/lrzsz).
Comparing that to an ancient version I found on my old CP/M-68K disks
shows a lot has remained the same:

#define VERSION "sz 1.22 01-01-87"
#define PUBDIR "/usr/spool/uucppublic"

/*% cc -O -K -i -DCRCTABLE -DREADCHECK sz.c -lx -o sz; size sz

* sz.c By Chuck Forsberg
*
* cc -O sz.c -o sz USG (SYS III/V) Unix
* cc -O -DSVR2 sz.c -o sz Sys V Release 2 with
non-blocking input
* Define to allow reverse channel
checking
* cc -O -DV7 sz.c -o sz Unix Version 7, 2.8 - 4.3 BSD
*

--
http://davesrocketworks.com
David Schultz

On Friday, January 13, 2023 at 9:03:09 PM UTC-8, David Albert wrote:
......<snip>.....
>> As you note, minicom and most other linux serial comms programs just call sx to do the transfer.
>> Unfortunately, it looks like sx (lrzsz) is broken for xmodem and has been for a very long time.
>> Oddly there doesn't seem to be a replacement...it might be worth writing one.

Hmmmm.... Interesting. I use the xmodem transfer protocol in minicom all the time, and it works very well.

I'm running (primarily) SuSE Linux, but also use Slackware. Never had any problems with it.

Roger

On 1/13/23 11:03 PM, David Albert wrote:
> I'm having a similar problem on a different target. XModem/CRC works great from Windows using TeraTerm but fails immediately with sx on linux. An interesting thing that may point in the right direction is that at 9600bps, the transfer proceeds although very slowly due to a retry between each packet (Retry 0: NAK on sector)
>
Reading that old lrzsz code is frustrating. I hate it when code is
poorly commented. Or almost not at all.

The XMODEM protocol mostly resides in the function wcputsec(). (Why that
name?) Dead simple and nothing that can go wrong. It sends data and
waits for an ACK.

But it was written when serial ports were serial ports. So makes some
assumptions. One of which is that the target is not echoing what it
receives. If a special character (ACK, NAK, CAN, etc.) appears in the
data, it will upset things. This is where TeraTerm could differ. But
given all the buffering that goes on (kernel, USB to serial, etc.) I can
think of no easy way to compensate for a target that echoes everything
it receives.

A CP/M target can be a problem if the data passes through the BDOS
console I/O routines.

The precise reason why the target sent a NAK would be helpful. Timeout?
bad checksum/CRC?

--
http://davesrocketworks.com
David Schultz

On Sunday, 15 January 2023 at 15:39:33 UTC, David Schultz wrote:
> On 1/13/23 11:03 PM, David Albert wrote:
> > I'm having a similar problem on a different target. XModem/CRC works great from Windows using TeraTerm but fails immediately with sx on linux. An interesting thing that may point in the right direction is that at 9600bps, the transfer proceeds although very slowly due to a retry between each packet (Retry 0: NAK on sector)
> >
> Reading that old lrzsz code is frustrating. I hate it when code is
> poorly commented. Or almost not at all.
>
> The XMODEM protocol mostly resides in the function wcputsec(). (Why that
> name?) Dead simple and nothing that can go wrong. It sends data and
> waits for an ACK.
>
> But it was written when serial ports were serial ports. So makes some
> assumptions. One of which is that the target is not echoing what it
> receives. If a special character (ACK, NAK, CAN, etc.) appears in the
> data, it will upset things. This is where TeraTerm could differ. But
> given all the buffering that goes on (kernel, USB to serial, etc.) I can
> think of no easy way to compensate for a target that echoes everything
> it receives.
>
> A CP/M target can be a problem if the data passes through the BDOS
> console I/O routines.
>
> The precise reason why the target sent a NAK would be helpful. Timeout?
> bad checksum/CRC?
> --
> http://davesrocketworks.com
> David Schultz
David
Have you tried the python implementation of XMODEM rather than lrzsz?
https://pypi.org/project/xmodem/ has links to the documentation and download files.
Mark Ogden

On Tuesday, 17 January 2023 at 11:44:25 UTC, Mark Ogden wrote:
> On Sunday, 15 January 2023 at 15:39:33 UTC, David Schultz wrote:
> > On 1/13/23 11:03 PM, David Albert wrote:
> > > I'm having a similar problem on a different target. XModem/CRC works great from Windows using TeraTerm but fails immediately with sx on linux. An interesting thing that may point in the right direction is that at 9600bps, the transfer proceeds although very slowly due to a retry between each packet (Retry 0: NAK on sector)
> > >
> > Reading that old lrzsz code is frustrating. I hate it when code is
> > poorly commented. Or almost not at all.
> >
> > The XMODEM protocol mostly resides in the function wcputsec(). (Why that
> > name?) Dead simple and nothing that can go wrong. It sends data and
> > waits for an ACK.
> >
> > But it was written when serial ports were serial ports. So makes some
> > assumptions. One of which is that the target is not echoing what it
> > receives. If a special character (ACK, NAK, CAN, etc.) appears in the
> > data, it will upset things. This is where TeraTerm could differ. But
> > given all the buffering that goes on (kernel, USB to serial, etc.) I can
> > think of no easy way to compensate for a target that echoes everything
> > it receives.
> >
> > A CP/M target can be a problem if the data passes through the BDOS
> > console I/O routines.
> >
> > The precise reason why the target sent a NAK would be helpful. Timeout?
> > bad checksum/CRC?
> > --
> > http://davesrocketworks.com
> > David Schultz
> David
> Have you tried the python implementation of XMODEM rather than lrzsz?
> https://pypi.org/project/xmodem/ has links to the documentation and download files.
> Mark Ogden
David
There is also the newer rzsz at https://github.com/coderfordev/rzsz
Mark

On 1/17/23 5:44 AM, Mark Ogden wrote:

> David
> Have you tried the python implementation of XMODEM rather than lrzsz?
> https://pypi.org/project/xmodem/ has links to the documentation and download files.
> Mark Ogden

Actually, I haven't used XMODEM in a very long time.

I got to looking and I do see a problem with its use with minicom.

The way XMODEM worked is that you dialed into a BBS or some such, told
it to send you a file, and then started your XMODEM receive program.
That sent a NAK to tell the sender it was ready.

Using it with minicom to send a file to a SBC is a problem. You tell
minicom to start XMODEM, then it tells you to give the command to start
receiving. Except you can't since XMODEM is now using that serial link.

So you have to start the XMODEM receive program before telling minicom
to start sending. This is a problem. (Of course with a short hardwired
connection like this, XMODEM isn't needed at all.)

The receive program sends a NAK to tell the sending program it is ready.
But since lrzsz hasn't started yet, nothing is listening. After a
timeout the receive program sends another NAK.

So when sz starts, more than one NAK can be buffered up waiting for it.
It consumes the first and sends the first packet. It then looks for an
ACK but finds one of those leftover NAKs.

I think that you could make lsz a little more tolerant of this. When it
reads data it uses a read() call and buffers extra characters. The code
includes a function to purge that buffer but it is never called. Adding
a call to readline_purge() at an appropriate point (after getting that
initial NAK would be good) would dump that extra data.

Calling it just before starting to send each packet might be excessive.
But there shouldn't be anything to purge.

--
http://davesrocketworks.com
David Schultz

I used minicom/lrzsz a lot with the z80pack machines. The virtual SIO's won't overrun like a real one
and buffers all and everything, but I have not seen this problem.

On Tuesday, January 17, 2023 at 1:16:47 PM UTC-5, David Schultz wrote:
> On 1/17/23 5:44 AM, Mark Ogden wrote:
>
> > David
> > Have you tried the python implementation of XMODEM rather than lrzsz?
> > https://pypi.org/project/xmodem/ has links to the documentation and download files.
> > Mark Ogden
> Actually, I haven't used XMODEM in a very long time.
>
> I got to looking and I do see a problem with its use with minicom.
>
> The way XMODEM worked is that you dialed into a BBS or some such, told
> it to send you a file, and then started your XMODEM receive program.
> That sent a NAK to tell the sender it was ready.
>
> Using it with minicom to send a file to a SBC is a problem. You tell
> minicom to start XMODEM, then it tells you to give the command to start
> receiving. Except you can't since XMODEM is now using that serial link.
>
> So you have to start the XMODEM receive program before telling minicom
> to start sending. This is a problem. (Of course with a short hardwired
> connection like this, XMODEM isn't needed at all.)
>
> The receive program sends a NAK to tell the sending program it is ready.
> But since lrzsz hasn't started yet, nothing is listening. After a
> timeout the receive program sends another NAK.
>
> So when sz starts, more than one NAK can be buffered up waiting for it.
> It consumes the first and sends the first packet. It then looks for an
> ACK but finds one of those leftover NAKs.
>
> I think that you could make lsz a little more tolerant of this. When it
> reads data it uses a read() call and buffers extra characters. The code
> includes a function to purge that buffer but it is never called. Adding
> a call to readline_purge() at an appropriate point (after getting that
> initial NAK would be good) would dump that extra data.
>
> Calling it just before starting to send each packet might be excessive.
> But there shouldn't be anything to purge.
> --
> http://davesrocketworks.com
> David Schultz

Um... lrz.c line 762 and 645... should not be a problem. Call is to purgeline()
which first does readline_purge() and then actually purges... via ioctl() or lseek()

so, I think, good to go. Never been a problem for me, anyway -- I do not think there is
bug in THOSE particular hills.

On 1/17/23 1:36 PM, Udo Munk wrote:
> I used minicom/lrzsz a lot with the z80pack machines. The virtual SIO's won't overrun like a real one
> and buffers all and everything, but I have not seen this problem.

The virtual SIO isn't the problem.

You start up the XMODEM receive program on the target and it sends a
NAK. But you haven't started the send side yet so the character doesn't
make it to lsz. Most likely minicom sees it and it vanishes.

Minicom eventually (the menu system is a bit clunky) starts up lsz which
looks for a NAK. Depending on how long of a delay there was between
starting the receiver, its timeout behaviour, and buffering of the
serial input on the Linux side, there could be more than one NAK sitting
in the queue. Is minicom looking at the serial port while you are in the
menu system trying to start lsz? Doubtful.

Of course if you take too long starting up lsz the receive program will
reach its retry limit and give up.

--
http://davesrocketworks.com
David Schultz

On 1/17/23 2:07 PM, fridtjof.ma...@gmail.com wrote:

> Um... lrz.c line 762 and 645... should not be a problem. Call is to purgeline()
> which first does readline_purge() and then actually purges... via ioctl() or lseek()
>
Which effects lsz.c not at all.

There are calls to purgeline() in lsz.c but as near as I can tell, only
in code active for ZMODEM.

> so, I think, good to go. Never been a problem for me, anyway -- I do not think there is
> bug in THOSE particular hills.
>

It isn't a bug so much as it is using the tool in a way it wasn't
designed for. You are supposed to start the receiver after the sender is
running.

--
http://davesrocketworks.com
David Schultz

David Schultz schrieb am Dienstag, 17. Januar 2023 um 23:32:50 UTC+1:
> It isn't a bug so much as it is using the tool in a way it wasn't
> designed for. You are supposed to start the receiver after the sender is
> running.

There are lots of devices where one can update firmware using xmodem protocol.
These all have one in common, you connect with a terminal to the device and start
the receiver, and then on the PC you start the sender with the firmware file.
Always works.

Forgot, if you start the receiver from the terminal you will see the NAK the receiver sends,
so this one is eaten up by the terminal.

On 1/17/23 18:03, Udo Munk wrote:
> Forgot, if you start the receiver from the terminal you will see the NAK the receiver sends,
> so this one is eaten up by the terminal.

I stopped having an of these problems when Kermit showed up. :-)

bill

> It isn't a bug so much as it is using the tool in a way it wasn't
> designed for. You are supposed to start the receiver after the sender is
> running.

You _can't_ start the receiver after starting the sender in an upload.

Seems like sx should flush the buffer either immediately at start, or
at least after receiving a NAK.

But it's still an open question whether this is the problem experience
when talking to a MyZ80, I believe.

De

Bill Gunshannon schrieb am Mittwoch, 18. Januar 2023 um 00:54:36 UTC+1:
> I stopped having an of these problems when Kermit showed up. :-)

It isn't really a problem, is normal that the terminal shows a bit garbage when you start
a xmodem transmission. Whatever, of course you can start the receiver first and then the
sender to get the transmission going, doesn't matter.

Had some time to try this stuff again, on CP/M I'm using Martin Eberhardt's xmodem.
You really have to start transceiver first with minicom/lrzsz, if you first start xmodem on
the CP/M side the transfer will not start and times out after a while.

On Wednesday, January 18, 2023 at 1:48:23 PM UTC+1, Udo Munk wrote:
> Had some time to try this stuff again, on CP/M I'm using Martin Eberhardt's xmodem.
> You really have to start transceiver first with minicom/lrzsz, if you first start xmodem on
> the CP/M side the transfer will not start and times out after a while.

If you have to send a file from Minicom to CP/M, how do you start the transceiver first with Minicom/lrzsz?