AppendPatch.sdk (Shrinkit DOS 3.3 disk, 12346
bytes)
AppendPatch.zip (ZIPped disk image, 9691
bytes)LLX >
Neil Parker >
Apple II > Fixing APPEND
APPEND in DOS 3.3For most of its life, Apple's DOS has had problems with its
APPEND command. It's supposed to open a text file and leave
it positioned at the end of the file, so that subsequent WRITEs
will add new data after the existing data. But because DOS doesn't store
an end-of-file pointer on disk, finding the end of the file can only be done
by reading through the file one byte at a time until the end is reached.
This has proven to be a surprisingly difficult procedure to get right:
Each of these fixes costs memory. The code that implements Apple's last two fixes overflows into a couple of previously-unoccupied memory areas inside DOS—memory that many third-party programs had been using to install their own modifications to DOS. The potential for disaster here should be obvious.
All three layers of Apple's fixes are applied to the APPEND
code in DOS's outer section, the command interpreter, or to the command
interpreter's error-handlng routines (in particular, the handling of the
END OF DATA error). None of them could be called "beautiful", and the 1983
fixes resort to an especially bizarre contortion.
So is there a better way to do it?
Yes. As we'll see below, the problem actually originates not in the
command interpreter where the APPEND command resides, but in
DOS's middle section, the file manager, and a fix at the file manager level
turns out to be quite a bit smaller and simpler than Apple's fixes.
Replacing Apple's fixes with a file manager fix not only results in an
APPEND that suffers from none of its traditional tribulations,
it also gives back all of the memory consumed in the 1983 releases and then
some—DOS's internal free spaces are bigger than ever. Plus, putting
APPEND in the file manager makes it faster—the
(expensive) file manager entry/exit overhead is incurred only once, rather
than for every byte examined in the search for the end of the file.
If you care about the technical details, read on. If all you want is
results (and if you trust me to have gotten it right), scroll to the bottom,
where you can download a disk image with programs that patch DOS with
the new, improved APPEND.
In preparing what follows, I've relied heavily on the classic DOS reference, Beneath Apple DOS by Don Worth and Pieter Lechner, and on the final DOS source code, rescued from oblivion by David T. Craig and findable on the usual Apple II download sites. Several snippets of assembly language below are taken from an assembly listing made from this source code—the addresses in the listing differ from those in Beneath Apple DOS because the source code assembles a "master" DOS that loads from $1B00 to $3FFF and then relocates itself. Addresses can be matched with normal 48K DOS, and with Beneath Apple DOS, by adding $8000.
Without further ado, here's the smoking gun:
2CA8: 230 ; 2CA8: 231 * GETBYT - GET A DATA BYTE 2CA8: 232 ; 2CA8: 2CA8 233 GETBYT EQU * 2CA8:20 B6 30 234 JSR LOCNXB ; LOCATE NEXT BYTE 2CAB:B0 0B 2CB8 235 BCS EOFIN ; BR IF EOF 2CAD:B1 42 236 LDA (ZPGFCB),Y ; GET DAT BYTE 2CAF:48 237 PHA ; SAVE IT 2CB0:20 5B 31 238 JSR INCRRB ; INCR REC BYTE 2CB3:20 94 31 239 JSR INCSCB ; INCR SEC BYTE 2CB6:68 240 PLA ; GET SAVED BYTE 2CB7:60 241 RTS ; RETURN 2CB8: 242 ; 2CB8:4C 6F 33 243 EOFIN JMP ERROR5 ; GO TO EOF RTN
This routine is part of DOS's file manager, and is responsible for reading
the next byte from a file. All bytes read from a DOS file pass through
this routine, including APPEND's search for the file's first
zero byte or unallocated sector.
The routine LOCNXB checks whether the next byte to be
accessed is in the current data sector, and if not, searches the file's
track/sector list to find and load the correct sector. If the sector
exists, it returns with the carry flag clear and the offset to the data byte
in the Y register. If the sector doesn't exist, it return with the carry
set. INCRRB advances the record/byte-within-record pointer,
and INCSCB advances the sector/byte-within-sector pointer.
The logic here is simple. It tries to get the next byte from the file,
and if it can't because there are no more sectors in the file, it exits the
file manager with an END OF DATA error. Otherwise it advances the
position-in-file pointers and returns the data byte (ZPGFCB
holds a pointer to the data sector buffer).
Note the exit conditions. After success, the accumulator holds the byte from the file, and the position-in-file pointers are pointing to the byte after it. But after END OF DATA, no byte is returned, and the position-in-file pointers are still pointing at what would be the first byte of the nonexistant sector if it existed.
This behaviour is right (or at least reasonable) for almost everything
else that DOS does, but it's poison for APPEND. Remember,
APPEND is searching for either a zero byte or
END OF DATA (no more sectors). The position-in-file pointers are correct
after END OF DATA, but if a zero byte was read, the pointers are pointing
to the byte after the zero byte, and need to be moved back so the
next write can overwrite the zero byte instead of the byte after it.
It's that backward move, which sometimes needs to be done and and
sometimes not, that's at the root of APPEND's woes.
APPENDed.APPEND flag can't carry over to a different file, and does its
own calculation of the new file position instead of calling the
random-access positioner. But it only adjusts the low-order byte of the
three-byte sector/byte-within-sector pointer. (The random-access
positioner in this version has been patched to handle record numbers up
to 65,535, but that no longer affects APPEND.)All of this grief could be avoided by makng APPEND (and
only APPEND) bypass the above routine completely, and
instead scan for the end of the file with a loop something like this:
SCANTOEOF JSR LOCNXB
BCS FOUNDEOF
LDA (ZPGFCB),Y
BEQ FOUNDEOF
JSR INCRRB
JSR INCSCB
BNE SCANTOEOF ;(Always taken)
FOUNDEOF ...
When the label FOUNDEOF is reached, either a zero byte has
been seen, in which case the pointers are pointing to it, or we tried to
read a sector that's not part of the file, in which case the pointers are
pointing at what would be the nonexistant sector's first byte. In both
cases the pointers are correct and need no further adjustment.
An aside: How can we be sure that the BNE above is always
taken? Here's the code for INCSCB:
3194: 241 ; 3194: 242 ;INCSCB - INCREMENT SECTOR BYTE 3194: 243 ; 3194: 3194 244 INCSCB EQU * 3194:EE E6 35 245 INC DCBCSB ; INC SECTOR BYTE 3197:D0 08 31A1 246 BNE INCS2 ; BR IF NOT FULL 3199:EE E4 35 247 INC DCBCRS ; AND INCR 319C:D0 03 31A1 248 BNE INCS2 ; RELATIVE SECTOR 319E:EE E5 35 249 INC DCBCRS+1 31A1: 250 ; 31A1: 251 ; 31A1: 31A1 252 INCS2 EQU * 31A1:60 253 RTS ; DONE
Clearly the only way this can return a BEQ result is if all
three bytes that it looks at wrap around to zero, which, in the context of
APPEND, can only happen if 65,536 sectors have been read
from the file without finding a zero byte or running out of sectors. Since
the theoretical maximum size of a DOS 3.3 disk is 1,600 sectors (that being
the number of bits available in the VTOC free sector map), no file can
ever have that many sectors allocated to it, and the wraparound can never
happen. Thus we can be certain that after JSR INCSCB,
the BNE will always branch.
All three of the routines called by our new code (LOCNXB,
INCRRB, and INCSCB) are part of the file manager,
so our code needs to part of the file manager too. The fact that
APPEND is a special form of OPEN suggests that it
should be patched into the file manager's Open routine, which looks like
this:
2B22: 2 ; 2B22: 3 ; FOPEN - OPEN A FILE 2B22: 4 ; 2B22: 2B22 5 FOPEN EQU * 2B22:20 28 2B 6 JSR DOPEN 2B25:4C 7F 33 7 JMP GOODIO
All the work is done by the call to DOPEN; GOODIO
exits from the file manager with no error. If we replace
JMP GOODIO with a jump to a patch that either jumps to
GOODIO or to our SCANTOEOF from above, depending
on whether or not Open was called by APPEND, we'll have most of
what we need. Our patch now looks like this:
LDA $AA5F ;Command number of most recent DOS command, times 2
CMP #$1C ;APPEND?
BEQ SCANTOEOF
PATCHDONE JMP GOODIO
SCANTOEOF JSR LOCNXB
BCS PATCHDONE
LDA (ZPGFCB),Y
BEQ PATCHDONE
JSR INCRRB
JSR INCSCB
BNE SCANTOEOF ;(Always taken)
That's the entirety of what needs to be added to the file manager to do
a proper APPEND. (Where to put this patch will be discussed
below.)
Next we need to let the commnand interpreter know that APPEND
is now being handled in the file manager. Here's the APPEND
command handler:
2298: 199 ; 2298: 200 ; EAPND - OPEN FILE FOR APPEND 2298: 201 ; 2298: 2298 202 EAPND EQU * 2298:20 A3 22 203 JSR EOPEN ; GO OPEN 229B: 229B 204 AP1 EQU * 229B:20 8C 26 205 JSR RBYTE ; READ A BYTE 229E:D0 FB 229B 206 BNE AP1 ; BR IF NOT ZERO 22A0: 207 ; 22A0:4C 71 36 208 JMP BUMPER ; GO TO PATCH FOR APPEND FIX
This just opens the file (EOPEN is the handler for the
OPEN command), reads bytes until it finds the end, and then
goes to the patch that adjusts the file pointer. But since the read loop is
now being done by our new file manager patch, we no longer need any
of this. We can just patch the APPEND pointer in the command
interpreter's command address table to point to the OPEN
handler instead, and the file manager will take care of the rest all by
itself. (The pointer in question lives at $9D3A in a 48K DOS, and changing
its first byte to $A2 accomplishes this.)
Now that this code is no longer called, it's eleven bytes worth of memory that can be reused for other purposes.
But we're not done yet. There's still the issue of where to put the new
file manager patch, and the issue of backing out of Apple's old
APPEND patches. Removing Apple's patches answers the question
of where to put our patch—we just put it where Apple's patches used to
be.
Unhooking the original APPEND handler, with its jump to
BUMPER, frees up one of the Apple patches. But there are
three others, all of them called from the command interpreter's routine
that calls the file manager, and one of them does things that aren't
limited to fixing APPEND, so we need to keep parts of it.
Here's the file manager caller, and the patch that we can't just eliminate entirely:
26A8: 2 ; 26A8: 3 ; DOSGO - GOTO DOS 26A8: 4 ; 26A8: 26A8 5 DOSGO EQU * 26A8:20 06 2B 6 JSR DOSENT ; GO TO DOS 26AB:90 16 26C3 7 BCC DG3 ; BR IF NOT ERROR 26AD: 8 ; 26AD:AD C5 35 9 LDA CCBSTA ;GET RETURN CODE 26B0:C9 05 10 CMP #5 26B2:F0 03 26B7 11 BEQ YESEOF 26B4:4C 5E 36 12 JMP CLOSFILE ;NO. CLOSE & COMPLAIN 26B7:4C 92 36 13 YESEOF JMP EOFFIX ;MABYE FIX IT UP? 26BA:EA 14 NOP 26BB: 15 *************************** 26BB: 16 * 26BB: 17 * DOS 3.3 (REV B) PATCH 26BB: 18 * 26BB: 19 *************************** 26BB: 0001 20 DO DOS33B 26BB:20 69 3A 21 DOSGO2A JSR MOVEOF ; MOVE END OF FILE PATCH 26BE: 22 ELSE S 23 NOP S 24 DOSGO2A NOP S 25 NOP 26BE: 26 FIN 26BE:A2 00 27 LDX #0 ; SET OTHER EIF 26C0:8E C3 35 28 STX CCBDAT ; DONE 26C3:60 29 DG3 RTS
365D:00 102 EOFFLAG DFB 0 365E: 365E 103 CLOSFILE EQU * 365E:20 64 27 104 JSR FILSRC ;FILE BUFFER FOUND? 3661:B0 08 366B 105 BCS NOTFOUND ;=> NO, SO SKIP IT. 3663:A9 00 106 LDA #0 ;YES, CLOSE IT 3665:A8 107 TAY 3666:8D 5D 36 108 STA EOFFLAG 3669:91 40 109 STA (ZPGWRK),Y ;RIGHT NOW 366B:AD C5 35 110 NOTFOUND LDA CCBSTA ;ORIGINAL INSTRUCTION 366E:4C D2 26 111 JMP ERROR ;BACK TO ERROR HANDLER
The patches called EOFFIX (which jumps back to
DOSGO2A when it's done) and MOVEEOF (the
latter was added in the January 1983 version of DOS) are purely
APPEND-related and can be eliminated. But we need to keep
everything in CLOSFILE except for the clearing of
EOFFLAG (which is the flag mentioned earlier that tells the
position-adjuster whether or not it needs to skip the adjustment).
Part of the CLOSFILE patch can be fit directly into the
DOSGO routine, replacing the no-longer-needed calls to the
other patches:
DOSGO EQU *
JSR DOSENT ; GO TO DOS
BCC DG3 ; BR IF NOT ERROR
;
LDA CCBSTA ;GET RETURN CODE
CMP #5
BEQ YESEOF
JSR FILSRC ;FILE BUFFER FOUND?
JSR CLOSFILE ;THE REST OF THE PATCH
JMP ERROR
NOP
YESEOF LDX #0 ; SET OTHER EIF
STX CCBDAT ; DONE
DG3 RTS
The rest, by a convenient coincidence, takes exactly eleven bytes, which
is the same as the amount of space that was made available by bypassing the
command interpreter's APPEND handler. Ths is too good an
opportunity to ignore, so the following will replace the original
APPEND handler, at $A298 in a 48K DOS:
CLOSFILE EQU *
BCS NOTFOUND ;=> NO, SO SKIP IT.
LDA #0 ;YES, CLOSE IT
TAY
STA (ZPGWRK),Y ;RIGHT NOW
NOTFOUND LDA CCBSTA ;ORIGINAL INSTRUCTION
RTS
All of Apple's APPEND patches have now been moved or
removed. In a normal 48K DOS, this frees everything from $B65D through
$B685, and from $B692 through $B6FC (the island in the middle, from $B686 to
$B691, is used by an unrelated patch that does an automatic
VERIFY after every SAVE or BSAVE).
We should probably leave $B65D alone, because that's the APPEND
patch flag—it's no longer used, but there might be some software that
tries to improve its chances of a successful APPEND by
POKEing it (as recommended, for example, by Tom Weishaar in his
DOSTalk column in the April
1983 issue of Softalk). That means the file manager patch
described above should start at $B65E, and that's the address that should
be patched into the file manager's Open handler.
That's it for fixing APPEND, but there's still one little bit
of uncleanliness that can be taken care of. The 1983 versions of DOS
include a patch that turns off an Apple IIe/c/GS 80-column card on boot,
which sits (in a 48K DOS) at $BA76, which is right in the middle of the
$BA69-to-$BA95 block that used to be free in the 1980 version. By moving
the patch elsewhere, we can consolidate the free space. Here's the
patch:
3A76: 133 *********************************************** 3A76: 134 * 3A76: 135 * TURN Apple //e 80 COLUMN CARD 3A76: 136 * OFF & INIT APPLE 3A76: 137 * 3A76: 138 *********************************************** 3A76: 3A76 139 OFF80 EQU * 3A76:A9 FF 140 LDA #$FF 3A78:8D FB 04 141 STA $4FB ; CLEARS FUNNY 80 COL STUFF 3A7B:8D 0C C0 142 STA $C00C ; TURNS 80 COL OFF 3A7E:8D 0E C0 143 STA $C00E ; TURN OFF ALT CHAR SET 3A81:4C 2F FB 144 JMP $FB2F ; MONITOR INIT ROUTINE
And its caller:
3FC8: 3FC8 20 RCPATCH EQU * 3FC8:20 93 FE 21 JSR SETVID 3FCB:AD 81 C0 22 LDA $C081 3FCE:AD 81 C0 23 LDA $C081 3FD1:A9 00 24 LDA #0 3FD3:8D 00 E0 25 STA $E000 3FD6: 0001 26 DO DOS33B 3FD6:20 76 3A 27 JSR OFF80 3FD9:4C 44 37 28 JMP RCBACK 3FDC: 29 ELSE S 30 JMP RCBACK S 31 DS 3,0 3FDC: 32 FIN
The OFF80 patch is fourteen bytes long, and we have have
thirteen bytes available between the end of the file manager patch at
$B679 and the beginning of the VERIFY patch at $B686.
Fortunately a slight rewrite can squeeze a byte out of the patch and make
it fit—this requires a modification to OFF80's caller,
starting at $BFD1 (in a 48K DOS):
LDX #0
STX $E000
JSR OFF80
JMP RCBACK
Now we can fit OFF80 starting at $B679, like this:
OFF80 DEX
STX $4FB ; CLEARS FUNNY 80 COL STUFF
STA $C00C ; TURNS 80 COL OFF
STA $C00E ; TURN OFF ALT CHAR SET
JMP $FB2F ; MONITOR INIT ROUTINE
And that's it. With these patches (assuming I got it all right), DOS
does everything that unpatched DOS does, plus APPEND
works correctly in all cases (and it's faster), and there's
more internal free memory than the original 1980 version of DOS
3.3 had—$B692 through $B6FC are free, and $BA69 through $BA95, and
$BCDF through $BCFF.
After these patches, the so-called DOS ID byte, at
PEEK (46725) ($B685) is 251 ($FB).
Here are the patches described above, in hex, for a 48K DOS.
A6B3:0A 20 64 A7 20 98 A2 4C D2 A6 EA (error patch, part 1) A298:B0 05 A9 00 48 91 40 AD C5 B5 60 (error patch, part 2) B65E:AD 5F AA C9 1C F0 03 4C 7F B3 20 B6 B0 B0 F8 B1 (APPEND patch) B66E:42 F0 F4 20 5B B1 20 94 B1 D0 EF AB26:5E B6 (hook into file manager) 9D3A:A2 (Make APPEND the same as OPEN) BFD1:A2 00 8E 00 E0 20 79 B6 4C 44 B7 (80-col patch part 1) B679:CA 8E FB 04 8D 0C C0 8D 0E C0 4C 2F FB (80-col patch part 2)
Note that the last two lines may be omitted (if you don't mind some memory fragmentation), but rest must be applied, or not applied, as a unit. A partial patch will result in a badly broken DOS.
These patches should be applied only to DOS 3.3 as originally supplied by Apple (any of the three official versions should be OK). Applying them to DOS 3.2.1 or earlier, or to any of the various third-party modified DOSes (DAVID-DOS, Diversi-DOS, ProntoDOS, etc.) is likely to be disastrous.
For the 1980 version of DOS, it would also be a good idea to add the patch to the random-access positioner to match the 1983 versions. If the value at $B33E is not $18, then apply the following:
B33E:18 AD BF B5 8D EC B5 6D E6 B5 8D E6 B5 AD C0 B5 B34E:8D ED B5 6D E4 B5 8D E4 B5 90 03 EE E5 B5 60 00 B35E:00
If you trust that my arguments above are sound, and that I haven't missed something important, here's a DOS 3.3 disk with software on it that applies these patches.
AppendPatch.sdk (Shrinkit DOS 3.3 disk, 12346
bytes) AppendPatch.zip (ZIPped disk image, 9691
bytes)The disk image contains two programs, one which patches the currently-running DOS 3.3 in memory, and one which patches the DOS image on a disk.
LLX >
Neil Parker >
Apple II > Fixing APPEND
Original: November 8, 2022