Rabbit with 1000 Repos

The IBM PC: More Memory, Courtesy Your Keyboard

February 27, 2021

One of the best things about networked computers is being able to download more RAM when you need it. It’s pretty handy, I’ve been doing that since the BBS and modem days. I still download RAM on my smartphone, and it–

I tell Bamboo that RAM doublers don't actually work and he smashes his phone.

Well…what if I told you that, on older IBM PCs, there was a key you could hit on your keyboard that would give you…

Bamboo says 'All the RAM you could possibly ever need, I say no, he drops an IBM M Series keyboard.

Ok, fine…sigh…it’s not actually a key on your keyboard, but it is in the thing your keyboard plugs into to talk to the rest of the computer. And…yes, it can give you more RAM. Kinda sorta.

The IBM PC was designed from the start to have max 1MB of RAM, because that’s the most the 8086 could handle and RAM was expensive back then. Due to how the chip was designed, accessing all that 1MB was…a little odd.

Processors use memory addresses, a number starting from 0, to figure out the place in RAM where data can be written to and read from. The 8086 was a 16 bit processor, as the size of pieces of data it was designed to work with was 16 bits - 2 bytes, or the numbers 0-65535 - wide. Programmers call the size of data a processor is designed to work with a word. This means that, if an 8086 only used one word as a pmemory address, you’d have at most 64 kilobytes to use. Even the Commodore 128 could do better than that.

Processors will have a different sized value, separate from the word size, as the address width, which they use to find stuff in RAM. Usually the address width is double the word size, or some multiple of 8 bits so you can stick with bytes for everything. Keeping it in bytes makes it easier for processors, and easier on programmers when writing software. The 8086 has an address width of 20 bits, which is 2 and a half bytes, or one and one quarter words.

Of course it has to be different.

So instead of taking two words and sticking them together, like on a Commodore 64 whose word size is 8 bits:

C64 memory addresses

Or, doing what Apple did on the 68000 and taking two 16 bit words, sticking them together, and using one of the bytes in there as data for the garbage collector:

68000 Mac memory addresses, made from two words, one byte in the word being for flags

The 8086 stuck two 16 bit words, a segment and an offset, together with MATH. Hexadecimal math.

Segment:offset 1234:ABCD wraps to 1C0FD
Segment:offset FFFF:FFFF wraps to 10FFEF

It’s a neat approach. But it has a problem. The 8086 would lop off everything that went over those last five characters, like how letterboxing a show shot at 16:9 and broadcast on 4:3 TVs would cut out the stuff no one expected to see:

A sitcom shot with two characters in a scene, letterboxed.

Take away that assumption by releasing the show on DVD in 16:9 format, and you start running into some unexpected results:

The same scene, but with the letterboxing removed, and you can see additional things on the sides.

The addresses FFFF:FFFF and 0FFE:000F go to the same memory location, 0FFEF, on an 8086, but not on a 286, where they would go to 10FFEF and 0FFEF, respectively. A program could get quite confused if it writes to 10FFEF and then hopes to find the data again at 0FFEF.

So what’s the solution? You hard wire the part of the 286 (Addressing line 20, or the A20 line) that allows addresses to go over 20 bits to a switch on the motherboard. If the switch is off, addresses wrap around, and if it’s on, they don’t. And the best place for that switch? Why, an unused part of the keyboard controller, of course!

Now, you can’t expect humans to know to push a switch on their keyboard to activate this. The switch stays off, unless a program like Rebel Assault or the operating system specifically requests it. But when that switch is turned on?

It’s 10000% totally exactly like downloading more RAM.

Changelog

  • 2021-02-27: Initial post

The Commodore 128: Meet the Memory Magician

February 19, 2021

My second computer was a Commodore 128. I remember two things about it: it felt like a real computer as opposed to the weird brown toy that the original C64 looked like, and it was the first computer that I learned to program on.

The C128 had double the RAM of the Commodore 64 (128 kilobytes vs. 64 kilobytes! Yeah!), and an upgraded processor. The MOS 8502 in a Commodore 128 ran at almost double the speed of the C64 (2 MHz vs. 1 MHz! Yeah!). However, its address width was still only 16 bits, which means it could technically only see 64K of RAM at a time. So how the heck did the Commodore 128 double that? that

A magician, standing in front of a Commodore help wanted sign, saying 'Well this would certainly be a step up from balloon animals...'

They hired a memory magician!

With older computers, you had to trick them into thinking they only had as much RAM that a particular constraint could handle. With IBM PCs, you had to lop off the top parts of addresses with a hardware switch for older software to work. With Macs, you had to pretend memory above 16MB didn’t exist with a software switch. With the Commodore 128, it was by putting each 64K block of RAM behind a door, and putting in a hardware memory management unit – a memory magician – to trick the CPU into only working with 64K at a time.

The CPU in the Commodore knew how to point to a spot in a 64K block of RAM and get and set data there. With only one 64K block, no magic is needed. It just works. To take advantage of that other 64K, the magician has to, very quickly and on command, swap which block of RAM the CPU is accessing, without the CPU knowing what’s up.

A magician showing off bank switching to an 8502 and a Commodore representative

It was standard Commodore protocol that program code went in the first 64K block – Bank 0 – and data went in the second 64K block – Bank 1. All the necessary computer-y things (operating system, how to run BASIC programs, floppy and tape drive access, display) also lived in Bank 0, but there were ways to turn parts of those off and free up some RAM if you didn’t need to use them. You, the programmer, could tell the magician to deactivate parts of the computer you didn’t need, like the tape drive or BASIC, which gave you more RAM to use.

If you wrote your program in BASIC, the BASIC interpreter already knew to put all of your data into Bank 1, and handled talking to the magician for you. If you really needed performance, you’d write your program in assembly. In that case, you’d have to work out with the magician the times when youall, the programmer/magician team, needed to trick the CPU.

As part of your program, you’d tell the CPU to put a value (0 or 1) into a particular address in memory. The CPU does not know why it’s doing this. The magician watched this particular address and, when it changed, would swap the Bank it was presenting to the CPU based on that value. Think of it as a light up sign the magician would act upon. The magician would see the new value on the sign and switch to that. This was done right before the CPU accessed RAM, and the CPU has no idea a switch took place – the magician simply presents the 64K the programmer asked it to.

The magician watches a memory location for changes and, when it changes, swaps out the memory bank presented to the processor.

Since only one Bank was visible to the CPU at a time, and since the CPU had to always know about things like joystick and keyboard inputs, certain parts of the Banks were shared – a change in one changed the other. This included the place where you change the light up sign. The magician handled the duplication for both the developer and the CPU.

Magicians – I mean, memory management units – became a critical component of computers, both to work around limitations and to give the CPU some help doing the thing it does a heck of a lot of – reading and writing RAM.

Changelog

  • 2021-02-19: Initial post

The IBM PC: Getting Rebel Assault Running

February 10, 2021

I was big into Star Wars as a kid. Long before the Mandalorian and Clone Wars, before JJ and Rian, heck, even before Jar Jar and Amidala. I watched the original movies on VHS every year on Christmas afternoon, I struggled with the SNES side scrolling games, and I played X-Wing and TIE Fighter to death.

In the mid 90s, I had my Amiga 500 (or maybe my 2000 by this point) and my dad had some generic PC. He had a CD-ROM drive in his machine, and he got LucasArts’s first CD-ROM only game, Star Wars: Rebel Assault. This game was super cool, with pre-rendered 3D flight sequences, because there was no way that computer was rendering anything more complex on its own than the original DOOM.

The trickiest part of Rebel Assault was not the spaceflight or taking down the Death Star…it was giving it enough memory to run! Rebel Assault ran on DOS, and required loading and configuring the sound card driver and the CD-ROM driver, by hand, using a pair of files called CONFIG.SYS and AUTOEXEC.BAT. These files have been around since the beginning of DOS, and only truly went away with Windows ME. Even in a simple OS like MS-DOS, you need to load a bunch of stuff at the start to make it work.

My dad’s PC was a 386, and most likely it had between 4 and 16 MB of RAM. In IBM PCs back then, the first MB of memory was divided up in a special way. The first 640K was roped off for direct program and data access in a place called Conventional Memory.

The breakdown of the first MB of RAM in an IBM PC, with an 8086 standing in front of it

Above that 640K is the Upper Memory Area, a place where the code for graphics card drivers and the BIOS lived. The original memory address boundaries and sizes of these areas are:

  • A0000-CFFFF, or 196608 bytes, for the graphics card
  • D0000-EFFFF, or 131072 bytes, for the graphic card ROM
  • F0000-FFFFF, or 65536 bytes, for the BIOS

Go past the 1MB mark and now you’re in the High Memory Area, a special 64k segment that, along with a special hardware switch called an A20 Gate, kept older DOS programs in their original 1MB playground.

Go a little further, and now you’re in to the good stuff. The memory above 1MB+64K. Normal DOS programs can’t get in here without tools like HIMEM.SYS and EMM386.EXE, but you’re Rebel Assault, a protected mode program that can throw open the A20 Gate and get access to it all. We just…have to…get it started…

Ugh, we’re out of conventional RAM, and/or the game is slow as heck, and/or the CD-ROM or sound card are acting up. Guess it’s time…

Bamboo, holds up a floppy disk and says '...to make a boot disk!'

A boot disk contains a minimal CONFIG.SYS and AUTOEXEC.BAT, tuned precisely for your hardware and the game you want to play. It also includes all the drivers you need. Since PCs of this era could not boot off of CD-ROMs, and since you didn’t want to constantly modify your hard drive’s boot files, you’ll need to break out a floppy.

Loading in a CD-ROM driver or a Terminate-And-Stay-Resident program (a program that executes once and keeps running) that your hardware needs means taking up precious conventional memory, and Rebel Assault needed enough available up to start. Normally we could load drivers into the Upper Memory Area, but the tools to do that, HIMEM.SYS and EMM386.EXE, don’t play nice with Rebel Assault. Additionally, some TSRs need a large amount of RAM to get started, then release most of that back to the computer, which means load order counts. I bet my sound card needed a TSR, which is why I cared about this.

So what does an eager 15 year old do?

Bamboo, holds up a QBasic window and says 'Write a QBasic program that helps you sort through all these options and spit out the most optimal CONFIG.SYS and AUTOEXEC.BAT files!'

It took a few days of coding after school, but I was eventually able to work out all the pieces I needed to put onto the boot disk, how to track memory usage before and after each piece loaded, and what file each part belonged in. Then I could stick the data into a QBasic program and get the results, giving me…

The Perfect Rebel Assault Boot Disk

Changelog

  • 2021-02-10: Initial post

New Self-Portrait for February 2021

February 2, 2021

I painted a new Bamboo self-portait for February 2021!

Hopefully this will be one of the only ones where I’m wearing a face mask. New one in a month or two!

The Macintosh: It's Free Real Estate

February 1, 2021

If you’re a software developer, you’ve probably written some code that, at the time, seemed like a great idea but turned out to be a not-so-great one after a while. I’ve done it hundreds of times. Often it’s no big deal – it’ll get cleaned up or removed eventually, with varying amounts of pain involved.

Every once in a while, though, a decision is made that causes real big problems, and sticks around much, much longer than you’d expect.

Just like how IBM engineers were only thinking about that first megabyte of RAM when designing how the Intel 8086/8088 would work in the first PCs, Apple engineers were only thinking about that first 16 MB, since that’s all the Motorola 68000 processor used in earlier Macintoshes could handle without any hacks.

A 68000 processor with 16 MB of RAM, and a 8060 with 1MB

Apple engineers also wanted to solve a classic memory management problem – how do ensure that the memory a program asked to use is actually done with it? Multitasking was still in its early days – a difficult thing to do on a Motorola 68K processor anyway – and you needed some way to make sure each program plays nice with the memory they’re given, and gives it back to the system when they’re done.

The 68K could use up to 16 MB, which amounts to a 24-bit address bus (2^24 bits = 16777216 bytes). The native data size – the “word” size – for a 68K is 16 bits, which means it would take two words, or 32 bits, to represent a spot in memory. This address – the pointer – had an extra 8 bits available for basically whatever you wanted to use them for. On the Mac, they used these 8 bits to flag blocks of memory as “locked” or “purgeable”, so that the garbage collector, a process that cleans up RAM, knows to leave it alone (“locked”), or that it’s safe to be cleaned up if needed (“purgeable”).

However, technology progresses. The 68020 came out which could address 4GB of memory and, with a 32 bit address bus, needed all 32 bits of that memory pointer. Memory prices started dropping. Users wanted more RAM. Apple advertised that their machines could handle more RAM. But Apple couldn’t open the floodgates to all this RAM, because these hybrid (24 bit address + 8 bit garbage collection data) memory pointers, when turned loose on a system that expected all 32 bits of a memory pointer to point to an actual address, would crash horrifically.

A program hands a 68020 a hybrid address, and the 020 hits the program in the head with it

Apple had to force their operating system to operate with 24 bit addresses until they could get everything using these hybrid addresses cleaned up, and this took quite a long time, almost up until Apple switched from 68K processors to PowerPC processors in the late 90s.

Changelog

  • 2021-02-01: Initial post
  • 2021-02-10: Add link to IBM PC memory map post

The Amiga: What the heck is NoFastMem?

January 22, 2021

Young Bamboo using an Amiga: “What the heck is NoFastMem?

The Amiga was a marvel of computing at release, with a custom set of processor chips that took over audio, video, and I/O from the CPU, allowing the computer to do so much more than other machines of the era.

The Amiga Original Chipset - Agnus, Denise, and Paula - all working hard.

These custom chips - Agnus, Denise, and Paula - made the Amiga the Amiga, and when the Amiga first came out, all of the RAM in the machine was made accessible to them, to really show off the machine’s potential. The RAM these custom chips had access to was called Chip RAM, ‘cause the custom chip Agnus, and not the CPU, managed access to that memory. My first real computer, an Amiga 500, came with 512KB of Chip RAM. Back then, this was a lot, but, as with all things computers, memory needs steadily grew, and the 512KB it was released with in 1987 quickly became insufficient.

A program looks at Agnus and the 68K process, wonder who to allocate RAM from

All Amigas supported memory expansions of various types. Even the original Amiga 1000 could be upgraded to about 10MB of storage. Memory on the Amiga came in two types: Chip RAM, which Agnus managed, and Fast RAM, which only the CPU had access to. Since the custom chips couldn’t access Fast RAM, programs that could use it could move data in and out much more quickly without Agnus standing in the way.

I had an A501 trapdoor expansion in my Amiga 500, which provided an additional 512KB, bringing the machine up to a whopping 1MB! When you booted the machine, it showed the 512KB of Chip RAM and 512KB of Fast RAM…but in my case, it wasn’t technically Fast RAM. The trapdoor expansions sat on the same path the custom chips used to access normal Chip RAM, but without replacing Agnus with a newer version, the trio couldn’t use it. I had more RAM, it was treated as Fast RAM by AmigaOS, but it was actually “Slow” RAM, since Agnus still controlled it.

Over the years, there were improvements to the custom chips and CPUs to allow more Chip and Fast RAM, eventually reaching 2MB Chip and 16MB Fast with the final Amiga, the Amiga 4000T. These computers could handle more and more applications and bigger, more complex games as time went on.

Bamboo: “Pretty impressive, huh?” The NoFastMem icon appears. “Oh, right, NoFastMem”

See, some really really old Amiga software didn’t have any clue how to handle that Fast RAM being in the system, and, since the operating system and CPU provided almost no memory management whatsoever beyond “play nice”, poorly written software could tell the OS to write anywhere in RAM and it would do so, with disastrous results. If the computer’s reporting back 1MB of RAM, and half of that is actually Fast RAM, that software could make some very poor assumptions.

NoFastMem is a resident program that disables all Fast RAM in the system while it’s running, even the “Slow” Fast RAM. You only get access to Chip RAM, however much that may be. That means you can run that older software without problems, and then you can stop NoFastMem to get all of your memory back when you’re done. Neat, huh?

Bamboo makes a heart around the Boing ball on his shirt

Changelog

  • 2021-01-22: Initial post
Older Posts
Home
Newer Posts