Running from hard disk - basic concepts

[Petari]

Atari ST games were made mostly to work from floppy drives. To achieve work from hard disk on real Atari machines we need to change some parts of games and/or do some other measures.
The whole thing is not simple at all, and talking about all details would took a lot of space and your time, so I will try to describe essence of probably hardest cases: games using all 512KB RAM, direct floppy access. Like Microprose F1 GP, EPIC, Wrath of the Demon, etc ...

Lowest 40KB or more in Atari ST is reserved for TOS workspace. That area must be intact, so we can use loading from floppies and other things via TOS function calls. But some games use that space for own purposes. The reasons are need for all RAM, copy protections and similar. Usually we say that SW kills TOS in such cases.Then game must have own code for floppy access. And it can be smaller, faster that TOS code. Even 200 bytes may be enough for it. To make such games to run from hard disk we need following:  replacing floppy access code with hard disk access code. Ensure somehow that can use standard hard disk format - what means our Atari hard disk with files and driver. But TOS is killed, and low RAM is occupied by game. So called RAM conflict.
To override it we can:   1.  change game so, that it run in higher RAM.  
2. Using custom hard disk loader, what can work from High RAM.
3. Swapping low RAM content with game's and TOS'es content.
4. Change TOS so that it's workspace is in high RAM.

1 - it is very hard  - to relocate some game, and we should do it with all involved. Plus some code is made specificly for low RAM.    Not used ever, as I know.
2 -It is used by FFLS hard disk loading system by Superior, some 20 years ago. Pretty good and fast, but limited only to ACSI drives and max 32 MB partitions. Read only.  I guess that it is actually floppy file loader, changed from FAT12 to FAT16 +  ACSI hard disk driver.
3 - Is WHDLoad concept, or ULS by Ataris. Downside is slowness when many short disk access is needed. Good side is that works with most of hard disk drivers.
4 - Disassembling TOS core (GEMDOS), and changing so, that workspace is not at 0, but may be at any RAM loc. The funny thing is that something like is planned by Atari (DR) self, right from beginning, but never done. There are traces of it in TOS code - addresses indexed by a5, but not always. It is mentioned too in Atari Profi Buch. Why it is not made completely remains secret.  Anyway, I did it with TOS 1.04 core. The result is possibility to access hard disk files when low RAM and standard TOS is killed, and all it fast. It was lot of work, and needed to change thousands lines of code.
Access is read/write, and we have same limits as TOS 1.04 - so max 512MB partitions. But speed has it's price - hard disk driver must be installed in high RAM too, what by usual ones means performing somekind of autoboot again. So, more code needed.
Loading, even many short accessing goes fast. But we still have some interrupts, visible by flashing, music interrupts. The reason is that usual hard disk drivers need regular low RAM content (system variables), Timers etc. So, we must restore them before any hard disk access and after it restore game's ones. MFP makes here most problems - reading state is slow for instance.
Can it be better ?  Yes, if we make hard disk driver which can work complete on own, without need for timers, system variables in low RAM etc. + we need TOS filesystem part with same.  
This is what I made in last years, gradually. Everyone can test it. Loading from hard disk during gameplay is smooth, as no breaking of gamecode needed. Practically, same as with RAMdisk, but no need for many RAM. 1MB is enough for most of games.
If you read it carefully, should know that it works only with special made hard disk drivers, and such are only mine - latest versions.

This leads us to next thing:  what hard disks, adapters to use in 2011/12 ... ? How to transfer files on Atari hard disks ?
I guess that everyone will agree that Flash cards are today best solution - sonsidering prices, place usage on table, etc.
Data transfer can be very easy with USB card readers.
So, normal is that I came on idea to release complete hard disk images, with games and driver. DL it in minutes, write on card in few minutes, attach to US and enjoy. That simple.  This is what is my priority - to support modern devices and pulling out max. from HW.

Atari people should be grateful that can choose between different solution considering hard disk adaptations (patches).

P.S.  Some ideas about further improvements:  to make RAM swap faster we could use PMMU on TT and Falcon - then swap can be practically instant, in microseconds. Something like is possible on ST(E)s too, with additional circuit, what I made actually - not too complicated. But hardly will be spread ever.