TBA Software is not all about TAG and I've been reading through Forum posts and giving some thought to two areas in particular. Threading and a 64bit file system.
Pre-emptive multi-tasking
A few people have had a go at implementing threads over the years but all have been an add on, rather than being in the OS code. To make it work properly I think it has to be in the task control system of RISC OS, and it has to provide backwards compatibility for old code.
A new type of task is needed that is managed by a new process manager that has the common process, thread, local storage and synchronisation objects that are present in other systems such as Win32. My idea is that a single threaded process would be created to deal with all the co-operative stuff, and communicate with the "new world" through synchronisation objects. The SWI despatcher would need to understand what is thread safe and what isn't, and block if more than one call is pending to non 'safe' code (from outside the co-op thread). Modules / SWI calls would need a flag to say they are thread safe. WIMP poll would glue into the new process management. Memory protection should be implemented as standard, even if in a basic form, so that thread safe code expects memory protection from the outset.
There is of course a lot more to it technically than this, but I think it important to get a structure in place before more effort is spent on code that ultimately doesn't achieve what is required.
TBAFS64
Following a short discussion on the ROOL forums I have drafted a spec for a 64bit version of TBAFS which I've dubbed TBAFS64. TBAFS is an image file system so requires 64bit enhancements to Fileswitch, Filecore, and ADFS, before it could be implemented. However it could provide the new disc format that RISC OS needs to handle large disc - the Filecore formats use some quite traditional methods and seem to require quite a large amount of RAM.
TBAFS quite simply turns disc space into a heap into which it allocates blocks. It uses a single layer indexed directory structure and free block structure, plus a tree-indexed file data structure. The tree index gives very high performance when random accessing compressed files as it only has to work with the blocks in question. When files are not compressed they are still treated in blocks for simplicity.
Rather than tying the TBAFS format to the hardware disc layout in some way (as has been traditional with some file systems) TBAFS simply uses caching to limit the amount of I/O that is required.
The TBAFS64 spec includes an option for a journal, and dual copies of all the index data at each end of the disc. The idea with the journal is to keep the file structures intact in the event of a power loss. All writes for a particular "operation" are comitted to the journal before being committed elsewhere, and are only marked as complete when the journal has been flushed. In the event a power loss the journal will still contain entries that still require committing to disc. Of course care has to be taken in the underlying hardware design to avoid lazy write back caches that can defeat the integrity of the journal. One design note is to provide an option to put the journal on a separate disc, greatly improving potential I/O throughput.
It is possible to extend the journal to provide fully transaction safe operations such as those used by a database. This requires the addition of "transaction begin" and "transaction commit / roll-back" calls into the TBAFS API. This would cause headaches with blocking (locking in database terms) if a second attempt was made to alter structure (typically file table data) already modified by an open transaction.
As it stands someone needs to look at the 64bit code for Fileswitch and Filecore before this can get off the ground. I might be brave enough to take a look if time allows over the coming months but a lot needs to happen before then! :-)
