K1n9_Duk3 wrote:I doubt that turning DOS executables into DLLs is going to work, at least as far as executing low-level hardware stuff is concerned. On WinXP and above, you can't even read/write ports from a Win32 program. Only device drivers are allowed to execute such instructions. I don't know a thing about the structure of a DLLs and EXEs, so I can't say if it would be possible to actually convert a DOS executable into a Windows DLL in the first place.
I think that in Hendricks266's example, the idea is running the engine code (which is open source) as-is, while most of the Blood-specific code (from BLOOD.EXE) will be run under an x86 emulator. Some JIT compilation may be used for the latter. Basically, it's a mix of emulated/translated x86 and native code, while hardware accesses (e.g., port reads/writes) are appropriately translated.
As for recreating source code from scratch, I can confirm that it would take roughly half a year to get something decent. My first attept at disassembling a game and porting the assembly code back to C code took about two months. The game's file formats were already well documented, but I had never written any x86 assembly code before, nor had I programmed any DOS games in C. I used a manual to look up what each assembly instruction does, and I wrote little test programs and disassembled them to figure out how the compiler translates certain code patterns into assembly code. After those two months, I had code that compiled properly, but it had tons of bugs.
It is great to read about this from you! This includes the point about learning x86 assembly and DOS-related stuff on the way. I do share the sentiment, that getting something like this to compile is less difficult than getting it to compile *and* run well.
The biggest issue is that when you are manually converting assembly code into C code, you are bound to make mistakes here and there. Unless you manage to reproduce the original EXE byte-by-byte, you can't say for sure that the C code you created is equivalent to the C code that was used to build the original EXE. That's actually one of the big problems in computer science. If you're lucky, the game has a demo record/playback feature that makes it easier to detect where the new EXE differs from the old one.
Oh yeah, these are all true. There are cases in which a sufficiently similar EXE (say, differing by just 1000 bytes) may be ok, but getting even to that is quite a challenge, especially without having original sources. As it should be clear for some, in order to even try and reproduce original machine code as-is, there are various factors that one should take into account, apart from the source code itself; These are project settings and versions of tools in use (e.g., compiler and linker), if not more than these.
When NY00123 (and others) were working on the Commander Keen 1-3 recreation, they actually made a patch that unlocked the demo record/playback routines in the original executable, so that the reimplementation could be tested against the original game. But that will only prove that the new EXE differs. It can't actually prove that the new EXE is equivalent to the old one.
Heh, I forgot that there's a topic about the Keen 1-3 recreation in these forums! Re-using the demo routines surely made me discover a few differences from vanilla Keen 1-3, including some jump height miscalculations, as-well-as a minor episode-specific thing in Vorticon's behaviors (Vorticon's state changes to "jumping" upon falling from a floor, but only in Keen 2-3).
So yeah, I should probably tell a short summary on the process of getting that reimplementation up:
1. First, QuantumG, who has great knowledge about reverse engineering, did some RE work for Keen 1 by modifying the DOSBox emulator. Currently his work can be found here:
http://www.quantumg.net/keen1.c.txt
2. Afterwards, lemm, who's known for his earlier patching work, as well as his later source modding efforts, took QuantumG's work, including a lot of readable function and variable names, and applied most of it (if not all) to a disassembly of KEEN1.EXE (in x86 ASM).
3. Near the beginning of September of 2012 (possibly in August), I found out this disassembly. A bit later, I started doing some work on the reimplementation of Keen 1. I also used a few chunks of Commander Genius code for decompressing data files (this might be the most difficult to understand code piece in the original DOS EXE, from what I recall).
4. I uploaded an initial release near the end of October 2012. So, maybe a bit less than 2 months, *with* all the earlier work done by other individuals beforehand. Also, I did get to have earlier experience with 16-bit x86 ASM (from patching Bio Menace 1), which must had helped.
5. About a month later (November), I got a feature-complete reimplementation of Keen 1.
6. Later, lemm assisted with adding support for Keen 2 and 3. As expected, most of the coding was already done while reimplementing Keen 1.
In some cases, I had to disassemble my new executable to track down the bugs. Stupid stuff like having to use "myfunction((x >> 8))" instead of "myfunction(x >> 8)" because the stupid compiler decided to convert x to a byte value
before doing the shift operation...
This reminds me of a case where I had to force some right shift being signed, or maybe unsigned; I don't recall now. This was required in order to make a bridge in some Keen 2 level operate as expected (when using a switch).
There are also the MIDI-related differences in Super 3-D Noah's Ark posted by me earlier, including something to do with integer casting:
viewtopic.php?p=7475#p7475