I have these giant data structures stored on the disk; I certainly want to make sure that it's safe. In case of a crash, power outage, or some similar catastrophe, it is possible that GameCracker shuts down at a point when the stored data is in an inconsistent state. In order to avoid that, I have created a consistent storage library.
My goal is similar to some aspects of databases, namely I want that the single operation I defined - the expansion of a move - either finish completely or not at all. Note that I have to ensure this requirement across multiple data files. In this entry, I describe how I do it, first from the point of view of GameCracker itself, and then discuss the implementation.
First, a manager instance has to be created, and through it can all the different data files be opened. The main program can read and write these files using familiar interfaces. I have two kinds: a sequential store, which can just write different kinds of data in a row without the ability to read anything back or to set position; and a random access file which can be read and written in a random access manner.
The program has to signal consistent states by calling a commit function on the manager. The guarantee of the system is that whenever it restarts, the files will reflect one of these states, even if some writes followed the last commit during the last execution. A final important operation is what I call sync; this causes the system to immediately update the files so that they reflect the last commit. Note that the system must ensure consistency even if the sync operation itself is interrupted.
Now on to the inner workings. All the files have a corresponding in-memory data structure which stores all modifications since the last sync, and also mark the last commit state. Write operations update these data structures instead of the files themselves, and reads ensure that they return the latest data, either from the file or from the in-memory structure. For sequential files, all I have is a byte array accumulating writes which have been performed since the last sync, with a pointer marking the position where the latest commit was performed. Upon a new commit, I move this pointer to the end of the array, and upon a sync, all the data before the pointer is written to the file and removed from the store.
For random access files, I have two maps, storing the bytes written to various positions, one corresponding to the latest commit, the other storing the uncommitted writes. All new writes are translated to adding elements to this second map. A commit operation comprises moving the contents of the second map to the first, committed one. Syncing flushes the changes stored in the committed map to the disk. Reads have to check first the uncommitted map, then the committed map for any updates before turning to the file on the disk.
Thinking that even for random access files, it will be common that data is appended to the end, I have also given them an 'append buffer', which is very similar to the data structure corresponding to the sequential files. This buffer complements the maps, providing an efficient method to administer writes which append to the file. (At least I hope. I haven't actually carried out any performance tests.)
The actual files are only updated by the sync operation, and otherwise reflect the state of the commit just preceding the latest sync. So the only time when the files can become inconsistent is when the sync is interrupted. When that happens, it is detected upon the next startup, and the files are fixed. Here's how.
The sync operation queries the data describing the changes since the previous sync up to the latest commit from every file structure belonging to the manager. I call these chunks of information commit data. (It is a bit confusing, but in my head I consider the sync operation as a commit from the point of view of the actual files.) This commit data is not immediately applied to the files themselves, but first they are written to a commit file. I make sure that the commit file is entirely written to disk before going ahead and performing the file updates. After all the updates are complete and safe on the hard drive, I delete the commit file.
If the program is interrupted during the sensitive time of updating the data files, the commit file doesn't get deleted; this is how the problem is detected when the program starts the next time. In this case, I perform restoration as part of the initialization of the library, before anything else. I read the contents of the commit file, and with all the commit data in hand, I virtually redo the sync operation, fixing the state of the files. When that is done, I delete the commit file, and now the program is ready to start.
Now, if the sync operation is interrupted not during the file updates but during the creation of the commit file, then the restoration will find an incomplete, incorrect commit file. In this case, this commit is deleted and abandoned. The actual data files are in a consistent state, since the last sync operation did not finish writing the commit file, it didn't get to the part where it even touched the data files themselves.
That's about all I wanted to say about this. The system of course doesn't protect from disk corruption or from outside modification (or deletion) of the data files, but that wasn't my goal. Now I can be safe that even if the program is interrupted during a file update, the graph and all related data structure will be fixed, and the results of all the work cannot get lost so easily by an accidental reboot or the free space running out or something.
I must say, I'm quite proud of how this system works.
