I wanted to write this to share what things tripped me up and what I thought about the experience of porting Cakelisp to Windows. Maybe these notes will help others in porting their projects.
What is Cakelisp?
See Cakelisp: a programming language for games. Cakelisp is an S-expression syntax language which outputs C/C++ code. It also contains a powerful C/C++ build system and other features.
Cakelisp has three major platform-specific requirements:
- Dynamic loading: Compile-time code execution works via compiling and loading dynamically linked libraries while the Cakelisp compiler is still running
- Process execution: Cakelisp runs external processes for compilation and linking. Projects in Gamelib use process execution to run CMake on 3rd party dependencies, use ImageMagick to convert images to
.dds, run Blender to process 3D assets, etc.
- File system: Various path manipulation tools are necessary, e.g. getting a filename from a path, or creating an absolute path from a relative one. I also rely on file modification times to detect changes which necessitate rebuilds
Windows has some differences to Linux which needed to be taken care of for Cakelisp.
Close them! Permission violations are usually because I still have an open handle, for example. This was one of the biggest time sinks while working on Cakelisp. I had forgotten to close handles for some files, which manifested in strange File Explorer behavior (
Permission denied on deletion of some files created by Cakelisp) as well as failure on second execution of Cakelisp.
I used Process Explorer to find why I cannot delete a file/folder Cakelisp created. The
Find Handle or DLL operation finds which process is preventing deletion, which hints that your process has not closed a handle.
Redirecting std in/out has a subtle difference when compared to pipes in Linux. On Windows,
stdout needs to be polled in order to not get stuck waiting for output. On Linux, the kernel will buffer it all for you so the application doesn't get locked up.
I still haven't completely locked down how stdout from child processes should be buffered, because these decisions affect the performance of the child process. If the child process fills its stdout buffer, it will wait until the parent reads the buffer, which means e.g. child process compilation will be slower than if executed directly.
Be very careful when converting
int64, which isn't convertable to
unsigned long. This tripped me up because I stored Linux file modification times with
unsigned long, but Windows
unsigned long is only 4 bytes. When I tried to store Windows file modification times in
unsigned long, I'd lose 32 whole bits, causing the file modification time to be very wrong. I fixed this by ensuring I use a
int64_t on both platforms.
There was another conversion issue, though this time it was that
strtol. This came up because command CRCs weren't being read in properly, causing unnecessary rebuilds. This is a good example of how using a variety of compilers can help find issues in your code.
Dynamic library differences
LoadLibrary() is strict about paths. Use Process Monitor to see what it's failing to load. This application will list all the Windows events, including when DLLs are requested for loading. I had to add some additional paths in order to load my dlls.
I used Dependency Walker when debugging my DLL loading. This showed me I needed to load
cakelisp.lib for my compile-time code DLLs in order to get symbols from the parent
This introduced a very annoying difference between Windows and Linux DLLs: Windows DLLs need
declspec. See this page for options of how to export symbols. This is annoying because compile-time code execution in Cakelisp expects near complete access to Cakelisp's functions and types. Windows requires I add an ugly
CAKELISP_API tag to nearly all my function declarations. I could create an index file instead, but those would easily become out of sync with the header files.
In contrast, Linux linkers provide options like
--export-dynamic, which will automatically export all symbols.
Another difference is that you have to load the parent
.lib file in order to access its symbols. With Unix's
libdl, the parent's symbols can automatically be resolved to from the dynamic library.
Finally, one pitfall I encountered was that you must have the exact same build settings for DLLs as the loading
.exe, otherwise random crashes in CRT will occur. See
/MDd options especially. I encountered these crashes/asserts during allocations usually. I had to make sure Visual Studio and my command lines all had matching options.
It is obnoxious to require a
vcvars environment when using MSVC (see Use the Microsoft C++ toolset from the command line). Why can't all these settings be on the command line? The worst part of this is that the
vcvars*.bat scripts are terribly slow (hundreds of milliseconds); they dominate the "nothing to do" build time. As a result, I'm now going to have to implement my own code to set these variables.
Herding PDBs is a pain in the ass. In Linux, debug symbols are included in the artifact by default, rather than having to be maintained as a separate file. If you're writing a build system which moves around artifacts and caches them (which Cakelisp does), the PDBs add complexity to that system.
Overall, the Windows documentation is not too bad! It can be overly verbose, but the linking is generally good and the examples are helpful. I would say manpages and Windows docs are pretty close in quality.
One interesting thing is that searching for help on Windows problems seemed harder than Linux problems. The cynic would claim that evidence that Windows works better, so fewer people are asking for help. I don't buy that: I would guess there are just more amateurs running syscalls in Linux than Windows (e.g., for college courses), causing there to be more questions.
The Windows port took several days and many hours of debugging. It was my first significant "low-level" Windows experience (at work I had done some
WM_PAINT UI stuff), so I learned a lot. I am proud that Cakelisp has good Windows support that doesn't require anything crazy like Cygwin or Mingw, which can have intimidating installs for many users.
I fixed bugs in the Linux version revealed by this port, which is good. It is important as a developer to have good Windows support because they still hold the majority market share, and will for some time.