Porting Cakelisp to Windows

By Macoy Madson. Published on .

I wanted to write this to share what things tripped me up and what I thought about the experience of porting Cakelisp to Windows. Maybe these notes will help others in porting their projects.

What is Cakelisp?

See Cakelisp: a programming language for games. Cakelisp is an S-expression syntax language which outputs C/C++ code. It also contains a powerful C/C++ build system and other features.

Cakelisp has three major platform-specific requirements:

Platform differences

Windows has some differences to Linux which needed to be taken care of for Cakelisp.

Handles

Close them! Permission violations are usually because I still have an open handle, for example. This was one of the biggest time sinks while working on Cakelisp. I had forgotten to close handles for some files, which manifested in strange File Explorer behavior (Permission denied on deletion of some files created by Cakelisp) as well as failure on second execution of Cakelisp.

I used Process Explorer to find why I cannot delete a file/folder Cakelisp created. The Find Handle or DLL operation finds which process is preventing deletion, which hints that your process has not closed a handle.

Standard I/O

Redirecting std in/out has a subtle difference when compared to pipes in Linux. On Windows, stdout needs to be polled in order to not get stuck waiting for output. On Linux, the kernel will buffer it all for you so the application doesn't get locked up.

I still haven't completely locked down how stdout from child processes should be buffered, because these decisions affect the performance of the child process. If the child process fills its stdout buffer, it will wait until the parent reads the buffer, which means e.g. child process compilation will be slower than if executed directly.

Types

Be very careful when converting int64, which isn't convertable to unsigned long. This tripped me up because I stored Linux file modification times with unsigned long, but Windows unsigned long is only 4 bytes. When I tried to store Windows file modification times in unsigned long, I'd lose 32 whole bits, causing the file modification time to be very wrong. I fixed this by ensuring I use a int64_t on both platforms.

There was another conversion issue, though this time it was that uint32_t needs strtoul not strtol. This came up because command CRCs weren't being read in properly, causing unnecessary rebuilds. This is a good example of how using a variety of compilers can help find issues in your code.

Dynamic library differences

LoadLibrary() is strict about paths. Use Process Monitor to see what it's failing to load. This application will list all the Windows events, including when DLLs are requested for loading. I had to add some additional paths in order to load my dlls.

I used Dependency Walker when debugging my DLL loading. This showed me I needed to load cakelisp.lib for my compile-time code DLLs in order to get symbols from the parent .exe.

This introduced a very annoying difference between Windows and Linux DLLs: Windows DLLs need declspec. See this page for options of how to export symbols. This is annoying because compile-time code execution in Cakelisp expects near complete access to Cakelisp's functions and types. Windows requires I add an ugly CAKELISP_API tag to nearly all my function declarations. I could create an index file instead, but those would easily become out of sync with the header files.

In contrast, Linux linkers provide options like --export-dynamic, which will automatically export all symbols.

Another difference is that you have to load the parent .exe's generated .lib file in order to access its symbols. With Unix's libdl, the parent's symbols can automatically be resolved to from the dynamic library.

Finally, one pitfall I encountered was that you must have the exact same build settings for DLLs as the loading .exe, otherwise random crashes in CRT will occur. See /MD and /MDd options especially. I encountered these crashes/asserts during allocations usually. I had to make sure Visual Studio and my command lines all had matching options.

Environment

It is obnoxious to require a vcvars environment when using MSVC (see Use the Microsoft C++ toolset from the command line). Why can't all these settings be on the command line? The worst part of this is that the vcvars*.bat scripts are terribly slow (hundreds of milliseconds); they dominate the "nothing to do" build time. As a result, I'm now going to have to implement my own code to set these variables.

Herding PDBs is a pain in the ass. In Linux, debug symbols are included in the artifact by default, rather than having to be maintained as a separate file. If you're writing a build system which moves around artifacts and caches them (which Cakelisp does), the PDBs add complexity to that system.

Documentation

Overall, the Windows documentation is not too bad! It can be overly verbose, but the linking is generally good and the examples are helpful. I would say manpages and Windows docs are pretty close in quality.

One interesting thing is that searching for help on Windows problems seemed harder than Linux problems. The cynic would claim that evidence that Windows works better, so fewer people are asking for help. I don't buy that: I would guess there are just more amateurs running syscalls in Linux than Windows (e.g., for college courses), causing there to be more questions.

In sum

The Windows port took several days and many hours of debugging. It was my first significant "low-level" Windows experience (at work I had done some WM_PAINT UI stuff), so I learned a lot. I am proud that Cakelisp has good Windows support that doesn't require anything crazy like Cygwin or Mingw, which can have intimidating installs for many users.

I fixed bugs in the Linux version revealed by this port, which is good. It is important as a developer to have good Windows support because they still hold the majority market share, and will for some time.

© 2023 Macoy Madson.