Update: See feedback on /r/gamedev and /r/gameenginedevs.

I originally wrote about this as a reply to this comment (I am /u/makuto9 on reddit). To me, the comment is arguing that learning how games can be built from scratch is useless, because game engines can accomplish so much more in the same amount of time.

In this article, I don't necessarily want to argue for writing games from scratch. I'm mainly arguing that it is still valuable to learn how games work at that level, even if you never code games at that level.

Hitting walls

I think one major thing people omit when talking about game engines is when you try to do something the engine isn't already designed to do (more on this later). You hit a wall that seems insurmountable, especially if, like Unity, you don't even have source code to change.

Learning about game development without using an engine is eye-opening because you learn there are no more "artificial" walls. It's just you and the graphics API, no millions of lines of code for features you don't use getting in the way, and confusing you.

If you need to simulate 10,000 entities for your city sim, for example, chances are Unreal's Actor class is going to bring the framerate to its knees. You've just hit a wall, and now you need to make this whole new system to circumvent Unreal's. When you've only ever used an engine, you'll instead think it can't be done, that the engine just can't handle it.

When you start from scratch, you have a better idea what the computer is actually capable of, that 10,000 entities is within reach, as long as you program carefully, keep your cache locality good, etc. etc. You've broken out of the box, now you're dealing with the actual box, i.e. your computer's real physical constraints. Engines tend to have so much complexity in-between you and your machine that finding the real walls is much more challenging.

Time box

There is another box, which is the box of things you're capable of implementing in a reasonable amount of time. Engines often, but not always, help make this box larger, but mostly for certain types of things. For example, engines help easily gain modern rendering features, but not gameplay features.

It's also important to know that some things are made harder by using an engine, i.e. the box is smaller. The classic example for me is procedural generation: engines typically have advanced binning schemes, baked lighting, etc. optimizations all designed to serve mostly static levels, which covers most games. But as soon as you want procedural levels, characters, etc., all the sudden all that code is useless, and potentially even harmful, because the engine is designed with different assumptions.

Implementing gameplay

The time required to implement gameplay features I will argue is largely unchanged with or without an engine. No matter what, if you want to get a good feeling character controller, you're going to need to write finely tuned behaviors for your game.

The eye-opening example of this for me was the idea of surface followers, which are custom code designed to keep your player character "glued" to the floor, stairs, elevators, etc. they walk on. If you take the "engine approach" and slap in a physics engine to handle character movement and gravity, you will quickly encounter issues with the cases I described before: the character will seem to try to jump down the staircase, their head will hit the elevator ceiling, they'll slide right off the moving platform, etc.

This surface follower code is going to look more or less identical whether you use an engine vs. your own thing (low-level libraries, OS/graphics APIs only, etc.). The difference for me personally is that using the engine feels more like jumping through hoops to figure out their myriad APIs, inherit from all the proper UCharacter classes, etc., whereas my own setup is so much smaller that I can focus on what the feature actually needs, i.e., what fundamentally needs to happen¹.

Engine design

Engines are absolutely designed with certain classes of games in mind. They need to be, otherwise they would be too general to be considered an "engine".

Unreal is built for 1st- and 3rd-person 3D action/adventure games. As soon as you make a procedurally generated, massively multiplayer, thousand-agent sim or something, the engine will be fighting you more than helping you. While I am not very familiar with Unreal nor Unity's 2D facilities, I hear they are lacking compared to the much smaller Godot's, because the former were not designed with 2D games as a priority.

Making good software requires focused effort, and no piece of software can do everything better than two pieces of software with more focus each. An example of this is Unreal vs. Blender: Unreal doesn't try to compete with Blender on 3D asset authoring, because it would clearly be inferior. Blender tried to have a game engine, and it was clearly inferior.

Another angle of this is the industry one. Games are often rewarded by their novelty. If you aren't making something that pushes the limits of the engine, or requires custom technology, you might not be making something novel enough for the market.

Useful even for high-level development

I think learning the low-level is good even if you're working at higher levels, because you'll have a better understanding of what's possible physically, vs. what's only a constraint of that engine.

One of the reasons² I started learning C way back when was because the free version of Game Maker arbitrarily forbid rotating sprites programmatically. I knew it was possible, but I found this arbitrary constraint a breaking of my suspension of disbelief that I could do "anything" in that engine. They did that for business reasons, so it's not directly comparable to technological/architectural limitations, but from that point on I was committed to never again letting someone else's technology impose such arbitrary limitations.

Very high-level languages (especially graphical)

What about the case where you don't know how to program using C/C++ (or even C#), and use Unreal's Blueprints or something instead? If you're at that level, your only option is to use an engine, because you do not have the skills necessary to go without.

I would recommend you pick up a "real" programming language³, so that you can more easily recognize the constraints, both technical and ergonomic, that something like Blueprints impose. Additionally, you will become less locked-in and in turn better equipped to choose and move to the appropriate technology for your projects.

In conclusion

Knowing more isn't going to hurt, but it does take time. Understanding comes at a price, but I believe it quickly pays for itself. If you are working in a team, you will quickly become a valuable source of knowledge if you habitually dig in and try to really learn things.

When I notice I am making random small changes in a futile attempt to get something to work, it is a sign that I need to go back to the basics, read the documentation, write out the math by hand, etc.

On a larger scale, learning game development without an engine will help you develop the skill to get much larger things set up, like entire novel gameplay systems.