When we make worlds, there are maps, star charts, languages, fashion, technology, currencies, governments, peoples, and so much more. This post is about creating a magic system. More posts will follow on world-building.
We're going to make a magic system of runes and spells that can be cast on the runes. This is but one approach of many. I made a prototype toy for this activity if you wish to play. I recommend spell books be made with ink on paper under a full moon.
Runes
A rune is a symbol, a glyph, an object that you give power.
To start, let's make a rune (or two). First, make a 4x4 grid:

And doodle some runes:

If you prefer a different grid, try hexagonal:

or polar:

Have fun, give them names, powers, and meanings.
Spells
Spells are what we can cast onto runes to change them. Let's use these runes:

with this spell,

...to make new runes for our magic system.

Now we have six runes:

There are other spells we can try, and if we like them, we can add them to our spell book:

Some spells might need two runes to work:

Some spells might do strange things:

Others might only work on some runes because they have conditions:

You might need to make runes to test your spells. Sometimes an experiment can help you find the capabilities of a spell, even if you don't keep the rune you used to test it.
Now that we know how to make runes, write spells, and cast them, let's make some spell books. Let's define a spell book as closed when the spells and runes within can only generate the runes within (no surprise runes).
This is also where a magic system can get deep and sometimes complicated. What does your set of runes and spells look like? Below are some ideas of how you might diagram how they interact.

Some examples from the diagrams:
The Pair ( involution, Z/2)
Two runes and one spell that toggles back and forth.

Orbit

The sink
Every rune becomes the same rune (the sink) when the spell is cast.

Even the sink stays itself:

The Cycle and Tail
All runes will eventually end up in a cycle.

Connections
Runes and spells connect to our lives, math, language, and so much more. Can you see if your spell book maps to the same structure as something else in your world?