Unit testing a new language

Unit testing is one strategy to achieve a goal: verify that a piece of software meets a stated specification. Let's assume you are more interested in the goal, instead of exclusively using unit testing to achieve it.

The question of verifying that a language meets a specification or exhibits specific desirable qualities is profound. The earliest work led to type theory, in which one usually extends the language with new syntax and rules to allow one to talk about well-typed programs: programs that obey these new rules.

Hand-in-hand with these extensions are mathematical proofs demonstrating that any well-typed program will exhibit various desired qualities. For example, perhaps a well-typed program will never attempt to perform integer arithmetic on a string, or try to access an out-of-bounds element of an array.

By requiring programs to be well-typed before allowing them to execute, one can effectively extend these guarantees from well-typed programs to the language itself.

Type systems can be classified by the kinds of rules they include, which in turn determine their expressive power. For example, most typed languages in common use can verify my first case above, but not the second. With the added power comes greater complexity: their type verification algorithms are correspondingly harder to write, reason about, etc.

If you want to learn more, I suggest you read this book, which will take you from the foundations of functional programming up through the common type system families.

You could lookup what other languages do for testing. When I was developing a language I was thinking about doing something like Python. They have tests written in python itself.

You could lookup their tests. These are some of then: grammar, types, exceptions and so on.

Offcourse, there is a lot of useful stuff there if you are looking for examples, so I recomend that you dig in :).