I wrote this over a decade ago.

I shouldn't be _too_ out of date, but maybe it would help?

An I pass in a little background theorie

A scripting language is something that is not compiled entirely to assembly code but interpreted from some kind of text/bytecode interpreter. Simple scripting languages often start by just throwing simple commands together that might look more or less like a command line application. So for example from my engine's shell

if "Content" not exists "mkdir Content"
if "Source" not exists "mkdir Source"

This is a simple syntax read line by line, split for single strings into peaces and then read from the front to the end. The system then seeks for a function with the given name ('if' in this case) and passes the left strings as args to that function. As we are in C# for this special tool, it is easy to fill some configuration structs via reflection.

Lua is another but not so complicated case here. Lua instructions are parsed/compiled to bytecode and read during runtime into the Lua language runtime. Different instructions have different byte or 'OpCodes' that are treated as if they would run on a real CPU. Different OpCodes trigger different peaces of code, for example store things in memory, call Lua and/or external functions with a given set of parameters and so on.

C# works the same way. The C# compiler generates OpCodes from your source code that is byte code and looks like follows

IL_0000: nop
IL_0001: ldstr "Hello, World!"
IL_0006: stloc.0
IL_0007: br.s IL_0009
IL_0009: ldloc.0
IL_000a: ret

(Wikipedia)

Now to write a scripting language, you would need the following:

• A tokenizer that will evaluate and split your source files into tokens. From my experience, a length of 256 bit/ 1 byte words is enougth to even parse a C++ style language properly
• (If you want to support the convinience of a preprocessor) A preprocessor that handles preprocessor tags like #define, #if, #ifdef, #elif, #else, #undef and so on. This is just a consumer class that reads tokens one by one and converts output by for example replacing any occurance of a define with the defined value or handles linking unclude files together
• The parser unit processing tokens and generates OpCodes from certain rules. This rules define for exmaple that a - 2 is a valid subtraction operation and prints out matching OpCodes for

  
  OpCode:Subtract OpCode:Identifier<a> OpCode:IntegerConstant<2>

   

• And last but not least; An interpreter that understands how to handle certain OpCodes. This isn't much different than the systems above, it reads those OpCodes one by one and applies certain rules that result in certain actions of your engine code. The above example maybe end in this (simple) code

  
  bool ReadInstruction(Drough::TokenStream& opCodes)
  {
      uint64 streamPos = opCodes.Position();
      bool result = false;
  
      switch(opCodes.Get())
      {
          case OpCodes::Add_I32: result = AddInstruction_I32(opCodes); break;
          case OpCodes::Sub_I32: result = SubInstruction_I32(opCodes); break;
      }
  
      if(!result)
          opCodes.Position(streamPos); //reset for error analysis
  
      return result;
  }
  ...
  bool SubInstruction_I32(Drough::TokenStream& opCodes)
  {
      int32 op1;
      //read either from memory, a register or constant value
  
      int32 op2;
      //read either from memory, a register or constant value
  
      op1 -= op2;
      //restore either to memory or register
  }

  And of course you can also handle function pointers here. Maybe with a prefix if it is an external or <your-script-language> internal function call. External function calls may then be executed with a C-style function pointer or wrapper struct arround one and the arguments passed to it in your script code passed as a piece of stack memory (where stack memory means a block of memory you allocated before, that is handled like a stack of data). I did this several times before with different (other) systems like my Task Scheduler that is capable to handle functions as C# like tasks or runtime compiled lambda expressions

At the end, there are fore sure several options you can use, either Lua as the above posts suggested or you can add a Mono Runtime to your C++ code like for example ElectronJs does to have C# run as WebApp. Mono has a how-to for this topic here while you should know that using Mono, you need a compatible Compiler to make CLR OpCodes from your C# source

I'm interested in this topic as well. I'd like to add Python or Ruby scripting to my GUI library. PyBind11 and SWIG seemed like natural choices to make the connection between C++ and the scripting language but I haven't really had time to explore these options much.

Basically I just need something to make the bindings without generating a giant mess. I don't really like LUA much. Too much like assembler code with all the pushing and popping junk on the stack. LUA lacks the high level constructs I want in my scripting engine.

At end of day you want performance in games and this is what LUA provides with the OpCode approach, simple performance and that is what it makes successfull

I used to use Lua quite a bit back in the day when it was fairly new. I even had some email discussions with the boys from Brazil.  In general I liked it. However I then learned JavaScript which I love. This is probably because I knew Scheme before and I instantly recognized it as a form of Scheme (which itself is a dialect of lisp) with a more standard syntax.  In any case when I get to that stage with my engine I think I'll try to see if I can use JavaScript for my extension language.

You can also roll your own. I've done about three little scripting languages over the years and it's really not as hard as you might think. However unless you have a specific need to do such a thing, or are just doing it out of interest, you might consider it a wast of time. 

Another reason for running your own scripting language or CLR could be the to not expose any engine function to the outside. This is a thing of closed source engines for example from the big players so in theory an external dependencie could be modified to expose some functions to modders or whatever is prevented