Embedding Lua in C# .Net – Part I

Recently I had to write a tool, which would be able to map one XML format to another.
I needed to be able to map a source XML and all its fields to a destination xml format with different kind of fields.

The list of requirements looked was looking something like this:
• Each field should be allowed a custom business rule to be applied when mapping to the destination, and these rules could be anything from doing some math, to look up a particular string depending on the nth element of any of the source XML arrays.
• It should also be possible to map a source array of elements of any type (custom included), to any destination array of any type.
• The mapping formats should be based on multiple fairly large XSD schemas
• The was no telling when business development department decided to change rules, mappings or the schemas themselves

This was looking to be a seriously pain in code modifications and recompilations of the tool, with endless support needed to maintain a valid tool.

… Or a seriously flexible and generic configuration engine that could manage every XSD schema, every mapping and every rule, so that all would be reconfigurable and the need for recompilations would be eliminated.

Thus any change in the schemas, mappings or business rules would just be configured by a user in the graphical UI of the tool itself, instantly changing the behaviour of the tool, without waiting for a new release of the tool, which would implement the new changes.

Choosing the components of the tool
I really like .Net, especially because of the many build in stuff, the memory management and the easy way a useful GUI can be implemented rather quickly.
Also I’ve been using Lua when coding some small mods for World of Warcraft, and I thought the whole WoW extension experience with Lua scripting engine was some of the most amazing I had seen in flexibility ever.

I wanted to combine the two, and get this sort of flexibility into my tool, as I didn’t quite see how I was able to fulfil my requirements without some sort of dynamic scripting engine.
Lua has very good integration with C/C++ but not so straightforward with .Net.

Making .Net and Lua talk
Go to http://www.lua.org/ and download the latest source of this great great great engine.
Lua is free software distributed in source code. It can be used for any purpose, including commercial purposes, at absolutely no cost.

The calling convention problem
Unfortunately using a standard Lua dll directly from .Net will quickly crash your program.
This is due to different calling conventions in C/Lua vs. .Net.

Calling conventions is all about how functions go about calling other functions.
In particular how function parameters should be passed and who should remove them when the function returns.
In C/C++ the __cdecl (caller cleans) calling convention is the default, and in .Net the __stdcall (callee cleans) calling convention is the default.

If you don’t know anything about calling conventions, and don’t care, then please just skip the explanation stuff and go straight to the guide helping you build a Lua dll for use with .Net
.Net implements platform invoke calls as __stdcalls per default and you cannot change this behaviour, except modifying and reassembling .Net IL code.
At the return of the call from a .Net function to a Lua C function the program will crash because the functions can’t agree about who’s responsible for cleaning up the stack, and this will corrupt the stack.

Some examination
__cdecl marked functions:
The caller is responsible for cleaning any parameters sent on the stack to the callee (the called function).

Typically it will look like this, when calling a function named C with 1 parameter:
C(5);

It will look similar to this in assembly code
004017EB  push       5                         — prepare an integer argument on the stack
004017ED  call        C (4017D0h)  — push return address 004017F2 on the stack and change instruction pointer to  4017D0h

Callee processes something and finally issues a RET instruction which pops the return address into the IP instruction pointer register and proceeds execution at:

004017F2  add         esp,4               — the caller now cleans up the stack, freeing the 4 bytes containing the 32 bit integer with the value  5.

__stdcall marked functions:
The callee is responsible for cleaning any parameters sent to it on the stack.

Calling C(5);

004017EB  push       5                         — again preparing an integer argument on the stack
004017ED  call        C (4017D0h)  — again issuing a call instruction and push return address
…
In function C do stuff
…
004017D4  ret         4
As the function C returns it now calls RET with an argument 4, again freeing 4 bytes of stack space, cleaning the 32 bit argument passed to it.

Lua with .Net integration solutions
Based on the knowledge above, there are basically two paths you can take to integrate Lua with .Net.

A.
Configure a visual studio solution to correctly build the Lua source code so that the Lua dll will be directly accessible from .Net.
This is simply achieved by compiling Lua as a dynamic dll with the calling convention of the project explicitly set to __stdcall as default instead.

The drawback
Choosing this method disables the use of some functions with variable arguments.
Some features of the Lua API can no longer be called with variable number of arguments, the caller must match the exact signature of the function.
This is for most types of applications and usages not a major drawback, but in some cases it’s a really cool feature which would greatly ease some tasks.
Im not sure precisely where this is an issue, so I would appreciate any comments about it, if somebody else can make a precise statement!

B.
Write a bridging dll between Lua and .Net in C++/CLI.

C++/CLI (Common Language Infrastructure) is Microsoft’s language specification intended to supersede Managed Extensions for C++.
With this option you can create all the bridging to Lua in C++/CLI with support for both the c++ runtime and the .net common language runtime
The produced integration assembly can then be used by regular .Net solutions, just by adding a reference to it and working on it like any other .Net assembly.

Example of a project of a Lua bridging dll written in C++/CLI is LuaInterface which can be found here

The plusses of this method are that it enables the compilation of the functions communicating directly with Lua, to be declared with the __cdecl calling convention.
Any other method that will be called from extern .Net assemblies can be declared with the __stdcall calling convention.

This enabled the use of variable argument functions
(which will just discard arguments that does not fit its signature).
This is due to the fact that the __cdecl calling convention specifies that ‘Caller cleans the stack’.
The callee does not need to know how many arguments were passed to it, and the callee is not responsible for cleaning out the arguments, from the stack upon returning from the function.

Get to the point already!
I’ll be focusing on method A, as I found it the easiest to get up and running quickly, and I did not need the bit of functionality the method prohibited.

Building the Lua Dll with __stdcall
Create a new c++ dll project.

Select DLL as the project type

After project creation, delete all the default files created in the project.
Add all the Lua source files, except ‘lua.c’ and ‘luac.c’, downloaded from lua.org to the projects header files folder and source files folder, lua.c and luac.c are standalone consoles that don’t belong in the core Lua dll.
 
Right click the project and choose properties.
In the configuration properties locate the calling convention setting like so.

Set it to __stdcall.
OK your way out.

As per default Lua is setup to be build as a library .lib, and therefore no functions are exported when building as a dll.
This can be changed in luaconf.h, by setting the corresponding pre-processor directives, but the result is not exactly what I want in regards to .Net.
I’ll show it anyway, but you can skip the addition of adding the directives.

Add the LUA_BUILD_AS_DLL define
Add the LUA_CORE define

Note that this method results in the dll containing decorated names, which is OK for any other C/C++ program that can use the .lib that goes with the DLL but not for a .Net assembly if you want to use the original functions names in your imports.
Using decorated names
Build the solution, your now almost have the final Lua core dll that you can use from .Net.

Actually I ended up with using a module definition file (.def) instead for my exports from Lua.
This ensures that the dll exposes the exported function names without decoration.

I have uploaded a word doc with the contents of a complete example .def file for the dll if you prefer using the same method as me, just paste the contents into a text file and give it the .def extension and add to dll project.
Lua dll module definition file

Change the configuration properties of the dll project to use the .def file:

Using this method, you should just skip the other exporting method, i.e. changing of pre-processor directives etc.

MS documentation on dll export using a .def file here:
http://msdn.microsoft.com/en-us/library/d91k01sh(VS.80).aspx

Handling the issues with the C runtime manifest
OK, so far so good all we need now is too handle the dependence of the C runtime, and embed a manifest in our dll that tells which C runtime too use.

From http://lua-users.org:
The distributions for Lua 5.1.3 have moved (and probably sensibly) to assuming MSVCR80.DLL is present, and provide the required manifest resource to properly handle side by side versioning of the runtime

Were going to solve this using the Microsoft manifest tool.
Place the manifest and your Lua dll in the same folder and fire up a Visual Studio command line, and execute the following:
mt.exe /manifest Microsoft.VC80.CRT.manifest /outputresource:lua514.dll;2
(1 for an EXE, 2 for a DLL.)

The documentation for mt.exe describes using [-] as command prefix, but somehow my console translates that to [!!], so I used [/] and that worked…

DLL Done
We are now ready to use the Lua dll from .Net.
Anytime you use it, just plant it in your bin folder along with the C runtime ‘msvcr80.dll’.
I don’t know why but even though the manifest lies embedded into the Lua dll, the manifest file must also still be present as standalone in the bin directory, this kind of annoys me as I’m probably doing something wrong 🙂

Viewing the exported function names of your Lua dll
Open a Visual Studio cmd.exe and cd to the directory of the dll.
Use the Dumpbin tool like this: Dumpbin /EXPORTS <name of dll>

Using Lua from a C# .Net project
Create a new .Net console application
In the example I use a console app, because Lua will then be able to output to the stdout of the console window.

I suggest you make a static class to wrap the Lua dll.
The class will contain definitions from lua.h, and the wrappers for the Lua API.

The Lua state object is the single most central part of the Lua engine, you will use this state in any call to a Lua API function from .Net.
Here’s a very small sample wrapper class that implements some basic Lua API functionality.
With this class you can open a valid Lua State, execute a Lua script held in a string, outputting some text to the console window, and finally close the Lua state.

public static class Lua
{
   //some Lua defines
   /* option for multiple returns in `lua_pcall' and `lua_call' */
   public const int LUA_MULTRET = (-1);
     
   //the first of any calls to Lua.
   //get a valid Lua state to operate on
   [DllImport("lua514.dll")]
   public static extern IntPtr luaL_newstate();
  
   //open all Lua libraries
   [DllImport("lua514.dll")]
   public static extern void luaL_openlibs(IntPtr lua_State);
  
   //close Lua
   [DllImport("lua514.dll")]
   public static extern void lua_close(IntPtr lua_State);
  
   //load a Lua script string into the Lua state
   [DllImport("lua5.1.dll")]
   public static extern int luaL_loadstring(IntPtr lua_State, string s);
  
   //call a lua function, a function can be a Lua script loaded into the Lua state
   [DllImport("lua5.1.dll")]
   public static extern int lua_pcall(IntPtr lua_State, int nargs, int nresults, int errfunc);
  
   //simplify the execution of a Lua script
   public static int luaL_dostring(IntPtr lua_State, string s)
   {
      if(luaL_loadstring(lua_State, s) != 0)
          return 1;
      return lua_pcall(lua_State, 0, LUA_MULTRET, 0);
   }
  
   /*
   public static void LogError()
   {
      //log error
      string errMsg = null;
      if (Lua.lua_isstring(m_lua_State, -1) > 0)
          errMsg = Lua.lua_tostring(m_lua_State, -1);
 
      //clear the Lua stack
      Lua.lua_settop(m_lua_State, 0);
         
      //log or show the error somewhere in your program
   }
   */
}

With our class defined we can now call Lua to do a simple calculation, this is our HelloWorld.
Codesnippet:

IntPtr lua_State = Lua.luaL_newstate();
if (lua_State == IntPtr.Zero)
{//error}

//open the Lua libraries for table, string, math etc.
Lua.luaL_openlibs(lua_State);

string luaScriptString = "TwoPlusTwo = 2+2; print('Hello World'); print('TwoPlusTwo:', TwoPlusTwo)";
if (Lua.luaL_dostring(lua_State, luaScriptString) != 0)
{//error}

//clean up nicely
Lua.lua_close(lua_State);

You should see something similar to this, outputted to the console window:

I leave it to you to add the DllImports to get the LogError method working.
Any other Lua API function can be added to the class, just remember to add lua.h definitions as you go.

This wraps up my post on integrating Lua with C#/.Net, hope you enjoyed it, and that it was useful to you.

There are lots of things I haven’t shown here, like registering C# methods to Lua and to be able to call them from Lua script.

I have added how to call from Lua into c# in another post, Embedding Lua in C# .Net – Part II
Full source code for a sample working program including a simple script engine, and a full managed wrapper can be found in part III in this post

Thue Tuxen

Advertisement

Privacy Settings

Tags: .Net, C#, embedding, integrating, Integration, Lua, Lua source code, Lua.org, script, scripting, Thue Tuxen, tuxen consulting