HGE Go work
I just realized I never actually posted about why I was doing a C binding of C++. Basically I bound most of Haaf’s Game Engine (HGE) from C++ to C. I ended up getting a bit stuck on the GUI elements and getting a nice way to let them do simple inheritance and polymorphism based on the current C++ classes. This actually wasn’t a problem for my end-game which was to take that C binding and make a Go version with cgo.
I had initially planned on this to be a pure cgo binding, but as I looked more closely at how the HGE helpers were written I quickly realized that a cgo/port hybrid might be more interesting. I’ll quickly describe HGE’s architecture so you can see how this ended up being a very natural fit.
First, HGE has a singleton class, HGE, that handles various things for the engine. Such as, providing a game loop, state management, drawing to the screen, audio output, resource loading, and a few other things. This is really the meat of the engine in which everything else is built upon. The engine also has various “helpers” which take this main singleton and add some specific higher-level functionality, like font loading, Sprite loading/rendering, GUI controls, particle effects, and a few others. As mentioned before, all functionality in the helpers are built using (and in some cases other helpers) the HGE class.
Since all the helpers were fairly straight forwardly written around the base HGE class, it was very easy to bind the HGE class to C and then to Go. After that, the helpers were pretty straight forward to port from C++ to Go.
Here’s an example from the Particle System:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | void hgeParticleSystem::Render() { int i; DWORD col; hgeParticle *par=particles; col=info.sprite->GetColor(); for(i=0; i < nParticlesAlive; i++) { info.sprite->SetColor(par->colColor.GetHWColor()); info.sprite->RenderEx(par->vecLocation.x+fTx, par->vecLocation.y+fTy, par->fSpin*par->fAge, par->fSize); par++; } info.sprite->SetColor(col); } |
And this is the ported code to Go:
1 2 3 4 5 6 7 8 9 10 11 | func (ps *ParticleSystem) Render() { col := ps.Info.Sprite.Color() for i := 0; i < ps.particlesAlive; i++ { par := ps.particles[i] ps.Info.Sprite.SetColor(par.color.HWColor()) ps.Info.Sprite.RenderEx(par.location.X+ps.tx, par.location.Y+ps.ty, par.spin*par.age, par.size) } ps.Info.Sprite.SetColor(col) } |
As you can see, they do the exact same thing. The only real difference is the for loop in Go uses the array directly and the C++ code moves the par pointer in the loop.
As another example, the Sprite render function:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | void hgeSprite::Render(float x, float y) { float tempx1, tempy1, tempx2, tempy2; tempx1 = x-hotX; tempy1 = y-hotY; tempx2 = x+width-hotX; tempy2 = y+height-hotY; quad.v[0].x = tempx1; quad.v[0].y = tempy1; quad.v[1].x = tempx2; quad.v[1].y = tempy1; quad.v[2].x = tempx2; quad.v[2].y = tempy2; quad.v[3].x = tempx1; quad.v[3].y = tempy2; hge->Gfx_RenderQuad(&quad); } |
1 2 3 4 5 6 7 8 9 10 11 12 13 | func (sprite *Sprite) Render(x, y float64) { tempx1 := float32(x - sprite.HotX) tempy1 := float32(y - sprite.HotY) tempx2 := float32(x + sprite.W - sprite.HotX) tempy2 := float32(y + sprite.H - sprite.HotY) sprite.Quad.V[0].X, sprite.Quad.V[0].Y = tempx1, tempy1 sprite.Quad.V[1].X, sprite.Quad.V[1].Y = tempx2, tempy1 sprite.Quad.V[2].X, sprite.Quad.V[2].Y = tempx2, tempy2 sprite.Quad.V[3].X, sprite.Quad.V[3].Y = tempx1, tempy2 sprite.Quad.Render() } |
As you can see, this one is identical with the syntax ported to Go from C++. The biggest difference is that I tried to make the ported version as Go-like as possible. So you see sprite.Quad.Render() instead of something like sprite.hge.Gfx_Render(sprite.quad)
To be clear, the majority of the binding and helpers are written and work (mostly, as do all ported tutorial examples.) There is at least one spot that is still totally unimplemented, and that's the hgeResourceManager port to Go. This is the class that does things like parses a resource script and can (does?) pre-caches resources.
Even with the few know issues left, I decided I wanted to port the rest of HGE to Go. I haven't had nearly enough time to work on it, but even still some things are already working. It's currently depending upon cgo for SDL and OpenGL. It'll likely also use cgo for OpenAL. What currently works is some things like: input, timer, & rand are all complete, opening up a window and setting some states, and drawing lines and untextured quads (the texture code doesn't seem to be working yet.) Overall though, I'm pleased with the progress so far, and think it's going to end up working out very well.
C Bindings for C++
Lately I’ve been playing around with writing C bindings for a C++ library. Your first thought might be, “Why would you bind a C++ to C?” Those of you who have ever had to do bindings to other languages may already know the answer. In general, it’s much easier to bind C to another language than it is to bind C++ directly. Another, albeit less likely, reason might be because a client wants a C interface instead.
You may be asking your self at this point, “Why not using SWIG or (insert specific language binding tools/libraries here)?” and that’s an extremely valid question. In general, I’d say, use SWIG, unless you’re unhappy with the bindings they generate. As a specific example for C++ to Lua binding, OOLua did a benchmark of OOLua and SWIG (as well as Luabind) in which OOLua typically did better then the other two bindings as far as execution speed. That’s not to say that in real world cases you’re going to see considerable improvements over SWIG by writing your own bindings (or using something else), but it could be a reason to consider it.
The reason I’ve personally chosen to write my own is to have complete control of the resulting binding(s). With Go in particular, I don’t care for the resulting API.
There are some things you’ll want to consider when taking this route. First and foremost. You’ll have to explicitly handle each binding you may do. A change to the original API may result in breakage and the need to refactor several bindings (depending on how many you want to support.) On the other hand, SWIG allows your to fairly easily only have to change things in one location and potentially have support for as many languages as SWIG supports (though, you’re likely to only support a few, or perhaps even just one.)
Next, it’s possible your bindings may actually be slower than one’s generated by SWIG. As SWIG is a mature project, a lot of work has been put into SWIG to make it efficient and also portable. While this isn’t exactly a killer, some care must be taken to make sure your bindings don’t become a jumbled mess of non-portable/slow code.
Finally, it’s going to take considerably more time to write and maintain your own bindings in comparison to SWIG. The main reason is that SWIG obviously does 95% of the work for you with the last 5% being tweaks you may want or need. With your own bindings, you not only have to carefuly plan out your own API’s (for C as well as any additional bindings), but any refactoring has to be done in more places. SWIG is a clear winner in this case (as mentioned before.)
Now, that’s not to say SWIG is without faults. One instance in particular (for Go) the SWIG documentation has this to say:
Often the APIs generated by swig are not very natural in Go, especially if there are output arguments.
As well as:
For classes, since swig generates an interface, you can add additional methods by defining another interface that includes the swig-generated interface. Of course, if you have to rewrite most of the methods, instead of just a few, then you might as well define your own struct that includes the swig-wrapped object, instead of adding methods to the swig-generated object. This only works if your wrappers do not need to import other go modules. There is at present no way to insert import statements in the correct place in swig-generated Go. If you need to do that, you must put your Go code in a separate file.
SWIG Documentation: Go adding additional code
Basically this means if the resulting binding is very unnatural, extend or wrap the wrapper. Also, in this case, you’re starting to need special logic on a per-language binding basis. For many, this is still a cheaper situation for large code bases. For smaller code bases that need a lot of this added code in the SWIG interface file could argue there’s less reason to use SWIG.
All of that aside, you may still decide to write your own C bindings. The process is actually quite straight forward in many cases. Though, there are some things to keep in mind. For instance, templates cannot be used in C bindings. You can however bind specific types instances templates. The reasons should be obvious to anyone who understands the various stages of the C++ compiler. In short, templates are only evaluated at compile time and only for instances. Bindings in general require that type information to be compiled into the binary to be referenced later. This isn’t unique to writing your own C bindings, SWIG, OOLua, and others will all require these types to be declared ahead of time. As an example, you can’t bind std::vector without first knowing the type. You’re free to bind std::vector<int>, std::vector<char *>, std::vector<MyClass>, etc. These will all resolve to a specific type, and thus can be bound. You can’t, however, expect std::vector<flat> to work if the template was never compiled for that type.
What I’ll cover in the rest of the post are some of the basics. A full example with source is included at the end of the post that you can download and examine. It’s thoroughly commented and is actually quite straight forward. The very first thing you should know, the C bindings are written in C++, but provide C linkage. If you don’t already, is that C++ has built in support for generating C linkage with:
1 | extern "C" |
You can either put this in front of a function or type or enclose multiple statements with curly braces. Now, because you’re still in C++, you can leverage a lot of the library without having to drop to C-ism’s unless needed. For instance, when creating new C structs, you may use new/delete instead of malloc/free. Inside the C binding functions you may also use std::strings, std::vectors, and other C++ containers if convenient. Keep in mind you’ll really want to do as little in the C binding interface as possible to keep overhead to a minimum. Another thing you’ll need to do is protect the C library from seeing the ‘extern “C” {‘ lines. These will cause parse errors in C. That’s why you should protect it with:
1 2 3 4 5 6 7 8 9 | #ifdef __cplusplus extern "C" { #endif void someCLinkageFunction(); #ifndef __cplusplus } #endif |
Next, if you have complete control over both the library your binding as well as the C binding, you may want to take a bit of time to pull all enums into one file. The reason for this is that it will allow your C++ and C code to much more easily share the enums without having to redeclare them. In at least the case where you just copy the enum over to the C header, you must protect against letting it be included twice. While compiling your C interface, you’ll see “duplicate definition of FU” if you’re including the enum twice. One way around this is to put an #ifndef around it. Then, while compiling our C interface, only the C++ one is included. Note that this works because enums do not create a type that is put into the library.
Free functions tend to be fairly straight forward. The only really tricky part might be if you pass data in that must be converted. For example:
1 2 3 4 5 6 7 | int sum(const std::vector<int> &x) { int sum = 0; for(int i = 0; i < x.size(); i++) { sum += x.at(i); } return sum; } |
Might be wrapped in C like this:
1 2 3 4 | int c_sum(const int *x, size_t i) { std::vector<int> v(x, x + i); return sum(v); }</int> |
Because you’re writing your own C bindings, you have a bit of flexibility with how you implement your C interface. This would also work:
1 2 3 4 5 6 7 8 | typedef struct { int x[10]; } Sum_t; int c_struct_sum(const Sum_t *x) { std::vector<int> v(x->x, x->x + sizeof(x->x)/sizeof(int)); return sum(v); }</int> |
Here’s an example of how a C++ class might be wrapped:
1 2 3 4 5 6 7 8 9 | class Test { public: Test(); virtual ~Test(); void AFunction(); protected: int x; }; |
Might have this for the C API:
1 2 3 4 | typedef struct Test_s Test_t; Test_t *Test_New(); void Test_Free(Test_t *t); void Test_AFunction(Test_t *t); |
The method I’ve chosen here is to hide the internal interface of the struct from the user and we take care of the internal details of the struct within the wrapper. While this method still allows for inherited types to be passed as base types, this style of C wrapper does effectively hide the object hierarchy. While this might be a bit confusing if someone opts to use the C wrapper for something other than bindings, that percentage will actually be very small to non-existent. Since you have the C++ (at least headers) in front of you, you’re free to hide the details in the C wrapper.
For the implementation of the above C interface, I’ve implemented it like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | typedef struct Test_s { Test *t; } Test_t; Test_t *Test_New() { Test_t *t = new Test_t; t->t = new Test(); return t; } void Test_Free(Test_t *t) { delete t->t; delete t; } void Test_AFunction(Test_t *t) { t->t->AFunction(); } |
As you can see, The C struct has a pointer to the wrapped class. We then create a new structure and new instance of the class and pass back the struct in the _New() function. In _Delete we clean up the implementation class and wrapper struct. Finally, the _AFunction we just call the function like a normal function in C++, the only difference is we’re passing the class wrapped in a struct to a function to make the call for us instead of calling it directly.
As a sort of note, inheritance works exactly the same. You can even pass the child struct to Test_AFunction with a type cast and it will work as you’d expect it. In this case, you may want to provide a function or define to do the typecast automatically so you can pass it to a class prefixed with the same class name (so if you have “class Inher : public Test;” You might “#define Inher_AFunction(i) Test_AFunction((Test_t *)i)”)
When you get to polymorphic abstract classes, things get a bit more tricky. The method I’ve opted to use is have one struct which like before hides the underlying implementation, and a second which provides function pointers to the C functions which are to be used by the object. This second struct is then passed to the _New function and a specific instance of the abstract class is created using that struct with a pointer put in the first struct and passed back tho be used as before. Perhaps an example will be clearer. Given the following C++ abstract class:
1 2 3 4 5 | class TestAbst { public: virtual ~TestAbst() {} virtual void Print(const char *str) const = 0; }; |
I’d define this in the C header:
1 2 3 4 | typedef struct TestAbst_s TestAbst_t; typedef struct TestAbst_Impl_s { void (*Print)(const char *str); } TestAbst_Impl_t; |
You can see that the implementing structure provides a function pointer that must be setup by the implementing code. The implementation wrapper will be defined like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | class _implTestAbst : public TestAbst { public: _implTestAbst(TestAbst_Impl_t *tai) : tai(tai) {} void Print(const char *str) const { tai->Print(str); } private: TestAbst_Impl_t *tai; }; typedef struct TestAbst_s { TestAbst *ta; } TestAbst_t; TestAbst_t *TestAbst_New(TestAbst_Impl_t *tai) { TestAbst_t *ta = new TestAbst_t; ta->ta = new _implTestAbst(tai); return ta; } void TestAbst_Free(TestAbst_t *ta) { delete ta->ta; delete ta; } void TestAbst_Print(TestAbst_t *ta, const char *str) { ta->ta->Print(str); } |
So, as I said before, we create a derived class from the abstract class which takes the implementation struct for its constructor and store the pointer to that struct internally. (Note this is a trivial example. You may actually want to either copy the struct or maintain ownership of the struct so it can be properly freed. This currently assumes either the caller is going to explicitly delete the struct or that it was a non pointer passed by address.)
If you're wanting to bind C++ to C for other language bindings, this should give you a good idea on how to start. That said, after having done this for a current project, I'd recommend against manually doing this.
As an anecdote to finish this off, I'll provide a description of the project for which I used this technique. Basically the game engine has two parts: the core which deals will all the low-level stuff (such as setting up the rendering context, provides input handling, resource loading, etc) and the other part are helper classes that make dealing with fonts, particle effects, etc easier. The core is basically a singleton class which was trivial to wrap to C. The helpers were kind of here-and-there. Most are single classes, but there's some obvious interaction between them and a few that implement GUI controls make use of inheritance. As you've seen, inheritance isn't trivial, and handling this correctly for various cases can be error prone. The route I ended up going was wrapping the core class to C, binding the C to Go, and then porting the helpers to Go.
This has an immediate obvious drawback. We now have two implementations. A bug in one, may or may not show up in the other, so it's twice as much work to make sure everything is working as it should. Another issue is that any other binding will now need to reimplement all these helpers as well. Because of this later point, I ended up wrapping all the C++ classes to C (well I didn't actually finish, but I was close enough for now.) This way, any other binding can use the wrapped C if desired. There was one reason for me to port the C++ over to Go instead of using the wrapped C; because I wanted to improve my understanding of Go. This also had an added bonus of verifying the Go binding of the core wrapped and bound class from C++ as working correctly in Go. It also allowed me to tweak the API a bit because I was already using the main API in the target language! Obviously if I had interest in binding to other languages, such as Lua, I'd probably want all of them using the same C wrapper (or as I said before, SWIG), but since that's not really my plan at the moment, my current approach seems to be working well enough.
I'll leave it at that for now. This wasn't a full tutorial of binding C++ to C, but it should give you a pretty good start if you're really interested in taking this route but aren't exactly sure where to begin. (Note that this was written oven several weeks when I had a few minutes here and there, so please let me know if you spot any glaring issue with this post.)
Source code for this post: c_binding-0.1.tar.gz
Guitar Painting and Modifications
This post is a bit different than what I normally write about. This post is about my guitar that I’ve had now for about eight years. It was a birthday present and was basically refurbished by one of the guitar shops in my home town. The guitar, a Peavey Raptor Exp Plus, originally looked like this:
This was fine and all, but after eight years and quite frankly wanting to upgrade the internals, I decided I’d take the time to first paint the guitar and also modify the pickguard. I have, for some time now, wanted to switch the pickups to H-S-H and with needing to route out the guitar, this was really the best time to make any other modifications (including painting.)
I first stripped the guitar and pickguard of all hardware and electronics. I then made a template from the current pickguard out of newspaper, which I then free hand cut out into a new design I had envisioned. It ended up being exactly what I wanted so that first template was later used to put the pattern on the real pickguard to cut out.
Just showing the guitar body totally stripped.
The guitar with the original pickguard laying on top
The guitar with the new template pickguard laying on top
After I had this picture of the new pickguard template I decided to GIMP the image a bit to see roughly what I’d like the guitar to look like. I basically just traced over the pickguard to make it white and I colorized the guitar body to about the color I wanted. I was wanting to go with an orange body and a white pickguard. Basically it came down to me needing to use the existing pickguard so I didn’t have to buy a new black pickguard sheet to start fresh.
Just a quick mockup of what I wanted the guitar to look like.
Same as before, but this also has the H-S-H drawn it to kind of give me an idea if everything would fit.
At about this point I was ready to start. I don’t have the tools to route, sand, pant, etc, but my father does. So, I packed up everything and took it to my parents’ house so I could prep it, route it, paint it, etc. I unfortunately didn’t take any pictures of the process, but basically I started with the pickguard. After transferring the pattern to the pickguard with a dry-erase marker, the new shape was cutout with an air grinder/cutter. It was then ready to rough sand the edges to get it to the correct shape. A bevel around the top edge was put on with a sanding block. Then it was sanded smooth with a fine grit sandpaper. An auto body air buffer was then used to buff out most of the scratches it acquired from years of playing.
We then moved on to take the clear off the body of the guitar and also route out for the second humbucker. We ended up going a bit too far with the routing. If we had not done something about this, the pickguard may not have covered the entire hole. Thus, we were required to Bondo it a bit to reshape the area as well as filling the three holes where the pickguard used to be held down.
After this, we were ready to paint. I ended up cheapening out and not buying paint. My father had some automotive fire red paint around that I decide to use. We applied a couple layers of red, and then a few thicker layers of clear. This ended my first weekend and I had to go back home before we could finish.
After three weeks, I was able to return to a fairly shiny (albeit flawed) red guitar body. There were a few sags in the clear, a few small insects landed in the last clear coat, and a couple other minor things. My father went about teaching me how to wet sand the clear to get all (most) the imperfections out of the clear paint. This involved lots of water and lots of careful, air powered, sanding. (We used an air powered tool to wet sand, but you can do it by hand.) His one word of warning when he handed it over, “if you see the base paint because you went through the clear, stop immediately!” Luckily at I didn’t wet sand through anywhere.
With that, we were finally ready to finish buffing the clear coat to a high shine. First we did the front, then the back, and then the sides. It was along the sides that we first accidentally buffed through the clear and into the base. Luckily, this was right about where the strap button goes on the base of the guitar. Unlikely anyone would see it there. So we decided to move on with the buffing. And then, we burned through again, this time on the underside basically directly under the neck pickup along the edge. This one was big enough that we decided to go ahead and touch it up with a bit more clear. So after we finished buffing the rest, we put a bit more clear on and waited till the next day for it to dry (since it was around 6 at night by that point.)
After that dried, we did some light wet sanding, followed by very careful buffing, and then the second stage buffing and we called it good. I put the guitar back together and everything was as it should be.
Fully assembled and painted guitar.
The backside which was totally wet sanded and buffed by me.
Look very closely, can you see where we had to touch it up?
And this is what it looks like in full.
And the back.
As you can see, it turned out very well. I’m extremely pleased with the pickguard’s shape as well as the colors. The red and white complement each other very well. It may not be the orange/white that I had wanted, but it is a huge improvement over the black. With the pickguard on, you obviously can’t see that the guitar is routed and ready to have a second humbucker put in. For this though, I’m either going to need to get a new blank pickguard to cut, or my father mentioned we may be able to plastic weld the two current pickup screw holes so two new ones can be drilled for the humbucker. In any case, that’s a bit down the road yet because I’m still not totally sure which pickups I want yet. I’m probably going to go with a less vintage sound than the TV Jones Classics that I want to buy to put in my Ibanez Artcore. After a quick search, perhaps something like the DiMarzio PAF Joe, but like I said, I haven’t given this guitar’s pickups serious thought. Perhaps now is a good time to start since the guitar looks like new again.
Chipmunk Physics
I’ve been hard at work at my project which I’m yet to call by name on here (and will continue to be that way until I have a playable demo.) First, a status update. I’ve ditched the idea of doing the entire engine myself and have now migrated to HGE (Haaf’s Game Engine) which fairly recently had a port to *nix and OS X by Ryan C. Gordon. I’ve worked a bit on the engine itself after the source release and my changes were pulled in upstream (mostly it adding a CMake build system, but I also did some things like get the tutorial/examples working, silenced warnings, etc.) Basically, I was pretty impressed with the engine and its features. Using OpenGL for 2D rendering was something I had kind of wanted to do, but had decided to just do software SDL rendering. That, in addition to being a pretty nice all-in-one 2D engine, had me sold on it. I was able to pretty quickly rip out the SDL work and put HGE in its place. What I currently have is a player that has basic animations. When I say basic, I mean, you press left, and one animation is played. You press up, and another is played. I also have the level layout rendering, and can change the (currently single) level file and have the changes take effect on the next run.
At this point I thought it would be prudent to the project to take the plunge into 2D physics engines. It was either that, or implement some basic physics myself (which wouldn’t be too big of a deal in this case, at least to begin with.) So, I did some research. The first one that I knew of off the top of my head was Box2D. The other one I found for this project was Chipmunk Physics. Honestly, this isn’t going to be a comparison between the two, because quite frankly, I didn’t even give Box2D a proper try. Not to mention that after I did a few tests with Chipmunk, it seemed like a good fit. It has a very easy to use C API, a very permissive license (MIT), it seems quite fast, and like I said, it seems like it’ll be a good fit.
The first thing I did was do a quick test with a single box that was affected by gravity. It was simple enough to implement. Next, I put the logic for the box into a class that would then take care of the Chipmunk memory management in the constructor and destructor. So far, so good. Next I added these boxes to a vector dynamically when the mouse is clicked. I then observed how the boxes reacted as they fell and collided with each other. This all seemed to work well, which made me want to try to add some things like being able to clear out the boxes without having to restart the program, giving all the boxes a random velocity, and simulating a explosion at the mouse pointer on a click. All-in-all, this all went together fairly quickly (few hours at most.)
I thought this simple demo might be of interest to others as well. So I’m going to post the code here. It requires HGE (I’ve only tested against hge-unix) and Chipmunk. Other than that, it require C++ and uses std::vector for dynamic storage. All the features I described above are implemented, and it comes in at just under 300 lines of code. The buttons for various actions are as follows:
- Space: clears all boxes from the screen
- Left click: adds boxes at the current mouse possition
- Right click: gives all boxes a random velocity
- Middle click: simulate an explosion at the mouse’s location
- Mousewheel: Increases/decreases size of boxes spawned (v0.2+)
- Backspace: Removes some of the most recent boxes (v0.2+)
- Delete: Removes some of the first boxes spawned (v0.2+)
- c: Changes from box->circle or circle->box (v0.3+)
chipmunk_demo-0.2.tar.bz2
chipmunk_demo-0.3.tar.bz2
chipmunk_demo-0.3.1.tar.bz2
Updated
Changelog:
0.1 Initial release
0.2 Added proper box rotation, ability to remove boxes (from front or back of vector), ability to change box size, and varying box mass based on size.
0.3 Ability to add circles as well (requires the image be in the same directory as the executable.)
0.3.1 final This is most likely the final release. It basically just corrects the mass of the objects.
0.3.2 Very minor update to CMake FindHGE.cmake to define HGE_HELPERS_LIBRARY as well and HGE_LIBRARIES (instead of HGE_LIBRARY) includes both.
Enjoy.
WordPress Backup Script
After my server was down for a couple days (because I was physically moving the machine to another location) I decided to check if there were any updates. Sure enough, there just happened to be an update for 3.2 and it had the usual warning of “be sure to backup your database and files” which I either ignore or partially ignore. I do typically do a mysqldump before running any of the updates, but I almost never backup the WordPress directory itself. For whatever reason, today I thought it’d be a good idea to do that.
Rather than doing it manually I thought I’d like for a script to do it for me so I can more easily do it in the future. Now, this could have been a pretty simple script with about 20 lines to include parsing arguments and running mysqldump and/or tar to backup the database and the WordPress directory, respectively. Of course, if that were the case, there wouldn’t be much to write about would there? Instead, I thought it’d be nice, if I were ever to get a faster server, to be able to change some things easily like what compression method to use, or if I’m ever to change server/reorganize this one to be able to quickly/easily change the WordPress directory, database name, etc. So, the script grows a bit. After that, I thought it’d be easier to control the flow if I were to put the logic into functions. Now the script grows a bit more. I then decide to use some Bash substring logic to try to figure out the file extension, and the script grows some more. Just because I was at it, I thought, why not show the compression ratio and before/after file sizes? So the script nearly doubles in size!
After all that, and learning a clever way to return non-numeric values from functions, I have a backup script that seems to work quite well for me. It’ll likely break on other systems, and likely break on non-Bash shells, but I’ll post it here anyway because it may be of use or at least interest to others.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 | #!/bin/bash WPDIR="/srv/www/wp" # Must be dumped to stdout, usually -c for bzip2, gzip, and xz COMPRESS="bzip2 -c9" # If DB_NAME is left blank, we'll try to fill these in from the WP config DB_NAME="" DB_USER="" DB_PASSWORD="" DATE="`date +%Y-%m-%d`" # This function tries to get the database info from the WordPress config # TODO write me :) function get_database_info() { echo "To be written" exit 1 } # "Returns" the extention based on the COMPRESS variable # Keep in mind that only one echo can be in this function, and it's used with # a call to it via: `file_ext` function file_ext() { local EXT="${COMPRESS:0:2}" if [ "$EXT" = "bz" ]; then EXT="bz2" fi # our "return" value echo $EXT } # $1 is the old file/dir and $2 is the new file/dir function compressed_stats() { if [ "$#" -ne 2 ]; then echo "Incorrect amount of arguments passed to compressed_stats" exit -1 fi local OLDSIZE local NEWSIZE # Check if the passed argument is already a number. # This fails if the file passed in is only numeric. # This check also fails if $1 is a decimal number, lucky for us, it's a byte # count # Credit goes to jilles on StackOverflow for pointing this out. case $1 in # It's most likely a file ''|*[!0-9]*) OLDSIZE="`du -b $1 | cut -f1`" ;; # Otherwise, it's a number, just set the size *) OLDSIZE=$1 ;; esac case $2 in ''|*[!0-9]*) NEWSIZE="`du -b $2 | cut -f1`" ;; *) NEWSIZE=$2 ;; esac # We were most likely given filenames that don't exist if [ "$OLDSIZE" = "" -o "$NEWSIZE" = "" ]; then echo "Not giving stats on files that don't exist!" return fi # Give some statistics about how much the database was compressed if [ "`which awk`" ]; then echo -n "compression ratio: "; echo - | awk "{ print $OLDSIZE/$NEWSIZE }" echo -n "Old size: "; echo - | awk "{ print $OLDSIZE/1024 }" echo -n "New size: "; echo - | awk "{ print $NEWSIZE/1024 }" elif [ "`which bc`" ]; then echo compression ratio: `bc $OLDSIZE/$NEWSIZE` echo Old size: `bc $OLDSIZE/1024` echo New size: `bc $NEWSIZE/1024` fi } # Dump the database function dump_db() { local FILENAME="wp_backup_$DATE" local EXT="`file_ext`" if [ "$DB_NAME" = "" ]; then get_database_info fi echo "Dumping mysql database to $FILENAME" mysqldump -p"$DB_PASSWORD" -u$DB_USER $DB_NAME > $FILENAME echo "Compressing $FILENAME with $COMPRESS" $COMPRESS $FILENAME > $FILENAME.$EXT compressed_stats $FILENAME $FILENAME.$EXT rm "$FILENAME" } # Backup the WordPress directory function backup_wp() { local EXT="`file_ext`" local FILENAME="wp-$DATE.tar.$EXT" local CURPWD="$PWD" echo "Backing up $WPDIR to $FILENAME" (cd $WPDIR/..; tar -cf - -X $CURPWD/$0.excludes "`basename $WPDIR`") | $COMPRESS > $FILENAME compressed_stats `(cd $WPDIR/..; tar -cf - -X $CURPWD/$0.excludes "$(basename $WPDIR)") | wc -c` $FILENAME } # Help dummy function print_help() { echo "Usage: $0 [-d] [-b] [-h]" echo " -d Dump wordpress database" echo " -b Backup WordPress directory" echo " -h This message" } # tar fails without this file, make sure it exists [ -e $0.excludes ] || touch $0.excludes # Parse our arguments if [ $# -ne 0 ]; then while getopts ":dbh" Opt; do case $Opt in d) dump_db ;; b) backup_wp ;; h) print_help ;; *) echo "Unknown option"; exit ;; esac done else # Default to backing up the database and directory dump_db backup_wp fi |
As I mentioned, I learned a way to pass non-numeric values back from shell functions. Normally, when you return a value from a shell function you do something like this:
1 2 3 4 | function f() { [ -e "$1" ] || return 0 return 1 } |
In this silly example, I return 0 if a filename passed in exists, and 0 if it does not. If you wanted to return something like “This is a returned string” with “return”, it’d fail because “return” returns a numeric value only. There’s a trick you can use that I hadn’t thought of before (and one I found while browsing online, but I can’t seem to find the link again.) Basically you have a single (executed) “echo” in the function. Then you can call the function with `func`. So the above example would be written as:
1 2 3 4 5 6 7 8 9 10 | function f() { if [ -e "$1" ]; then echo "Exists" else echo "Failed" fi } v=`f "test.file"` echo $v echo `f "file.test"` |
This allows for some interesting possibilities. For example, in my backup script, I used it to return the extension for the compressed file. How the script is currently implemented, this is a totally unneeded function as the code could be executed once, and be done with it. I mostly did it to try out this new technique, but also, in the future, I may want to use different compression methods for the two steps. E.G. I’d use xz -c9 for the database compression, but bzip2 -c9 for the wp directory compression. I’d want that, because xz uses a lot of RAM (which my server doesn’t have).
– Updated 2011-12-13 to make script more fail proof for numeric test –
CrabEmu for the Didj
Well, I’m releasing what I’ll call version 0.1.0 of CrabEmu for the Didj. It includes a custom built rootfs with no questionable code from the LF distribution (for instance, their scripts for checking various things like USB connections.) This initial release requires a DJHI or other third party card that is SD capable. The SD must also have two partitions: Fat32 & Ext 3. Because of the later, you’ll also need an OS that’s able to read/write to Ext 3 file systems.
The distribution itself has CrabEmu startup when the system is powered on. This port uses SDL Audio, Video, & Input. The code will eventually make its way upstream after I work out a few more issues with the SDL (and Didj) port. For now though, the patch is provided in the archive. The supplied README also has more information on what’s needed, how to install, what works, what doesn’t, what may be included in a future release, and more.
I’ll try to make some time tonight to update this post with video and pictures of the emulator in action. For now, here’s the link to download it and give it a spin.
You can download it here:
http://dl.dropbox.com/u/27591164/crabemu-didj-0.1.0.tar.bz2
Sega Master System Coding
Besides updating my server, I’ve been quite busy with some Sega Master System (SMS from here-on-out) coding. I’ve decided to switch to SMS for now instead of Game Gear for now. It seems I’ve kind of went my own way with this project. From the looks of it, most people either use WLA-DX or, to a much lesser extent, z88dk. I don’t see too much about any other assemblers/compilers. Granted, I haven’t looked terribly hard for the topics though. So even from the get-go with using z80asm instead of WLA-DX I was using a different assembler than the rest.
I did an evaluation after a bit of a hiatus and it was pretty clear that I still have a long ways to go in terms of how I organize my assembly code. Enter C.
I had mentioned before that I had considered trying to use SDCC for a C compiler, but when I first started playing around, I didn’t have much of a foundation to work from. Really, the initial tests I did in pure Z80 proved to be very useful when it came to getting SDCC working. I had no problems editing the default crt0.s to work with the SMS. Getting that to compile and run on CrabEmu ended up being easy too. From the start, I knew I’d probably want some kind of utility library to help me out along the way. The first thing I did was create a couple simple macros to that do in/out in Z80 assembly so I could actually do something useful with the hardware. From there I converted my assembly to C and used the in/out macros.
With basic I/O in place, I quickly went on to write some functions. First I changed in/out to functions, added some register defines, tested, and things still worked. With some excitement I went to write some utilities functions for the SN79489 and TMS9918a. All of these functions ended up calling the in/out functions. Did some testing, worked out a few bugs here and there, but for the most part these worked well too. From there I decided it was time to try some real usage. The first thing I did was did a clear_vram function. The code was basically this:
1 2 3 4 5 6 7 | void clear_vram() { uint16_t x; tms9918a_vram_set_write_address(0); for(x = 0; x < 0x4000; ++x) tms9918a_ram_write(0); } |
tms9918a_vram_set_write_address sets the VDP’s RAM address to the start of the RAM, and then loops through the entire 16K writing zero’s to every byte. After this I did a few tiles (and after struggling for quite some time) drew them to the screen.
There was a problem here. It took a couple seconds before my couple tiles were displayed after the bios screen. I ended up blaming the fact that tms9918a_ram_write was a function call that called another function that ran inline assembly that then returned from a return. This may not sound like much, but on a Z80, 32K of function calls add up to quite a few CPU cycles and memory access. So I went to the SDCC documentation and found something very useful under the Z80 Storage Class Language Extensions: “sfr (in/out to 8-bit addresses).”
1 2 | __sfr __at 0x78 IoPort; /* define a var in I/O space at 78h called IoPort */ IoPort = 0x01; |
generates the following assembly code:
1 2 | ld a,#0x01 out (_IoPort),a |
Alright, you've got my attention with that. Sound extremely straight forward, even if it is an SDCC only language extension. So I tried it out. I created the following macros:
1 2 | #define io_vdp_data __sfr __at 0xBE io_vdp_data; #define tms9918a_write(d) io_vdp_data = d |
Replaced tms9918a_ram_write with tms9918a_write. Ran it again, and now the clear_vram ran in roughly half the time (ok, this is just a best guess since I didn't do any kind of real performance measurements.) So I ended up defining a few more macros to expose all useful SMS I/O ports. With these in place I can do useful things like:
1 2 3 | if(!(io_ab_reg & 0x20)) { // Do something on player one press of button 1 } |
Where I'm currently at is I draw the entire tile based level map and can do trivial animation on tiles. I can also output trivial tones, but I've been focusing on graphics currently so I don't have too much to say about sound right now. It's probably going to be cutting it very close for submitting something to the 2011 SMS Coding Competition, but I've been trying my hardest to keep on it. I have a feeling that I'll miss the deadline though. Hopefully I'll be able to get it mostly working since it seems like a good deal of work went very smoothly the last few days.
A few other random thoughts on the subject. While I'm currently targeting the SMS, I'll probably also a "port" of this current project to the Game Gear. It'll take advantage of the much larger color palette for sure. On the other hand, I'll also have to have at least a basic AI in place for single player play. As for z88dk and why I went with SDCC instead, well, to start with because I didn't know about z88dk. Later because I was having fun writing my own functions for my project. Basically, that's all there is to it. It was more fun to do it myself then to get started faster. Eventually the GPL'd project source and LGPL'd library source (still not totally sure on these licenses) will be released. For now though, I haven't released anything, and the library isn't in a very releasable form yet. Hell, I'd even say that without fairly intimate knowledge of the SMS/GG hardware, it's not even that useful. Another small utility that I wrote before I noticed there was (most likely) one already was a header checker/correcter. This should fix the header so it can be played on actual hardware. It was thrown together extremely quickly and doesn't do much besides add in the correct checksum and a few other things. Again, it's likely not going to be terribly useful, and there's already the source to a tool that does basically this. In any case, it's currently integrated into my build processes and happens automatically. It seems to work fine against a real bios dump and the CrabEmu-Bios. This leaves me with little reason currently to change it to something else.
In any case, that's what I'v been primarily up to for the past 3 months. Really though, the C code has all been over the last month (Jan 27 was the Git commit date.) I'll try to remember to post an image of the current level map animation that I have.
Server updated
I’ve now updated my home server to Debian 6.0. In the process I’ve decided to also switch to lighttpd from Apache HTTP. I do miss the fact that mod_chroot in Apache was able to basically take care of the dependencies for me and build the chroot daemon startup, but it wasn’t too much work to get everything up and going in a chroot. It ended up one of my plugins (Last.fm for WordPress) was causing the entire site to stall while it waited for a response, so that’s been disabled now. It’s nice having my server up and running again. It’s also running ddclient because my IP seems to change fairly frequently. All-in-all it was well worth getting set back up.
Adventures in Lua Binding
I’ve mentioned before (actually, probably only in that last post) about a game I’ve been working on on and off for the last few years (I think I said three, but it’s actually been five, with it being inactive for most of that time.) I had decided long ago that it would probably be scripted with Lua and I’m just now finally getting around to experimenting with that. I’ve ultimately decided it’s going to either be using OOLua or Luabind. I’ve given both a fairly quick look over and attempted to use each to bind a specific class in my project.
For the sake of conversation I’m going to use a very basic C++ class that’s based on my project’s Configuration class. I’ll go through and show how both libraries bind the class to Lua.
First, here’s the C++ class:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | class Configuration { public: enum ConfigurationValue { // String values DATADIR, MODE, // Int Values RESOLUTIONX, RESOLUTIONY, DEPTH, WINDOWED, SOUND, MUSIC, ENUMCONFLENGTH }; Configuration(std::string directory = "") { // if we don't have a path, save the config in the same dir if(directory == "") { m_Cfg = "config.cfg"; } else { m_Cfg = directory + "/config.cfg"; } // Then we'd do things like loading the configuration/creating a new one } void set(ConfigurationValue cfg, std::string value) { m_ConfStr[cfg] = value; } void set(ConfigurationValue cfg, int value) { m_ConfInt[cfg] = value; } std::string getStr(ConfigurationValue cfg) const { return m_ConfStr[cfg]; } int getInt(ConfigurationValue cfg) const { return m_ConfInt[cfg]; } bool save() { std::cout < < "Saving (but not really) Configuration" << std::endl; std::cout << "datadir = " << getStr(DATADIR) << std::endl << "mode = " << getStr(MODE) << std::endl << "resolution_x = " << getInt(RESOLUTIONX) << std::endl << "resolution_y = " << getInt(RESOLUTIONY) << std::endl << "depth = " << getInt(DEPTH) << std::endl << "windowed = " << getInt(WINDOWED) << std::endl << "sound = " << getInt(SOUND) << std::endl << "music = " << getInt(MUSIC) << std::endl; return true; } bool load() { set(DATADIR, "/usr/local/game/some_game_dir"); set(MODE, "opengl"); set(RESOLUTIONX, 800); set(RESOLUTIONY, 600); set(DEPTH, 32); set(WINDOWED, 1); set(SOUND, 1); set(MUSIC, 1); return true; } private: std::string m_Cfg; std::string m_ConfStr[ENUMCONFLENGTH]; int m_ConfInt[ENUMCONFLENGTH]; }; |
This code should look quite trivial to anyone with a background in C++, but not so trivial that it didn’t introduce some challenges in binding them with each library (explained later.) All this class is doing is mimicing what a real configuration class might do, and that is, store and load configuration values. In this case, the data isn’t saved to any kind of persistance. For the sake of an example, that would only complitate things.
Here’s the basic rundown of the code:
- A constructor with an argument (in this case, one that doesn’t do anything useful.)
- An enumeration which is used to represent the values available in the configuration.
- Getters and setters for ints and strings.
- A Save function which prints out the configuration.
- A load function which sets some defaults to the configuration values.
- A few private variables to save the fake configuration path, and two arrays to store the configuration in memory.
Now, we can start to get an idea of what the binding libraries are going to need to handle.
- Constructors & overloaded constructors
- Enumerations (or at least being able to let the decay to ints)
- Member functions
- Overloaded member functions with differing parameters
So, let’s start with Luabind.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | using namespace luabind; // assume s in defined as a lua_State* open(s); module(s) [ class_<Configuration>("Configuration") .def(constructor<std::string>()) .def("getStr", &Configuration::getStr) .def("getInt", &Configuration::getInt) .def("setInt", (void(Configuration::*)(Configuration::ConfigurationValue, int))&Configuration::set) .def("setStr", (void(Configuration::*)(Configuration::ConfigurationValue, std::string))&Configuration::set) .def("save", &Configuration::save) .def("load", &Configuration::load) .enum_("ConfigurationValue") [ value("DATADIR", Configuration::DATADIR), value("MODE", Configuration::MODE), value("RESOLUTIONX", Configuration::RESOLUTIONX), value("RESOLUTIONY", Configuration::RESOLUTIONY), value("DEPTH", Configuration::DEPTH), value("WINDOWED", Configuration::WINDOWED), value("SOUND", Configuration::SOUND), value("MUSIC", Configuration::MUSIC) ] ]; |
So lets go through the above requirements:
- Supported. Luabind looks to provide good support for constructors and multiple constructors.
- Supported. Again, Luabind has a built-in method for dealing with enumerations.
- Supported. This one should obviously be supported by any C++ Lua binding library worth a damn.
- Supported. As you can see, Luabind allows/requires you to tell it what the function will be called in Lua, thus allowing overloaded member functions without any issue.
Other thoughts: Luabind seems to have pretty good support for binding C++ classes, functions, and enums to Lua. The syntax for the binding seems quite clean and well thought out.
So now lets do a binding to the same class in OOLua
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | // I use the Varadic macros, otherwise, I'd have to give the length of each // function/constructor definition (and also to the EXPORT* macros) OOLUA_CLASS_NO_BASES(Configuration) // most likely not really needed in our example here, but it forces the use // of Configuration(std::string) OOLUA_TYPEDEFS No_default_constructor OOLUA_END_TYPES OOLUA_CONSTRUCTORS_BEGIN OOLUA_CONSTRUCTOR(std::string) OOLUA_CONSTRUCTORS_END OOLUA_MEM_FUNC(bool, load) OOLUA_MEM_FUNC(bool, save) OOLUA_MEM_FUNC_CONST(std::string, getStr, Configuration::ConfigurationValue) OOLUA_MEM_FUNC_CONST(int, getInt, Configuration::ConfigurationValue) OOLUA_MEM_FUNC_RENAME(setInt, void, set, Configuration::ConfigurationValue, int) OOLUA_MEM_FUNC_RENAME(setStr, void, set, Configuration::ConfigurationValue, std::string) OOLUA_CLASS_END EXPORT_OOLUA_FUNCTIONS_CONST(Configuration, getStr, getInt) EXPORT_OOLUA_FUNCTIONS_NON_CONST(Configuration, load, save, setInt, setStr) OOLUA::Script s; s.register_class<Configuration>(); s.register_class_static<Configuration>("DATADIR", Configuration::DATADIR); s.register_class_static<Configuration>("MODE", Configuration::MODE); s.register_class_static<Configuration>("RESOLUTIONX", Configuration::RESOLUTIONX); s.register_class_static<Configuration>("RESOLUTIONY", Configuration::RESOLUTIONY); s.register_class_static<Configuration>("DEPTH", Configuration::DEPTH); s.register_class_static<Configuration>("WINDOWED", Configuration::WINDOWED); s.register_class_static<Configuration>("SOUND", Configuration::SOUND); s.register_class_static<Configuration>("MUSIC", Configuration::MUSIC); |
Again, we'll go through my above requirements:
- Supported. OOLua too has support for multiple constructors.
- Supported. At the time I started this post, it was actually broken. After asking about it on the mailing list, a bug report was filed for it, and it was fixed shortly after.
- Supported. Again, this should obviously be provided by any C++ to Lua library.
- Supported. OOLua uses a different macro than what's used to define normal functions.
Other thoughts: What I really like about OOLua is how fast the developer's response time is for fixing bugs/issues. It's also nice that it strives to be as fast as possible. I also really like the Script class. Script->run_file(string) just feels like a natural way to run a Lua script. OOLua works, and works well. That said, I'm not personally very fond of the syntax for declaring/exporting classes. Mostly, I'd like to, just as an example, not have to type OOLUA_ as a prefix for every macro.
For my pretty basic example both OOLua and Luabind support all four requirements. So I suppose you're now asking yourself, "which is better?" Ultimately, I think Luabind supports using more of C++'s features than OOLua. For instance, I don't believe OOLua supports overloaded operators, but Luabind does. Depending on the classes you're binding, OOLua may not be able to fully handle it, and you may be better off with Luabind. In most cases, Luabind will probably work for you. OOLua should not be overlooked though. Even though it may not support quite as many things as Luabind does, what it does support, it supports well and is fast. I also believe there's a lot of potential for OOLua. So go try them out for yourself.
I've included the full source to the above test bindings with a quick run of each. The code is licensed under the MIT liscense (just like Lua, Luabind, and OOLua.) It's also only been tested on Linux. The makefile should work for any POSIX environment, but the source is should be easy enough to compile anywhere. Lua, Luabind, and OOLua all need to have their headers & libraries installed or put in the project path (or really anywhere) to be included by the compiler, as pointed out by Liam.