The overhead of std::string

Recently, I have been interested in the overhead of standard string and decided to do an investigation using Visual Studio 2010. I optimized my code for size and wrote a few different scenario functions. Firstly I noted the size of the particular std::string I am using is 20 bytes. That seems quite high. So, I turned off run-time type checking and noted the size was still 20 bytes.

OK. What next? I decided to write several functions which return a string in various ways. I know that even though this is in release build with all size optizations turned to maximum, that function alignment might be off, so I also decided in the debugger to take a look at the code. I noticed that all the functions were packed close together without any nops in between functions .

First, I will show you just the sizes of the various functions:

1. const char *ReturnAConstCharString() { return "test"; } =6 bytes
   
2. Sizeof(String ReturnAString()) { return String("test"); } = 22 bytes
   
3. Sizeof(void FillingAStringReference(String &reference)) { reference = "test"; } = 30 bytes
4. Sizeof(auto_ptr ReturnAutoPtr() { return auto_ptr(new String(""")); } = 43 bytes.
   

Wow, not what I expected at all. The one that most people think is the most optimized solution (using a reference), apparently takes more space than just returning a string. Naturally I would have expected the const char * version to be the lightest and 6 bytes is extremely light. However one can't do much with such a function, which is the same effect as referencing a static variable.

Looking at the underlying code, only (4) had a loop. So, the auto_ptr, performance wise, would probably be the poorest.

For most practical situations, the best solution is (2). Simply return the String. I wonder if this is always true for all classes? Probably not for the bigger classes however.

The next thing I wanted to investigate is the return load of each call. I mean, are the all the same weight or do they come with an overhead? The requirement of my return overhead functions is that the all return their values into a std::string. I also decided just to count instructions in the debugger, as there was really no other good way of doing it.

1. sizeof(string value = ReturnAConstCharString()) = 17 bytes and 7 instructions (2 calls).
   
2. sizeof(string value = ReturnAString()) = 43 bytes and 14 instructions (3 calls).
   
3. sizeof(string value = FillingAStringReference) = 9 bytes and 4 instructions (1 call).
   
4. sizeof(auto value = new auto_ptr(ReturnAutoPtr())) = 20 bytes and 8 instructions (2 calls).

Now, this starts to paint a more clear picture as the overhead of each method. The truth is that passing a string by reference takes the least overhead at least when being called. So, when a function is referenced frequently, this can save a lot of space when a reference is returned.

1. Const char returning: 6 + 17 = 23 bytes.
2. Returning a string: 22 + 43 = 55 bytes.
3. Fill a reference: 30 + 9 = 39 bytes.
4. auto_ptr: 43 + 20 = 63 bytes.

Ok, so, our original assumptions are starting to prove correct. References seems to be outpacing returning a string. This is probably true in terms of performance as well. But what about practically, in a program, which calls each function 5 times? Five seems like a good number for a small program.

1. Const char returning: 6 + 5*17 = 91 bytes. 10 calls.
   
2. Returning a string: 22 + 5*43 = 237 bytes. 15 calls.
   
3. Fill a reference: 30 + 5*9 = 75 bytes. 5 calls.
   
4. auto_ptr: 43 + 5*20 = 143 bytes. 10 calls.
   

Slightly larger programs probably end up calling functions which returns string at least 100 times, but probably contain up to 30 different functions. What would that look like?

1. Const char returning: 30*6 + 100*17 = 180 + 17=1876 bytes. 230 calls.
   
2. Returning a string: 30*22 + 100*43 = 660+4300= 4960 bytes. 330 calls.
   
3. Fill a reference: 30*30 + 100*9 = 900 + 930 bytes = 1830. 130 calls.
   
4. auto_ptr: 30*43 + 100*20 = 1290+2000 = 3290 bytes. 230 calls.

Still I find the idea of returning a String so much easier than a reference and frankly adding 3K extra, for this convenience is, in my mind acceptable. However, many programmers may feel that 3K is too much, or the overhead may be too great.

The main reason, why, is that to return a string, actually requires far less typing than all the above options and just for that reason alone, I usually pick this. When speed becomes an issue, I slip back to using references. I have used auto_ptrs in the past, but my feeling is that auto_ptrs are more appropriate for larger classes which exceed 30 bytes. Also not mentioned here is the overhead of allocating memory on the heap. Each call to new is much more costly that to use a stack variable.

Just remember the old adage, "premature optimization is the root of all evil", and you will be just fine.

Fast Unique File Name Generation

A colleague and I recently got into a discussion about generating unique file names, for temporary files. We discussed the different ways of doing this, using GUIDs or Windows GetTempFileName() function and other options.

I started writing file system drivers over 20 years ago, so I have watched a lot of stack traces down the file system stack. Generating a guid is much much faster, since it requires far less overhead than searching for a unique file name. GetTempFileName actually creates a file, which means it has to call through the entire file system driver stack (who knows how many calls that would be and a switch into kernel mode.) GetRandomFileName sounds like it is faster, however I would trust the GUID call to be even faster. What people don't realize is that even testing for the existence of a file requires a complete call through the driver stack. It actually results in an open, get attributes and close (which is at least 3 calls, depending on the level.) In reality it is a minimum of 20 function calls and a transition to kernel mode. GUIDS guarentee of uniqueness is good enough for most purposes.

My recommendation was to generate the name and create the file only if it doesn't exist. If it does, throw an exception and catch it, then generate a new guid and try again. That way, you have zero chance of errors and can sleep easy at night.

On a side note, checking for errors is so overdone. Code should be designed to crash if assumptions are wrong, or catch exceptions and deal with it then. It's much faster to push and pop an address on the exception stack, than to check everytime on every function for an error.

Creating Temporary Files

Recently a collegue and I got into a discussion over generating temporay file names. Several options were discussed including using system functions, etc.. On primary focus was on speed, since we had to process several million files.

Generating a guid is much much faster, since it requires far less overhead than searching for a unique file name. GetTempFileName actually creates a file, which means it has to call through the entire file system driver stack (who knows how many calls that would be and a switch into kernel mode.) GetRandomFileName sounds like it is faster, however I would trust the GUID call to be even faster. What people don't realize is that even testing for the existence of a file requires a complete call through the driver stack. It actually results in an open, get attributes and close (which is at least 3 calls, depending on the level.) In reality it is a minimum of 20 function calls and a transition to kernel mode. GUIDS guarentee of uniqueness is good enough for most purposes.

My recommendation is to generate the name and create the file only if it doesn't exist. If it does, throw an exception and catch it, then generate a new guid and try again. That way, you have zero chance of errors and can rest easy at night.

On a side note, checking for errors is so overdone. Code should be designed to crash if assumptions are wrong, or catch exceptions and deal with it then. Its much faster to push and pop and address on the exception stack, than to check everytime on every function for an error.

The Theory of Everything, Anti-Gravity and Free Energy

Someone asked me if "free-energy" is possible. I suppose free, should really mean: "fueless energy," because no device is free to build and will required maintenance. I don't really know how to build a practical device which does this, but I can explain the theory of how it might be done. With that we need to explain how everything really works, at a micro and macroscopic level.

So, what is happening is that energy is constantly radiating from all mass, as it loses energy (in the form of scalar waves, like rippling waves in a pond.) Everything in the universe is "losing energy" at a constant rate, due to its rotation. The energy trapped in the atom is radiating out and becoming space-time and why the universe is always expanding. It's not really expanding, its just that matter is shrinking. The big bang is really a misnomer. There was no bang, just this evolution. Particles are points of singularity, where the field goes outside the space-time contiuum. EM and Flavor are sub-oscillations in the extra dimensions of the sub-atomic particles. Space is where there are no-singularties, but only warps in the different dimensions (11 of them), and these warps are the field strengths as well as the energy contained therein. So, space holds energy and when it has enough, sometimes, it flashes over into a singularity (after the minimum threshold is reached.) and this is where anti-matter comes from inside of a particle accelerator. The amount of energy in a region of space, is the amount of distance the space contains, so most things keep the same relative distance to each other. If the energy within a region of space goes to zero, so does the distance (and this is called a wormhole.)

Things fall to the ground because there is a very slight shielding of energy by the Earth. This is overcome several ways: The simple way is to use a rocket engine, which applies a push or to use a wing, which causes lift. But these ways cause acceleration to be felt by the recipients and thus, the time field is altered, which makes for relativistic effects, such as the ever annoying time-dilation effect.

So, to make anti-gravity, you have to balance the energy coming from the Earth, from that coming from the cosmos. Since these are scalar waves, at high frequencies, they are difficult to shield (or even detect.) The idea is to utilize the non-linearities of the particle fields themselves, i.e. the nucleus, and perform a reverse phase-conjugation, i.e. pumping energy from one source (such as a spinning disk or fluid) into the nucleus. The military found a way to use a rotating super-fluid, and then they stuck a triangle on it (with silent rockets), and then they flew it around and people saw it, including my cousin (40 feet over his head in tehachipi.)

Ok, so free energy is therefore taking the non-homogenuity of this ZPE field (the radiating-rippling spherical lake-like waves) and transferring them into a new force. How this would be done, I am not really sure. Harold Puthof did it with rotating electrons bursts, but his device resulted in very little excess energy. Ok, so now people claim this guy, Howard Johnson, seems to have done it with magnets. It might be possible, just really don't know exactly how. All the devices I saw were deceptions, sometimes self-deceptions by their inventors. Perhaps Howard Johnson actually figured it out.

What I need is a model, first, on my computer. I have made an open-source project called video-physics on Google and with that I hope to finish the modeling of the ZPE-field. Please feel free to help.

Get SqlLite to work with .NET 4.0 and VS 2010

Of extreme use is SqlLite under .NET 4.0 and Visual Studio 2010. I am writing this short article to help those in need, get quickly up and running. This example is made with a 64 bit version of Windows 7. I made sure to activate all of the features, so that linq can be used within Visual Studio. Here is a list the process from beginning to end:

1. First step is to download ADO.NET 2.0 Provider for SQLite here.
2. Run the file SQLite-1.0.66.0-setup.exe.
   
3. Let it install into the default location.
4. Make sure to check the box allowing the installer to change Visual Studio 2010.
5. Check to make sure it installed correctly, by starting Visual Studio and opening the Server-Explorer pane, then connecting to a Data Source:
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
6. Close all instances of Visual Studio.
   
7. Download x86 and x64 versions of the system.data.sqlite dlls. For our purposes, sqlite-netFx40-binary-Win32-2010-1.0.79.0.zip and sqlite-netFx40-binary-x64-2010-1.0.79.0.zip.
8. Expand each zip into a corresponding temporary directory and label them x86 and x64 to differentiate between them.
9. Do NOT execute the installer, but rather copy the x86 files to C:\program files (x86)\SQLite.NET\bin and the x64 files to C:\program files (x86)\SQLite.NET\bin\x64.
10. The program SQLite-1.0.66.0-setup.exe, when installed, registered the incorrect files into the gac. You must point instead to the files you just copied. Open the Visual Studio command prompt in admin mode.
11. Navigate to C:\Program Files (x86)\SQLite.NET\bin and enter the following commands: gacutil /if SQLite.Designer.dll
    gacutil /if System.Data.SQLite.dll
    gacutil /if System.Data.SQLite.Linq.dll
    
12. Proceed until you have registered every dll with the GAC. After you are done there, go to the bin\x64 directory and repeat again.
    gacutil /if SQLite.Designer.dll
    gacutil /if System.Data.SQLite.dll
    gacutil /if System.Data.SQLite.Linq.dll
    
13. Download an SQLite adminstration program, such as http://sqliteadmin.orbmu2k.de/ and create a database.
14. Start Visual Studio and use Server Explorer to view this database.
    
15. Create a .NET 4.0 application project.
16. Add an App.config file to your project, by right-clicking over your project and inserting a new item: Application Configuration File.
    
17. To App.config, add the following line, directly under the configuration level:
    

    startup uselegacyv2runtimeactivationpolicy="true" supportedruntime="" version="v4.0" blockquote="">
    

18. Add a new component: ADO.NET Entity Data Model. Give it an appropriate name.
19. Choose to generate the model from the database you previously created.
20. Give the name something useful. You will be typing this namespace often.
21. Then select the tables you want and choose plural names if you want (recommended.)
22. Your model will then appear as an opened edmx file.
23. Include a reference to System.Data.Linq.
24. Go ahead and create a function and start typing your linq code.
25. i.e. from d in (new MyEntities()).myTable select d.myValue.

Have fun!

Die Singletons, Die!

Why, oh why? I love them. What could be more easy to understand than this?

class Singleton

{

public:

Singleton& Instance();

void DoSomething();

private

Singleton *_instance;

}

void func()

{

Singleton::Instance().DoSomething();

}

For one thing, what about threading? What if two threads instantiate the Singleton at the same time? Well, you could use a critical section, mutex or semaphore to serialize access. Problem solved? Yes, but still the object is created and when does it call its destructor? If I look above, the object is instantiated but not freed until the application terminates. It also allows any class to access this singleton without restriction; its methods are public. The truth is that the life-cycle as well as access is not clear, not tight, not proper OOP.

OOP methodology calls for something better. Objects must maintain tight scope; meaning they should instantiate and destruct in a predictable order. Singletons violate this principle, by letting any class instantiate and access the public methods, on demand. This can lead to difficult-to-find bugs.

There is another more OOP acceptable solution: The collaborator pattern. In this, the order of access is tightly controlled. The collaborator pattern gives a hierarchy of control to each class. For example: Instead of a singleton, we simply make singleton a friend of a parent class, which we will call Application. Application then contains a function to return the current instance of the singleton. This works on down the line, from parent to child. Each class is then assigned a responsible parent class.

class FakeSingleton

{

friend Application;

public:

DoSomething();

private:

FakeSingleton();

}

class Application

{

public:

FakeSingleton &fakeResource()

{

if(_fakeSingleton==null)

_fakeSingleton = new FakeSingleton();

return *_fakeSingleton;

}

~Application() {

if(_fakeSingleton !=null)

delete _fakeSingleton;

}

private:

FakeSingleton *_fakeSingleton;

}

Another variation is to not return an instance, but control access through wrapper functions.

This pattern therefore allows us to determine the sequence of initialization, as well as maximum lifetime. But what if we want to free the resource upon last use? A factory class can be useful in this case. A factory class, wraps the collaborator object. When this wrapper destructs, it frees the collaborator objects when the usage count goes to zero.

void Func()

{

// _fakeSingleton count incremented.

FakeSingletonFactory fakeSingleton();

fakeSingleton->DoSomething();

// _fakeSingleton count decremented, possibly freed.

}

Another nifty trick is the idea of the stack-only class:

class A{

private:

void * operator new(size_t size) {}

};

int main(){

A a;

A * b;

A * c = new A;

}

With this class, we are able to wrap our object in a stack variable.

I will post a full example on CodeProject, showing different ways the collaborator pattern can be used in C++, C# and PHP.