[CLOSED] Hybrid GM (Dev-Log)

[.]

Finally, I got a basic/untested implementation of the GlobalEvent system. Now comes the hard part. Implementing the LocalEvent system. And starting to test what I've implemented so far. The LocalEvent system is going to be harder to tame because it has to be extremely efficient and it involves a ton of internal management for which I have to find a clever way of keeping it to a minimum

I still haven't implemented the structures for entities and other things because from what I've noticed in Stormeus's version of the Squirrel plugin there will be some changes in the API so I'll have to wait until the changelog is released before I begin working on those. Right now I have just the basic structures for them to allow me to work on the rest of the plugin. More updates coming soon
Tons of new features are planned:

• Local/Global user data and tags for custom personalization. Status: Almost completed but tested only quickly.
• Entity Queues for distribution of entity changes. Status: Incomplete. Awaiting until the changelog is released.
• Area System for custom collisions. Status: Half complete. Awaiting for an alpha/preview release first.
• Extensive built-in library. Status: Incomplete. Awaiting for an alpha/preview release first.
• etc.

These will be discussed here when I get the chance to implement them.

[.]

I got 99 problems but an event system ain't one of them. Just finished implementing a primitive version of the event system. Took a bit longer than I expected because the VC:MP server is a b!tch sometimes and debugging it is painful. But that's not gonna stop me After having a few more wasted prototypes I ended up having 3 event types instead of 2. I had to make another event type which doesn't include entity filtering when it's not needed. Mostly for performance reasons. So now you have BasicEvent, GlobalEvent and LocalEvent at your disposal.

Now that I got this out of the way I can start going through the rest of the source to do a few more improvements and probably look for some bugs. And later tonight, upload the source for anyone who wants to see what's new and what I'm working on. The implementation will be incomplete of course but should be functional for testing and learning about the plugin.

[.]

Did a small and very unrealistic benchmark on the current implementation to see how many events I can attach on the onFrame callback to get an idea of how it performs. So I started by creating one huge file with about 250 events that attach to the on frame event. Something like this:

Code Select Expand

local event1 = BasicEvent(EVENTID.SERVER_FRAME);

event1.on_trigger = function(delta)
{
    ++_I;
}

local event2 = BasicEvent(EVENTID.SERVER_FRAME);

event2.on_trigger = function(delta)
{
    ++_I;
}

local event3 = BasicEvent(EVENTID.SERVER_FRAME);

event3.on_trigger = function(delta)
{
    ++_I;
}

.....
..........
...............


local event250 = BasicEvent(EVENTID.SERVER_FRAME);

event250.on_trigger = function(delta)
{
    ++_I;
}

Then I duplicated that file about 50 times. At the end I created one more file with one event to count how many events were called in that frame. Something like this:

Code Select Expand

_I <- 0;

local event = BasicEvent(EVENTID.SERVER_FRAME);

event.on_trigger = function(delta)
{
    print(delta + " ---- " + _I);
    _I = 0;
}

Then I added those files to the configuration file to be loaded on start-up:

Code Select Expand

[Scripts]
Source = bench/1.nut
Source = bench/2.nut
Source = bench/3.nut

.....
..........
...............

Source = bench/50.nut

Source = bench/X.nut
The X.nut file is the last binded event and it's the one counting the calls. Which means that we should get about 12,501 events binded to the onFrame callback. Here's a screen:

The first number is the time since last frame. I'm assuming it's milliseconds because I multiplied that float number by 1000 to get an integer. The second number is the number of events triggered before it reached the last event which is in the file X.nut. The reason I said that this is an unrealistic benchmark is that in a real situation an actual script wouldn't use such simple functions. But still 12,501 events executed in just 27 milliseconds shouldn't be that bad (if that number is indeed milliseconds).

So that's about it. A small idea of how the event system performs

[.]

The repository was made public. I'll try to upload the source before I go to sleep and if possible, also leave a x32 bit dll available for download just for anyone who wants to test anything. I'll also begin working on the wiki (which can be found on the repository) to have a lightweight documentation available for anyone who can't get the information directly from C++.

Like I said, it's incomplete and probably looks like a primitive implementation, but it's a start. I'll slowly complete the source and wiki in the days to come.

Edit:

Temporary download link for x32bit binaries with a hello world script can be found here.

[.]

Latest updates to the repository include:

• Entity iterators implemented. (Untested)
• Entity data types implementation completed. (Untested)
• Entity queues partially implemented. (Untested)
• Event filter iterators. (Untested)
• Event filter hint/specifier. (Untested)
• Partial miscellaneous data types implementation. (Untested)
• Various things removed until further testing.
• Tons of other changes and fixes too minor to mention.

Temporary download link for x32bit binaries with a hello world script can be found here.

[.]

Lately I've been thinking how to have a better administration system that everyone can use. So, I've spent some time thinking about it. But nothing ideal came to my mind for a while. Until I've remembered that I can register custom functions in SQLite. So, I've been thinking of a small hack (so to speak) to use SQLite to my advantage.

At run-time I would keep a database in :memory: that would track everything that's happening in the server. And then I'd register a bunch of functions to manipulate that data. So, now I have an expressive management system that I can use. Imagine if you could simply execute strings like:

Code Select Expand

select player_ban(id) from [players] where country_code = 'US';Bans all players from US

Code Select Expand

select player_give_money(id, 300) from [players] where class = 1;Gives 300$ to all players with the class id 1

Code Select Expand

select player_ban(id) from [players] where name = 'MasterX';Bans the player with the name 'MasterX'

Code Select Expand

select player_pos_near(id, 143.34, 764.2342, 565.643)
    from [players] where class = 0 limit 3;Teleport 3 players with the class id 0 near the specified position

Code Select Expand

select player_health(id, 0) from [players] where vehicle = 4 offset 2;Kill players in the vehicle with id 4 and start from the third player

There's way more things you can do with this. That's just a tiny example. Imagine how much code you'd have to write to do that and now you just have to execute a query string to do it.

I have a small example of this implemented in C++ to prove the concept here. Anyway, this is something I've been thinking about and I'm curious how will it perform. Would you find this useful? I wonder if it's worth to implement something like this and whether it's safe or not.

[.]

I forgot to mention guys but the development of this plugin was resumed here. It's a completely revised version of the previous one that had some limitations. It's still under development and when I get the time I'll explain the new features that I've added and the issues that I've come across which forced me to revise the whole thing.

EDIT: If you have any suggestions, requests, bugs to report then you can always find me at @LUNet on channel #SLC

Keep in mind that the plugin is barely 15% completed and some of the features that I have in plan are missing currently.

[.]

Added hashing to the standard library:

Code Select Expand

local crc32 = CHashCRC32(); // Create CRC32 encoder
local keccak = CHashKeccak(); // Create Keccak encoder
local md5 = CHashMD5(); // Create MD5 encoder
local sha1 = CHashSHA1(); // Create SHA1 encoder
local sha256 = CHashSHA256(); // Create SHA256 encoder
local sha3 = CHashSHA3(); // Create SHA3 encoder

local data = @"f*in awesome";

local crc32_hash = crc32.compute(data);
local keccak_hash = keccak.compute(data);
local md5_hash = md5.compute(data);
local sha1_hash = sha1.compute(data);
local sha256_hash = sha256.compute(data);
local sha3_hash = sha3.compute(data);

if (HashCRC32(data) == crc32_hash) print(crc32_hash);
else print("individual hash didn't match with encoder instance");

if (HashKeccak(data) == keccak_hash) print(keccak_hash);
else print("individual hash didn't match with encoder instance");

if (HashMD5(data) == md5_hash) print(md5_hash);
else print("individual hash didn't match with encoder instance");

if (HashSHA1(data) == sha1_hash) print(sha1_hash);
else print("individual hash didn't match with encoder instance");

if (HashSHA256(data) == sha256_hash) print(sha256_hash);
else print("individual hash didn't match with encoder instance");

if (HashSHA3(data) == sha3_hash) print(sha3_hash);
else print("individual hash didn't match with encoder instance");

Working on an adding a fully featured IRC client on the standard library as well. It's actually implemented. I just need to work on the event processing loop to integrate it into the server frame event. Next post will be about the integrated IRC client.

[.]

The IRC implementation is almost done. All that's left to implement is the DCC and file transfer things. Which is not something that even makes sense to have on a game server. But because this is supposed to be a fully compliant IRC client, why not.

A sample of how to it looks like:

Code Select Expand

// A single IRC session can connect to one IRC server but multiple channels on that server
local irc_session = IRC.Session();

// Create a basic event that listens to the server start up event
local startup_event = BasicEvent(EVENTID.SERVERSTARTUP)
// Tell the event instance to call this function when it's being triggered
startup_event.on_trigger = function()
{
    // Connect the IRC session to this an IRC server (on success it returns 0)
    // Args: server_name, port, nickname (optional: password, username, realname)
    if (irc_session.connect("irc.liberty-unleashed.co.uk", 6667, "SqMod") != 0)
    {
        // Output the last error message (if any)
        print("Unable to connect to IRC server: " + irc_session.err_str);
    }
    // Can't join channels here because the session didn't connect yet

    // Tell the session to remove extract SLC from [email protected]
    // automatically when sending the `origin` to events
    irc_session.set_option(EIRCOPT.STRIPNICKS);
}

// Create a basic event that listens to the server shut down event
local shutdown_event = BasicEvent(EVENTID.SERVERSHUTDOWN)
// Tell the event instance to call this function when it's being triggered
shutdown_event.on_trigger = function()
{
    /* The session is closed automatically so... */
}

// Tell the irc session instance to call this funtion when it successfully connects to the server
irc_session.on_connect = function(event, origin, params)
{
    // Confirm in console
    print("Connected to IRC...");
    // Join a channel (on success it returns 0)
    if (irc_session.cmd_join("#SLC") != 0)
    {
        print("Unable to join channel: " + irc_session.err_str);
    }
    // Say something on some channel or user
    else
    {
        // Say a message in #SLC channel
        irc_session.cmd_msg("#SLC", "Hi. I am SqMod");
        // Say a message to SLC user
        irc_session.cmd_msg("SLC", "Sup?");
        // Do a /me command in #SLC channel
        irc_session.cmd_me("#SLC", "goes home");
    }
}

// Tell the irc session instance to call this funtion when someone says something in the channel
irc_session.on_channel = function(event, origin, params)
{
    // event is the event type in IRC [CHANNEL]
    // origin is who said the message
    // params[0] is the channel where the message was said
    // params[1] is the text that was said

    // See if the used who said this message was SLC
    print( format("%s said on %s : %s", params[0], origin, params[1]) );

    // Because the EIRCOPT.STRIPNICKS was enabled
    // `origin` will contain the clean user name instead of [email protected]
}

// Using IRC to evaluate code on the server from private messages

// Tell the irc session instance to call this funtion when someone when someone sent you a message
irc_session.on_priv_msg = function(event, origin, params)
{
    // event is the event type in IRC [CHANNEL]
    // origin is who said the message
    // params[0] is the channel where the message was said
    // params[1] is the text that was said

    // See if the used who said this message was SLC
    if (origin == "SLC")
    {
        // Compile message as valid script code
        local code = compilestring(params[1]);
        // Execute it
        code();
    }

    // Because the EIRCOPT.STRIPNICKS was enabled
    // `origin` will contain the clean user name instead of [email protected]
}

How to get a list of users on channels:

Code Select Expand

// Table of channels and arrays of connected users
IRC_USERS <- { /* ... */ };

// Create a basic event that listens to the server frame event
local fake_timer = BasicEvent(EVENTID.SERVERFRAME);
// Tell the event instance to sleep for 3000 milliseconds (3 seconds)
fake_timer.idle = 3000;
// Tell the event instance to call this function when it's being triggered
fake_timer.on_trigger = function(delta)
{
    // Ask the server for a list of users on #SLC channel
    irc_session.cmd_list("#SLC");

    print("------ List of users retrieved every 3 seconds");
    // Print the list of users retrieved on the last call
    foreach (chanel, users in IRC_USERS)
    {
        print("Users in " + chanel);
        foreach (user in users)
        {
            print(user);
        }
    }

    if (IRC_USERS.rawin("#SLC") && IRC_USERS.rawget("#SLC").find("[member=135]Doom_Kill3R[/member]") != null)
    {
        print("Found the nab");
    }

    // Tell it to sleep for another 3000 milliseconds (3 seconds)
    fake_timer.idle = 3000;
}

// Tell the irc session instance to call this funtion when an IRC RFC code is sent
irc_session.on_numeric = function(event, origin, params)
{
    switch (event)
    {
        case 353: /* RFC 1459 RPL_NAMREPLY */
            // params[0] Our nick name
            // params[1] = ? (dunno what this is)
            // params[2] Channel
            // params[3] List of users separated by space

            IRC_USERS.rawset(params[2], split(params[3], " "));
        break;

        case 366: /* RFC 1459 RPL_ENDOFNAMES */
            // params[0] Our nick name
            // params[1] Channel
            // params[2] End of /NAMES list.
        break;

        default:
            /* Unknown or unhandled RFC code */
    }
}


[.]

I've been working on a simple builtin command system to avoid having to see people writing a god function containing all commands and to take out the duplicate code used to extract command arguments and validate types.

The command system is very simple and works similarly to the event system. You begin by creating a command listener and specify a command name and optionally some argument type filters.

Just to be clear by invoker I mean the player who called the command.

So let's begin simply by creating a command listener that listens to the "test" command:

Code Select Expand

local cmd_test = Cmd.Listener("test");
Now in order to actually do something when this command is used we bind a function to the `on_exec` slot:

Code Select Expand

cmd_test.on_exec = function(invoker, args)
{
    print("test cmd called by " + invoker);
}
The function will receive the ID of the player that invoked the command and an aray with the passed arguments. A command supports up to 12 arguments to be specified if necessary.

So let's print the type and value of each received argument by modifying that function a bit:

Code Select Expand

cmd_test.on_exec = function(invoker, args)
{
    print("test cmd called by " + invoker);

    foreach (i, v in args)
    {
        print("type <" + typeof(v) + "> arg " + i + " has value: " + v);
    }
}
Now let's assume that someone calls this command using the following:

Code Select Expand

/test some_text "text with spaces" 786 false 'some other text' 2.343
The out might be as follows:

Code Select Expand

test cmd called by 0
type <string> arg 0 has value: some_text
type <string> arg 1 has value: text with spaces
type <integer> arg 0 has value: 786
type <bool> arg 0 has value: false
type <string> arg 0 has value: some other text
type <float> arg 0 has value: 2.343

As you can see the internal parser splits arguments where a space is found. However you can include a text as a single argument by enclosing it in single or double quotes. Similar to what Squirrel does. If the value is not enclosed in quotes then it tries to convert it to an integer and if that fails then to a float and if that fails then to a boolean and finally falling back to a string.

As its the case for the `some_text` argument which is not enclosed in quotes but it's a string. Obviously that cannot convert to an integer, float or boolean so it's used as a string. Argument two on the other hand `text with spaces` instead of being considered three distinct arguments it'll be grouped into a single one because of the quoting.

Including quotes inside quotes works the same way as in squirrel. Simply by escaping them with the back slash character \' and \" respectively.

But that's not the thing that saves up most of the code. The thing that prevents you from writing a ton of bloat code is the integrated type checking. You can specify what kind of types of values you command may receive as arguments.

Let's begin by adjusting the creation to also include type checking:

Code Select Expand

local cmd_test = Cmd.Listener("test", "i,f|s|b");
Hopefully you notified the second constructor argument value "i,f|s|b". That's a string which specifies what type of value each argument expects. The pipe '|' character separates the arguments while the comma separates the allowed types which are:

• i for integer
• f for float numbers
• b for booleans
• s for strings

Basically the second string specified previously tells that the command may only receive integer or float in the first argument, string as the second argument and boolean as the third.

Therefore if we try to call the command with something like this:

Code Select Expand

/test turbo "goes here" bool
Will throw an error saying that argument 0 (yes, argument counting starts from 0) is incompatible. Since it expects an integer or float and we passed the string `turbo`.

If you wish to catch that message and send information to the player then you can set a custom error handler:

Code Select Expand

Cmd.set_on_error(function(type, msg) {
    switch (type)
    {
        case ECMDERR.UNKNOWN_COMMAND:
            // Get the player that invoked the command and send him a message
            Cmd.get_invoker().message("No such command exists");
        break;
        case ECMDERR.UNSUPPORTED_ARG:
            // Get the player that invoked the command and send him a message
            Cmd.get_invoker().message("Wrong command arguments");
        break;
        case ECMDERR.EXECUTION_FAILED:
            // Get the player that invoked the command and send him a message
            Cmd.get_invoker().message("Failed to execute the command");
        break;
    }
});
Allowing you to customize what you want to say and how do you want to react when something fails.

But what if I don't want anyone to use my command? Well, there's something for that as well. There's another slot besides `on_exec` caled `on_auth` which takes a function that must return true or false if the invoker is allowed to use the command. But before that the command must have explicit authority turned on. That's done through the `auth` member. Let's see how that looks:

Code Select Expand

cmd_test.auth = true;
cmd_test.level = 3;

cmd_test.on_auth = function(invoker)
{
    local player = CPlayer(invoker);
    if (player.level >= cmd_test.level)
    {
        return true;
    }
    return false;
}
So let's see what we did. We enabled explicit authority clearance by setting `auth` property to true. We used the built in property `level` to store a number and use that as a way of saying that our command needs a level higher or equal to three to be called.

The same `level` property is built into players as well. This is provided by the plugin and it's optional if you don't have more complex systems. Each player can have it's own level. The level is nothing but an integer. Preferably the integer should start from 0. And let's say that a basic user has a level of 1 a vip user has a level of 2 a moderator has a level of 3 and an admin has a level of 4.

Basically we said that our command needs to be called by someone who's at least a moderator to be allowed to call it. Finally we set out custom function which inspects the invoker level and compares it with our command to see if the execution should be allowed.

Of course you don't have to use the built in authority system and you may implement your own with more fine grained privileges. For example, with the built in level system we don't even need to set a function to the `on_auth` slot.

We just have to enable explicit authority by setting `auth` to true and set the minimum level required to executed the command. if the level is not less than 0 (negative) then it will be used to compare it to the invoker's level.

There are tons of other customization. Setting a minimum and maximum allowed arguments required to call your command. Suspend command listener to prevent further calls. Set up a help or informational text or generate them automatically based on the argument types specified and other things.

Help describes what the command is supposed to do and what it needs. While information describes how to be used. Basically that means the syntax that is allowed.

Dynamically update it's name and specifiers. And a ton of other things to come. This was meant to prevent you from having to write a ton of code for checking types and dealing with permissions. It's still a work in progress but it's something and worked just fine in the basic tests.

[.]

UPDATES (major only):

• Dropped the current design and every relation to C++11. Apparently Squirrel and Sqrat (the binding utility) don't behave well in C++11. So now the project is back on C++03
• Implemented the INI library. (examples in following posts)
• Implemented the XML library. (examples in following posts)
• Various other libraries partially implemented. Such as Time, Random, Numeric etc.
• Dropped the previous event system due to its complexity in favor of a more simplistic one allowing and recommending OOP designs.

Examples of new mentioned features will be available in the following posts.

[.]

Implemented the INI library featuring Unicode support (which is irrelevant since the plugin uses ASCII), As well as MultiKey and MultiLine support. As implied, the MultiKey feature allows INI files to have multiple keys with the same name under a section. While MultiLine allows values to span across multiple lines.

For the sake of this example, I'll use a string as the source instead of a file. Also, the things demonstrated here only show a portion of what can be achieved with the current implementation.

Assuming we have the following INI:

Code Select Expand

local doc = @"
[Database]
Host=127.0.0.1
Port=5443

[Options]
Test=23
Test=false
Test=Text goes here

[Data]
Text= <<<MULTI_LINE
Multi line
    content
allowed
MULTI_LINE
";
We begin by creating an INI document. And load the string into the instance. If any errors occur, a proper exception is thrown.

Code Select Expand

local ini = SqIni.Document(false, true, true);

ini.LoadData(doc);

The three arguments passed when creating the document disable UTF8 and enable MultiKey and MultiLine.

So now that we parsed the string and we have data in our document. Let's try reading individual values:

Code Select Expand

print(ini.GetValue("Database", "Host", "*"));
print(ini.GetInteger("Database", "Port", 3432));
print(ini.GetValue("Database", "Name", "default value"));
The first two arguments describe the section and the key from which the value should be retrieved. The third one is a value that should be returned if that key doesn't exist. And if everything went well, we should get the following output:

Code Select Expand

[USR] 127.0.0.1
[USR] 5443
[USR] default value
How about getting all sections and printing them? Let's try:

Code Select Expand

local inisec = ini.GetSections();

printf("Sections: %d", inisec.Size);

while (inisec.Valid)
{
    print(inisec.Item);
    inisec.Next();
}

First we obtained a list of sections and then we tested to see if the returned iterator points to a valid section. If it does, then we print it's name and move to the next one. Then we test again if the iterator points to a valid section and print it again. And we do so until we reach the end.

But how about those multi keys under the [Options] section? Well, we apply almost the same method that we did with sections:

Code Select Expand

local inival = ini.GetValues("Options", "Test");

printf("Keys: %d", inival.Size);

while (inival.Valid)
{
    print(inival.Item);
    inival.Next();
}

First we retrieve a list of values from all the keys named "Test" under the "Options" section. And we iterate over them like we did with sections. Optionally, we can sort them into the order they were defined in the INI file if that's a requirement inival.SortByLoadOrder();

And sure enough, we should have the following output:

Code Select Expand

[USR] Keys: 3
[USR] 23
[USR] false
[USR] Text goes here
So now lets try the multi line text. Which we can access just like an individual item:

Code Select Expand

print(ini.GetValue("Data", "Text", ""));
Output should be:

Code Select Expand

[USR] Multi line
    content
allowed
This is only a small example. You can modify documents, save them and much more.

[.]

This is a demonstration of the built in time library that allows easy manipulation of time with microsecond precision. This is a small demonstration and more features remain to be implemented.

NOTE: SLongInt is part of the Numeric library and allows the user to work with 64 bit integers on 32 bit builds.

Code Select Expand

print( "[0]  " + time() ); // Only option in squirrel
// Options in this plugin
print( "[1]  " + SqTime.Now().SecondsI );

print( "[2]  " + SqTime.Now().Milliseconds );
print( "[3]  " + SqTime.Now().Milliseconds / SLongInt(1000) );
print( "[4]  " + SqTime.Now().Microseconds );
print( "[5]  " + SqTime.Now().MinutesI );
print( "[6]  " + SqTime.Now().HoursI );
print( "[7]  " + SqTime.Now().DaysI );
print( "[8]  " + SqTime.Now().YearsI );

print( "[9]  " + ( SqTime.Now() + SqTime.Seconds(10) ).SecondsI );

print( "[10]  " + SqTime.Days(1.5).HoursI );
print( "[11]  " + SqTime.Years(1).DaysI );
print( "[12]  " + SqTime.Minutes(5.2).SecondsI );
print( "[13]  " + SqTime.Minutes(45).HoursF );
print( "[14]  " + SqTime.Milliseconds(10000).SecondsI );

print( "[15]  " + ( SqTime.Minutes(2) - SqTime.Seconds(30) ).SecondsI );

NOTE: The suffixed I and F at the end stand for Integer and Float. Since sometimes you need to preserve precision with fractional numbers.

Output:

Code Select Expand

[USR] [0]  1456131000
[USR] [1]  1456131000
[USR] [2]  1456131000991
[USR] [3]  1456131000
[USR] [4]  1456131000991210
[USR] [5]  24268850
[USR] [6]  404480
[USR] [7]  16853
[USR] [8]  46
[USR] [9]  1456131010
[USR] [10]  36
[USR] [11]  365
[USR] [12]  311
[USR] [13]  0.75
[USR] [14]  10
[USR] [15]  90

Code Select Expand

local tm = SqTime.Timer();

// Performing some arbitrary computations
for (local i = 0; i < 10000000; i++)
{
    i += 1;
    i -= 1;
}

print("Microseconds elapsed: " + tm.Elapsed.Microseconds);
Output:

Code Select Expand

[USR] Microseconds elapsed: 525408

[.]

This in response to a popular debate here on the forums on who's faster INI or SQLite. The series of these benchmarks are meant to show that both are decent candidates for storing data if implemented well enough.

These benchmarks should not be taken too seriously as they rely on the fact that the user needs to process large amounts of data in bulk. Which in real code it's rarely the case.

[Benchmark] SQLite vs INI

Unfortunately that benchmark is a bit outdated and tbh it is unfair. None of the plugins/modules make use of all the features that SQLite provides or does any kind of optimizations as is the case of the INI plugin/module. Why? Because the entire INI file is parsed every time you need to read/write something and the SQLite benchmark does not make use of transactions and/or pre-compiled statements. And the simulation is not fair or process the same amount of data.

I am going to run a series of benchmarks and describe further methods of optimizations which I've implemented in my plugin during it's development. Hopefully by the end of this benchmark you will get and idea how further you can go with optimizations if the storage is the bottleneck.

BENCHMARK CODE

I'll be using the following code to time my benchmarks and generate a nice and structured output:

Code Select Expand

local _Timer, _Count;

function StartBenchmark(name)
{
    print(SqStr.Center(name, '-', 70));
    print(""); // Just some spacing
    _Timer = SqTime.Timer();
    _Count = 0;
}

function StepBenchmark(step)
{
    local elapsed = _Timer.Restart().Microseconds.Num;
    SqLog.Inf("%s:%s microseconds",
                SqStr.Left(step+"", ' ', 16)
                SqStr.Right(elapsed+"", ' ', 12));
    _Count += elapsed;
}

function StopBenchmark(name)
{
    SqLog.Inf("%s:%s microseconds",
                SqStr.Left("Total", ' ', 16)
                SqStr.Right(_Count+"", ' ',  12));
    print(""); // Just some spacing
    print(SqStr.Center(name, '-', 70));
}


[.]

INI: WRITE

For this benchmark I will use a single INI file and write 10,000 dummy accounts:

Code Select Expand

StartBenchmark(" INI WRITE ");

local ini = SqIni.Document(false, false, false);

StepBenchmark("Creation");

for (local i = 0; i <= 10000; ++i)
{
    local section = "Account_" + i;
    ini.SetInteger(section, "ID", i);
    ini.SetValue(section, "Name", "user_" + i);
    ini.SetValue(section, "Password", "password_" + i);
    ini.SetBoolean(section, "Active", i % 2 == 0);
}

StepBenchmark("Insertion");

ini.SaveFile("Accounts.ini");

StepBenchmark("Saving");

StopBenchmark(" INI WRITE ");

Benchmark results:

Code Select Expand

[USR] ------------------------------ INI WRITE -----------------------------
[USR]
[INF] Creation        :          32 microseconds
[INF] Insertion       :      308865 microseconds
[INF] Saving          :       93330 microseconds
[INF] Total           :      402227 microseconds
[USR]
[USR] ------------------------------ INI WRITE -----------------------------
Contents of the new INI file:

Code Select Expand

[Account_0]
ID = 0
Name = user_0
Password = password_0
Active = true


[Account_1]
ID = 1
Name = user_1
Password = password_1
Active = false

...

[Account_10000]
ID = 10000
Name = user_10000
Password = password_10000
Active = true

Optimizations behind the scene:

• When inserting elements, they are created in memory. The only time disk IO is necessary is when writing everything at once.

Bottlenecks behind the scene:

• The internal containers have to scale on each insert and therefore some allocations have to occur. If this was a pressing matter I could've used some methods of pre-allocating space upfront. But unfortunately it's not.
• I'm suspecting that each element is written to the file individually rather than writing it all in a memory buffer and then to disk. I'll test this in the further optimization stage.

INI: READ

For this bench mark I'll be reading and processing the file I've previously created and also do some minor validation on the data:

Code Select Expand

StartBenchmark(" INI READ ");

local ini = SqIni.Document(false, false, false);

StepBenchmark("Creation");

ini.LoadFile("Accounts.ini");

StepBenchmark("Loading");

local acc_id, acc_name, acc_pass, acc_active;

for (local i = 0; i <= 10000; ++i)
{
    local section = "Account_" + i;

    local acc_id = ini.GetInteger(section, "ID", -1);

    if (acc_id != i)
    {
        throw "account has a bad ID";
    }

    acc_name = ini.GetValue(section, "Name", "");
    acc_pass = ini.GetValue(section, "Password", "");
    acc_active = ini.GetBoolean(section, "Active", false);
}

StepBenchmark("Processing");

StopBenchmark(" INI READ ");

Benchmark results:

Code Select Expand

[USR] ------------------------------ INI READ ------------------------------
[USR]
[INF] Creation        :          32 microseconds
[INF] Loading         :       82465 microseconds
[INF] Processing      :       90007 microseconds
[INF] Total           :      172504 microseconds
[USR]
[USR] ------------------------------ INI READ ------------------------------
Optimizations behind the scene:

• The entire file is read into memory at once and disk IO is no longer necessary after.

Bottlenecks behind the scene:

• The internal containers have to iterated whenever a value is retrieved. I'm planning to further optimize this later by allowing to fetch every key/value pair in a section at once.