Does arrays in squirrel have a maximum size?

[Mötley]

If so what would the value be?

I looked on the web and could not find anything. Unless I missed something

Not being serious with this but

Code Select Expand

Index <- array( infinite, null )
Index <- array( MAX_ARRAY_SIZE, null )
Should give you the idea

[Anik]

I also had the same question. And asked @Xmair about it. He said there is no limit.

[Thijn]

I don't think they have a fixed maximum size, but please remember that everything you put in there is store in RAM so your usage can be high if you store a lot of stuff in arrays.

[.]

The arrays in Squirrel are nothing but a wrapper around a vector of script objects.

Code Select Expand

struct SQArray : public CHAINABLE_OBJ
{
    ...
    SQObjectPtrVec _values;
};
And looking at the _realloc function within the vector class:

Code Select Expand

template<typename T> class sqvector
{
    ...
    void _realloc(SQUnsignedInteger newsize)
    {
        newsize = (newsize > 0)?newsize:4;
        _vals = (T*)SQ_REALLOC(_vals, _allocated * sizeof(T), newsize * sizeof(T));
        _allocated = newsize;
    }
};
I can see it's using a macro:

Code Select Expand

#define SQ_REALLOC(__ptr,__oldsize,__size) sq_vm_realloc((__ptr),(__oldsize),(__size));
Which is just a wrapper around a function:

Code Select Expand

void *sq_vm_realloc(void *p,SQUnsignedInteger oldsize,SQUnsignedInteger size);
Which is also a wrapper around the standard C `realloc` function:

Code Select Expand

void *sq_vm_realloc(void *p, SQUnsignedInteger SQ_UNUSED_ARG(oldsize), SQUnsignedInteger size){ return realloc(p, size); }

Which means that Squirrel itself does not impose any specific limits and it all falls back to the operating system and architecture.

So now let's talk architecture. On an architectural level, the maximum amount of memory you can require is limited by the maximum number you can represent with an unsigned integer of equivalent size.

However, Squirrel uses signed integers. To find out the limits of an integer value in Squirrel, we can use some bit-wise nastiness like:

Code Select Expand

~(1<<(_intsize_*8)-1)
Which gives us 2147483647 on 32 bit architecture and 9223372036854775807 on 64 bit architecture.

From there we look at the structure that squirrel uses to represent an object:

Code Select Expand

typedef struct tagSQObject
{
    SQObjectType _type;
    SQObjectValue _unVal;
}SQObject;
Where the `_type` is likely to be an integer and `_unVal` is a pointer to the actual thing. Then including a pointer that an array will use to reference such object. Plus the reference counter used internally and whatever management squirrel does to perform garbage collection. Excluding the fact that the object could be a string/table/class etc. which in turn may have additional requirements.

We're going to assume a minimum of 32 bytes on 32 bit and 64 bytes on 64 bits for each Squirrel object. We're just going to assume, meaning we're not that precise.

From then on, you can do some simple math like `(2147483647ull * 32)` which gives you roughly 68719476704 bytes and `(9223372036854775807ull * 64)`which gives you roughly HOLY F* THIS NUMBER IS TOO HUGE FOR YOUR COMPUTER. So we're not even going to bother about the 64 bit limit until the day we have computers with terabytes of RAM.

So now, let's talk OS limitations. As you can see on 32 bits you exceed the maximum operating system memory by a lot. Which means, your operating system is likely to run out of memory before you reach the limit of a signed integer.

And that is assuming the OS even gives that much memory to a single application. Forgot the exact numbers but on windows, a single application can't really get more than 1-2 GB of memory. Somewhere around that number. Even on 64 bits you may run into some of these limitations. Assuming your hardware even has that much memory.

So why don't we find out how many objects can we get with about 1gb of memory? Well, one gigabyte has about 1073741824 bytes. So if we do `(1073741824 / 32)`we're going to get around 33,554,432. So there you go, you can assume your limit is around that number on 32 bit. Of course, also assuming your application only needs a single array of such size.

I am not in a mood to allocate an array with that much memory to force to OS to move my stuff to swap memory and make it slow as f*. But you are free to poke around and allocate an array of this magnitude incrementally to see when your OS is going to say NO and let you 'die'.

Although I would suggest specifying a fill value when you create the array, otherwise you're just going to fill it with null values which may not represent the correct limit.

Have phun.

[Mötley]

Can squirrels Architecture with vcmp handle increasing the array value while the server is running? 

If so this would be preferred.. The value should not be but so high, but what if, I would prefer to start from 0 - 1 and work my way up..

[.]

If so this would be preferred.. The value should not be but so high, but what if, I would prefer to start from 0 - 1 and work my way up..

I don't get that?

[EK.IceFlake]

If so this would be preferred.. The value should not be but so high, but what if, I would prefer to start from 0 - 1 and work my way up..

I don't get that?

Code Select Expand

local ar1;
local idx = 1;
while (true)
{
    ar1 = array(idx, "SomeValue");
    print(idx);
    ++idx;
}

[Mötley]

Basically in the format that EK.CrystalBlue used.

For what I am doing if there is nothing but 77 indexs then there is no need to have a massive array.. Until the index grows in size overtime,

Due to player count I doubt there would ever be a size issue what so ever causing hardware issues

[.]

Last time I'll reply to topics like these. Topics where you people ask questions where you are not capable of understanding the answer. Also, wtf is that snippet supposed to demonstrate? If anyone finds himself in a situation where a piece of code replicates that behavior from the snippet above. Then please, PLEASE. For the sake of humanity, press the poweroff button and walk away from the computer.

How the f* did this simple question of 'is there a limit on array size' turned into a question of 'I don't want my arrays to be that big' is beyond me. Can so eone please explain to me how did we get here? Or it's just me, finally, loosing my sanity.

[Mötley]

How the f* did this simple question of 'is there a limit on array size' turned into a question of 'I don't want my arrays to be that big' is beyond me. Can so eone please explain to me how did we get here? Or it's just me, finally, loosing my sanity.

You went in full depth about what happens when you have higher array limits to your os ram etc.. You kind of steered this topic into hardware etc, I attempted to go into a route that should prevent that.,


^ Then you went crazy and off topic,,,

  You okay ,
  You good

I just figured if we were going to go into safety measure why not go for per cations...

[KAKAN]

^ Then you went crazy and off topic,,,

No, he didn't.
He simply meant this all depends on your architecture and the amount of RAM available in your system to save that much data.

[Thijn]

If the question you're asking, can arrays grow dynamicly. The answer is yes.
If you start with an array of, say, 10 elements. You can push to that array as much as you want.

Code Select Expand

local smallArray = array(10,null);
for( local i = 0; i < 20; i++)
{
smallArray.push('somevalue');
}
print( smallArray.len() );
Returns 30.

[Mötley]

Thank you @Thijn, Thats the answer I was looking for


And my apologies. My eye is not the greatest anymore, Over a course of time of staring at code on the monitor text starts to not look right out of my left eye causing confusion, I honestly had a hard time reading that message from S.L.C..

[Mötley]

I will combine this post latter,

So I would like to think I have done something wrong.

To make a long story short the account storage never gets called for a update,

I modified this so many times but I am seriously so mixed up between learning to add values to the array and looping threw to an array index the currently does not exist...

I can add the accounts to the array/class on script load just fine, But either there's an aspect of the array I am missing out on or that entire loop if fu%$ed

Code Select Expand

// I started at 0
SQL_Accounts <- array( 0 )

function Register_Player( Player, password )
{
local iCounter = 0, Number_of_Accounts = SQL_Accounts.len();

        //Set the account class first!
        Account[ Player.ID ].Name = Player.Name;
        Account[ Player.ID ].Password = ::SHA1( password ); // Encrypt the password
        Account[ Player.ID ].IP = Player.IP; // Store the players ip
        Account[ Player.ID ].Autologin = 0; // Set the autologin to 0
        Account[ Player.ID ].Active = true; // Set them to logged in

        while ( iCounter < Number_of_Accounts )
        {
                if ( !SQL_Accounts[iCounter] )
                {
                        SQL_Accounts.push(iCounter);
                        SQL_Accounts[ iCounter ] = SQL_Accounts();
                        SQL_Accounts[ iCounter ].Name = Player.Name;
                        SQL_Accounts[ iCounter ].Password = :SHA1( password );
                        SQL_Accounts[ iCounter ].IP = Player.IP;
                        SQL_Accounts[ iCounter ].Autologin = 0; // Default Value
                        print( "Number_of_Accounts " + Number_of_Accounts + "Counter "+ iCounter );
                        //break;
                }
                iCounter++
        }

/* Add the player to the database */
/*Account[ Player.ID ].register( Player );*/
}
Any help on learning this would be nice

Also when the server loads

Sorry I use pointers to convert scripts from VCMP to the other MP

Code Select Expand

// Load the accounts into the class
function Authorize::Accounts( )
{
local iCounter = 0; // Add the counter
    local query = ::sqlite_query( g_AccountDB, "SELECT * FROM Accounts" ) // Grab the query
    local string = ::sqlite_column_data(query, 0); // Stores the first name.

while (string)
{
SQL_Accounts.push(iCounter); // Increase the array Value
SQL_Accounts[ iCounter ] = SQL_Account_Class();
SQL_Accounts[ iCounter ].Name = ::sqlite_column_data(query, 0);
SQL_Accounts[ iCounter ].Password = ::sqlite_column_data(query, 1);
SQL_Accounts[ iCounter ].IP = ::sqlite_column_data(query, 2);
SQL_Accounts[ iCounter ].Autologin = ::sqlite_column_data(query, 3);

print("" + SQL_Accounts[ iCounter ].Name)
print("" + SQL_Accounts[ iCounter ].Password)
print("" + SQL_Accounts[ iCounter ].IP)
print("" + SQL_Accounts[ iCounter ].Autologin)

iCounter ++;
string = sqlite_next_row(query);
}
sqlite_free( query );
}

FML I Messed up selecting the class, If you feel I could of done better PLEASE let me know as this is a learning experience
Everything is working fine now.

Code Select Expand

function Register_Player( Player, password )
{
    local iCounter = 0, Number_of_Accounts = SQL_Accounts.len();

    //Set the account class first!
    Account[ Player.ID ].Name = Player.Name;
    Account[ Player.ID ].Password = ::SHA1( password ); // Encrypt the password (THIS IS VERY IMPORTANT)
    Account[ Player.ID ].IP = Player.IP; // Store the players ip
    Account[ Player.ID ].Autologin = 0; // Set the autologin to 0
    Account[ Player.ID ].Active = true; // Set them to logged in
   
    if ( iCounter < Number_of_Accounts  )
    {
            iCounter += Number_of_Accounts;
            SQL_Accounts.push(iCounter);
            SQL_Accounts[ iCounter ] = SQL_Account_Class();
            SQL_Accounts[ iCounter ].Name = Account[ Player.ID ].Name ;
            SQL_Accounts[ iCounter ].Password = Account[ Player.ID ].Password ;
            SQL_Accounts[ iCounter ].IP = Account[ Player.ID ].IP;
            SQL_Accounts[ iCounter ].Autologin = Account[ Player.ID ].Autologin;
            print( "Number_of_Accounts: " + Number_of_Accounts + "| Counter "+ iCounter );
    }

    /* Add the player to the database */
    Account[ Player.ID ].register_player( Player );
}