Vice City: Multiplayer

VC:MP Discussion => General Discussion => Topic started by: NewK on Aug 06, 2018, 03:37 AM

Title: Javascript plugin
Post by: NewK on Aug 06, 2018, 03:37 AM
Would anyone be interested in a javascript plugin?  I've got a basic vcmp javascript plugin working at the moment, still needs some testing but it's working. It runs on V8 and uses a nodejs-style event loop that allows you to do IO operations asynchronously without blocking the main thread. That's one of the advantages over squirrel. Another one is that it's also alot faster than squirrel and this is very noticeable for CPU intensive tasks. Some of my benchmarks showed it to be about 30x faster than squirrel.

All that's left to do right now is create some documentation, although I'd rather not go through the painful process of creating documentation and wasting days setting that up if there's no people interested in the plugin. So with this topic I'd just like to know first if there's anyone that would be interested in this plugin or not, I don't want to invest my time maintaining and documenting it if there's no interest from people ;D

Title: Re: Javascript plugin
Post by: D4rkR420R on Aug 06, 2018, 04:50 AM
I might be just unfamiliar with the language, however can you provide me examples of what Javascript can perform in terms of server development?
Title: Re: Javascript plugin
Post by: MatheuS on Aug 06, 2018, 06:31 AM
Sounds great.
Title: Re: Javascript plugin
Post by: MEGAMIND on Aug 06, 2018, 07:39 AM
ahm interested
Quote from: Takanaue on Aug 06, 2018, 06:31 AMSounds great.
Title: Re: Javascript plugin
Post by: (SpCy)Alex on Aug 06, 2018, 08:05 AM
yes xD
Title: Re: Javascript plugin
Post by: vitogta on Aug 06, 2018, 10:42 AM
I'd like to use javascript in vc-mp since this language is part of my job, especially if it comes with multi thread but I'm using rel004 and I never going to move to rel006. So if this plugin will work with 004 I will use that.
Title: Re: Javascript plugin
Post by: Thijn on Aug 06, 2018, 07:23 PM
Quote from: vitogta on Aug 06, 2018, 10:42 AMSo if this plugin will work with 004 I will use that.
Obviously not
Title: Re: Javascript plugin
Post by: vitogta on Aug 06, 2018, 07:26 PM
Quote from: Thijn on Aug 06, 2018, 07:23 PM
Quote from: vitogta on Aug 06, 2018, 10:42 AMSo if this plugin will work with 004 I will use that.
Obviously not
Then not intresting.
Title: Re: Javascript plugin
Post by: NewK on Aug 06, 2018, 10:54 PM
Quote from: KuRuMi^ on Aug 06, 2018, 04:50 AMI might be just unfamiliar with the language, however can you provide me examples of what Javascript can perform in terms of server development?
Well as far as VCMP features go, it has the same VCMP features that the squirrel plugin has. The same events, and the same server functions. The main advantages here in comparison to the squirrel plugin is that your code will run alot faster due to the JIT, and the IO operations can be done asynchronously. For example if you had to execute a slow DB operation that would take 5 seconds to complete, your whole server would be "paused" for the duration of this operation. That means for the players inside your server they would see everyone warping/lagging until the operation was finished. Using this plugin, that would not happen if you executed this operation asynchronously. The plugin would execute this in a background thread instead of using the main server thread and would return a callback with the results. For example if I wanted to query a mysql "users" table and print the nickname of each user to the console, it would look something like this:

    //open mysql connection
    var conn = sql.connect("dbName", "localhost", 3306, "dbUser", "dbPassword");

    //queries users table
    conn.query("select * from  users", function (accounts) {
        accounts.forEach(account => {
            console.log(account.nick);
        });
    });

If you wanted to do it in a synchronous manner you could do that too you'd just need to remove the callback and it would behave synchronously:
var accounts = conn.query("select * from  users");

Quote from: vitogta on Aug 06, 2018, 07:26 PM
Quote from: Thijn on Aug 06, 2018, 07:23 PM
Quote from: vitogta on Aug 06, 2018, 10:42 AMSo if this plugin will work with 004 I will use that.
Obviously not
Then not intresting.
Actually, the plugin will work on rel004 if you use a rel004 server executable.  You just wont be able to use any of the new features/bugfixes from rel005 and rel006 versions obviously.

This plugin relies on the vcmp java plugin to expose the vcmp "environment" to the javacript context. VCMP server functions and events are exposed from the java plugin and can be used seemlessly from javascript. Java will also be used to implement  IO, multithreading and any other external library that's needed. It will then provide access to all of that through simple javascript APIs.
Title: Re: Javascript plugin
Post by: ysc3839 on Aug 07, 2018, 09:18 AM
I wonder how you deal with asynchronous event loop?
As for Python, it requires a blocking event loop to run async code.So I have to run it on another thread.
Another problem is if it's safe to call server function on another thread?
Title: Re: Javascript plugin
Post by: . on Aug 07, 2018, 11:40 AM
Quote from: NewK on Aug 06, 2018, 10:54 PM...
This plugin relies on the vcmp java plugin to expose the vcmp "environment" to the javacript context. VCMP server functions and events are exposed from the java plugin and can be used seemlessly from javascript. Java will also be used to implement  IO, multithreading and any other external library that's needed. It will then provide access to all of that through simple javascript APIs.

I kinda knew that was the case. Doing this via c/c++ for v8 or some other engine would be a little overkill.
Title: Re: Javascript plugin
Post by: EK.IceFlake on Aug 07, 2018, 01:13 PM
It'd be nice, but one of the main advantages of JavaScript is the number of libraries there are out there. Most of the libraries that deal with IO stuff won't work because they rely on Node.JS's API.
Title: Re: Javascript plugin
Post by: NewK on Aug 07, 2018, 01:49 PM
Quote from: ysc3839 on Aug 07, 2018, 09:18 AMI wonder how you deal with asynchronous event loop?
I implemented an event loop similar to the way libuv did it for node.js. The event loop is single threaded and runs on the main thread (server thread) just like all the javascript code. Event loop iterations are processed onServerFrame. The event loop has a thread-safe linked queue that simply receives the results of the asynchronous operations that are run on the other threads. On each event loop iteration, if the queue is not empty, the results are dispatched to the javascript context through callbacks.

Quote from: ysc3839 on Aug 07, 2018, 09:18 AMAnother problem is if it's safe to call server function on another thread?
Nah, it's not safe to call server functions from another thread. Infact, on the java plugin if you try to do this, it will not let you and will throw an exception. But the java plugin provides a proper way to do it through server.sync(). That will pause the main thread for a few moments to ensure thread safety and will let you use the server API from other threads. But that's something to use with the actual java plugin, it won't be used on this javascript plugin

Quote from: . on Aug 07, 2018, 11:40 AMI kinda knew that was the case. Doing this via c/c++ for v8 or some other engine would be a little overkill.
Yea, the idea here was to rely on the already existing java plugin to facilitate maintainability. Easier to maintain this way. The next time a new VCMP update is released I just need to update the java plugin and this plugin will automatically be updated aswell. No need to mantain two separate codebases this way :P

Quote from: EK.IceFlake on Aug 07, 2018, 01:13 PMIt'd be nice, but one of the main advantages of JavaScript is the number of libraries there are out there. Most of the libraries that deal with IO stuff won't work because they rely on Node.JS's API.
No, that's not one of the main advantages of javascript, that's one of the main advantages of node.js. Do not mix up the 2, those are 2 very different things. Javascript is not node.js and node.js is not javascript. Node.js is simply a set of C++ APIs that are made available to a javascript context running on top of V8. I never claimed this to be a node.js plugin, I said this is a javascript plugin, as in, actual ecmascript.  Think of it this way: Nodejs is a set of C++ APIs exposed to a javascript context running on V8. This plugin is a set of Java APIs exposed to a javascript context running on V8. Java was used to rely on the already existing VCMP Java plugin to expose VCMP functionality and to share the same codebase so it's easier to maintain in the future.
Title: Re: Javascript plugin
Post by: vitogta on Aug 07, 2018, 02:20 PM
Can you write a simple example, I want to get player's position and some other player's data (from instance), then I want to make some heavy calculation with this data in another theread (without blocking main theread) then (when calculations are done) I want to call a function with the result of those calculations as an argument in main theread (to interact player's instance by this function).

ps also is it possible to interact with another plugins like database or hashing in another (not-main) theread?
Title: Re: Javascript plugin
Post by: NewK on Aug 07, 2018, 11:23 PM
Quote from: vitogta on Aug 07, 2018, 02:20 PMCan you write a simple example, I want to get player's position and some other player's data (from instance), then I want to make some heavy calculation with this data in another theread (without blocking main theread) then (when calculations are done) I want to call a function with the result of those calculations as an argument in main theread (to interact player's instance by this function).
At the moment it's not possible to manually spawn threads yourself, javascript is not a very good language to do this type of thing because it would involve spawning a new V8 Runtime instance per thread, starting a new V8 instance is a bit slow and memory intensive. This was never the objective of the plugin, for such control over threads, the java plugin should be used instead. This plugin spawns the threads itself on the background, executes the operation, and simply calls a javascript function when that operation is finished with the results, but it never spawns any additional V8 instances, it simply returns to the already created V8 instance. Maybe in the future I'll look into this again but there's no "good" solutions. Node.js has this implemented, but even so, the node.js team still recommends not using it due the same issues I've described above.

For what you want to do, I'd just send an http request to an http server running on localhost, do the heavy calculation there, and return the result to the vcmp server. I already have an httpclient made that allows you to execute requests asynchronously without blocking the main thread, it would look something like this:

function onPlayerEnterVehicle(player, vehicle, slot) {
    var data = {
        X: player.pos.X,
        Y: player.pos.Y,
        Z: player.pos.Z
    };

    http.postJson("https://httpbin.org/post", data,
        function (response) {
            var resp = response.body().string();

            //if the response is a json string you can convert it to a JSON object like this
            var calculationResultFromServer = JSON.parse(resp);

            // call your function
            functionToCall(calculationResultFromServer);
        })
}
That's just an example on the PlayerEnterVehicle event but you could use it anywhere you want to. I sent the http request to httpbin.org just to test it but you'd just replace that URL with your http server endpoint.

Quoteps also is it possible to interact with another plugins like database or hashing in another (not-main) theread?
Well as I said above, you can't manually spawn threads but you can use multiple different async operations inside each other. For example, combining the examples above:

  //open mysql connection
    var conn = sql.connect("dbName", "localhost", 3306, "dbUser", "dbPassword");
    var data = {
        X: player.pos.X,
        Y: player.pos.Y,
        Z: player.pos.Z
    };

    http.postJson("https://httpbin.org/post", data,
        function (response) {
            var resp = response.body().string();

            //if the response is a json string you can convert it to a JSON object like this
            var obj = JSON.parse(resp);
             
            //create a prepared statement to avoid any sql injection attacks
           var stm = conn.instance.prepareStatement("select * from  users where level=? and admin=? ");
           stm.setInteger(1, obj.level);
           stm.setBoolean(2, obj.isAdmin);

            conn.query(stm, function (accounts) {
             //print results to console
               accounts.forEach(account => {
                   console.log(account.nick);
               });
            });
     })
Title: Re: Javascript plugin
Post by: Klaus on Aug 07, 2018, 11:56 PM
Nice one NewK, keep this going.
Title: Re: Javascript plugin
Post by: EK.IceFlake on Aug 08, 2018, 04:59 PM
Quote from: NewK on Aug 07, 2018, 01:49 PM
Quote from: EK.IceFlake on Aug 07, 2018, 01:13 PMIt'd be nice, but one of the main advantages of JavaScript is the number of libraries there are out there. Most of the libraries that deal with IO stuff won't work because they rely on Node.JS's API.
No, that's not one of the main advantages of javascript, that's one of the main advantages of node.js. Do not mix up the 2, those are 2 very different things. Javascript is not node.js and node.js is not javascript. Node.js is simply a set of C++ APIs that are made available to a javascript context running on top of V8. I never claimed this to be a node.js plugin, I said this is a javascript plugin, as in, actual ecmascript.  Think of it this way: Nodejs is a set of C++ APIs exposed to a javascript context running on V8. This plugin is a set of Java APIs exposed to a javascript context running on V8. Java was used to rely on the already existing VCMP Java plugin to expose VCMP functionality and to share the same codebase so it's easier to maintain in the future.
I'm aware of that, yeah. But the majority of JavaScript libraries that support networking or disk IO are made for node.js. So you won't really get much benefit in using this. ECMAScript by itself is a terrible language. There was this infamous picture which I can't find right now that shows what happens when you're using some operands with different types.

But yeah, overall, it'll still be better than what we currently have.
Title: Re: Javascript plugin
Post by: NewK on Aug 08, 2018, 07:39 PM
Quote from: EK.IceFlake on Aug 08, 2018, 04:59 PMI'm aware of that, yeah. But the majority of JavaScript libraries that support networking or disk IO are made for node.js.  So you won't really get much benefit in using this.
Those are node.js libraries. As I said, this is not a nodejs plugin, so obviously, it's not going to support node.js APIs, that was never my intention with this plugin. It's like comparing LU and VCMP Squirrel IO libraries. Just because both use the same language (squirrel) that doesn't mean all IO operations will look the same. Squirrel's just the "frontend" language, lower level APIs are implemention specific. This is the same case with Javascript, there's many different implementations, but still I wouldn't say there's not "much benefit" seeing as this brings async/non-blocking operations which squirrel does not have and also brings like a 30x speed up improvement. I'd say that's quite some benefits in comparison with squirrel.

Quote from: EK.IceFlake on Aug 08, 2018, 04:59 PMECMAScript by itself is a terrible language. There was this infamous picture which I can't find right now that shows what happens when you're using some operands with different types.
Yea I'm aware of the many javascript quirks and faults. There's some very ugly parts no doubt, but it's getting better and with V8 it becomes a JIT'ed language so yea like you said, overall still better.

Title: Re: Javascript plugin
Post by: ysc3839 on Aug 08, 2018, 08:57 PM
That's not what I want. I made Python plugin because Python have much libraries. We can use them instead of write another one.
Title: Re: Javascript plugin
Post by: NewK on Aug 09, 2018, 01:32 PM
Quote from: ysc3839 on Aug 08, 2018, 08:57 PMThat's not what I want. I made Python plugin because Python have much libraries. We can use them instead of write another one.
Yea python's nice, has a big community and is a very productive language but it's just as slow as squirrel. In fact, I found python to be even slower than squirrel. A while back I made a few benchmarks using a recursive function to calculate the first 35 fibonnaci numbers and python was the slowest, these were the results I got:

java - 69ms
javascript - 166ms
squirrel (using SLC's plugin) -  4596ms
python - 4978ms

Title: Re: Javascript plugin
Post by: vitogta on Aug 09, 2018, 06:00 PM
If plugin is done I suggest to write a blank script with few examples inside, then we all could to test it and decide to move on it or no.
The example blank script should include main kill-features of plugin.
Title: Re: Javascript plugin
Post by: NewK on Aug 09, 2018, 06:33 PM
Quote from: vitogta on Aug 09, 2018, 06:00 PMIf plugin is done I suggest to write a blank script with few examples inside, then we all could to test it and decide to move on it or no.
The example blank script should include main kill-features of plugin.
Yea I intend to do this soon. I'm just doing some final adjustments.
Title: Re: Javascript plugin
Post by: Sir. Cucuruchito on Aug 09, 2018, 06:39 PM
I am waiting for this with enthusiasm. Good news approaches.

Everything what we need is a good documentation and it's quite.
Title: Re: Javascript plugin
Post by: NewK on Aug 14, 2018, 02:40 PM
I've created a sample script that you can see here: https://github.com/newk5/vcmp-javascript-plugin/tree/master/sampleScript/src

It's a very simple script, with basic functionality, it could've been organized better but I chose to keep it as simple as possible to make it easy to understand for everyone. It only has 3 commands: /register,  /login and /stats. It uses SQLite. I've included the DB, it only has a "Players" table with the following columns: id, nickname,password, ip, kills, joins, registerDate, lastActive and level. After registering and logging in, you can use the /stats command to see account info. Note that there is a node_modules folder, this folder should always exist and contains the available modules which at the moment are:

- crypto (for hashing),
- filesystem (disk IO like reading/writing from/to files),
- httpclient (http requests)
- ircbot
- sql (this module works for both SQLite and MySQL and uses the exact same API for both)

I have some more modules planned for the future, like websockets, a discord client and a few other, but for now I just want to make sure the existing modules are stable. I tried to comment most of the code, take a look at the script and if you have any questions feel free to ask.

I'm just doing some final tests at the moment. Soon, I'll provide plugin download links and instructions on how to set it up. After that I'll create some documentation.

Download here (https://forum.vc-mp.org/?topic=6173.msg42498#msg42498)