Is it possible download server files through HTTP?

[ysc3839]

Is it possible download server files through HTTP protocol?

[.]

Mmm, if you want to create a module in which you bind the cURL library then yes you could (any many other protocols) Otherwise, you could use the system(...) function to work with the cURL binary from your OS or another utility if you prefer and use temporary files as output.

Something similar to what this snippet does.

[Thijn]

With server files, do you mean the files the client downloads before joining?
If so, probably not.

[KAKAN]

With server files, do you mean the files the client downloads before joining?
If so, probably not.

Yea, he was talking about that. I too wanted that. The downloader inbuilt in VCMP is extremely slow.

[Thijn]

The downloader works fine on normal connections, but on servers with high latency it can time out pretty easily.

[.]

It's normal to be slow. It's meant to be slow so that it won't lag your server if multiple people download files from your server. Imagine your server has a 100 MB/s connection. 10 people connect and start downloading files at 10 MB/s What about people that play on the server? Do you think they'll enjoy the lag?

Of course you could ask the devs to offer a way to manually set this limit. That way, you can't blame the devs they lagged your server and you are in control of stuff.

[Stormeus]

It's normal to be slow. It's meant to be slow so that it won't lag your server if multiple people download files from your server. Imagine your server has a 100 MB/s connection. 10 people connect and start downloading files at 10 MB/s What about people that play on the server? Do you think they'll enjoy the lag?

That's not really it. File transfers are handled through RakNet over UDP, and RakNet tries its hardest to handle it, but we don't impose a cap on transfer speeds. It's just that, to everyone's surprise, UDP is a terrible protocol for reliably transferring large files. Using a TCP-based protocol, or giving the option of using an external HTTP server, would very likely improve things for people with unreliable or slow connections. This is a point I raised with the other devs a couple of times over the summer.

[.]

I remembered CS 1.6 did the same and I thought you did as well. Though the limit was about ~8 kb/s there

[ysc3839]

It's normal to be slow. It's meant to be slow so that it won't lag your server if multiple people download files from your server. Imagine your server has a 100 MB/s connection. 10 people connect and start downloading files at 10 MB/s What about people that play on the server? Do you think they'll enjoy the lag?

That's not really it. File transfers are handled through RakNet over UDP, and RakNet tries its hardest to handle it, but we don't impose a cap on transfer speeds. It's just that, to everyone's surprise, UDP is a terrible protocol for reliably transferring large files. Using a TCP-based protocol, or giving the option of using an external HTTP server, would very likely improve things for people with unreliable or slow connections. This is a point I raised with the other devs a couple of times over the summer.

Thanks! I hope I could see this in future updates!

[EK.IceFlake]

Also if you send it in small chunks instead of a single packet for the file I think it can be faster and maybe more reliable.

[.]

Also if you send it in small chunks instead of a single packet for the file I think it can be faster and maybe more reliable.

Sorry but how do you think they're sent actually? You just send a 4,10,16,20 ... mb file in one packet? The problem here is that UDP is unreliable and unordered. Therefore, when the file is sent into chunks, you need to wait for all the chunks on the other end and then reply back what chunks you have received so that the server doesn't think certain chunks got lost and tries to send them again. And then assemble them into the order they were disassembled.

Either way, it's like taking a house apart into to small pieces and then throwing them over a river and expecting to form the same house again by putting them together in the order you receive them. The only issue is that some parts might reach quicker than others and some parts might fall into the river and you get a mess on the other side if you don't know how to manage things.

[EK.IceFlake]

Look at this example of a few UDP packets their checksum isn't real
Packet 1

Code Select Expand

CHKSUM000000000001ID0000000000BYTES00016BYTE0NAMELEN00024store/objects/bridge.xml
<?xml version="1Packet 2

Code Select Expand

CHKSUM000000000002ID0000000000BYTES00016BYTE0NAMELEN00024store/objects/bridge.xml
.6" encoding="ASand so on...
btw they are shown here in a format used to reduce file size by reducing useless whitespace or punctuation since they are interpreted by a machine.
If a client notices a checksum is wrong it requests that bit instead of the whole file again better than having to send 16 mb files a few times.
Also the size could be reduced by using some hex codes or simply omitting them and using things such as checksum=byte 0 length 12 hardcoded into the client.
For example

Code Select Expand

000000000001000000000000016000024store/objects/bridge.xml
<?xml version="1Packet 2

Code Select Expand

000000000002000000000000016000024store/objects/bridge.xml
.6" encoding="ASThis is just an example which could be greatly improved.
For example instead of specifying the length(which is sent in the UDP header) you can use a \n or you could define the length of the lengths to save more space.
Don't try to find mistakes there might be some but I'm just giving an example.

[Thijn]

S.L.C. just told you the client already does this. It's not going to download a whole file in one packet.
Do you actually read the replies?

[EK.IceFlake]

'Kay but I can't find him saying that