0
0
mirror of https://github.com/sp-tarkov/modules.git synced 2025-02-13 09:50:43 -05:00
Merijn Hendriks 9c89c31c68 Modernize HTTP v2 (!104)
This PR won't affect modders who have already updated their mods to 3.8.0. They just need to re-compile.

This is a resubmission of my previous PR (SPT-AKI/Modules#99) with additional code cleanup and fixes.
Instead of outright removing the functionality, this time I deprecate it instead (marked for removal in next release).

Requires SPT-AKI/Server#274 to function.

## Overview

- HTTP modernization
  - Adds `Aki.Common.Http.Client`, a replacement for `Aki.Common.Http.Request` and builds on top of `System.Net.Http.HttpClient`
  - Implements failsafe retries when requesting during busy connections
  - Improved debugging
  - Improved performance
  - Deprecades old request code
- Bundle system
  - Fixes remote downloaded bundles using external IP
  - Implements functional bundle caching from remote sources
  - Implements multi-threaded bundle downloads
  - Implements Unity-compatible bundle format support
  - Extensive cleanup
  - Deprecated unneccecary models

## Why?

In it's current state, the bundle system is ducktaped together in 2021, fundumentally broken and in desperate need of a cleanup and fixes.
The HTTP code hasn't been updated since 2021, and `HttpWebRequest` has been deprecated by Microsoft for a while now.
There was also a lot of opportunity left for simple performance gains that even reduces the complexity of the code.

As for why not two separate PRs (HTTP modernization, bundle rework): both were deeply interconnected. A change in one requires modification in the other. Hence the current approach.

## Testing

The code has been validated and tested by @TheSparta and me.
A large section of the code has been implemented and tested extensively by modders.

### Local

1. Start the game from 127.0.0.1
2. The game starts loading bundles from the mods path

### Remote, full re-aquire

1. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
2. Start the game from LAN IP
3. A folder named `user/cache/bundles` is created

### Remote, partial-aquire (deleted)

1. Ensure all bundles are cached
2. Delete one of the aquired bundles from cache
3. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
4. Start the game from LAN IP
5. The bundle is redownloaded

### Remote, partial-aquire (invalid crc)

1. Ensure all bundles are cached
2. Update a bundle mod with a new bundle on the same path
3. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
4. Start the game from LAN IP
5. The bundle is redownloaded

### Remote, use cache

1. Ensure all bundles are cached
2. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
3. Start the game from LAN IP
4. The game starts loading bundles from the cache path

## Risk assessment

In order to reduce friction between releases, this PR introduces a deprecation system.
Obsolete classes and methods have been marked deprecated.
These will remain available for the current release and continue to function as normal for this release.
A warning will be displayed during build when a modder relies on the deprecated functionality.
The marked classes and methods are to be removed in the next release.

The server-side changes have no impact on modders.

## Deprecation

The following classes are affected:

- `Aki.Common.Http.Request`: Replaced by `Aki.Common.Http.Request`
- `Aki.Common.Http.WebConstants`: Replaced by functionality from `System.Net.Http`
- `Aki.Custom.Models.BundleInfo`: Replaced by `Aki.Custom.Models.BundleItem`

The following methods are affected:

- `Aki.Common.Http.RequestHandler.GetData(path, hasHost)`: `hasHost` enables connection outside intended host.
- `Aki.Common.Http.RequestHandler.GetJson(path, hasHost)`: `hasHost` enables connection outside intended host.
- `Aki.Common.Http.RequestHandler.PostJson(path, json, hasHost)`: `hasHost` enables connection outside intended host.
- `Aki.Common.Http.RequestHandler.PutJson(path, json, hasHost)`: `hasHost` enables connection outside intended host.

The deprecated methods and `Aki.Custom.Models.BundleInfo` are self-contained and can be removed independently.
The deprecated classes require removal of all deprecated code at once.

Reviewed-on: SPT-AKI/Modules#104
Reviewed-by: TheSparta <thesparta@noreply.dev.sp-tarkov.com>
Co-authored-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
Co-committed-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
2024-03-29 18:43:46 +00:00

162 lines
4.6 KiB
C#

using System;
using System.IO;
using System.Net.Http;
using Aki.Common.Http;
using Aki.Common.Utils;
namespace Aki.Common.Http
{
// NOTE: you do not want to dispose this, keep a reference for the lifetime
// of the application.
// NOTE: cannot be made async due to Unity's limitations.
public class Client : IDisposable
{
protected readonly HttpClient _httpv;
protected readonly string _address;
protected readonly string _accountId;
protected readonly int _retries;
public Client(string address, string accountId, int retries = 3)
{
_address = address;
_accountId = accountId;
_retries = retries;
var handler = new HttpClientHandler()
{
// force setting cookies in header instead of CookieContainer
UseCookies = false
};
_httpv = new HttpClient(handler);
}
private HttpRequestMessage GetNewRequest(HttpMethod method, string path)
{
return new HttpRequestMessage()
{
Method = method,
RequestUri = new Uri(_address + path),
Headers = {
{ "Cookie", $"PHPSESSID={_accountId}" }
}
};
}
protected byte[] Send(HttpMethod method, string path, byte[] data, bool compress = true)
{
HttpResponseMessage response = null;
using (var request = GetNewRequest(method, path))
{
if (data != null)
{
// if there is data, convert to payload
byte[] payload = (compress)
? Zlib.Compress(data, ZlibCompression.Maximum)
: data;
// add payload to request
request.Content = new ByteArrayContent(payload);
}
// send request
response = _httpv.SendAsync(request).Result;
}
if (!response.IsSuccessStatusCode)
{
// response error
throw new Exception($"Code {response.StatusCode}");
}
using (var ms = new MemoryStream())
{
using (var stream = response.Content.ReadAsStreamAsync().Result)
{
// grap response payload
stream.CopyTo(ms);
var bytes = ms.ToArray();
if (bytes != null)
{
// payload contains data
return Zlib.IsCompressed(bytes)
? Zlib.Decompress(bytes)
: bytes;
}
}
}
// response returned no data
return null;
}
public byte[] Get(string path)
{
var error = new Exception("Internal error");
// NOTE: <= is intentional, 0 is send, 1,2,3 is retry
for (var i = 0; i <= _retries; ++i)
{
try
{
return Send(HttpMethod.Get, path, null, false);
}
catch (Exception ex)
{
error = ex;
}
}
throw error;
}
public byte[] Post(string path, byte[] data, bool compressed = true)
{
var error = new Exception("Internal error");
// NOTE: <= is intentional, 0 is send, 1,2,3 is retry
for (var i = 0; i <= _retries; ++i)
{
try
{
return Send(HttpMethod.Post, path, data, compressed);
}
catch (Exception ex)
{
error = ex;
}
}
throw error;
}
public void Put(string path, byte[] data, bool compressed = true)
{
var error = new Exception("Internal error");
// NOTE: <= is intentional, 0 is send, 1,2,3 is retry
for (var i = 0; i <= _retries; ++i)
{
try
{
Send(HttpMethod.Put, path, data, compressed);
return;
}
catch (Exception ex)
{
error = ex;
}
}
throw error;
}
public void Dispose()
{
_httpv.Dispose();
}
}
}