This patch contains the following:
- Initial async VFS code (for reading / writing files)
- Simplified Http Client code
- Added async support to Http Client, RequestHandler
- Improved RequestHandler logging
- Deferred bundle loading to EasyAssetPatch
- Make GetManifestJson run async
This comes with a number of benefits:
- When downloading bundles, it will mention which files succeeded or failed to download
- Bundle loading happens in the initial screen, not the white screen
- Fixed the issue where bundle loading could break bepinex loading (too long load time)
- Modders can now make async http request and read/write files async
I removed logging of sessionid inside the RequestHandler for each request, sessionid is already visible from bepinex log startup parameters.
At last, sorry for the amount of commits it took. I initially wanted to target the 3.9.0 branch, but decided to use 3.8.1 instead as async request can really help out some mods.
Reviewed-on: SPT-AKI/Modules#117
Co-authored-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
Co-committed-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
Inform user via messagebox that they need to run Aki.Launcher.exe
Refactored a bit of the validation code in Aki.Prepatch
Co-authored-by: DrakiaXYZ <565558+TheDgtl@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#116
Co-authored-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Co-committed-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Added as a check in the PrePatch, because while users delete `BepInEx/plugins/spt` often, it would be uncommon for them to also delete the prepatcher
No idea why there's a second commit in this PR, it doesn't actually change anything. Just Git Things™
Co-authored-by: Terkoiz <terkoiz@spt.dev>
Co-authored-by: DrakiaXYZ <565558+TheDgtl@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#109
Co-authored-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Co-committed-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
This PR won't affect modders who have already updated their mods to 3.8.0. They just need to re-compile.
This is a resubmission of my previous PR (SPT-AKI/Modules#99) with additional code cleanup and fixes.
Instead of outright removing the functionality, this time I deprecate it instead (marked for removal in next release).
Requires SPT-AKI/Server#274 to function.
## Overview
- HTTP modernization
- Adds `Aki.Common.Http.Client`, a replacement for `Aki.Common.Http.Request` and builds on top of `System.Net.Http.HttpClient`
- Implements failsafe retries when requesting during busy connections
- Improved debugging
- Improved performance
- Deprecades old request code
- Bundle system
- Fixes remote downloaded bundles using external IP
- Implements functional bundle caching from remote sources
- Implements multi-threaded bundle downloads
- Implements Unity-compatible bundle format support
- Extensive cleanup
- Deprecated unneccecary models
## Why?
In it's current state, the bundle system is ducktaped together in 2021, fundumentally broken and in desperate need of a cleanup and fixes.
The HTTP code hasn't been updated since 2021, and `HttpWebRequest` has been deprecated by Microsoft for a while now.
There was also a lot of opportunity left for simple performance gains that even reduces the complexity of the code.
As for why not two separate PRs (HTTP modernization, bundle rework): both were deeply interconnected. A change in one requires modification in the other. Hence the current approach.
## Testing
The code has been validated and tested by @TheSparta and me.
A large section of the code has been implemented and tested extensively by modders.
### Local
1. Start the game from 127.0.0.1
2. The game starts loading bundles from the mods path
### Remote, full re-aquire
1. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
2. Start the game from LAN IP
3. A folder named `user/cache/bundles` is created
### Remote, partial-aquire (deleted)
1. Ensure all bundles are cached
2. Delete one of the aquired bundles from cache
3. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
4. Start the game from LAN IP
5. The bundle is redownloaded
### Remote, partial-aquire (invalid crc)
1. Ensure all bundles are cached
2. Update a bundle mod with a new bundle on the same path
3. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
4. Start the game from LAN IP
5. The bundle is redownloaded
### Remote, use cache
1. Ensure all bundles are cached
2. Start the server from LAN IP (http.json, set host to `cmd > ipconfig` address, example: `192.168.178.32`)
3. Start the game from LAN IP
4. The game starts loading bundles from the cache path
## Risk assessment
In order to reduce friction between releases, this PR introduces a deprecation system.
Obsolete classes and methods have been marked deprecated.
These will remain available for the current release and continue to function as normal for this release.
A warning will be displayed during build when a modder relies on the deprecated functionality.
The marked classes and methods are to be removed in the next release.
The server-side changes have no impact on modders.
## Deprecation
The following classes are affected:
- `Aki.Common.Http.Request`: Replaced by `Aki.Common.Http.Request`
- `Aki.Common.Http.WebConstants`: Replaced by functionality from `System.Net.Http`
- `Aki.Custom.Models.BundleInfo`: Replaced by `Aki.Custom.Models.BundleItem`
The following methods are affected:
- `Aki.Common.Http.RequestHandler.GetData(path, hasHost)`: `hasHost` enables connection outside intended host.
- `Aki.Common.Http.RequestHandler.GetJson(path, hasHost)`: `hasHost` enables connection outside intended host.
- `Aki.Common.Http.RequestHandler.PostJson(path, json, hasHost)`: `hasHost` enables connection outside intended host.
- `Aki.Common.Http.RequestHandler.PutJson(path, json, hasHost)`: `hasHost` enables connection outside intended host.
The deprecated methods and `Aki.Custom.Models.BundleInfo` are self-contained and can be removed independently.
The deprecated classes require removal of all deprecated code at once.
Reviewed-on: SPT-AKI/Modules#104
Reviewed-by: TheSparta <thesparta@noreply.dev.sp-tarkov.com>
Co-authored-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
Co-committed-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
This is a workflow that triggers the build workflow of the Build project whenever this project has a tag pushed to it.
Reviewed-on: SPT-AKI/Modules#103
Co-authored-by: Refringe <me@refringe.com>
Co-committed-by: Refringe <me@refringe.com>
Most of this code was (re-)written in 0.12.9, almost 3 years ago.
Current understanding of how it worked was limited, so I went back and broke it until I understood it properly.
I added comments where I could and slightly altered the logic of `GetManifestJson` to make it easier to read.
Regarding removal of `GetManifestBundle`: you can't. While `Windows.json` contains most of the bundle info, some info (like `Doge`) resides outside of this. Removing `GetManifestBundle` results in the game failing to load.
Reviewed-on: SPT-AKI/Modules#101
Co-authored-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
Co-committed-by: Merijn Hendriks <merijn.d.hendriks@gmail.com>
This reverts changes to the directory paths made in commit: c7bc0b05143c7062a0c70150cb872b5070f4859c
fixes [SPT-AKI/Issues#588](SPT-AKI/Issues#588)
Co-authored-by: MadByte <madbyte@tuta.io>
Reviewed-on: SPT-AKI/Modules#100
Co-authored-by: MadByte <madbyte@noreply.dev.sp-tarkov.com>
Co-committed-by: MadByte <madbyte@noreply.dev.sp-tarkov.com>
This patch can be used to dump the data contained in the `allExtracts.json` files in the server locations database. When enabled, load into each map sequentially, and the extract data will be dumped to `ExfilDumps/{mapName}/allExtracts.json`, ready to be copy/pasted into the server `database/locations` directory
Co-authored-by: DrakiaXYZ <565558+TheDgtl@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#98
Co-authored-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Co-committed-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Added a patch to handle Closing all extracts but 1 for when cultist ritual succeeds in halloween event.
Halloween Event doesnt seem to work well when map has newspawn system enabled. I think bots are getting stuck getting loaded. causing delayed spawns of cultists.
Fixes exception thrown when trace logging is enabled and you accept a quest
Co-authored-by: DrakiaXYZ <565558+TheDgtl@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#97
Co-authored-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Co-committed-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Needs merged with: SPT-AKI/Server#262
Short story first: Last night I was poking around the code as I always do looking for the cause of an exception and got sick and tired of the generic bad exceptions we always work off of. So I fixed it.
This allows us access to BSG's internal logging using Nlog. This is useful for everybody. Not just SPT dev's, but mod dev's alike. It can be enabled from `core.json` with the option to send the output to the server.
Here's an example of the improvement, its huge. Bepinex console is the gerneric logging we always use, the editor behind it is the improved logging from Nlog. Its night and day.
The levels are based off of Nlog ordinals where low is more logging, high is less. (Stupid I know, but what can you do)
Verbosity is as follows:
* In all cases, except off, better exceptions will be logged.
* to see the logging output in your bepinex logs, make sure you have debug logging enabled in bepinex.
* WARNING: trace-info logging will quickly create log files in the megabytes.
* 0 - trace
* 1 - debug
* 2 - info
* 3 - warn
* 4 - error
* 5 - fatal
* 6 - off
Co-authored-by: Cj <161484149+CJ-SPT@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#96
Co-authored-by: Cj <cj@noreply.dev.sp-tarkov.com>
Co-committed-by: Cj <cj@noreply.dev.sp-tarkov.com>
needs merged with: SPT-AKI/Server#258
Shows if mods are loaded in the cool debug message, will show if either server mods or client mods are present.
Co-authored-by: Cj <161484149+CJ-SPT@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#95
Co-authored-by: Cj <cj@noreply.dev.sp-tarkov.com>
Co-committed-by: Cj <cj@noreply.dev.sp-tarkov.com>
Rewrite the BTRTurretView attach patch, so we no longer need the BotInit patch
Co-authored-by: DrakiaXYZ <565558+TheDgtl@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#92
Co-authored-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Co-committed-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
To handle in-raid LK task unlocking, when a task is switched from AvailableAfter to Locked, instead switch it to AvailableForStart
This handles the lack of `status` values for tasks, where the target state would normally be read from, without risking breaking non-LK tasks by adding that data.
Co-authored-by: DrakiaXYZ <565558+TheDgtl@users.noreply.github.com>
Reviewed-on: SPT-AKI/Modules#91
Co-authored-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
Co-committed-by: DrakiaXYZ <drakiaxyz@noreply.dev.sp-tarkov.com>
