waffle.lord/Patcher
0
0
Fork 0
Code Issues Pull Requests Projects Releases 22 Wiki Activity

Improve patch creation performance #20

Merged
waffle.lord merged 1 commits from :perf-nocompression into main 2024-06-25 08:54:21 -04:00
Conversation 1 Commits 1 Files Changed 1 +7 -1
DrakiaXYZ commented 2024-06-25 00:21:54 -04:00
Contributor
Copy Link
  • Skip MD5 hash generation if the file size differs, as it's guaranteed to be different
  • Disable xdelta compression, LZMA does a better job and quicker
- Skip MD5 hash generation if the file size differs, as it's guaranteed to be different - Disable xdelta compression, LZMA does a better job and quicker
DrakiaXYZ added 1 commit 2024-06-25 00:21:54 -04:00 
- Skip MD5 hash generation if the file size differs, as it's guaranteed to be different
- Disable xdelta compression, LZMA does a better job and quicker
DrakiaXYZ commented 2024-06-25 00:22:39 -04:00
Author
Contributor
Copy Link

From some quick testing, these are the runtimes of a few different scenarios (All creating patches from 30626 to 29197):
Base patch creation: 7:38
Skip MD5 when file size differs: 7:21
Skip MD5 + Disable compression: 3:54

And some 7z compression sizes (Excluding patcher.exe, this is just patch files):
Base delta files, compressed with 7z normal: 1,619,961 KB
Uncompressed delta files, compressed with 7z normal: 1,576,445 KB

And on-disk sizes of the deltas (Also excluding patcher.exe):
Base delta files: 1.54GB
Uncompressed delta files: 1.95GB

The tradeoff for faster patcher creation and better compression for upload/download is slightly more disk usage on end user systems during the installation process

From some quick testing, these are the runtimes of a few different scenarios (All creating patches from 30626 to 29197): Base patch creation: 7:38 Skip MD5 when file size differs: 7:21 Skip MD5 + Disable compression: 3:54 And some 7z compression sizes (Excluding patcher.exe, this is just patch files): Base delta files, compressed with 7z normal: 1,619,961 KB Uncompressed delta files, compressed with 7z normal: 1,576,445 KB And on-disk sizes of the deltas (Also excluding patcher.exe): Base delta files: 1.54GB Uncompressed delta files: 1.95GB The tradeoff for faster patcher creation and better compression for upload/download is slightly more disk usage on end user systems during the installation process
waffle.lord merged commit e5f6c681de into main 2024-06-25 08:54:21 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: waffle.lord/Patcher#20
No description provided.
Delete Branch ":perf-nocompression"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?