fix(performance): Do not set up filesystem on every call by miaulalala · Pull Request #36589 · nextcloud/server

miaulalala · 2023-02-07T14:56:50Z

Removed old OO_FileChunking logic that produced GC- collectable chunks

Added BG Job that will run a garbage collection for every user

Signed-off-by: Anna Larch anna@nextcloud.com

Resolves: Basic auth requests always set up the filesystem #31357

Summary

Comparison for OCS requests:
https://blackfire.io/profiles/compare/2c3c3699-137e-49a3-a363-e0950bb278d4...41a33cf0-71ee-42aa-9791-9ef74f99e6c3/graph?settings%5Bdimension%5D=wt&settings%5Bdisplay%5D=landscape&settings%5BtabPane%5D=nodes&selected=&callname=main()&constraintDoc=

TODO

Testing

Checklist

Code is properly formatted
Sign-off message is added to all commits
Tests (unit, integration, api and/or acceptance) are included
~~Screenshots before/after for front-end changes~~
Documentation (manuals or wiki) has been updated or is not required
~~Backports requested where applicable (ex: critical bugfixes)~~

+	public function run($argument) {
+		$this->userManager->callForSeenUsers(function (IUser $user): void {
+			$this->logger->debug('Running chunk cleanup job for user '. $user->getUID());
+			$fileCache = new File();


+	/**
+	 * This job cleans up all backups except the latest 3 from the updaters backup directory
+	 */
+	public function run($argument): void {


stale

miaulalala · 2023-02-13T15:09:29Z

/rebase

juliusknorr · 2023-02-17T13:32:27Z

Last commit seems to indeed break subfolder listing of mount points :/ Needs another dive into why Depth > 0 requests for dav fail with shares being moved into those subfolders without that then.

Also remove old Oc_FileChunking logis that produced GC- collectable chunks Signed-off-by: Anna Larch <anna@nextcloud.com>

Signed-off-by: Julius Härtl <jus@bitgrid.net>

juliusknorr · 2023-02-20T07:59:15Z

I finally figured out why the failure was happening. When using the nodes api in dav we initialise it with the wrong view fake root /user/files instead of ``. This doesn't cause issues as dav rarely calls nodes api endpoints that then access the view. Pushed a fix in this PR to reuse already passed LazyFolder instances. The full fix is separate in #36774

DaphneMuller · 2023-02-21T13:01:15Z

hello @miaulalala
Thank you so much for your work on this pull request! This ticket seems to have the tag 'pending documentation'.
Is there any chance you could clarify what documentation is missing? Is this for admins or for app developers?

miaulalala · 2023-02-21T14:16:24Z

This PR was reverted @DaphneMuller

miaulalala self-assigned this Feb 7, 2023

miaulalala marked this pull request as draft February 7, 2023 14:57

miaulalala added 2. developing Work in progress performance 🚀 labels Feb 7, 2023

miaulalala requested review from come-nc, icewind1991 and juliusknorr February 7, 2023 15:13

icewind1991 previously requested changes Feb 7, 2023

View reviewed changes

Comment thread lib/base.php

miaulalala force-pushed the enh/perf-remove-icache branch from a0a28a9 to aadd950 Compare February 7, 2023 17:01

come-nc approved these changes Feb 7, 2023

View reviewed changes

Comment thread apps/dav/lib/Connector/Sabre/Directory.php

Comment thread apps/dav/lib/Connector/Sabre/Directory.php

come-nc reviewed Feb 7, 2023

View reviewed changes

Comment thread lib/private/Server.php Outdated

miaulalala requested review from ChristophWurst and icewind1991 February 7, 2023 18:59

github-advanced-security AI found potential problems Feb 7, 2023

View reviewed changes

Comment thread apps/files/lib/BackgroundJob/FileChunkCleanupJob.php Fixed

Comment thread apps/files/lib/BackgroundJob/FileChunkCleanupJob.php Fixed

Comment thread apps/files/lib/BackgroundJob/FileChunkCleanupJob.php Fixed

Comment thread apps/files/lib/BackgroundJob/FileChunkCleanupJob.php Fixed

github-advanced-security AI found potential problems Feb 7, 2023

View reviewed changes

miaulalala marked this pull request as ready for review February 7, 2023 19:35

miaulalala force-pushed the enh/perf-remove-icache branch from ac9d919 to 60f1f7f Compare February 7, 2023 19:48

come-nc approved these changes Feb 8, 2023

View reviewed changes

miaulalala force-pushed the enh/perf-remove-icache branch from a7e914a to a7b96c5 Compare February 8, 2023 13:26

juliusknorr mentioned this pull request Feb 9, 2023

perf(dav): Do not call general setupFS on ever dav auth #36626

Merged

4 tasks

miaulalala added pending documentation This pull request needs an associated documentation update 3. to review Waiting for reviews and removed 2. developing Work in progress labels Feb 9, 2023

miaulalala changed the title ~~Do not set up filesystem on every call~~ fix(performance): Do not set up filesystem on every call Feb 9, 2023

miaulalala force-pushed the enh/perf-remove-icache branch from a90f63e to 0b25e61 Compare February 9, 2023 15:08

juliusknorr approved these changes Feb 9, 2023

View reviewed changes

juliusknorr added this to the Nextcloud 26 milestone Feb 9, 2023

juliusknorr force-pushed the enh/perf-remove-icache branch from 6473d0d to 1a7041e Compare February 9, 2023 21:44

juliusknorr force-pushed the enh/perf-remove-icache branch 2 times, most recently from 756cc5e to 270828b Compare February 10, 2023 10:54

nextcloud-command force-pushed the enh/perf-remove-icache branch from 21d3ea8 to cbdd2b2 Compare February 13, 2023 15:31

juliusknorr force-pushed the enh/perf-remove-icache branch 2 times, most recently from 15b85a1 to 248c224 Compare February 14, 2023 21:02

miaulalala and others added 5 commits February 17, 2023 19:18

Do not set up filesystem on every call

5d4efb4

Also remove old Oc_FileChunking logis that produced GC- collectable chunks Signed-off-by: Anna Larch <anna@nextcloud.com>

tests: Fix test isolation on object storage

20058eb

Signed-off-by: Julius Härtl <jus@bitgrid.net>

tests: Remove legacy checksum test

3493099

Signed-off-by: Julius Härtl <jus@bitgrid.net>

tests: Remove legacy chunking tests

ec35650

Signed-off-by: Julius Härtl <jus@bitgrid.net>

fix: Use passed LazyFolder when building sabre nodes

3bdf332

Signed-off-by: Julius Härtl <jus@bitgrid.net>

juliusknorr force-pushed the enh/perf-remove-icache branch from 248c224 to 3bdf332 Compare February 17, 2023 18:21

juliusknorr mentioned this pull request Feb 18, 2023

fix: always use proper path on node api when calling the view #36774

Merged

juliusknorr merged commit bba3a1c into master Feb 20, 2023

juliusknorr deleted the enh/perf-remove-icache branch February 20, 2023 07:59

This was referenced Feb 21, 2023

Regression in filesystem handling breaks Talk/ attachment folder #36787

Closed

Revert "fix(performance): Do not set up filesystem on every call" #36788

Merged

juliusknorr mentioned this pull request Feb 21, 2023

Basic auth requests always set up the filesystem #31357

Open

1 task

skjnldsv mentioned this pull request Feb 23, 2023

26.0.0 beta 5 #36834

Merged

ChristophWurst removed the pending documentation This pull request needs an associated documentation update label Mar 16, 2023

icewind1991 mentioned this pull request Jun 19, 2024

chore: remove chunking-v1 #45973

Merged

provokateurin mentioned this pull request May 27, 2025

perf(base): Stop setting up the FS for every basic auth request #53141

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(performance): Do not set up filesystem on every call#36589

fix(performance): Do not set up filesystem on every call#36589
juliusknorr merged 5 commits intomasterfrom
enh/perf-remove-icache

miaulalala commented Feb 7, 2023 •

edited by juliusknorr

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check notice

Check notice

miaulalala commented Feb 13, 2023

Uh oh!

juliusknorr commented Feb 17, 2023

Uh oh!

juliusknorr commented Feb 20, 2023

Uh oh!

DaphneMuller commented Feb 21, 2023

Uh oh!

miaulalala commented Feb 21, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Conversation

miaulalala commented Feb 7, 2023 • edited by juliusknorr Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

TODO

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check notice

Check notice

miaulalala commented Feb 13, 2023

Uh oh!

juliusknorr commented Feb 17, 2023

Uh oh!

juliusknorr commented Feb 20, 2023

Uh oh!

DaphneMuller commented Feb 21, 2023

Uh oh!

miaulalala commented Feb 21, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

miaulalala commented Feb 7, 2023 •

edited by juliusknorr

Loading