Add index to properties table by mario · Pull Request #20716 · nextcloud/server

mario · 2020-04-29T12:12:00Z

Signed-off-by: Mario Danic mario@lovelyhq.com

mario · 2020-04-29T14:05:20Z

Would be awesome to get this in 19.

mario · 2020-04-29T19:34:51Z

This speeds up access to folders with lots of files considerably.

mario · 2020-04-29T19:35:34Z

I'm pretty sure the odd test failure is not related to me adding indexes.

icewind1991 · 2020-04-29T20:14:01Z

The autoloaders are not up to date
Please run: bash build/autoloaderchecker.sh
And commit the result

is yours

mario · 2020-04-29T20:16:13Z

The autoloaders are not up to date
Please run: bash build/autoloaderchecker.sh
And commit the result

is yours

Thanks.

gary-kim

I think you may have accidentally committed composer.phar?

mario · 2020-04-29T22:13:35Z

I think you may have accidentally committed composer.phar?

Good catch, fixed.

Addressed

MorrisJobke

Please add to the check for the missing indices so that admins are notified about this:

server/core/Application.php

Line 74 in cb05782

$eventDispatcher->addListener(IDBConnection::CHECK_MISSING_INDEXES_EVENT,

mario · 2020-04-30T12:22:19Z

Please add to the check for the missing indices so that admins are notified about this:

server/core/Application.php

Line 74 in cb05782

$eventDispatcher->addListener(IDBConnection::CHECK_MISSING_INDEXES_EVENT,

Fix pushed.

mario · 2020-04-30T12:44:45Z

So lint is complaining about lib/private/Files/ObjectStore/Swift.php which I didn't touch and I can't run composer run cs:fix because it errors out with:

Fatal error: Uncaught Error: Class 'Nextcloud\CodingStandard\Config' not found in /Users/mario/Projects/server/.php_cs.dist:9

MorrisJobke · 2020-04-30T12:59:40Z

So lint is complaining about lib/private/Files/ObjectStore/Swift.php which I didn't touch and I can't run composer run cs:fix because it errors out with:

It is already fixed in #20742

core/Application.php

core/Command/Db/AddMissingIndices.php

core/Migrations/Version19000Date20200429140134.php

mario · 2020-04-30T13:07:12Z

@MorrisJobke thanks, fixed. Thanks to you and @icewind1991 for mentoring, and @gary-kim and @ChristophWurst of course! :)

Fixed.

MorrisJobke · 2020-04-30T13:08:30Z

core/Migrations/Version19000Date20200429140134.php

+use OCP\Migration\IOutput;
+use OCP\Migration\SimpleMigrationStep;
+
+class Version19000Date20200429140134 extends SimpleMigrationStep {


I just noticed that you want to do this in a migration. Then the step in the "add-missing-indices" is not needed. I guess we should not do the migration here (because it can take really long and thus break the update). Better is to put the addIndex() to an old migration, so that it is executed on new installs. And for existing installations we do the approach with the admin notification and the add-missing-indices command to add the index while the instance is online.

@mario I just noticed that I had this as pending open. That's why I said that in the chat back then ;)

What migration do you want me to put it in @MorrisJobke ?

Right below that line I would say:

server/core/Migrations/Version13000Date20170718121200.php

Line 291 in cb05782

$table->setPrimaryKey(['userid', 'appid', 'configkey']);

(it's the only migration for preferences in core)

Just for reference - that's how we did it for other indices as well: https://github.com/nextcloud/server/pull/13213/files

You just confused the hell out of me - I hope you meant few lines further where the migration for properties is, and not preferences? :D

This is for preferences, I need it for properties :P But found it, thanks!

Ah yes - sorry. Far better now 👍

Signed-off-by: Mario Danic <mario@lovelyhq.com>

mario · 2020-04-30T21:05:05Z

Let's hope its good now :p

dennisTGC · 2020-06-09T07:49:47Z

Will this be back-ported to a minor v18 release?

mario marked this pull request as draft April 29, 2020 12:12

mario requested review from icewind1991 and rullzer April 29, 2020 12:12

mario force-pushed the index-propertypath branch from 8cca823 to 4946e7c Compare April 29, 2020 13:13

mario marked this pull request as ready for review April 29, 2020 13:27

mario added the 3. to review Waiting for reviews label Apr 29, 2020

mario added this to the Nextcloud 19 milestone Apr 29, 2020

rullzer mentioned this pull request Apr 29, 2020

19 beta 6 #20611

Merged

11 tasks

icewind1991 approved these changes Apr 29, 2020

View reviewed changes

mario force-pushed the index-propertypath branch from 4946e7c to 2db36ad Compare April 29, 2020 20:15

gary-kim previously requested changes Apr 29, 2020

View reviewed changes

mario force-pushed the index-propertypath branch from 2db36ad to 9e2aecb Compare April 29, 2020 22:13

ChristophWurst approved these changes Apr 30, 2020

View reviewed changes

MorrisJobke previously requested changes Apr 30, 2020

View reviewed changes

mario requested review from MorrisJobke and gary-kim April 30, 2020 12:24

mario force-pushed the index-propertypath branch from bad5a14 to 4243d5d Compare April 30, 2020 12:36

mario force-pushed the index-propertypath branch from 4243d5d to b94240c Compare April 30, 2020 13:03

MorrisJobke reviewed Apr 30, 2020

View reviewed changes

core/Application.php Outdated Show resolved Hide resolved

MorrisJobke reviewed Apr 30, 2020

View reviewed changes

core/Command/Db/AddMissingIndices.php Outdated Show resolved Hide resolved

MorrisJobke reviewed Apr 30, 2020

View reviewed changes

core/Migrations/Version19000Date20200429140134.php Outdated Show resolved Hide resolved

mario force-pushed the index-propertypath branch from d8cc68d to f282975 Compare April 30, 2020 13:06

mario requested a review from MorrisJobke April 30, 2020 13:18

rullzer mentioned this pull request Apr 30, 2020

19 beta 7 #20754

Merged

2 tasks

MorrisJobke reviewed Apr 30, 2020

View reviewed changes

Add index to properties table

6e28c28

Signed-off-by: Mario Danic <mario@lovelyhq.com>

mario force-pushed the index-propertypath branch from f282975 to 6e28c28 Compare April 30, 2020 21:05

mario changed the title ~~Add index to oc_properties~~ Add index to properties table Apr 30, 2020

rullzer merged commit 80372a3 into master May 1, 2020

rullzer deleted the index-propertypath branch May 1, 2020 10:39

This was referenced May 4, 2020

Very slow access (copy, ls, etc) files in folders with lots of files through webdav + proposed fix #8962

Closed

performance issue when deleting folder of user with many files #3021

Closed

ChristophWurst mentioned this pull request Dec 23, 2021

Add missing index for propertypath only queries of DAV properties #30392

Merged

1 task

solracsf mentioned this pull request Oct 24, 2023

Remove out-of-date issue iterate-ch/docs#430

Merged

Uh oh!

Conversation

mario commented Apr 29, 2020

Uh oh!

mario commented Apr 29, 2020

Uh oh!

mario commented Apr 29, 2020

Uh oh!

mario commented Apr 29, 2020

Uh oh!

icewind1991 commented Apr 29, 2020

Uh oh!

mario commented Apr 29, 2020

Uh oh!

gary-kim left a comment

Choose a reason for hiding this comment

Uh oh!

mario commented Apr 29, 2020

Uh oh!

MorrisJobke left a comment

Choose a reason for hiding this comment

Uh oh!

mario commented Apr 30, 2020

Uh oh!

mario commented Apr 30, 2020

Uh oh!

MorrisJobke commented Apr 30, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mario commented Apr 30, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mario commented Apr 30, 2020

Uh oh!

dennisTGC commented Jun 9, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Comments