Commit Graph

894 Commits (2ff0fffbd32a8aa5df9198ffe0ca1638fa1810bf)

Author SHA1 Message Date
John Molakvoæ 301ecd0fc0
Merge pull request #30660 from nextcloud/backport/29622/stable21 2022-01-20 10:22:22 +07:00
Carl Schwan 396157af18 Fix psalm issues
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-14 14:16:59 +07:00
Carl Schwan 73e402f715 Add helper method in Wrapper
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-14 14:16:54 +07:00
John Molakvoæ 6b94630f44
Implement multibucket shift for ObjectStore
Signed-off-by: John Molakvoæ <skjnldsv@protonmail.com>
2022-01-14 08:39:37 +07:00
Vincent Petry 79d26e4d7f Fix fail when keys/files folder already exists
Fixes an issue with transfer ownership in move mode where the folder
"files_encryption/keys/files" already exists.

Instead of failing, its existence is checked before calling mkdir.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2022-01-11 10:15:40 +07:00
Robin Appelman f6024a895c
also use case sensitive like when searching in a folder
this case was missed from https://github.com/nextcloud/server/pull/29080

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-12-14 17:49:48 +07:00
Vincent Petry 65d4d30251 Check resource before closing in encryption wrapper
In case of error there is no guarantee that $source or $target is set or
is a resource when handling an error.

Without this fix, there's a risk that fclose will fail and the actual
exception will not be thrown, making it impossible to find out about the
root cause.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-12-10 00:10:44 +07:00
Louis f41c1fffff
Merge pull request #30056 from nextcloud/backport/29735/stable21
[stable21] find users for background scan one by one
2021-12-08 18:06:00 +07:00
John Molakvoæ fda86f8c8c
Merge pull request #29834 from nextcloud/carl/bugfix/file-search-21 2021-12-06 14:51:55 +07:00
Robin Appelman bd677f6741 background scan the source storage when a background scan on a storage jail is triggered
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-12-02 20:00:45 +07:00
Carl Schwan 8f9d1b31df
Fix search results filtering
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-11-26 16:30:47 +07:00
MichaIng 6d8cf37f06
Merge pull request #29907 from nextcloud/backport/29902/stable21
[stable21] Check for invalid characters before trimming
2021-11-26 14:22:04 +07:00
Joas Schilling f700abc889 Check for invalid characters before trimming
Signed-off-by: Joas Schilling <coding@schilljs.com>
2021-11-25 15:34:35 +07:00
MichaIng 9db07032d5 Use pre-v22 query commands
Signed-off-by: MichaIng <micha@dietpi.com>
2021-11-24 14:47:23 +07:00
Robin Appelman a3dc399e2f better cleanup of filecache when deleting an external storage
this way it can delete the cache entries even with per-user credentials

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-11-24 14:47:23 +07:00
Vincent Petry ca37664887 Normalize directory entries in Encoding wrapper
Directory entry file names are now normalized in getMetaData(),
getDirectoryContents() and opendir().

This makes the scanner work properly as it assumes pre-normalized names.

In case the names were not normalized, the scanner will now skip the
entries and display a warning when applicable.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-19 14:51:39 +07:00
Vincent Petry 88b5860e70 Move storage encoding compatibility warning logic
The encoding check for file names is now happening the Scanner, and an
event will be emitted only if the storage doesn't contain the encoding
compatibility wrapper.

The event is listened to by the occ scan command to be able to display a
warning in case of file name mismatches when they have NFD encoding.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-19 14:51:39 +07:00
Vincent Petry 2e79444828 Normalize file name before existence check in scanner
The scanner would not find a NFD-encoded file name in an
existing file list that is normalized.

This normalizes the file name before scanning.

Fixes issues where scanning repeatedly would make NFD files flicker in
and out of existence in the file cache.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-19 14:51:39 +07:00
John Molakvoæ 5b89d34397
Revert "[21] tell mysql to ignore the sort index for search queries" 2021-11-04 11:10:46 +07:00
John Molakvoæ 07c20b4f22
Merge pull request #29322 from nextcloud/mysql-search-ignore-index-21 2021-11-04 09:00:16 +07:00
Sijmen Schoon 43e829a93e Move query outside the loop and reduce chunk size to 1000
This involved changing CacheQueryBuilder\whereParentIn to take a
parameter name, renaming the function accordingly.

Signed-off-by: Sijmen Schoon <me@sijmenschoon.nl>
2021-11-01 11:27:36 +07:00
Sijmen Schoon bb198c635f Limit parameter count per query in Cache.removeChildren
Signed-off-by: Sijmen Schoon <me@sijmenschoon.nl>
2021-11-01 11:27:36 +07:00
MichaIng e250aef6b4
Merge pull request #29416 from nextcloud/backport/27440/stable21
[stable21] Handle files with `is_file` instead of `file_exists`
2021-10-30 00:19:35 +07:00
Carl Schwan 00a9b0c1db Fix permissions when copying from ObjectStorage
Make sure that when a user copy a file from a directory they don't have
all permissions to a directory where they have more permissions, the
permissions are correctly set to the one from the parent taget folder.

This was caused by the ObjectStoreStorage::copyFromStorage using
the jailed storage and cache entry instead of the unjailed one like other
storages (the local one).
Steps to reproduce

+ Use object storage
+ Create a groupfolder with one group having full permission and another one
  who can just read files.
+ With an user who is in the second group, copy a file from the groupfolder to
  the home folder of this user.
+ The file in the home folder of the user will be read only and can't be deleted
  even though it is in their home folder and they are the owner. In oc_filecache,
  the permissions stored for this file are 1 (READ)

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-28 12:55:53 +07:00
Robin Appelman 99ba96524e
more reliable return value for Watcher::checkUpdate
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-26 14:45:08 +07:00
Robin Appelman f18a76fec9
s3 external storage listing rework
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-26 14:41:46 +07:00
acsfer eea1c23719 Handle files with is_file instead of file_exists
Should fix things like `fread(): read of 8192 bytes failed with errno=21 Is a directory`
2021-10-23 09:28:25 +07:00
Carl Schwan ec8b2a90e6 Fix security issues when copying groupfolder with advanced ACL
Using advanced ACL, it is possible that an user has access to a
directory but not to a subdirectory, so the copying use
Common::copyFromStorage instead of Local::copyFromStorage.

Fix https://github.com/nextcloud/groupfolders/issues/1692

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-21 16:34:46 +07:00
Robin Appelman 5002bf9cc1
tell mysql to ignore the sort index for search queries
mysql really likes to pick an index for sorting if it can't fully satisfy the where
filter with an index, since search queries pretty much never are fully filtered by index
mysql often picks an index for sorting instead of the *much* more useful index for filtering.

To bypass this, we tell mysql explicitly not to use the mtime (the default order field) index,
so it will instead pick an index that is actually useful.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-18 18:05:48 +07:00
Robin Appelman 7f272dd98f
allow specifying index hints for mysql search queries
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-18 18:03:34 +07:00
acsfer f57bbaf146 Simplify :) 2021-10-18 08:02:44 +07:00
acsfer aeb0ad8cea Replace `file_exists()` method by `is_file()` 2021-10-18 08:02:44 +07:00
acsfer 0c1ccde7d2 Get `filesize()` if `file_exists()`
Should make sense.
2021-10-18 08:02:44 +07:00
Daniel 1ea652d611
Merge pull request #29159 from nextcloud/backport/24185/stable21
[stable21] Properly handle folder deletion on external s3 storage
2021-10-15 11:36:56 +07:00
Vincent Petry 5735a55509
Merge pull request #29080 from nextcloud/path-prefix-filter-21
[21] generate a better optimized query for path prefix search filters
2021-10-13 08:37:28 +07:00
Robin Appelman 916a838873
[21] generate a better optimized query for path prefix search filters
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-12 15:55:57 +07:00
Julius Härtl df3ed040e4 Make sure that a empty directory can still be deleted when copied from another storage
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-10-11 08:01:08 +07:00
Daniel Kesselberg b77b1d6bbd
explicitly close source stream on encryption storage
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2021-10-08 14:30:27 +07:00
Daniel Kesselberg 98d3836650
explicitly close source stream on local storage
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2021-10-08 14:30:27 +07:00
Vincent Petry 9722bb5c54
Fix psalm issue in Encryption stream
Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-10-08 12:15:04 +07:00
Robin Appelman 4180ca7a0b handle case where storage can't be created in getStorageRootId
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-09-30 13:21:40 +07:00
MichaIng ea39319d2e
Merge pull request #28905 from nextcloud/backport/26832/stable21
[stable21] explicitly close source stream on object store upload even if count…
2021-09-24 13:35:41 +07:00
Pauli Järvinen 61acaf1fbd Support seeking also from the end of file on S3 storage
The PR https://github.com/nextcloud/server/pull/20033 added support
for `fseek` for  the S3 storage backend. However, the seek mode SEEK_END
was left out that time. This PR fills this gap.

Signed-off-by: Pauli Järvinen <pauli.jarvinen@gmail.com>
2021-09-20 18:13:18 +07:00
Daniel Kesselberg dd9a08fa82
explicitly close source stream on object store upload even if countwrapper isn't needed
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2021-09-20 18:27:12 +07:00
Jasper Knockaert bb092dd7cc fix even more brackets
Signed-off-by: Jasper Knockaert jasper@knockaert.nl
2021-09-03 13:40:42 +07:00
Jasper Knockaert c1dcd06fe1 fix brakcets 2021-09-03 13:40:42 +07:00
Jasper Knockaert 5f9663d2c8 consolidation of boolean expression 2021-09-03 13:40:41 +07:00
Jasper Knockaert 82482adb48 avoid fread on directories and unencrypted files
Reworking the logic in order to first check the filecache and only then reading the fileheader.
This in order to solve #21578.
2021-09-03 13:40:40 +07:00
Vincent Petry 2032ca7457
Unregister enc stream wrapper for any exception
This prevents side effects in tests by properly cleaning up
even with expected exceptions.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-08-26 14:20:04 +07:00
Robin Appelman ca3a6251d2
better cleanup of user files on user deletion
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-08-12 18:16:51 +07:00