Commit Graph

1406 Commits (feat/database/query-result-fetch-associative-fetch-num)

Author SHA1 Message Date
Côme Chilliet 966a3e6963
Tidy up typing in OC\Files\View
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-04-03 10:52:33 +07:00
jld3103 dcf6ebebab
Fix inherited parameter names
Signed-off-by: jld3103 <jld3103yt@gmail.com>
2023-04-03 08:29:33 +07:00
Robin Appelman c9e4e30523 clear encrypted flag when moving away from encrypted storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-01 15:36:37 +07:00
Rsplwe d85e623916
s3: get remaining truncation part
Signed-off-by: Rsplwe <i@rsplwe.com>
2023-04-01 22:56:15 +07:00
Rsplwe ccac296821
Adjust the value of 'maxparts' to 1000
Signed-off-by: Rsplwe <i@rsplwe.com>
2023-04-01 21:20:52 +07:00
Côme Chilliet 8104d9f5d8
Remove useless coalescing operator on non-null return value
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-04-01 14:20:26 +07:00
Côme Chilliet da618c457a
Fix quota typing for 32bits after forward port from 25
Signed-off-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
2023-04-01 14:20:25 +07:00
Côme Chilliet 3f84d41226
Silence a warning from psalm with explanation
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-04-01 14:20:25 +07:00
Côme Chilliet 1740311283
Adapt types of subclasses
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-04-01 14:20:24 +07:00
Côme Chilliet ed863fb47c
Make it clear that file sizes may be float on 32bits
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-04-01 14:20:24 +07:00
Vincent Petry c6b683342e
Remove quota value type for 32-bit systems
Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2023-04-01 14:20:20 +07:00
Robin Appelman 36835b7fec
Merge pull request #36810 from nextcloud/setup-user-onetime-before-mounts
do onetime user setup before getting any mount from providers
2023-03-24 16:45:08 +07:00
Côme Chilliet c60182065f
Merge pull request #34773 from nextcloud/artonge/feat/dispatch_entry_removed_event_for_all_entries
Dispatch event for all removed entries
2023-03-21 11:42:18 +07:00
Côme Chilliet 9c3e06c046
Merge pull request #33061 from ibpl/master-IB#1121253
Paths added to messages and message cleanup
2023-03-14 17:28:22 +07:00
Arthur Schiwon 315510df84
Merge pull request #27034 from nextcloud/enh/s3-multipart-upload-api
Use MultipartUpload for uploading chunks to s3
2023-03-09 13:55:57 +07:00
Arthur Schiwon 61aa0978ae
Merge pull request #36609 from nextcloud/lazyuserfolder-mountpoint
optimize LazyUserFolder::getMountPoint
2023-03-09 09:46:34 +07:00
Julius Härtl e23aa8883e
feat(s3): Use multipart upload for chunked uploading
This allows to stream file chunks directly to S3 during upload.

Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-03-08 14:00:04 +07:00
Joas Schilling 6b0cef6b9a Add transaction around mass mounts operations
Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-03-07 19:58:55 +07:00
Git'Fellow 28231ae515
Add function_exists() check
Co-authored-by: Simon L. <szaimen@e.mail.de>
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
2023-03-07 17:09:24 +07:00
Git'Fellow 9ac065fb92
Fix free space problems
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
2023-03-07 00:13:57 +07:00
Julius Härtl d515da502f fix: Use proper path for quota fetching
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-03-06 22:46:07 +07:00
Robin Appelman fb1095f0f6
do onetime user setup before getting any mount from providers
this fixes a race condition where wrappers weren't properly applied if when a mount provider creates the storage instance one mountpoint creation instead of lazily

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-22 18:03:38 +07:00
Robin Appelman d6818ba894
optimize LazyUserFolder::getMountPoint
no need to do a full setup

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-13 15:13:50 +07:00
Robin Appelman 936f634069
Merge pull request #36643 from nextcloud/fs-setup-instrimuntion
more filesystem setup performance instrumentation
2023-02-13 14:37:57 +07:00
Robin Appelman 91d5f6a826 more filesystem setup performance instrumentation
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-10 11:11:13 +07:00
Julius Härtl d9cd8b1d37
Merge pull request #36610 from nextcloud/folder-lazy-submounts 2023-02-10 09:09:08 +07:00
Robin Appelman 5bcf37b7ff only fetch the data for mounts inside a folder when needed
for most operations we don't actually care about any mounts inside a folder, only for metadata that needs to propagate across storage boundaries (size, etag, mtime) do we need all the submount info.

By only loading this data when needed we can save a bunch of storage setup in a number of cases

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-09 11:39:00 +07:00
Julius Härtl 08802ad13f
Merge pull request #36566 from nextcloud/mount-find-by-numeric-id 2023-02-08 20:04:57 +07:00
Robin Appelman 9f3dbb699a fix tests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-07 15:50:10 +07:00
Côme Chilliet 308fcf4526
Respect OCP interface in private classes
Because the parameter type was moved to phpdoc it needs to be removed
 from implementations

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:35 +07:00
Côme Chilliet 028c225d96
Fix psalm errors
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:32 +07:00
Côme Chilliet cd885b5705
Type sizes as int|float throughout the code base
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:31 +07:00
Côme Chilliet 5c1a0e0307
Allow float sizes in FileInfo for 32bits
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:31 +07:00
Côme Chilliet d9dbed9105
Fix psalm errors related to filesizes
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:29 +07:00
Côme Chilliet ff776a90b1
Strong type filesize related methods to ease 32bits problem findings
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:28 +07:00
Côme Chilliet e156f8339c
Revert "remove 32-bit workarounds"
This reverts commit dd8774389e.

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:28 +07:00
Robin Appelman e3bafcc7a8 cache numeric id in mountpoint
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-06 18:20:08 +07:00
Robin Appelman 9281359fa4 deduplicate getStorage() logic in Mount\Manager
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-06 18:17:40 +07:00
Robin Appelman 43589312cc dont delegate Mount\Manager::getByNumericId to getByStorageId
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-06 18:16:23 +07:00
Joas Schilling 4bdc5834fa
Make sure name and path are strings
Otherwise Oracle returns NULL for empty strings and PHP 8.2
throws on null in string functions like trim() and md5()

Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-02-02 12:05:22 +07:00
Joas Schilling c27c9fac98
Try fixing Oracle
Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-02-02 12:05:21 +07:00
Louis Chemineau 2830eeac7e Dispatch event for all remove entry
Signed-off-by: Louis Chemineau <louis@chmn.me>
2023-01-31 13:30:10 +07:00
Richard Steinmetz f0a3b77d0f fix(encryption): don't throw on missing file
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2023-01-30 10:07:01 +07:00
Vincent Petry 9ecec2fd75
Merge pull request #35894 from nextcloud/move-from-encryption
fix moving files of encrypted local storage to unencrypted local storage
2023-01-26 09:45:12 +07:00
Julius Härtl 919a840f34
Merge pull request #32798 from nextcloud/enh/sse-c
[S3] Add option to specify an SSE-C customer provided key
2023-01-25 15:59:26 +07:00
Robin Appelman 33b90d28f8 fix moving files of encrypted local storage to unencrypted local storage
for example when moving encrypted files to a groupfolder

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-01-24 16:40:41 +07:00
Thomas Citharel 00d9d49303 Add app config for default template path
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2023-01-24 09:39:31 +07:00
Julius Härtl 159a0c8411
feat(s3): Add option to specify an SSE-C customer provided key
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-01-24 10:36:03 +07:00
Côme Chilliet f5c361cf44
composer run cs:fix
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-20 11:45:08 +07:00
Côme Chilliet 5e090d044d
Merge pull request #36075 from fmenabe/s3-storage-class
Add support for s3 storage classes
2023-01-16 09:43:53 +07:00
François Ménabé cf9bdc0274 Manage s3 storage class in objectstore
Signed-off-by: François Ménabé <francois.menabe@gmail.com>
2023-01-12 14:58:27 +07:00
Git'Fellow e855298b3c Switch logical operators
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
2023-01-11 16:27:33 +07:00
Robin Appelman ac5cfd10e2
extend path-prefix optimizer to remove all cases of path_hash= when encountering a path prefix filter
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-01-11 16:38:41 +07:00
Côme Chilliet a7740c0ae6
Suppress last psalm error until upstream fix is released and included
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-02 15:17:38 +07:00
Côme Chilliet a372564850
Fix psalm spotted errors with new requirements
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-02 15:02:28 +07:00
Vincent Petry 3bcbc4701a
Merge pull request #34724 from nextcloud/encryption-read-before-cache
fix reading newly written encrypted files before their cache entry is written
2022-12-20 17:16:12 +07:00
Vincent Petry 3b7cc31f64
Merge pull request #34924 from nextcloud/cache-jail-path-prefix
escape path prefix when doing cache jail search
2022-12-16 17:27:15 +07:00
Julius Härtl 59ae6e9dd6
Also update the unencrypted size of files when creating them with empty content
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-12-07 19:12:06 +07:00
Julius Härtl 77edd36583
Update the unencrypted file size when closing streams
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-12-07 16:26:31 +07:00
Simon L 2e0d262864
Merge pull request #35576 from nextcloud/handle-possible-null-value-in-new-simple-file
Handle possible null value for file in OC\Files\SimpleFS\NewSimpleFile
2022-12-06 19:52:55 +07:00
Thomas Citharel a5b6119ec4
Handle possible null value for file in OC\Files\SimpleFS\NewSimpleFile
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2022-12-03 17:25:32 +07:00
Maxence Lange cc639a8877 compare cached filesize on download
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
2022-12-01 15:42:15 +07:00
Robin Appelman e3a0e64698
clear is-encrypted cache when trying to fix encrypted version
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-12-01 14:36:27 +07:00
Robin Appelman e9b87c73fa
fix reading newly written encrypted files before their cache entry is written
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-30 14:54:56 +07:00
Andy Xheli e69c7c4d26
Fix Executing a query: SQLSTATE[42000]
Signed-off-by: Andy Xheli <axheli@axtsolutions.com>

Fix https://github.com/nextcloud/server/issues/32007#issuecomment-1329405245

Signed-off-by: Andy Xheli <axheli@axtsolutions.com>
2022-11-29 11:24:56 +07:00
Louis Chemineau 66ba90ceff Update cache when file size === 0
The conditions were false when $result === 0.
$results here contains the number of written bits.
The correct way of checking for operation success is to check if $result === false

Signed-off-by: Louis Chemineau <louis@chmn.me>
2022-11-23 12:43:57 +07:00
Robin Appelman 841a6a084e
delay updating setup providers untill we register the mounts
otherwise the fallback to a full setup for a missing cached mount provider will lead to a race condition

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-17 15:21:57 +07:00
Simon L 9d45845d2a
Merge pull request #34994 from Glandos/fileinfo_perf
[Performance] Test valid path only if file info has to be retrieved
2022-11-11 17:11:21 +07:00
Côme Chilliet a78595a45f
Make sure that path is normalized and then checked,
and not the other way around

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-11-10 15:03:15 +07:00
Glandos 48ca33761f Test valid path only if file info has to be retrieved
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2022-11-06 19:23:53 +07:00
Pawel Boguslawski 4a46c747e9 Merge master 2022-11-04 13:34:21 +07:00
Robin Appelman f76b4473ee
escape path prefix when doing cache jail search
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-02 12:44:51 +07:00
szaimen 3f7c620f26 fix psalm
Signed-off-by: szaimen <szaimen@e.mail.de>
2022-11-02 11:13:34 +07:00
szaimen dd8774389e remove 32-bit workarounds
Signed-off-by: szaimen <szaimen@e.mail.de>
2022-11-02 11:13:34 +07:00
Robin Appelman a5036a973c allow disabling object store write check
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-10-26 18:45:59 +07:00
Vincent Petry 4aa8e80511
Merge pull request #34579 from nextcloud/unencrypted-size-null
fix null error in getUnencryptedSize
2022-10-21 16:53:43 +07:00
Carl Schwan 0f0f132959
Merge pull request #34624 from nextcloud/fix/files_external_psalm
Multiple fix for files external
2022-10-20 16:53:43 +07:00
Carl Schwan a5ea677370 Rename file1 and file2 to source and target in Storage abstraction
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 17:21:47 +07:00
Carl Schwan 54031e370a Fix various other small psalm issues
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-17 12:40:35 +07:00
John Molakvoæ e4ff84d468
Scope the appdata theming storage for global and users
Signed-off-by: John Molakvoæ <skjnldsv@protonmail.com>
2022-10-14 16:18:37 +07:00
Robin Appelman 63316fc572
fix null error in getUnencryptedSize
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-10-13 15:34:48 +07:00
Julius Härtl 214a50d071
Avoid allocating too much memory for the buffer
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-10-11 15:34:27 +07:00
raul 95bc770a37 Fix unencrypted size calculation for files created before the Encryption storage is enabled
Signed-off-by: raul <raul@nextcloud.com>
2022-10-03 13:44:14 +07:00
raul 45a1ebcc4c Fix: Prevent deadlocks during mtime/size/etag propagation
Signed-off-by: raul <raul@nextcloud.com>
2022-10-03 13:44:14 +07:00
blizzz ca958de7e9
Merge pull request #33566 from nextcloud/fopen-not-found-rescan
trigger a rescan when trying to fopen a file that exists in cache but not on disk
2022-10-01 21:47:26 +07:00
blizzz f323c5b854
Merge pull request #34269 from nextcloud/td/noid/remove-unused-listener
remove listeners to OC_Filesystem::(write|rename) old style hooks
2022-10-01 16:15:39 +07:00
Carl Schwan 92a5a8f075 Cleanup tags and Share component
- Port to LoggerInterface
- Use IDBConnection and IQueryBuilder instead of raw SQL and OC_DB
- Use IEventListener instead of hooks
- Remove the now unused OC_DB and OC_DB_StatementWrapper legacy utils

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-09-29 15:15:28 +07:00
Robin Appelman cb51564356 trigger a rescan when trying to fopen a file that exists in cache but not on disk
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-09-29 11:17:16 +07:00
Arthur Schiwon 9b7ef2962e
remove listeners to OC_Filesystem::(write|rename) old style hooks
- the events are not emitted anymore
- OC_Filesystem::isBlacklisted() is not called from anywhere else

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2022-09-26 21:07:08 +07:00
Carl Schwan 3950deb42f
Merge pull request #32740 from jasperweyne/patch-2
Fix open_basedir issues by AWS SDK
2022-09-19 10:19:15 +07:00
Vincent Petry 1025d049c7
Merge pull request #24596 from kofemann/dcache-worm
make NextCloud WORM file system friendly
2022-09-16 14:39:13 +07:00
Vincent Petry b3ca186d2d
Merge pull request #34020 from nextcloud/logical-op
Switch logical operators (and|or)
2022-09-16 13:35:19 +07:00
Tigran Mkrtchyan f41209a061 config: add a switch to control truncate before update
To avoid extra truncate on non WORM file systems, add a new config
option `localstorage.unlink_on_truncate`, which defaults to false.

The OC\Files\Storage\Local is update to respect that option.

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2022-09-16 11:10:32 +07:00
Tigran Mkrtchyan 8fc4cf67f1 files: remove destination file before copying new content (WORM)
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2022-09-16 10:57:17 +07:00
Robin Appelman d8961ed10f
fix using FSEEK_END with SeekableHttpStream to get file size
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-09-15 17:23:16 +07:00
Git'Fellow 5300f0d693
Switch logical operators (and|or)
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
2022-09-12 09:35:12 +07:00
Carl Schwan eac54c4660
Merge pull request #33788 from nextcloud/improve-recent
Improve getting recent files performance
2022-09-08 16:19:41 +07:00
blizzz 417f46e467
Merge pull request #33600 from nextcloud/lazy-quota
get quota for user only when needed
2022-09-07 18:10:33 +07:00
Carl Schwan 67551f379f
Filter out old files when trying to get recent files
Only do so when asking for less than 100 files and having an offset
equal to 0.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-09-05 12:53:32 +07:00
Robin Appelman 9f5f970957
handle errors from hash_final
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-09-02 12:27:13 +07:00
Robin Appelman bd91c56539
get quota for user only when needed
always apply the wrapper but have the wrapper get the quota lazily only when an operation where quota is applied is performed

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-09-01 19:32:24 +07:00
Robin Appelman a9575a7029
handle cases where the hash context gets cleaned up before the hash wrapper
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-31 15:05:10 +07:00
Robin Appelman e9ba58ca61
remove leading slash for search results at mountpoint root
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-29 17:33:43 +07:00
Tigran Mkrtchyan b6065a236f files: make OC\Files\Storage\Local WORM friendly
Some filesystems run as a Write-Once-Read-Many storages. This
makes them impossible to use with NexeCloud, as the file system
layers uses `truncate` syscall (through file_put_contents function).

As Nextcloud is never updates existing files, removing the old entry
and creatint a new one on update will allow NextCoud to update on such
file systems.

Update Local#fopen and Local#file_put_contents to remote existing
file before truncating.

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2022-08-25 11:11:49 +07:00
Julius Härtl 83b1415906
Only pass parent if paths match
As the user folder might be initialized by the root from two levels
down the hierarchy, passing this as a parent only works if the path matches

Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-08-24 22:20:32 +07:00
Julius Härtl b2d6435737
Avoid extra query for the parent node if we can pass it along
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-08-24 22:20:32 +07:00
Robin Appelman b6f8b8da60 directly build the search filter for shared storage instead of setting up the source cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-17 11:58:57 +07:00
Vincent Petry 312b719acf
Merge pull request #32943 from nextcloud/unencrypted-size-revert-interface-changes
Revert interface changes from "store unencrypted size in the unencrypted_size column"
2022-08-17 11:36:16 +07:00
Vincent Petry 11d5a1d3df
Merge pull request #33550 from nextcloud/jail-search-post
optimize search post-processing for jail wrapper
2022-08-17 10:55:41 +07:00
Vincent Petry 2360d880fa
Merge pull request #33540 from nextcloud/mount-provider-migration
fix updating cached mounts that didn't have their mount provider set previously
2022-08-17 10:41:31 +07:00
Vincent Petry 082432e01c
Merge pull request #33551 from nextcloud/scanner-dont-update-same-size
don't update the folder size if we know it hasn't changed
2022-08-17 09:21:30 +07:00
Julius Härtl 999dced07c
Merge pull request #33559 from yeyulantu/bugfix-search-cache 2022-08-16 23:54:59 +07:00
Robin Appelman 457822c144 don't update the folder size if we know it hasn't changed
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:56:51 +07:00
Robin Appelman de63f6363f fix updating size when folder is empty
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:54:26 +07:00
Robin Appelman 1374cbee3e store unencrypted size in the unencrypted_size column
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:54:26 +07:00
Robin Appelman 10767d7779 fix updating cached mounts that didn't have their mount provider set previously
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:47:41 +07:00
Robin Appelman 9d6c2ce301 optimize search post-processing for jail wrapper
don't both asking the wrapped cache if we know it's not in our jail anyway

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:36:52 +07:00
Robin Appelman 5e375d9092 Revert "store unencrypted size in the unencrypted_size column"
This reverts commit 8238582e59.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 12:07:14 +07:00
David fe3f4afd77
fixed the cache searchBuilder negative map
Signed-off-by: David <37280718+yeyulantu@users.noreply.github.com>
2022-08-16 17:59:02 +07:00
Carl Schwan 51b9847fad
Merge branch 'master' into display-name-cache-public
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-08-15 15:28:30 +07:00
Jasper Weyne 44f6c931e7
Merge branch 'master' into patch-2 2022-08-11 08:54:08 +07:00
Côme Chilliet 7ae6791737
Document all getIncomplete implementations as returning string|false
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-08-02 14:23:14 +07:00
Côme Chilliet cb271b759e
Fix dynamic property creations in test files
This fixes warnings in PHP 8.2

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-08-01 15:07:53 +07:00
Paweł Bogusławski a850424b1b Update Folder.php
Signed-off-by: Pawel Boguslawski <pawel.boguslawski@ib.pl>
2022-07-29 13:34:00 +07:00
Paweł Bogusławski 097b78594a Update Folder.php
Signed-off-by: Pawel Boguslawski <pawel.boguslawski@ib.pl>
2022-07-29 13:34:00 +07:00
Paweł Bogusławski 04baf69fea Update lib/private/Files/Node/Folder.php
Co-authored-by: Vincent Petry <vincent@nextcloud.com>
Signed-off-by: Pawel Boguslawski <pawel.boguslawski@ib.pl>
2022-07-29 13:34:00 +07:00
Pawel Boguslawski 9eb1a92fa4 Paths added to messages and message cleanup
This mod adds path to error messages. Without it it's difficult for example
to find what path causes permission problems.

Related: https://help.nextcloud.com/t/ocp-files-notpermittedexception-could-not-create-folder/133818
Related: https://help.nextcloud.com/t/could-not-create-folder-false-error/65962/
Author-Change-Id: IB#1121253
Signed-off-by: Pawel Boguslawski <pawel.boguslawski@ib.pl>
2022-07-29 13:34:00 +07:00
Côme Chilliet 472f4cad47
Merge pull request #33380 from luzpaz/fix/lib-private-typos
Fix typos in lib/private subdirectory
2022-07-28 09:17:48 +07:00
luz paz 368f83095d Fix typos in lib/private subdirectory
Found via `codespell -q 3 -S l10n -L jus ./lib/private`

Signed-off-by: luz paz <luzpaz@github.com>
2022-07-27 08:52:17 +07:00
Robin Appelman 61fdd91dcf
don't set `null` as a bundle path
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-07-27 12:58:07 +07:00
Robin Appelman 13eef55a92
always triger setup of builtin storage wrappers
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-07-21 16:36:32 +07:00
Carl Schwan 6c3c7a5e4d
Merge pull request #32981 from nextcloud/typing/carl/simplefs
Add typing to SimpleFS
2022-07-18 12:34:11 +07:00
Carl Schwan f6b37e6912
Merge pull request #33064 from nextcloud/feat/capped-memorycache-ocp
Move CappedMemoryCache to OCP
2022-07-18 10:32:11 +07:00
Daniel 52dc51cde3
Merge pull request #32963 from nextcloud/s3-crt-bundle-download
also use nextcloud certificate bundle when downloading from s3
2022-07-18 09:51:19 +07:00
Carl Schwan 19a36b58a6 Add typing to SimpleFS
- Fix putContent sometimes return a bool and sometimes nothing

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-07-14 17:20:51 +07:00
Carl Schwan d5c23dbb9f Move CappedMemoryCache to OCP
This is an helpful helper that should be used in more place than just
server and this is already the case with groupfodlers, deck, user_oidc
and more using it, so let's make it public

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-07-14 15:54:31 +07:00
Jonas 74e9ef0fb1
Fix listening for circle events in SetupManager
So far, SetupManager listened for deprecated events that are no longer
triggered. Instead, use the circle events that actually get triggered
when adding or removing a circle or circle member. Also, these events
get triggered on each instance of a globalscale setup.

Fixes: #33210

Signed-off-by: Jonas <jonas@freesources.org>
2022-07-12 13:28:21 +07:00
Robin Appelman f1486890d7
only use nextcloud bundle when explicitly enabled
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-07-05 16:06:55 +07:00
Robin Appelman bffa67c48b
also use nextcloud certificate bundle when downloading from s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-07-05 15:57:54 +07:00
Jonas 7d07e06bfe Check whether entry is of type ICacheEntry in Cache->remove()
In some scenarios (file not in cache, but partial data of it in the
object), Cache->get() might return an array, which leads to errors like
"Call to a member function getId() on array".

So check whether the returned entry is of type ICacheEntry before doing
operations on it in Cache->remove().

Fixes: #33023

Signed-off-by: Jonas <jonas@freesources.org>
2022-06-29 10:47:07 +07:00
Carl Schwan 46a49dee8a
Merge pull request #32994 from nextcloud/fix/extended-query-search
Search without join on filecache_extended
2022-06-29 11:58:20 +07:00
Carl Schwan a1bc770043
Merge pull request #32982 from nextcloud/fix/metadata-extraction
Fix metadata extraction
2022-06-28 15:34:38 +07:00
Côme Chilliet 64bff27c99
Merge pull request #32960 from nextcloud/fix/avoid-using-undeclared-properties
Fix PHP 8.2 warnings about undeclared properties
2022-06-28 12:05:16 +07:00
Carl Schwan f326b54e53 Search without join on filecache_extended
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-06-23 16:23:52 +07:00
Carl Schwan 940b5e8f0a Fix metadata extraction
The metadata extraction only happens when the size is not equal to 0,
but due to a regression in FileInfo the size is always zero.

This fix the regression.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-06-22 17:27:09 +07:00
blizzz 2e10378052
Merge pull request #31475 from nextcloud/kofemann-patch-1
files: close open directory file descriptor on error path
2022-06-21 18:44:05 +07:00
Côme Chilliet 1bd5222224
Fix PHP 8.2 warnings about undeclared properties
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-06-21 16:17:52 +07:00
Vincent Petry 8809de1eef
Merge pull request #31966 from nextcloud/unencrypted-size
store unencrypted size in the unencrypted_size column
2022-06-13 11:55:37 +07:00
Vincent Petry bea755f153
Merge pull request #32723 from MartinBrugnara/master
Expose umask override value as config parameter: localstorage.umask
2022-06-10 11:23:31 +07:00
Vincent Petry 56b1453945
Merge pull request #32768 from nextcloud/fs-setup-onetime-earlier
perform onetime setup earlier to ensure wrappers are registered on time
2022-06-10 11:21:54 +07:00
Robin Appelman 5c768f980e
fix mounts mounted at the users home
this fixes external storages with '/' as mountpoint

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-06-09 19:21:56 +07:00
Robin Appelman a937ab03dd
perform onetime setup earlier to ensure wrappers are registered on time
this fixes an issue with wrappers like encryption not always being applied to mountpoint that create the storage object directly (such as external storage)

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-06-08 18:49:54 +07:00
Robin Appelman 499995b37a
handle stream wrappers in SeekableHttpStream
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-06-08 17:52:27 +07:00
Jasper Weyne 0633a1d9f5
Disable ~/.aws/config access by S3ConnectionTrait
This commit sets the 'use_aws_shared_config_files' option to false, in order to disable configuration loading from ~/.aws/config by the AWS SDK, specifically the S3Client. It is a continuation of #27040, as that PR only changed the behaviour of the CredentialsProvider; this change affects the ConfigurationProvider as well.

Signed-off-by: Jasper Weyne <jasperweyne@gmail.com>
2022-06-07 13:39:14 +07:00
Martin Brugnara 7409569994
Expose umask override value as config parameter: localstorage.umask
Commit 451c06d introduced override for umask value.
This is needed to avoid broken env configuration or dirty workers
to mess with the permissions when creating new files.

Most Nextcloud, that does not integrate with external software
would work fine with an hard-coded value (451c06d set it at 022).

Advanced install may require more flexibility, as such this commit
exposes the "umask override value" as configuration parameter:
`localstorage.umask`

It defaults to 0022 both in code and in config/config.sample.php .

Signed-off-by: Martin Brugnara <martin@0x6d62.eu>
2022-06-05 19:19:31 +07:00
Robin Appelman 8238582e59
store unencrypted size in the unencrypted_size column
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-06-02 16:07:00 +07:00
Julius Härtl 0f0e74a78c Cleanup temporary files after finishing the write to object storage
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-06-02 12:57:18 +07:00
Daniel Kesselberg f399fd41ec
Fix docblock for return type
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2022-05-27 20:05:46 +07:00
Simon L 18dd460720
Merge pull request #26813 from nextcloud/feature/files-search-tags
Files: Extend search to also cover tags
2022-05-18 10:58:02 +07:00
Robin Appelman 6d6662ec68
expose displayname cache trough a public interface
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-05-18 03:47:34 +07:00
Marcel Klehr f755ee0868 Files: Extend search to also cover tags
fixes #326

Signed-off-by: Marcel Klehr <mklehr@gmx.net>
2022-05-17 21:05:09 +07:00
John Molakvoæ fe33e9c08c
Merge pull request #32250 from nextcloud/cleanup/cappedmemorycache 2022-05-13 08:27:28 +07:00
Carl Schwan ec6b83cc18 Add stricter psalm type for CappedMemoryCache
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-12 15:06:18 +07:00
Carl Schwan a392cd70fb Make it possible to get the appdata folder using the public API
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-11 23:28:37 +07:00
Vincent Petry fe24091ffb
Merge pull request #32110 from plumbeo/binary-encoding-4
Save encrypted files in binary format
2022-05-05 18:04:06 +07:00
Vincent Petry 4147e1233a
Merge pull request #32152 from nextcloud/bugfix/32151/allow-compatibility-wrapper-local-ext-storage
Allow compatibility wrapper on local external storage
2022-05-05 08:18:52 +07:00
plumbeo 1258caeab7 Save encrypted files in binary format
Default to the more space-efficient binary encoding for newly encrypted files
instead of the traditional base64 encoding, eliminating the 33% overhead.

The new option 'encryption.use_legacy_encoding' allows to force the legacy
encoding format if needed. Files encoded in the old format remain readable.

Based on https://github.com/owncloud/encryption/pull/224 and
https://github.com/owncloud/core/pull/38249 by karakayasemi.

Signed-off-by: plumbeo <plumbeo@users.noreply.github.com>
2022-05-04 17:38:25 +07:00
Côme Chilliet 459202d54c
Merge pull request #31752 from nextcloud/fix/remove-still-more-ilogger
Move away from deprecated ILogger
2022-05-02 10:15:10 +07:00
Carl Schwan dbdb9f6f11
Merge pull request #32034 from nextcloud/bugfix/noid/satisfy-binary-operation-order
Make the binary operation order explicit
2022-04-27 11:53:18 +07:00
Côme Chilliet 2b07031d3e
composer run cs:fix
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:53 +07:00
Côme Chilliet 23f7a37c03
Remove ILogger usages in lib/private/Files/Storage
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:53 +07:00
Côme Chilliet 069477e608
Migrate more classes of lib/private to LoggerInterface
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:52 +07:00
Vincent Petry 23403ef693
Allow compatibility wrapper on local external storage
The check was likely in place to prevent adding the wrapper on the root
and home storage, which is not possible anyway since the encoding option cannot
be set on that mount.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2022-04-26 09:27:29 +07:00
Vincent Petry 6935baa568
Merge pull request #32084 from nextcloud/root-setup-groupfolders
only need to setup root for groupfolders
2022-04-25 19:35:59 +07:00
Robin Appelman f7413b9afc
Merge pull request #32074 from nextcloud/mysql-search-ignore-index-2
tell mysql to ignore the sort index for search queries
2022-04-25 11:23:51 +07:00
Vincent Petry ff385dc679
Merge pull request #32082 from nextcloud/directory-content-lazy-owner
use a lazy user for the file owner when listing a directory
2022-04-25 11:44:44 +07:00
Vincent Petry e693781c21
Merge pull request #32079 from nextcloud/directory-listing-optimize-add-submount
optimize adding submount info to directory content
2022-04-25 11:44:26 +07:00
Robin Appelman b36dfd8f11 Only need to setup root for groupfolders
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-22 17:47:08 +07:00
Julius Härtl fb59875071
Merge pull request #32040 from nextcloud/full-setup-only-new
only register mounts that are new from providers that are new during a full setup
2022-04-22 17:03:09 +07:00
Robin Appelman 7a6c724a81 Use a lazy user for the file owner when listing a directory
Only getUID and getDisplayName are called on the file owner objects anyway
and we can get this information often without DB request

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-22 17:02:37 +07:00
Robin Appelman 9f0ba3adc2
optimize adding submount info to directory content
no need to loop when we can do a hashtable lookup

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-22 15:50:14 +07:00
Robin Appelman a4e120c203
tell mysql to ignore the sort index for search queries
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-22 13:42:33 +07:00
Robin Appelman bbf283f818
cache fullSetupRequired locally
no need to hit redis every time

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-21 16:00:28 +07:00
Robin Appelman 561a2071e4
only register mounts that are new from providers that are new during a full setup
this fixes cases where during the (partial) setup of a shared mount a full setup is triggered

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-21 12:53:50 +07:00
Joas Schilling 49e62149d8
Make the binary operation order explicit
Signed-off-by: Joas Schilling <coding@schilljs.com>
2022-04-21 10:47:26 +07:00
Robin Appelman caebdc2a55
Merge pull request #32024 from nextcloud/local-metadata-forbidden
ignore forbidden files while scanning local storage
2022-04-20 12:34:23 +07:00
Robin Appelman 468d6a9cf1
ignore forbidden files while scanning local storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-20 13:01:16 +07:00
Arthur Schiwon 1ec4563c2e
log stacktrace for 'Backends provided no user object'
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2022-04-20 12:15:39 +07:00
Carl Schwan 7817845538 Add a metadata service to store file metadata
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-04-13 14:06:29 +07:00
Robin Appelman b65d214f15
perform a setup if we can't find any mounts containing a file
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-11 19:15:58 +07:00
Vincent Petry 1b819b8649
Merge pull request #31901 from nextcloud/fs-setup-tweaks
Filesystem setup tweaks
2022-04-08 17:47:36 +07:00
Vincent Petry 0d7d28e530
Merge pull request #31676 from nextcloud/enh/ocp-owner-lock
Add public API for owner based file locking
2022-04-08 17:43:26 +07:00
Robin Appelman 499749dbc1
more preset metadata for LazyUserFolder
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-08 15:09:40 +07:00
Robin Appelman 54e52e9b5e
always do full setup for home folder with submounts
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-08 15:05:43 +07:00
Robin Appelman bdc86e9486
add missing return
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-08 15:04:35 +07:00
Robin Appelman 9834350ac3
Merge pull request #31713 from nextcloud/fed-performance
Federated share performance improvements
2022-04-07 18:53:36 +07:00
Julius Härtl 9b408d556d
Adapt LockScope to LockContext rename
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-04-07 11:52:16 +07:00
Robin Appelman 5e69f98c16
sh
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-06 18:08:13 +07:00
Robin Appelman 151c800397
allow reusing known folder info when getting directory contents
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-06 14:40:34 +07:00
Robin Appelman 9b1abd6fac
save filesystem node in dav node
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-06 14:40:31 +07:00