02f4a82088
fix: make bucket mapper work with new multi-object-store config
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-28 17:51:25 +07:00
b3c53c7436
feat: allow object store configuration aliases for easier migrations
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-14 13:06:51 +07:00
2d4bba7b0c
feat: add command to get user objectstore config mappings
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-14 13:06:51 +07:00
385dd36ff8
feat: allow configuring multiple objectstore configurations
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-14 13:06:51 +07:00
97efc95efc
fix: better object store write error propagation
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-05 19:17:58 +07:00
83b8a390cd
fix: always do stream counting for object store upload
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-05 19:17:58 +07:00
398b106f0c
fix: validate written size for s3 multipart uploads
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-05 19:17:58 +07:00
5981b7eb51
chore: apply new CSFixer rules
...
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
# Conflicts:
# apps/settings/lib/SetupChecks/PhpOpcacheSetup.php
2025-07-01 16:26:50 +07:00
2e1c316582
fix: remove double cache write in ObjectStoreStorage::touch
...
the file_put_contents already puts all of those values into the cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-06-18 22:22:51 +07:00
ea98e4b3f5
Merge pull request #53419 from nextcloud/leftybournes/fix/object_storage_large_uploads
...
fix(s3): retry failed multipart uploads with decreased concurrency
2025-06-18 21:07:06 +07:00
d7ae952dc8
fix(ObjectStore): Make S3 "connect_timeout" option configurable
...
The hardcoded connection timeout of 5 seconds may not be enough in some
cases, so now it is got from the ObjectStore arguments in Nextcloud
configuration, falling back to 5 if not set.
The connection timeout is set in seconds, but decimal precision can be
used for subsecond accuracy (for example, 4.2 for 4200 milliseconds).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2025-06-16 20:17:50 +07:00
06b5ce5859
fix(s3): retry failed multipart uploads with decreased concurrency
...
Signed-off-by: Kent Delante <kent.delante@proton.me>
2025-06-10 17:39:59 +07:00
f4cb78b905
Merge pull request #52242 from nextcloud/artonge/fix/copy_subfolders_s3
2025-06-02 11:55:54 +07:00
5b3f4cd8fe
perf: set the folder size in the same query as we create it
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-05-28 15:52:00 +07:00
1f8b823992
fix(S3): Use original folder size during copy
...
This prevent having copied folders with a wrongly set size of 0KB.
- Fix https://github.com/nextcloud/server/issues/51916
Signed-off-by: Louis Chemineau <louis@chmn.me>
2025-05-28 15:52:00 +07:00
7599162c7b
feat: move primary object store configuration to a single place
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-05-28 15:32:31 +07:00
9592390070
Merge pull request #51779 from nextcloud/object-store-filename
2025-05-06 15:21:01 +07:00
49baa914cf
fix(objectstorage): add retry attempts to S3 connection
...
Signed-off-by: Anna Larch <anna@nextcloud.com>
2025-04-23 12:15:06 +07:00
8fdf2a7eae
fix: Transfer ownership with S3 as primary
...
When using S3 as primary storage, transferring ownership with the `--move` option fail with the following error:
`SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8-45b963397aa40d4a0063e0d85e4fe7a1' for key 'fs_storage_path_hash'`
The `--move` option moves the entire home folder from one account to another.
The error means that the move failed because the destination folder already exist in `oc_filecache`.
- With S3 as primary storage, folders only exists as entries in `oc_filecache`.
- With S3 as primary storage, `moveFromStorage(...)` only moves the cache entry, as nothing needs to be moved on disk. This cache move does not delete potentially pre-existing destination folder.
- With Local storage, `moveFromStorage(...)` calls `rename(...)` which delete pre-existing folder.
- `transfer(...)`: 687a4d9ac7/apps/files/lib/Service/OwnershipTransferService.php (L112)687a4d9ac7/lib/private/Files/SetupManager.php (L261-L262)687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L91-L135)687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L635-L636)
2025-04-01 13:53:56 +07:00
8aae332622
feat: store original storage id and path in object store metadata
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-03-31 15:10:58 +07:00
94114b99f7
feat: more generic way of passing metadata to object storage backends for new objects
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-03-31 14:55:18 +07:00
fcde776683
feat: add command to list objects
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-03-31 14:30:40 +07:00
c3bc362f48
feat: add command to get object metadata
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-03-31 14:30:40 +07:00
12a92a2e8b
Merge branch 'master' into feat/support-aws-session-token
2025-03-04 18:27:54 +07:00
9bdee76535
perf(files): faster query to fetch incomplete directories
...
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2025-02-13 10:11:02 +07:00
6b4c859a41
feat(object_store): Add support for session token in AWS credentials
...
- Pass session token, either null or with value, to the AWS Credentials constructor
Signed-off-by: Hector Valcarcel <hmvalcarcel@gmail.com>
2025-02-06 16:24:19 +07:00
f1c025dfd3
fix(s3): treat empty sse_c_key as not set
...
Co-authored-by: Christian Becker <christian@dabecka.de>
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2025-01-28 22:59:59 +07:00
b48ee2e924
fix: Harden files scanner for invalid null access
...
Co-authored-by: Ferdinand Thiessen <opensource@fthiessen.de>
Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2025-01-28 20:08:46 +07:00
5896f8c37f
chore: cleanup leftover debug statement
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-09 15:40:01 +07:00
6cf66f95ce
fix: get object size from stream where possible
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-05 15:16:22 +07:00
ca547bb913
fix: don't perform the extra buffering in s3 stream write when the stream size is known
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-05 14:07:40 +07:00
7da8951c57
fix: use php://temp instead of php://memory for multi-part upload buffer
...
this should reduce potential memory issues if the limit is set very high
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-05 14:07:37 +07:00
df3a9e221d
fix: throw correct exception type when we can't verify if an s3 bucket exists
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-11-26 16:49:55 +07:00
0de4843b73
refactor(Storage): Align all Storage constructors
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-23 13:24:18 +07:00
f28e74b7a8
refactor(Storage): Make all parameter types strong types
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-07 15:00:05 +07:00
3e12e1e789
fix: rework move into object store to better preserve fileids
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-27 14:46:08 +07:00
888d06dff9
fix: preserve fileid when moving from objectstore to non-objectstore
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-27 14:45:42 +07:00
da21acfb3f
fix: ensure source folder is removed from cache when moving to objectstore
...
otherwise this causes confusion down the line as it's contents will be moved to the new cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-27 14:45:42 +07:00
7cdccd058f
fix(Storage): Fix IStorage return types
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-26 18:13:03 +07:00
47d281865e
perf: reuse cache info we already have when moving to object store
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-19 19:06:06 +07:00
9836e9b164
chore(deps): Update nextcloud/coding-standard to v1.3.1
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-19 14:21:20 +07:00
3c0c81bb94
fix: ensure parent folder exists when writing a file to object storage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 14:11:27 +07:00
e0fb25f846
fix: fix object store id for test object store
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
60f6347c97
fix: cleanup objectstore file_put_content
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
3e94ee391f
fix: create intermediate directories for objectstore moveFromStorage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
879fcc4253
fix: get source file before moving the cache item in objectstore moveFromStorage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
f25acfb799
Merge pull request #47986 from cfiehe/fix_move_on_same_bucket
...
perf(ObjectStoreStorage): Improve (slow) move on same object bucket
2024-09-17 16:35:19 +07:00
d2d1e3284d
fix(Storage): Fix method docs inheritance
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-17 10:20:38 +07:00
8ca6fcace7
fix(Storage): Document getOwner() can return false
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-17 10:10:50 +07:00
492e6997d8
chore: Fix psalm issues, put back private versions of getter in private Storage interface
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-09-16 18:11:38 +07:00
Robin Appelman
Ferdinand Thiessen
Kent Delante
Daniel Calviño Sánchez
Louis
Kate
Anna Larch
Hector Manuel
Benjamin Gaussorgues
Hector Valcarcel
Richard Steinmetz
provokateurin
Stephan Orbaugh
Côme Chilliet