e67c8e739e
feat(ObjectStore): Allow overriding arguments per bucket
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2025-12-01 16:44:41 +07:00
3140daa1aa
feat(CertificateManager): Add option to specify the default certificates bundle path
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2025-12-01 11:57:26 +07:00
8706dadcf0
Merge pull request #56616 from nextcloud/backport/56565/stable31
...
[stable31] feat: emit an event when an S3 bucket is created
2025-11-27 10:26:42 +07:00
bf4c094741
feat: emit an event when an S3 bucket is created
...
Signed-off-by: Kent Delante <kent.delante@proton.me>
2025-11-25 20:16:21 +07:00
d1a760d618
fix: ensure all object store configuration have distict bucket names
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-11-20 15:36:14 +07:00
6d487ac990
fix: make bucket mapper work with new multi-object-store config
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-11-20 15:36:13 +07:00
f21dcac9ef
feat: allow object store configuration aliases for easier migrations
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-11-20 15:36:11 +07:00
832c77ab52
feat: add command to get user objectstore config mappings
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-11-20 15:16:43 +07:00
ff74df3528
feat: allow configuring multiple objectstore configurations
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-11-20 15:16:39 +07:00
3cfc34ecfa
fix(S3): Only append streams if non-seekable
...
Later, when we rewind the stream in `writeMultiPart` during retry, both streams were rewinded, so the resulting stream was bigger than expected.
Inspired by https://github.com/aws/aws-sdk-php/blob/master/src/S3/ObjectUploader.php#L136-L146
Signed-off-by: Louis Chmn <louis@chmn.me>
2025-10-10 12:43:19 +07:00
6842d0746d
Merge pull request #55288 from nextcloud/backport/52816/stable31
...
[stable31] feat: move primary object store configuration to a single place
2025-09-30 10:35:42 +07:00
d615109126
fix: small typo fix
...
Signed-off-by: yemkareems <yemkareems@gmail.com>
2025-09-29 12:24:43 +07:00
7162c3e8ed
feat: move primary object store configuration to a single place
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-09-24 18:39:21 +07:00
05e64418a7
fix: better object store write error propagation
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-07 13:48:43 +07:00
d1af0f34a2
fix: always do stream counting for object store upload
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-07 13:48:42 +07:00
902cb3dbb9
fix: validate written size for s3 multipart uploads
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-07 13:48:42 +07:00
e483387189
feat: more generic way of passing metadata to object storage backends for new objects
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-07 13:48:37 +07:00
f40d14abac
fix(s3): retry failed multipart uploads with decreased concurrency
...
Signed-off-by: Kent Delante <kent.delante@proton.me>
2025-06-20 18:24:16 +07:00
019b535868
fix(ObjectStore): Make S3 "connect_timeout" option configurable
...
The hardcoded connection timeout of 5 seconds may not be enough in some
cases, so now it is got from the ObjectStore arguments in Nextcloud
configuration, falling back to 5 if not set.
The connection timeout is set in seconds, but decimal precision can be
used for subsecond accuracy (for example, 4.2 for 4200 milliseconds).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2025-06-17 03:30:55 +07:00
edfb7b2393
perf: set the folder size in the same query as we create it
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-06-02 09:56:29 +07:00
9d26571eb2
fix(S3): Use original folder size during copy
...
This prevent having copied folders with a wrongly set size of 0KB.
- Fix https://github.com/nextcloud/server/issues/51916
Signed-off-by: Louis Chemineau <louis@chmn.me>
2025-06-02 09:56:29 +07:00
780766b63c
fix: keep IObjectStoreMetaData in private namespace for backports
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-05-16 10:41:27 +07:00
f0cab800ab
feat: add command to list objects
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-05-16 10:41:27 +07:00
a324dbec2e
feat: add command to get object metadata
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-05-16 10:41:27 +07:00
d393e98c9e
fix(objectstorage): add retry attempts to S3 connection
...
Signed-off-by: Anna Larch <anna@nextcloud.com>
2025-04-23 18:40:23 +07:00
27ed921ad5
fix: Transfer ownership with S3 as primary
...
When using S3 as primary storage, transferring ownership with the `--move` option fail with the following error:
`SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8-45b963397aa40d4a0063e0d85e4fe7a1' for key 'fs_storage_path_hash'`
The `--move` option moves the entire home folder from one account to another.
The error means that the move failed because the destination folder already exist in `oc_filecache`.
- With S3 as primary storage, folders only exists as entries in `oc_filecache`.
- With S3 as primary storage, `moveFromStorage(...)` only moves the cache entry, as nothing needs to be moved on disk. This cache move does not delete potentially pre-existing destination folder.
- With Local storage, `moveFromStorage(...)` calls `rename(...)` which delete pre-existing folder.
- `transfer(...)`: 687a4d9ac7/apps/files/lib/Service/OwnershipTransferService.php (L112)687a4d9ac7/lib/private/Files/SetupManager.php (L261-L262)687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L91-L135)687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L635-L636)
2025-04-16 10:19:17 +07:00
d056e4dc74
perf(files): faster query to fetch incomplete directories
...
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2025-02-13 14:24:44 +07:00
936ef804cd
fix(s3): treat empty sse_c_key as not set
...
Co-authored-by: Christian Becker <christian@dabecka.de>
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2025-01-29 17:35:52 +07:00
26c93519f2
fix: Harden files scanner for invalid null access
...
Co-authored-by: Ferdinand Thiessen <opensource@fthiessen.de>
Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2025-01-28 19:49:03 +07:00
5896f8c37f
chore: cleanup leftover debug statement
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-09 15:40:01 +07:00
6cf66f95ce
fix: get object size from stream where possible
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-05 15:16:22 +07:00
ca547bb913
fix: don't perform the extra buffering in s3 stream write when the stream size is known
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-05 14:07:40 +07:00
7da8951c57
fix: use php://temp instead of php://memory for multi-part upload buffer
...
this should reduce potential memory issues if the limit is set very high
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-12-05 14:07:37 +07:00
df3a9e221d
fix: throw correct exception type when we can't verify if an s3 bucket exists
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-11-26 16:49:55 +07:00
0de4843b73
refactor(Storage): Align all Storage constructors
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-23 13:24:18 +07:00
f28e74b7a8
refactor(Storage): Make all parameter types strong types
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-07 15:00:05 +07:00
3e12e1e789
fix: rework move into object store to better preserve fileids
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-27 14:46:08 +07:00
888d06dff9
fix: preserve fileid when moving from objectstore to non-objectstore
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-27 14:45:42 +07:00
da21acfb3f
fix: ensure source folder is removed from cache when moving to objectstore
...
otherwise this causes confusion down the line as it's contents will be moved to the new cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-27 14:45:42 +07:00
7cdccd058f
fix(Storage): Fix IStorage return types
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-26 18:13:03 +07:00
47d281865e
perf: reuse cache info we already have when moving to object store
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-19 19:06:06 +07:00
9836e9b164
chore(deps): Update nextcloud/coding-standard to v1.3.1
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-19 14:21:20 +07:00
3c0c81bb94
fix: ensure parent folder exists when writing a file to object storage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 14:11:27 +07:00
e0fb25f846
fix: fix object store id for test object store
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
60f6347c97
fix: cleanup objectstore file_put_content
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
3e94ee391f
fix: create intermediate directories for objectstore moveFromStorage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
879fcc4253
fix: get source file before moving the cache item in objectstore moveFromStorage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 13:54:55 +07:00
f25acfb799
Merge pull request #47986 from cfiehe/fix_move_on_same_bucket
...
perf(ObjectStoreStorage): Improve (slow) move on same object bucket
2024-09-17 16:35:19 +07:00
d2d1e3284d
fix(Storage): Fix method docs inheritance
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-17 10:20:38 +07:00
8ca6fcace7
fix(Storage): Document getOwner() can return false
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-17 10:10:50 +07:00
provokateurin
Kent Delante
Robin Appelman
Louis Chmn
yemkareems
Daniel Calviño Sánchez
Anna Larch
Benjamin Gaussorgues
Richard Steinmetz
Ferdinand Thiessen
Stephan Orbaugh