This is faster than going back to nextcloud to download the files.
This is an opt-in setting that can be enabled by setting
use_presigned_url in the object store config.
Additionally add support for the proxy config which is needed in a
docker setup. See https://github.com/juliusknorr/nextcloud-docker-dev/pull/431
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
Add retriesMaxAttempts parameter to S3 objectstore configuration
to allow customization of AWS SDK retry behavior for handling
unreliable network conditions or proxy issues.
Defaults to 5 retries (AWS SDK default) if not specified.
Signed-off-by: nfebe <fenn25.fn@gmail.com>
Otherwise, we call doesBucketExist all the time which does a network
request to the S3 server adding some non-trivial latency when creating a
S3 connection object.
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
Fix https://github.com/nextcloud/server/issues/56077
This commit makes the configuration settings 'request_checksum_calculation' and 'response_checksum_validation' of the S3Client from the AWS SDK for PHP configurable.
Signed-off-by: Fiehe Christoph <c.fiehe@eurodata.de>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
to avoid 'object::%' to be considered as a column
(`la colonne \\u00ab object::% \\u00bb n'existe pas` on PG)
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
* Simplify migration by not moving the actual files and just updating
the DB
* Don't store the storageid in the preview table as it is not needed
* Start adding tests
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
The new oc_previews table is optimized for storing previews and should
decrease significantly the space taken by previews in the filecache
table.
This attend to reuse the IObjectStore abstraction over S3/Swift/Azure
but currently only support one single bucket configuration.
Signed-off-by: Carl Schwan <carl.schwan@nextclound.com>
The hardcoded connection timeout of 5 seconds may not be enough in some
cases, so now it is got from the ObjectStore arguments in Nextcloud
configuration, falling back to 5 if not set.
The connection timeout is set in seconds, but decimal precision can be
used for subsecond accuracy (for example, 4.2 for 4200 milliseconds).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When using S3 as primary storage, transferring ownership with the `--move` option fail with the following error:
`SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8-45b963397aa40d4a0063e0d85e4fe7a1' for key 'fs_storage_path_hash'`
The `--move` option moves the entire home folder from one account to another.
The error means that the move failed because the destination folder already exist in `oc_filecache`.
- With S3 as primary storage, folders only exists as entries in `oc_filecache`.
- With S3 as primary storage, `moveFromStorage(...)` only moves the cache entry, as nothing needs to be moved on disk. This cache move does not delete potentially pre-existing destination folder.
- With Local storage, `moveFromStorage(...)` calls `rename(...)` which delete pre-existing folder.
- `transfer(...)`: 687a4d9ac7/apps/files/lib/Service/OwnershipTransferService.php (L112)
- `oneTimeUserSetup(...)`: 687a4d9ac7/lib/private/Files/SetupManager.php (L261-L262)
- `mkdir(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L91-L135)
- `moveFromStorage(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L635-L636)
Delete pre-existing folder in `moveFromStorage(...)`
Signed-off-by: Louis Chemineau <louis@chmn.me>
Carl Schwan
Daniel Kesselberg
provokateurin
Kent Delante
Andy Scherzinger
Carl Schwan
nfebe
Robin Appelman
Carl Schwan
Kate
Stephan Orbaugh
Fiehe Christoph
Daan Selen
Thomas Citharel
Louis Chmn
yemkareems
Ferdinand Thiessen
Daniel Calviño Sánchez
Anna Larch
Hector Manuel
Benjamin Gaussorgues
Hector Valcarcel
Richard Steinmetz