Commit Graph

95 Commits (c0cce94d22f79dfdf5472845ea4e0712cd9fca2d)

Author SHA1 Message Date
Christian Becker 5b51e1b0dc feat(S3): add SSE-C support in S3 External Storage
Just need to add the parameter and fix two calls. All other logic is already implemented and implicitly called through the S3ConnectionTrait

fixes #33283

Signed-off-by: Christian Becker <christian@dabecka.de>
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2025-01-31 12:01:09 +07:00
provokateurin 0de4843b73 refactor(Storage): Align all Storage constructors
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-23 13:24:18 +07:00
Côme Chilliet 1580c8612b
chore(apps): Apply new rector configuration to autouse classes
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-10-15 10:40:25 +07:00
provokateurin 14e6734547
refactor(files_external): Add Storage parameter strong types
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-07 15:00:07 +07:00
provokateurin 1302b48ca2
fix(files_external): Fix all IStorage return types
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-26 17:29:48 +07:00
Côme Chilliet 1a4978c4ea
chore: Apply rector configuration to apps folder
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-09-20 17:51:00 +07:00
Daniel Kesselberg af6de04e9e
style: update codestyle for coding-standard 1.2.3
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2024-08-25 19:34:58 +07:00
Robin Appelman c2b1f079b1
feat: add a specialized writeStream implementation for s3 external storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-07-24 17:37:20 +07:00
Andy Scherzinger c1555fc33e
chore: Add SPDX header
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
2024-06-06 18:05:37 +07:00
Joas Schilling aa5f037af7
chore: apply changes from Nextcloud coding standards 1.1.1
Signed-off-by: Joas Schilling <coding@schilljs.com>
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2023-11-23 10:36:13 +07:00
Louis Chemineau e66e8bad1a
Return correct status if touch failed
Signed-off-by: Louis Chemineau <louis@chmn.me>
2023-10-23 15:26:43 +07:00
Robin Appelman eab8dd3ca0 use multipart copy for s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-09-20 17:28:48 +07:00
Robin Appelman 7c9edbff32
handle more exceptions in AmazonS3::fopen
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-07-13 15:37:42 +07:00
James Guo bb173e092b
Update AmazonS3.php
Signed-off-by: James Guo <i@ze3kr.com>
2023-04-06 14:34:08 +07:00
François Ménabé 710f3fd405 Support of s3 storage class in External Storage application
Signed-off-by: François Ménabé <francois.menabe@gmail.com>
2023-01-12 14:59:13 +07:00
Robin Appelman ed8fedc685
fix s3 directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-15 17:17:59 +07:00
Carl Schwan a5ea677370 Rename file1 and file2 to source and target in Storage abstraction
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 17:21:47 +07:00
Carl Schwan 732badf552 Additional type hinting found by psalm
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 12:28:34 +07:00
Carl Schwan cf1381a6f9 Port away from deprecated clearBucket call
This was removed a long time ago in the library that we use, but never
detected since this was inside a try catch block...

Remove the call for now and we should investigate newer api in the
future.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:49 +07:00
Carl Schwan 1d90f84141 Port to LoggerInterface
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:28 +07:00
Matthieu Gallien 54c05bcdb9 handle AccessDenied error when checking if external s3 support versions
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2022-07-28 09:20:12 +07:00
Carl Schwan d5c23dbb9f Move CappedMemoryCache to OCP
This is an helpful helper that should be used in more place than just
server and this is already the case with groupfodlers, deck, user_oidc
and more using it, so let's make it public

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-07-14 15:54:31 +07:00
Carl Schwan ec6b83cc18 Add stricter psalm type for CappedMemoryCache
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-12 15:06:18 +07:00
Carl Schwan f945c0cbc6 Add a public replacement for OC::$server->get
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-10 18:51:12 +07:00
Erik van Velzen 17f3e5e4b5 AmazonS3: allow not implemented versioning
In case the S3 implementation does not implement versioning, set it to
false. Versioning was introduced in Nexcloud in commit
09ffac5e6d

This is needed for compatibility with the Storj gateway.

Signed-off-by: Erik van Velzen <erik@evanv.nl>
2022-03-15 00:40:38 +07:00
Julius Härtl 4e57146617
Use unique combination of hostname/bucket/key for external storages
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-11-04 11:34:13 +07:00
Robin Appelman a1ca901e58
cache versioning enabled status
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 15:03:22 +07:00
Robin Appelman eb6e6e3a85
minor directory detect improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 14:58:40 +07:00
Robin Appelman 247e12da96
always set Key field in `headObject`
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:43:50 +07:00
Robin Appelman 4bd08af2ad
more reliable hasUpdated for s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:02:57 +07:00
Robin Appelman 34637697e1
remove old migration method
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:19 +07:00
Robin Appelman d3bd0b5a1b
optimize filetype for s3 directories a bit
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:16 +07:00
Robin Appelman 294af4275c
more reliable directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:09 +07:00
Robin Appelman 5e3c8b3af2
doesDirectoryExist fixes
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:06 +07:00
Robin Appelman 09ffac5e6d
s3 external storage listing rework
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:02 +07:00
Vincent Petry d02936bfb3
Revert "Delete S3 versions in rmdir" 2021-10-12 17:06:02 +07:00
Vincent Petry 09ab7a40fe
Delete S3 versions in rmdir
When deleting a complete folder in a bucket that has versioning enabled,
also make sure to delete all associated versions and delete markers

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-10-11 13:45:02 +07:00
Louis Chemineau 735fd94cc0 Fix folder size contained in S3 buckets
If 'filesystem_check_changes' was set to never, the cached size was alway set to -1 (Pending) on every access

Signed-off-by: Louis Chemineau <louis@chmn.me>
2021-08-19 15:03:46 +07:00
Julius Härtl 8a14f64ac2 Delete object to cleanup leftover of paths when removing directories
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-08-13 16:34:48 +07:00
John Molakvoæ (skjnldsv) 215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +07:00
Robin Appelman effb7dc8ba
set mimetype for objects uploaded to object storages
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-21 15:25:58 +07:00
Christoph Wurst d89a75be0b
Update all license headers for Nextcloud 21
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-16 18:48:22 +07:00
Christoph Wurst fd649afb1f
Remove the deprecated update.php
* It was documented as deprecated.
* The app code checker warned about it
* It's been three years

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-11-27 13:02:59 +07:00
Roeland Jago Douma b7be09ab2c
Check if array elements exist before using them
It seems that in some recent upgrade here. Not always the entries we
expect are returned. So we should first check if they exist. As to not
spam the log.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-10-25 22:03:40 +07:00
Julius Härtl 2004ae4603
Set delimiter so that the CommonPrefixes response can properly be determined
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-10 08:11:55 +07:00
Robin Appelman fc96fa0cc7
fix s3 doesDirectoryExist check for empty directories
if a directory is empty, only the 'marker' object `/` exists.

since not all s3 implementations return just the prefix when listing objects by prefix,
when listing objects by the folder prefix, nothing will be returned if the directory is empty.
by not including the trailing slash in the prefix, the folder marked will always be returned if it exists

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-10 08:11:55 +07:00
Roeland Jago Douma bb06b6cce4
Fix reading empty files from objectstorage
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-09 20:45:13 +07:00
Christoph Wurst 2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +07:00
Morris Jobke ac73982395
Merge pull request #21734 from nextcloud/external-s3-no-touch
disable touch to update mtime for external s3 storage
2020-08-20 16:07:59 +07:00
Robin Appelman 0d6910ac13
disable touch to update mtime for external s3 storage
since updating the mtime requires doing a copy this can have a significant overhead and it messes with versioning

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-07 15:55:38 +07:00