Commit Graph

82 Commits (140c2541edffe136f0bbf1fba74b8bbbaecc7e02)

Author SHA1 Message Date
James Guo bb173e092b
Update AmazonS3.php
Signed-off-by: James Guo <i@ze3kr.com>
2023-04-06 14:34:08 +07:00
François Ménabé 710f3fd405 Support of s3 storage class in External Storage application
Signed-off-by: François Ménabé <francois.menabe@gmail.com>
2023-01-12 14:59:13 +07:00
Robin Appelman ed8fedc685
fix s3 directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-15 17:17:59 +07:00
Carl Schwan a5ea677370 Rename file1 and file2 to source and target in Storage abstraction
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 17:21:47 +07:00
Carl Schwan 732badf552 Additional type hinting found by psalm
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 12:28:34 +07:00
Carl Schwan cf1381a6f9 Port away from deprecated clearBucket call
This was removed a long time ago in the library that we use, but never
detected since this was inside a try catch block...

Remove the call for now and we should investigate newer api in the
future.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:49 +07:00
Carl Schwan 1d90f84141 Port to LoggerInterface
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:28 +07:00
Matthieu Gallien 54c05bcdb9 handle AccessDenied error when checking if external s3 support versions
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2022-07-28 09:20:12 +07:00
Carl Schwan d5c23dbb9f Move CappedMemoryCache to OCP
This is an helpful helper that should be used in more place than just
server and this is already the case with groupfodlers, deck, user_oidc
and more using it, so let's make it public

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-07-14 15:54:31 +07:00
Carl Schwan ec6b83cc18 Add stricter psalm type for CappedMemoryCache
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-12 15:06:18 +07:00
Carl Schwan f945c0cbc6 Add a public replacement for OC::$server->get
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-10 18:51:12 +07:00
Erik van Velzen 17f3e5e4b5 AmazonS3: allow not implemented versioning
In case the S3 implementation does not implement versioning, set it to
false. Versioning was introduced in Nexcloud in commit
09ffac5e6d

This is needed for compatibility with the Storj gateway.

Signed-off-by: Erik van Velzen <erik@evanv.nl>
2022-03-15 00:40:38 +07:00
Julius Härtl 4e57146617
Use unique combination of hostname/bucket/key for external storages
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-11-04 11:34:13 +07:00
Robin Appelman a1ca901e58
cache versioning enabled status
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 15:03:22 +07:00
Robin Appelman eb6e6e3a85
minor directory detect improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 14:58:40 +07:00
Robin Appelman 247e12da96
always set Key field in `headObject`
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:43:50 +07:00
Robin Appelman 4bd08af2ad
more reliable hasUpdated for s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:02:57 +07:00
Robin Appelman 34637697e1
remove old migration method
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:19 +07:00
Robin Appelman d3bd0b5a1b
optimize filetype for s3 directories a bit
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:16 +07:00
Robin Appelman 294af4275c
more reliable directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:09 +07:00
Robin Appelman 5e3c8b3af2
doesDirectoryExist fixes
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:06 +07:00
Robin Appelman 09ffac5e6d
s3 external storage listing rework
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:02 +07:00
Vincent Petry d02936bfb3
Revert "Delete S3 versions in rmdir" 2021-10-12 17:06:02 +07:00
Vincent Petry 09ab7a40fe
Delete S3 versions in rmdir
When deleting a complete folder in a bucket that has versioning enabled,
also make sure to delete all associated versions and delete markers

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-10-11 13:45:02 +07:00
Louis Chemineau 735fd94cc0 Fix folder size contained in S3 buckets
If 'filesystem_check_changes' was set to never, the cached size was alway set to -1 (Pending) on every access

Signed-off-by: Louis Chemineau <louis@chmn.me>
2021-08-19 15:03:46 +07:00
Julius Härtl 8a14f64ac2 Delete object to cleanup leftover of paths when removing directories
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-08-13 16:34:48 +07:00
John Molakvoæ (skjnldsv) 215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +07:00
Robin Appelman effb7dc8ba
set mimetype for objects uploaded to object storages
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-21 15:25:58 +07:00
Christoph Wurst d89a75be0b
Update all license headers for Nextcloud 21
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-16 18:48:22 +07:00
Christoph Wurst fd649afb1f
Remove the deprecated update.php
* It was documented as deprecated.
* The app code checker warned about it
* It's been three years

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-11-27 13:02:59 +07:00
Roeland Jago Douma b7be09ab2c
Check if array elements exist before using them
It seems that in some recent upgrade here. Not always the entries we
expect are returned. So we should first check if they exist. As to not
spam the log.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-10-25 22:03:40 +07:00
Julius Härtl 2004ae4603
Set delimiter so that the CommonPrefixes response can properly be determined
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-10 08:11:55 +07:00
Robin Appelman fc96fa0cc7
fix s3 doesDirectoryExist check for empty directories
if a directory is empty, only the 'marker' object `/` exists.

since not all s3 implementations return just the prefix when listing objects by prefix,
when listing objects by the folder prefix, nothing will be returned if the directory is empty.
by not including the trailing slash in the prefix, the folder marked will always be returned if it exists

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-10 08:11:55 +07:00
Roeland Jago Douma bb06b6cce4
Fix reading empty files from objectstorage
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-09 20:45:13 +07:00
Christoph Wurst 2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +07:00
Morris Jobke ac73982395
Merge pull request #21734 from nextcloud/external-s3-no-touch
disable touch to update mtime for external s3 storage
2020-08-20 16:07:59 +07:00
Robin Appelman 0d6910ac13
disable touch to update mtime for external s3 storage
since updating the mtime requires doing a copy this can have a significant overhead and it messes with versioning

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-07 15:55:38 +07:00
Julius Härtl f5cc34c9c2
Reuse cache for directory mtime/size if filesystem changes can be ignored
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-07-07 10:22:22 +07:00
Julius Härtl 69f4c50032
Remove rescanDelay from directory mtime
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-06-22 14:15:57 +07:00
Christoph Wurst caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +07:00
Christoph Wurst 14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +07:00
Christoph Wurst 1a9330cd69
Update the license headers for Nextcloud 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-31 14:52:54 +07:00
Christoph Wurst b80ebc9674
Use the short array syntax, everywhere
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-26 16:34:56 +07:00
Christoph Wurst 5bf3d1bb38
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +07:00
Roeland Jago Douma 68748d4f85
Some php-cs fixes
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-11-22 20:52:10 +07:00
Daniel Kesselberg 230f685987
Remove superfluous fclose
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-10-21 11:31:07 +07:00
Daniel Kesselberg 72d22a4828
Use local variable for directory name
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +07:00
Daniel Kesselberg 75b8944061
Don't strip path from directory prefix
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +07:00
Daniel Kesselberg 80340a8d46
Fill directory cache from opendir
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +07:00
Daniel Kesselberg 12863e0d24
Change visibility to private (from protected)
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +07:00