3950deb42f
Merge pull request #32740 from jasperweyne/patch-2
...
Fix open_basedir issues by AWS SDK
2022-09-19 10:19:15 +07:00
457822c144
don't update the folder size if we know it hasn't changed
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:56:51 +07:00
44f6c931e7
Merge branch 'master' into patch-2
2022-08-11 08:54:08 +07:00
61fdd91dcf
don't set `null` as a bundle path
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-07-27 12:58:07 +07:00
f1486890d7
only use nextcloud bundle when explicitly enabled
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-07-05 16:06:55 +07:00
bffa67c48b
also use nextcloud certificate bundle when downloading from s3
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-07-05 15:57:54 +07:00
0633a1d9f5
Disable ~/.aws/config access by S3ConnectionTrait
...
This commit sets the 'use_aws_shared_config_files' option to false, in order to disable configuration loading from ~/.aws/config by the AWS SDK, specifically the S3Client. It is a continuation of #27040 , as that PR only changed the behaviour of the CredentialsProvider; this change affects the ConfigurationProvider as well.
Signed-off-by: Jasper Weyne <jasperweyne@gmail.com>
2022-06-07 13:39:14 +07:00
0f0e74a78c
Cleanup temporary files after finishing the write to object storage
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-06-02 12:57:18 +07:00
069477e608
Migrate more classes of lib/private to LoggerInterface
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:52 +07:00
7817845538
Add a metadata service to store file metadata
...
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-04-13 14:06:29 +07:00
ea23523c70
Adapt more code to migration to LoggerInterface
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +07:00
6be7aa112f
Migrate from ILogger to LoggerInterface in lib/private
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +07:00
1156214a26
don't try to get custom certs for s3 primary storage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-03-15 15:50:08 +07:00
194a21f374
use the nextcloud certificate bundle for s3
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-03-14 18:09:48 +07:00
bc2c02f451
Wrap S3 multipart upload exception
...
Signed-off-by: Louis Chemineau <louis@chmn.me>
2022-02-16 14:56:14 +07:00
0b768c17d2
Wrap S3 multipart upload exception
...
Signed-off-by: Louis Chemineau <louis@chmn.me>
2022-02-15 17:35:12 +07:00
2aac757805
Make max size for single put uploads configurable
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-12-29 13:30:22 +07:00
e475dfe16c
Fix php-cs for S3ConnectionTrait.php
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-12-21 21:35:46 +07:00
1acfbd0b06
Merge pull request #27493 from cuppett/cuppett/simplify-aws-credential-provider
2021-12-21 11:58:52 +07:00
66bdda568a
Fix Guzzle function calls
...
Since 1.7.0 and 2.0 these functions are turned into static methods.
See https://github.com/guzzle/psr7#upgrading-from-function-api
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2021-11-23 09:28:57 +07:00
4a6a2c2b48
Simplify S3ConnectionTrait to defaultProvider plus option
...
When we initially added the EC2 and ECS IAM role support in #24700 ,
we had to use a workaround by explicitly ordering the various providers
due to an inconsistency in the AWS SDK for PHP. We submitted a PR there
to get that squared away. Now, we've consumed that version upstream
for the SDK and can update our code here to be the most concise version
as well as position ourselves to pick up new methods as those become
available and prevalent in AWS (for acquiring credentials).
See also: https://github.com/nextcloud/server/pull/24700#issuecomment-747650892
See also: https://github.com/aws/aws-sdk-php/pull/2172
Signed-off-by: Stephen Cuppett <steve@cuppett.com>
2021-11-22 07:04:48 +07:00
6ab2feaa54
Implement multibucket shift for ObjectStore
...
Signed-off-by: John Molakvoæ <skjnldsv@protonmail.com>
2021-11-11 15:09:28 +07:00
df4e6bab69
Merge pull request #29115 from nextcloud/work/carl/correct-permissions-when-copying
...
Fix permissions when copying from ObjectStorage
2021-10-28 14:38:20 +07:00
bfa60aaf27
Fix permissions when copying from ObjectStorage
...
Make sure that when a user copy a file from a directory they don't have
all permissions to a directory where they have more permissions, the
permissions are correctly set to the one from the parent taget folder.
This was caused by the ObjectStoreStorage::copyFromStorage using
the jailed storage and cache entry instead of the unjailed one like other
storages (the local one).
Steps to reproduce
+ Use object storage
+ Create a groupfolder with one group having full permission and another one
who can just read files.
+ With an user who is in the second group, copy a file from the groupfolder to
the home folder of this user.
+ The file in the home folder of the user will be read only and can't be deleted
even though it is in their home folder and they are the owner. In oc_filecache,
the permissions stored for this file are 1 (READ)
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-28 13:29:50 +07:00
09ffac5e6d
s3 external storage listing rework
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:02 +07:00
3866f388b1
Refactor writeObject to only use MultipartUpload when required
...
Signed-off-by: Bernd Rederlechner <Bernd.Rederlechner@t-systems.com>
Co-authored-by: Julius Härtl <jus@bitgrid.net>
2021-08-20 17:02:25 +07:00
3e67637a4a
Merge pull request #26463 from Worteks/fix-s3proxy
2021-07-14 23:32:05 +07:00
62675eb5c2
Merge pull request #27586 from nextcloud/bugfi/noid/objectstore-checksum
...
Reset checksum when writing files to object store
2021-07-01 17:30:54 +07:00
388a458cd5
fix(proxy): use accessor
...
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
2021-06-30 09:25:04 +07:00
6d5cfe0c66
Move DateTime::RFC2822 to DateTimeInterface::2822
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2021-06-23 15:30:43 +07:00
21db463bc9
Reset checksum when writing files to object store
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-06-21 12:04:49 +07:00
215aef3cbd
Update php licenses
...
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +07:00
333665b43d
Merge pull request #27040 from nextcloud/FlorentCoppint-master
...
Avoid reading ~/.aws/config when using S3 provider
2021-05-25 10:19:22 +07:00
d4444f2472
Avoid reading ~/.aws/config when using S3 provider
...
When using S3 storage, we sometimes have open_basedir errors due to aws-sdk-php trying to read ~/.aws/config which is out of open_basedir restrictions.
Christoph Wurst already added csm=false config in #21406 but it wasn't enough, we also need to set use_arn_region=false, added in this commit.
Signed-off-by: Florent <florent@coppint.com>
2021-05-20 12:21:15 +07:00
ef6f2e68f0
explicitly close source stream on object store upload even if count wrapper isn't needed
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-29 17:01:19 +07:00
effb7dc8ba
set mimetype for objects uploaded to object storages
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-21 15:25:58 +07:00
547438527d
fix(proxy): reaching s3 storage behind some http proxy
...
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
2021-04-09 10:15:57 +07:00
03fe74b95e
fix(proxy): reaching s3 storage behind some http proxy
...
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
2021-04-09 10:15:57 +07:00
ad636ae12d
use returned file id for object store copy
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-12 17:22:40 +07:00
32551b9ff7
Merge pull request #25722 from nextcloud/objectstore-copy-cross
...
apply object store copy optimization when 'cross storage' copy is wit…
2021-03-12 16:10:18 +07:00
3eb764e65d
remove explicit fclose from S3->writeStream
...
streams get closed automatically when dropped, and in some cases the stream seems to be already closed by the S3 library, in which case trying to close it again will raise an error
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-11 18:05:04 +07:00
c87b1a50d9
apply object store copy optimization when 'cross storage' copy is within the same object store
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-09 20:31:27 +07:00
eb502c02ff
Bump nextcloud/coding-standard from 0.3.0 to 0.5.0
...
Bumps [nextcloud/coding-standard](https://github.com/nextcloud/coding-standard ) from 0.3.0 to 0.5.0.
- [Release notes](https://github.com/nextcloud/coding-standard/releases )
- [Changelog](https://github.com/nextcloud/coding-standard/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nextcloud/coding-standard/compare/v0.3.0...v0.5.0 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2021-02-18 13:31:24 +07:00
137636b651
Merge pull request #24700 from Imajie/s3-creds
...
Resolves #24699 , Support ES2 and ECS instance providers for S3 buckets
2021-02-02 21:48:22 +07:00
9ce3ea3368
Update license headers
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-30 14:07:05 +07:00
73c7d0dc81
Bump icewind/streams from 0.7.1 to 0.7.2
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-29 09:07:36 +07:00
5094e29ebd
Merge pull request #24594 from kofemann/dcache
2020-12-22 09:26:07 +07:00
4f2dc18f58
storage: update IStorage#file_put_contents docs to match usage
...
The current phpdoc of IStorage#file_put_contents doesnt corresponds to
it's actual usage in code, e.g.
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2020-12-21 19:04:18 +07:00
d89a75be0b
Update all license headers for Nextcloud 21
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-16 18:48:22 +07:00
45a02ee30c
Fix failing tests
...
Signed-off-by: James Letendre <james.letendre@gmail.com>
2020-12-15 19:06:06 +07:00
Carl Schwan
Robin Appelman
Jasper Weyne
Julius Härtl
Côme Chilliet
Louis Chemineau
Stephen Cuppett
John Molakvoæ
Bernd Rederlechner
blizzz
Maxime Besson
Christoph Wurst
Morris Jobke
Florent
Samuel
dependabot-preview[bot]
Roeland Jago Douma
Tigran Mkrtchyan
James Letendre