Technically, saying that a storage has no updates when it's not
available is correct.
This makes it possible to retrieve the cache entry for the mount point
and also to list and remove unavailable federated shares.
Signed-off-by: Vincent Petry <vincent@nextcloud.com>
When using and object store as primary storage and using the default
encryption module at the same time, any encrypted file would be truncated
when read, and a text error message added to the end.
This was caused by a combination of the reliance of the read functions on
on knowing the unencrypted file size, and a bug in the function which
calculated the unencrypted file size for a given file.
In order to calculate the unencrypted file size, the function would first
skip the header block, then use fseek to skip to the last encrypted block
in the file. Because there was a corresponence between the encrypted and
unencrypted blocks, this would also be the last encrypted block. It would
then read the final block and decrypt it to get the unencrypted length of
the last block. With that, the number of blocks, and the unencrypted block
size, it could calculate the unencrypted file size.
The trouble was that when using an object store, an fread call doesn't
always get you the number of bytes you asked for, even if they are
available. To resolve this I adapted the stream_read_block function from
lib/private/Files/Streams/Encryption.php to work here. This function
wraps the fread call in a loop and repeats until it has the entire set of
bytes that were requested, or there are no more to get.
This fixes the imediate bug, and should (with luck) allow people to get
their encrypted files out of Nextcloud now. (The problem was purely on
the decryption side). In the future it would be nice to do some
refactoring here.
I have tested this with image files ranging from 1kb to 10mb using
Nextcloud version 22.1.0 (the nextcloud:22.1-apache docker image), with
sqlite and a Linode object store as the primary storage.
Signed-off-by: Alan Meeson <alan@carefullycalculated.co.uk>
Using advanced ACL, it is possible that an user has access to a
directory but not to a subdirectory, so the copying use
Common::copyFromStorage instead of Local::copyFromStorage.
Fix https://github.com/nextcloud/groupfolders/issues/1692
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
The current phpdoc of IStorage#file_put_contents doesnt corresponds to
it's actual usage in code, e.g.
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
The OC\Files\Storage\Local#writeStream use system provided file_put_contents.
However, it overrides file_put_contents, thus expects that the default behaviour
can be different.
Use Local#file_put_contents in writeStream to benefit from class specific functionality.
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
* removes the ability for users to import their own certificates (for external storage)
* reliably returns the same certificate bundles system wide (and not depending on the user context and available sessions)
The user specific certificates were broken in some cases anyways, as they are only loaded if the specific user is logged in and thus causing unexpected behavior for background jobs and other non-user triggered code paths.
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
This fixes an issue where the files_trashbin hierarchy of a user could
not been created as the mkdir operations were blocked by the quota
storage wrapper. Even with 0 quota, users should be able to have a
trashbin for external storages.
Signed-off-by: Julius Härtl <jus@bitgrid.net>
Co-authored-by: Christoph Wurst <christoph@winzerhof-wurst.at>
Signed-off-by: Julius Härtl <jus@bitgrid.net>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
having the "cache rename" after the "storage move" caused the target
to get the fileid from the source file, without taking care that the object
is stored under the original file id.
By doing the "cache rename" first, we trigger the "update existing file"
logic while moving the file to the object store and the object gets stored for the
correct file id
Signed-off-by: Robin Appelman <robin@icewind.nl>
the php ftp streamwrapper doesn't handle hashes correctly and will break when it tries to enter a path containing a hash.
By filtering out paths containing a hash we can at least stop the external storage from breaking completely
Signed-off-by: Robin Appelman <robin@icewind.nl>
while some code paths do wrap the "raw" locking exception into one with a proper path, not all of them do
by adding the proper path to the original exception we ensure that we always have the usefull information in out logs
Signed-off-by: Robin Appelman <robin@icewind.nl>
while this scan *should* never be triggered, it's good to have some failsafe to ensure
that the users home contents don't end up getting scanned in the root storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
Else if a lot of writes happen. It might happen that an old stat result
is used. Resulting in a wrong file size for the file. For example the
text app when a lot of people edit at the same time.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.
While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)
With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.
Signed-off-by: Robin Appelman <robin@icewind.nl>
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.
This also removes and empty lines from method/function bodies at the
beginning and end.
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
If you try to do something on a DAV mount (external or federated share)
that is not allowed. We should not mark the storage as not available but
just fail somewhat gracefully.
Now by catching this and just properly returning the operation will just
fail (and notify the user) which is already a lot better then marking
the storage as unavailable and doing boom.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
the target storage doesn't need additional handling for wrappers as the wrappers implementation of moveFromStorage already deals with that
Any storage based on local storage isn't affected by this as local storage already has it's own way of handling with this
Signed-off-by: Robin Appelman <robin@icewind.nl>
We already catch the result value. Having the warning being logged
explicitly doesn't help and polutes the log.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
This can happen for valid reasons (multiple users writing at the same
time) with for example the text app. Apps should properly handle it. No
reason to log it by default.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
we can only store etags up to 40 characters long in the database, so when we get an etag that's longer we simply hash it to bring down the length
Signed-off-by: Robin Appelman <robin@icewind.nl>
Right now we propogate a lof of changes in appdata. So for example we
propogate each and every preview that is added to the system. This has
no real added value as far as I can tell.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
this removes the need for temporary storages with some external storage backends.
The new method is added to a separate interface to maintain compatibility with
storage backends implementing the storage interface directly (without inheriting common)
Currently the interface is implemented for objectstorage based storages and local storage
and used by webdav uploads
Signed-off-by: Robin Appelman <robin@icewind.nl>
Fixes#11637
If we do not normalize the unjailed path we might end up with a path
like files/user/folder/. which can break on objectstores
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Apperently Sabre and Onedrive are not friends when requesting a single
404 property. I need to dig deeper on why this is. Anyways requesting a
valid property makes it work like a charm.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Fixes#4577
Users with a quota of 0 are a special case. Since they can't (ever)
create files on their own storage. Therefor it makes no real that they
can create folders (and possible share those etc).
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Fixes#7444
Since the quota is calculated on the files. We should allow apps etc to
store temp stuff in the <user>/cache
Else users can't upload avatars for example.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
If an external storage test fails the exception's message is now written
into the logfile (level INFO). Additionally, the stack trace is printed
as DEBUG. This helps to resolve the reason for a failing connection.
Signed-off-by: Roland Tapken <roland@bitarbeiter.net>
* found while adding the strict_typing for PHP 7+ migration
* first argument is a boolean - second one is the path
* see http://php.net/manual/en/function.clearstatcache.php
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
If we have a jailed storage we must also fix the internal path on copy.
Else we pass in the wrong path.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Vincent Petry
alanmeeson
John Molakvoæ
Carl Schwan
Robin Appelman
acsfer
Daniel Kesselberg
Julius Härtl
Lukas Reschke
Rid
John Molakvoæ (skjnldsv)
Roeland Jago Douma
Roeland Jago Douma
dependabot-preview[bot]
Jasper Knockaert
Johannes Leuker
Morris Jobke
Christoph Wurst
Tigran Mkrtchyan
aler9
J0WI
Christoph Wurst
Greta Doci
Arthur Schiwon
Joas Schilling
Florian Schunk
Bjoern Schiessle
Roland Tapken