Commit Graph

317 Commits (feat/database/query-result-fetch-associative-fetch-num)

Author SHA1 Message Date
Côme Chilliet d9dbed9105
Fix psalm errors related to filesizes
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:29 +07:00
Côme Chilliet ff776a90b1
Strong type filesize related methods to ease 32bits problem findings
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:28 +07:00
Côme Chilliet e156f8339c
Revert "remove 32-bit workarounds"
This reverts commit dd8774389e.

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-02-07 11:23:28 +07:00
Richard Steinmetz f0a3b77d0f fix(encryption): don't throw on missing file
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2023-01-30 10:07:01 +07:00
Robin Appelman 33b90d28f8 fix moving files of encrypted local storage to unencrypted local storage
for example when moving encrypted files to a groupfolder

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-01-24 16:40:41 +07:00
Côme Chilliet f5c361cf44
composer run cs:fix
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-20 11:45:08 +07:00
Côme Chilliet a7740c0ae6
Suppress last psalm error until upstream fix is released and included
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-02 15:17:38 +07:00
Côme Chilliet a372564850
Fix psalm spotted errors with new requirements
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-02 15:02:28 +07:00
Robin Appelman e3a0e64698
clear is-encrypted cache when trying to fix encrypted version
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-12-01 14:36:27 +07:00
Robin Appelman e9b87c73fa
fix reading newly written encrypted files before their cache entry is written
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-30 14:54:56 +07:00
szaimen 3f7c620f26 fix psalm
Signed-off-by: szaimen <szaimen@e.mail.de>
2022-11-02 11:13:34 +07:00
szaimen dd8774389e remove 32-bit workarounds
Signed-off-by: szaimen <szaimen@e.mail.de>
2022-11-02 11:13:34 +07:00
Carl Schwan a5ea677370 Rename file1 and file2 to source and target in Storage abstraction
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 17:21:47 +07:00
Robin Appelman cb51564356 trigger a rescan when trying to fopen a file that exists in cache but not on disk
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-09-29 11:17:16 +07:00
Vincent Petry 1025d049c7
Merge pull request #24596 from kofemann/dcache-worm
make NextCloud WORM file system friendly
2022-09-16 14:39:13 +07:00
Tigran Mkrtchyan f41209a061 config: add a switch to control truncate before update
To avoid extra truncate on non WORM file systems, add a new config
option `localstorage.unlink_on_truncate`, which defaults to false.

The OC\Files\Storage\Local is update to respect that option.

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2022-09-16 11:10:32 +07:00
Tigran Mkrtchyan 8fc4cf67f1 files: remove destination file before copying new content (WORM)
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2022-09-16 10:57:17 +07:00
Git'Fellow 5300f0d693
Switch logical operators (and|or)
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
2022-09-12 09:35:12 +07:00
Robin Appelman bd91c56539
get quota for user only when needed
always apply the wrapper but have the wrapper get the quota lazily only when an operation where quota is applied is performed

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-09-01 19:32:24 +07:00
Tigran Mkrtchyan b6065a236f files: make OC\Files\Storage\Local WORM friendly
Some filesystems run as a Write-Once-Read-Many storages. This
makes them impossible to use with NexeCloud, as the file system
layers uses `truncate` syscall (through file_put_contents function).

As Nextcloud is never updates existing files, removing the old entry
and creatint a new one on update will allow NextCoud to update on such
file systems.

Update Local#fopen and Local#file_put_contents to remote existing
file before truncating.

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2022-08-25 11:11:49 +07:00
Carl Schwan d5c23dbb9f Move CappedMemoryCache to OCP
This is an helpful helper that should be used in more place than just
server and this is already the case with groupfodlers, deck, user_oidc
and more using it, so let's make it public

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-07-14 15:54:31 +07:00
blizzz 2e10378052
Merge pull request #31475 from nextcloud/kofemann-patch-1
files: close open directory file descriptor on error path
2022-06-21 18:44:05 +07:00
Vincent Petry 8809de1eef
Merge pull request #31966 from nextcloud/unencrypted-size
store unencrypted size in the unencrypted_size column
2022-06-13 11:55:37 +07:00
Martin Brugnara 7409569994
Expose umask override value as config parameter: localstorage.umask
Commit 451c06d introduced override for umask value.
This is needed to avoid broken env configuration or dirty workers
to mess with the permissions when creating new files.

Most Nextcloud, that does not integrate with external software
would work fine with an hard-coded value (451c06d set it at 022).

Advanced install may require more flexibility, as such this commit
exposes the "umask override value" as configuration parameter:
`localstorage.umask`

It defaults to 0022 both in code and in config/config.sample.php .

Signed-off-by: Martin Brugnara <martin@0x6d62.eu>
2022-06-05 19:19:31 +07:00
Robin Appelman 8238582e59
store unencrypted size in the unencrypted_size column
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-06-02 16:07:00 +07:00
Côme Chilliet 2b07031d3e
composer run cs:fix
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:53 +07:00
Côme Chilliet 23f7a37c03
Remove ILogger usages in lib/private/Files/Storage
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:53 +07:00
Robin Appelman 468d6a9cf1
ignore forbidden files while scanning local storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-20 13:01:16 +07:00
Robin Appelman f9a5fb99fd
use the same cached propfind for free space for dav storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-04 16:48:31 +07:00
Louis 7d2cb35988
Merge pull request #31632 from Maaxxs/fix-undefined-index-dav
Fixes the undefined index error with the DAV property getlastmodified
2022-03-29 13:53:15 +07:00
Max Kunzelmann 2952c7d01f Use the null coalescing operator to set the value of `size`.
Signed-off-by: Max Kunzelmann <maxdev@posteo.de>
2022-03-28 22:26:45 +07:00
Côme Chilliet e8c1f75064
Fixing psalm errors
Encryption constructor is problematic and should still be fixed later.

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:26 +07:00
Côme Chilliet 6be7aa112f
Migrate from ILogger to LoggerInterface in lib/private
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +07:00
Max Kunzelmann 3a135f0c5f Set `mtime` to null instead of false if `getlastmodified` does not exist.
Signed-off-by: Max Kunzelmann <maxdev@posteo.de>
2022-03-23 15:02:01 +07:00
Max Kunzelmann e0a5fe649a Fix typo.
Signed-off-by: Max Kunzelmann <maxdev@posteo.de>
2022-03-20 11:07:36 +07:00
Max Kunzelmann ee888875bb Fixes the undefined index error with the DAV property getlastmodified
Signed-off-by: Max Kunzelmann <maxdev@posteo.de>
2022-03-20 00:38:07 +07:00
Robin Appelman 84f464550a
some file scanner performance improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-03-17 22:21:16 +07:00
Tiramisu Mokka 8190441335
files: close open directory file descriptor on error path 2022-03-07 10:19:58 +07:00
Côme Chilliet 5cca86ce19
Remove redundant if clause
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-01-25 16:53:35 +07:00
Côme Chilliet 35dcda36a3
Fix OC\Files\Storage\DAV::hasUpdated when cache is not set
Fixes the test OCA\Files_External\Tests\Storage\WebdavTest::testStat on
 PHP>=7.4

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-01-25 16:53:19 +07:00
Vincent Petry cfafd79706
Merge pull request #30631 from nextcloud/fix/update_encrypted_version
updateEncryptedVersion: cleanup on target if cache already got renamed
2022-01-14 09:45:08 +07:00
Carl Schwan cbf9064b8e
Fix psalm issues
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-13 12:30:27 +07:00
Carl Schwan 695165260f
Add helper method in Wrapper
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-13 10:39:36 +07:00
Jonas Meurer a2f4c62c19
updateEncryptedVersion: cleanup on target if cache already got renamed
When moving a file to trash with encryption enabled, the cache gets
moved before the actual file. According to @icewind1991 this is in order
to not break object storage.

When moving a file from an unencrypted storage (e.g. a collectives
storage) to the encrypted trashbin storage, this causes errors, see

This commit fixes it by doing `updateEncryptedVersion()` on the target
cache entry *if* the source cache entry doesn't exist anymore, but the
corresponding target cache entry does exist already.

Fixes: #26544

Signed-off-by: Jonas Meurer <jonas@freesources.org>
2022-01-12 15:47:08 +07:00
Vincent Petry be590df588
Fix fail when keys/files folder already exists
Fixes an issue with transfer ownership in move mode where the folder
"files_encryption/keys/files" already exists.

Instead of failing, its existence is checked before calling mkdir.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2022-01-10 11:15:22 +07:00
Vincent Petry 6f81e60bce
Check resource before closing in encryption wrapper
In case of error there is no guarantee that $source or $target is set or
is a resource when handling an error.

Without this fix, there's a risk that fclose will fail and the actual
exception will not be thrown, making it impossible to find out about the
root cause.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-12-09 11:28:10 +07:00
Joas Schilling 65d79bb592
Check for invalid characters before trimming
Signed-off-by: Joas Schilling <coding@schilljs.com>
2021-11-25 08:48:26 +07:00
Vincent Petry 289624b79e
Merge pull request #29758 from nextcloud/disk_free_space-fix-php8
Check for `disk_free_space`
2021-11-19 17:14:08 +07:00
acsfer 152e2bfdaa
Check for `disk_free_space`
Check for `disk_free_space` before calling it.
Fix https://github.com/nextcloud/server/issues/29748
2021-11-18 11:46:13 +07:00
Vincent Petry c92a0e4152
Normalize directory entries in Encoding wrapper
Directory entry file names are now normalized in getMetaData(),
getDirectoryContents() and opendir().

This makes the scanner work properly as it assumes pre-normalized names.

In case the names were not normalized, the scanner will now skip the
entries and display a warning when applicable.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-17 09:43:25 +07:00
Vincent Petry 93fb33d863
Update lib/private/Files/Storage/Wrapper/Availability.php add void
Signed-off-by: Vincent Petry <vincent@nextcloud.com>

Co-authored-by: Carl Schwan <carl@carlschwan.eu>
2021-11-05 10:28:41 +07:00
Vincent Petry db29fd29ee
Return false in hasUpdated when storage is not available
Technically, saying that a storage has no updates when it's not
available is correct.

This makes it possible to retrieve the cache entry for the mount point
and also to list and remove unavailable federated shares.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-05 09:48:52 +07:00
alanmeeson 16f70e8647 Add type hint to fread_block $blockSize param & apply cs:fix.
Signed-off-by: alanmeeson <alan@carefullycalculated.co.uk>
2021-10-23 15:11:23 +07:00
Alan Meeson 44c332a46e Fix truncation of files upon read when using object store and encryption.
When using and object store as primary storage and using the default
encryption module at the same time,  any encrypted file would be truncated
when read, and a text error message added to the end.

This was caused by a combination of the reliance of the read functions on
on knowing the unencrypted file size,  and a bug in the function which
calculated the unencrypted file size for a given file.

In order to calculate the unencrypted file size,  the function would first
skip the header block, then use fseek to skip to the last encrypted block
in the file.  Because there was a corresponence between the encrypted and
unencrypted blocks, this would also be the last encrypted block.  It would
then read the final block and decrypt it to get the unencrypted length of
the last block.  With that, the number of blocks, and the unencrypted block
size, it could calculate the unencrypted file size.

The trouble was that when using an object store, an fread call doesn't
always get you the number of bytes you asked for, even if they are
available.  To resolve this I adapted the stream_read_block function from
lib/private/Files/Streams/Encryption.php to work here.  This function
wraps the fread call in a loop and repeats until it has the entire set of
bytes that were requested,  or there are no more to get.

This fixes the imediate bug, and should (with luck) allow people to get
their encrypted files out of Nextcloud now.  (The problem was purely on
the decryption side).  In the future it would be nice to do some
refactoring here.

I have tested this with image files ranging from 1kb to 10mb using
Nextcloud version 22.1.0 (the nextcloud:22.1-apache docker image), with
sqlite and a Linode object store as the primary storage.

Signed-off-by: Alan Meeson <alan@carefullycalculated.co.uk>
2021-10-23 15:11:23 +07:00
John Molakvoæ f4e4a85fcf
Merge pull request #27440 from nextcloud/is-file-handle 2021-10-23 11:18:52 +07:00
John Molakvoæ e673ec0b87
Merge branch 'master' into imountpoint-ocp-storage 2021-10-22 11:47:37 +07:00
Carl Schwan 9408f8ae69
Fix security issues when copying groupfolder with advanced ACL
Using advanced ACL, it is possible that an user has access to a
directory but not to a subdirectory, so the copying use
Common::copyFromStorage instead of Local::copyFromStorage.

Fix https://github.com/nextcloud/groupfolders/issues/1692

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-21 17:11:52 +07:00
John Molakvoæ a1c8b6e9eb
Merge pull request #28768 from nextcloud/check-file-exists 2021-10-18 09:54:22 +07:00
John Molakvoæ 739b67031b
Merge pull request #27194 from nextcloud/new-ftp 2021-10-14 10:29:24 +07:00
Robin Appelman b3766fc99b
make param names consistent with interface for copy directory polyfill
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-07 17:19:44 +07:00
acsfer 98eac0fc05
Simplify :) 2021-10-04 17:21:37 +07:00
acsfer 8a8df49bc0
Replace `file_exists()` method by `is_file()` 2021-10-04 17:20:17 +07:00
acsfer 3a25183b66 Get `filesize()` if `file_exists()`
Should make sense.
2021-10-01 21:02:38 +07:00
Daniel Kesselberg be3f4edf1f
explicitly close source stream on encryption storage
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2021-09-21 17:42:22 +07:00
Daniel Kesselberg e60a829b42
explicitly close source stream on local storage
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2021-09-21 17:41:59 +07:00
acsfer 9324a74bba Handle files with is_file instead of file_exists
Should fix things like `fread(): read of 8192 bytes failed with errno=21 Is a directory`
2021-08-15 11:55:00 +07:00
Julius Härtl bc3c46362a Make sure that a empty directory can still be deleted when copied from another storage
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-08-13 16:34:48 +07:00
Lukas Reschke 8193a0d1cf
Merge pull request #27923 from nextcloud/remove-flysystem-storage-class
Remove Flysystem storage class
2021-08-10 13:18:46 +07:00
Carl Schwan 28970563a2
Remove some mentions of ownCloud from our api documentation
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-07-29 15:56:30 +07:00
Lukas Reschke 28359571de Remove Flysystem storage class
This seems unused as per https://github.com/nextcloud/server/issues/27768
and may allow us to get rid of one more dependency.

Signed-off-by: Lukas Reschke <lukas@statuscode.ch>
2021-07-12 13:21:06 +07:00
Rid 0469acfb20 Fix scanner mistaking socket files for directories
Signed-off-by: Rid <rid@cylo.io>
2021-06-17 11:44:44 +07:00
John Molakvoæ (skjnldsv) 215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +07:00
Robin Appelman ed2d02d5f1
better cleanup of user files on user deletion
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-27 20:29:51 +07:00
Lukas Reschke 4b4971ab52
Merge pull request #24966 from nextcloud/jknockaert-patch-1
avoid fread on directories and unencrypted files
2021-04-06 13:45:10 +07:00
Roeland Jago Douma f3738eeff7
Merge pull request #25280 from nextcloud/explicit-file-permissions
Set umask before operations that create local files
2021-03-30 21:55:50 +07:00
Robin Appelman aee4caed07
show better error messages when a file with a forbidden path is encountered
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-24 17:13:09 +07:00
Robin Appelman e5dc1a8085
Set umask before operations that create local files
this solves issues where "other php stuff" is messing with the umask

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-23 14:52:31 +07:00
Robin Appelman 7995c6e1dc
dont return private storage interface from public mount interface
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-23 12:25:37 +07:00
Robin Appelman c87b1a50d9 apply object store copy optimization when 'cross storage' copy is within the same object store
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-09 20:31:27 +07:00
Roeland Jago Douma 618805b14f Fix RedundantPropertyInitializationCheck
For #25839

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2021-03-09 09:49:45 +07:00
Robin Appelman b844293536 catch notfound and forbidden exception in smb::getmetadata
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-04 21:34:54 +07:00
dependabot-preview[bot] eb502c02ff
Bump nextcloud/coding-standard from 0.3.0 to 0.5.0
Bumps [nextcloud/coding-standard](https://github.com/nextcloud/coding-standard) from 0.3.0 to 0.5.0.
- [Release notes](https://github.com/nextcloud/coding-standard/releases)
- [Changelog](https://github.com/nextcloud/coding-standard/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nextcloud/coding-standard/compare/v0.3.0...v0.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2021-02-18 13:31:24 +07:00
Robin Appelman d182043e83
explicitly set permissions on newly created folders
this works around any `umask` that might be set and limiting the folder permissions

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-01-20 17:15:57 +07:00
Jasper Knockaert 40fde94b4d
fix even more brackets
Signed-off-by: Jasper Knockaert jasper@knockaert.nl
2021-01-16 14:49:53 +07:00
Jasper Knockaert 252d1ae512
fix brakcets 2021-01-16 14:38:19 +07:00
Jasper Knockaert f9a0380eb1
consolidation of boolean expression 2021-01-16 14:33:44 +07:00
Johannes Leuker ac88bcbd5f Add 'allow Symlinks' as an option to config.php
Signed-off-by: Johannes Leuker <j.leuker@hosting.de>
2021-01-12 10:38:39 +07:00
Morris Jobke 24d436cb60
Remove unneeded casts that were found by Psalm
In preparation of the update of Psalm from 4.2.1 to 4.3.1+ (see https://github.com/nextcloud/server/pull/24521)

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2021-01-11 13:14:41 +07:00
Jasper Knockaert 81e3ae4753
avoid fread on directories and unencrypted files
Reworking the logic in order to first check the filecache and only then reading the fileheader.
This in order to solve #21578.
2021-01-05 11:14:49 +07:00
Julius Härtl 98560c80ea
Properly handle result when copying with failure and throw in tests
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-01-05 09:20:35 +07:00
Christoph Wurst 9ce3ea3368
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-30 14:07:05 +07:00
Christoph Wurst 73c7d0dc81
Bump icewind/streams from 0.7.1 to 0.7.2
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-29 09:07:36 +07:00
Roeland Jago Douma adc4f1a811
Merge pull request #22916 from J0WI/unifiy-links-to-php.net
Unify links to php.net
2020-12-22 09:53:31 +07:00
Julius Härtl 5094e29ebd
Merge pull request #24594 from kofemann/dcache 2020-12-22 09:26:07 +07:00
Tigran Mkrtchyan 4f2dc18f58 storage: update IStorage#file_put_contents docs to match usage
The current phpdoc of IStorage#file_put_contents doesnt corresponds to
it's actual usage in code, e.g.

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2020-12-21 19:04:18 +07:00
Christoph Wurst d89a75be0b
Update all license headers for Nextcloud 21
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-16 18:48:22 +07:00
Tigran Mkrtchyan f3513f3fe4 files: Local#writeStream should use it's own file_put_contents
The OC\Files\Storage\Local#writeStream use system provided file_put_contents.
However, it overrides file_put_contents, thus expects that the default behaviour
can be different.

Use Local#file_put_contents in writeStream to benefit from class specific functionality.

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
2020-12-07 18:11:40 +07:00
aler9 ac0c7a8fe0 Fix file size computation on 32bit platforms
Signed-off-by: aler9 <46489434+aler9@users.noreply.github.com>
2020-12-02 12:05:14 +07:00
John Molakvoæ (skjnldsv) 6a453ac184
Make sure root storage is valid before checking its size
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2020-11-09 08:49:30 +07:00
Morris Jobke dc479aae2d
Improve CertificateManager to not be user context dependent
* removes the ability for users to import their own certificates (for external storage)
* reliably returns the same certificate bundles system wide (and not depending on the user context and available sessions)

The user specific certificates were broken in some cases anyways, as they are only loaded if the specific user is logged in and thus causing unexpected behavior for background jobs and other non-user triggered code paths.

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-11-03 00:13:01 +07:00