ea678c0280
feat: support migrating an instance to sharding
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-02-25 21:43:54 +07:00
8a2437927d
fix: Initialize lastConnectionCheck after first connection
...
We are checking whether the DB connection is alive once every 30 seconds. But when we are lacking the last check time, we are skipping the check and reconnect logic. This is causing the reconnect logic to never fire in those cases.
It seems to me that "those cases", are actually always the case, as upon initialization, we are not using the proper connection name to store the time.
In the `connect()` logic, when `$this->_conn` is null, `$this->getConnectionName()` is returning `replica`, so `$this->lastConnectionCheck` will be equal to `['replica' => time()];`
60711ea4cf/lib/private/DB/Connection.php (L215-L221)60711ea4cf/lib/private/DB/Connection.php (L891-L893)2b6d7bf65f/doctrine/dbal/src/Connections/PrimaryReadReplicaConnection.php (L136-L139)60711ea4cf/lib/private/DB/Connection.php (L874-L880)https://github.com/nextcloud/server/pull/41819
Signed-off-by: Louis Chemineau <louis@chmn.me>
2025-02-25 11:07:34 +07:00
a8f46af20f
chore: Add proper deprecation dates where missing
...
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2024-09-20 00:46:03 +07:00
9836e9b164
chore(deps): Update nextcloud/coding-standard to v1.3.1
...
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-19 14:21:20 +07:00
edaa4d1432
fix(DB): do not assume sharding is always enabled
...
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2024-08-29 19:55:10 +07:00
fc05a67f19
fix: only allow pre-defined shards
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-08-28 10:21:19 +07:00
62f8b6517f
feat: implement distributing partitioned queries over multiple shards
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-08-28 10:21:19 +07:00
f5b3486744
feat: add option to automatically partition queries by specific tables
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-08-28 10:18:53 +07:00
bdcfe5b8a9
fix(db): Increase log level for very slow transactions
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-08-27 10:12:14 +07:00
af6de04e9e
style: update codestyle for coding-standard 1.2.3
...
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2024-08-25 19:34:58 +07:00
7fbb981877
feat: add additional logging for database errors
...
including the stack trace of the current database transaction
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-07-24 15:35:07 +07:00
b656edc47c
fix(db): Fix md5 for oracle >= 20
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2024-07-19 14:13:56 +07:00
eeb6ddb176
fix(db): Deprecate `IExpressionBuilder::or()` and `IExpressionBuilder::and()` without parameters
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2024-07-19 11:21:16 +07:00
829f2b9bc7
fix(db): Promote the use of `getDatabaseProvider` to reduce the impage of removed upstream platforms
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2024-07-19 11:21:14 +07:00
c84580d53a
fix(db): `Doctrine\DBAL\Connection::executeUpdate()` is deprecated
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2024-07-19 11:21:12 +07:00
a4c1d7291f
fix(db): Use `createSchemaManager()` method as `getSchemaManager()` is deprecated
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2024-07-19 11:21:11 +07:00
258bb03cf5
Merge branch 'master' into refactor/OC-Server-getSecureRandom
...
Signed-off-by: John Molakvoæ <skjnldsv@users.noreply.github.com>
2024-05-30 14:24:22 +07:00
3bfba2042c
fix(db): Prevent two connections for single node databases
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-05-28 10:38:36 +07:00
b0f5c759a8
Merge pull request #39589 from nextcloud/enh/set-nest-transactions-with-savepoints
...
set Doctrine to use nest transactions with savepoints
2024-05-27 16:53:53 +07:00
48b47f7903
fix(db): Log loong transaction times at debug level
...
Signed-off-by: Josh <josh.t.richards@gmail.com>
2024-05-25 09:07:43 +07:00
dae7c159f7
chore: Add SPDX header
...
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
2024-05-24 13:11:22 +07:00
d6be80ceaf
feat: add option to add backtrace to the query logger
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-05-01 18:12:16 +07:00
8c10c78099
feat: add request id as comment to all queries
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-04-17 16:36:24 +07:00
5f519a2e7d
chore(db): move to OCP\Server
...
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
2024-04-13 18:29:22 +07:00
ec5133b739
fix: Apply new coding standard to all files
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-04-02 14:16:21 +07:00
2a0815401e
feat(db): set Doctrine to use nest transactions with savepoints
...
Using nested transactions without savepoints is actually deprecated by Doctrine:
https://www.doctrine-project.org/projects/doctrine-dbal/en/current/reference/transactions.html#transaction-nesting
Without savepoints, a nested transaction can be rollbacked but not
handled properly in the "real" transaction, leading to the following
error:
Transaction commit failed because the transaction has been marked for rollback only.
Ref
https://github.com/nextcloud/server/pull/36528#issuecomment-1639913965
(and possibly) https://github.com/nextcloud/server/issues/38902#issuecomment-1598075391
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2024-02-27 14:27:11 +07:00
d52ebaa7cd
Merge pull request #43297 from nextcloud/fix/db/dirty-read-cool-off
...
fix(db): Let dirty writes cool off
2024-02-19 10:53:05 +07:00
1f46e4b854
fix(db): Let dirty writes cool off
...
We can assume that after a few seconds a read will be clean again.
This is helpful for false warnings in long running processes.
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-02-19 10:13:24 +07:00
cf17a20835
fix(db): Give dirty read exception a message
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-02-15 09:14:47 +07:00
aef28e191a
fix(db): Unify long transaction log/exception message
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-02-13 08:17:49 +07:00
97152de9bf
fix(db): Execute dirty reads on the primary node
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-30 12:17:39 +07:00
911ab393c0
feat(db): Make dirty query logging available in production
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-30 10:31:03 +07:00
033a654389
Merge pull request #42929 from nextcloud/fix/db/transacted-read-not-dirty
...
fix(db): Do not log transacted reads as dirty read
2024-01-23 09:34:20 +07:00
ab0d7c007d
fix(db): Remove very verbose dirty query logs
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-18 18:33:07 +07:00
f54b08c224
fix(db): Do not log transacted reads as dirty read
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-18 13:37:36 +07:00
296096e069
fix: Add reconnect check in case of timeouts on the db side
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2024-01-12 14:38:01 +07:00
c17c42a0b2
feat: First attempt to track dirty tables after writes and switch back to replicas if reads go to other tables
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
debug: error log
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2024-01-12 11:02:50 +07:00
3e60092b7d
feat: Add logging for transaction time
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2024-01-12 11:02:50 +07:00
79c4986354
enh: Implement PrimaryReadReplicaConnection
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-12-16 20:27:37 +07:00
aa5f037af7
chore: apply changes from Nextcloud coding standards 1.1.1
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2023-11-23 10:36:13 +07:00
160298c556
fix(mysql): Remove custom MySQL workaround from 2015
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-10-06 15:29:25 +07:00
b202b139dd
fix(postgres): Remove old Postgres 9.4 workaround
...
Postgres 10 is the minimum in the meantime
and doctrine/dbal fixed this in 2.6.0 already
ref https://github.com/doctrine/dbal/pull/2614
Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-10-06 15:29:25 +07:00
ad839dbb0a
fix(sqlite): Remove no longer required autoincrement fix
...
- I installed current master and exported the schema as SQL
- Then I went to this branch, removed the content of the run() method (so made it no-op)
- I installed again and exported the schema as SQL
- The files are exactly the same, so whatever we tried to fix was fixed since 2015 in doctrine dbal
Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-10-06 15:29:25 +07:00
e966cfbc4b
Cleanup plsam issues in DB/ContactsManager and Console
...
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2023-09-05 09:20:26 +07:00
1395a53602
Refactor `OC\Server::getSecureRandom`
...
Signed-off-by: Andrew Summers <18727110+summersab@users.noreply.github.com>
2023-08-29 21:32:40 +07:00
997c2a2a79
fix DBAL exception handling in setValues
...
This seems to be a left over after abstracting DBAL. Nowadays,
IQueryBuilder::executeStatement() only throws a \OCP\DB\Exception, where
previously original DBAL exceptions where thrown. These are now wrapped,
the orignal classes are now mapped to a reason.
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-03-31 17:01:17 +07:00
8a16e7e8a1
Fix more psalm issues in OC\DB
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-11-21 11:30:20 +07:00
378630f8a0
Fix psalm issues following doctrine/dbal bump
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-11-21 10:56:15 +07:00
a2a7150d6d
Migrate Repair events to OCP\EventDispatcher\Event
...
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-08-25 16:15:48 +07:00
1537c3d23b
record backtrace when profiling db requests
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-04 18:12:30 +07:00
Robin Appelman
Louis Chemineau
Ferdinand Thiessen
provokateurin
Arthur Schiwon
Christoph Wurst
Daniel Kesselberg
Joas Schilling
John Molakvoæ
Thomas Citharel
Josh
Andy Scherzinger
Git'Fellow
Côme Chilliet
Thomas Citharel
Christoph Wurst
Julius Härtl
Carl Schwan
Andrew Summers