Sorry to bother you now.
There are two pointers in ybdb: regular_db and intents_db. I have two questions to ask:
- Every time you read or write data, write the Intents_db first, and then migrate the Data migration to the regulardb after the entire transaction is completed? If this mode is used, how does migration occur? Do data replays? Or still directly copy data between dbs?
- Or is it just an intermediate state for storing transaction control, or it is just a distributed lock, it do not store specific data?
Hi, don’t be sorry, forums are there for questions and that’s an interesting one
The answer is close to 1) but let me add more details
In the general case, all transaction read + write intents go to the IntentsDB first, as provisional records. This includes new rows, new values for columns, read or write lock information, transaction control (begin, commit, savepoints…). The other transactions look also there, and act depending on the transaction status and isolation level.
When the transaction is committed, the transaction status is updated and that’s sufficient to acknowledge the commit. In the background, each tablet will move the committed transactional records to the RegularDB and assign the transaction timestamp to them so that future reads will not have to check the transaction status.
Note that there’s a “fast path” for single-shard transactions. They can go directly to the RegularDB. This is also called ‘non-transactional’ in the sense that ACID properties are single-row only. This is transparent: YugabyteDB detects it. Or can be set for special operations like bulk load.
Thanks for your reply!
Excuse me, how does the ‘each tablet will move the committed transactional records to the RegularDB’ moved? Is it a write operation? Or the movement of record files?
Thank you very much.
Yes, normal write operations in rocksdb.
Thanks for your reply.
If there is a large amount of data, will it affect normal read and write operations’s performance?
@ZhenNan2016 Yes it affects read (because there’s two RocksDB to lookup) and write operations (because there’s additional writes) but there’s no choice if the transaction is multi-shard. All database has some intermediate writes (rollback segments, tuple copy,…) and post-commit cleanup (delayed cleanout, vacuum,…) because of SQL transaction complexity.
However, YugabyteDB detects when the IntentsDB can be bypassed (single-row transactions) and use a “fast path”. I have put an example, tracing all write operations, in a new blog post:
YugabyteDB fast path write
ok，I got it now. Thank you very much.