The purpose of the regular_db and the intents_db

General
1

Sorry to bother you now.
There are two pointers in ybdb: regular_db and intents_db. I have two questions to ask:

  1. Every time you read or write data, write the Intents_db first, and then migrate the Data migration to the regulardb after the entire transaction is completed? If this mode is used, how does migration occur? Do data replays? Or still directly copy data between dbs?
  2. Or is it just an intermediate state for storing transaction control, or it is just a distributed lock, it do not store specific data?
2

Hi, donā€™t be sorry, forums are there for questions and thatā€™s an interesting one :slight_smile:
The answer is close to 1) but let me add more details

In the general case, all transaction read + write intents go to the IntentsDB first, as provisional records. This includes new rows, new values for columns, read or write lock information, transaction control (begin, commit, savepointsā€¦). The other transactions look also there, and act depending on the transaction status and isolation level.
When the transaction is committed, the transaction status is updated and thatā€™s sufficient to acknowledge the commit. In the background, each tablet will move the committed transactional records to the RegularDB and assign the transaction timestamp to them so that future reads will not have to check the transaction status.

Note that thereā€™s a ā€œfast pathā€ for single-shard transactions. They can go directly to the RegularDB. This is also called ā€˜non-transactionalā€™ in the sense that ACID properties are single-row only. This is transparent: YugabyteDB detects it. Or can be set for special operations like bulk load.

3

Hi, @FranckPachot
Thanks for your reply!
Excuse me, how does the ā€˜each tablet will move the committed transactional records to the RegularDBā€™ moved? Is it a write operation? Or the movement of record files?
Thank you very much.

4

Yes, normal write operations in rocksdb.

1 Like
5

Hi, @dorian_yugabyte
Thanks for your reply.
If there is a large amount of data, will it affect normal read and write operationsā€™s performance?

6

@ZhenNan2016 Yes it affects read (because thereā€™s two RocksDB to lookup) and write operations (because thereā€™s additional writes) but thereā€™s no choice if the transaction is multi-shard. All database has some intermediate writes (rollback segments, tuple copy,ā€¦) and post-commit cleanup (delayed cleanout, vacuum,ā€¦) because of SQL transaction complexity.
However, YugabyteDB detects when the IntentsDB can be bypassed (single-row transactions) and use a ā€œfast pathā€. I have put an example, tracing all write operations, in a new blog post:
YugabyteDB fast path write

1 Like
7

okļ¼ŒI got it now. Thank you very much.