* fix: should transform checkout event
* chore: update fuzz dep
* chore: add pos to error info
* fix: clear undo/redo stack when checkingout
* test: update fuzz dep
* test: a new failed test case
* fix: tree transform
* chore: fuzz
* chore: add log
* chore: add more logs
* fix: compose err
* chore: fuzz test dep
* test: a failed tree case
* fix: undo tree event
* fix: do not compare tree position in fuzz
* fix: fuzz rev
* test: a failed tree case
* fix: add tree compose
* chore: add comment
* chore: fuzz
* fix: test
* fix: tree transform
* fix: tree transform index
* fix: sort tree index
* chore: fuzz
* fix: undo/redo remote change effect compose
* bk
* fix: tree undo redo (#385)
* fix: event hint none
* chore: fuzz version
* ci: fuzz
* bk: weird err
* fix: type err
* fix: fractional index between
* fix: wasm counter feature
* test: a new failed case
* fix: recursively create child nodes
* fix: filter empty event
* bk
* bk
* fix: tree undo redo remap
* chore: clean
* bk
* fix: tree need remap first
* fix: tree undo effect
* fix: tree diff calc
* fix: tree fuzz check eq func
* fix: remove EventHint None
* chore: cargo fix
* fix: tree uncreate
* fix: fuzz tree assert only structure
* refactor: rename methods
* fix: movable tree apply delta
* fix: another movable list issue
* chore: fuzz only check 1 actor's history
---------
Co-authored-by: Leon Zhao <leeeon233@gmail.com>
* feat: delta rope support init
* perf: use generic-btree v0.9.0
* refactor: improve readability and maintainability
* fix: fix several issues about composing
* fix: a few more issue about composing deletions
* test: rich text
* fix: cover more edge cases
* refactor: use deltarope for list event
* refactor: replace text delta with DeltaRope
* fix: list fuzz err
* fix: safety issue on insert_many
* chore: refine impl of text delta
* refactor: use Replace instead of insert+del in DeltaItem (#330)
* refactor: use Replace instead of insert+del in DeltaItem
* fix: each deltaitem should have non-zero rle_len
Updated generic-btree dependency to version 0.10.3 and refactored DeltaItem and DeltaRope implementations in loro-delta. Refine compose impl
* fix: update generic-btree to fix the update leaf issue
* chore: lockfile
* chore: clippy fix
* refactor: make composing easier to understand
* refactor: simplify the impl of composing
This PR includes a BREAKING CHANGE.
It enables you to create containers before attaching them to the document, making the API more intuitive and straightforward.
A container can be either attached to a document or detached. When it's detached, its history/state does not persist. You can attach a container to a document by inserting it into an attached container. Once a container is attached, its state, along with all of its descendants's states, will be recreated in the document. After attaching, the container and its descendants, will each have their corresponding "attached" version of themselves?
When a detached container x is attached to a document, you can use x.getAttached() to obtain the corresponding attached container.
This PR implements a new encode schema that is more extendible and more compact. It’s also simpler and takes less binary size and maintaining effort. It is inspired by the [Automerge Encoding Format](https://automerge.org/automerge-binary-format-spec/).
The main motivation is the extensibility. When we integrate a new CRDT algorithm, we don’t want to make a breaking change to the encoding or keep multiple versions of the encoding schema in the code, as it will make our WASM size much larger. We need a stable and extendible encoding schema for our v1.0 version.
This PR also exposes the ops that compose the current container state. For example, now you can make a query about which operation a certain character quickly. This behavior is required in the new snapshot encoding, so it’s included in this PR.
# Encoding Schema
## Header
The header has 22 bytes.
- (0-4 bytes) Magic Bytes: The encoding starts with `loro` as magic bytes.
- (4-20 bytes) Checksum: MD5 checksum of the encoded data, including the header starting from 20th bytes. The checksum is encoded as a 16-byte array. The `checksum` and `magic bytes` fields are trimmed when calculating the checksum.
- (20-21 bytes) Encoding Method (2 bytes, big endian): Multiple encoding methods are available for a specific encoding version.
## Encode Mode: Updates
In this approach, only ops, specifically their historical record, are encoded, while document states are excluded.
Like Automerge's format, we employ columnar encoding for operations and changes.
Previously, operations were ordered by their Operation ID (OpId) before columnar encoding. However, sorting operations based on their respective containers initially enhance compression potential.
## Encode Mode: Snapshot
This mode simultaneously captures document state and historical data. Upon importing a snapshot into a new document, initialization occurs directly from the snapshot, bypassing the need for CRDT-based recalculations.
Unlike previous snapshot encoding methods, the current binary output in snapshot mode is compatible with the updates mode. This enhances the efficiency of importing snapshots into non-empty documents, where initialization via snapshot is infeasible.
Additionally, when feasible, we leverage the sequence of operations to construct state snapshots. In CRDTs, deducing the specific ops constituting the current container state is feasible. These ops are tagged in relation to the container, facilitating direct state reconstruction from them. This approach, pioneered by Automerge, significantly improves compression efficiency.
* feat: use resolved diff as event
* feat: wasm ValueOrContainer event
* fix: cargo fix
* fix: avoid state clone
* chore: add resolve event bench
* test: add handler in event test
* refactor: use resolved as external diff
* chore: cargo fix
* fix: typescript loro value type
* fix: use Arc::new_cyclic
* refactor: bring back sub container
* feat: new tree state
* fix: emit meta event
* fix: semantic tree event
* fix: diff calc bring_back
* chore: clear comments
* fix: merge
* fix: tree snapshot
* fix: filter empty bring back
* feat: tree add external diff
* fix: imported changes were not mergeable (#147)
* fix: imported changes were not mergeable
now the small encoding size is supported in example
* fix: stupid err in richtext checkout
* fix: rle oplog encode err
- support pending changes
- start counters were wrong
* fix: utf16 query err (#151)
* fix: tree movable node lamport
* fix: merge
* perf: bring back
* doc: add deep value meta doc
* refactor: bring back only when record diff
---------
Co-authored-by: Zixuan Chen <remch183@outlook.com>
- Allow changes to be merged when possible. This makes realtime collaboration more efficient with Loro.
- Refactor the code to make modifications of changes in oplog in one place
- Optimize the diff calculation so that it doesn't have to go back to the beginning of the change.
Note: we still keep the invariants that dependency pointers in Loro always point to the last op in a change
* feat: tree state
* feat: tree value
* feat: tree handler
* fix: tree diff
* test: fuzz tree
* feat: tree snapshot
* fix: tree default value
* fix: test new node
* fix: tree diff
* fix: tree unresolved value
* fix: tree fuzz
* fix: tree fuzz move
* fix: sort by tree id
* fix: tree diff sorted by lamport
* fix: sort roots before tree converted to string
* fix: rebase main
* fix: tree fuzz
* fix: delete undo
* fix: tree to json children sorted
* fix: diff calculate
* fix: diff cycle move
* fix: tree old parent cache
* feat: cache
* fix: local op add tree cache
* fix: don't add same tree move to cache
* fix: need update cache
* feat: new cache
* bench: add checkout bench
* chore: clean
* fix: apply node uncheck
* perf: lamport bound
* fix: calc old parent
* feat: tree wasm
* fix: change tree diff
* fix: tree diff retreat
* fix: tree diff should not apply when add node
* feat: new tree loro value
* chore: typo
* fix: tree deep value
* fix: snapshot tree index -1
* fix: decode tree snapshot use state
* fix: release state lock when emit event
* fix: tree node meta container
* fix: need set map container when covert to local tree op
* fix: tree value add deleted
* fix: more then one op in a change
* fix: tree fuzz deleted equal
* fix: tree calc min lamport
* feat: tree encoding v2
* doc: movable tree
* fix: test tree meta
* test: remove import bytes check
* refactor: diff of text and map
* refactor: del span
* perf: tree state use deleted cache
* fix: some details
* fix: loro js tree create
* feat: add un exist tree node
* bench: tree depth
* fix: check out should emit event
* refactor: event
* fix: fuzz err
* fix: pass all tests
* fix: fuzz err
* fix: list child cache err
* chore: rm debug code
* fix: encode enhanced err
* fix: encode enchanced
* fix: fix several richtext issue
* fix: richtext anchor err
* chore: rm debug code
* fix: richtext fuzz err
* feat: speedup text snapshot decode
* perf: optimize snapshot encoding
* perf: speed up decode & insert
* fix: fugue span merge err
* perf: speedup delete & id cursor map
* fix: fugue merge err
* chore: update utils
* fix: fix merge
* fix: return err apply op
* fix: fix merge
* fix: get map container as tree meta
* feat: checkout to frontiers
* feat: record timestamp
* fix: use unicode len by default for text
now "你好" has length of 2 instead of 6
* chore: rm dbg!
