Commit graph

99 commits

Author SHA1 Message Date
Leon Zhao
4414053a82
feat: add FFI for Loro (#420)
* chore: init ffi

* feat: impl doc and LoroList

* feat: impl containers

* feat: unknown container

* feat: event ffi

* chore: clean

* feat: ffi undo manager

* chore: cargo fix

* chore: cargo fix

* fix: ffi value or container

* fix: ffi arc

* fix: is attached for movable list

* bk

* feat: all LoroDoc func

* feat: refine vv

* feat: ffi frontiers

* feat: ffi awareness

* fix: merge
2024-09-29 07:41:59 +08:00
Zixuan Chen
812de54ad0
refactor: replace all .lock() with .try_lock()
to avoid deadlock when using Loro with multi-thread
2024-09-27 02:43:00 +08:00
Zixuan Chen
35854743f0
fix: checkout into middle of marks 2024-09-25 23:00:01 +08:00
Zixuan Chen
672062c9d1
fix: gc snapshot error (#481)
it should retain all containers created after the `from` version
2024-09-25 22:35:29 +08:00
Zixuan Chen
66047f7909
feat: update text by line (#480) 2024-09-25 20:50:36 +08:00
Zixuan Chen
78aaa4080d
Fix tree cache checkout issue and add a few invariance checks (#479)
* test: test failed if use non-cached diff calc

* fix: tree diff calc retreat cache current vv

* feat(wasm): commit message & get pending ops length

* bk

* chore: add tree one doc fuzz

* fix: fix the problem and added a few checks

* chore: rm debugging code

* fix: encode snapshot when detached

---------

Co-authored-by: leeeon233 <leeeon233@gmail.com>
2024-09-25 20:50:10 +08:00
Leon Zhao
261cdda674
fix: get correct tree_cache current vv when retreating (#476)
* test: test failed if use non-cached diff calc

* fix: tree diff calc retreat cache current vv

* fix: better tree cache current vv

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-09-25 20:38:26 +08:00
Zixuan Chen
bef39ce6b5
Feat: allow editing on detached mode (#473)
* refactor: add detached editing config and prepare the architecture for editing detached doc

* feat: subscribe for peer id change

* fix: undo after checkout & add tests for detached editing

* test: add fuzzer for detached editing

* feat: expose detached editing configure to wasm

* test: add wasm test for detached editing
2024-09-24 11:16:59 +08:00
Zixuan Chen
1082af0d46
Fix: get tree's alive children correctly (#474)
* test: add a failed case

* fix: get tree's alive children correctly
2024-09-24 10:27:57 +08:00
Leon Zhao
3fe619bc81
refactor: use hierarchy value for tree value (#470)
* refactor: use hierarchy value for tree value

* fix: get_container_deep_value_with_id

* feat: get tree nodes by flat format
2024-09-23 09:05:12 +08:00
Leon Zhao
aab6730b4d
fix: tree undo when converting movement to creation, new target id should be used (#468)
* test: a checkout event err

* fix: tree undo convert moving to creating should use new target tree id

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-09-20 16:51:37 +08:00
Leon Zhao
7f9bfa8a85
refactor: tree contains & isDeleted (#467) 2024-09-19 11:21:54 +08:00
Zixuan Chen
1ea2a24d74
fix: export snapshot error on a gc doc 2024-09-19 11:08:34 +08:00
Zixuan Chen
8dcb619147
fix: raise error if perform action on a deleted container (#465)
- Add ContainerDeleted error variant to LoroError
- Implement is_deleted() method for container handlers
- Check for deleted containers before applying operations
- Add dead_containers_cache to DocState to track deleted containers
- Update apply_diff to optionally clear dead container cache
- Add tests for handling operations on deleted containers
- Implement checkout to reset container deleted cache
- Minor code cleanup and typo fixes
2024-09-17 12:09:20 +08:00
Zixuan Chen
2f38708a50
Fix a tree checkout issue on GcSnapshot (#464)
* test: a failed test case

* fix: tree checkout init err

* fix: export from

* chore: add cov

* fix: movable list undo/redo apply delta err

* refactor: refactor by o1-preview 🤯

* refactor: extract arguments

* test: add cov

* fix: use correct method to get was_alive for tree node

* fix: tree event was alive err

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-09-17 00:19:29 +08:00
Zixuan Chen
0f1df49b9e
feat: add state only snapshot & refine check slow test 2024-09-13 22:35:43 +08:00
Zixuan Chen
c18cec15a9
fix: lazy load dag node 2024-09-13 18:39:44 +08:00
Zixuan Chen
38d1c279d9
feat: add new export api to wasm 2024-09-12 22:16:55 +08:00
Zixuan Chen
9040747620
fix: ensure all alive containers are encoded in doc state snapshot 2024-09-12 20:53:50 +08:00
Zixuan Chen
a9d4de7a18
Test: Add fuzzing tests for gc mode correctly & fix several failed cases (#461)
* test: fuzz gc correctly

* fix: lots of gc snapshot issues

* fix: vv to frontiers

* test: add an arbtest for gc fuzzing tests

* test: fix a few test issues

* fix: apply diff of a dangling container that was deleted before trimmed version
2024-09-12 20:07:57 +08:00
leeeon233
5450e511d0
chore: merge gc 2024-09-12 17:24:40 +08:00
Zixuan Chen
db1111cf83
test: fix gc fuzz preprocess 2024-09-11 23:56:37 +08:00
Zixuan Chen
81c9f3aacc
Merge branch 'test-add-gc-fuzz-test' into feat-gc 2024-09-11 23:36:16 +08:00
Zixuan Chen
88f9e5fa45
test: add fuzzing tests for gc snapshot 2024-09-11 23:34:05 +08:00
Zixuan Chen
21e3ffea45
perf: refine state fast snapshot & fix a few tree event issues (#459)
* perf: refine state fast snapshot

* fix: tree apply diff err

* fix: get child index

* fix: use better tree event
2024-09-11 22:54:14 +08:00
Zixuan Chen
d7778f15fd
test: fix a few outdated test 2024-09-11 11:05:42 +08:00
Zixuan Chen
4a27a0645f
Merge branch 'dev' into feat-gc 2024-09-11 10:54:29 +08:00
Leon Zhao
07671ea9fd
feat: add old parent and old index in tree diff (#452)
* feat: add old parent in tree diff

* chore: enable ci

* feat: add old_index to tree diff

* fix: new fractional index config

* fix: cargo fix

* fix: add FractionalIndexNotEnabled error

* fix: move config to tree state

* fix: error string

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-09-09 16:16:02 +08:00
Leon Zhao
cb134fc7c7
feat: add with fractional index config (#442)
* feat: add `with_fractional_index` config

* fix: share config state
2024-09-05 16:03:47 +08:00
Zixuan Chen
68ffd31ba0
Feat when exporting gc snapshot with short history, don't encode the latest state (#445)
* feat: export gc snapshot without latest state when ops len is small

* test: add a test for time tracker usage

* chore: add gc snapshot bench

* chore: simplify

* test: record timestamp for time tracker
2024-09-04 21:00:17 +08:00
Leon Zhao
c1620fdb37
feat: memkv export import all (#422)
* feat: sstable

* fix: add magic number version

* feat: new mem kv store based sstable

* feat: binary_search

* fix: sstable iter scan

* fix: new mem kv

* feat: add cache for sstable

* fix: encode schema comment

* fix: sstable iter scan

* chore: clean

* fix: export all

* fix: sstable scan bound

* fix: sstable iter scan next==prev

* fix: merge iter next_back

* fix: mem kv export

* chore: clean

* fix: prev to key

* fix: prev find block

* fix: get prev block idx

* refactor: kv store

* fix: checksum when import

* fix: meta first last key

* Revert "fix: meta first last key"

This reverts commit a069c1ed37.

* fix: skip empty iter

* fix: remove key from large block

* chore: comment

* feat: compress block

* fix: remove key in large block

* chore: const

* doc: intro sstable encode

* test: add kv store fuzz

* style: format file

* feat: add fuzz to kv store (#428)

* fix: kv fuzzer

* fix: debug

* bk

* fix: block iter next back

* fix: block prev iter left = next idx

* feat: move kv store a crate

* fix: remove value len from normal block

* doc: sstable format

* test: add more test

* test: add test

* feat: new merge iter

* chore: revert

* fix: rename next back

* fix: rename mem sstable

* fix: rename to mem

* fix: use Bytes as key

* fix: use simple merge iter

* feat: compress option

* fix: remove empty iter

* style: refine some impl details

* fix: large block compress

* feat: use write read for encode

* doc: refine doc

* fix: simplify the first chunk

* feat: import many times

* refactor: refine styles

* test: fuzz merge iter

* fix: rename peek_xxx()

* fix: better sstable iter inner

* fix: use mem kv store

* pref: mem kv store

* perf: export mem kv

* chore: clean

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-08-30 11:44:34 +08:00
Zixuan Chen
e084eca580
Merge branch 'main' into dev 2024-08-25 21:12:07 +08:00
Leon Zhao
2f0a3ab92f
fix: delete the **bring back** tree node from the undo container remap (#423) 2024-08-25 20:34:05 +08:00
Zixuan Chen
8fbfea4bda
test: skip undo-redo assertion in fuzz for now 2024-08-25 19:42:57 +08:00
Zixuan Chen
95bec549d7
feat: support commit message (#429)
* feat: support commit message

* test: refine test
2024-08-24 14:15:40 +08:00
Zixuan Chen
c2c727de6f
test: use less strict tree tracker for fuzzing 2024-08-23 23:36:28 +08:00
Zixuan Chen
5ba3fba3f3
Fix real_id setting err in tracker (#427)
* test: add a failed test case

* fix: tracker real_id setting err
2024-08-22 21:42:00 +08:00
Zixuan Chen
bcabf682e9
test: a failed case 2024-08-22 13:25:19 +08:00
Zixuan Chen
3f2c7d9498
fix: ignore undo failures
It could happen when using movable list in collab env. But it's extremely rare.
2024-08-21 23:27:43 +08:00
Leon Zhao
7402ff6788
chore: new fuzz test 2024-08-21 22:56:58 +08:00
Zixuan Chen
86aeaad97d
fix: movable list apply diff err 2024-08-21 18:38:46 +08:00
Zixuan Chen
f142afd431
test: minimize 2024-08-21 18:21:25 +08:00
Leon Zhao
2b2270ab99
test: add test 2024-08-21 18:21:25 +08:00
Zixuan Chen
37046f8e41
chore: rm log 2024-08-20 19:54:37 +08:00
Zixuan Chen
be0f32eed6
fix: iter change error 2024-08-20 19:50:47 +08:00
Zixuan Chen
a7d9f3f5d3
test: add a failed test case 2024-08-20 08:34:11 +08:00
Zixuan Chen
544a8ebfcc
test: add a multithread test case minify method 2024-08-20 01:41:44 +08:00
Leon Zhao
79e9f29656
fix: delete the **bring back** tree node from the undo container remap (#423) 2024-08-19 21:52:30 +08:00
Zixuan Chen
c8bd656658
test: avoid super slow fuzzing unit 2024-08-19 17:15:08 +08:00
Zixuan Chen
756f0c9ecf
fix: fix a few place that violate the dag's invariants 2024-08-19 15:12:20 +08:00