Zixuan Chen
ea2beae48e
refactor: make gc store accessible for history cache
2024-08-26 23:01:42 +08:00
Zixuan Chen
87bc464f3b
fix: make exporting & importing gc snapshot more correctly
2024-08-26 22:33:44 +08:00
Zixuan Chen
4e64051c58
fix: detect importing outdated updates on gc doc
2024-08-26 22:14:23 +08:00
Zixuan Chen
cfe8652415
feat: basic gc mode impl
2024-08-26 21:21:33 +08:00
Zixuan Chen
82d035442c
feat: add new export from mode for new storage format
2024-08-26 17:10:38 +08:00
Zixuan Chen
f293a796f2
Merge branch 'dev' into feat-gc
2024-08-25 21:14:04 +08:00
Zixuan Chen
e084eca580
Merge branch 'main' into dev
2024-08-25 21:12:07 +08:00
Leon Zhao
2f0a3ab92f
fix: delete the **bring back** tree node from the undo container remap ( #423 )
2024-08-25 20:34:05 +08:00
Zixuan Chen
4f338dc14d
refactor: org fast snapshot
2024-08-25 19:27:46 +08:00
Zixuan Chen
03c3e0b0b8
refactor: rm arc in tree hist cache
2024-08-24 15:05:38 +08:00
Zixuan Chen
ca9298507e
refactor: make tree history cache fields private
2024-08-24 15:02:59 +08:00
Zixuan Chen
1812caea65
Refactor: use kv internally for docstate ( #426 )
...
* refactor: use kv in state
* refactor: do not load the state into the inner fxhashmap if not needed
* refactor: calc offset without unsafe code
* style: replace unsafe code
2024-08-24 14:16:06 +08:00
Zixuan Chen
95bec549d7
feat: support commit message ( #429 )
...
* feat: support commit message
* test: refine test
2024-08-24 14:15:40 +08:00
Zixuan Chen
81515ba6d3
refactor: reuse the peer id in map fast snapshot if we can ( #430 )
2024-08-24 14:15:10 +08:00
Zixuan Chen
c2c727de6f
test: use less strict tree tracker for fuzzing
2024-08-23 23:36:28 +08:00
Zixuan Chen
5ba3fba3f3
Fix real_id setting err in tracker ( #427 )
...
* test: add a failed test case
* fix: tracker real_id setting err
2024-08-22 21:42:00 +08:00
leeeon233
a6f8c9c2d2
fix: tree apply diff when before dead after alive
2024-08-22 16:44:52 +08:00
Zixuan Chen
3f2c7d9498
fix: ignore undo failures
...
It could happen when using movable list in collab env. But it's extremely rare.
2024-08-21 23:27:43 +08:00
Zixuan Chen
86aeaad97d
fix: movable list apply diff err
2024-08-21 18:38:46 +08:00
Zixuan Chen
5a03e7fa3a
fix: issue when getting container value
2024-08-21 12:02:35 +08:00
Zixuan Chen
d689db619f
feat: get path to container
2024-08-21 12:02:07 +08:00
Zixuan Chen
be0a7979cd
chore: fix type err
2024-08-21 11:30:03 +08:00
Zixuan Chen
0717aadd79
feat(dev): add a way to analyze the containers of the doc
2024-08-21 11:18:06 +08:00
Zixuan Chen
be0f32eed6
fix: iter change error
2024-08-20 19:50:47 +08:00
Zixuan Chen
5b7ac90e54
refactor: simplify state code
2024-08-20 00:20:25 +08:00
Leon Zhao
79e9f29656
fix: delete the **bring back** tree node from the undo container remap ( #423 )
2024-08-19 21:52:30 +08:00
Zixuan Chen
1d58d69f28
test: add load + get value bench
2024-08-19 21:30:03 +08:00
Zixuan Chen
8ac4322bb6
test: add text bench
2024-08-19 21:15:48 +08:00
Zixuan Chen
c8bd656658
test: avoid super slow fuzzing unit
2024-08-19 17:15:08 +08:00
Zixuan Chen
756f0c9ecf
fix: fix a few place that violate the dag's invariants
2024-08-19 15:12:20 +08:00
Zixuan Chen
65b9263231
refactor: add external_vv to change_store
2024-08-19 12:01:27 +08:00
Zixuan Chen
5f70310681
chore: fix warning - rm a redundant field
2024-08-19 11:39:35 +08:00
Zixuan Chen
ea5f91f6a6
chore: fix typos
2024-08-19 11:36:59 +08:00
Zixuan Chen
0619d7ae89
refactor: refine history cache (use type as constrain)
2024-08-19 11:34:34 +08:00
Zixuan Chen
c206365455
chore: fix warnings
2024-08-19 11:26:10 +08:00
Zixuan Chen
3e70302d51
refactor: org change_store
2024-08-19 10:59:21 +08:00
Zixuan Chen
b06f617d3b
fix: frontiers_to_vv err
2024-08-18 21:45:11 +08:00
Zixuan Chen
6fd6aeb13d
fix: map event error
2024-08-18 20:40:53 +08:00
Zixuan Chen
b5eb176b3f
fix: mark flush on push_change
2024-08-18 18:32:41 +08:00
Zixuan Chen
6f9329adc2
fix: change_store loading cache error
2024-08-18 18:15:53 +08:00
Zixuan Chen
21c915163b
fix: avoid dag internal lock issue
2024-08-18 16:38:44 +08:00
Zixuan Chen
32922d20f5
fix: potential child parent link issue & map entry idlp bug
2024-08-18 16:23:17 +08:00
Zixuan Chen
3ba8ad5f12
fix: getting parent for root container error
2024-08-18 15:07:56 +08:00
Zixuan Chen
c97f1eec58
fix: tree snapshot encode decode err
2024-08-18 14:40:23 +08:00
Zixuan Chen
0c59bd2dcc
refactor: simplify dag and inserting change
2024-08-18 14:34:27 +08:00
Zixuan Chen
4df37ba170
fix: tree fast snapshot err
2024-08-18 14:32:39 +08:00
Zixuan Chen
fbd5d6725a
fix: bugs related to fast snapshot
2024-08-18 00:54:27 +08:00
Zixuan Chen
6c88847756
feat: now support treating fast snapshot as updates
2024-08-18 00:31:18 +08:00
Zixuan Chen
d8e2cdb756
refactor: rm outdated code
2024-08-17 23:53:18 +08:00
Zixuan Chen
c23e29ddb1
perf: replace md5 with xxhash for checksum
2024-08-17 23:32:09 +08:00
Zixuan Chen
1fc4b01f56
refactor: refine change store inner
2024-08-17 22:17:04 +08:00
Zixuan Chen
24a4f9839e
fix: add text example for bench & fix a few related bugs
2024-08-17 22:00:39 +08:00
Zixuan Chen
6126906ec2
perf: make idlp to id much faster
...
by using binary search on top of .range
2024-08-17 21:16:44 +08:00
Zixuan Chen
89327c3d68
feat: fast snapshot mode init
2024-08-17 19:04:19 +08:00
Zixuan Chen
f2d02f9fca
fix: container store fast snapshot
2024-08-17 16:52:46 +08:00
Zixuan Chen
24583ba33a
refactor: refine parent info register
2024-08-17 15:05:54 +08:00
Zixuan Chen
a7816af173
fix: lock issues
2024-08-16 23:52:58 +08:00
Zixuan Chen
a134bf88e3
feat: make dag able to lazy load the graph
2024-08-16 23:39:42 +08:00
Zixuan Chen
14178eeb2c
refactor: move handle new change inside loro_dag
2024-08-16 15:29:51 +08:00
Zixuan Chen
7a460414d4
fix: get lamport lte
...
fix failed wasm tests
2024-08-16 12:14:07 +08:00
Zixuan Chen
59e31c1370
refactor: provide better encapsulation for dag
...
make all the fields private
2024-08-16 11:40:29 +08:00
Zixuan Chen
35f0f811eb
refactor: move dag logic to dag mod
2024-08-16 10:56:35 +08:00
Zixuan Chen
cb3458b862
docs: leave some comments
2024-08-16 10:47:56 +08:00
Zixuan Chen
dd5c1bea72
refactor: reduce op inner content size by 1/6
2024-08-16 10:45:04 +08:00
Zixuan Chen
c0a8d4fc36
refactor: rm the unsound workaround and just use clone
2024-08-15 22:48:39 +08:00
Zixuan Chen
ea20f62d31
fix: make it work
...
but it may be unsound
2024-08-15 22:21:57 +08:00
Zixuan Chen
6568af86b9
chore: bk did not fix lifetime issue
2024-08-15 18:12:55 +08:00
Zixuan Chen
e05d991954
refactor: rm the need for refresh frontiers
2024-08-15 10:33:46 +08:00
Zixuan Chen
0aa534a9ba
chore: save frontiers to kv as well
2024-08-15 08:44:35 +08:00
Zixuan Chen
de8f9e38aa
refactor: change store internal
2024-08-15 08:28:11 +08:00
Zixuan Chen
957a002741
refactor: org tree state
2024-08-14 17:51:45 +08:00
Zixuan Chen
e19e1af543
perf: better tree fast snapshot
2024-08-13 22:47:55 +08:00
Zixuan Chen
a8c1180810
refactor: add an optional field about bool_rle in tree for future use
2024-08-13 21:27:19 +08:00
Zixuan Chen
1e12fd6b20
perf: optimize tree checkout diff a bit
2024-08-13 20:34:07 +08:00
Zixuan Chen
2881b45bfe
perf: reduce mem use of tree history cache
2024-08-13 20:30:20 +08:00
Zixuan Chen
c8f505539e
refactor: wrap a arc around tree op
2024-08-13 17:51:24 +08:00
Zixuan Chen
0c1e1d47a2
fix: refine tree ds a bit
2024-08-13 17:38:37 +08:00
Zixuan Chen
fac3f07a70
perf: optimize how tree history cache store ops mapping
2024-08-13 16:55:18 +08:00
Zixuan Chen
68b717cfb3
chore: tree ds
2024-08-13 16:43:39 +08:00
Zixuan Chen
32687f61e8
chore: add rle vec
2024-08-13 11:47:57 +08:00
Zixuan Chen
7b81bed19d
refactor: rm unused code
2024-08-12 13:42:36 +08:00
Zixuan Chen
a615104fb1
perf: speedup id_to_cursor worst case
...
https://linear.app/loro/issue/LORO-834
2024-08-11 22:56:36 +08:00
Zixuan Chen
bf5a03e152
refactor: bk, before enable tree insert set
2024-08-11 20:48:43 +08:00
Zixuan Chen
84fe79e63f
test: bench large movable list
2024-08-11 17:36:58 +08:00
Zixuan Chen
0faa860d59
docs: rm warning
2024-08-11 11:04:53 +08:00
Zixuan Chen
870230c3e1
perf: optimize mem layout of history cache of map and movable list (experimental)
...
This change has not been benchmarked yet. Maybe need to be reverted.
2024-08-10 23:49:14 +08:00
Zixuan Chen
e00337d7d8
chore: cleanup
2024-08-10 17:38:24 +08:00
Zixuan Chen
0bac73b029
perf: optimize how to store fractional index in change block
2024-08-10 15:40:56 +08:00
Zixuan Chen
0ff6a736e0
test: add tree mem bench
2024-08-10 15:36:35 +08:00
Zixuan Chen
bdc8b4b908
fix: split large change when importing
2024-08-09 17:27:07 +08:00
Zixuan Chen
42329a20ea
feat: add a few methods to free memory
2024-08-09 16:03:13 +08:00
Zixuan Chen
ce842a3aee
chore: add 10M cells bench
2024-08-09 14:56:56 +08:00
Zixuan Chen
65cf79da66
feat: api to free history cache
2024-08-09 12:02:15 +08:00
Zixuan Chen
94f8acd224
refactor: lazy load the history cache
2024-08-08 23:27:55 +08:00
Zixuan Chen
480e74b083
refactor: differentiate two different history cache
2024-08-08 21:26:03 +08:00
Zixuan Chen
868ee2e8cc
docs: add notes about apply diff
2024-08-08 18:27:49 +08:00
Zixuan Chen
2b6dae8393
fix: tree event when using import greater updates
2024-08-08 17:58:12 +08:00
Zixuan Chen
b40f273b7a
refactor: add import greater updates mode to tree import
2024-08-08 16:33:02 +08:00
Zixuan Chen
62d4c9ca5d
fix: add importGreaterUpdates variants to diffmode
2024-08-08 14:57:11 +08:00
Zixuan Chen
de78cf9636
refactor: add linear diff calc mode to tree
2024-08-08 10:39:04 +08:00
Zixuan Chen
9d8221ca29
fix: text diff calc refactor err
2024-08-08 00:36:59 +08:00
Zixuan Chen
af274eac79
refactor: add linear mode for text
2024-08-07 23:42:15 +08:00
Zixuan Chen
a9c6c32b3e
refactor: rename op group to history cache
2024-08-07 19:26:21 +08:00
Zixuan Chen
76ff328290
refactor: use less history cache for movable list diff calc
2024-08-07 19:25:40 +08:00
Zixuan Chen
fd26e6c71a
refactor: rm group list pos to elem id map in movable list history cache
2024-08-07 00:05:18 +08:00
Zixuan Chen
e79397c6b4
refactor: rename move's from_id to elem_id
2024-08-06 23:47:13 +08:00
Zixuan Chen
b791157494
refactor: make map import independent from history cache
2024-08-06 23:22:24 +08:00
Zixuan Chen
06ea79b900
refactor: add diff mode
2024-08-06 17:10:51 +08:00
Zixuan Chen
7e7fe782b4
refactor: refine movable list internal event
2024-08-06 15:34:55 +08:00
东灯
251c9c7787
feat: check invalid root container name ( #411 )
2024-07-29 15:06:31 +08:00
东灯
9d7b5b8474
feat & perf: add map.contains_key and perf map.len ( #409 )
...
* feat & perf: add map.contains_key and perf map.len
* chore: remove unused import
2024-07-25 16:40:08 +08:00
东灯
7e4500620d
style: use clippy to perf code ( #407 )
2024-07-24 17:39:29 +08:00
Zixuan Chen
e13c7ff174
chore: add method to ensure cids in block header are decoded
2024-07-24 16:06:33 +08:00
Zixuan Chen
618c38c7b9
fix: resolve LORO-766
2024-07-24 15:44:48 +08:00
东灯
e01505e013
feat: add text update API ( #404 )
...
* feat: add diff algorithm
* feat: implement DiffHook
* feat: add rust/typescript API
* feat: use u8 to enable SIMD
* feat: add SIMD support
* test: add a failed test case
* revert: change to unicode index
* fix: text event index correctness when using utf8 or unicode
* fix: use unicode index
* fix: apply delta
* fix: remove splice_unicode
---------
Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-07-23 21:22:47 +08:00
Patrick Arminio
b8f9fafed0
Add code to the default rich text configuration ( #406 )
2024-07-21 19:14:22 +08:00
Zixuan Chen
3d516a3e18
refactor: op group -> history cache
2024-07-18 15:44:50 +08:00
Zixuan Chen
06cd6b577d
refactor: rename "OpGroup" to "ContainerHistoryCache"
2024-07-17 12:32:03 +08:00
Zixuan Chen
59f09e7162
Merge branch 'main' into zxch3n/kv-store-trait
2024-07-17 12:16:53 +08:00
Zixuan Chen
ca6a7c897e
fix: warnings
2024-07-17 12:15:16 +08:00
Zixuan Chen
b39c45fa2e
chore: rm export blocks and import blocks
2024-07-17 12:10:58 +08:00
Yang Si
ce88b326dd
fix: when computing the len of the map, do not count elements that are None ( #402 )
2024-07-14 16:40:46 +08:00
Zixuan Chen
bc63d250e2
Merge branch 'main' into zxch3n/kv-store-trait
2024-07-14 13:59:14 +08:00
东灯
c710ec3dd3
feat: add more text api ( #398 )
...
* feat: add char_at
* feat: add slice
* feat: add splice
* feat: add rust/typescript API
* fix: ci test
* fix: patch
* fix: patch
* Update crates/loro-wasm/src/lib.rs
Co-authored-by: Zixuan Chen <remch183@outlook.com>
* Update crates/loro-wasm/src/lib.rs
Co-authored-by: Zixuan Chen <remch183@outlook.com>
* Update crates/loro/src/lib.rs
Co-authored-by: Zixuan Chen <remch183@outlook.com>
* perf: use entity index
* fix: patch
* fix: error brackets
---------
Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-07-14 13:06:04 +08:00
东灯
2f95480e96
feat: add text iter ( #400 )
...
* feat: add text iter
* doc: fix return error
* doc: add callback explain
* perf: change to iterate spans
2024-07-13 23:34:37 +08:00
Leon Zhao
00e9c8d031
fix: tree diff calc children should be sorted by idlp ( #401 )
2024-07-13 23:26:38 +08:00
Zixuan Chen
9f7783516d
refactor: use BTree in dag
2024-07-12 22:51:13 +08:00
Zixuan Chen
2a4e089208
chore: rm logs
2024-07-12 18:12:21 +08:00
Zixuan Chen
46000420e8
Merge main
2024-07-12 16:15:54 +08:00
Zixuan Chen
23fbae1f80
refactor: make vv in dag optional
2024-07-12 15:19:40 +08:00
Leon Zhao
71e5afa6cc
fix: tree undo when processing deleted node ( #399 )
2024-07-12 10:53:48 +08:00
Zixuan Chen
a924b3e55b
fix: change block encode & decode
2024-07-11 23:54:09 +08:00
Leon Zhao
00e7bf2911
chore: rename position to fractional_index ( #381 )
...
Breaking Change:
- rename `position` to `fractional_index`
2024-07-11 22:03:48 +08:00
东灯
86c760abd0
feat: add insert_utf8 and delete_utf8 for Rust Text API ( #396 )
...
* feat: add insert_utf8
* chore: merge insert functions
* fix: use utf8_to_unicode_index to reslove index
* fix: add bound-check and use unicode PosType
* feat: add delete_utf8
* perf: O(LogN) insert_utf8
* feat: add utf-16 cross unicode check
* perf: O(LogN) delete_utf8
* chore: add api
* chore: remove unused function
* fix: api name and bindgen name
* test: add utf8 js test
---------
Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-07-10 10:20:08 +08:00
Zixuan Chen
354f429381
feat: impl import_all export_all apis
2024-07-09 23:07:57 +08:00
Zixuan Chen
27daa08b77
pr: impl kv store
2024-07-09 00:32:16 +08:00
Zixuan Chen
f802ede4b9
refactor: impl HasCounter explicitly to speedup
2024-07-09 00:02:41 +08:00
Zixuan Chen
d69e4c9c30
Refactor ChangeStore to use external KV store and improve block management
...
- Add ID serialization methods in loro-common
- Update KvStore trait with binary search and double-ended iterator
- Modify export/import blocks to use bytes::Bytes
- Refactor ChangeStore to use external KV store for persistence
- Implement flush, encode_all, and decode_all methods for ChangeStore
- Update change retrieval and iteration methods to work with new block management
- Improve block parsing and caching mechanisms
2024-07-08 17:33:57 +08:00
Zixuan Chen
391bd5ad0a
bk: kv store init; refine block encode format
2024-07-04 23:41:53 +08:00
Zixuan Chen
9047065843
Fix undo with checkout ( #375 )
...
* fix: should transform checkout event
* chore: update fuzz dep
* chore: add pos to error info
* fix: clear undo/redo stack when checkingout
* test: update fuzz dep
* test: a new failed test case
* fix: tree transform
* chore: fuzz
* chore: add log
* chore: add more logs
* fix: compose err
* chore: fuzz test dep
* test: a failed tree case
* fix: undo tree event
* fix: do not compare tree position in fuzz
* fix: fuzz rev
* test: a failed tree case
* fix: add tree compose
* chore: add comment
* chore: fuzz
* fix: test
* fix: tree transform
* fix: tree transform index
* fix: sort tree index
* chore: fuzz
* fix: undo/redo remote change effect compose
* bk
* fix: tree undo redo (#385 )
* fix: event hint none
* chore: fuzz version
* ci: fuzz
* bk: weird err
* fix: type err
* fix: fractional index between
* fix: wasm counter feature
* test: a new failed case
* fix: recursively create child nodes
* fix: filter empty event
* bk
* bk
* fix: tree undo redo remap
* chore: clean
* bk
* fix: tree need remap first
* fix: tree undo effect
* fix: tree diff calc
* fix: tree fuzz check eq func
* fix: remove EventHint None
* chore: cargo fix
* fix: tree uncreate
* fix: fuzz tree assert only structure
* refactor: rename methods
* fix: movable tree apply delta
* fix: another movable list issue
* chore: fuzz only check 1 actor's history
---------
Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-07-04 18:15:44 +08:00
Zixuan Chen
1e94248128
Feat fork ( #393 )
...
* feat: fork
* chore: add changeset
2024-06-26 21:05:04 +08:00
Leon Zhao
fb028f861d
fix: export CommitOptions & JsonSchema ( #389 )
2024-06-20 16:52:36 +08:00
Zixuan Chen
3848db5a9e
fix: warnings
2024-06-20 16:30:03 +08:00
Zixuan Chen
a276010128
feat: replace states with container store
2024-06-20 13:17:32 +08:00
Zixuan Chen
fcdbaed172
fix: warnings
2024-06-18 19:38:10 +08:00
Zixuan Chen
9c050b8b0b
refactor: fix name err & add counter state fast snapshot
2024-06-18 18:17:46 +08:00
Zixuan Chen
d6c8a632ad
Merge branch 'main' into zxch3n/container-store
2024-06-18 18:09:14 +08:00
Zixuan Chen
572635dad2
feat: tree state
2024-06-18 14:36:14 +08:00
Zixuan Chen
a290f29bb5
bk: tree impl (untested)
2024-06-14 18:37:51 +08:00
Zixuan Chen
3729deb12d
feat: richtext snapshot
2024-06-14 12:32:08 +08:00
