Leon Zhao
dfc99c1746
feat: block encode header ( #458 )
...
* bk
* feat: encode header
* chore: use columnar 0.3.9
* fix: not need encode last lamport and length
* chore: cargo fix
* fix: print
* bk
* chore: columnar 0.3.10
2024-09-11 22:53:10 +08:00
Leon Zhao
f7bd2aefff
feat: do not compress if larger after compressing ( #455 )
...
* feat: do not compress if larger after compressing
* test: add related tests and ensure coverage
---------
Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-09-11 18:13:41 +08:00
Zixuan Chen
255a7bd9e6
chore: add deprecated warning for old export methods
2024-09-11 14:25:30 +08:00
Zixuan Chen
dfe5b15a83
test: add loro value encode size check
2024-09-11 12:50:52 +08:00
Zixuan Chen
69ef0ac8c2
refactor: rename fractional index config function
...
and fix a few timestmap err
2024-09-11 11:16:31 +08:00
Zixuan Chen
d7778f15fd
test: fix a few outdated test
2024-09-11 11:05:42 +08:00
Zixuan Chen
4a27a0645f
Merge branch 'dev' into feat-gc
2024-09-11 10:54:29 +08:00
Zixuan Chen
71e46f65f1
perf: export block directly for updates
...
It can reduce the overhead for small updates
2024-09-10 13:09:45 +08:00
Leon Zhao
07671ea9fd
feat: add old parent and old index in tree diff ( #452 )
...
* feat: add old parent in tree diff
* chore: enable ci
* feat: add old_index to tree diff
* fix: new fractional index config
* fix: cargo fix
* fix: add FractionalIndexNotEnabled error
* fix: move config to tree state
* fix: error string
---------
Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-09-09 16:16:02 +08:00
Zixuan Chen
ab6ee76b1e
style: refine a few impl ( #454 )
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
2024-09-09 15:58:00 +08:00
Zixuan Chen
7be915e87d
chore: bump rust crates version
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
2024-09-07 20:01:33 +08:00
github-actions[bot]
243708943c
chore: version packages ( #451 )
...
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-07 19:44:32 +08:00
Zixuan Chen
46e21fccc7
fix: two tree move (inside the same parent) issues ( #450 )
...
Release WASM / Release (push) Waiting to run
Test All / build (push) Waiting to run
* fix: two tree move (inside the same parent) issues
* chore: add release note
2024-09-07 16:49:28 +08:00
Zixuan Chen
1c4d7e2182
chore: add some debug log
2024-09-06 22:57:19 +08:00
Zixuan Chen
3d2d9d9c18
refactor: optimize block encoder
2024-09-06 16:26:50 +08:00
Zixuan Chen
00a25510f1
refactor!: use timestamp in seconds by default ( #448 )
2024-09-05 22:21:18 +08:00
Zixuan Chen
dd6bb10fff
Refactor(wasm) rename Loro to LoroDoc ( #443 )
...
* chore: reduce wasm size
* refactor(wasm): rename Loro to LoroDoc
2024-09-05 20:27:31 +08:00
Zixuan Chen
c04da30572
fix: avoid import issue on rich text doc
...
by avoid splitting doc on a version that splits the stylestart and styleend
2024-09-05 17:48:59 +08:00
Zixuan Chen
6e8048bc28
test: add test for export snapshot on gc doc
2024-09-05 16:50:41 +08:00
Zixuan Chen
e9be422046
fix: handle on new container correctly
...
when using diff calc for unknown span
2024-09-05 16:26:00 +08:00
Leon Zhao
cb134fc7c7
feat: add with fractional index config ( #442 )
...
* feat: add `with_fractional_index` config
* fix: share config state
2024-09-05 16:03:47 +08:00
github-actions[bot]
622c881605
chore: version packages ( #447 )
...
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-05 14:31:14 +08:00
Zixuan Chen
e7bf12cc14
(GC) Checkout on text/list that needs to retrieve content from the trimmed state ( #446 )
...
* feat: export gc snapshot without latest state when ops len is small
* test: add a test for time tracker usage
* chore: add gc snapshot bench
* chore: simplify
* test: record timestamp for time tracker
* fix: checkout to a version that needs to use gc state
* refactor: make list internal event more reasonable
* fix: checkout list/movable list on gc doc
2024-09-05 14:30:22 +08:00
Zixuan Chen
dce00abd8a
Fix using loro in cloudflare worker ( #441 )
...
* fix: try fix generated wasm bundler
* fix: loro-quill example
and fix bundler patch issue
* chore: add release info
2024-09-05 13:10:27 +08:00
Zixuan Chen
68ffd31ba0
Feat when exporting gc snapshot with short history, don't encode the latest state ( #445 )
...
* feat: export gc snapshot without latest state when ops len is small
* test: add a test for time tracker usage
* chore: add gc snapshot bench
* chore: simplify
* test: record timestamp for time tracker
2024-09-04 21:00:17 +08:00
Zixuan Chen
3d5a5da425
feat: remove dropped containers on exported gc snapshot
2024-09-04 19:52:53 +08:00
Zixuan Chen
4c325ab87c
Merge branch 'dev' into feat-gc
2024-09-03 21:21:14 +08:00
Zixuan Chen
08d53cae93
feat: subscribe for local updates ( #444 )
2024-09-03 14:04:43 +08:00
Zixuan Chen
64e5d30da8
chore: reduce wasm size
2024-09-02 00:30:00 +08:00
Zixuan Chen
cdb7498e90
chore: setup tracing feature only on loro-wasm
2024-08-30 20:58:18 +08:00
Leon Zhao
c1620fdb37
feat: memkv export import all ( #422 )
...
* feat: sstable
* fix: add magic number version
* feat: new mem kv store based sstable
* feat: binary_search
* fix: sstable iter scan
* fix: new mem kv
* feat: add cache for sstable
* fix: encode schema comment
* fix: sstable iter scan
* chore: clean
* fix: export all
* fix: sstable scan bound
* fix: sstable iter scan next==prev
* fix: merge iter next_back
* fix: mem kv export
* chore: clean
* fix: prev to key
* fix: prev find block
* fix: get prev block idx
* refactor: kv store
* fix: checksum when import
* fix: meta first last key
* Revert "fix: meta first last key"
This reverts commit a069c1ed37#428 )
* fix: kv fuzzer
* fix: debug
* bk
* fix: block iter next back
* fix: block prev iter left = next idx
* feat: move kv store a crate
* fix: remove value len from normal block
* doc: sstable format
* test: add more test
* test: add test
* feat: new merge iter
* chore: revert
* fix: rename next back
* fix: rename mem sstable
* fix: rename to mem
* fix: use Bytes as key
* fix: use simple merge iter
* feat: compress option
* fix: remove empty iter
* style: refine some impl details
* fix: large block compress
* feat: use write read for encode
* doc: refine doc
* fix: simplify the first chunk
* feat: import many times
* refactor: refine styles
* test: fuzz merge iter
* fix: rename peek_xxx()
* fix: better sstable iter inner
* fix: use mem kv store
* pref: mem kv store
* perf: export mem kv
* chore: clean
---------
Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-08-30 11:44:34 +08:00
Zixuan Chen
94252102b3
chore: update categories
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
2024-08-29 23:09:45 +08:00
Zixuan Chen
4fa34a318a
chore: rm keywords
2024-08-29 23:07:47 +08:00
Zixuan Chen
cf6489b320
chore: rm keywords
2024-08-29 23:06:23 +08:00
Zixuan Chen
543bbe9226
chore: rm keyword
2024-08-29 23:03:45 +08:00
Zixuan Chen
a38cf178d1
chore: bump rust crates version
2024-08-29 23:02:03 +08:00
Zixuan Chen
0cc90937cd
fix: insert change split error
2024-08-29 22:18:11 +08:00
Leon Zhao
c1f0a40f4b
feat: movable list tree gc ( #439 )
...
* feat: movable list tree gc
* fix: record value in movable history when gc
* fix: movable list gc
* fix: remove value from pos
* refactor: tree init
* test: add tree gc checkout test
2024-08-29 09:53:32 +08:00
Zixuan Chen
60036bc39c
Merge branch 'main' into dev
2024-08-28 23:33:44 +08:00
Zixuan Chen
227d1aad2c
fix: find last_delete_op can return none
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
2024-08-28 01:17:01 +08:00
github-actions[bot]
efc50fa9c5
chore: version packages ( #437 )
...
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-28 00:09:38 +08:00
Zixuan Chen
7cf54e8aa3
fix: should not use snapshot importing when it's inside a batch importing ( #436 )
...
* fix: should not use snapshot importing when it's inside a batch importing
* chore: bk
2024-08-28 00:06:24 +08:00
github-actions[bot]
c36664e784
chore: version packages ( #435 )
...
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-27 21:34:04 +08:00
Zixuan Chen
a76143068a
Chore build err ( #434 )
...
* chore: rm needless fuzz code
* fix(wasm): build script fix
* chore: release info
2024-08-27 21:30:18 +08:00
github-actions[bot]
6333cf0a09
chore: version packages ( #433 )
...
Release WASM / Release (push) Waiting to run
Test All / build (push) Waiting to run
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-27 21:08:11 +08:00
Zixuan Chen
38b4bcfee5
chore: rm crypto patch ( #432 )
2024-08-27 21:03:40 +08:00
Zixuan Chen
1a80cb7572
fix: one tricky case
2024-08-27 00:17:22 +08:00
Zixuan Chen
819d1f86b9
feat: checkout map
2024-08-27 00:05:11 +08:00
Zixuan Chen
ea2beae48e
refactor: make gc store accessible for history cache
2024-08-26 23:01:42 +08:00
Zixuan Chen
87bc464f3b
fix: make exporting & importing gc snapshot more correctly
2024-08-26 22:33:44 +08:00
Zixuan Chen
4e64051c58
fix: detect importing outdated updates on gc doc
2024-08-26 22:14:23 +08:00
Zixuan Chen
cfe8652415
feat: basic gc mode impl
2024-08-26 21:21:33 +08:00
Zixuan Chen
82d035442c
feat: add new export from mode for new storage format
2024-08-26 17:10:38 +08:00
Zixuan Chen
f293a796f2
Merge branch 'dev' into feat-gc
2024-08-25 21:14:04 +08:00
Zixuan Chen
e084eca580
Merge branch 'main' into dev
2024-08-25 21:12:07 +08:00
Leon Zhao
2f0a3ab92f
fix: delete the **bring back** tree node from the undo container remap ( #423 )
2024-08-25 20:34:05 +08:00
Zixuan Chen
8fbfea4bda
test: skip undo-redo assertion in fuzz for now
2024-08-25 19:42:57 +08:00
Zixuan Chen
4f338dc14d
refactor: org fast snapshot
2024-08-25 19:27:46 +08:00
Zixuan Chen
03c3e0b0b8
refactor: rm arc in tree hist cache
2024-08-24 15:05:38 +08:00
Zixuan Chen
ca9298507e
refactor: make tree history cache fields private
2024-08-24 15:02:59 +08:00
Zixuan Chen
1812caea65
Refactor: use kv internally for docstate ( #426 )
...
* refactor: use kv in state
* refactor: do not load the state into the inner fxhashmap if not needed
* refactor: calc offset without unsafe code
* style: replace unsafe code
2024-08-24 14:16:06 +08:00
Zixuan Chen
95bec549d7
feat: support commit message ( #429 )
...
* feat: support commit message
* test: refine test
2024-08-24 14:15:40 +08:00
Zixuan Chen
81515ba6d3
refactor: reuse the peer id in map fast snapshot if we can ( #430 )
2024-08-24 14:15:10 +08:00
Zixuan Chen
c2c727de6f
test: use less strict tree tracker for fuzzing
2024-08-23 23:36:28 +08:00
Zixuan Chen
5ba3fba3f3
Fix real_id setting err in tracker ( #427 )
...
* test: add a failed test case
* fix: tracker real_id setting err
2024-08-22 21:42:00 +08:00
leeeon233
a6f8c9c2d2
fix: tree apply diff when before dead after alive
2024-08-22 16:44:52 +08:00
Zixuan Chen
bcabf682e9
test: a failed case
2024-08-22 13:25:19 +08:00
Zixuan Chen
3f2c7d9498
fix: ignore undo failures
...
It could happen when using movable list in collab env. But it's extremely rare.
2024-08-21 23:27:43 +08:00
Leon Zhao
7402ff6788
chore: new fuzz test
2024-08-21 22:56:58 +08:00
Zixuan Chen
86aeaad97d
fix: movable list apply diff err
2024-08-21 18:38:46 +08:00
Zixuan Chen
f142afd431
test: minimize
2024-08-21 18:21:25 +08:00
Leon Zhao
2b2270ab99
test: add test
2024-08-21 18:21:25 +08:00
Zixuan Chen
5a03e7fa3a
fix: issue when getting container value
2024-08-21 12:02:35 +08:00
Zixuan Chen
d689db619f
feat: get path to container
2024-08-21 12:02:07 +08:00
Zixuan Chen
be0a7979cd
chore: fix type err
2024-08-21 11:30:03 +08:00
Zixuan Chen
0717aadd79
feat(dev): add a way to analyze the containers of the doc
2024-08-21 11:18:06 +08:00
Zixuan Chen
37046f8e41
chore: rm log
2024-08-20 19:54:37 +08:00
Zixuan Chen
be0f32eed6
fix: iter change error
2024-08-20 19:50:47 +08:00
Zixuan Chen
a7d9f3f5d3
test: add a failed test case
2024-08-20 08:34:11 +08:00
Zixuan Chen
544a8ebfcc
test: add a multithread test case minify method
2024-08-20 01:41:44 +08:00
Zixuan Chen
5b7ac90e54
refactor: simplify state code
2024-08-20 00:20:25 +08:00
Leon Zhao
79e9f29656
fix: delete the **bring back** tree node from the undo container remap ( #423 )
2024-08-19 21:52:30 +08:00
Zixuan Chen
1d58d69f28
test: add load + get value bench
2024-08-19 21:30:03 +08:00
Zixuan Chen
8ac4322bb6
test: add text bench
2024-08-19 21:15:48 +08:00
Zixuan Chen
c8bd656658
test: avoid super slow fuzzing unit
2024-08-19 17:15:08 +08:00
Zixuan Chen
d192c8607f
chore: display wasm size after building
2024-08-19 15:30:15 +08:00
Zixuan Chen
756f0c9ecf
fix: fix a few place that violate the dag's invariants
2024-08-19 15:12:20 +08:00
Zixuan Chen
65b9263231
refactor: add external_vv to change_store
2024-08-19 12:01:27 +08:00
Zixuan Chen
5f70310681
chore: fix warning - rm a redundant field
2024-08-19 11:39:35 +08:00
Zixuan Chen
ea5f91f6a6
chore: fix typos
2024-08-19 11:36:59 +08:00
Zixuan Chen
0619d7ae89
refactor: refine history cache (use type as constrain)
2024-08-19 11:34:34 +08:00
Zixuan Chen
c206365455
chore: fix warnings
2024-08-19 11:26:10 +08:00
Zixuan Chen
1c6221cf45
test: add a new failed test case
2024-08-19 10:59:57 +08:00
Zixuan Chen
3e70302d51
refactor: org change_store
2024-08-19 10:59:21 +08:00
Zixuan Chen
b06f617d3b
fix: frontiers_to_vv err
2024-08-18 21:45:11 +08:00
Zixuan Chen
6fd6aeb13d
fix: map event error
2024-08-18 20:40:53 +08:00
Zixuan Chen
b5eb176b3f
fix: mark flush on push_change
2024-08-18 18:32:41 +08:00
Zixuan Chen
6f9329adc2
fix: change_store loading cache error
2024-08-18 18:15:53 +08:00
Zixuan Chen
21c915163b
fix: avoid dag internal lock issue
2024-08-18 16:38:44 +08:00
Zixuan Chen
32922d20f5
fix: potential child parent link issue & map entry idlp bug
2024-08-18 16:23:17 +08:00
Zixuan Chen
3ba8ad5f12
fix: getting parent for root container error
2024-08-18 15:07:56 +08:00
Zixuan Chen
c97f1eec58
fix: tree snapshot encode decode err
2024-08-18 14:40:23 +08:00
Zixuan Chen
0c59bd2dcc
refactor: simplify dag and inserting change
2024-08-18 14:34:27 +08:00
Zixuan Chen
4df37ba170
fix: tree fast snapshot err
2024-08-18 14:32:39 +08:00
Zixuan Chen
fbd5d6725a
fix: bugs related to fast snapshot
2024-08-18 00:54:27 +08:00
Zixuan Chen
6c88847756
feat: now support treating fast snapshot as updates
2024-08-18 00:31:18 +08:00
Zixuan Chen
d8e2cdb756
refactor: rm outdated code
2024-08-17 23:53:18 +08:00
Zixuan Chen
c23e29ddb1
perf: replace md5 with xxhash for checksum
2024-08-17 23:32:09 +08:00
Zixuan Chen
1fc4b01f56
refactor: refine change store inner
2024-08-17 22:17:04 +08:00
Zixuan Chen
24a4f9839e
fix: add text example for bench & fix a few related bugs
2024-08-17 22:00:39 +08:00
Zixuan Chen
6126906ec2
perf: make idlp to id much faster
...
by using binary search on top of .range
2024-08-17 21:16:44 +08:00
Zixuan Chen
89327c3d68
feat: fast snapshot mode init
2024-08-17 19:04:19 +08:00
Zixuan Chen
f2d02f9fca
fix: container store fast snapshot
2024-08-17 16:52:46 +08:00
Zixuan Chen
24583ba33a
refactor: refine parent info register
2024-08-17 15:05:54 +08:00
Zixuan Chen
a7816af173
fix: lock issues
2024-08-16 23:52:58 +08:00
Zixuan Chen
a134bf88e3
feat: make dag able to lazy load the graph
2024-08-16 23:39:42 +08:00
Zixuan Chen
14178eeb2c
refactor: move handle new change inside loro_dag
2024-08-16 15:29:51 +08:00
Zixuan Chen
7a460414d4
fix: get lamport lte
...
fix failed wasm tests
2024-08-16 12:14:07 +08:00
Zixuan Chen
59e31c1370
refactor: provide better encapsulation for dag
...
make all the fields private
2024-08-16 11:40:29 +08:00
Zixuan Chen
35f0f811eb
refactor: move dag logic to dag mod
2024-08-16 10:56:35 +08:00
Zixuan Chen
cb3458b862
docs: leave some comments
2024-08-16 10:47:56 +08:00
Zixuan Chen
dd5c1bea72
refactor: reduce op inner content size by 1/6
2024-08-16 10:45:04 +08:00
Zixuan Chen
c0a8d4fc36
refactor: rm the unsound workaround and just use clone
2024-08-15 22:48:39 +08:00
Zixuan Chen
ea20f62d31
fix: make it work
...
but it may be unsound
2024-08-15 22:21:57 +08:00
Zixuan Chen
6568af86b9
chore: bk did not fix lifetime issue
2024-08-15 18:12:55 +08:00
Zixuan Chen
e05d991954
refactor: rm the need for refresh frontiers
2024-08-15 10:33:46 +08:00
Zixuan Chen
0aa534a9ba
chore: save frontiers to kv as well
2024-08-15 08:44:35 +08:00
Zixuan Chen
de8f9e38aa
refactor: change store internal
2024-08-15 08:28:11 +08:00
Zixuan Chen
957a002741
refactor: org tree state
2024-08-14 17:51:45 +08:00
Zixuan Chen
e19e1af543
perf: better tree fast snapshot
2024-08-13 22:47:55 +08:00
Zixuan Chen
a8c1180810
refactor: add an optional field about bool_rle in tree for future use
2024-08-13 21:27:19 +08:00
Zixuan Chen
1e12fd6b20
perf: optimize tree checkout diff a bit
2024-08-13 20:34:07 +08:00
Zixuan Chen
2881b45bfe
perf: reduce mem use of tree history cache
2024-08-13 20:30:20 +08:00
Zixuan Chen
c8f505539e
refactor: wrap a arc around tree op
2024-08-13 17:51:24 +08:00
Zixuan Chen
0c1e1d47a2
fix: refine tree ds a bit
2024-08-13 17:38:37 +08:00
Zixuan Chen
fac3f07a70
perf: optimize how tree history cache store ops mapping
2024-08-13 16:55:18 +08:00
Zixuan Chen
68b717cfb3
chore: tree ds
2024-08-13 16:43:39 +08:00
Zixuan Chen
32687f61e8
chore: add rle vec
2024-08-13 11:47:57 +08:00
Zixuan Chen
7b81bed19d
refactor: rm unused code
2024-08-12 13:42:36 +08:00
Zixuan Chen
a615104fb1
perf: speedup id_to_cursor worst case
...
https://linear.app/loro/issue/LORO-834
2024-08-11 22:56:36 +08:00
Zixuan Chen
bf5a03e152
refactor: bk, before enable tree insert set
2024-08-11 20:48:43 +08:00
Zixuan Chen
84fe79e63f
test: bench large movable list
2024-08-11 17:36:58 +08:00
Zixuan Chen
0faa860d59
docs: rm warning
2024-08-11 11:04:53 +08:00
Zixuan Chen
870230c3e1
perf: optimize mem layout of history cache of map and movable list (experimental)
...
This change has not been benchmarked yet. Maybe need to be reverted.
2024-08-10 23:49:14 +08:00
Zixuan Chen
e00337d7d8
chore: cleanup
2024-08-10 17:38:24 +08:00
Zixuan Chen
0bac73b029
perf: optimize how to store fractional index in change block
2024-08-10 15:40:56 +08:00
Zixuan Chen
c8bab8f663
chore: add rand
2024-08-10 15:38:06 +08:00
Zixuan Chen
0ff6a736e0
test: add tree mem bench
2024-08-10 15:36:35 +08:00
Zixuan Chen
bdc8b4b908
fix: split large change when importing
2024-08-09 17:27:07 +08:00
Zixuan Chen
42329a20ea
feat: add a few methods to free memory
2024-08-09 16:03:13 +08:00
