Commit graph

99 commits

Author SHA1 Message Date
Zixuan Chen
6d04066cd0
refactor: avoid footgun of impl ord for cid 2024-09-30 18:09:49 +08:00
Zixuan Chen
bef39ce6b5
Feat: allow editing on detached mode (#473)
* refactor: add detached editing config and prepare the architecture for editing detached doc

* feat: subscribe for peer id change

* fix: undo after checkout & add tests for detached editing

* test: add fuzzer for detached editing

* feat: expose detached editing configure to wasm

* test: add wasm test for detached editing
2024-09-24 11:16:59 +08:00
Zixuan Chen
bdf006ed2e
feat: fork doc at the target version (#469) 2024-09-23 09:51:07 +08:00
Zixuan Chen
d460346585
feat: jsonpath experimental support (#466)
* feat: jsonpath experimental support

* fix: add support for negative index unionindex and unionkey

* chore: export lorodoc in loro-js and fix a few tests related to map's entries order

* chore: fix type err
2024-09-19 19:22:39 +08:00
Zixuan Chen
e592d4f830
fix: raise error if moving a deleted node 2024-09-17 16:04:00 +08:00
Zixuan Chen
8dcb619147
fix: raise error if perform action on a deleted container (#465)
- Add ContainerDeleted error variant to LoroError
- Implement is_deleted() method for container handlers
- Check for deleted containers before applying operations
- Add dead_containers_cache to DocState to track deleted containers
- Update apply_diff to optionally clear dead container cache
- Add tests for handling operations on deleted containers
- Implement checkout to reset container deleted cache
- Minor code cleanup and typo fixes
2024-09-17 12:09:20 +08:00
Zixuan Chen
f1374c3bd0
Merge branch 'main' into dev 2024-09-12 22:30:42 +08:00
Zixuan Chen
4a27a0645f
Merge branch 'dev' into feat-gc 2024-09-11 10:54:29 +08:00
Leon Zhao
07671ea9fd
feat: add old parent and old index in tree diff (#452)
* feat: add old parent in tree diff

* chore: enable ci

* feat: add old_index to tree diff

* fix: new fractional index config

* fix: cargo fix

* fix: add FractionalIndexNotEnabled error

* fix: move config to tree state

* fix: error string

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-09-09 16:16:02 +08:00
Zixuan Chen
7be915e87d
chore: bump rust crates version
Some checks failed
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
2024-09-07 20:01:33 +08:00
Zixuan Chen
e7bf12cc14
(GC) Checkout on text/list that needs to retrieve content from the trimmed state (#446)
* feat: export gc snapshot without latest state when ops len is small

* test: add a test for time tracker usage

* chore: add gc snapshot bench

* chore: simplify

* test: record timestamp for time tracker

* fix: checkout to a version that needs to use gc state

* refactor: make list internal event more reasonable

* fix: checkout list/movable list on gc doc
2024-09-05 14:30:22 +08:00
Zixuan Chen
3d5a5da425
feat: remove dropped containers on exported gc snapshot 2024-09-04 19:52:53 +08:00
Zixuan Chen
4c325ab87c
Merge branch 'dev' into feat-gc 2024-09-03 21:21:14 +08:00
Leon Zhao
c1620fdb37
feat: memkv export import all (#422)
* feat: sstable

* fix: add magic number version

* feat: new mem kv store based sstable

* feat: binary_search

* fix: sstable iter scan

* fix: new mem kv

* feat: add cache for sstable

* fix: encode schema comment

* fix: sstable iter scan

* chore: clean

* fix: export all

* fix: sstable scan bound

* fix: sstable iter scan next==prev

* fix: merge iter next_back

* fix: mem kv export

* chore: clean

* fix: prev to key

* fix: prev find block

* fix: get prev block idx

* refactor: kv store

* fix: checksum when import

* fix: meta first last key

* Revert "fix: meta first last key"

This reverts commit a069c1ed37.

* fix: skip empty iter

* fix: remove key from large block

* chore: comment

* feat: compress block

* fix: remove key in large block

* chore: const

* doc: intro sstable encode

* test: add kv store fuzz

* style: format file

* feat: add fuzz to kv store (#428)

* fix: kv fuzzer

* fix: debug

* bk

* fix: block iter next back

* fix: block prev iter left = next idx

* feat: move kv store a crate

* fix: remove value len from normal block

* doc: sstable format

* test: add more test

* test: add test

* feat: new merge iter

* chore: revert

* fix: rename next back

* fix: rename mem sstable

* fix: rename to mem

* fix: use Bytes as key

* fix: use simple merge iter

* feat: compress option

* fix: remove empty iter

* style: refine some impl details

* fix: large block compress

* feat: use write read for encode

* doc: refine doc

* fix: simplify the first chunk

* feat: import many times

* refactor: refine styles

* test: fuzz merge iter

* fix: rename peek_xxx()

* fix: better sstable iter inner

* fix: use mem kv store

* pref: mem kv store

* perf: export mem kv

* chore: clean

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-08-30 11:44:34 +08:00
Zixuan Chen
4fa34a318a
chore: rm keywords 2024-08-29 23:07:47 +08:00
Zixuan Chen
a38cf178d1
chore: bump rust crates version 2024-08-29 23:02:03 +08:00
Zixuan Chen
819d1f86b9
feat: checkout map 2024-08-27 00:05:11 +08:00
Zixuan Chen
4e64051c58
fix: detect importing outdated updates on gc doc 2024-08-26 22:14:23 +08:00
Zixuan Chen
e084eca580
Merge branch 'main' into dev 2024-08-25 21:12:07 +08:00
Zixuan Chen
1812caea65
Refactor: use kv internally for docstate (#426)
* refactor: use kv in state

* refactor: do not load the state into the inner fxhashmap if not needed

* refactor: calc offset without unsafe code

* style: replace unsafe code
2024-08-24 14:16:06 +08:00
Zixuan Chen
6f9329adc2
fix: change_store loading cache error 2024-08-18 18:15:53 +08:00
东灯
251c9c7787
feat: check invalid root container name (#411) 2024-07-29 15:06:31 +08:00
Patrick Arminio
b8f9fafed0
Add code to the default rich text configuration (#406) 2024-07-21 19:14:22 +08:00
Zixuan Chen
59f09e7162
Merge branch 'main' into zxch3n/kv-store-trait 2024-07-17 12:16:53 +08:00
东灯
c710ec3dd3
feat: add more text api (#398)
* feat: add char_at

* feat: add slice

* feat: add splice

* feat: add rust/typescript API

* fix: ci test

* fix: patch

* fix: patch

* Update crates/loro-wasm/src/lib.rs

Co-authored-by: Zixuan Chen <remch183@outlook.com>

* Update crates/loro-wasm/src/lib.rs

Co-authored-by: Zixuan Chen <remch183@outlook.com>

* Update crates/loro/src/lib.rs

Co-authored-by: Zixuan Chen <remch183@outlook.com>

* perf: use entity index

* fix: patch

* fix: error brackets

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-07-14 13:06:04 +08:00
Zixuan Chen
46000420e8
Merge main 2024-07-12 16:15:54 +08:00
Zixuan Chen
a924b3e55b
fix: change block encode & decode 2024-07-11 23:54:09 +08:00
东灯
86c760abd0
feat: add insert_utf8 and delete_utf8 for Rust Text API (#396)
* feat: add insert_utf8

* chore: merge insert functions

* fix: use utf8_to_unicode_index to reslove index

* fix: add bound-check and use unicode PosType

* feat: add delete_utf8

* perf: O(LogN) insert_utf8

* feat: add utf-16 cross unicode check

* perf: O(LogN) delete_utf8

* chore: add api

* chore: remove unused function

* fix: api name and bindgen name

* test: add utf8 js test

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-07-10 10:20:08 +08:00
Zixuan Chen
f802ede4b9
refactor: impl HasCounter explicitly to speedup 2024-07-09 00:02:41 +08:00
Zixuan Chen
d69e4c9c30
Refactor ChangeStore to use external KV store and improve block management
- Add ID serialization methods in loro-common
- Update KvStore trait with binary search and double-ended iterator
- Modify export/import blocks to use bytes::Bytes
- Refactor ChangeStore to use external KV store for persistence
- Implement flush, encode_all, and decode_all methods for ChangeStore
- Update change retrieval and iteration methods to work with new block management
- Improve block parsing and caching mechanisms
2024-07-08 17:33:57 +08:00
Zixuan Chen
9047065843
Fix undo with checkout (#375)
* fix: should transform checkout event

* chore: update fuzz dep

* chore: add pos to error info

* fix: clear undo/redo stack when checkingout

* test: update fuzz dep

* test: a new failed test case

* fix: tree transform

* chore: fuzz

* chore: add log

* chore: add more logs

* fix: compose err

* chore: fuzz test dep

* test: a failed tree case

* fix: undo tree event

* fix: do not compare tree position in fuzz

* fix: fuzz rev

* test: a failed tree case

* fix: add tree compose

* chore: add comment

* chore: fuzz

* fix: test

* fix: tree transform

* fix: tree transform index

* fix: sort tree index

* chore: fuzz

* fix: undo/redo remote change effect compose

* bk

* fix: tree undo redo (#385)

* fix: event hint none

* chore: fuzz version

* ci: fuzz

* bk: weird err

* fix: type err

* fix: fractional index between

* fix: wasm counter feature

* test: a new failed case

* fix: recursively create child nodes

* fix: filter empty event

* bk

* bk

* fix: tree undo redo remap

* chore: clean

* bk

* fix: tree need remap first

* fix: tree undo effect

* fix: tree diff calc

* fix: tree fuzz check eq func

* fix: remove EventHint None

* chore: cargo fix

* fix: tree uncreate

* fix: fuzz tree assert only structure

* refactor: rename methods

* fix: movable tree apply delta

* fix: another movable list issue

* chore: fuzz only check 1 actor's history

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-07-04 18:15:44 +08:00
Zixuan Chen
d6c8a632ad
Merge branch 'main' into zxch3n/container-store 2024-06-18 18:09:14 +08:00
Zixuan Chen
0d3709b79e
feat: map snapshot 2024-06-13 17:50:29 +08:00
Leon Zhao
afac34755f
feat: implementing Counter and expose to js side (#384) 2024-06-13 14:43:19 +08:00
Zixuan Chen
d71fee144d
bk: container store impl 2024-06-12 11:43:02 +08:00
Leon Zhao
2df2a52b05
feat: Stable JSON representation for history (#368)
---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-06-07 13:18:30 +08:00
Zixuan Chen
54637a7b42
Publish v0.16.2 to crates.io (#374)
* chore: mark dev-utils not published

* chore: release rust crates

* chore: publish fi

* chore: pre publish fi

* chore: update lockfile

* test: change fuzz rev
2024-05-29 22:55:55 +08:00
Zixuan Chen
321e0e72a4
Undo (#361)
https://github.com/loro-dev/loro/pull/361

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-05-21 06:14:49 +08:00
Leon Zhao
6e14e5b117
Feat: Make the encoding format forward and backward compatible (#329) 2024-05-13 21:37:10 +08:00
Gábor Szabó
d958767b13
add repository to Cargo.toml (#358) 2024-05-10 21:14:17 +08:00
Zixuan Chen
ea7d8b0669
fix: refine error message on corrupted data (#356)
* fix: refine error message on corrupted data

* fix: comment
2024-05-10 11:06:01 +08:00
Zixuan Chen
0660b1a1be
fix: upgrade wasm-bindgen to fix str free err (#353)
* fix: upgrade wasm-bindgen to fix str free err

* chore: fix ci
2024-05-09 15:22:34 +08:00
Leon Zhao
fffd49b5fa
Use fractional index to order the children of the tree (#298)
* feat: fractional index

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-05-07 14:01:13 +08:00
Zixuan Chen
4021ad820c
chore: loro-rs v0.5.1 2024-05-06 13:37:55 +08:00
Leon Zhao
cd8b9a31e7
fix: missing MovableList in all container type (#343) 2024-05-02 11:28:28 +08:00
Zixuan Chen
5660bcc4a5
chore: bump loro-rs to v0.5.0 2024-04-29 18:09:07 +08:00
Zixuan Chen
31a8569840
Movable List (#293)
* bk: add move op content

* bk: add inner_movable_list diff and related stuff

* perf: high perf state

* fix: update old list item cache

* fix: should use id in del

* feat: two kinds of len for movable list state

* bk: add op index to movable list

* bk: make basic handler test pass

* refactor: add move_from to list event

* fix: make all existing tests pass

* bk: list move event hint into event

* bk: convert inner event into user event

Co-authored-by: Leon <leeeon233@gmail.com>

* fix: convert issue when inserting new value

* feat: add op group for movable list

* feat: diff calc

* feat: add mov support to tracker

* fix: when applying diff, state should be force update

* feat: encoded op

* feat: snapshot encode

* fix: pass basic sync

* fix: snapshot encode/decode

* fix: warnings

* feat: expose mov list to loro crate

* test: fuzz movable list

* test: fix fuzz integration

* fix: movable list basic move sync

* fix: movable list events

* fix: movable event err

* fix: register child container on movable list

* fix: should not return child index if the value is already overwritten

* fix: local event err in movable list

* fix: get elem at pos

* refactor: extract mut op that could break invariants

* fix: event err

* fix: child container to elem err

* fix: bringback event issue

* fix: event err

* fix: event emit

* fix: id to cursor iter issue

* chore: fix a few warnings

* fix: warnings

* fix: fix move in tracker

* test: add consistency check

* test: fix tracker

* refactor: simplify event conversion in docstate

* refactor: refine move event

* refactor: simplify the maintain of parent child links

* fix: revive err

* fix: warnings

* fix: it's possible that pos change but cannot find the respective list item

* fix: elem may be dropped after snapshot

* fix: warnings

* fix: richtext time travel issue

* fix: move op used wrong delete id on tracker

* fix: handle events created by concurrent moves correctly

* fix: event hint error, used op index for list event

* fix: move_from flag err

* fix: id to cursor get err

* test: add mov fuzz target

* fix: the pos of inserting new container

* fix: used wrong event hint index

* fix: del event hint

* fix: warnings

* fix: internal diff to event err

* fix: event's move flag error
This "move" flag does not actually mean that the insertion
is caused by the move op.
就算是 move 造成的它不一定就能是 true
它得是下游真的能在“前一个版本的 array 里找到“,才能是 true

* fix: remove redundant elements from the movable list

The Movable List is currently flawed; an element may not exist on the movable list state, yet there are operations that revive its corresponding list item. In such cases, the diff calculation does not send back the corresponding element state (this occurs when tracing back, which fuzz testing currently does not cover. It might only be exposed by randomly switching to a version and then checking for consistency; otherwise, as long as all elements are in memory, this problem does not arise).

Moreover, there is no need to store elements in the state that do not have a corresponding list item. They will be deleted during the Snapshot, and relying on "them still being in the state" is incorrect behavior. Such adjustments also eliminate the need to maintain the `pending_elements` field.

By allowing the opgroup to record the mapping from pos id to state id, we can ensure that the events sent to the movable list state will include the corresponding state.

Movable List 现在是有错的,elem 可能不存在 movable list state 上,但是又有操作把它对应的 list item 复活了,此时 diff calc 不会把对应 element 状态发送回来(往前回溯的时候会出现,fuzz 现在没覆盖到。得有随意切换一个版本然后 check consistency 才可能会暴露;否则现在大家 elements 都在内存,就没这个问题)

而且我们没有必要在状态中存储没有对应 list item 的 element。在 Snapshot 的时候它们都会被删掉,如果依赖了“它们还会在 state 内”就是错误的行为。这样的调整也让我们不需要去维护 pending_elements 这个 field 了

通过让 opgroup 记录了 pos id → state id 的映射,可以保证发给 movable list state 的事件中会带上对应的 state

* test: make fuzzer stricter

* test: test expectation error

* refactor: rename stable pos to cursor

* tests: chore list bench init

* test: add bench

* bench: add mov & set bench

* feat(wasm): movable list js api

* fix: make movablelist able to attach even if it's already attached & refine the type of subscribe

* fix: remove the loro doc param in .unsub

* refactor: refine ts types and export setContainer api

* chore: fix warnings

* chore: rm debug logs

* perf: reduce mem usage of opgroup

* bench: add list criterion bench

---------

Co-authored-by: Leon <leeeon233@gmail.com>
2024-04-26 12:08:53 +08:00
Zixuan Chen
f99bfd8e21
Refactor rm unused code (#328)
* chore: init

* fix: fuzz config

* refactor: rm unused code
2024-04-22 21:20:00 +08:00
Zixuan Chen
a32b75ad3b
fix(js): allow convert from undefined to LoroValue (#323) 2024-04-18 09:47:00 +08:00
Zixuan Chen
01b45598d8
refactor: parse uint8array to binary (#320)
and refine Awareness type in ts
2024-04-17 15:09:03 +08:00