Commit graph

1324 commits

Author SHA1 Message Date
Zixuan Chen
996ce70e45
docs: add warn(missing_docs) to loro and loro-wasm (#339) 2024-04-29 19:35:21 +08:00
Zixuan Chen
f869b959f3
chore: specify loro-delta version in loro 2024-04-29 18:16:00 +08:00
Zixuan Chen
5660bcc4a5
chore: bump loro-rs to v0.5.0 2024-04-29 18:09:07 +08:00
Leon Zhao
6224974124
chore: rename js toJSON (#338) 2024-04-29 16:23:00 +08:00
Leon Zhao
a1ec948c32
fix: fuzz tree delete nested (#336) 2024-04-28 17:06:47 +08:00
github-actions[bot]
41d198d9e4
chore: version packages (#333)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-26 12:29:22 +08:00
Zixuan Chen
35b9b6e7b1
chore: release 2024-04-26 12:14:54 +08:00
Zixuan Chen
31a8569840
Movable List (#293)
* bk: add move op content

* bk: add inner_movable_list diff and related stuff

* perf: high perf state

* fix: update old list item cache

* fix: should use id in del

* feat: two kinds of len for movable list state

* bk: add op index to movable list

* bk: make basic handler test pass

* refactor: add move_from to list event

* fix: make all existing tests pass

* bk: list move event hint into event

* bk: convert inner event into user event

Co-authored-by: Leon <leeeon233@gmail.com>

* fix: convert issue when inserting new value

* feat: add op group for movable list

* feat: diff calc

* feat: add mov support to tracker

* fix: when applying diff, state should be force update

* feat: encoded op

* feat: snapshot encode

* fix: pass basic sync

* fix: snapshot encode/decode

* fix: warnings

* feat: expose mov list to loro crate

* test: fuzz movable list

* test: fix fuzz integration

* fix: movable list basic move sync

* fix: movable list events

* fix: movable event err

* fix: register child container on movable list

* fix: should not return child index if the value is already overwritten

* fix: local event err in movable list

* fix: get elem at pos

* refactor: extract mut op that could break invariants

* fix: event err

* fix: child container to elem err

* fix: bringback event issue

* fix: event err

* fix: event emit

* fix: id to cursor iter issue

* chore: fix a few warnings

* fix: warnings

* fix: fix move in tracker

* test: add consistency check

* test: fix tracker

* refactor: simplify event conversion in docstate

* refactor: refine move event

* refactor: simplify the maintain of parent child links

* fix: revive err

* fix: warnings

* fix: it's possible that pos change but cannot find the respective list item

* fix: elem may be dropped after snapshot

* fix: warnings

* fix: richtext time travel issue

* fix: move op used wrong delete id on tracker

* fix: handle events created by concurrent moves correctly

* fix: event hint error, used op index for list event

* fix: move_from flag err

* fix: id to cursor get err

* test: add mov fuzz target

* fix: the pos of inserting new container

* fix: used wrong event hint index

* fix: del event hint

* fix: warnings

* fix: internal diff to event err

* fix: event's move flag error
This "move" flag does not actually mean that the insertion
is caused by the move op.
就算是 move 造成的它不一定就能是 true
它得是下游真的能在“前一个版本的 array 里找到“,才能是 true

* fix: remove redundant elements from the movable list

The Movable List is currently flawed; an element may not exist on the movable list state, yet there are operations that revive its corresponding list item. In such cases, the diff calculation does not send back the corresponding element state (this occurs when tracing back, which fuzz testing currently does not cover. It might only be exposed by randomly switching to a version and then checking for consistency; otherwise, as long as all elements are in memory, this problem does not arise).

Moreover, there is no need to store elements in the state that do not have a corresponding list item. They will be deleted during the Snapshot, and relying on "them still being in the state" is incorrect behavior. Such adjustments also eliminate the need to maintain the `pending_elements` field.

By allowing the opgroup to record the mapping from pos id to state id, we can ensure that the events sent to the movable list state will include the corresponding state.

Movable List 现在是有错的,elem 可能不存在 movable list state 上,但是又有操作把它对应的 list item 复活了,此时 diff calc 不会把对应 element 状态发送回来(往前回溯的时候会出现,fuzz 现在没覆盖到。得有随意切换一个版本然后 check consistency 才可能会暴露;否则现在大家 elements 都在内存,就没这个问题)

而且我们没有必要在状态中存储没有对应 list item 的 element。在 Snapshot 的时候它们都会被删掉,如果依赖了“它们还会在 state 内”就是错误的行为。这样的调整也让我们不需要去维护 pending_elements 这个 field 了

通过让 opgroup 记录了 pos id → state id 的映射,可以保证发给 movable list state 的事件中会带上对应的 state

* test: make fuzzer stricter

* test: test expectation error

* refactor: rename stable pos to cursor

* tests: chore list bench init

* test: add bench

* bench: add mov & set bench

* feat(wasm): movable list js api

* fix: make movablelist able to attach even if it's already attached & refine the type of subscribe

* fix: remove the loro doc param in .unsub

* refactor: refine ts types and export setContainer api

* chore: fix warnings

* chore: rm debug logs

* perf: reduce mem usage of opgroup

* bench: add list criterion bench

---------

Co-authored-by: Leon <leeeon233@gmail.com>
2024-04-26 12:08:53 +08:00
github-actions[bot]
1b59909490
chore: version packages (#332)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-24 19:29:58 +08:00
Leon Zhao
24cf9b9138
chore: prerelease 2024-04-24 19:26:20 +08:00
Leon Zhao
684d121702
fix: insert attach container (#331)
* fix: insert attached container

* test: insert attached container

* test: add more assert for insert container
2024-04-24 16:50:05 +08:00
Zixuan Chen
9d4f7aa8a3
DeltaRope (#327)
* feat: delta rope support init

* perf: use generic-btree v0.9.0

* refactor: improve readability and maintainability

* fix: fix several issues about composing

* fix: a few more issue about composing deletions

* test: rich text

* fix: cover more edge cases

* refactor: use deltarope for list event

* refactor: replace text delta with DeltaRope

* fix: list fuzz err

* fix: safety issue on insert_many

* chore: refine impl of text delta

* refactor: use Replace instead of insert+del in DeltaItem (#330)

* refactor: use Replace instead of insert+del in DeltaItem

* fix: each deltaitem should have non-zero rle_len
Updated generic-btree dependency to version 0.10.3 and refactored DeltaItem and DeltaRope implementations in loro-delta. Refine compose impl

* fix: update generic-btree to fix the update leaf issue

* chore: lockfile

* chore: clippy fix

* refactor: make composing easier to understand

* refactor: simplify the impl of composing
2024-04-24 13:53:26 +08:00
Zixuan Chen
f99bfd8e21
Refactor rm unused code (#328)
* chore: init

* fix: fuzz config

* refactor: rm unused code
2024-04-22 21:20:00 +08:00
github-actions[bot]
4700ead1c1
chore: version packages (#326)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-18 19:56:47 +08:00
Zixuan Chen
73e3ba59c3
chore: release (#325) 2024-04-18 17:02:21 +08:00
Zixuan Chen
ce992b9de0
refactor: refine ts type (#322) 2024-04-18 09:47:14 +08:00
Zixuan Chen
a32b75ad3b
fix(js): allow convert from undefined to LoroValue (#323) 2024-04-18 09:47:00 +08:00
Zixuan Chen
2fc44206fb
Fix rich text cursor pos (#324)
* test: add text cursor test

* fix: should return event index
2024-04-18 09:46:50 +08:00
github-actions[bot]
e8352ad38d
chore: version packages (#321)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-17 15:15:17 +08:00
Zixuan Chen
598d97e013
chore: prerelease 2024-04-17 15:11:07 +08:00
Zixuan Chen
01b45598d8
refactor: parse uint8array to binary (#320)
and refine Awareness type in ts
2024-04-17 15:09:03 +08:00
Zixuan Chen
27566ed13a
docs: update how to publish new npm pkgs 2024-04-14 22:21:58 +08:00
github-actions[bot]
d51c566326
chore: version packages (#319)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-14 22:15:07 +08:00
Zixuan Chen
8f8d0989ca
ci: fix release-wasm 2024-04-14 22:04:59 +08:00
Zixuan Chen
a1fc2e3db6
ci: add release 2024-04-14 21:55:35 +08:00
Zixuan Chen
37311a52c8
feat: awareness (#318)
* feat: awareness

* refactor: new api wrapper for js & fix consistency

* chore: rm redundant file

* refactor: expose it to the rust crate
2024-04-14 20:44:20 +08:00
Zixuan Chen
08253f9d87
chore(ci): fix deno type conflict 2024-04-12 18:21:49 +08:00
Zixuan Chen
df8bf029a6
chore: experimentally publish to jsr 2024-04-12 17:33:28 +08:00
Zixuan Chen
8752c229f9
chore(js): bump version 2024-04-10 12:30:21 +08:00
Zixuan Chen
1918cd7ac3
refactor: rename stable pos to cursor (#317) 2024-04-10 12:26:12 +08:00
Zixuan Chen
454b4088a6
chore(rs): bump versions of rust crates 2024-04-09 16:23:48 +08:00
Zixuan Chen
095e93b327
chore(wasm): bump version 2024-04-09 16:10:28 +08:00
Zixuan Chen
aca14fda10
fix(ts): types 2024-04-09 16:05:30 +08:00
Zixuan Chen
81ae2abacc
Cursors (#290)
This PR introduces support for retrieving and querying cursors.

## Motivation

Using "index" to denote cursor positions can be unstable, as positions may shift with document edits. To reliably represent a position or range within a document, it is more effective to leverage the unique ID of each item/character in a List CRDT or Text CRDT.

## Updating Cursors

Loro optimizes State metadata by not storing the IDs of deleted elements. This approach, while efficient, complicates tracking cursor positions since they rely on these IDs for precise locations within the document. The solution recalculates position by replaying relevant history to update stable positions accurately. To minimize the performance impact of history replay, the system updates cursor info to reference only the IDs of currently present elements, thereby reducing the need for replay.

Each position has a "Side" information, indicating the actual cursor position is on the left, right, or directly in the center of the target ID.

Note: In JavaScript, the offset returned when querying a Stable Position is based on the UTF-16 index.

# Example

```ts
const loro = new Loro();
const list = loro.getList("list");
list.insert(0, "a");
const pos0 = list.getStablePos(0);
list.insert(1, "b");
{
  const ans = loro.queryStablePos(pos0!);
  expect(ans.offset).toEqual(0);
  expect(ans.side).toEqual(0);
  expect(ans.update).toBeUndefined();
}
list.insert(0, "c");
{
  const ans = loro.queryStablePos(pos0!);
  expect(ans.offset).toEqual(1);
  expect(ans.side).toEqual(0);
  expect(ans.update).toBeUndefined();
}
list.delete(1, 1);
{
  const ans = loro.queryStablePos(pos0!);
  expect(ans.offset).toEqual(1);
  expect(ans.side).toEqual(-1);
  expect(ans.update).toBeDefined();
}
```
2024-04-09 16:01:37 +08:00
Zixuan Chen
205fa94653
chore: bump loro-crdt 2024-04-05 02:25:25 +08:00
Zixuan Chen
9365cc049c
refactor(ts): make method types better (#315)
* refactor(ts): make method types better

* docs: add missing docs omitted by skip_typescript
2024-04-05 01:43:36 +08:00
Zixuan Chen
a215d88c03
refactor: add concrete type for each different container (#313) 2024-04-03 19:24:48 +08:00
Zixuan Chen
7561448323
refactor: replace "local" and "fromCheckout" in event with "triggeredBy" (#312)
* refactor: replace "local" and "fromCheckout" in event with "triggeredBy"

* refactor: simplify the name to `by`
2024-04-03 17:56:01 +08:00
Zixuan Chen
fc20f1f7d0
chore: add coverage report cli (#311)
* chore: add coverage report cli

* chore: ignore trace-*
2024-04-02 22:27:45 +08:00
Zixuan Chen
756d7a9404
fix: type error caused by merging 2024-04-01 17:42:02 +08:00
Leon Zhao
51890ff8d8
fix: decode iter return result by updating columnar to 0.3.4 (#309) 2024-04-01 17:29:07 +08:00
Zixuan Chen
3da28459b0
feat: decode import blob meta (#307)
* feat: decode import blob meta

* chore: rm logs
2024-04-01 17:25:43 +08:00
Zixuan Chen
3c124f454b
feat: access value/container by path (#308)
* feat: access elem by path

* fix: parse tree node index by str

* fix: reuse treeid try_from

* chore: rename methods on lib.rs

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-04-01 17:25:12 +08:00
Zixuan Chen
c788ff1727
docs: clarify that peer id should be convertable to a u64 (#306)
#305
2024-04-01 10:06:13 +08:00
Zixuan Chen
25b0f8096e
docs: refine wasm docs (#304) 2024-04-01 09:03:32 +08:00
Zixuan Chen
1015ee33a7
chore: bump loro-crdt 2024-03-31 02:12:06 +08:00
Zixuan Chen
367975e1ac
fix: type error 2024-03-31 02:10:49 +08:00
Zixuan Chen
c3d5e92407
refactor: rm isAttached type on containers (#303) 2024-03-31 02:06:57 +08:00
Zixuan Chen
a8c059d96b
fix: type mismatch and add type info for whether a container is attached (#302) 2024-03-31 01:12:08 +08:00
Zixuan Chen
3fe9cc1969
fix: avoid bundler change the classname (#301) 2024-03-31 01:11:55 +08:00