Commit graph

729 commits

Author SHA1 Message Date
Zixuan Chen
d08a865d50
fix: getOrCreateContainer should not throw if value is null (#576)
Some checks are pending
Release WASM / Release (push) Waiting to run
Test All / build (push) Waiting to run
* fix: getOrCreateContainer should not throw if value is null

resolve #575

* chore: update changeset release info
2024-12-10 15:01:12 +08:00
Zixuan Chen
01fccc5a7d
feat: return import status in import_batch method (#573)
Some checks are pending
Release WASM / Release (push) Waiting to run
Test All / build (push) Waiting to run
* feat: return import status in import_batch method

* feat(wasm): add importUpdateBatch returned status support

* chore: add changeset update

* chore: update ffi return type
2024-12-09 16:31:21 +08:00
Zixuan Chen
0325061476
fix: dead loop when importing updates (#570)
Fix the dead loop issue here: https://gist.github.com/sunflowerdeath/c0e2b46b6f5d2e32d368f8e04f730237

# Reason for the Dead Loop in `find_common_ancestor`

### Original Assumptions

1. **Dependency Assumption**

   If a DagNode is depended upon by other nodes, the direction of this dependency will only point to before the end of the node. This assumption is used in system design.

2. **Node Overlap Assumption**

   DagNodes do not overlap with each other. When retrieving DagNodes from two different positions:
   - They are either completely identical
   - Or they do not overlap at all
   - If there is overlap, it means they are the same node

### Current Issues

1. **Issues Caused by Lazy Loading**

   Due to the use of lazy loading, new situations have arisen. Initially:
   - A certain DagNode is not depended upon by other DagNodes
   - Therefore, the initially retrieved form is complete

2. **Impact of Subsequent Loading**

   After loading additional DagNodes:
   - The newly loaded DagNode may depend on the internal position of the previously complete DagNode
   - This causes the originally complete DagNode to be split into multiple smaller DagNodes

3. **Violation of Original Assumptions**

   In this case, when retrieving the content of the original DagNode again:
   - Other DagNode dependencies may appear in the middle
   - There may be overlaps on the DagNode
   - This violates the original design assumptions

# Fix Approach

Remove the assumption in the original implementation that "overlapping DagNodes will not occur." When overlapping DagNodes are found, retain the shorter one and remove the longer one.
2024-11-29 13:01:15 +08:00
Zixuan Chen
d6966aca34
fix: fractional index is enabled by default now (#561)
* fix: tree should use jitter 0 by default

Otherwise, there may be inconsistency between the event and the actual data. But this will increase the cost when the index property is not used.

* chore: add changeset

* chore: fix warning

* refactor: enable tree move by default

* docs: update related docs
2024-11-18 14:32:56 +08:00
Zixuan Chen
35e7ea5f54
fix: calc change count (#556)
Some checks failed
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
* fix: calc change count

* fix: len ops
2024-11-13 16:52:41 +08:00
Zixuan Chen
ee26952fc0
feat: Add isDeleted() method to containers (#555)
* feat: Add isDeleted() method to containers

- Add isDeleted() method to all container types (Text, Map, List, Tree, etc.)
- Fix deletion tracking for containers in tree operations
- Add tests to verify deletion state across different scenarios

* chore: fix redundant field names

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-11-12 21:15:46 +08:00
Zixuan Chen
9abeb81747
feat(wasm): add methods to modify vv (#552)
* feat(wasm): add methods to modify vv

* test: add js test for updating vv
2024-11-10 16:36:09 +08:00
Zixuan Chen
cc570d2b80
chore: bump Rust crates to v1.1.0 2024-11-09 23:35:17 +08:00
Zixuan Chen
778ca5452d
feat: allow users to query the changed containers in the target id range (#549)
* feat: allow users to query the changed containers in the target id range

* chore: add changeset note

* chore: update cargo toml

* test: add related tests and add a commit before get_changed_container_in
2024-11-09 21:00:07 +08:00
Zixuan Chen
6e878d216a
Feat add API to query creators, the last editors/movers (#548)
* feat: LoroMap::get_last_editor

* feat: get creator/mover/editor of items

* chore: add changeset

* chore: fix warnings
2024-11-09 19:17:56 +08:00
Zixuan Chen
4f0d499d4b
perf: use a priority-queue-based search for updating text (#544)
* perf: use a priority-queue-based search for updating text

It tends to produce diffs with more continuous edits,
which is more efficient when we need to apply them to CRDTs

* fix: use better text diff calc

* refactor: add text update options struct

* chore: update text.update comments

* chore: fix warnings

* fix: rm a dumb optimization
2024-11-09 16:35:15 +08:00
Zixuan Chen
661610165b
perf: import batch optimize - use the largest snapshot first (#547)
* perf: import batch optimize - use the largest snapshot first

* chore: add changeset info
2024-11-09 16:10:13 +08:00
Zixuan Chen
8486234aba
fix: get correct encode blob info (#545)
* fix: get correct encode blob info

* fix: is_snapshot

* test: fix test case

* test: update ts tests

* chore: add change file

* chore: fix warnings
2024-11-09 15:43:11 +08:00
Zixuan Chen
715bf759c3
refactor!: reduce footguns when using for_each(fn) on rust (#540)
Some checks are pending
Release WASM / Release (push) Waiting to run
Test All / build (push) Waiting to run
2024-11-09 01:07:38 +08:00
Zixuan Chen
044a1fd31c
fix: avoid creating non-root containers that doesn't exist by get_container api (#541) 2024-11-09 00:51:21 +08:00
Zixuan Chen
9b60d01be2
fix: define the fork behavior when the doc is detached (#537)
* fix: define the fork behavior when the doc is detached

* chore: add release note
2024-11-04 22:34:33 +08:00
Zixuan Chen
7bf6db7398
feat: add push/push_container to LoroText/LoroList/LoroMovableList (#534)
Some checks are pending
Release WASM / Release (push) Waiting to run
Test All / build (push) Waiting to run
* feat: add push_container and push_str to list/text

* chore: add changeset file
2024-11-04 17:03:24 +08:00
Zixuan Chen
f78e1da8a0
refactor(rust): extract LoroStringValue, LoroMapValue, LoroBinaryValue, LoroListValue (#535)
* refactor: extract LoroStringValue LoroMapValue LoroListValue LoroBinaryValue

* chore: clippy fix

* chore: fix warning
2024-11-04 17:03:14 +08:00
Zixuan Chen
9731e6ace2
chore: bump rust crates versions 2024-10-22 23:07:38 +08:00
Zixuan Chen
484d6db7a1
fix: fork at should restore detached state (#523)
Some checks failed
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
2024-10-20 00:21:04 +08:00
Zixuan Chen
efad275e1a
fix: forkAt should inherit the config and auto commit from the original doc 2024-10-19 22:46:41 +08:00
Leon Zhao
3e4005d113
chore: rename tree event (#520) 2024-10-19 11:24:13 +08:00
Leon Zhao
dedc8e087e
chore: rename wasm export from (#519)
* chore: rename wasm export from

* fix: clippy
2024-10-18 19:49:40 +08:00
Zixuan Chen
d872ac6488
fix: avoid auto unsubscribe (due to gc) in js env (#516)
* fix: avoid auto unsubscribe (due to gc) in js env

* fix: drop subscription if the subscriber is dropped

* perf: use manually drop

* fix: rm once_cell dep

* fix: should call unsubscribe
2024-10-17 13:33:29 +08:00
Zixuan Chen
9650c52a30
chore: rm unused deps 2024-10-17 02:01:16 +08:00
Zixuan Chen
e0b7280024
perf: use better data structure for frontiers (#515)
* perf: optimize high concurrency performance

* refactor: use new fronteirs impl init

* refactor: fix more type err

* refactor: fix more type err

* fix: fix more type err

* fix: all type errors and warnings

* fix: type error in tests

* fix: Debug impl

* fix: singular value

* fix: err

* fix: avoid potential err of frontiers merging

* fix: a few test issues

* test: add frontiers tests

* fix: all test errors

* chore: fix warnings

* chore: rm testing code

* perf: optimize dag related performance

* chore: rm outdated marks

* refactor: frontiers's map variant should contains at least 2 elements
2024-10-16 16:22:36 +08:00
Zixuan Chen
e14f81aa37
perf: optimize high concurrency performance (#514)
* perf: optimize high concurrency performance

* fix: err

* chore: fix warnings
2024-10-16 12:01:31 +08:00
Zixuan Chen
84fbd6d368
fix: isContainerDeleted cache err (#513) 2024-10-15 11:22:46 +08:00
Zixuan Chen
344613cd76
fix: tracker skip applied deletion error (#512) 2024-10-15 02:18:03 +08:00
Zixuan Chen
13898f0af9
Perf: commit speed & text insert cache (#511)
* perf: reduce the cost of is_deleted check

* perf: speedup get_or_insert container

* perf: reduce the cost of get or insert with

* fix: store len

* perf: cache cursor for text insertions

* chore: fix warnings
2024-10-15 02:09:30 +08:00
Zixuan Chen
40e6c524c7
perf: optimize batch container registrations on arena (#510) 2024-10-14 13:26:58 +08:00
Zixuan Chen
2b9194cf42
perf: optimize shrink frontiers 2024-10-12 13:54:47 +08:00
Zixuan Chen
f4823191e7
fix: movable list undo del impl 2024-10-12 13:49:36 +08:00
Zixuan Chen
ba7c1b7b94
fix: movable list undo impl (#509) 2024-10-12 11:53:50 +08:00
Zixuan Chen
23a99f53c9
Test fuzz oom (#508)
* test: avoid fuzz oom

* fix: get pending dag node err
2024-10-12 00:49:23 +08:00
Zixuan Chen
b5e153a33d
fix: movable list undo/redo error (#507)
* fix: movable list undo/redo error

* chore: rm log

* chore: rm unused

* chore: use deno 2.x

* ci: remove build step
2024-10-11 23:36:04 +08:00
Zixuan Chen
f2365a837b
feat: redact (#504) 2024-10-10 12:47:41 +08:00
Zixuan Chen
66b8a76fdf
test: add compatibility tests (#503) 2024-10-09 17:42:05 +08:00
Zixuan Chen
59efca6fe5
perf: optimize kv export_all by reusing encoded block (#501)
* perf: optimize kv export_all by reusing encoded block

* chore: fix warnings

* perf: reuse encoded block bytes

* perf: optimize encode/decode speed further

* chore: fix warning
2024-10-09 10:22:17 +08:00
Zixuan Chen
afbcee99b3
perf: avoid memory leak when forking repeatedly (#500)
* perf: avoid memory leak when forking repeatedly

* fix: use the safest way to fork
2024-10-08 17:24:08 +08:00
Zixuan Chen
c8f776f018
Refactor rename trimmed snapshot to shallow snapshot (#499)
* refactor: rename trimmed snapshot to shallow snapshot

* refactor: rename trimmed_vv and trimmed_frontiers

* refactor: rename rest of trimmed* into shallow*

* chore: rename cov message

* chore: fix warnings
2024-10-08 13:15:14 +08:00
Leon Zhao
66ad26aeeb
fix: create event cannot find parent (#498)
* test: add a failed test

* fix: tree parent may be deleted when importGreaterUpdates

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-10-06 19:36:56 +08:00
Zixuan Chen
27e3a593a6
Perf: optimize speed of large maps (#496)
* perf: optimize speed of large maps

* chore: record current perf

* chore: throw error when clippy has warnings
2024-10-05 14:35:56 +08:00
Leon Zhao
405cbe047c
feat: add import status (#494) 2024-10-05 08:18:12 +08:00
Zixuan Chen
e1bf4a858d
test: bench large folder with 1M files & 100M ops (#495)
* test: bench large folder with 1M files & 100M ops

* test: update task

* test: display trimmed snapshot size

* chore: record current test result

* perf: optimize the speed of ensure_vv
2024-10-05 01:10:54 +08:00
Zixuan Chen
b117880fed
fix: type err 2024-10-04 09:55:49 +08:00
Zixuan Chen
e3a7757610
refactor: change the first param of travel change from id to ids (#492)
* refactor: change the first param of travel change from id to ids

* fix: fix all warnings and refine the impl of subscription

* chore: use pnpm
2024-10-03 21:09:18 +08:00
Zixuan Chen
51c9b40022
fix: fork error (#493)
* fix: fork error
fixed #490

* chore: fix warnings
2024-10-03 21:08:05 +08:00
Leon Zhao
b1e03d914e
fix: do not set peer id with max (#491) 2024-10-03 21:07:11 +08:00
Zixuan Chen
a93efd5f51
refactor!: better event api (#489)
- (js) Refactor subscription mechanism to return unsubscribe function
- (rust) Implement must-use struct for subscriptions in Rust
- Enhance API ergonomics for JavaScript-like environments
- Improve resource cleanup and prevent potential memory leaks
2024-10-03 15:14:21 +08:00