Commit graph

107 commits

Author SHA1 Message Date
Zixuan Chen
c8f776f018
Refactor rename trimmed snapshot to shallow snapshot (#499)
* refactor: rename trimmed snapshot to shallow snapshot

* refactor: rename trimmed_vv and trimmed_frontiers

* refactor: rename rest of trimmed* into shallow*

* chore: rename cov message

* chore: fix warnings
2024-10-08 13:15:14 +08:00
Zixuan Chen
27e3a593a6
Perf: optimize speed of large maps (#496)
* perf: optimize speed of large maps

* chore: record current perf

* chore: throw error when clippy has warnings
2024-10-05 14:35:56 +08:00
Leon Zhao
405cbe047c
feat: add import status (#494) 2024-10-05 08:18:12 +08:00
Zixuan Chen
e1bf4a858d
test: bench large folder with 1M files & 100M ops (#495)
* test: bench large folder with 1M files & 100M ops

* test: update task

* test: display trimmed snapshot size

* chore: record current test result

* perf: optimize the speed of ensure_vv
2024-10-05 01:10:54 +08:00
Leon Zhao
b1e03d914e
fix: do not set peer id with max (#491) 2024-10-03 21:07:11 +08:00
Zixuan Chen
de93d34a9c
fix: return err if snapshot container has unknown container (#488) 2024-10-02 14:24:57 +08:00
Zixuan Chen
09a004e365
refactor: Loro import function should return LoroEncodeError (#487)
* refactor: Loro import function should return LoroEncodeError

- Change return type
- Add support for handling potential errors:
  - Invalid or non-existent frontiers
  - Exporting old snapshot format from trimmed snapshot doc
- Improve compatibility with trimmed docs

* fix: add tests and fix a few places that need to return err
2024-10-02 11:11:08 +08:00
Zixuan Chen
c8a60977c1
Refactor!: List state snapshot schema for v1.0 (#485)
* refactor!: change how we encode list state for fast-snapshot

* perf: speedup tree decode

* test: add movable list state snapshot test

* chore: add state encoding overhead test

* fix: error about invalid tree node creation

* chore: update example

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-10-01 16:11:53 +08:00
Zixuan Chen
6d04066cd0
refactor: avoid footgun of impl ord for cid 2024-09-30 18:09:49 +08:00
Zixuan Chen
bef39ce6b5
Feat: allow editing on detached mode (#473)
* refactor: add detached editing config and prepare the architecture for editing detached doc

* feat: subscribe for peer id change

* fix: undo after checkout & add tests for detached editing

* test: add fuzzer for detached editing

* feat: expose detached editing configure to wasm

* test: add wasm test for detached editing
2024-09-24 11:16:59 +08:00
Zixuan Chen
bdf006ed2e
feat: fork doc at the target version (#469) 2024-09-23 09:51:07 +08:00
Zixuan Chen
d460346585
feat: jsonpath experimental support (#466)
* feat: jsonpath experimental support

* fix: add support for negative index unionindex and unionkey

* chore: export lorodoc in loro-js and fix a few tests related to map's entries order

* chore: fix type err
2024-09-19 19:22:39 +08:00
Zixuan Chen
e592d4f830
fix: raise error if moving a deleted node 2024-09-17 16:04:00 +08:00
Zixuan Chen
8dcb619147
fix: raise error if perform action on a deleted container (#465)
- Add ContainerDeleted error variant to LoroError
- Implement is_deleted() method for container handlers
- Check for deleted containers before applying operations
- Add dead_containers_cache to DocState to track deleted containers
- Update apply_diff to optionally clear dead container cache
- Add tests for handling operations on deleted containers
- Implement checkout to reset container deleted cache
- Minor code cleanup and typo fixes
2024-09-17 12:09:20 +08:00
Zixuan Chen
f1374c3bd0
Merge branch 'main' into dev 2024-09-12 22:30:42 +08:00
Zixuan Chen
4a27a0645f
Merge branch 'dev' into feat-gc 2024-09-11 10:54:29 +08:00
Leon Zhao
07671ea9fd
feat: add old parent and old index in tree diff (#452)
* feat: add old parent in tree diff

* chore: enable ci

* feat: add old_index to tree diff

* fix: new fractional index config

* fix: cargo fix

* fix: add FractionalIndexNotEnabled error

* fix: move config to tree state

* fix: error string

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-09-09 16:16:02 +08:00
Zixuan Chen
7be915e87d
chore: bump rust crates version
Some checks failed
Release WASM / Release (push) Has been cancelled
Test All / build (push) Has been cancelled
2024-09-07 20:01:33 +08:00
Zixuan Chen
e7bf12cc14
(GC) Checkout on text/list that needs to retrieve content from the trimmed state (#446)
* feat: export gc snapshot without latest state when ops len is small

* test: add a test for time tracker usage

* chore: add gc snapshot bench

* chore: simplify

* test: record timestamp for time tracker

* fix: checkout to a version that needs to use gc state

* refactor: make list internal event more reasonable

* fix: checkout list/movable list on gc doc
2024-09-05 14:30:22 +08:00
Zixuan Chen
3d5a5da425
feat: remove dropped containers on exported gc snapshot 2024-09-04 19:52:53 +08:00
Zixuan Chen
4c325ab87c
Merge branch 'dev' into feat-gc 2024-09-03 21:21:14 +08:00
Leon Zhao
c1620fdb37
feat: memkv export import all (#422)
* feat: sstable

* fix: add magic number version

* feat: new mem kv store based sstable

* feat: binary_search

* fix: sstable iter scan

* fix: new mem kv

* feat: add cache for sstable

* fix: encode schema comment

* fix: sstable iter scan

* chore: clean

* fix: export all

* fix: sstable scan bound

* fix: sstable iter scan next==prev

* fix: merge iter next_back

* fix: mem kv export

* chore: clean

* fix: prev to key

* fix: prev find block

* fix: get prev block idx

* refactor: kv store

* fix: checksum when import

* fix: meta first last key

* Revert "fix: meta first last key"

This reverts commit a069c1ed37.

* fix: skip empty iter

* fix: remove key from large block

* chore: comment

* feat: compress block

* fix: remove key in large block

* chore: const

* doc: intro sstable encode

* test: add kv store fuzz

* style: format file

* feat: add fuzz to kv store (#428)

* fix: kv fuzzer

* fix: debug

* bk

* fix: block iter next back

* fix: block prev iter left = next idx

* feat: move kv store a crate

* fix: remove value len from normal block

* doc: sstable format

* test: add more test

* test: add test

* feat: new merge iter

* chore: revert

* fix: rename next back

* fix: rename mem sstable

* fix: rename to mem

* fix: use Bytes as key

* fix: use simple merge iter

* feat: compress option

* fix: remove empty iter

* style: refine some impl details

* fix: large block compress

* feat: use write read for encode

* doc: refine doc

* fix: simplify the first chunk

* feat: import many times

* refactor: refine styles

* test: fuzz merge iter

* fix: rename peek_xxx()

* fix: better sstable iter inner

* fix: use mem kv store

* pref: mem kv store

* perf: export mem kv

* chore: clean

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-08-30 11:44:34 +08:00
Zixuan Chen
4fa34a318a
chore: rm keywords 2024-08-29 23:07:47 +08:00
Zixuan Chen
a38cf178d1
chore: bump rust crates version 2024-08-29 23:02:03 +08:00
Zixuan Chen
819d1f86b9
feat: checkout map 2024-08-27 00:05:11 +08:00
Zixuan Chen
4e64051c58
fix: detect importing outdated updates on gc doc 2024-08-26 22:14:23 +08:00
Zixuan Chen
e084eca580
Merge branch 'main' into dev 2024-08-25 21:12:07 +08:00
Zixuan Chen
1812caea65
Refactor: use kv internally for docstate (#426)
* refactor: use kv in state

* refactor: do not load the state into the inner fxhashmap if not needed

* refactor: calc offset without unsafe code

* style: replace unsafe code
2024-08-24 14:16:06 +08:00
Zixuan Chen
6f9329adc2
fix: change_store loading cache error 2024-08-18 18:15:53 +08:00
东灯
251c9c7787
feat: check invalid root container name (#411) 2024-07-29 15:06:31 +08:00
Patrick Arminio
b8f9fafed0
Add code to the default rich text configuration (#406) 2024-07-21 19:14:22 +08:00
Zixuan Chen
59f09e7162
Merge branch 'main' into zxch3n/kv-store-trait 2024-07-17 12:16:53 +08:00
东灯
c710ec3dd3
feat: add more text api (#398)
* feat: add char_at

* feat: add slice

* feat: add splice

* feat: add rust/typescript API

* fix: ci test

* fix: patch

* fix: patch

* Update crates/loro-wasm/src/lib.rs

Co-authored-by: Zixuan Chen <remch183@outlook.com>

* Update crates/loro-wasm/src/lib.rs

Co-authored-by: Zixuan Chen <remch183@outlook.com>

* Update crates/loro/src/lib.rs

Co-authored-by: Zixuan Chen <remch183@outlook.com>

* perf: use entity index

* fix: patch

* fix: error brackets

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-07-14 13:06:04 +08:00
Zixuan Chen
46000420e8
Merge main 2024-07-12 16:15:54 +08:00
Zixuan Chen
a924b3e55b
fix: change block encode & decode 2024-07-11 23:54:09 +08:00
东灯
86c760abd0
feat: add insert_utf8 and delete_utf8 for Rust Text API (#396)
* feat: add insert_utf8

* chore: merge insert functions

* fix: use utf8_to_unicode_index to reslove index

* fix: add bound-check and use unicode PosType

* feat: add delete_utf8

* perf: O(LogN) insert_utf8

* feat: add utf-16 cross unicode check

* perf: O(LogN) delete_utf8

* chore: add api

* chore: remove unused function

* fix: api name and bindgen name

* test: add utf8 js test

---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-07-10 10:20:08 +08:00
Zixuan Chen
f802ede4b9
refactor: impl HasCounter explicitly to speedup 2024-07-09 00:02:41 +08:00
Zixuan Chen
d69e4c9c30
Refactor ChangeStore to use external KV store and improve block management
- Add ID serialization methods in loro-common
- Update KvStore trait with binary search and double-ended iterator
- Modify export/import blocks to use bytes::Bytes
- Refactor ChangeStore to use external KV store for persistence
- Implement flush, encode_all, and decode_all methods for ChangeStore
- Update change retrieval and iteration methods to work with new block management
- Improve block parsing and caching mechanisms
2024-07-08 17:33:57 +08:00
Zixuan Chen
9047065843
Fix undo with checkout (#375)
* fix: should transform checkout event

* chore: update fuzz dep

* chore: add pos to error info

* fix: clear undo/redo stack when checkingout

* test: update fuzz dep

* test: a new failed test case

* fix: tree transform

* chore: fuzz

* chore: add log

* chore: add more logs

* fix: compose err

* chore: fuzz test dep

* test: a failed tree case

* fix: undo tree event

* fix: do not compare tree position in fuzz

* fix: fuzz rev

* test: a failed tree case

* fix: add tree compose

* chore: add comment

* chore: fuzz

* fix: test

* fix: tree transform

* fix: tree transform index

* fix: sort tree index

* chore: fuzz

* fix: undo/redo remote change effect compose

* bk

* fix: tree undo redo (#385)

* fix: event hint none

* chore: fuzz version

* ci: fuzz

* bk: weird err

* fix: type err

* fix: fractional index between

* fix: wasm counter feature

* test: a new failed case

* fix: recursively create child nodes

* fix: filter empty event

* bk

* bk

* fix: tree undo redo remap

* chore: clean

* bk

* fix: tree need remap first

* fix: tree undo effect

* fix: tree diff calc

* fix: tree fuzz check eq func

* fix: remove EventHint None

* chore: cargo fix

* fix: tree uncreate

* fix: fuzz tree assert only structure

* refactor: rename methods

* fix: movable tree apply delta

* fix: another movable list issue

* chore: fuzz only check 1 actor's history

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-07-04 18:15:44 +08:00
Zixuan Chen
d6c8a632ad
Merge branch 'main' into zxch3n/container-store 2024-06-18 18:09:14 +08:00
Zixuan Chen
0d3709b79e
feat: map snapshot 2024-06-13 17:50:29 +08:00
Leon Zhao
afac34755f
feat: implementing Counter and expose to js side (#384) 2024-06-13 14:43:19 +08:00
Zixuan Chen
d71fee144d
bk: container store impl 2024-06-12 11:43:02 +08:00
Leon Zhao
2df2a52b05
feat: Stable JSON representation for history (#368)
---------

Co-authored-by: Zixuan Chen <remch183@outlook.com>
2024-06-07 13:18:30 +08:00
Zixuan Chen
54637a7b42
Publish v0.16.2 to crates.io (#374)
* chore: mark dev-utils not published

* chore: release rust crates

* chore: publish fi

* chore: pre publish fi

* chore: update lockfile

* test: change fuzz rev
2024-05-29 22:55:55 +08:00
Zixuan Chen
321e0e72a4
Undo (#361)
https://github.com/loro-dev/loro/pull/361

---------

Co-authored-by: Leon Zhao <leeeon233@gmail.com>
2024-05-21 06:14:49 +08:00
Leon Zhao
6e14e5b117
Feat: Make the encoding format forward and backward compatible (#329) 2024-05-13 21:37:10 +08:00
Gábor Szabó
d958767b13
add repository to Cargo.toml (#358) 2024-05-10 21:14:17 +08:00
Zixuan Chen
ea7d8b0669
fix: refine error message on corrupted data (#356)
* fix: refine error message on corrupted data

* fix: comment
2024-05-10 11:06:01 +08:00
Zixuan Chen
0660b1a1be
fix: upgrade wasm-bindgen to fix str free err (#353)
* fix: upgrade wasm-bindgen to fix str free err

* chore: fix ci
2024-05-09 15:22:34 +08:00