Repository navigation
Release/2.6.0 - #136
Merged
Merged
Release/2.6.0#136
Conversation
busyTimeout never took effect on Android: SQLite waited 2500 ms, as the framework sets it, for a lock that another connection held. On Android 9 and higher, it was given to OpenParams' deprecated setIdleConnectionTimeout, which means how long a connection may be idle before the framework closes it, and which SQLiteOpenHelper drops anyway, as the OpenParams.Builder it copies the parameters into doesn't copy that one, on every version from Android 9 to 16. It is now set with PRAGMA busy_timeout, when the database is configured. From Android 13 on, a database in WAL mode reads through several connections, so execPerConnectionSQL runs it on each of them, and on those opened later; it only runs a PRAGMA that returns a row, as this one does, from Android 13 on. Before, a database that SQLlin opens has a single connection, which runs it through rawQuery. The guides to getting started listed busyTimeout, and in Chinese journalMode, among the parameters that only work on Android 9 and higher; only inMemory and the lookaside parameters do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The default lookasideSlotSize and lookasideSlotCount, 0, which the documentation said left SQLite's default configuration, disabled the lookaside memory allocator on Kotlin/Native and on Android 9 and higher, as sqlite3_db_config and setLookasideConfig take 0 to disable it, while the JVM, where sqlite-jdbc can't set it, and lower versions of Android kept SQLite's default configuration. Their default is now -1, in DatabaseConfiguration and DSLDBConfiguration, which leaves SQLite's default configuration on every platform, as the Android framework's own default does; 0 still disables the allocator. The Android driver only calls setLookasideConfig when both values are 0 or more, as it rejects negative ones. The examples of the guides show the new defaults, which they explain, and the guides name sqlite3_db_config() as what sqlite-jdbc doesn't support, rather than sqlite3_config(). The error of the Kotlin/Native driver had a stray '%'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
F66 is the primitive that a reader which has to invalidate itself, as a
PagingSource does, needs: versioned { } returns the results of a
read-only scope with an opaque token of the versions of the tables it
read, captured under the lock that the read runs under, after the
tables are tracked, so that no change between the read and the capture
is missed. Tokens combine with +, taking the earlier version of a table,
and reject the tokens of another Database. awaitChange suspends until a
table changes, and onChange registers a one-shot callback, run after
the writing scope releases the lock, returning null when the token is
already stale, as it registers before it checks.
It comes from the design discussed with the MyMovie project, which
found that results, as observe gives, don't fit a PagingSource, and
that a version read without the read can't be sound, as a table only
has one once it is tracked. F13 now builds on it, with the callback
held weakly by the registry, so that a source that Pager abandons
doesn't stay registered, and the three traps of offset paging that the
project reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the JVM and Kotlin/Native, the connection is locked for each
statement rather than for a transaction, so a blocking database { }
scope on another thread can have its transaction open while the tracker
of observed queries creates its TEMP table and triggers. They are
rolled back with that transaction, while the tracker still counts their
tables as tracked: the next write failed with "no such table", and once
another table was observed, which created the TEMP table again, the
changes of the lost table were no longer counted. Android isn't
affected, as a transaction holds the primary connection there.
Before it reads the counts, the tracker now checks that the TEMP table
exists and holds a row for each tracked table, and that each tracked
table has its three triggers, in a transaction, as on Android a query
outside one may run on a reader connection. It creates again whatever
is gone, and counts a change of those tables above the version last
read, as their changes may have been missed. A failure to tell the
observed queries of a change is added to the failure of the statement
that made it, rather than taking its place.
The guide to observed queries, and the documentation of invoke and
observe, now say to write through suspendedScope while queries are
observed, as they run on another thread, and that a blocking scope
isn't serialized with them, as the guide to concurrency safety already
said of sharing a database between threads. The test stands for the
rollback by removing the row of a table from the TEMP table, and by
dropping it, through an entity of the TEMP table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The example called database suspendScope { }, which doesn't exist, in
a function that wasn't a suspend one, in English and in Chinese. It is
database suspendedScope { }, in a suspend function.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Transactions began DEFERRED on the JVM and Kotlin/Native, and EXCLUSIVE on Android. So on the JVM and Kotlin/Native, a transaction that read before it wrote failed at once with "database is locked" while another connection wrote, as SQLite doesn't wait for busyTimeout where waiting could deadlock, while on Android it waited. They now begin IMMEDIATE on every platform, taking the write lock at once, for which SQLite waits as long as busyTimeout. On Android, beginTransactionNonExclusive does it, which is what EXCLUSIVE did in WAL mode, and in DELETE mode lets other connections read until the transaction commits. On the JVM and Kotlin/Native, a read-only connection begins a deferred transaction, as it can't take the write lock. The JVM driver now begins and ends transactions with SQL, in auto-commit mode: sqlite-jdbc begins another transaction of its configured kind as soon as one is committed or rolled back, which an IMMEDIATE one couldn't do while another connection writes, so that a COMMIT that succeeded would fail. F27, choosing how a transaction begins, no longer says that the platforms differ. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SQLite rolls some transactions back itself: for a constraint with ON
CONFLICT ROLLBACK, which onConflict = OnConflict.ROLLBACK declares, for
a trigger's RAISE(ROLLBACK), and for some I/O errors. Ending such a
transaction then failed too, with "cannot rollback - no transaction is
active", and withTransaction threw that failure from its finally
block, in place of the one that made SQLite roll it back, on every
platform, and so did the DSL's transaction { }, which runs through it.
withTransaction now throws the block's failure, with the failure to end
the transaction added to it as a suppressed exception. It keeps ending
the transaction in a finally block, as a non-local return from the
block, which an inline function allows, has to end it too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bug fixes for 2.6.0
…fix/generated-key-last
…docs/correct-guide-api-examples
Fix INSERT when the generated primary key is the last property
Correct API names and generated-key types in guide examples Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The entries that came with the two pull requests were short. B7's now says why INSERT failed, a trailing comma in the column list and a VALUES tuple never closed, as the key that the database assigns is left out, and which forms weren't affected: a key declared before another property, and INSERT_WITH_ID, INSERT_OR_IGNORE and INSERT_OR_REPLACE, which write the key. B8's only named the query guide's getResult(); it now lists every correction: getResult() in the READMEs too, the import of DSLDBConfiguration, the CollateNoCase annotation, and the type of an autoIncrement key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bug fixes for 2.6.0: B7 and B8
… relation (F7) 2.5.0 replaced the joins written as clauses of SELECT, such as table SELECT INNER_JOIN<R>(other) USING column, with joins of relations, which check the type their rows are read into against the joined relations, can join any number of relations, each with its own condition, and select result columns. It announced that the old ones would be removed in the next version, which this commit does: the functions CROSS_JOIN, JOIN, INNER_JOIN, NATURAL_JOIN, NATURAL_INNER_JOIN, LEFT_OUTER_JOIN and NATURAL_LEFT_OUTER_JOIN that took relations as arguments, the SELECT and SELECT_DISTINCT that took what they returned, and the types BaseJoinClause, JoinClause, NaturalJoinClause, JoinStatementWithoutCondition and JoinSelectStatement, with the ON, USING, WHERE, GROUP_BY, ORDER_BY and LIMIT that took them. It is a breaking change: such a join becomes (table INNER_JOIN other USING column) SELECT X<R>(). With them gone, a join can start with its first relation, as in PersonTable INNER_JOIN BookTable ON (...), without FROM: every join operator is now an infix function of a relation too, which starts the join FROM(relation) would. It couldn't be before, as such a function took the place of the old CROSS_JOIN(CategoryTable) in the lambda of a table, whose implicit receiver is a relation. FROM still starts a join. The test of the old joins now covers the joins that start from a relation, of every kind each platform has; the other tests and the sample use them too. The sample only keeps the joins that its Student type fits, as the new ones check it. The guide to advanced queries starts joins from their first relation, and describes how to migrate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2.5.0 deprecated the UNION {} and UNION_ALL {} blocks, and the
beginUnion, createUnionSelectStatement and endUnion functions they
used, for bugs that wouldn't be fixed: a block read its rows into its
first SELECT's result type while returning the table's row type, so a
union of projections, result columns or joins failed with a
ClassCastException when its results were used, and a nested block was
flattened into the SQL, which SQLite evaluates from left to right, so
that it changed its meaning. The infix UNION, UNION_ALL, INTERSECT and
EXCEPT replaced them. This removes them, with UnionSelectStatementGroup,
the container they collected their SELECTs in, so that a SELECT is now
always added to the scope, transaction or trigger body it is written
in. It is a breaking change: UNION { a; b } becomes (a) UNION (b).
The test of the block is removed, as the infix operators have their
own, and the guide to advanced queries describes the migration in place
of the deprecated block.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An observed query gives results, while a reader that reads on its own
and only has to learn when the tables it read change, such as a
PagingSource that invalidates itself, needs a signal. And a version
read apart from the read can't be sound, as a table only has one once
it is tracked, which the read itself starts.
Database#versioned { } runs a scope of SELECTs, and returns its result
with a TableVersions token of the versions of the tables they read.
Like observe, it tracks the tables, captures their versions and runs
the SELECTs under the executive lock, in that order, so that no change
after the read is missed. A scope that writes is rejected before any of
its statements runs, as its token couldn't tell its own changes from
others'. Tokens combine with +, which takes the earlier version of a
table that both read, so that the combined token is stale when either
is, as for a query built from the results of another; tokens of
another Database are rejected.
awaitChange suspends until a table of the token changes. onChange
registers a one-shot callback, and registers before it checks: when
the token is already stale, it removes the registration and returns
null, so that the caller acts on the change itself, as a PagingSource
returns LoadResult.Invalid(). The registrations are a list replaced
atomically, so that a registration is either fired by a writer or
removed by onChange, never both. Callbacks run on the thread that
wrote, after the writing scope, blocking or suspended, has released the
lock, so that they may use the database.
The tracker's weak registrations for F13 are left to F13.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SQLlin had INSERT OR IGNORE and INSERT OR REPLACE. This adds the other
resolutions a statement can give: INSERT_OR_ABORT, INSERT_OR_FAIL and
INSERT_OR_ROLLBACK, which take objects, expressions, DEFAULT_VALUES and
a SELECT, as INSERT does, and UPDATE_OR_ABORT, UPDATE_OR_FAIL,
UPDATE_OR_IGNORE, UPDATE_OR_REPLACE and UPDATE_OR_ROLLBACK, written as
UPDATE. A statement's resolution takes the place of the conflict clause
of a constraint, which onConflict declares since F17: INSERT_OR_ABORT
fails where a UNIQUE ON CONFLICT IGNORE column would have skipped the
row, which is the use of a resolution that is otherwise the default.
Like INSERT_OR_IGNORE and INSERT_OR_REPLACE, the new INSERTs write the
primary key even when the database assigns it, as a conflict on it
couldn't be seen otherwise, so the three now share one builder,
Insert.insertOr, and Update.update takes the keyword it starts with.
The guide to modifying the database describes the resolutions in a new
section, including that in transaction { }, where a failing statement
rolls the transaction back, ABORT, FAIL and ROLLBACK only differ outside
one. The test checks that each failure is the constraint's, rather than
one of the SQL, and runs every form once successfully.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…VE (F27)
Since B5, every transaction begins IMMEDIATE, taking the write lock at
once. This lets a transaction begin in another TransactionMode:
EXCLUSIVE, which also keeps other connections from reading in DELETE
journal mode, and is IMMEDIATE in WAL mode; and DEFERRED, which takes
no lock until the transaction reads, nor the write lock until it
writes, but makes a transaction that reads before it writes fail at
once while another connection writes, without waiting for the busy
timeout.
In sqllin-driver, DatabaseConnection#beginTransaction(mode) begins one,
and withTransaction takes a mode. The new beginTransaction has a default
implementation, which only begins an IMMEDIATE transaction, so that a
DatabaseConnection implemented outside SQLlin still compiles; the JVM
and Kotlin/Native drivers run BEGIN with the mode, or a deferred BEGIN
on a read-only connection, as they did. Android's framework can't begin
a deferred transaction that writes, as it turns a BEGIN statement into
an EXCLUSIVE transaction of its own, so there DEFERRED begins an
IMMEDIATE one, which only takes the write lock earlier, while EXCLUSIVE
is its beginTransaction.
In the DSL, transaction(mode) { } begins one; a transaction in another
one joins it and keeps its mode. The guide to modifying the database
describes the modes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
INDEXED_BY makes a statement look its rows up through an index, and NOT_INDEXED through none, rather than through the one SQLite's query planner chooses: (PersonTable INDEXED_BY "idx_person_name") SELECT WHERE(...) and PersonTable.NOT_INDEXED DELETE WHERE(...). They return an IndexedTable, a Relation whose FROM is the table with the clause, and whose name and columns are the table's, so that a SELECT reads it as it reads the table, joins included, and an observed query or a versioned read tracks the table. It isn't a Table, so INSERT, which SQL doesn't allow it in, doesn't take it, which is checked at compile time; UPDATE, the UPDATE_OR_ ones and DELETE take it through overloads, which give the table with the clause to their builders as the target they write. The test shows the clauses are in the SQL of each statement by giving an index that doesn't exist, which fails with "no such index". The guide to modifying the database describes them with indexes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to 3.54.1.0 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sqllin-processor only gave its Java classes a JVM target, 21, through the java block, which left its Kotlin classes, all of its code, to the default: the JDK that runs the build. Built with JDK 25, they were Java 25 class files, major version 69, which a KSP run on an older JDK can't load, failing with UnsupportedClassVersionError; the Kotlin Gradle plugin warned of the inconsistent targets. CI, on JDK 21, built them for 21. The processor now uses the JVM toolchain of the other modules, JDK 21, which compiles its Java and Kotlin classes for it, whichever JDK runs the build: the jar's classes are major version 65. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EXPLAIN_QUERY_PLAN(select) tells how SQLite would run a SELECT without running it: which tables it scans, which indexes it searches, and where it sorts with a temporary B-tree. Any SELECT can be planned, joins, subqueries and compound ones included. It is taken out of the statements the scope runs, as a subquery is, and planned with its parameters, as it would run. The QueryPlanStatement it returns is a statement of its own, rather than a SELECT, so that it can't become a subquery or part of a compound, which SQL doesn't allow, and isn't counted among the tables a versioned read covers; it changes nothing. It reads the plan when it runs and closes its cursor, as the plan is small. getResults() returns a QueryPlan: its steps, each with its detail and its children, the details of all of them, and a toString() that writes them as a tree, as the sqlite3 shell does. The plan is read by the columns SQLite returns, rather than by its version, which no platform tells reliably: since SQLite 3.24.0 they are id, parent, notused and detail, and a step's parent is the id of the step it belongs to; before, as on Android below API 30, they tell no such relation, so the steps are listed without children. The details are SQLite's own text, which differs between versions, so they are left as they are, and the guide to advanced queries says to check a plan by the names of tables and indexes and by phrases such as USING INDEX, rather than by whole details. The helper that takes a SELECT out of the scope's statements is renamed takeOut(), as it no longer serves subqueries only, and gets back its documentation, which a later section had separated from it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A clause appended to a statement, as ORDER_BY, LIMIT, OFFSET, GROUP_BY, HAVING, WHERE after result columns, and UPDATE's WHERE, replaced the last statement of its container, whatever it was, as long as it was a SELECT or an UPDATE without WHERE. When another statement had been built since, as when a statement is held in a variable and completed later, that statement was dropped and never ran, while the incomplete one still did: an UPDATE held as `val update = table UPDATE SET(...)`, followed by a SELECT and then completed with `update WHERE (...)`, updated every row of the table. UPDATE's WHERE also added its parameters to the list of the UPDATE without it, which then failed with too many parameters when its SET had some. The clauses now replace the statement they complete through replaceStatement, which finds it wherever it is, as CREATE INDEX's WHERE already did, and changeLastStatement is removed. A clause appended to a statement that a clause was already appended to, or that became part of another one, as a subquery, a compound SELECT or EXPLAIN QUERY PLAN, fails with an IllegalStateException that says so, while the scope is built, so that none of its statements runs, rather than replacing another statement. UPDATE's WHERE merges the parameters into a new list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A transaction in another one, which the documentation of DatabaseScope
says can be nested, ended the outer one when it ended, since 2.0.0 at
least: transaction { } called endTransaction() in its finally block
whether its beginTransaction() had begun a transaction or joined one,
and endTransaction() cleared the transaction being built. The
statements after the inner transaction were then no longer in the
transaction, so they ran on their own, and when one of them failed,
those before it weren't rolled back; a second transaction in the outer
one began a transaction of its own, in its own mode. The F27 test put
the inner transaction last, so it didn't show.
beginTransaction() and endTransaction() now count how deeply the
transactions are nested, and only the outermost endTransaction() ends
the transaction. The fix is in these two functions rather than in the
inline transaction { } and transaction(mode) { }, which call them, so
that it also applies to code compiled against an earlier version, whose
inlined bodies still call them as before, and to nested calls of the
two functions themselves. beginTransaction() still returns true only for
the outermost transaction. endTransaction() gets documentation that
says what it does.
The test rolls back an outer transaction whose statement after an inner
one fails, with two inner ones in a row, with modes, after an inner one
whose block threw, and with nested calls of beginTransaction() and
endTransaction(), and commits the statements of the inner ones with the
outer one; it fails without the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
transactionStatementsGroup and transactionDepth were @volatile, which suggested that a scope could be shared between threads. It can't: the statements are collected in unsynchronized lists, and beginning a transaction checks the depth, creates a group and adds it in several steps, which neither @volatile nor an atomic would make safe. A scope is used by the code that runs its block; a suspending block may resume on another thread, but a coroutine's steps run one after another, its dispatcher making the writes before a suspension visible after it, so the fields need no @volatile, as the other fields of the scope have none. The documentation of DatabaseScope now says that a scope isn't thread-safe, and that its functions mustn't be called from threads or coroutines that run at the same time, as coroutines launched on a dispatcher of several threads in the block of suspendedScope would. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t 1) The first part of F30: PRAGMA_INTEGRITY_CHECK(maxErrors) and PRAGMA_QUICK_CHECK(maxErrors), which return the problems, or an empty list rather than SQLite's "ok"; PRAGMA_FOREIGN_KEY_CHECK(), of all tables or of one; PRAGMA_TABLE_INFO(table), of a table, a view or a name, PRAGMA_INDEX_LIST(table), PRAGMA_INDEX_XINFO(index) and PRAGMA_FOREIGN_KEY_LIST(table); PRAGMA_WAL_CHECKPOINT(mode), which a transaction rejects, as SQLite fails it there with "database table is locked"; PRAGMA_PAGE_COUNT(), PRAGMA_FREELIST_COUNT() and PRAGMA_PAGE_SIZE(); and PRAGMA_DEFER_FOREIGN_KEYS(flag), which only a transaction takes, as SQLite turns it off when one ends. A PragmaStatement runs a PRAGMA as a query when it returns rows, and with execSQL when it returns none, as Android's framework doesn't run a statement that returns rows with execSQL, and sqlite-jdbc doesn't run one that returns none as a query. It reads the rows when it runs and closes its cursor; getResults() returns them once the scope has run, as ColumnInfo, IndexInfo, IndexColumn, ForeignKeyInfo, ForeignKeyViolation, a CheckpointResult or a value, in com.ctrip.sqllin.dsl.sql.pragma. A PRAGMA changes no table, so it can be in a versioned read. Names are written as string literals, and columns are looked up by name once there is a row, as an older SQLite names no columns for a PRAGMA that returns no rows. PRAGMA_RECURSIVE_TRIGGERS and reading PRAGMA_FOREIGN_KEYS(), which the first part was to have, are left out: on Android 13 and higher, a database in WAL mode has several connections, and the framework steps a PRAGMA that sqlite3_stmt_readonly() reports as read-only on another connection than the one that writes, which only prepares it, so the statements cached there keep running as they were compiled, which a test on Robolectric API 37 showed for both; they need the PRAGMA to run on every connection, which PRAGMA_FOREIGN_KEYS(flag) needs too. The test reads auto_vacuum before checking freelist_count after a delete, as Android's SQLite creates a database with auto_vacuum FULL, which leaves no unused pages. The guide to modifying the database describes the PRAGMAs in a new section. The ROADMAP leaves case_sensitive_like out of F30, as SQLite deprecates it, and adds F67, the PRAGMAs that set how each connection runs statements, as parameters of the configurations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nnection on Android (B12) On Android 13 and higher, PRAGMA foreign_keys = ON, and OFF, and PRAGMA recursive_triggers, run with execSQL, as the DSL's PRAGMA_FOREIGN_KEYS does, didn't apply to the statements whose SQL had run before: they kept checking foreign keys or not, and firing triggers recursively or not. There, a database in WAL mode has several connections, the one that writes and those that read outside a transaction. The framework prepares such a PRAGMA on the connection that writes, but steps it on one that reads, as sqlite3_stmt_readonly() reports it doesn't write. SQLite changes the setting when it prepares the PRAGMA, but only expires the statements the connection has compiled when it steps it, so the statements the connection that writes had cached kept running as they were compiled, while those it compiled afterwards had the new setting. On Robolectric API 37, an INSERT whose SQL had run before PRAGMA_FOREIGN_KEYS(true) inserted a row that violated a foreign key. The Android driver's execSQL now runs these two PRAGMAs, outside a transaction, on every connection of the database, those opened later included, with execPerConnectionSQL, as the busy timeout is set, so that each connection steps them. Inside a transaction, which runs on the connection that writes, it runs them there, where SQLite ignores a change of foreign_keys, as on the other platforms, and applies recursive_triggers to the statements after it. Below Android 13, a database that SQLlin opens has a single connection, and nothing changes. No API changes: the driver recognizes the two PRAGMAs in the SQL. The test fails without the fix on API 37, and passes on API 26 and the other platforms either way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second part of F30, with the two the first part left out, which B12 made work on every connection: - PRAGMA_FOREIGN_KEYS(), which reads what PRAGMA_FOREIGN_KEYS(flag) sets; PRAGMA_RECURSIVE_TRIGGERS(flag), PRAGMA_LEGACY_ALTER_TABLE(flag) and PRAGMA_IGNORE_CHECK_CONSTRAINTS(flag), which the Android driver now runs on every connection too, as it does foreign_keys; - PRAGMA_INTEGRITY_CHECK(table) and PRAGMA_QUICK_CHECK(table), which need SQLite 3.33.0, below which they check the whole database; - PRAGMA_TABLE_XINFO(table), with ColumnInfo.kind for the hidden and generated columns, which needs SQLite 3.26.0, and PRAGMA_TABLE_LIST(), which needs 3.37.0; - PRAGMA_COMPILE_OPTIONS(), which Android's SQLite is built without, PRAGMA_COLLATION_LIST(), PRAGMA_DATABASE_LIST(), and PRAGMA_FUNCTION_LIST(), PRAGMA_MODULE_LIST() and PRAGMA_PRAGMA_LIST(), which need SQLite 3.30.0; - PRAGMA_APPLICATION_ID() and PRAGMA_APPLICATION_ID(id), and PRAGMA_AUTO_VACUUM() and PRAGMA_AUTO_VACUUM(mode). SQLite ignores a PRAGMA it lacks rather than failing, so those that only newer versions or some builds have check that the rows SQLite returns have their first column, and throw an IllegalStateException that tells what they need rather than return nothing, without asking SQLite for its version; they are marked with @PlatformDependentSQLiteAPI. SQLite takes auto_vacuum and page_size from what the connection last read of the header of the file, which it reads again when a transaction begins, so they are read in a transaction of their own outside one: on Robolectric API 37, auto_vacuum read outside a transaction, on a connection that reads, still told FULL after INCREMENTAL and VACUUM. Left out: incremental_vacuum, which returns a row without columns for each page it removes, so that sqlite-jdbc runs one step of it and can't query it, and Android's framework below 13 rejects it in execSQL; journal_size_limit, wal_autocheckpoint, max_page_count, secure_delete and cell_size_check, which each connection has, and some of which Android's framework sets on each connection it opens, so they go to F67. Android's SQLite turns legacy_alter_table on by default, which the test and the documentation take into account, and its own test reads the auto_vacuum and the build options of the SQLite it runs on. The guide to modifying the database describes them; the ROADMAP moves F30 to Supported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Kotlin/Native and on Android 8.1 and lower, journalMode didn't take effect, since 1.2.0: a database stayed in the journal mode it was in, DELETE for a new one on Kotlin/Native and that of the framework on Android, rather than WAL, the default. updateJournalMode ran `PRAGMA journal_mode=WAL` as a query whose cursor it never read, and a query there only runs as its cursor is read: NativeCursor steps the statement in next(), and Android's framework fills its window when the cursor is first moved. On Robolectric API 26, the mode stayed `memory`, and became `wal` once the row was read. The JVM sets the mode through SQLiteConfig, and Android 9 and higher through OpenParams. The row is now read. A connection that is actually read-only, as one that isReadOnly opens where the file can't be written, keeps the mode the file is in, as SQLite writes WAL mode to the header of the file: switching a file in DELETE mode to WAL fails there with "attempt to write a readonly database". testJournalMode checks the mode after opening with the default, with DELETE, and with the default again, and fails on API 26 without the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PRAGMA incremental_vacuum removes a page each time it returns a row, which has no columns, so it has to run as a query whose rows are all read. F30 left it out as "no driver steps through it", which was wrong: measured, Android's framework runs it fully as a query, on API 26 and 37, though execSQL removes one page and then throws, and Kotlin/Native would, as it steps each row of a query. Only sqlite-jdbc can't: it doesn't run a statement that returns no columns as a query, failing with "Query does not return results", and runs it as a statement, as DB.exec does too, only as far as its first row, removing one page. The KDoc of PRAGMA_AUTO_VACUUM(mode), the guide to modifying the database, in English and Chinese, the CHANGELOG entry of F30 and the ROADMAP line of F30 now say so. The JVM would have to run `PRAGMA incremental_vacuum(1)` once for each page, in a transaction, and few databases are in INCREMENTAL mode, which Android's SQLite doesn't create them in, so it goes to a new part of the ROADMAP, Shelved, of the items that aren't planned until a need for them comes up, as F68. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n created A connection reads the schema once and keeps it, and when the PRAGMAs that read the schema of a table or an index by its name don't find it, SQLite doesn't always check that schema is still current: index_list, index_info, index_xinfo and foreign_key_list never do, up to its trunk, and table_info and table_xinfo don't before 3.33.0, Android 13 and lower. So a table or index that another connection, such as one in another process, has just created looks as if it didn't exist, until this connection runs a statement that reads or writes a table, which does check it. Measured on SQLite 3.43.2 with two connections, and on Robolectric API 33. A connection's own DDL updates its schema at once, and on Android 13 and higher, where a Database reads through several connections, the framework closes those that read after each DDL that runs through execSQL, so a single Database, which SQLlin's apps almost always are, doesn't meet it. It isn't worked around: that would cost an extra query, and a transaction on Android, for every such PRAGMA. The guide to modifying the database, in English and Chinese, now says so where it says a table or index that doesn't exist has none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
F46 already added the COLLATE operator, with BINARY, NOCASE and RTRIM, for conditions and ORDER BY, so two parts of F19 were left: - @CollateRTrim, experimental, for COLLATE RTRIM in a column's definition, which compares the column's values ignoring the spaces at their ends, as its UNIQUE constraint and indexes do, while storing them as they are written. The processor reports a property that has both it and @CollateNoCase, as a column has one collation, and one that isn't a String or a Char, and rejects it on views and FTS tables, as it does the other constraints. A column needs no @CollateBinary, as BINARY is its default. - COLLATE on the columns of an index. CREATE_INDEX already wrote the operator's expression without the table's name, and SQLite reads `(name COLLATE NOCASE)` in an index as the column with that collation, as index_xinfo tells, rather than as an expression, so it only needed saying and testing, with an order too, as `(name COLLATE NOCASE) to DESC`. testCollations checks the column's SQL, a lookup that ignores the trailing spaces of a stored value, which keeps them, the UNIQUE constraint that rejects the value without them, the collations and orders of two indexes through PRAGMA_INDEX_XINFO, one of them BINARY on a NOCASE column, and that a query comparing by NOCASE uses the NOCASE index, through EXPLAIN_QUERY_PLAN. The guides to getting started, to modifying the database and to SQL functions, in English and Chinese, the KDoc of CREATE_INDEX and of COLLATE, the CHANGELOG and the ROADMAP, where F19 moves to Supported, say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`PersonTable DELETE WHERE(PersonTable.age LT 18) ORDER_BY (PersonTable.age
to ASC) LIMIT 10`, and the same after DELETE X, UPDATE ... WHERE and an
UPDATE without WHERE, with OFFSET after LIMIT, experimental.
SQLite only takes ORDER BY and LIMIT in a DELETE or UPDATE when it is
built with SQLITE_ENABLE_UPDATE_DELETE_LIMIT. sqlite-jdbc and Apple's
SQLite are; Android's isn't, as AOSP's Android.bp shows, nor the
libraries SQLlin links on Linux and Windows, where the statement fails
with `near "ORDER": syntax error`, as it does on Python's SQLite 3.53.4.
When it is, sqlite3LimitWhere() turns them into a subquery of the rowids
of the rows they choose, so SQLlin writes that subquery itself, which
every SQLite runs:
DELETE FROM person WHERE rowid IN (SELECT rowid FROM person
WHERE person.age<? ORDER BY person.age ASC LIMIT 10)
EXPLAIN QUERY PLAN of the two is the same, for DELETE and UPDATE, and
with INDEXED BY, which SQLite keeps for the subquery while the statement
finds the rows by their rowids: SQLlin puts INDEXED BY and NOT INDEXED in
the subquery only, as with both the statement scans the index. Unlike
SQLite's syntax, the subquery is allowed in the body of a trigger. A
table with a column named rowid, which rowid then means, reads its rowid
as _rowid_ or oid, which FTS tables take too.
ORDER BY without LIMIT, which SQLite rejects, fails the scope with an
IllegalArgumentException before any of its statements runs: an
UPDATE or DELETE with ORDER BY is an OrderByUpdateDeleteStatement, whose
checkComplete() throws. checkComplete() moves to SingleStatement, which
SelectStatement overrides, so that the scope, a transaction and
CREATE_TRIGGER check every statement alike.
DELETE X and DELETE WHERE(...), of a Table and of an IndexedTable, now
return a WhereUpdateDeleteStatement, which ORDER_BY and LIMIT follow,
rather than Unit, which only needs code compiled against an earlier
version to be compiled again; UPDATE ... WHERE returns one too rather
than the String of its SQL, which is a Breaking change. Their parts are
kept in UpdateDeleteParts, from which each clause builds the next
statement. ROADMAP F15 now says that WITHOUT ROWID tables will have to
choose the rows by their primary key.
testUpdateDeleteOrderByLimit covers DELETE with WHERE and without,
UPDATE with WHERE and without, with the parameters of SET before those
of WHERE, OFFSET with a negative LIMIT, LIMIT without ORDER BY,
INDEXED BY, a column named rowid, a transaction, a trigger that keeps the
latest rows, and ORDER BY without LIMIT in a scope, in a transaction and
in a trigger, after which nothing ran.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new module, com.ctrip.kotlin:sqllin-dsl-paging, on every platform of
sqllin-dsl, which androidx.paging's paging-common 3.5.3, its latest
release, has too. `database.pagingSource { PersonTable SELECT
ORDER_BY(PersonTable.id to ASC) }`, experimental, returns a
PagingSource<Int, T> that loads the rows a page at a time by their
offsets, as Room's LimitOffsetPagingSource does:
- a page is `SELECT * FROM (select) LIMIT ... OFFSET ...`, so the SELECT
can have a WHERE, a join or compound operators; the first page also
counts the rows, `SELECT count(*) FROM (select)`, so that each page
tells how many rows are before and after it, and a refresh past the
last row loads the last rows;
- the keys are offsets at the borders between pages: a prepend loads at
most as many rows as there are before its key, so that two pages never
share a row wherever a refresh starts, which needs no grid of the page
size, as the ROADMAP said; a page after the first row can always
prepend, as after a RemoteMediator invalidates the source;
- getRefreshKey starts half the initial load before the row last read,
adding the first page's itemsBefore without placeholders, whose
anchorPosition doesn't count the rows before it.
It is built on the table version tokens of F66: it registers with
onChange from its first page, through a callback that only holds the
source weakly, so that a source its Pager drops can be collected, and
closes the registration when it is invalidated otherwise. A page read
after a change since the first is LoadResult.Invalid, and every Invalid
is returned through invalidate(), so that the source is invalid once it
returns one, also for a TestPager, which doesn't invalidate it as a
Pager does; a source invalidated before it loads reads nothing. Pages
load in Dispatchers.IO by default.
The module can only reach sqllin-dsl's public API, so sqllin-dsl marks
what it loads through with the new @InternalSQLlinAPI, an opt-in of
ERROR level for SQLlin's own modules: Database.loadPage, LoadedPage and
TableVersions.isStale, over the internal DatabaseScope.page() and
RowCountStatement, which reads its count under the lock it runs under.
The module's build script, which a template made, now follows
sqllin-dsl's: its package, targets without iosX64, the SDK levels of the
version catalog and Maven Central publishing; the template's lint
plugin, kotlin-stdlib, kotlin-test, test runner and junit entries, its
.gitignore, manifest and example tests are gone. CI builds it on each
host, and publish_all.sh publishes it.
CommonPagingTest, in sqllin-dsl-test, loads it through TestPager and a
Pager: the keys and counts of refreshes, appends and prepends, at the
start, the middle, the end and past it; the refresh key with
placeholders and without; invalidation by a change, not by one of
another table nor before the first page, before loading, between the
two reads of a refresh, and before a later page; WHERE, an outer join
and a compound SELECT; and a Pager that reloads around the position read
after a delete. Each of four fixes that are undone fails it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A property of a type SQLite doesn't store can be a column when its serializer encodes it as one value: the serializer kotlinx.serialization writes the property with is the converter, so that no annotation of SQLlin's is needed. That covers the built-in serializers of Uuid, Instant and Duration, value classes, types whose classes declare their serializers, as kotlinx-datetime's do, and any type, a List stored as JSON included, given a serializer with @serializable(with = ...) on the property, on the use of its type or in a type alias, or by @file:UseSerializers. - sqllin-processor finds that serializer as the plugin does, and generates a ClauseConverted element that holds it, with a SET {} setter that stores what it encodes; a type no column can hold, a collection or a class serialized as a structure, or a type serialized contextually, without a SerializersModule to find its serializer in, is an error whose message says how to give it one. - The column's type is the one the serializer stores, which the table's createSQL, then a getter, computes with Table#columnType, and which ALTER_ADD_COLUMN now computes too, through the kind of the descriptor; a serializer that encodes a structure fails CREATE, as does a collation of a column that doesn't store text. The tables of every other entity are generated as before, byte for byte. - ClauseConverted, experimental, converts the values of EQ, NEQ, LT, LTE, GT, GTE, IN, of values and of a subquery, BETWEEN, IS and ISNOT with the serializer, and checks that it is the property's, so that a serializer the processor took wrongly fails rather than compares other values than those stored. Comparisons and ORDER BY compare the stored values, which the guide warns the ISO-8601 strings of Instant aren't in the order of. B14: an enum whose class declares a serializer, as one storing its name, or any property given a serializer of its own, was written by it in INSERT but compared and set as its type is by default, as an enum's ordinal, in a column of INT, so that conditions didn't match and a row SET {} changed couldn't be read. Such a property is now converted by its serializer, which changes the type of its element: a breaking change. An enum with its default serializer, or serialized contextually, is still stored as its ordinal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An entity annotated with an opt-in marker, as @ExperimentalUuidApi on its class, requires every use of the class to opt in, which the object generated for it, a Table, a View or an FtsTable of it, didn't, so it didn't compile, with a converted column or without. The object now opts in to the markers that the class, its properties and its file are annotated with, the annotations that are themselves annotated with @RequiresOptIn, besides those that their @optins give, which a table with converted columns needs for their types. The objects of the other entities are generated as before. Found by the review of F12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It could let the table version tokens of F66, and the PagingSource of F13, see the commits of other connections and processes, which they don't, but it doesn't tell which tables changed, and each connection counts on its own, which Android's pool of connections needs a design for. It was left out of F30 without being written down. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A SELECT kept its cursor, and its statement, open until its results
were read, which they may never be, as the SELECTs of versioned { }
whose token alone is wanted. Meanwhile the statement kept reading its
tables, which couldn't be dropped, failing with "database table is
locked", and on Kotlin/Native it kept the connection open after
close(), as sqlite3_close_v2 waits for the statements to end, holding
the database file. Windows can't delete an open file, so on CI every
test after testTableVersions opened its database, with its rows and
without the tables of its own configuration, and 78 of 116 failed;
Linux and macOS delete it anyway.
The SELECT now reads and decodes its rows when it runs, under the lock
of its scope and in its transaction, and closes the cursor, so that its
statement ends with the scope; getResults() returns the rows, and an
error decoding them is thrown by the scope. testUnreadSelect drops the
table of an unread SELECT, which failed with SQLITE_LOCKED on the JVM,
and deletes the closed database, which failed on Windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The macOS, Windows and Linux jobs uploaded sqllin-dsl/build/reports on failure, but the DSL's tests run in sqllin-dsl-test, so that the report of a failed run, as that of the Windows tests of #135, wasn't uploaded. The Android job already uploaded the right one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New features and fixes for 2.6.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
All
AGP's version to9.5.1Gradle's version to9.9.0kotlinx.serialization's version to1.12.0sqllin-dsl
lookasideSlotSizeandlookasideSlotCountofDSLDBConfigurationdisabled SQLite's lookaside memory allocator on Kotlin/Native and on Android 9 and higher, as those ofDatabaseConfigurationdid; they are now -1, see sqllin-driverdatabase { }scope on another thread could take the TEMP table and triggers that count the changes of observed queries with it, as its transaction can be open while they are created, the connection being locked for each statement rather than for a transaction: its rollback removed them, so the next write failed with "no such table", and once another table was observed, the changes of the table they belonged to were no longer counted. The tracker now creates them again where they are gone, and counts a change of their tables. A failure to tell the observed queries of a change no longer takes the place of the failure of the statement that made it. The guide to observed queries and the documentation ofDatabase#invokeandDatabase#observenow say to write throughsuspendedScopewhile queries are observed, as they run on another thread, and that a blocking scope isn't serialized with themdatabase suspendScope { }, which doesn't exist, in a function that wasn't asuspendone; it isdatabase suspendedScope { }, in asuspendfunctionINSERTfailed withnear ")": syntax errorwhen the primary key that the database assigns, aLong?property annotated with@PrimaryKey, was the last property of the class: asINSERTleaves such a key out, the column list kept a trailing comma, and theVALUEStuple was never closed. A key declared before another property wasn't affected, and neither wereINSERT_WITH_ID,INSERT_OR_IGNOREandINSERT_OR_REPLACE, which write the keySelectStatement.getResult(), which doesn't exist, rather thangetResults(); the guide to getting started importedDSLDBConfigurationfromcom.ctrip.sqllin.driverrather thancom.ctrip.sqllin.dsl, and, in English, the annotationNoCase, which doesn't exist, rather thanCollateNoCase; and the guide to modifying the database declared anautoIncrementprimary key asLong = 0, which the processor rejects, rather thanLong? = nullval update = table UPDATE SET { ... }, followed by a SELECT and then completed withupdate WHERE (...), updated every row of the table, and the SELECT returned nothing.WHEREalso added its parameters to those of the UPDATE without it. A clause now replaces the statement it completes, wherever it is, and appending one to a statement that a clause was already appended to, or that became part of another one, as a subquery, a compound SELECT orEXPLAIN_QUERY_PLANdoes, fails with anIllegalStateExceptionbefore any statement of the scope runs, rather than replacing another statementDatabaseScopesays can be nested, ended the outer one when it ended, since 2.0.0 at least: the statements after it in the outer block weren't in the transaction, so they ran on their own, and when one of them failed, those before it weren't rolled back; a second transaction in the outer one began a transaction of its own, in its own mode.beginTransaction()andendTransaction()now count how deeply transactions are nested, and only the outermostendTransaction()ends the transaction, so the statements of an inner transaction and those after it are the outer one's, which commits or rolls back all of them. As the fix is in these two functions, rather than in the inlinetransaction { }, it also applies to code compiled against an earlier version, and to nested calls of the two functionsPRAGMA_FOREIGN_KEYS(flag)didn't apply to the INSERT, UPDATE and DELETE statements whose SQL had run before, which kept checking foreign keys or not as before, such as after a migration that turned them off and on again; see sqllin-driverjournalModeofDSLDBConfiguration, WAL by default, didn't take effect, since 1.2.0; see sqllin-driverCROSS_JOIN,JOIN,INNER_JOIN,NATURAL_JOIN,NATURAL_INNER_JOIN,LEFT_OUTER_JOINandNATURAL_LEFT_OUTER_JOINthat took relations as arguments, theSELECTandSELECT_DISTINCTthat took what they returned, and the typesBaseJoinClause,JoinClause,NaturalJoinClause,JoinStatementWithoutConditionandJoinSelectStatement, with theON,USING,WHERE,GROUP_BY,ORDER_BYandLIMITthat took them. To migrate, write the join as a join of relations:table SELECT INNER_JOIN<R>(other) USING columnbecomes(table INNER_JOIN other USING column) SELECT X<R>(). Its result type is checked against the joined relations when the statement is built, so a type that reads a column two relations have has to select it withAS, and one that reads a column an outer join can leave NULL has to make its property nullable. The guide to advanced queries describes the migration(PersonTable INNER_JOIN BookTable ON (PersonTable.id EQ BookTable.authorId)) SELECT X<R>(), withoutFROM, with each join operator:INNER_JOIN,LEFT_OUTER_JOIN,RIGHT_OUTER_JOIN,FULL_OUTER_JOIN,CROSS_JOINand the NATURAL joins.FROM(relation)still starts one tooUNION {}andUNION_ALL {}blocks, and theDatabaseScope#beginUnion,createUnionSelectStatementandendUnionfunctions they used, which 2.5.0 deprecated for bugs that wouldn't be fixed: a union of projections, result columns or joins failed with aClassCastException, and a nested block was flattened into the SQL, which changed its meaning. To migrate, use the infix operators:UNION { a; b }becomes(a) UNION (b), andUNION_ALL { a; b }becomes(a) UNION_ALL (b). The guide to advanced queries describes itPagingSourcethat invalidates itself.Database#versioned { }runs a scope of SELECTs, and returns its result with aTableVersionstoken of the versions of the tables they read, captured under the lock that they run under, after the tables are tracked, so that no change after them is missed; a scope that writes is rejected.Database#awaitChange(token)suspends until a statement run through the database changes one of those tables, andDatabase#onChange(token) { }registers a callback that runs once, after the writing scope releases the lock, on the thread that wrote, and returns anAutoCloseablethat closes the registration, or null when the token is already stale, as it registers before it checks. Tokens combine with+, which takes the earlier version of a table that both read, and rejects the tokens of anotherDatabaseINSERT_OR_ABORT,INSERT_OR_FAILandINSERT_OR_ROLLBACK, for SQL syntaxINSERT OR ABORT,INSERT OR FAILandINSERT OR ROLLBACK, which take objects, expressions,DEFAULT_VALUESand a SELECT, asINSERTdoes, and write the primary key, asINSERT_OR_IGNOREandINSERT_OR_REPLACEdo; andUPDATE_OR_ABORT,UPDATE_OR_FAIL,UPDATE_OR_IGNORE,UPDATE_OR_REPLACEandUPDATE_OR_ROLLBACK, forUPDATE OR .... A statement's resolution takes the place of the conflict clause of a constraint, whichonConflictdeclares. The guide to modifying the database describes them in a new sectiontransaction(mode) { }, which begins the transaction in aTransactionModeof sqllin-driver, for SQL syntaxBEGIN DEFERRED,BEGIN IMMEDIATEandBEGIN EXCLUSIVE, rather than IMMEDIATE, the mode oftransaction { }. A transaction in another one keeps the outer one's modeINDEXED_BYandNOT_INDEXED, for SQL syntaxINDEXED BYandNOT INDEXED, which make a statement look its rows up through an index, or none, rather than through the one SQLite's query planner chooses:(PersonTable INDEXED_BY "idx_person_name") SELECT WHERE(...)andPersonTable.NOT_INDEXED DELETE WHERE(...). TheIndexedTablethey return is read by a SELECT, joins included, and written by UPDATE, theUPDATE_OR_ones and DELETE, but not by INSERT, which SQL doesn't allow, which is checked at compile timeEXPLAIN_QUERY_PLAN(select), for SQL syntaxEXPLAIN QUERY PLAN, which tells how SQLite would run a SELECT without running it: which tables it scans, which indexes it searches, and where it sorts with a temporary B-tree. Any SELECT can be planned, joins, subqueries and compound ones included; it becomes part of the statement, and is planned with its parameters. TheQueryPlanStatementit returns gives, once the scope has run, aQueryPlan: itssteps, each with itsdetailand itschildren, thedetailsof all of them, and atoString()that writes them as a tree, as thesqlite3shell does. The details are SQLite's own text, which differs between SQLite versions, and a SQLite older than 3.24.0, as Android's before API 30, lists the steps without children; the plan is read by the columns SQLite returns, so the same on every platform, without the version of SQLite. The guide to advanced queries describes it in a new sectionPRAGMA_INTEGRITY_CHECK(maxErrors)andPRAGMA_QUICK_CHECK(maxErrors), which return the problems found, or an empty list rather than SQLite'sok;PRAGMA_FOREIGN_KEY_CHECK(), of all tables or of one;PRAGMA_FOREIGN_KEYS(), which reads whatPRAGMA_FOREIGN_KEYS(flag)sets;PRAGMA_TABLE_INFO(table), of a table, a view or a name,PRAGMA_INDEX_LIST(table),PRAGMA_INDEX_XINFO(index)andPRAGMA_FOREIGN_KEY_LIST(table);PRAGMA_WAL_CHECKPOINT(mode), which throws anIllegalStateExceptioninside a transaction, where SQLite can't checkpoint;PRAGMA_PAGE_COUNT(),PRAGMA_FREELIST_COUNT()andPRAGMA_PAGE_SIZE();PRAGMA_DEFER_FOREIGN_KEYS(flag), which throws anIllegalStateExceptionoutside a transaction, as SQLite turns it off when one ends;PRAGMA_RECURSIVE_TRIGGERS(flag),PRAGMA_LEGACY_ALTER_TABLE(flag), which Android's SQLite turns on by default, andPRAGMA_IGNORE_CHECK_CONSTRAINTS(flag);PRAGMA_APPLICATION_ID()andPRAGMA_APPLICATION_ID(id);PRAGMA_AUTO_VACUUM()andPRAGMA_AUTO_VACUUM(mode), which VACUUM applies to a database that has tables, whilePRAGMA incremental_vacuumis left out, as it removes a page each time it returns a row, which has no columns, and sqlite-jdbc runs such a statement only as far as its first row;PRAGMA_COLLATION_LIST()andPRAGMA_DATABASE_LIST(); and, marked with@PlatformDependentSQLiteAPI,PRAGMA_TABLE_XINFO(table), which needs SQLite 3.26.0, Android API 30,PRAGMA_FUNCTION_LIST(),PRAGMA_MODULE_LIST()andPRAGMA_PRAGMA_LIST(), which need SQLite 3.30.0, Android API 31,PRAGMA_TABLE_LIST(), which needs SQLite 3.37.0, Android API 34,PRAGMA_INTEGRITY_CHECK(table)andPRAGMA_QUICK_CHECK(table), which need SQLite 3.33.0, Android API 34, below which they check the whole database, andPRAGMA_COMPILE_OPTIONS(), which Android's SQLite is built without. A PRAGMA that SQLite lacks is ignored rather than failing, so those that only newer versions have check the columns of the rows SQLite returns, and throw anIllegalStateExceptionthat tells what they need, rather than return nothing, without asking SQLite for its version. auto_vacuum and page_size are read in a transaction of their own outside one, as SQLite takes them from what the connection last read of the header of the file, which on Android 13 and higher may be a connection that reads, which would tell the mode before the last VACUUM. A PRAGMA that reads returns aPragmaStatement, whosegetResults()returns, once the scope has run, a list ofColumnInfo,IndexInfo,IndexColumn,ForeignKeyInfo,ForeignKeyViolation,RelationInfo,DatabaseInfoorFunctionInfo, aCheckpointResult, anAutoVacuum, or a value. Each PRAGMA runs as a query or not, as it returns rows or not, which neither Android's framework nor sqlite-jdbc would otherwise run, and a PRAGMA changes no table, so it can be in a versioned read. The guide to modifying the database describes them in a new section@CollateRTrim, for SQL syntaxCOLLATE RTRIMin a column's definition, which compares the column's values ignoring the spaces at their ends, as its UNIQUE constraint and indexes do, while storing them as they are written; a property can't have it and@CollateNoCaseboth, which the processor reports, as it does on a property that isn't aStringor aChar. A column without either compares its values as theBINARYcollation does, which a column has by default. And theCOLLATEoperator, which 2.5.0 added for conditions and ORDER BY, gives a column of an index its collation, as inPersonTable.CREATE_INDEX("idx_person_name_nocase", PersonTable.name COLLATE NOCASE), for SQL syntaxCREATE INDEX ... ON person(name COLLATE NOCASE), also with an order, as(PersonTable.name COLLATE NOCASE) to DESC, which the queries that compare by that collation use. The guides to getting started and to modifying the database describe themORDER_BY,LIMITandOFFSETafter aDELETEor anUPDATE, with itsWHEREor without, which only write the first rows in an order, or as many asLIMITsays, as inPersonTable DELETE WHERE(PersonTable.age LT 18) ORDER_BY (PersonTable.age to ASC) LIMIT 10, for SQL syntaxDELETE ... ORDER BY ... LIMIT ...andUPDATE ... ORDER BY ... LIMIT .... SQLite only takes them when it is built withSQLITE_ENABLE_UPDATE_DELETE_LIMIT, which Android's SQLite isn't, and then turns them into a subquery of the rowids of the rows they choose, so SQLlin writes that subquery itself, which every platform runs, with the same query plan:DELETE FROM person WHERE rowid IN (SELECT rowid FROM person WHERE person.age<? ORDER BY person.age ASC LIMIT 10). It can be in the body of a trigger too, where SQLite's own syntax isn't allowed;INDEXED_BYandNOT_INDEXEDdecide how the subquery looks the rows up; a table with a column namedrowidreads its rowid as_rowid_oroid; and a negativeLIMITis none, so thatOFFSETkeeps the first rows, as inORDER_BY (time to DESC) LIMIT -1 OFFSET 100.ORDER_BYhas to be followed byLIMIT, as SQLite says: a scope where it isn't throws anIllegalArgumentExceptionbefore any of its statements runs, as doesCREATE_TRIGGERfor a trigger whose body has one. The guide to modifying the database describes them in a new sectionUPDATE ... WHEREreturns aWhereUpdateDeleteStatement, whichORDER_BYandLIMITcan follow, rather than theStringof its SQL, which itssqlStris; andDELETE XandDELETE WHERE(...), of a table and of anIndexedTable, return one too, rather thanUnit, which changes no source but code compiled against an earlier version, which has to be compiled again@InternalSQLlinAPI, which marks the APIs of sqllin-dsl that only SQLlin's other modules use, as sqllin-dsl-paging loads its pages through, which apps shouldn't use, as they may change in any version@Serializable(with = ...), was written by that serializer inINSERT, but compared in conditions, and set inSET {}, as its type is by default, as the ordinal of an enum entry, in a column of the type of its Kotlin type,INTfor an enum: a condition didn't match the values stored, and a row thatSET {}changed couldn't be read. Such a property is now a column converted by its serializer, see below: code that names the type of its element, asClauseEnum<Status>, or uses an operator ofClauseStringorClauseNumberon it, asLIKEor+, has to change, and its operators, which are experimental, need an opt-in. A table created before keeps the type of its column, which holds values of any type in SQLite, so it needs no migration. An enum with its default serializer, or serialized contextually, is still stored as its ordinaldatabase table is locked, and on Kotlin/Native the connection stayed open afterclose(), assqlite3_close_v2waits for the statements to end, holding the database file, which Windows then couldn't delete. A SELECT now reads its rows when its scope runs it, under the scope's lock and in its transaction, so that an error decoding them is thrown by the scope, andgetResults()returns themkotlin.uuid.Uuid,kotlin.time.Instantandkotlin.time.Duration, with their built-in serializers, value classes, types whose classes declare their serializers, as those of kotlinx-datetime do, and any type given a serializer with@Serializable(with = ...)on the property, on the use of its type or in a type alias, or by@file:UseSerializers, as aListstored as JSON. The serializer is the converter: the column has the type of the value it stores, whichCREATEandALTER_ADD_COLUMNcompute from it, and the column's element, aClauseConverted<T>, converts the values of its conditions with it,EQ,NEQ,LT,LTE,GT,GTE,IN, of values and of a subquery,BETWEEN,ISandISNOT, asSET {}converts the values it assigns.literal(value, serializer)is a literal of such a type. A comparison other than an equality, andORDER_BY, compare the stored values, which are in the order of the Kotlin values only if the serializer keeps it, as the ISO-8601 strings of the built-in serializer ofInstantaren't. A serializer that encodes a structure, as the generated one of a data class, failsCREATE. See the guide to getting startedTable#columnType(elementIndex, collation)andTable#appendConverted(...)ofSetClause, which the tables that sqllin-processor generates call for the columns converted by their serializerssqllin-dsl-paging
com.ctrip.kotlin:sqllin-dsl-paging, on every platform of sqllin-dsl, which depends on androidx.paging'spaging-common3.5.3: new experimental APIDatabase#pagingSource(context) { }, which returns androidx.paging'sPagingSourceof the rows of the SELECT it builds, loaded a page at a time by their offsets, as Room'sLimitOffsetPagingSourcedoes. A page isSELECT * FROM (select) LIMIT ... OFFSET ..., so the SELECT can have a WHERE, a join or compound operators; the first page also counts the rows, so that each page tells how many rows are before and after it, which placeholders show, and a refresh past the last row loads the last ones. The keys are offsets at the borders between pages, so that two pages never share a row, and a page after the first row can always prepend the rows above it, as after aRemoteMediatorinvalidates the source; a refresh starts half the initial load before the row last read, also without placeholders, whose positions don't count the rows before the first page. It is built on the table version tokens of sqllin-dsl: it registers withDatabase#onChangefrom its first page, weakly, so that a source itsPagerdrops can still be collected, and closes the registration when it is invalidated otherwise; a change of a table the SELECT reads invalidates it, and a page that reads rows changed since the first one isLoadResult.Invalid(), with which the source is invalid, also for aTestPageror another caller ofload; a source invalidated before it loads reads nothing. Pages load inDispatchers.IOby default. The guide to advanced queries describes it, and the guide to getting started its dependencysqllin-driver
sqlite-jdbc's version to3.54.1.0lookasideSlotSizeandlookasideSlotCountofDatabaseConfiguration, 0, which its documentation said left SQLite's default configuration, disabled SQLite's lookaside memory allocator on Kotlin/Native and on Android 9 and higher, assqlite3_db_configandsetLookasideConfigtake 0 to disable it, while the JVM and lower versions of Android kept SQLite's default configuration. Their default is now -1, which leaves SQLite's default configuration on every platform, and 0 still disables the allocator. The guides to getting started now say so, and that sqlite-jdbc doesn't supportsqlite3_db_config(), rather thansqlite3_config()busyTimeoutdidn't take effect, so SQLite waited 2500 ms, as the framework sets it, for a lock that another connection held. On Android 9 and higher, it was given toSQLiteDatabase.OpenParams.Builder#setIdleConnectionTimeout, which is deprecated and means another thing, how long a connection may be idle before the framework closes it, and whichSQLiteOpenHelperdrops anyway.busyTimeoutis now set withPRAGMA busy_timeout: from Android 13 on, on every connection of the database, as one in WAL mode reads through several, and below, on its only connection. The guides to getting started listedbusyTimeout, and in ChinesejournalMode, among the parameters that only work on Android 9 and higher; onlyinMemory,lookasideSlotSizeandlookasideSlotCountdobusyTimeoutthere, where waiting could deadlock. Transactions now begin IMMEDIATE on every platform, taking the write lock at once, for which SQLite waits as long asbusyTimeout. On Android, it is what EXCLUSIVE did in WAL mode, and in DELETE mode, other connections can now read until the transaction commits. On the JVM and Kotlin/Native, a read-only connection begins a deferred transaction, as it can't take the write lock. The JVM driver begins and ends transactions with SQL rather than by leaving and entering auto-commit mode, as sqlite-jdbc then begins another transaction of the configured kind at once, which an IMMEDIATE one couldn't while another connection wroteON CONFLICT ROLLBACK, whichonConflict = OnConflict.ROLLBACKdeclares, for a trigger'sRAISE(ROLLBACK)and for some I/O errors, ending the transaction failed too, as no transaction was active, and that failure took the place of the one that made SQLite roll it back, inwithTransactionand so in the DSL'stransaction { }. The failure that made SQLite roll it back is now thrown, with the failure to end the transaction added to it as a suppressed exceptionexecSQL("PRAGMA foreign_keys = ON"), andOFF, andPRAGMA recursive_triggers,legacy_alter_tableandignore_check_constraints, didn't apply to the statements whose SQL had run before, which kept checking foreign keys or not, and firing triggers recursively or not, as before. A database in WAL mode has several connections there, and the framework only prepares such a PRAGMA on the connection that writes, stepping it on one that reads, assqlite3_stmt_readonly()reports it doesn't write; SQLite changes the setting when it prepares the PRAGMA, but only expires the statements the connection has compiled when it steps it, so those the connection that writes had cached kept the setting they were compiled with. These PRAGMAs now run on every connection of the database, those opened later included, withexecPerConnectionSQL, outside a transaction, and on the connection of the transaction inside one, where SQLite ignores a change of foreign_keys, as on the other platforms. Before Android 13, a database that SQLlin opens has a single connection, which wasn't affectedjournalModedidn't take effect, since 1.2.0: a database stayed in the journal mode it was in, DELETE for a new one on Kotlin/Native and that of the framework on Android, rather than that of the configuration, WAL by default. SQLlin set it there with the PRAGMA, as a query, whose cursor it never read, and a query there only runs as its cursor is read. The row it returns is now read. A connection that is read-only, as one thatisReadOnlyopens where the file can't be written, keeps the mode the file is in, as SQLite writes WAL mode to the header of the fileTransactionMode,DEFERRED,IMMEDIATEorEXCLUSIVE, whichDatabaseConnection#beginTransaction(mode)andwithTransaction(mode) { }begin a transaction in, rather than IMMEDIATE, the mode ofbeginTransaction().EXCLUSIVEalso keeps other connections from reading in DELETE journal mode, andDEFERREDtakes no lock until the transaction reads or writes; Android's framework can't begin a deferred transaction that writes, so on AndroidDEFERREDbegins an IMMEDIATE one. The newbeginTransactionhas a default implementation, which only begins an IMMEDIATE transaction, so aDatabaseConnectionimplemented outside SQLlin still compilessqllin-processor
UnsupportedClassVersionError. The processor is now built with the JVM toolchain of the other modules, JDK 21, which its Kotlin classes target as well@file:UseSerializersor by its class, and generates aClauseConvertedelement with it. The objects of the tables that have such columns opt in toExperimentalDSLDatabaseAPI, and to the markers that the@OptIns of the entity's class, properties and file give, as the type of a column can need, asUuiddoes. A property of a type that no column can hold, as a collection, a class serialized as a structure or a type serialized contextually, is still an error, whose message says how to give it a serializer that stores one value@DBRowor@DBViewclass annotated with an opt-in marker, as@ExperimentalUuidApi, which requires the uses of the class to opt in, didn't opt in to it, so it didn't compile; it now opts in to the markers that the class, its properties and its file are annotated with🤖 Generated with Claude Code