Radish alpha
H
rad:z3QDZAW2FAfuLvihrhiyDC9fAD8G9
HardenedBSD Package Manager
Radicle
Git
Merge branch 'upstream-master'
Shawn Webb committed 3 years ago
commit fa6223faacf11e0e3f6399fc3c2bdd1e3451938c
parent e294fea
167 files changed +25325 -16895
modified .cirrus.yml
@@ -8,9 +8,6 @@ freebsd_task:
                - name: freebsd12.3-amd64
                  freebsd_instance:
                     image: freebsd-12-3-release-amd64
-
                - name: freebsd13-amd64
-
                  freebsd_instance:
-
                     image: freebsd-13-0-release-amd64
                - name: freebsd13.1-amd64
                  freebsd_instance:
                     image: freebsd-13-1-release-amd64
@@ -78,7 +75,7 @@ alpine_task:
        container:
                image: alpine:latest
        install_script:
-
                - apk add musl-dev gcc g++ make kyua pkgconf gcompat tcl fts-dev libbsd-dev libarchive-dev openssl-dev bzip2-dev xz-dev zlib-dev m4
+
                - apk add musl-dev gcc g++ make kyua pkgconf gcompat tcl musl-fts-dev libbsd-dev libarchive-dev openssl-dev bzip2-dev xz-dev zlib-dev m4
        script:
                - ./configure
                - make -j4 || make V=1
modified .gitignore
@@ -31,6 +31,7 @@ scripts/completion/_pkg.bash
scripts/completion/_pkg.zsh
scripts/completion/_pkg
scripts/periodic/400.status-pkg
+
scripts/periodic/405.pkg-base-audit
scripts/periodic/410.pkg-audit
scripts/periodic/411.pkg-backup
scripts/periodic/460.pkg-checksum
@@ -82,8 +83,13 @@ scripts/sbin/pkg2ng
!/tests/frontend/*.sh
!/tests/frontend/*.in
/tests/frontend/test_environment.sh
+
/tests/lua
+
/tests/metalog
/tests/merge
+
/tests/packing
/tests/pkg_add_dir_to_del
/tests/pkg_printf
/tests/pkg_validation
/tests/plist
+
/tests/ssh
+
/tests/utils
modified FAQ.md
@@ -31,7 +31,6 @@ Table of Contents
* [Can it track user-edited configuration files that are associated with packages?](#24)
* [Can it do 2- or 3-way merges of package configuration files?](#25)
* [The README states "Directory leftovers are automatically removed if they are not in the MTREE."  How does this work for directories that are shared between multiple packages?  Does this mean that if I add a file to a directory that was created by a package, that file will be deleted automatically if I delete the package?](#26)
-
* [How to create a new plugin for pkgng?](#27)

<a name="0"></a>
### Q: How can I start using pkgng?
@@ -231,10 +230,3 @@ This is the standard way things work in the ports at the moment.
### Q: The README states "Directory leftovers are automatically removed if they are not in the MTREE."  How does this work for directories that are shared between multiple packages?  Does this mean that if I add a file to a directory that was created by a package, that file will be deleted automatically if I delete the package?

No.  Directories have to be empty before they will be removed.
-

-
<a name="27"></a>
-
### Q: How to create a new plugin for pkgng?
-

-
If you are interested in creating a new plugin for pkgng you might want to check the following handbook which is an introduction to plugins writing for pkgng.
-

-
* [Introduction to writing plugins for pkgng in FreeBSD](http://unix-heaven.org/writing-plugins-for-pkgng)
modified NEWS
@@ -1,3 +1,50 @@
+
Changes from 1.18.99.5 to1.18.99.6
+
- reduce the number of filedescriptors opened at the same time
+
- update sqlite to 3.40.0
+
- fix regression in pkg lock
+
- fix annotate --modify
+

+
Changes from 1.18.99.5 to1.18.99.6
+
- remove backup command, do backup pkg shell .dump should be preferred
+
  as already done via periodic script
+
- (r)query: stricter command line validation
+
- (r)query: new operators: =~ ==~ and !=~
+
- fix merging of config files accross split upgrades
+
- fix a regression when executing a shell script
+

+
Changes from 1.18.99.4 to 1.18.99.5
+
- delete: now handles properly provides/requipres
+

+
Changes from 1.18.99.3 to 1.18.99.4
+
- Accept '~' in the version or name of a package
+
- Autoremove will not remove anything which is required
+
  by another installed package (via provides or via shlibs_provides)
+
- Update linenoise to latest version
+
- Update sqlite to 3.39.3
+
- Update lua to 5.4.4
+
- doc: add example on how to restore a package
+
- query: fix unicity issue
+

+
Changes from 1.18.99.2 to 1.18.99.3
+
- fix regression creating hardlinks
+

+
Changes from 1.18.99.1 to 1.18.99.2
+
- fix directory path matching in the new extraction code
+

+
Changes from 1.17.99.11 to 1.18.99.1
+
- rework the extraction process to fix the case when a file becomes a directory
+
  during an upgrade
+
- fix a crash in pkg create
+
- fix a crash in pkg version -PvL=
+
- allow to deal with 240 character long filenames
+
- fix an issue when disabling a repository resulting in breakage of the
+
  repository list
+
- fix querying licenses
+
- fix packaging the lua deinstall scripts
+
- add pkg.symlink to allow creating symlinks from lua scripts
+
- performance improvements
+
- sync libfetch with base bringing in socks proxy support
+

Changes from 1.17.99.10 to 1.17.99.11
- Fix performance issue while computing upgrades
- Fix shlibs always considered as changed
modified README.md
@@ -35,7 +35,6 @@ Table of Contents:
* [Updating remote repositories](#pkgupdate)
* [Searching in remote package repositories](#pkgsearch)
* [Installing from remote repositories](#pkginstall)
-
* [Backing up your package database](#pkgbackup)
* [Creating a package repository](#pkgcreate)
* [Additional resources](#resources)

@@ -477,21 +476,6 @@ Or you could also install the packages using only one command, like this:
For more information on the remote package installs, please refer to
*pkg-install(1)*

-
<a name="pkgbackup"></a>
-
### Backing up your package database
-

-
It is a good idea that you backup your local package database on regular basis.
-

-
In order to backup the local package database, you should use the `pkg backup` command.
-

-
	# pkg backup -d /path/to/pkg-backup.dump
-

-
The above command will create a dump of your local package database in
-
/path/to/pkg-backup.dump
-

-
For more information on backing up your local package database, please
-
refer to *pkg-backup(1)*
-

<a name="pkgcreate"></a>
### Creating a package repository

modified auto.def
@@ -4,9 +4,9 @@
use cc cc-lib cc-shared pkg-config

set maj_ver 1
-
set med_ver 17
+
set med_ver 18
set min_ver 99
-
set dev_ver 11
+
set dev_ver 7
define PKG_API [expr $maj_ver * 1000000 + $med_ver * 1000 + $min_ver]
define VERSION $maj_ver.$med_ver.$min_ver[expr {$dev_ver ? ".$dev_ver" : ""}]

@@ -344,7 +344,7 @@ if {[get-define CCACHE] ne {none}} {
}

make-config-header pkg_config.h -bare { DEFAULT_* }
-
make-config-header external/sqlite/config.h
+
make-config-header external/sqlite/sqlite_cfg.h
make-template mk/defs.mk.in
make-template libpkg/pkg.h.in
make-template libpkg/pkg.pc.in
modified docs/Makefile.autosetup
@@ -14,7 +14,6 @@ MAN8= pkg-add.8 \
	pkg-annotate.8 \
	pkg-audit.8 \
	pkg-autoremove.8 \
-
	pkg-backup.8 \
	pkg-check.8 \
	pkg-clean.8 \
	pkg-config.8 \
modified docs/pkg-add.8
@@ -129,7 +129,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-alias.8
@@ -75,7 +75,6 @@ for alias
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-annotate.8
@@ -195,7 +195,6 @@ Show all packages with the foo annotation:
.Xr pkg-alias 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-audit.8
@@ -129,7 +129,6 @@ See
.Xr pkg-alias 8 ,
.Xr pkg-annotate 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-autoremove.8
@@ -75,7 +75,6 @@ See
.Xr pkg-alias 8 ,
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
deleted docs/pkg-backup.8
@@ -1,108 +0,0 @@
-
.\"
-
.\" FreeBSD pkg - a next generation package for the installation and maintenance
-
.\" of non-core utilities.
-
.\"
-
.\" Redistribution and use in source and binary forms, with or without
-
.\" modification, are permitted provided that the following conditions
-
.\" are met:
-
.\" 1. Redistributions of source code must retain the above copyright
-
.\"    notice, this list of conditions and the following disclaimer.
-
.\" 2. Redistributions in binary form must reproduce the above copyright
-
.\"    notice, this list of conditions and the following disclaimer in the
-
.\"    documentation and/or other materials provided with the distribution.
-
.\"
-
.\"
-
.\"     @(#)pkg.8
-
.\"
-
.Dd May 12, 2015
-
.Dt PKG-BACKUP 8
-
.Os
-
.Sh NAME
-
.Nm "pkg backup"
-
.Nd backup and restore the local package database
-
.Sh SYNOPSIS
-
.Nm
-
.Op Fl q
-
.Fl d Ar dest_file
-
.Nm
-
.Op Fl q
-
.Fl r Ar src_file
-
.Pp
-
.Nm
-
.Op Cm --quiet
-
.Cm --dump Ar dest_file
-
.Nm
-
.Op Cm --quiet
-
.Cm --restore Ar src_file
-
.Sh DESCRIPTION
-
.Nm
-
is used to backup and restore the local package database.
-
.Sh OPTIONS
-
The following options are supported by
-
.Nm :
-
.Bl -tag -width restore
-
.It Fl d Ar dest_file , Cm --dump Ar dest_file
-
Dumps the local package database to a file specified on the
-
command-line.
-
.It Fl q , Cm --quiet
-
Force quiet output.
-
.It Fl r Ar src_file , Cm --restore Ar src_file
-
Uses
-
.Ar src_file
-
in order to restore the local package database.
-
Useful in case of a database crash or loss, to restore your database
-
from a previous backup.
-
.El
-
.Sh ENVIRONMENT
-
The following environment variables affect the execution of
-
.Nm .
-
See
-
.Xr pkg.conf 5
-
for further description.
-
.Bl -tag -width ".Ev NO_DESCRIPTIONS"
-
.It Ev PKG_DBDIR
-
.El
-
.Sh FILES
-
See
-
.Xr pkg.conf 5 .
-
.Sh SEE ALSO
-
.Xr pkg_create 3 ,
-
.Xr pkg_printf 3 ,
-
.Xr pkg_repos 3 ,
-
.Xr pkg-keywords 5 ,
-
.Xr pkg-lua-script 5 ,
-
.Xr pkg-repository 5 ,
-
.Xr pkg-script 5 ,
-
.Xr pkg-triggers 5 ,
-
.Xr pkg.conf 5 ,
-
.Xr pkg 8 ,
-
.Xr pkg-add 8 ,
-
.Xr pkg-alias 8 ,
-
.Xr pkg-annotate 8 ,
-
.Xr pkg-audit 8 ,
-
.Xr pkg-autoremove 8 ,
-
.Xr pkg-check 8 ,
-
.Xr pkg-clean 8 ,
-
.Xr pkg-config 8 ,
-
.Xr pkg-create 8 ,
-
.Xr pkg-delete 8 ,
-
.Xr pkg-fetch 8 ,
-
.Xr pkg-info 8 ,
-
.Xr pkg-install 8 ,
-
.Xr pkg-lock 8 ,
-
.Xr pkg-query 8 ,
-
.Xr pkg-register 8 ,
-
.Xr pkg-repo 8 ,
-
.Xr pkg-rquery 8 ,
-
.Xr pkg-search 8 ,
-
.Xr pkg-set 8 ,
-
.Xr pkg-shell 8 ,
-
.Xr pkg-shlib 8 ,
-
.Xr pkg-ssh 8 ,
-
.Xr pkg-stats 8 ,
-
.Xr pkg-triggers 8 ,
-
.Xr pkg-update 8 ,
-
.Xr pkg-updating 8 ,
-
.Xr pkg-upgrade 8 ,
-
.Xr pkg-version 8 ,
-
.Xr pkg-which 8
modified docs/pkg-check.8
@@ -174,7 +174,6 @@ pkg check -sa
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
.Xr pkg-create 8 ,
modified docs/pkg-clean.8
@@ -88,7 +88,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-config 8 ,
.Xr pkg-create 8 ,
modified docs/pkg-config.8
@@ -53,7 +53,6 @@ Getting the directories where the repositories configuration are read:
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-create 8 ,
modified docs/pkg-create.8
@@ -489,7 +489,6 @@ Create package file for pkg:
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-delete.8
@@ -157,7 +157,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-fetch.8
@@ -157,7 +157,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-info.8
@@ -191,7 +191,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-install.8
@@ -215,7 +215,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-keywords.5
@@ -167,7 +167,6 @@ being in the example case,
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-lock.8
@@ -159,7 +159,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-lua-script.5
@@ -11,7 +11,7 @@
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
-
.Dd March 26, 2021
+
.Dd July 27, 2022
.Dt PKG-LUA-SCRIPT 5
.Os
.Sh NAME
@@ -92,7 +92,7 @@ arguments passed to
Boolean to inform the scripts that it is running or not in the context of an upgrade
.El
.Pp
-
The following function has been added:
+
The following functions have been added:
.Bl -tag -width Ds
.It Ft out Fn pkg.prefixed_path "in"
prepend
@@ -125,6 +125,11 @@ with the following fields:
.Va size ,
.Va uid ,
.Va gid
+
.It Fn pkg.symlink "source" "destination"
+
Create a symbolic link
+
.Va destination
+
pointing at
+
.Va source
.It Fn pkg.exec arguments
Will execute the command
.Ar arguments
@@ -159,7 +164,6 @@ are be filtered out.
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-query.8
@@ -14,7 +14,7 @@
.\"
.\"     @(#)pkg.8
.\"
-
.Dd November 18, 2016
+
.Dd October 06, 2022
.Dt PKG-QUERY 8
.Os
.Sh NAME
@@ -336,14 +336,22 @@ The numerical value of
is less than
.Op or equal to
the given number.
-
.It Va var Cm = Ns Oo = Oc Ar num
-
The numerical value of
+
.It Va var Cm = Ns Oo = Oc Oo Ar num | Ar string Oc
+
The value of
.Va var
-
is equal to the given number.
-
.It Va var Cm != Ar num
-
The numerical value of
+
is equal to the given number or string.
+
.It Va var Cm =~ Oo Ar num | Ar string Oc
+
The value of
+
.Va var
+
is equal (case insensitive) to the given number or string.
+
.It Va var Cm != Oo Ar num | Ar string Oc
+
The value of
+
.Va var
+
is not equal to the given number or string.
+
.It Va var Cm !=~  Oo Ar num | Ar string Oc
+
The value of
.Va var
-
is not equal to the given number.
+
is not equal case insensitive to the given number or string.
.El
.Sh ENVIRONMENT
The following environment variables affect the execution of
@@ -406,7 +414,6 @@ List unmaintained packages:
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-register.8
@@ -196,7 +196,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-repo.8
@@ -220,7 +220,6 @@ filename.
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-repository.5
@@ -305,7 +305,6 @@ in
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-rquery.8
@@ -14,7 +14,7 @@
.\"
.\"     @(#)pkg.8
.\"
-
.Dd July 15, 2018
+
.Dd October 06, 2022
.Dt PKG-RQUERY 8
.Os
.Sh NAME
@@ -309,14 +309,22 @@ The numerical value of
is less than
.Op or equal to
the given number.
-
.It Va var Cm = Ns Oo = Oc Ar num
-
The numerical value of
+
.It Va var Cm = Ns Oo = Oc Oo Ar num | Ar string Oc
+
The value of
.Va var
-
is equal to the given number.
-
.It Va var Cm != Ar num
-
The numerical value of
+
is equal to the given number or string.
+
.It Va var Cm =~ Oo Ar num | Ar string Oc
+
The value of
+
.Va var
+
is equal (case insensitive) to the given number or string.
+
.It Va var Cm != Oo Ar num | Ar string Oc
+
The value of
+
.Va var
+
is not equal to the given number or string.
+
.It Va var Cm !=~  Oo Ar num | Ar string Oc
+
The value of
.Va var
-
is not equal to the given number.
+
is not equal case insensitive to the given number or string.
.El
.Sh ENVIRONMENT
The following environment variables affect the execution of
@@ -353,7 +361,6 @@ for example usage.
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-script.5
@@ -82,7 +82,6 @@ Variable existing when the scripts is run in the context of an upgrade of the pa
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-search.8
@@ -478,7 +478,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-set.8
@@ -157,7 +157,6 @@ for further description.
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-shell.8
@@ -65,7 +65,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-shlib.8
@@ -84,7 +84,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-ssh.8
@@ -63,7 +63,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-stats.8
@@ -68,7 +68,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-triggers.5
@@ -91,7 +91,6 @@ the transaction.
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-triggers.8
@@ -56,7 +56,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-update.8
@@ -113,7 +113,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-updating.8
@@ -5,7 +5,7 @@
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
-
.\" 8. Redistributions of source code must retain the above copyright
+
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
@@ -14,7 +14,7 @@
.\"
.\"     @(#)pkg.8
.\"
-
.Dd April 3, 2018
+
.Dd July 29, 2022
.Dt PKG-UPDATING 8
.Os
.Sh NAME
@@ -63,8 +63,12 @@ The following environment variables affect the execution of
Location of the ports tree.
.El
.Sh FILES
-
See
-
.Xr pkg.conf 5 .
+
.Bl -tag -width /usr/ports/UPDATING
+
.It Pa /usr/ports/UPDATING
+
The default location of the
+
.Pa UPDATING
+
file.
+
.El
.Sh EXAMPLES
Shows all entries of all installed ports:
.Dl % pkg updating
@@ -97,7 +101,6 @@ installed ports:
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-upgrade.8
@@ -218,7 +218,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-version.8
@@ -132,7 +132,7 @@ matching against
case sensitive.
.It Fl h , Fl -help
Displays usage information.
-
.It Fl I Op Ar index , Fl -index Op Ar index
+
.It Fl I Oo Ar index Oc , Fl -index Oo Ar index Oc
Use
.Pa index
file for determining if a package is out of date.
@@ -306,7 +306,6 @@ The following command compares two package version strings:
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg-which.8
@@ -78,7 +78,6 @@ See
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg.8
@@ -209,8 +209,6 @@ List the command line aliases.
Audit installed packages against known vulnerabilities.
.It Ic autoremove
Delete packages which were automatically installed as dependencies and are not required any more.
-
.It Ic backup
-
Dump the local package database to a file specified on the command-line.
.It Ic bootstrap
This is for compatibility with the
.Xr pkg 7
@@ -367,6 +365,10 @@ Check for missing dependencies:
.Pp
Show the pkg-message of a package:
.Dl # pkg info -D perl-5.14
+
.Pp
+
Restore a backup database:
+
.Dl % rm /var/db/pkg/local.sqlite
+
.Dl % xzcat /var/backups/pkg.sql.xz | pkg shell
.\" ---------------------------------------------------------------------------
.Sh SEE ALSO
.Xr pkg_create 3 ,
@@ -383,7 +385,6 @@ Show the pkg-message of a package:
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg.conf.5
@@ -545,7 +545,7 @@ same repository the older installed version came from, as given in the
.Sy repository
annotation of the installed package.
See
-
.Xr pkg-repository 8
+
.Xr pkg-repository 5
for details.
.Pp
It is possible to specify more than one repository per file.
@@ -656,7 +656,6 @@ ignored), and reads the configuration files instead.
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified docs/pkg_printf.3
@@ -923,7 +923,6 @@ Insufficient storage space is available.
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
-
.Xr pkg-backup 8 ,
.Xr pkg-check 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
modified external/include/tllist.h
@@ -190,3 +190,65 @@
        (list).length = 0;                                  \
        (list).head = (list).tail = NULL;                   \
    } while (0)
+

+
#define tll_sort(list, cmp)                                                 \
+
    do {                                                                    \
+
	__typeof((list).head) __p;                                          \
+
	__typeof((list).head) __q;                                          \
+
	__typeof((list).head) __e;                                          \
+
	__typeof((list).head) __t;                                          \
+
        int __insize, __nmerges, __p_size, __q_size;                        \
+
	if ((list).head == NULL)                                            \
+
	    break;                                                          \
+
        __insize = 1;                                                       \
+
        while (1) {                                                         \
+
		__p = (list).head;                                          \
+
		(list).head = NULL;                                         \
+
                __t = NULL;                                                 \
+
		__nmerges = 0;                                              \
+
		while (__p != NULL) {                                       \
+
	            __nmerges++;                                            \
+
	            __q = __p;                                              \
+
		    __p_size = 0;                                           \
+
		    for (int _i = 0; _i < __insize; _i++) {                 \
+
			    __p_size++;                                     \
+
			    __q = __q->next;                                \
+
			    if (__q == NULL)                                \
+
			        break;                                      \
+
		    }                                                       \
+
	            __q_size = __insize;                                    \
+
		    while (__p_size > 0 || (__q_size > 0 && __q != NULL)) { \
+
			    if (__p_size == 0) {                            \
+
				    __e = __q;                              \
+
				    __q = __q->next;                        \
+
				    __q_size--;                             \
+
			    } else if (__q_size == 0 || __q == NULL) {      \
+
				    __e = __p;                              \
+
				    __p = __p->next;                        \
+
				    __p_size--;                             \
+
			    } else if (cmp(__p->item, __q->item) <= 0) {    \
+
				    __e = __p;                              \
+
				    __p = __p->next;                        \
+
				    __p_size--;                             \
+
			    } else {                                        \
+
				    __e = __q;                              \
+
				    __q = __q->next;                        \
+
				    __q_size--;                             \
+
			    }                                               \
+
			    if (__t != NULL) {                              \
+
				    __t->next = __e;                        \
+
			    } else {                                        \
+
				    (list).head = __e;                      \
+
			    }                                               \
+
			    __e->prev = __t;                                \
+
			    __t = __e;                                      \
+
		    }                                                       \
+
	            __p = __q;                                              \
+
		}                                                           \
+
		(list).tail = __t;                                          \
+
		__t->next = NULL;                                           \
+
		__insize *= 2;                                              \
+
		if (__nmerges <= 1)                                         \
+
			break;                                              \
+
	}                                                                   \
+
    } while (0)
modified external/libfetch/common.c
@@ -30,7 +30,7 @@
 */

#include "bsd_compat.h"
-
__FBSDID("$FreeBSD: head/lib/libfetch/common.c 347050 2019-05-03 06:06:39Z adrian $");
+
__FBSDID("$FreeBSD$");

#include <sys/param.h>
#include <sys/socket.h>
@@ -42,7 +42,9 @@ __FBSDID("$FreeBSD: head/lib/libfetch/common.c 347050 2019-05-03 06:06:39Z adria
#include <ctype.h>
#include <errno.h>
#include <fcntl.h>
+
#include <inttypes.h>
#include <netdb.h>
+
#include <paths.h>
#include <poll.h>
#include <pwd.h>
#include <stdarg.h>
@@ -77,6 +79,64 @@ static struct fetcherr netdb_errlist[] = {
	{ -1,		FETCH_UNKNOWN,	"Unknown resolver error" }
};

+
/*
+
 * SOCKS5 error enumerations
+
 */
+
enum SOCKS5_ERR {
+
/* Protocol errors */
+
	SOCKS5_ERR_SELECTION,
+
	SOCKS5_ERR_READ_METHOD,
+
	SOCKS5_ERR_VER5_ONLY,
+
	SOCKS5_ERR_NOMETHODS,
+
	SOCKS5_ERR_NOTIMPLEMENTED,
+
	SOCKS5_ERR_HOSTNAME_SIZE,
+
	SOCKS5_ERR_REQUEST,
+
	SOCKS5_ERR_REPLY,
+
	SOCKS5_ERR_NON_VER5_RESP,
+
	SOCKS5_ERR_GENERAL,
+
	SOCKS5_ERR_NOT_ALLOWED,
+
	SOCKS5_ERR_NET_UNREACHABLE,
+
	SOCKS5_ERR_HOST_UNREACHABLE,
+
	SOCKS5_ERR_CONN_REFUSED,
+
	SOCKS5_ERR_TTL_EXPIRED,
+
	SOCKS5_ERR_COM_UNSUPPORTED,
+
	SOCKS5_ERR_ADDR_UNSUPPORTED,
+
	SOCKS5_ERR_UNSPECIFIED,
+
/* Configuration errors */
+
	SOCKS5_ERR_BAD_HOST,
+
	SOCKS5_ERR_BAD_PROXY_FORMAT,
+
	SOCKS5_ERR_BAD_PORT
+
};
+

+
/*
+
 * Error messages for SOCKS5 errors
+
 */
+
static struct fetcherr socks5_errlist[] = {
+
/* SOCKS5 protocol errors */
+
	{ SOCKS5_ERR_SELECTION,		FETCH_ABORT,	"SOCKS5: Failed to send selection method" },
+
	{ SOCKS5_ERR_READ_METHOD,	FETCH_ABORT,	"SOCKS5: Failed to read method" },
+
	{ SOCKS5_ERR_VER5_ONLY,		FETCH_PROTO,	"SOCKS5: Only version 5 is implemented" },
+
	{ SOCKS5_ERR_NOMETHODS,		FETCH_PROTO,	"SOCKS5: No acceptable methods" },
+
	{ SOCKS5_ERR_NOTIMPLEMENTED,	FETCH_PROTO,	"SOCKS5: Method currently not implemented" },
+
	{ SOCKS5_ERR_HOSTNAME_SIZE,	FETCH_PROTO,	"SOCKS5: Hostname size is above 256 bytes" },
+
	{ SOCKS5_ERR_REQUEST,		FETCH_PROTO,	"SOCKS5: Failed to request" },
+
	{ SOCKS5_ERR_REPLY,		FETCH_PROTO,	"SOCKS5: Failed to receive reply" },
+
	{ SOCKS5_ERR_NON_VER5_RESP,	FETCH_PROTO,	"SOCKS5: Server responded with a non-version 5 response" },
+
	{ SOCKS5_ERR_GENERAL,		FETCH_ABORT,	"SOCKS5: General server failure" },
+
	{ SOCKS5_ERR_NOT_ALLOWED,	FETCH_AUTH,	"SOCKS5: Connection not allowed by ruleset" },
+
	{ SOCKS5_ERR_NET_UNREACHABLE,	FETCH_NETWORK,	"SOCKS5: Network unreachable" },
+
	{ SOCKS5_ERR_HOST_UNREACHABLE,	FETCH_ABORT,	"SOCKS5: Host unreachable" },
+
	{ SOCKS5_ERR_CONN_REFUSED,	FETCH_ABORT,	"SOCKS5: Connection refused" },
+
	{ SOCKS5_ERR_TTL_EXPIRED,	FETCH_TIMEOUT,	"SOCKS5: TTL expired" },
+
	{ SOCKS5_ERR_COM_UNSUPPORTED,	FETCH_PROTO,	"SOCKS5: Command not supported" },
+
	{ SOCKS5_ERR_ADDR_UNSUPPORTED,	FETCH_ABORT,	"SOCKS5: Address type not supported" },
+
	{ SOCKS5_ERR_UNSPECIFIED,	FETCH_UNKNOWN,	"SOCKS5: Unspecified error" },
+
/* Configuration error */
+
	{ SOCKS5_ERR_BAD_HOST,		FETCH_ABORT,	"SOCKS5: Bad proxy host" },
+
	{ SOCKS5_ERR_BAD_PROXY_FORMAT,	FETCH_ABORT,	"SOCKS5: Bad proxy format" },
+
	{ SOCKS5_ERR_BAD_PORT,		FETCH_ABORT,	"SOCKS5: Bad port" }
+
};
+

/* End-of-Line */
static const char ENDL[2] = "\r\n";

@@ -326,7 +386,6 @@ syserr:
}


-

/*
 * Bind a socket to a specific local address
 */
@@ -349,31 +408,241 @@ fetch_bind(int sd, int af, const char *addr)


/*
+
 * SOCKS5 connection initiation, based on RFC 1928
+
 * Default DNS resolution over SOCKS5
+
 */
+
int
+
fetch_socks5_init(conn_t *conn, const char *host, int port, int verbose)
+
{
+
	/*
+
	 * Size is based on largest packet prefix (4 bytes) +
+
	 * Largest FQDN (256) + one byte size (1) +
+
	 * Port (2)
+
	 */
+
	unsigned char buf[BUFF_SIZE];
+
	unsigned char *ptr;
+
	int ret = 1;
+

+
	if (verbose)
+
		fetch_info("Initializing SOCKS5 connection: %s:%d", host, port);
+

+
	/* Connection initialization */
+
	ptr = buf;
+
	*ptr++ = SOCKS_VERSION_5;
+
	*ptr++ = SOCKS_CONNECTION;
+
	*ptr++ = SOCKS_RSV;
+

+
	if (fetch_write(conn, buf, 3) != 3) {
+
		ret = SOCKS5_ERR_SELECTION;
+
		goto fail;
+
	}
+

+
	/* Verify response from SOCKS5 server */
+
	if (fetch_read(conn, buf, 2) != 2) {
+
		ret = SOCKS5_ERR_READ_METHOD;
+
		goto fail;
+
	}
+

+
	ptr = buf;
+
	if (ptr[0] != SOCKS_VERSION_5) {
+
		ret = SOCKS5_ERR_VER5_ONLY;
+
		goto fail;
+
	}
+
	if (ptr[1] == SOCKS_NOMETHODS) {
+
		ret = SOCKS5_ERR_NOMETHODS;
+
		goto fail;
+
	}
+
	else if (ptr[1] != SOCKS5_NOTIMPLEMENTED) {
+
		ret = SOCKS5_ERR_NOTIMPLEMENTED;
+
		goto fail;
+
	}
+

+
	/* Send Request */
+
	*ptr++ = SOCKS_VERSION_5;
+
	*ptr++ = SOCKS_CONNECTION;
+
	*ptr++ = SOCKS_RSV;
+
	/* Encode all targets as a hostname to avoid DNS leaks */
+
	*ptr++ = SOCKS_ATYP_DOMAINNAME;
+
	if (strlen(host) > FQDN_SIZE) {
+
		ret = SOCKS5_ERR_HOSTNAME_SIZE;
+
		goto fail;
+
	}
+
	*ptr++ = strlen(host);
+
	strncpy(ptr, host, strlen(host));
+
	ptr = ptr + strlen(host);
+

+
	port = htons(port);
+
	*ptr++ = port & 0x00ff;
+
	*ptr++ = (port & 0xff00) >> 8;
+

+
	if (fetch_write(conn, buf, ptr - buf) != ptr - buf) {
+
		ret = SOCKS5_ERR_REQUEST;
+
		goto fail;
+
	}
+

+
	/* BND.ADDR is variable length, read the largest on non-blocking socket */
+
	if (!fetch_read(conn, buf, BUFF_SIZE)) {
+
		ret = SOCKS5_ERR_REPLY;
+
		goto fail;
+
	}
+

+
	ptr = buf;
+
	if (*ptr++ != SOCKS_VERSION_5) {
+
		ret = SOCKS5_ERR_NON_VER5_RESP;
+
		goto fail;
+
	}
+

+
	switch(*ptr++) {
+
	case SOCKS_SUCCESS:
+
		break;
+
	case SOCKS_GENERAL_FAILURE:
+
		ret = SOCKS5_ERR_GENERAL;
+
		goto fail;
+
	case SOCKS_CONNECTION_NOT_ALLOWED:
+
		ret = SOCKS5_ERR_NOT_ALLOWED;
+
		goto fail;
+
	case SOCKS_NETWORK_UNREACHABLE:
+
		ret = SOCKS5_ERR_NET_UNREACHABLE;
+
		goto fail;
+
	case SOCKS_HOST_UNREACHABLE:
+
		ret = SOCKS5_ERR_HOST_UNREACHABLE;
+
		goto fail;
+
	case SOCKS_CONNECTION_REFUSED:
+
		ret = SOCKS5_ERR_CONN_REFUSED;
+
		goto fail;
+
	case SOCKS_TTL_EXPIRED:
+
		ret = SOCKS5_ERR_TTL_EXPIRED;
+
		goto fail;
+
	case SOCKS_COMMAND_NOT_SUPPORTED:
+
		ret = SOCKS5_ERR_COM_UNSUPPORTED;
+
		goto fail;
+
	case SOCKS_ADDRESS_NOT_SUPPORTED:
+
		ret = SOCKS5_ERR_ADDR_UNSUPPORTED;
+
		goto fail;
+
	default:
+
		ret = SOCKS5_ERR_UNSPECIFIED;
+
		goto fail;
+
	}
+

+
	return (ret);
+

+
fail:
+
	socks5_seterr(ret);
+
	return (0);
+
}
+

+
/*
+
 * Perform SOCKS5 initialization
+
 */
+
int
+
fetch_socks5_getenv(char **host, int *port)
+
{
+
	char *socks5env, *endptr, *ext;
+
	const char *portDelim;
+
	size_t slen;
+

+
	portDelim = ":";
+
	if ((socks5env = getenv("SOCKS5_PROXY")) == NULL || *socks5env == '\0') {
+
		*host = NULL;
+
		*port = -1;
+
		return (-1);
+
	}
+

+
	/*
+
	 * IPv6 addresses begin and end in brackets.  Set the port delimiter
+
	 * accordingly and search for it so we can do appropriate validation.
+
	 */
+
	if (socks5env[0] == '[')
+
		portDelim = "]:";
+

+
	slen = strlen(socks5env);
+
	ext = strstr(socks5env, portDelim);
+
	if (socks5env[0] == '[') {
+
		if (socks5env[slen - 1] == ']') {
+
			*host = strndup(socks5env, slen);
+
		} else if (ext != NULL) {
+
			*host = strndup(socks5env, ext - socks5env + 1);
+
		} else {
+
			socks5_seterr(SOCKS5_ERR_BAD_PROXY_FORMAT);
+
			return (0);
+
		}
+
	} else {
+
		*host = strndup(socks5env, ext - socks5env);
+
	}
+

+
	if (*host == NULL) {
+
		fprintf(stderr, "Failure to allocate memory, exiting.\n");
+
		return (-1);
+
	}
+
	if (ext == NULL) {
+
		*port = 1080; /* Default port as defined in RFC1928 */
+
	} else {
+
		ext += strlen(portDelim);
+
		errno = 0;
+
		*port = strtoimax(ext, (char **)&endptr, 10);
+
		if (*endptr != '\0' || errno != 0 || *port < 0 ||
+
		    *port > 65535) {
+
			free(*host);
+
			*host = NULL;
+
			socks5_seterr(SOCKS5_ERR_BAD_PORT);
+
			return (0);
+
		}
+
	}
+

+
	return (2);
+
}
+

+

+
/*
 * Establish a TCP connection to the specified port on the specified host.
 */
conn_t *
-
fetch_connect(struct url *u, int af, int verbose)
+
fetch_connect(const char *host, int port, int af, int verbose)
{
	struct addrinfo *cais = NULL, *sais = NULL, *cai, *sai;
	const char *bindaddr;
	conn_t *conn = NULL;
	int err = 0, sd = -1;
+
	char *sockshost;
+
	int socksport;

-
	DEBUGF("---> %s:%d\n", u->host, u->port);
+
	DEBUGF("---> %s:%d\n", host, port);

-
	/* resolve server address */
-
	if (verbose)
-
		fetch_info("resolving server address: %s:%d", u->host, u->port);
-
	if ((sais = fetch_resolve(u->host, u->port, af)) == NULL)
+
	/*
+
	 * Check if SOCKS5_PROXY env variable is set.  fetch_socks5_getenv
+
	 * will either set sockshost = NULL or allocate memory in all cases.
+
	 */
+
	sockshost = NULL;
+
	if (!fetch_socks5_getenv(&sockshost, &socksport))
		goto fail;

-
	/* resolve client address */
-
	bindaddr = getenv("FETCH_BIND_ADDRESS");
-
	if (bindaddr != NULL && *bindaddr != '\0') {
+
	/* Not using SOCKS5 proxy */
+
	if (sockshost == NULL) {
+
		/* resolve server address */
+
		if (verbose)
+
			fetch_info("resolving server address: %s:%d", host,
+
			    port);
+
		if ((sais = fetch_resolve(host, port, af)) == NULL)
+
			goto fail;
+

+
		/* resolve client address */
+
		bindaddr = getenv("FETCH_BIND_ADDRESS");
+
		if (bindaddr != NULL && *bindaddr != '\0') {
+
			if (verbose)
+
				fetch_info("resolving client address: %s",
+
				    bindaddr);
+
			if ((cais = fetch_resolve(bindaddr, 0, af)) == NULL)
+
				goto fail;
+
		}
+
	} else {
+
		/* resolve socks5 proxy address */
		if (verbose)
-
			fetch_info("resolving client address: %s", bindaddr);
-
		if ((cais = fetch_resolve(bindaddr, 0, af)) == NULL)
+
			fetch_info("resolving SOCKS5 server address: %s:%d",
+
			    sockshost, socksport);
+
		if ((sais = fetch_resolve(sockshost, socksport, af)) == NULL) {
+
			socks5_seterr(SOCKS5_ERR_BAD_HOST);
			goto fail;
+
		}
	}

	/* try each server address in turn */
@@ -401,21 +670,27 @@ fetch_connect(struct url *u, int af, int verbose)
		sd = -1;
	}
	if (err != 0) {
-
		if (verbose)
-
			fetch_info("failed to connect to %s:%d", u->host, u->port);
+
		if (verbose && sockshost == NULL) {
+
			fetch_info("failed to connect to %s:%d", host, port);
+
			goto syserr;
+
		} else if (sockshost != NULL) {
+
			if (verbose)
+
				fetch_info(
+
				    "failed to connect to SOCKS5 server %s:%d",
+
				    sockshost, socksport);
+
			socks5_seterr(SOCKS5_ERR_CONN_REFUSED);
+
			goto fail;
+
		}
		goto syserr;
	}

	if ((conn = fetch_reopen(sd)) == NULL)
		goto syserr;

-
	strlcpy(conn->scheme, u->scheme, sizeof(conn->scheme));
-
	strlcpy(conn->host, u->host, sizeof(conn->host));
-
	strlcpy(conn->user, u->user, sizeof(conn->user));
-
	strlcpy(conn->pwd, u->pwd, sizeof(conn->pwd));
-
	conn->port = u->port;
-
	conn->af = af;
-

+
	if (sockshost)
+
		if (!fetch_socks5_init(conn, host, port, verbose))
+
			goto fail;
+
	free(sockshost);
	if (cais != NULL)
		freeaddrinfo(cais);
	if (sais != NULL)
@@ -423,9 +698,12 @@ fetch_connect(struct url *u, int af, int verbose)
	return (conn);
syserr:
	fetch_syserr();
-
	goto fail;
fail:
-
	if (sd >= 0)
+
	free(sockshost);
+
	/* Fully close if it was opened; otherwise just don't leak the fd. */
+
	if (conn != NULL)
+
		fetch_close(conn);
+
	else if (sd >= 0)
		close(sd);
	if (cais != NULL)
		freeaddrinfo(cais);
@@ -433,103 +711,6 @@ fail:
		freeaddrinfo(sais);
	return (NULL);
}
-
static conn_t *connection_cache;
-
static int cache_global_limit = 0;
-
static int cache_per_host_limit = 0;
-

-
/*
-
 * Initialise cache with the given limits.
-
 */
-
void
-
fetchConnectionCacheInit(int global_limit, int per_host_limit)
-
{
-

-
	if (global_limit < 0)
-
		cache_global_limit = INT_MAX;
-
	else if (per_host_limit > global_limit)
-
		cache_global_limit = per_host_limit;
-
	else
-
		cache_global_limit = global_limit;
-
	if (per_host_limit < 0)
-
		cache_per_host_limit = INT_MAX;
-
	else
-
		cache_per_host_limit = per_host_limit;
-
}
-

-
/*
-
 * Flush cache and free all associated resources.
-
 */
-
void
-
fetchConnectionCacheClose(void)
-
{
-
	conn_t *conn;
-

-
	while ((conn = connection_cache) != NULL) {
-
		connection_cache = conn->next;
-
		(*conn->close)(conn);
-
	}
-
}
-

-
/*
-
 * Check connection cache for an existing entry matching
-
 * protocol/host/port/user/password/family.
-
 */
-
conn_t *
-
fetch_cache_get(const struct url *url, int af)
-
{
-
	conn_t *conn, *last_conn = NULL;
-

-
	for (conn = connection_cache; conn; conn = conn->next) {
-
		if (conn->port == url->port &&
-
		    strcmp(conn->scheme, url->scheme) == 0 &&
-
		    strcmp(conn->host, url->host) == 0 &&
-
		    strcmp(conn->user, url->user) == 0 &&
-
		    strcmp(conn->pwd, url->pwd) == 0 &&
-
		    (conn->af == AF_UNSPEC || af == AF_UNSPEC ||
-
		     conn->af == af)) {
-
			if (last_conn != NULL)
-
				last_conn->next = conn->next;
-
			else
-
				connection_cache = conn->next;
-

-
			return conn;
-
		}
-
	}
-

-
	return NULL;
-
}
-

-
/*
-
 * Put the connection back into the cache for reuse.
-
 */
-
void
-
fetch_cache_put(conn_t *conn, int (*closecb)(conn_t *))
-
{
-
	conn_t *iter, *last;
-
	int global_count, host_count;
-

-
	global_count = host_count = 0;
-
	last = NULL;
-
	for (iter = connection_cache; iter;
-
	    last = iter, iter = iter->next) {
-
		++global_count;
-
		if (strcmp(conn->host, iter->host) == 0)
-
			++host_count;
-
		if (global_count < cache_global_limit &&
-
		    host_count < cache_per_host_limit)
-
			continue;
-
		--global_count;
-
		if (last != NULL)
-
			last->next = iter->next;
-
		else
-
			connection_cache = iter->next;
-
		(*iter->close)(iter);
-
	}
-

-
	conn->close = closecb;
-
	conn->next = connection_cache;
-
	connection_cache = conn;
-
}

#ifdef WITH_SSL
/*
@@ -885,9 +1066,7 @@ fetch_ssl_setup_transport_layer(SSL_CTX *ctx, int verbose)
{
	long ssl_ctx_options;

-
	ssl_ctx_options = SSL_OP_ALL | SSL_OP_NO_SSLv2 | SSL_OP_NO_TICKET;
-
	if (getenv("SSL_ALLOW_SSL3") == NULL)
-
		ssl_ctx_options |= SSL_OP_NO_SSLv3;
+
	ssl_ctx_options = SSL_OP_ALL | SSL_OP_NO_SSLv3 | SSL_OP_NO_TICKET;
	if (getenv("SSL_NO_TLS1") != NULL)
		ssl_ctx_options |= SSL_OP_NO_TLSv1;
	if (getenv("SSL_NO_TLS1_1") != NULL)
modified external/libfetch/common.h
@@ -61,14 +61,6 @@ struct fetchconn {
	const SSL_METHOD *ssl_meth;	/* SSL method */
#endif
	int		 ref;		/* reference count */
-
	char		 scheme[URL_SCHEMELEN+1];
-
	char		 user[URL_USERLEN+1];
-
	char		 pwd[URL_PWDLEN+1];
-
	char		 host[MAXHOSTNAMELEN+1];
-
	int		 port;
-
	int		 af;
-
	int		(*close)(conn_t *);
-
	conn_t		*next;
};

/* Structure used for error message lists */
@@ -78,17 +70,52 @@ struct fetcherr {
	const char	*string;
};

+
/* For SOCKS header size */
+
#define HEAD_SIZE	4
+
#define FQDN_SIZE	256
+
#define PACK_SIZE	1
+
#define PORT_SIZE	2
+
#define BUFF_SIZE	HEAD_SIZE + FQDN_SIZE + PACK_SIZE + PORT_SIZE
+

+
/* SOCKS5 Request Header */
+
#define SOCKS_VERSION_5		0x05
+
/* SOCKS5 CMD */
+
#define SOCKS_CONNECTION	0x01
+
#define SOCKS_BIND		0x02
+
#define SOCKS_UDP		0x03
+
#define SOCKS_NOMETHODS		0xFF
+
#define SOCKS5_NOTIMPLEMENTED	0x00
+
/* SOCKS5 Reserved */
+
#define SOCKS_RSV		0x00
+
/* SOCKS5 Address Type */
+
#define SOCKS_ATYP_IPV4		0x01
+
#define SOCKS_ATYP_DOMAINNAME	0x03
+
#define SOCKS_ATYP_IPV6		0x04
+
/* SOCKS5 Reply Field */
+
#define SOCKS_SUCCESS			0x00
+
#define SOCKS_GENERAL_FAILURE		0x01
+
#define SOCKS_CONNECTION_NOT_ALLOWED	0x02
+
#define SOCKS_NETWORK_UNREACHABLE	0x03
+
#define SOCKS_HOST_UNREACHABLE		0x04
+
#define SOCKS_CONNECTION_REFUSED	0x05
+
#define SOCKS_TTL_EXPIRED		0x06
+
#define SOCKS_COMMAND_NOT_SUPPORTED	0x07
+
#define SOCKS_ADDRESS_NOT_SUPPORTED	0x08
+

/* for fetch_writev */
struct iovec;

void		 fetch_seterr(struct fetcherr *, int);
void		 fetch_syserr(void);
void		 fetch_info(const char *, ...) __printflike(1, 2);
+
int		 fetch_socks5_getenv(char **host, int *port);
+
int		 fetch_socks5_init(conn_t *conn, const char *host,
+
		     int port, int verbose);
int		 fetch_default_port(const char *);
int		 fetch_default_proxy_port(const char *);
struct addrinfo *fetch_resolve(const char *, int, int);
int		 fetch_bind(int, int, const char *);
-
conn_t		*fetch_connect(struct url *, int, int);
+
conn_t		*fetch_connect(const char *, int, int, int);
conn_t		*fetch_reopen(int);
conn_t		*fetch_ref(conn_t *);
#ifdef WITH_SSL
@@ -105,13 +132,12 @@ int fetch_add_entry(struct url_ent **, int *, int *,
		     const char *, struct url_stat *);
int		 fetch_netrc_auth(struct url *url);
int		 fetch_no_proxy_match(const char *);
-
conn_t		*fetch_cache_get(const struct url *, int);
-
void		 fetch_cache_put(conn_t *conn, int (*closecb)(conn_t *));

#define ftp_seterr(n)	 fetch_seterr(ftp_errlist, n)
#define http_seterr(n)	 fetch_seterr(http_errlist, n)
#define netdb_seterr(n)	 fetch_seterr(netdb_errlist, n)
#define url_seterr(n)	 fetch_seterr(url_errlist, n)
+
#define socks5_seterr(n) fetch_seterr(socks5_errlist, n)

#ifndef NDEBUG
#define DEBUGF(...)							\
modified external/libfetch/fetch.h
@@ -133,17 +133,12 @@ struct url_ent *fetchList(struct url *, const char *);
struct url	*fetchMakeURL(const char *, const char *, int,
		     const char *, const char *, const char *);
struct url	*fetchParseURL(const char *);
-
struct url	*fetchDupURL(struct url *);
void		 fetchFreeURL(struct url *);

#ifdef __cplusplus
}
#endif

-
/* Connection caching */
-
void fetchConnectionCacheInit(int, int);
-
void fetchConnectionCacheClose(void);
-

/* Authentication */
typedef int (*auth_t)(struct url *);
extern auth_t		 fetchAuthMethod;
modified external/libfetch/http.c
@@ -73,6 +73,7 @@ __FBSDID("$FreeBSD: head/lib/libfetch/http.c 351573 2019-08-28 17:01:28Z markj $
#include <locale.h>
#include <netdb.h>
#include <stdarg.h>
+
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -131,7 +132,6 @@ struct httpio
{
	conn_t		*conn;		/* connection */
	int		 chunked;	/* chunked mode */
-
	int		 keep_alive;	/* keep-alive mode */
	char		*buf;		/* chunk buffer */
	size_t		 bufsize;	/* size of chunk buffer */
	size_t		 buflen;	/* amount of data currently in buffer */
@@ -317,24 +317,11 @@ static int
http_closefn(void *v)
{
	struct httpio *io = (struct httpio *)v;
-
	int r, val;
-

-
	if (io->keep_alive) {
-
		val = 0;
-
		setsockopt(io->conn->sd, IPPROTO_TCP, TCP_NODELAY, &val,
-
		    sizeof(val));
-
		fetch_cache_put(io->conn, fetch_close);
-
#ifdef TCP_NOPUSH
-
		val = 1;
-
		setsockopt(io->conn->sd, IPPROTO_TCP, TCP_NOPUSH, &val,
-
		    sizeof(val));
-
#endif
-
		r = 0;
-
	} else {
-
		r = fetch_close(io->conn);
-
	}
+
	int r;

-
	free(io->buf);
+
	r = fetch_close(io->conn);
+
	if (io->buf)
+
		free(io->buf);
	free(io);
	return (r);
}
@@ -343,7 +330,7 @@ http_closefn(void *v)
 * Wrap a file descriptor up
 */
static FILE *
-
http_funopen(conn_t *conn, int chunked, int keep_alive)
+
http_funopen(conn_t *conn, int chunked)
{
	struct httpio *io;
	FILE *f;
@@ -354,7 +341,6 @@ http_funopen(conn_t *conn, int chunked, int keep_alive)
	}
	io->conn = conn;
	io->chunked = chunked;
-
	io->keep_alive = keep_alive;
	f = funopen(io, http_readfn, http_writefn, NULL, http_closefn);
	if (f == NULL) {
		fetch_syserr();
@@ -375,7 +361,6 @@ typedef enum {
	hdr_error = -1,
	hdr_end = 0,
	hdr_unknown = 1,
-
	hdr_connection,
	hdr_content_length,
	hdr_content_range,
	hdr_last_modified,
@@ -390,7 +375,6 @@ static struct {
	hdr_t		 num;
	const char	*name;
} hdr_names[] = {
-
	{ hdr_connection,		"Connection" },
	{ hdr_content_length,		"Content-Length" },
	{ hdr_content_range,		"Content-Range" },
	{ hdr_last_modified,		"Last-Modified" },
@@ -992,13 +976,12 @@ http_base64(const char *src)
	    "0123456789+/";
	char *str, *dst;
	size_t l;
-
	int t, r;
+
	int t;

	l = strlen(src);
	if ((str = malloc(((l + 2) / 3) * 4 + 1)) == NULL)
		return (NULL);
	dst = str;
-
	r = 0;

	while (l >= 3) {
		t = (src[0] << 16) | (src[1] << 8) | src[2];
@@ -1007,7 +990,7 @@ http_base64(const char *src)
		dst[2] = base64[(t >> 6) & 0x3f];
		dst[3] = base64[(t >> 0) & 0x3f];
		src += 3; l -= 3;
-
		dst += 4; r += 4;
+
		dst += 4;
	}

	switch (l) {
@@ -1018,7 +1001,6 @@ http_base64(const char *src)
		dst[2] = base64[(t >> 6) & 0x3f];
		dst[3] = '=';
		dst += 4;
-
		r += 4;
		break;
	case 1:
		t = src[0] << 16;
@@ -1026,7 +1008,6 @@ http_base64(const char *src)
		dst[1] = base64[(t >> 12) & 0x3f];
		dst[2] = dst[3] = '=';
		dst += 4;
-
		r += 4;
		break;
	case 0:
		break;
@@ -1394,7 +1375,7 @@ http_authorize(conn_t *conn, const char *hdr, http_auth_challenges_t *cs,
 * Connect to the correct HTTP server or proxy.
 */
static conn_t *
-
http_connect(struct url *URL, struct url *purl, const char *flags, int *cached)
+
http_connect(struct url *URL, struct url *purl, const char *flags)
{
	struct url *curl;
	conn_t *conn;
@@ -1404,6 +1385,8 @@ http_connect(struct url *URL, struct url *purl, const char *flags, int *cached)
	int verbose;
	int af, val;
	int serrno;
+
	bool isproxyauth = false;
+
	http_auth_challenges_t proxy_challenges;

#ifdef INET6
	af = AF_UNSPEC;
@@ -1421,23 +1404,58 @@ http_connect(struct url *URL, struct url *purl, const char *flags, int *cached)

	curl = (purl != NULL) ? purl : URL;

-
	if ((conn = fetch_cache_get(curl, af)) != NULL) {
-
		*cached = 1;
-
		return (conn);
-
	}
-

-
	if ((conn = fetch_connect(curl, af, verbose)) == NULL)
+
retry:
+
	if ((conn = fetch_connect(curl->host, curl->port, af, verbose)) == NULL)
		/* fetch_connect() has already set an error code */
		return (NULL);
	init_http_headerbuf(&headerbuf);
	if (strcmp(URL->scheme, SCHEME_HTTPS) == 0 && purl) {
-
		http_cmd(conn, "CONNECT %s:%d HTTP/1.1",
-
		    URL->host, URL->port);
-
		http_cmd(conn, "Host: %s:%d",
-
		    URL->host, URL->port);
+
		init_http_auth_challenges(&proxy_challenges);
+
		http_cmd(conn, "CONNECT %s:%d HTTP/1.1", URL->host, URL->port);
+
		http_cmd(conn, "Host: %s:%d", URL->host, URL->port);
+
		if (isproxyauth) {
+
			http_auth_params_t aparams;
+
			init_http_auth_params(&aparams);
+
			if (*purl->user || *purl->pwd) {
+
				aparams.user = strdup(purl->user);
+
				aparams.password = strdup(purl->pwd);
+
			} else if ((p = getenv("HTTP_PROXY_AUTH")) != NULL &&
+
				    *p != '\0') {
+
				if (http_authfromenv(p, &aparams) < 0) {
+
					http_seterr(HTTP_NEED_PROXY_AUTH);
+
					fetch_syserr();
+
					goto ouch;
+
				}
+
			} else if (fetch_netrc_auth(purl) == 0) {
+
				aparams.user = strdup(purl->user);
+
				aparams.password = strdup(purl->pwd);
+
			} else {
+
				/*
+
				 * No auth information found in system - exiting
+
				 * with warning.
+
				 */
+
				warnx("Missing username and/or password set");
+
				fetch_syserr();
+
				goto ouch;
+
			}
+
			http_authorize(conn, "Proxy-Authorization",
+
			    &proxy_challenges, &aparams, purl);
+
			clean_http_auth_params(&aparams);
+
		}
		http_cmd(conn, "");
-
		if (http_get_reply(conn) != HTTP_OK) {
-
			http_seterr(conn->err);
+
		/* Get reply from CONNECT Tunnel attempt */
+
		int httpreply = http_get_reply(conn);
+
		if (httpreply != HTTP_OK) {
+
			http_seterr(httpreply);
+
			/* If the error is a 407/HTTP_NEED_PROXY_AUTH */
+
			if (httpreply == HTTP_NEED_PROXY_AUTH &&
+
			    ! isproxyauth) {
+
				/* Try again with authentication. */
+
				clean_http_headerbuf(&headerbuf);
+
				fetch_close(conn);
+
				isproxyauth = true;
+
				goto retry;
+
			}
			goto ouch;
		}
		/* Read and discard the rest of the proxy response */
@@ -1507,12 +1525,13 @@ http_get_proxy(struct url * url, const char *flags)
static void
http_print_html(FILE *out, FILE *in)
{
-
	size_t len = 0;
+
	ssize_t len = 0;
+
	size_t cap;
	char *line = NULL, *p, *q;
	int comment, tag;

	comment = tag = 0;
-
	while (getline(&line, &len, in) >= 0) {
+
	while ((len = getline(&line, &cap, in)) >= 0) {
		while (len && isspace((unsigned char)line[len - 1]))
			--len;
		for (p = q = line; q < line + len; ++q) {
@@ -1572,7 +1591,7 @@ http_request_body(struct url *URL, const char *op, struct url_stat *us,
	char hbuf[MAXHOSTNAMELEN + 7], *host;
	conn_t *conn;
	struct url *url, *new;
-
	int chunked, direct, ims, keep_alive, noredirect, verbose, cached;
+
	int chunked, direct, ims, noredirect, verbose;
	int e, i, n, val;
	off_t offset, clength, length, size;
	time_t mtime;
@@ -1594,7 +1613,6 @@ http_request_body(struct url *URL, const char *op, struct url_stat *us,
	noredirect = CHECK_FLAG('A');
	verbose = CHECK_FLAG('v');
	ims = CHECK_FLAG('i');
-
	keep_alive = 0;

	if (direct && purl) {
		fetchFreeURL(purl);
@@ -1616,14 +1634,13 @@ http_request_body(struct url *URL, const char *op, struct url_stat *us,
		length = -1;
		size = -1;
		mtime = 0;
-
		cached = 0;

		/* check port */
		if (!url->port)
			url->port = fetch_default_port(url->scheme);

		/* connect to server or proxy */
-
		if ((conn = http_connect(url, purl, flags, &cached)) == NULL)
+
		if ((conn = http_connect(url, purl, flags)) == NULL)
			goto ouch;

		/* append port number only if necessary */
@@ -1850,10 +1867,6 @@ http_request_body(struct url *URL, const char *op, struct url_stat *us,
			case hdr_error:
				http_seterr(HTTP_PROTOCOL_ERROR);
				goto ouch;
-
			case hdr_connection:
-
				/* XXX too weak? */
-
				keep_alive = (strcasecmp(p, "keep-alive") == 0);
-
				break;
			case hdr_content_length:
				http_parse_length(p, &clength);
				break;
@@ -2028,7 +2041,7 @@ http_request_body(struct url *URL, const char *op, struct url_stat *us,
	URL->length = clength;

	/* wrap it up in a FILE */
-
	if ((f = http_funopen(conn, chunked, keep_alive)) == NULL) {
+
	if ((f = http_funopen(conn, chunked)) == NULL) {
		fetch_syserr();
		goto ouch;
	}
modified external/linenoise/linenoise.c
@@ -125,6 +125,7 @@ static linenoiseHintsCallback *hintsCallback = NULL;
static linenoiseFreeHintsCallback *freeHintsCallback = NULL;

static struct termios orig_termios; /* In order to restore at exit.*/
+
static int maskmode = 0; /* Show "***" instead of input. For passwords. */
static int rawmode = 0; /* For atexit() function to check if restore is needed*/
static int mlmode = 0;  /* Multi line mode. Default is single line. */
static int atexit_registered = 0; /* Register atexit just 1 time. */
@@ -197,6 +198,19 @@ FILE *lndebug_fp = NULL;

/* ======================= Low level terminal handling ====================== */

+
/* Enable "mask mode". When it is enabled, instead of the input that
+
 * the user is typing, the terminal will just display a corresponding
+
 * number of asterisks, like "****". This is useful for passwords and other
+
 * secrets that should not be displayed. */
+
void linenoiseMaskModeEnable(void) {
+
    maskmode = 1;
+
}
+

+
/* Disable mask mode. */
+
void linenoiseMaskModeDisable(void) {
+
    maskmode = 0;
+
}
+

/* Set if to use or not the multi line mode. */
void linenoiseSetMultiLine(int ml) {
    mlmode = ml;
@@ -525,7 +539,11 @@ static void refreshSingleLine(struct linenoiseState *l) {
    abAppend(&ab,seq,strlen(seq));
    /* Write the prompt and the current buffer content */
    abAppend(&ab,l->prompt,strlen(l->prompt));
-
    abAppend(&ab,buf,len);
+
    if (maskmode == 1) {
+
        while (len--) abAppend(&ab,"*",1);
+
    } else {
+
        abAppend(&ab,buf,len);
+
    }
    /* Show hits if any. */
    refreshShowHints(&ab,l,plen);
    /* Erase to right */
@@ -579,7 +597,12 @@ static void refreshMultiLine(struct linenoiseState *l) {

    /* Write the prompt and the current buffer content */
    abAppend(&ab,l->prompt,strlen(l->prompt));
-
    abAppend(&ab,l->buf,l->len);
+
    if (maskmode == 1) {
+
        unsigned int i;
+
        for (i = 0; i < l->len; i++) abAppend(&ab,"*",1);
+
    } else {
+
        abAppend(&ab,l->buf,l->len);
+
    }

    /* Show hits if any. */
    refreshShowHints(&ab,l,plen);
@@ -647,7 +670,8 @@ int linenoiseEditInsert(struct linenoiseState *l, char c) {
            if ((!mlmode && l->plen+l->len < l->cols && !hintsCallback)) {
                /* Avoid a full update of the line in the
                 * trivial case. */
-
                if (write(l->ofd,&c,1) == -1) return -1;
+
                char d = (maskmode==1) ? '*' : c;
+
                if (write(l->ofd,&d,1) == -1) return -1;
            } else {
                refreshLine(l);
            }
modified external/linenoise/linenoise.h
@@ -65,6 +65,8 @@ int linenoiseHistoryLoad(const char *filename);
void linenoiseClearScreen(void);
void linenoiseSetMultiLine(int ml);
void linenoisePrintKeyCodes(void);
+
void linenoiseMaskModeEnable(void);
+
void linenoiseMaskModeDisable(void);

#ifdef __cplusplus
}
modified external/lua/Makefile
@@ -46,7 +46,7 @@ TO_MAN= lua.1 luac.1

# Lua version and release.
V= 5.4
-
R= $V.2
+
R= $V.4

# Targets start here.
all:	$(PLAT)
modified external/lua/README
@@ -1,5 +1,5 @@

-
This is Lua 5.4.2, released on 13 Nov 2020.
+
This is Lua 5.4.4, released on 13 Jan 2022.

For installation instructions, license details, and
further information about Lua, see doc/readme.html.
modified external/lua/doc/contents.html
@@ -32,7 +32,7 @@ For a complete introduction to Lua programming, see the book

<P>
<SMALL>
-
Copyright &copy; 2020 Lua.org, PUC-Rio.
+
Copyright &copy; 2020&ndash;2022 Lua.org, PUC-Rio.
Freely available under the terms of the
<A HREF="http://www.lua.org/license.html">Lua license</A>.
</SMALL>
@@ -396,6 +396,7 @@ Freely available under the terms of the
<A HREF="manual.html#lua_callk">lua_callk</A><BR>
<A HREF="manual.html#lua_checkstack">lua_checkstack</A><BR>
<A HREF="manual.html#lua_close">lua_close</A><BR>
+
<A HREF="manual.html#lua_closeslot">lua_closeslot</A><BR>
<A HREF="manual.html#lua_compare">lua_compare</A><BR>
<A HREF="manual.html#lua_concat">lua_concat</A><BR>
<A HREF="manual.html#lua_copy">lua_copy</A><BR>
@@ -663,10 +664,10 @@ Freely available under the terms of the

<P CLASS="footer">
Last update:
-
Tue Nov 10 20:58:52 UTC 2020
+
Thu Jan 13 11:32:22 UTC 2022
</P>
<!--
-
Last change: revised for Lua 5.4.2
+
Last change: revised for Lua 5.4.4
-->

</BODY>
modified external/lua/doc/manual.html
@@ -19,7 +19,7 @@ by Roberto Ierusalimschy, Luiz Henrique de Figueiredo, Waldemar Celes

<P>
<SMALL>
-
Copyright &copy; 2020 Lua.org, PUC-Rio.
+
Copyright &copy; 2020&ndash;2022 Lua.org, PUC-Rio.
Freely available under the terms of the
<a href="http://www.lua.org/license.html">Lua license</a>.
</SMALL>
@@ -143,6 +143,11 @@ The type <em>boolean</em> has two values, <b>false</b> and <b>true</b>.
Both <b>nil</b> and <b>false</b> make a condition false;
they are collectively called <em>false values</em>.
Any other value makes a condition true.
+
Despite its name,
+
<b>false</b> is frequently used as an alternative to <b>nil</b>,
+
with the key difference that <b>false</b> behaves
+
like a regular value in a table,
+
while a <b>nil</b> in a table represents an absent key.


<p>
@@ -434,7 +439,7 @@ under certain events.
You can change several aspects of the behavior
of a value by setting specific fields in its metatable.
For instance, when a non-numeric value is the operand of an addition,
-
Lua checks for a function in the field "<code>__add</code>" of the value's metatable.
+
Lua checks for a function in the field <code>__add</code> of the value's metatable.
If it finds one,
Lua calls this function to perform the addition.

@@ -901,7 +906,7 @@ For an object (table or userdata) to be finalized when collected,
you must <em>mark</em> it for finalization.

You mark an object for finalization when you set its metatable
-
and the metatable has a field indexed by the string "<code>__gc</code>".
+
and the metatable has a <code>__gc</code> metamethod.
Note that if you set a metatable without a <code>__gc</code> field
and later create that field in the metatable,
the object will not be marked for finalization.
@@ -956,11 +961,8 @@ these marks have no effect.


<p>
-
Finalizers cannot yield.
-
Except for that, they can do anything,
-
such as raise errors, create new objects,
-
or even run the garbage collector.
-
However, because they can run in unpredictable times,
+
Finalizers cannot yield nor run the garbage collector.
+
Because they can run in unpredictable times,
it is good practice to restrict each finalizer
to the minimum necessary to properly release
its associated resource.
@@ -1631,8 +1633,10 @@ before the adjustment


<p>
-
The assignment statement first evaluates all its expressions
-
and only then the assignments are performed.
+
If a variable is both assigned and read
+
inside a multiple assignment,
+
Lua ensures all reads get the value of the variable
+
before the assignment.
Thus the code

<pre>
@@ -1657,6 +1661,14 @@ cyclically permutes the values of <code>x</code>, <code>y</code>, and <code>z</c


<p>
+
Note that this guarantee covers only accesses
+
syntactically inside the assignment statement.
+
If a function or a metamethod called during the assignment
+
changes the value of a variable,
+
Lua gives no guarantees about the order of that access.
+

+

+
<p>
An assignment to a global name <code>x = val</code>
is equivalent to the assignment
<code>_ENV.x = val</code> (see <a href="#2.2">&sect;2.2</a>).
@@ -1992,16 +2004,8 @@ they are closed in the reverse order that they were declared.
If there is any error while running a closing method,
that error is handled like an error in the regular code
where the variable was defined.
-
However, Lua may call the method one more time.
-

-

-
<p>
After an error,
the other pending closing methods will still be called.
-
Errors in these methods
-
interrupt the respective method and generate a warning,
-
but are otherwise ignored;
-
the error reported is only the original one.


<p>
@@ -2419,16 +2423,21 @@ character is one byte.)
<p>
The length operator applied on a table
returns a border in that table.
-
A <em>border</em> in a table <code>t</code> is any natural number
+
A <em>border</em> in a table <code>t</code> is any non-negative integer
that satisfies the following condition:

<pre>
-
     (border == 0 or t[border] ~= nil) and t[border + 1] == nil
+
     (border == 0 or t[border] ~= nil) and
+
     (t[border + 1] == nil or border == math.maxinteger)
</pre><p>
In words,
-
a border is any (natural) index present in the table
-
that is followed by an absent index
-
(or zero, when index 1 is absent).
+
a border is any positive integer index present in the table
+
that is followed by an absent index,
+
plus two limit cases:
+
zero, when index 1 is absent;
+
and the maximum value for an integer, when that index is present.
+
Note that keys that are not positive integers
+
do not interfere with borders.


<p>
@@ -2439,12 +2448,9 @@ The table <code>{10, 20, 30, nil, 50}</code> has two borders (3 and 5),
and therefore it is not a sequence.
(The <b>nil</b> at index 4 is called a <em>hole</em>.)
The table <code>{nil, 20, 30, nil, nil, 60, nil}</code>
-
has three borders (0, 3, and 6) and three holes
-
(at indices 1, 4, and 5),
+
has three borders (0, 3, and 6),
so it is not a sequence, too.
The table <code>{}</code> is a sequence with border 0.
-
Note that non-natural keys do not interfere
-
with whether a table is a sequence.


<p>
@@ -2462,7 +2468,7 @@ the memory addresses of its non-numeric keys.)
<p>
The computation of the length of a table
has a guaranteed worst time of <em>O(log n)</em>,
-
where <em>n</em> is the largest natural key in the table.
+
where <em>n</em> is the largest integer key in the table.


<p>
@@ -3763,6 +3769,29 @@ will probably need to close states as soon as they are not needed.



+
<hr><h3><a name="lua_closeslot"><code>lua_closeslot</code></a></h3><p>
+
<span class="apii">[-0, +0, <em>e</em>]</span>
+
<pre>void lua_closeslot (lua_State *L, int index);</pre>
+

+
<p>
+
Close the to-be-closed slot at the given index and set its value to <b>nil</b>.
+
The index must be the last index previously marked to be closed
+
(see <a href="#lua_toclose"><code>lua_toclose</code></a>) that is still active (that is, not closed yet).
+

+

+
<p>
+
A <code>__close</code> metamethod cannot yield
+
when called through this function.
+

+

+
<p>
+
(Exceptionally, this function was introduced in release 5.4.3.
+
It is not present in previous 5.4 releases.)
+

+

+

+

+

<hr><h3><a name="lua_compare"><code>lua_compare</code></a></h3><p>
<span class="apii">[-0, +0, <em>e</em>]</span>
<pre>int lua_compare (lua_State *L, int index1, int index2, int op);</pre>
@@ -3959,6 +3988,10 @@ For more details about these options,
see <a href="#pdf-collectgarbage"><code>collectgarbage</code></a>.


+
<p>
+
This function should not be called by a finalizer.
+

+




@@ -4657,11 +4690,7 @@ except that it allows the called function to yield (see <a href="#4.5">&sect;4.5

<p>
Pops <code>n</code> elements from the stack.
-

-

-
<p>
-
This function can run arbitrary code when removing an index
-
marked as to-be-closed from the stack.
+
It is implemented as a macro over <a href="#lua_settop"><code>lua_settop</code></a>.



@@ -5140,10 +5169,12 @@ and then pops the top element.
Resets a thread, cleaning its call stack and closing all pending
to-be-closed variables.
Returns a status code:
-
<a href="#pdf-LUA_OK"><code>LUA_OK</code></a> for no errors in closing methods,
+
<a href="#pdf-LUA_OK"><code>LUA_OK</code></a> for no errors in the thread
+
(either the original error that stopped the thread or
+
errors in closing methods),
or an error status otherwise.
In case of error,
-
leaves the error object on the top of the stack,
+
leaves the error object on the top of the stack.



@@ -5466,28 +5497,24 @@ otherwise, returns <code>NULL</code>.

<p>
Marks the given index in the stack as a
-
to-be-closed "variable" (see <a href="#3.3.8">&sect;3.3.8</a>).
+
to-be-closed slot (see <a href="#3.3.8">&sect;3.3.8</a>).
Like a to-be-closed variable in Lua,
-
the value at that index in the stack will be closed
+
the value at that slot in the stack will be closed
when it goes out of scope.
Here, in the context of a C function,
to go out of scope means that the running function returns to Lua,
-
there is an error,
-
or the index is removed from the stack through
-
<a href="#lua_settop"><code>lua_settop</code></a> or <a href="#lua_pop"><code>lua_pop</code></a>.
-
An index marked as to-be-closed should not be removed from the stack
-
by any other function in the API except <a href="#lua_settop"><code>lua_settop</code></a> or <a href="#lua_pop"><code>lua_pop</code></a>.
+
or there is an error,
+
or the slot is removed from the stack through
+
<a href="#lua_settop"><code>lua_settop</code></a> or <a href="#lua_pop"><code>lua_pop</code></a>,
+
or there is a call to <a href="#lua_closeslot"><code>lua_closeslot</code></a>.
+
A slot marked as to-be-closed should not be removed from the stack
+
by any other function in the API except <a href="#lua_settop"><code>lua_settop</code></a> or <a href="#lua_pop"><code>lua_pop</code></a>,
+
unless previously deactivated by <a href="#lua_closeslot"><code>lua_closeslot</code></a>.


<p>
This function should not be called for an index
-
that is equal to or below an active to-be-closed index.
-

-

-
<p>
-
In the case of an out-of-memory error,
-
the value in the given index is immediately closed,
-
as if it was already marked.
+
that is equal to or below an active to-be-closed slot.


<p>
@@ -5495,7 +5522,7 @@ Note that, both in case of errors and of a regular return,
by the time the <code>__close</code> metamethod runs,
the C&nbsp;stack was already unwound,
so that any automatic C&nbsp;variable declared in the calling function
-
will be out of scope.
+
(e.g., a buffer) will be out of scope.



@@ -5919,7 +5946,10 @@ information about a function or an activation record.
<a href="#lua_getstack"><code>lua_getstack</code></a> fills only the private part
of this structure, for later use.
To fill the other fields of <a href="#lua_Debug"><code>lua_Debug</code></a> with useful information,
-
you must call <a href="#lua_getinfo"><code>lua_getinfo</code></a>.
+
you must call <a href="#lua_getinfo"><code>lua_getinfo</code></a> with an appropriate parameter.
+
(Specifically, to get a field,
+
you must add the letter between parentheses in the field's comment
+
to the parameter <code>what</code> of <a href="#lua_getinfo"><code>lua_getinfo</code></a>.)


<p>
@@ -6096,11 +6126,25 @@ you can write the following code:
<p>
Each character in the string <code>what</code>
selects some fields of the structure <code>ar</code> to be filled or
-
a value to be pushed on the stack:
+
a value to be pushed on the stack.
+
(These characters are also documented in the declaration of
+
the structure <a href="#lua_Debug"><code>lua_Debug</code></a>,
+
between parentheses in the comments following each field.)

<ul>

-
<li><b>'<code>n</code>': </b> fills in the field <code>name</code> and <code>namewhat</code>;
+
<li><b>'<code>f</code>': </b>
+
pushes onto the stack the function that is
+
running at the given level;
+
</li>
+

+
<li><b>'<code>l</code>': </b> fills in the field <code>currentline</code>;
+
</li>
+

+
<li><b>'<code>n</code>': </b> fills in the fields <code>name</code> and <code>namewhat</code>;
+
</li>
+

+
<li><b>'<code>r</code>': </b> fills in the fields <code>ftransfer</code> and <code>ntransfer</code>;
</li>

<li><b>'<code>S</code>': </b>
@@ -6108,9 +6152,6 @@ fills in the fields <code>source</code>, <code>short_src</code>,
<code>linedefined</code>, <code>lastlinedefined</code>, and <code>what</code>;
</li>

-
<li><b>'<code>l</code>': </b> fills in the field <code>currentline</code>;
-
</li>
-

<li><b>'<code>t</code>': </b> fills in the field <code>istailcall</code>;
</li>

@@ -6118,25 +6159,13 @@ fills in the fields <code>source</code>, <code>short_src</code>,
<code>nups</code>, <code>nparams</code>, and <code>isvararg</code>;
</li>

-
<li><b>'<code>f</code>': </b>
-
pushes onto the stack the function that is
-
running at the given level;
-
</li>
-

<li><b>'<code>L</code>': </b>
-
pushes onto the stack a table whose indices are the
-
numbers of the lines that are valid on the function.
-
(A <em>valid line</em> is a line with some associated code,
-
that is, a line where you can put a break point.
-
Non-valid lines include empty lines and comments.)
-

-

-
<p>
+
pushes onto the stack a table whose indices are
+
the lines on the function with some associated code,
+
that is, the lines where you can put a break point.
+
(Lines with no code include empty lines and comments.)
If this option is given together with option '<code>f</code>',
its table is pushed after the function.
-

-

-
<p>
This is the only option that can raise a memory error.
</li>

@@ -6494,7 +6523,7 @@ Adds the byte <code>c</code> to the buffer <code>B</code>


<hr><h3><a name="luaL_addgsub"><code>luaL_addgsub</code></a></h3><p>
-
<span class="apii">[-0, +0, <em>m</em>]</span>
+
<span class="apii">[-?, +?, <em>m</em>]</span>
<pre>const void luaL_addgsub (luaL_Buffer *B, const char *s,
                         const char *p, const char *r);</pre>

@@ -6548,7 +6577,7 @@ to the buffer <code>B</code>


<hr><h3><a name="luaL_addvalue"><code>luaL_addvalue</code></a></h3><p>
-
<span class="apii">[-1, +?, <em>m</em>]</span>
+
<span class="apii">[-?, +?, <em>m</em>]</span>
<pre>void luaL_addvalue (luaL_Buffer *B);</pre>

<p>
@@ -6702,7 +6731,7 @@ Note that any addition to the buffer may invalidate this address.


<hr><h3><a name="luaL_buffinit"><code>luaL_buffinit</code></a></h3><p>
-
<span class="apii">[-0, +0, &ndash;]</span>
+
<span class="apii">[-0, +?, &ndash;]</span>
<pre>void luaL_buffinit (lua_State *L, luaL_Buffer *B);</pre>

<p>
@@ -6740,7 +6769,7 @@ Equivalent to the sequence


<hr><h3><a name="luaL_buffsub"><code>luaL_buffsub</code></a></h3><p>
-
<span class="apii">[-0, +0, &ndash;]</span>
+
<span class="apii">[-?, +?, &ndash;]</span>
<pre>void luaL_buffsub (luaL_Buffer *B, int n);</pre>

<p>
@@ -7543,6 +7572,11 @@ on top of the library table.
These values are popped from the stack after the registration.


+
<p>
+
A function with a <code>NULL</code> value represents a placeholder,
+
which is filled with <b>false</b>.
+

+




@@ -7905,6 +7939,10 @@ See <a href="#2.5">&sect;2.5</a> for more details about garbage collection
and some of these options.


+
<p>
+
This function should not be called by a finalizer.
+

+



<p>
@@ -7922,7 +7960,7 @@ to its caller.

<p>
<hr><h3><a name="pdf-error"><code>error (message [, level])</code></a></h3>
-
Raises an error (see <a href="#2.3">&sect;2.3</a>) with @{message} as the error object.
+
Raises an error (see <a href="#2.3">&sect;2.3</a>) with <code>message</code> as the error object.
This function never returns.


@@ -8097,9 +8135,8 @@ use a numerical <b>for</b>.)


<p>
-
The behavior of <code>next</code> is undefined if,
-
during the traversal,
-
you assign any value to a non-existent field in the table.
+
You should not assign any value to a non-existent field in a table
+
during its traversal.
You may however modify existing fields.
In particular, you may set existing fields to nil.

@@ -8145,7 +8182,7 @@ This means that any error inside&nbsp;<code>f</code> is not propagated;
instead, <code>pcall</code> catches the error
and returns a status code.
Its first result is the status code (a boolean),
-
which is true if the call succeeds without errors.
+
which is <b>true</b> if the call succeeds without errors.
In such case, <code>pcall</code> also returns all results from the call,
after this first result.
In case of any error, <code>pcall</code> returns <b>false</b> plus the error object.
@@ -8398,7 +8435,9 @@ that is,
closes all its pending to-be-closed variables
and puts the coroutine in a dead state.
The given coroutine must be dead or suspended.
-
In case of error closing some variable,
+
In case of error
+
(either the original error that stopped the coroutine or
+
errors in closing methods),
returns <b>false</b> plus the error object;
otherwise returns <b>true</b>.

@@ -8423,7 +8462,7 @@ an object with type <code>"thread"</code>.


<p>
-
Returns true when the coroutine <code>co</code> can yield.
+
Returns <b>true</b> when the coroutine <code>co</code> can yield.
The default for <code>co</code> is the running coroutine.


@@ -8467,7 +8506,7 @@ If there is any error,

<p>
Returns the running coroutine plus a boolean,
-
true when the running coroutine is the main one.
+
<b>true</b> when the running coroutine is the main one.



@@ -9005,7 +9044,7 @@ otherwise, it returns <b>fail</b>.
A third, optional numeric argument <code>init</code> specifies
where to start the search;
its default value is&nbsp;1 and can be negative.
-
A value of <b>true</b> as a fourth, optional argument <code>plain</code>
+
A <b>true</b> as a fourth, optional argument <code>plain</code>
turns off the pattern matching facilities,
so the function does a plain "find substring" operation,
with no characters in <code>pattern</code> being considered magic.
@@ -9030,8 +9069,10 @@ following the description given in its first argument,
which must be a string.
The format string follows the same rules as the ISO&nbsp;C function <code>sprintf</code>.
The only differences are that the conversion specifiers and modifiers
-
<code>*</code>, <code>h</code>, <code>L</code>, <code>l</code>, and <code>n</code> are not supported
+
<code>F</code>, <code>n</code>, <code>*</code>, <code>h</code>, <code>L</code>, and <code>l</code> are not supported
and that there is an extra specifier, <code>q</code>.
+
Both width and precision, when present,
+
are limited to two digits.


<p>
@@ -9055,7 +9096,7 @@ may produce the string:
     "a string with \"quotes\" and \
      new line"
</pre><p>
-
This specifier does not support modifiers (flags, width, length).
+
This specifier does not support modifiers (flags, width, precision).


<p>
@@ -9960,23 +10001,23 @@ from <code>list[1]</code> to <code>list[#list]</code>.
If <code>comp</code> is given,
then it must be a function that receives two list elements
and returns true when the first element must come
-
before the second in the final order
-
(so that, after the sort,
-
<code>i &lt; j</code> implies <code>not comp(list[j],list[i])</code>).
+
before the second in the final order,
+
so that, after the sort,
+
<code>i &lt;= j</code> implies <code>not comp(list[j],list[i])</code>.
If <code>comp</code> is not given,
then the standard Lua operator <code>&lt;</code> is used instead.


<p>
-
Note that the <code>comp</code> function must define
-
a strict partial order over the elements in the list;
-
that is, it must be asymmetric and transitive.
-
Otherwise, no valid sort may be possible.
+
The <code>comp</code> function must define a consistent order;
+
more formally, the function must define a strict weak order.
+
(A weak order is similar to a total order,
+
but it can equate different elements for comparison purposes.)


<p>
The sort algorithm is not stable:
-
elements considered equal by the given order
+
Different elements considered equal by the given order
may have their relative positions changed by the sort.


@@ -10346,7 +10387,7 @@ or <b>fail</b> if <code>x</code> is not a number.

<p>
Returns a boolean,
-
true if and only if integer <code>m</code> is below integer <code>n</code> when
+
<b>true</b> if and only if integer <code>m</code> is below integer <code>n</code> when
they are compared as unsigned integers.


@@ -11908,10 +11949,10 @@ and LiteralString, see <a href="#3.1">&sect;3.1</a>.)

<P CLASS="footer">
Last update:
-
Fri Nov 13 15:35:22 UTC 2020
+
Thu Jan 13 11:33:16 UTC 2022
</P>
<!--
-
Last change: revised for Lua 5.4.2
+
Last change: revised for Lua 5.4.4
-->

</body></html>
modified external/lua/doc/readme.html
@@ -110,7 +110,7 @@ Here are the details.
<OL>
<LI>
Open a terminal window and move to
-
the top-level directory, which is named <TT>lua-5.4.2</TT>.
+
the top-level directory, which is named <TT>lua-5.4.4</TT>.
The <TT>Makefile</TT> there controls both the build process and the installation process.
<P>
<LI>
@@ -303,7 +303,7 @@ For details, see
<A HREF="http://www.lua.org/license.html">this</A>.

<BLOCKQUOTE STYLE="padding-bottom: 0em">
-
Copyright &copy; 1994&ndash;2020 Lua.org, PUC-Rio.
+
Copyright &copy; 1994&ndash;2022 Lua.org, PUC-Rio.

<P>
Permission is hereby granted, free of charge, to any person obtaining a copy
@@ -330,10 +330,10 @@ THE SOFTWARE.

<P CLASS="footer">
Last update:
-
Tue Nov 10 20:55:28 UTC 2020
+
Mon Jan  3 09:54:18 UTC 2022
</P>
<!--
-
Last change: revised for Lua 5.4.2
+
Last change: revised for Lua 5.4.4
-->

</BODY>
modified external/lua/src/Makefile
@@ -67,7 +67,7 @@ $(LUAC_T): $(LUAC_O) $(LUA_A)
	$(CC) -o $@ $(LDFLAGS) $(LUAC_O) $(LUA_A) $(LIBS)

test:
-
	./lua -v
+
	./$(LUA_T) -v

clean:
	$(RM) $(ALL_T) $(ALL_O)
@@ -79,7 +79,7 @@ echo:
	@echo "PLAT= $(PLAT)"
	@echo "CC= $(CC)"
	@echo "CFLAGS= $(CFLAGS)"
-
	@echo "LDFLAGS= $(SYSLDFLAGS)"
+
	@echo "LDFLAGS= $(LDFLAGS)"
	@echo "LIBS= $(LIBS)"
	@echo "AR= $(AR)"
	@echo "RANLIB= $(RANLIB)"
@@ -108,6 +108,8 @@ c89:
	$(MAKE) $(ALL) SYSCFLAGS="-DLUA_USE_C89" CC="gcc -std=c89"
	@echo ''
	@echo '*** C89 does not guarantee 64-bit integers for Lua.'
+
	@echo '*** Make sure to compile all external Lua libraries'
+
	@echo '*** with LUA_USE_C89 to ensure consistency'
	@echo ''

FreeBSD NetBSD OpenBSD freebsd:
modified external/lua/src/lapi.c
@@ -39,7 +39,7 @@ const char lua_ident[] =


/*
-
** Test for a valid index.
+
** Test for a valid index (one that is not the 'nilvalue').
** '!ttisnil(o)' implies 'o != &G(L)->nilvalue', so it is not needed.
** However, it covers the most common cases in a faster way.
*/
@@ -53,6 +53,10 @@ const char lua_ident[] =
#define isupvalue(i)		((i) < LUA_REGISTRYINDEX)


+
/*
+
** Convert an acceptable index to a pointer to its respective value.
+
** Non-valid indices return the special nil value 'G(L)->nilvalue'.
+
*/
static TValue *index2value (lua_State *L, int idx) {
  CallInfo *ci = L->ci;
  if (idx > 0) {
@@ -70,21 +74,28 @@ static TValue *index2value (lua_State *L, int idx) {
  else {  /* upvalues */
    idx = LUA_REGISTRYINDEX - idx;
    api_check(L, idx <= MAXUPVAL + 1, "upvalue index too large");
-
    if (ttislcf(s2v(ci->func)))  /* light C function? */
-
      return &G(L)->nilvalue;  /* it has no upvalues */
-
    else {
+
    if (ttisCclosure(s2v(ci->func))) {  /* C closure? */
      CClosure *func = clCvalue(s2v(ci->func));
-
      return (idx <= func->nupvalues) ? &func->upvalue[idx-1] : &G(L)->nilvalue;
+
      return (idx <= func->nupvalues) ? &func->upvalue[idx-1]
+
                                      : &G(L)->nilvalue;
+
    }
+
    else {  /* light C function or Lua function (through a hook)?) */
+
      api_check(L, ttislcf(s2v(ci->func)), "caller not a C function");
+
      return &G(L)->nilvalue;  /* no upvalues */
    }
  }
}


-
static StkId index2stack (lua_State *L, int idx) {
+

+
/*
+
** Convert a valid actual index (not a pseudo-index) to its address.
+
*/
+
l_sinline StkId index2stack (lua_State *L, int idx) {
  CallInfo *ci = L->ci;
  if (idx > 0) {
    StkId o = ci->func + idx;
-
    api_check(L, o < L->top, "unacceptable index");
+
    api_check(L, o < L->top, "invalid index");
    return o;
  }
  else {    /* non-positive index */
@@ -172,7 +183,7 @@ LUA_API int lua_gettop (lua_State *L) {

LUA_API void lua_settop (lua_State *L, int idx) {
  CallInfo *ci;
-
  StkId func;
+
  StkId func, newtop;
  ptrdiff_t diff;  /* difference for new top */
  lua_lock(L);
  ci = L->ci;
@@ -187,9 +198,26 @@ LUA_API void lua_settop (lua_State *L, int idx) {
    api_check(L, -(idx+1) <= (L->top - (func + 1)), "invalid new top");
    diff = idx + 1;  /* will "subtract" index (as it is negative) */
  }
-
  if (diff < 0 && hastocloseCfunc(ci->nresults))
-
    luaF_close(L, L->top + diff, LUA_OK);
-
  L->top += diff;  /* correct top only after closing any upvalue */
+
  api_check(L, L->tbclist < L->top, "previous pop of an unclosed slot");
+
  newtop = L->top + diff;
+
  if (diff < 0 && L->tbclist >= newtop) {
+
    lua_assert(hastocloseCfunc(ci->nresults));
+
    luaF_close(L, newtop, CLOSEKTOP, 0);
+
  }
+
  L->top = newtop;  /* correct top only after closing any upvalue */
+
  lua_unlock(L);
+
}
+

+

+
LUA_API void lua_closeslot (lua_State *L, int idx) {
+
  StkId level;
+
  lua_lock(L);
+
  level = index2stack(L, idx);
+
  api_check(L, hastocloseCfunc(L->ci->nresults) && L->tbclist == level,
+
     "no variable to close at given level");
+
  luaF_close(L, level, CLOSEKTOP, 0);
+
  level = index2stack(L, idx);  /* stack may be moved */
+
  setnilvalue(s2v(level));
  lua_unlock(L);
}

@@ -200,7 +228,7 @@ LUA_API void lua_settop (lua_State *L, int idx) {
** Note that we move(copy) only the value inside the stack.
** (We do not move additional fields that may exist.)
*/
-
static void reverse (lua_State *L, StkId from, StkId to) {
+
l_sinline void reverse (lua_State *L, StkId from, StkId to) {
  for (; from < to; from++, to--) {
    TValue temp;
    setobj(L, &temp, s2v(from));
@@ -420,7 +448,7 @@ LUA_API lua_CFunction lua_tocfunction (lua_State *L, int idx) {
}


-
static void *touserdata (const TValue *o) {
+
l_sinline void *touserdata (const TValue *o) {
  switch (ttype(o)) {
    case LUA_TUSERDATA: return getudatamem(uvalue(o));
    case LUA_TLIGHTUSERDATA: return pvalue(o);
@@ -612,7 +640,7 @@ LUA_API int lua_pushthread (lua_State *L) {
*/


-
static int auxgetstr (lua_State *L, const TValue *t, const char *k) {
+
l_sinline int auxgetstr (lua_State *L, const TValue *t, const char *k) {
  const TValue *slot;
  TString *str = luaS_new(L, k);
  if (luaV_fastget(L, t, str, slot, luaH_getstr)) {
@@ -629,11 +657,21 @@ static int auxgetstr (lua_State *L, const TValue *t, const char *k) {
}


+
/*
+
** Get the global table in the registry. Since all predefined
+
** indices in the registry were inserted right when the registry
+
** was created and never removed, they must always be in the array
+
** part of the registry.
+
*/
+
#define getGtable(L)  \
+
	(&hvalue(&G(L)->l_registry)->array[LUA_RIDX_GLOBALS - 1])
+

+

LUA_API int lua_getglobal (lua_State *L, const char *name) {
-
  Table *reg;
+
  const TValue *G;
  lua_lock(L);
-
  reg = hvalue(&G(L)->l_registry);
-
  return auxgetstr(L, luaH_getint(reg, LUA_RIDX_GLOBALS), name);
+
  G = getGtable(L);
+
  return auxgetstr(L, G, name);
}


@@ -677,7 +715,7 @@ LUA_API int lua_geti (lua_State *L, int idx, lua_Integer n) {
}


-
static int finishrawget (lua_State *L, const TValue *val) {
+
l_sinline int finishrawget (lua_State *L, const TValue *val) {
  if (isempty(val))  /* avoid copying empty items to the stack */
    setnilvalue(s2v(L->top));
  else
@@ -811,10 +849,10 @@ static void auxsetstr (lua_State *L, const TValue *t, const char *k) {


LUA_API void lua_setglobal (lua_State *L, const char *name) {
-
  Table *reg;
+
  const TValue *G;
  lua_lock(L);  /* unlock done in 'auxsetstr' */
-
  reg = hvalue(&G(L)->l_registry);
-
  auxsetstr(L, luaH_getint(reg, LUA_RIDX_GLOBALS), name);
+
  G = getGtable(L);
+
  auxsetstr(L, G, name);
}


@@ -861,12 +899,10 @@ LUA_API void lua_seti (lua_State *L, int idx, lua_Integer n) {

static void aux_rawset (lua_State *L, int idx, TValue *key, int n) {
  Table *t;
-
  TValue *slot;
  lua_lock(L);
  api_checknelems(L, n);
  t = gettable(L, idx);
-
  slot = luaH_set(L, t, key);
-
  setobj2t(L, slot, s2v(L->top - 1));
+
  luaH_set(L, t, key, s2v(L->top - 1));
  invalidateTMcache(t);
  luaC_barrierback(L, obj2gco(t), s2v(L->top - 1));
  L->top -= n;
@@ -1063,8 +1099,7 @@ LUA_API int lua_load (lua_State *L, lua_Reader reader, void *data,
    LClosure *f = clLvalue(s2v(L->top - 1));  /* get newly created function */
    if (f->nupvalues >= 1) {  /* does it have an upvalue? */
      /* get global table from registry */
-
      Table *reg = hvalue(&G(L)->l_registry);
-
      const TValue *gt = luaH_getint(reg, LUA_RIDX_GLOBALS);
+
      const TValue *gt = getGtable(L);
      /* set global table as 1st upvalue of 'f' (may be LUA_ENV) */
      setobj(L, f->upvals[0]->v, gt);
      luaC_barrier(L, f->upvals[0], gt);
@@ -1101,18 +1136,19 @@ LUA_API int lua_status (lua_State *L) {
LUA_API int lua_gc (lua_State *L, int what, ...) {
  va_list argp;
  int res = 0;
-
  global_State *g;
+
  global_State *g = G(L);
+
  if (g->gcstp & GCSTPGC)  /* internal stop? */
+
    return -1;  /* all options are invalid when stopped */
  lua_lock(L);
-
  g = G(L);
  va_start(argp, what);
  switch (what) {
    case LUA_GCSTOP: {
-
      g->gcrunning = 0;
+
      g->gcstp = GCSTPUSR;  /* stopped by the user */
      break;
    }
    case LUA_GCRESTART: {
      luaE_setdebt(g, 0);
-
      g->gcrunning = 1;
+
      g->gcstp = 0;  /* (GCSTPGC must be already zero here) */
      break;
    }
    case LUA_GCCOLLECT: {
@@ -1131,8 +1167,8 @@ LUA_API int lua_gc (lua_State *L, int what, ...) {
    case LUA_GCSTEP: {
      int data = va_arg(argp, int);
      l_mem debt = 1;  /* =1 to signal that it did an actual step */
-
      lu_byte oldrunning = g->gcrunning;
-
      g->gcrunning = 1;  /* allow GC to run */
+
      lu_byte oldstp = g->gcstp;
+
      g->gcstp = 0;  /* allow GC to run (GCSTPGC must be zero here) */
      if (data == 0) {
        luaE_setdebt(g, 0);  /* do a basic step */
        luaC_step(L);
@@ -1142,7 +1178,7 @@ LUA_API int lua_gc (lua_State *L, int what, ...) {
        luaE_setdebt(g, debt);
        luaC_checkGC(L);
      }
-
      g->gcrunning = oldrunning;  /* restore previous state */
+
      g->gcstp = oldstp;  /* restore previous state */
      if (debt > 0 && g->gcstate == GCSpause)  /* end of cycle? */
        res = 1;  /* signal it */
      break;
@@ -1160,7 +1196,7 @@ LUA_API int lua_gc (lua_State *L, int what, ...) {
      break;
    }
    case LUA_GCISRUNNING: {
-
      res = g->gcrunning;
+
      res = gcrunning(g);
      break;
    }
    case LUA_GCGEN: {
@@ -1240,8 +1276,7 @@ LUA_API void lua_toclose (lua_State *L, int idx) {
  lua_lock(L);
  o = index2stack(L, idx);
  nresults = L->ci->nresults;
-
  api_check(L, L->openupval == NULL || uplevel(L->openupval) <= o,
-
               "marked index below or equal new one");
+
  api_check(L, L->tbclist < o, "given index below or equal a marked one");
  luaF_newtbcupval(L, o);  /* create new to-be-closed upvalue */
  if (!hastocloseCfunc(nresults))  /* function not marked yet? */
    L->ci->nresults = codeNresults(nresults);  /* mark it */
modified external/lua/src/lapi.h
@@ -42,6 +42,8 @@

#define hastocloseCfunc(n)	((n) < LUA_MULTRET)

+
/* Map [-1, inf) (range of 'nresults') into (-inf, -2] */
#define codeNresults(n)		(-(n) - 3)
+
#define decodeNresults(n)	(-(n) - 3)

#endif
modified external/lua/src/lauxlib.c
@@ -190,7 +190,7 @@ LUALIB_API int luaL_argerror (lua_State *L, int arg, const char *extramsg) {
}


-
int luaL_typeerror (lua_State *L, int arg, const char *tname) {
+
LUALIB_API int luaL_typeerror (lua_State *L, int arg, const char *tname) {
  const char *msg;
  const char *typearg;  /* name for the type of the actual argument */
  if (luaL_getmetafield(L, arg, "__name") == LUA_TSTRING)
@@ -378,7 +378,7 @@ LUALIB_API int luaL_checkoption (lua_State *L, int arg, const char *def,
** but without 'msg'.)
*/
LUALIB_API void luaL_checkstack (lua_State *L, int space, const char *msg) {
-
  if (!lua_checkstack(L, space)) {
+
  if (l_unlikely(!lua_checkstack(L, space))) {
    if (msg)
      luaL_error(L, "stack overflow (%s)", msg);
    else
@@ -388,20 +388,20 @@ LUALIB_API void luaL_checkstack (lua_State *L, int space, const char *msg) {


LUALIB_API void luaL_checktype (lua_State *L, int arg, int t) {
-
  if (lua_type(L, arg) != t)
+
  if (l_unlikely(lua_type(L, arg) != t))
    tag_error(L, arg, t);
}


LUALIB_API void luaL_checkany (lua_State *L, int arg) {
-
  if (lua_type(L, arg) == LUA_TNONE)
+
  if (l_unlikely(lua_type(L, arg) == LUA_TNONE))
    luaL_argerror(L, arg, "value expected");
}


LUALIB_API const char *luaL_checklstring (lua_State *L, int arg, size_t *len) {
  const char *s = lua_tolstring(L, arg, len);
-
  if (!s) tag_error(L, arg, LUA_TSTRING);
+
  if (l_unlikely(!s)) tag_error(L, arg, LUA_TSTRING);
  return s;
}

@@ -420,7 +420,7 @@ LUALIB_API const char *luaL_optlstring (lua_State *L, int arg,
LUALIB_API lua_Number luaL_checknumber (lua_State *L, int arg) {
  int isnum;
  lua_Number d = lua_tonumberx(L, arg, &isnum);
-
  if (!isnum)
+
  if (l_unlikely(!isnum))
    tag_error(L, arg, LUA_TNUMBER);
  return d;
}
@@ -442,7 +442,7 @@ static void interror (lua_State *L, int arg) {
LUALIB_API lua_Integer luaL_checkinteger (lua_State *L, int arg) {
  int isnum;
  lua_Integer d = lua_tointegerx(L, arg, &isnum);
-
  if (!isnum) {
+
  if (l_unlikely(!isnum)) {
    interror(L, arg);
  }
  return d;
@@ -475,7 +475,7 @@ static void *resizebox (lua_State *L, int idx, size_t newsize) {
  lua_Alloc allocf = lua_getallocf(L, &ud);
  UBox *box = (UBox *)lua_touserdata(L, idx);
  void *temp = allocf(ud, box->box, box->bsize, newsize);
-
  if (temp == NULL && newsize > 0) {  /* allocation error? */
+
  if (l_unlikely(temp == NULL && newsize > 0)) {  /* allocation error? */
    lua_pushliteral(L, "not enough memory");
    lua_error(L);  /* raise a memory error */
  }
@@ -516,12 +516,21 @@ static void newbox (lua_State *L) {


/*
+
** Whenever buffer is accessed, slot 'idx' must either be a box (which
+
** cannot be NULL) or it is a placeholder for the buffer.
+
*/
+
#define checkbufferlevel(B,idx)  \
+
  lua_assert(buffonstack(B) ? lua_touserdata(B->L, idx) != NULL  \
+
                            : lua_touserdata(B->L, idx) == (void*)B)
+

+

+
/*
** Compute new size for buffer 'B', enough to accommodate extra 'sz'
** bytes.
*/
static size_t newbuffsize (luaL_Buffer *B, size_t sz) {
  size_t newsize = B->size * 2;  /* double buffer size */
-
  if (MAX_SIZET - sz < B->n)  /* overflow in (B->n + sz)? */
+
  if (l_unlikely(MAX_SIZET - sz < B->n))  /* overflow in (B->n + sz)? */
    return luaL_error(B->L, "buffer too large");
  if (newsize < B->n + sz)  /* double is not big enough? */
    newsize = B->n + sz;
@@ -531,10 +540,11 @@ static size_t newbuffsize (luaL_Buffer *B, size_t sz) {

/*
** Returns a pointer to a free area with at least 'sz' bytes in buffer
-
** 'B'. 'boxidx' is the relative position in the stack where the
-
** buffer's box is or should be.
+
** 'B'. 'boxidx' is the relative position in the stack where is the
+
** buffer's box or its placeholder.
*/
static char *prepbuffsize (luaL_Buffer *B, size_t sz, int boxidx) {
+
  checkbufferlevel(B, boxidx);
  if (B->size - B->n >= sz)  /* enough space? */
    return B->b + B->n;
  else {
@@ -545,10 +555,9 @@ static char *prepbuffsize (luaL_Buffer *B, size_t sz, int boxidx) {
    if (buffonstack(B))  /* buffer already has a box? */
      newbuff = (char *)resizebox(L, boxidx, newsize);  /* resize it */
    else {  /* no box yet */
-
      lua_pushnil(L);  /* reserve slot for final result */
+
      lua_remove(L, boxidx);  /* remove placeholder */
      newbox(L);  /* create a new box */
-
      /* move box (and slot) to its intended position */
-
      lua_rotate(L, boxidx - 1, 2);
+
      lua_insert(L, boxidx);  /* move box to its intended position */
      lua_toclose(L, boxidx);
      newbuff = (char *)resizebox(L, boxidx, newsize);
      memcpy(newbuff, B->b, B->n * sizeof(char));  /* copy original content */
@@ -583,11 +592,11 @@ LUALIB_API void luaL_addstring (luaL_Buffer *B, const char *s) {

LUALIB_API void luaL_pushresult (luaL_Buffer *B) {
  lua_State *L = B->L;
+
  checkbufferlevel(B, -1);
  lua_pushlstring(L, B->b, B->n);
-
  if (buffonstack(B)) {
-
    lua_copy(L, -1, -3);  /* move string to reserved slot */
-
    lua_pop(L, 2);  /* pop string and box (closing the box) */
-
  }
+
  if (buffonstack(B))
+
    lua_closeslot(L, -2);  /* close the box */
+
  lua_remove(L, -2);  /* remove box or placeholder from the stack */
}


@@ -622,6 +631,7 @@ LUALIB_API void luaL_buffinit (lua_State *L, luaL_Buffer *B) {
  B->b = B->init.b;
  B->n = 0;
  B->size = LUAL_BUFFERSIZE;
+
  lua_pushlightuserdata(L, (void*)B);  /* push placeholder */
}


@@ -639,10 +649,14 @@ LUALIB_API char *luaL_buffinitsize (lua_State *L, luaL_Buffer *B, size_t sz) {
** =======================================================
*/

-
/* index of free-list header */
-
#define freelist	0
-

+
/* index of free-list header (after the predefined values) */
+
#define freelist	(LUA_RIDX_LAST + 1)

+
/*
+
** The previously freed references form a linked list:
+
** t[freelist] is the index of a first free index, or zero if list is
+
** empty; t[t[freelist]] is the index of the second element; etc.
+
*/
LUALIB_API int luaL_ref (lua_State *L, int t) {
  int ref;
  if (lua_isnil(L, -1)) {
@@ -650,9 +664,16 @@ LUALIB_API int luaL_ref (lua_State *L, int t) {
    return LUA_REFNIL;  /* 'nil' has a unique fixed reference */
  }
  t = lua_absindex(L, t);
-
  lua_rawgeti(L, t, freelist);  /* get first free element */
-
  ref = (int)lua_tointeger(L, -1);  /* ref = t[freelist] */
-
  lua_pop(L, 1);  /* remove it from stack */
+
  if (lua_rawgeti(L, t, freelist) == LUA_TNIL) {  /* first access? */
+
    ref = 0;  /* list is empty */
+
    lua_pushinteger(L, 0);  /* initialize as an empty list */
+
    lua_rawseti(L, t, freelist);  /* ref = t[freelist] = 0 */
+
  }
+
  else {  /* already initialized */
+
    lua_assert(lua_isinteger(L, -1));
+
    ref = (int)lua_tointeger(L, -1);  /* ref = t[freelist] */
+
  }
+
  lua_pop(L, 1);  /* remove element from stack */
  if (ref != 0) {  /* any free element? */
    lua_rawgeti(L, t, ref);  /* remove it from list */
    lua_rawseti(L, t, freelist);  /* (t[freelist] = t[ref]) */
@@ -668,6 +689,7 @@ LUALIB_API void luaL_unref (lua_State *L, int t, int ref) {
  if (ref >= 0) {
    t = lua_absindex(L, t);
    lua_rawgeti(L, t, freelist);
+
    lua_assert(lua_isinteger(L, -1));
    lua_rawseti(L, t, ref);  /* t[ref] = t[freelist] */
    lua_pushinteger(L, ref);
    lua_rawseti(L, t, freelist);  /* t[freelist] = ref */
@@ -851,7 +873,7 @@ LUALIB_API lua_Integer luaL_len (lua_State *L, int idx) {
  int isnum;
  lua_len(L, idx);
  l = lua_tointegerx(L, -1, &isnum);
-
  if (!isnum)
+
  if (l_unlikely(!isnum))
    luaL_error(L, "object length is not an integer");
  lua_pop(L, 1);  /* remove object */
  return l;
@@ -859,6 +881,7 @@ LUALIB_API lua_Integer luaL_len (lua_State *L, int idx) {


LUALIB_API const char *luaL_tolstring (lua_State *L, int idx, size_t *len) {
+
  idx = lua_absindex(L,idx);
  if (luaL_callmeta(L, idx, "__tostring")) {  /* metafield? */
    if (!lua_isstring(L, -1))
      luaL_error(L, "'__tostring' must return a string");
@@ -1064,7 +1087,7 @@ static void warnfon (void *ud, const char *message, int tocont) {

LUALIB_API lua_State *luaL_newstate (void) {
  lua_State *L = lua_newstate(l_alloc, NULL);
-
  if (L) {
+
  if (l_likely(L)) {
    lua_atpanic(L, &panic);
    lua_setwarnf(L, warnfoff, L);  /* default is warnings off */
  }
modified external/lua/src/lauxlib.h
@@ -12,6 +12,7 @@
#include <stddef.h>
#include <stdio.h>

+
#include "luaconf.h"
#include "lua.h"


@@ -101,7 +102,7 @@ LUALIB_API lua_State *(luaL_newstate) (void);

LUALIB_API lua_Integer (luaL_len) (lua_State *L, int idx);

-
LUALIB_API void luaL_addgsub (luaL_Buffer *b, const char *s,
+
LUALIB_API void (luaL_addgsub) (luaL_Buffer *b, const char *s,
                                     const char *p, const char *r);
LUALIB_API const char *(luaL_gsub) (lua_State *L, const char *s,
                                    const char *p, const char *r);
@@ -130,10 +131,10 @@ LUALIB_API void (luaL_requiref) (lua_State *L, const char *modname,
  (luaL_checkversion(L), luaL_newlibtable(L,l), luaL_setfuncs(L,l,0))

#define luaL_argcheck(L, cond,arg,extramsg)	\
-
		((void)((cond) || luaL_argerror(L, (arg), (extramsg))))
+
	((void)(luai_likely(cond) || luaL_argerror(L, (arg), (extramsg))))

#define luaL_argexpected(L,cond,arg,tname)	\
-
		((void)((cond) || luaL_typeerror(L, (arg), (tname))))
+
	((void)(luai_likely(cond) || luaL_typeerror(L, (arg), (tname))))

#define luaL_checkstring(L,n)	(luaL_checklstring(L, (n), NULL))
#define luaL_optstring(L,n,d)	(luaL_optlstring(L, (n), (d), NULL))
@@ -153,11 +154,35 @@ LUALIB_API void (luaL_requiref) (lua_State *L, const char *modname,
#define luaL_loadbuffer(L,s,sz,n)	luaL_loadbufferx(L,s,sz,n,NULL)


+
/*
+
** Perform arithmetic operations on lua_Integer values with wrap-around
+
** semantics, as the Lua core does.
+
*/
+
#define luaL_intop(op,v1,v2)  \
+
	((lua_Integer)((lua_Unsigned)(v1) op (lua_Unsigned)(v2)))
+

+

/* push the value used to represent failure/error */
#define luaL_pushfail(L)	lua_pushnil(L)


/*
+
** Internal assertions for in-house debugging
+
*/
+
#if !defined(lua_assert)
+

+
#if defined LUAI_ASSERT
+
  #include <assert.h>
+
  #define lua_assert(c)		assert(c)
+
#else
+
  #define lua_assert(c)		((void)0)
+
#endif
+

+
#endif
+

+

+

+
/*
** {======================================================
** Generic Buffer manipulation
** =======================================================
modified external/lua/src/lbaselib.c
@@ -138,7 +138,7 @@ static int luaB_setmetatable (lua_State *L) {
  int t = lua_type(L, 2);
  luaL_checktype(L, 1, LUA_TTABLE);
  luaL_argexpected(L, t == LUA_TNIL || t == LUA_TTABLE, 2, "nil or table");
-
  if (luaL_getmetafield(L, 1, "__metatable") != LUA_TNIL)
+
  if (l_unlikely(luaL_getmetafield(L, 1, "__metatable") != LUA_TNIL))
    return luaL_error(L, "cannot change a protected metatable");
  lua_settop(L, 2);
  lua_setmetatable(L, 1);
@@ -182,11 +182,20 @@ static int luaB_rawset (lua_State *L) {


static int pushmode (lua_State *L, int oldmode) {
-
  lua_pushstring(L, (oldmode == LUA_GCINC) ? "incremental" : "generational");
+
  if (oldmode == -1)
+
    luaL_pushfail(L);  /* invalid call to 'lua_gc' */
+
  else
+
    lua_pushstring(L, (oldmode == LUA_GCINC) ? "incremental"
+
                                             : "generational");
  return 1;
}


+
/*
+
** check whether call to 'lua_gc' was valid (not inside a finalizer)
+
*/
+
#define checkvalres(res) { if (res == -1) break; }
+

static int luaB_collectgarbage (lua_State *L) {
  static const char *const opts[] = {"stop", "restart", "collect",
    "count", "step", "setpause", "setstepmul",
@@ -199,12 +208,14 @@ static int luaB_collectgarbage (lua_State *L) {
    case LUA_GCCOUNT: {
      int k = lua_gc(L, o);
      int b = lua_gc(L, LUA_GCCOUNTB);
+
      checkvalres(k);
      lua_pushnumber(L, (lua_Number)k + ((lua_Number)b/1024));
      return 1;
    }
    case LUA_GCSTEP: {
      int step = (int)luaL_optinteger(L, 2, 0);
      int res = lua_gc(L, o, step);
+
      checkvalres(res);
      lua_pushboolean(L, res);
      return 1;
    }
@@ -212,11 +223,13 @@ static int luaB_collectgarbage (lua_State *L) {
    case LUA_GCSETSTEPMUL: {
      int p = (int)luaL_optinteger(L, 2, 0);
      int previous = lua_gc(L, o, p);
+
      checkvalres(previous);
      lua_pushinteger(L, previous);
      return 1;
    }
    case LUA_GCISRUNNING: {
      int res = lua_gc(L, o);
+
      checkvalres(res);
      lua_pushboolean(L, res);
      return 1;
    }
@@ -233,10 +246,13 @@ static int luaB_collectgarbage (lua_State *L) {
    }
    default: {
      int res = lua_gc(L, o);
+
      checkvalres(res);
      lua_pushinteger(L, res);
      return 1;
    }
  }
+
  luaL_pushfail(L);  /* invalid call (inside a finalizer) */
+
  return 1;
}


@@ -260,6 +276,11 @@ static int luaB_next (lua_State *L) {
}


+
static int pairscont (lua_State *L, int status, lua_KContext k) {
+
  (void)L; (void)status; (void)k;  /* unused */
+
  return 3;
+
}
+

static int luaB_pairs (lua_State *L) {
  luaL_checkany(L, 1);
  if (luaL_getmetafield(L, 1, "__pairs") == LUA_TNIL) {  /* no metamethod? */
@@ -269,7 +290,7 @@ static int luaB_pairs (lua_State *L) {
  }
  else {
    lua_pushvalue(L, 1);  /* argument 'self' to metamethod */
-
    lua_call(L, 1, 3);  /* get 3 values from metamethod */
+
    lua_callk(L, 1, 3, 0, pairscont);  /* get 3 values from metamethod */
  }
  return 3;
}
@@ -279,7 +300,8 @@ static int luaB_pairs (lua_State *L) {
** Traversal function for 'ipairs'
*/
static int ipairsaux (lua_State *L) {
-
  lua_Integer i = luaL_checkinteger(L, 2) + 1;
+
  lua_Integer i = luaL_checkinteger(L, 2);
+
  i = luaL_intop(+, i, 1);
  lua_pushinteger(L, i);
  return (lua_geti(L, 1, i) == LUA_TNIL) ? 1 : 2;
}
@@ -299,7 +321,7 @@ static int luaB_ipairs (lua_State *L) {


static int load_aux (lua_State *L, int status, int envidx) {
-
  if (status == LUA_OK) {
+
  if (l_likely(status == LUA_OK)) {
    if (envidx != 0) {  /* 'env' parameter? */
      lua_pushvalue(L, envidx);  /* environment for loaded function */
      if (!lua_setupvalue(L, -2, 1))  /* set it as 1st upvalue */
@@ -355,7 +377,7 @@ static const char *generic_reader (lua_State *L, void *ud, size_t *size) {
    *size = 0;
    return NULL;
  }
-
  else if (!lua_isstring(L, -1))
+
  else if (l_unlikely(!lua_isstring(L, -1)))
    luaL_error(L, "reader function must return a string");
  lua_replace(L, RESERVEDSLOT);  /* save string in reserved slot */
  return lua_tolstring(L, RESERVEDSLOT, size);
@@ -393,7 +415,7 @@ static int dofilecont (lua_State *L, int d1, lua_KContext d2) {
static int luaB_dofile (lua_State *L) {
  const char *fname = luaL_optstring(L, 1, NULL);
  lua_settop(L, 1);
-
  if (luaL_loadfile(L, fname) != LUA_OK)
+
  if (l_unlikely(luaL_loadfile(L, fname) != LUA_OK))
    return lua_error(L);
  lua_callk(L, 0, LUA_MULTRET, 0, dofilecont);
  return dofilecont(L, 0, 0);
@@ -401,7 +423,7 @@ static int luaB_dofile (lua_State *L) {


static int luaB_assert (lua_State *L) {
-
  if (lua_toboolean(L, 1))  /* condition is true? */
+
  if (l_likely(lua_toboolean(L, 1)))  /* condition is true? */
    return lua_gettop(L);  /* return all arguments */
  else {  /* error */
    luaL_checkany(L, 1);  /* there must be a condition */
@@ -437,7 +459,7 @@ static int luaB_select (lua_State *L) {
** ignored).
*/
static int finishpcall (lua_State *L, int status, lua_KContext extra) {
-
  if (status != LUA_OK && status != LUA_YIELD) {  /* error? */
+
  if (l_unlikely(status != LUA_OK && status != LUA_YIELD)) {  /* error? */
    lua_pushboolean(L, 0);  /* first result (false) */
    lua_pushvalue(L, -2);  /* error message */
    return 2;  /* return false, msg */
modified external/lua/src/lcode.c
@@ -10,6 +10,7 @@
#include "lprefix.h"


+
#include <float.h>
#include <limits.h>
#include <math.h>
#include <stdlib.h>
@@ -314,15 +315,6 @@ void luaK_patchtohere (FuncState *fs, int list) {
}


-
/*
-
** MAXimum number of successive Instructions WiTHout ABSolute line
-
** information.
-
*/
-
#if !defined(MAXIWTHABS)
-
#define MAXIWTHABS	120
-
#endif
-

-

/* limit for difference between lines in relative line info. */
#define LIMLINEDIFF	0x80

@@ -337,13 +329,13 @@ void luaK_patchtohere (FuncState *fs, int list) {
static void savelineinfo (FuncState *fs, Proto *f, int line) {
  int linedif = line - fs->previousline;
  int pc = fs->pc - 1;  /* last instruction coded */
-
  if (abs(linedif) >= LIMLINEDIFF || fs->iwthabs++ > MAXIWTHABS) {
+
  if (abs(linedif) >= LIMLINEDIFF || fs->iwthabs++ >= MAXIWTHABS) {
    luaM_growvector(fs->ls->L, f->abslineinfo, fs->nabslineinfo,
                    f->sizeabslineinfo, AbsLineInfo, MAX_INT, "lines");
    f->abslineinfo[fs->nabslineinfo].pc = pc;
    f->abslineinfo[fs->nabslineinfo++].line = line;
    linedif = ABSLINEINFO;  /* signal that there is absolute information */
-
    fs->iwthabs = 0;  /* restart counter */
+
    fs->iwthabs = 1;  /* restart counter */
  }
  luaM_growvector(fs->ls->L, f->lineinfo, pc, f->sizelineinfo, ls_byte,
                  MAX_INT, "opcodes");
@@ -545,11 +537,14 @@ static void freeexps (FuncState *fs, expdesc *e1, expdesc *e2) {
** and try to reuse constants. Because some values should not be used
** as keys (nil cannot be a key, integer keys can collapse with float
** keys), the caller must provide a useful 'key' for indexing the cache.
+
** Note that all functions share the same table, so entering or exiting
+
** a function can make some indices wrong.
*/
static int addk (FuncState *fs, TValue *key, TValue *v) {
+
  TValue val;
  lua_State *L = fs->ls->L;
  Proto *f = fs->f;
-
  TValue *idx = luaH_set(L, fs->ls->h, key);  /* index scanner table */
+
  const TValue *idx = luaH_get(fs->ls->h, key);  /* query scanner table */
  int k, oldsize;
  if (ttisinteger(idx)) {  /* is there an index there? */
    k = cast_int(ivalue(idx));
@@ -563,7 +558,8 @@ static int addk (FuncState *fs, TValue *key, TValue *v) {
  k = fs->nk;
  /* numerical value does not need GC barrier;
     table has no metatable, so it does not need to invalidate cache */
-
  setivalue(idx, k);
+
  setivalue(&val, k);
+
  luaH_finishset(L, fs->ls->h, key, idx, &val);
  luaM_growvector(L, f->k, k, f->sizek, TValue, MAXARG_Ax, "constants");
  while (oldsize < f->sizek) setnilvalue(&f->k[oldsize++]);
  setobj(L, &f->k[k], v);
@@ -585,24 +581,41 @@ static int stringK (FuncState *fs, TString *s) {

/*
** Add an integer to list of constants and return its index.
-
** Integers use userdata as keys to avoid collision with floats with
-
** same value; conversion to 'void*' is used only for hashing, so there
-
** are no "precision" problems.
*/
static int luaK_intK (FuncState *fs, lua_Integer n) {
-
  TValue k, o;
-
  setpvalue(&k, cast_voidp(cast_sizet(n)));
+
  TValue o;
  setivalue(&o, n);
-
  return addk(fs, &k, &o);
+
  return addk(fs, &o, &o);  /* use integer itself as key */
}

/*
-
** Add a float to list of constants and return its index.
+
** Add a float to list of constants and return its index. Floats
+
** with integral values need a different key, to avoid collision
+
** with actual integers. To that, we add to the number its smaller
+
** power-of-two fraction that is still significant in its scale.
+
** For doubles, that would be 1/2^52.
+
** (This method is not bulletproof: there may be another float
+
** with that value, and for floats larger than 2^53 the result is
+
** still an integer. At worst, this only wastes an entry with
+
** a duplicate.)
*/
static int luaK_numberK (FuncState *fs, lua_Number r) {
  TValue o;
+
  lua_Integer ik;
  setfltvalue(&o, r);
-
  return addk(fs, &o, &o);  /* use number itself as key */
+
  if (!luaV_flttointeger(r, &ik, F2Ieq))  /* not an integral value? */
+
    return addk(fs, &o, &o);  /* use number itself as key */
+
  else {  /* must build an alternative key */
+
    const int nbm = l_floatatt(MANT_DIG);
+
    const lua_Number q = l_mathop(ldexp)(l_mathop(1.0), -nbm + 1);
+
    const lua_Number k = (ik == 0) ? q : r + r*q;  /* new key */
+
    TValue kv;
+
    setfltvalue(&kv, k);
+
    /* result is not an integral value, unless value is too large */
+
    lua_assert(!luaV_flttointeger(k, &ik, F2Ieq) ||
+
                l_mathop(fabs)(r) >= l_mathop(1e6));
+
    return addk(fs, &kv, &o);
+
  }
}


@@ -763,7 +776,7 @@ void luaK_dischargevars (FuncState *fs, expdesc *e) {
      break;
    }
    case VLOCAL: {  /* already in a register */
-
      e->u.info = e->u.var.sidx;
+
      e->u.info = e->u.var.ridx;
      e->k = VNONRELOC;  /* becomes a non-relocatable value */
      break;
    }
@@ -1036,7 +1049,7 @@ void luaK_storevar (FuncState *fs, expdesc *var, expdesc *ex) {
  switch (var->k) {
    case VLOCAL: {
      freeexp(fs, ex);
-
      exp2reg(fs, ex, var->u.var.sidx);  /* compute 'ex' into proper place */
+
      exp2reg(fs, ex, var->u.var.ridx);  /* compute 'ex' into proper place */
      return;
    }
    case VUPVAL: {
@@ -1276,7 +1289,7 @@ void luaK_indexed (FuncState *fs, expdesc *t, expdesc *k) {
  }
  else {
    /* register index of the table */
-
    t->u.ind.t = (t->k == VLOCAL) ? t->u.var.sidx: t->u.info;
+
    t->u.ind.t = (t->k == VLOCAL) ? t->u.var.ridx: t->u.info;
    if (isKstr(fs, k)) {
      t->u.ind.idx = k->u.info;  /* literal string */
      t->k = VINDEXSTR;
@@ -1303,7 +1316,8 @@ static int validop (int op, TValue *v1, TValue *v2) {
    case LUA_OPBAND: case LUA_OPBOR: case LUA_OPBXOR:
    case LUA_OPSHL: case LUA_OPSHR: case LUA_OPBNOT: {  /* conversion errors */
      lua_Integer i;
-
      return (tointegerns(v1, &i) && tointegerns(v2, &i));
+
      return (luaV_tointegerns(v1, &i, LUA_FLOORN2I) &&
+
              luaV_tointegerns(v2, &i, LUA_FLOORN2I));
    }
    case LUA_OPDIV: case LUA_OPIDIV: case LUA_OPMOD:  /* division by 0 */
      return (nvalue(v2) != 0);
modified external/lua/src/lcorolib.c
@@ -31,14 +31,14 @@ static lua_State *getco (lua_State *L) {
*/
static int auxresume (lua_State *L, lua_State *co, int narg) {
  int status, nres;
-
  if (!lua_checkstack(co, narg)) {
+
  if (l_unlikely(!lua_checkstack(co, narg))) {
    lua_pushliteral(L, "too many arguments to resume");
    return -1;  /* error flag */
  }
  lua_xmove(L, co, narg);
  status = lua_resume(co, L, narg, &nres);
-
  if (status == LUA_OK || status == LUA_YIELD) {
-
    if (!lua_checkstack(L, nres + 1)) {
+
  if (l_likely(status == LUA_OK || status == LUA_YIELD)) {
+
    if (l_unlikely(!lua_checkstack(L, nres + 1))) {
      lua_pop(co, nres);  /* remove results anyway */
      lua_pushliteral(L, "too many results to resume");
      return -1;  /* error flag */
@@ -57,7 +57,7 @@ static int luaB_coresume (lua_State *L) {
  lua_State *co = getco(L);
  int r;
  r = auxresume(L, co, lua_gettop(L) - 1);
-
  if (r < 0) {
+
  if (l_unlikely(r < 0)) {
    lua_pushboolean(L, 0);
    lua_insert(L, -2);
    return 2;  /* return false + error message */
@@ -73,10 +73,13 @@ static int luaB_coresume (lua_State *L) {
static int luaB_auxwrap (lua_State *L) {
  lua_State *co = lua_tothread(L, lua_upvalueindex(1));
  int r = auxresume(L, co, lua_gettop(L));
-
  if (r < 0) {  /* error? */
+
  if (l_unlikely(r < 0)) {  /* error? */
    int stat = lua_status(co);
-
    if (stat != LUA_OK && stat != LUA_YIELD)  /* error in the coroutine? */
-
      lua_resetthread(co);  /* close its tbc variables */
+
    if (stat != LUA_OK && stat != LUA_YIELD) {  /* error in the coroutine? */
+
      stat = lua_resetthread(co);  /* close its tbc variables */
+
      lua_assert(stat != LUA_OK);
+
      lua_xmove(co, L, 1);  /* move error message to the caller */
+
    }
    if (stat != LUA_ERRMEM &&  /* not a memory error and ... */
        lua_type(L, -1) == LUA_TSTRING) {  /* ... error object is a string? */
      luaL_where(L, 1);  /* add extra info, if available */
@@ -176,7 +179,7 @@ static int luaB_close (lua_State *L) {
      }
      else {
        lua_pushboolean(L, 0);
-
        lua_xmove(co, L, 1);  /* copy error message */
+
        lua_xmove(co, L, 1);  /* move error message */
        return 2;
      }
    }
modified external/lua/src/ldblib.c
@@ -33,7 +33,7 @@ static const char *const HOOKKEY = "_HOOKKEY";
** checked.
*/
static void checkstack (lua_State *L, lua_State *L1, int n) {
-
  if (L != L1 && !lua_checkstack(L1, n))
+
  if (l_unlikely(L != L1 && !lua_checkstack(L1, n)))
    luaL_error(L, "stack overflow");
}

@@ -152,6 +152,7 @@ static int db_getinfo (lua_State *L) {
  lua_State *L1 = getthread(L, &arg);
  const char *options = luaL_optstring(L, arg+2, "flnSrtu");
  checkstack(L, L1, 3);
+
  luaL_argcheck(L, options[0] != '>', arg + 2, "invalid option '>'");
  if (lua_isfunction(L, arg + 1)) {  /* info about a function? */
    options = lua_pushfstring(L, ">%s", options);  /* add '>' to 'options' */
    lua_pushvalue(L, arg + 1);  /* move function to 'L1' stack */
@@ -212,7 +213,7 @@ static int db_getlocal (lua_State *L) {
    lua_Debug ar;
    const char *name;
    int level = (int)luaL_checkinteger(L, arg + 1);
-
    if (!lua_getstack(L1, level, &ar))  /* out of range? */
+
    if (l_unlikely(!lua_getstack(L1, level, &ar)))  /* out of range? */
      return luaL_argerror(L, arg+1, "level out of range");
    checkstack(L, L1, 1);
    name = lua_getlocal(L1, &ar, nvar);
@@ -237,7 +238,7 @@ static int db_setlocal (lua_State *L) {
  lua_Debug ar;
  int level = (int)luaL_checkinteger(L, arg + 1);
  int nvar = (int)luaL_checkinteger(L, arg + 2);
-
  if (!lua_getstack(L1, level, &ar))  /* out of range? */
+
  if (l_unlikely(!lua_getstack(L1, level, &ar)))  /* out of range? */
    return luaL_argerror(L, arg+1, "level out of range");
  luaL_checkany(L, arg+3);
  lua_settop(L, arg+3);
@@ -377,7 +378,7 @@ static int db_sethook (lua_State *L) {
  }
  if (!luaL_getsubtable(L, LUA_REGISTRYINDEX, HOOKKEY)) {
    /* table just created; initialize it */
-
    lua_pushstring(L, "k");
+
    lua_pushliteral(L, "k");
    lua_setfield(L, -2, "__mode");  /** hooktable.__mode = "k" */
    lua_pushvalue(L, -1);
    lua_setmetatable(L, -2);  /* metatable(hooktable) = hooktable */
@@ -420,7 +421,7 @@ static int db_debug (lua_State *L) {
  for (;;) {
    char buffer[250];
    lua_writestringerror("%s", "lua_debug> ");
-
    if (fgets(buffer, sizeof(buffer), stdin) == 0 ||
+
    if (fgets(buffer, sizeof(buffer), stdin) == NULL ||
        strcmp(buffer, "cont\n") == 0)
      return 0;
    if (luaL_loadbuffer(L, buffer, strlen(buffer), "=(debug command)") ||
modified external/lua/src/ldebug.c
@@ -33,11 +33,9 @@

#define noLuaClosure(f)		((f) == NULL || (f)->c.tt == LUA_VCCL)

-
/* inverse of 'pcRel' */
-
#define invpcRel(pc, p)		((p)->code + (pc) + 1)

-
static const char *funcnamefromcode (lua_State *L, CallInfo *ci,
-
                                    const char **name);
+
static const char *funcnamefromcall (lua_State *L, CallInfo *ci,
+
                                                   const char **name);


static int currentpc (CallInfo *ci) {
@@ -48,10 +46,16 @@ static int currentpc (CallInfo *ci) {

/*
** Get a "base line" to find the line corresponding to an instruction.
-
** For that, search the array of absolute line info for the largest saved
-
** instruction smaller or equal to the wanted instruction. A special
-
** case is when there is no absolute info or the instruction is before
-
** the first absolute one.
+
** Base lines are regularly placed at MAXIWTHABS intervals, so usually
+
** an integer division gets the right place. When the source file has
+
** large sequences of empty/comment lines, it may need extra entries,
+
** so the original estimate needs a correction.
+
** If the original estimate is -1, the initial 'if' ensures that the
+
** 'while' will run at least once.
+
** The assertion that the estimate is a lower bound for the correct base
+
** is valid as long as the debug info has been generated with the same
+
** value for MAXIWTHABS or smaller. (Previous releases use a little
+
** smaller value.)
*/
static int getbaseline (const Proto *f, int pc, int *basepc) {
  if (f->sizeabslineinfo == 0 || pc < f->abslineinfo[0].pc) {
@@ -59,20 +63,12 @@ static int getbaseline (const Proto *f, int pc, int *basepc) {
    return f->linedefined;
  }
  else {
-
    unsigned int i;
-
    if (pc >= f->abslineinfo[f->sizeabslineinfo - 1].pc)
-
      i = f->sizeabslineinfo - 1;  /* instruction is after last saved one */
-
    else {  /* binary search */
-
      unsigned int j = f->sizeabslineinfo - 1;  /* pc < anchorlines[j] */
-
      i = 0;  /* abslineinfo[i] <= pc */
-
      while (i < j - 1) {
-
        unsigned int m = (j + i) / 2;
-
        if (pc >= f->abslineinfo[m].pc)
-
          i = m;
-
        else
-
          j = m;
-
      }
-
    }
+
    int i = cast_uint(pc) / MAXIWTHABS - 1;  /* get an estimate */
+
    /* estimate must be a lower bound of the correct base */
+
    lua_assert(i < 0 ||
+
              (i < f->sizeabslineinfo && f->abslineinfo[i].pc <= pc));
+
    while (i + 1 < f->sizeabslineinfo && pc >= f->abslineinfo[i + 1].pc)
+
      i++;  /* low estimate; adjust it */
    *basepc = f->abslineinfo[i].pc;
    return f->abslineinfo[i].line;
  }
@@ -305,8 +301,15 @@ static void collectvalidlines (lua_State *L, Closure *f) {
    sethvalue2s(L, L->top, t);  /* push it on stack */
    api_incr_top(L);
    setbtvalue(&v);  /* boolean 'true' to be the value of all indices */
-
    for (i = 0; i < p->sizelineinfo; i++) {  /* for all lines with code */
-
      currentline = nextline(p, currentline, i);
+
    if (!p->is_vararg)  /* regular function? */
+
      i = 0;  /* consider all instructions */
+
    else {  /* vararg function */
+
      lua_assert(GET_OPCODE(p->code[0]) == OP_VARARGPREP);
+
      currentline = nextline(p, currentline, 0);
+
      i = 1;  /* skip first instruction (OP_VARARGPREP) */
+
    }
+
    for (; i < p->sizelineinfo; i++) {  /* for each instruction */
+
      currentline = nextline(p, currentline, i);  /* get its line */
      luaH_setint(L, t, currentline, &v);  /* table[line] = true */
    }
  }
@@ -314,15 +317,9 @@ static void collectvalidlines (lua_State *L, Closure *f) {


static const char *getfuncname (lua_State *L, CallInfo *ci, const char **name) {
-
  if (ci == NULL)  /* no 'ci'? */
-
    return NULL;  /* no info */
-
  else if (ci->callstatus & CIST_FIN) {  /* is this a finalizer? */
-
    *name = "__gc";
-
    return "metamethod";  /* report it as such */
-
  }
-
  /* calling function is a known Lua function? */
-
  else if (!(ci->callstatus & CIST_TAIL) && isLua(ci->previous))
-
    return funcnamefromcode(L, ci->previous, name);
+
  /* calling function is a known function? */
+
  if (ci != NULL && !(ci->callstatus & CIST_TAIL))
+
    return funcnamefromcall(L, ci->previous, name);
  else return NULL;  /* no way to find a name */
}

@@ -594,16 +591,10 @@ static const char *getobjname (const Proto *p, int lastpc, int reg,
** Returns what the name is (e.g., "for iterator", "method",
** "metamethod") and sets '*name' to point to the name.
*/
-
static const char *funcnamefromcode (lua_State *L, CallInfo *ci,
-
                                     const char **name) {
+
static const char *funcnamefromcode (lua_State *L, const Proto *p,
+
                                     int pc, const char **name) {
  TMS tm = (TMS)0;  /* (initial value avoids warnings) */
-
  const Proto *p = ci_func(ci)->p;  /* calling function */
-
  int pc = currentpc(ci);  /* calling instruction index */
  Instruction i = p->code[pc];  /* calling instruction */
-
  if (ci->callstatus & CIST_HOOKED) {  /* was it called inside a hook? */
-
    *name = "?";
-
    return "hook";
-
  }
  switch (GET_OPCODE(i)) {
    case OP_CALL:
    case OP_TAILCALL:
@@ -629,12 +620,10 @@ static const char *funcnamefromcode (lua_State *L, CallInfo *ci,
    case OP_LEN: tm = TM_LEN; break;
    case OP_CONCAT: tm = TM_CONCAT; break;
    case OP_EQ: tm = TM_EQ; break;
-
    case OP_LT: case OP_LE: case OP_LTI: case OP_LEI:
-
      *name = "order";  /* '<=' can call '__lt', etc. */
-
      return "metamethod";
-
    case OP_CLOSE: case OP_RETURN:
-
      *name = "close";
-
      return "metamethod";
+
    /* no cases for OP_EQI and OP_EQK, as they don't call metamethods */
+
    case OP_LT: case OP_LTI: case OP_GTI: tm = TM_LT; break;
+
    case OP_LE: case OP_LEI: case OP_GEI: tm = TM_LE; break;
+
    case OP_CLOSE: case OP_RETURN: tm = TM_CLOSE; break;
    default:
      return NULL;  /* cannot find a reasonable name */
  }
@@ -642,19 +631,43 @@ static const char *funcnamefromcode (lua_State *L, CallInfo *ci,
  return "metamethod";
}

+

+
/*
+
** Try to find a name for a function based on how it was called.
+
*/
+
static const char *funcnamefromcall (lua_State *L, CallInfo *ci,
+
                                                   const char **name) {
+
  if (ci->callstatus & CIST_HOOKED) {  /* was it called inside a hook? */
+
    *name = "?";
+
    return "hook";
+
  }
+
  else if (ci->callstatus & CIST_FIN) {  /* was it called as a finalizer? */
+
    *name = "__gc";
+
    return "metamethod";  /* report it as such */
+
  }
+
  else if (isLua(ci))
+
    return funcnamefromcode(L, ci_func(ci)->p, currentpc(ci), name);
+
  else
+
    return NULL;
+
}
+

/* }====================================================== */



/*
-
** The subtraction of two potentially unrelated pointers is
-
** not ISO C, but it should not crash a program; the subsequent
-
** checks are ISO C and ensure a correct result.
+
** Check whether pointer 'o' points to some value in the stack
+
** frame of the current function. Because 'o' may not point to a
+
** value in this stack, we cannot compare it with the region
+
** boundaries (undefined behaviour in ISO C).
*/
static int isinstack (CallInfo *ci, const TValue *o) {
-
  StkId base = ci->func + 1;
-
  ptrdiff_t i = cast(StkId, o) - base;
-
  return (0 <= i && i < (ci->top - base) && s2v(base + i) == o);
+
  StkId pos;
+
  for (pos = ci->func + 1; pos < ci->top; pos++) {
+
    if (o == s2v(pos))
+
      return 1;
+
  }
+
  return 0;  /* not found */
}


@@ -677,9 +690,21 @@ static const char *getupvalname (CallInfo *ci, const TValue *o,
}


+
static const char *formatvarinfo (lua_State *L, const char *kind,
+
                                                const char *name) {
+
  if (kind == NULL)
+
    return "";  /* no information */
+
  else
+
    return luaO_pushfstring(L, " (%s '%s')", kind, name);
+
}
+

+
/*
+
** Build a string with a "description" for the value 'o', such as
+
** "variable 'x'" or "upvalue 'y'".
+
*/
static const char *varinfo (lua_State *L, const TValue *o) {
-
  const char *name = NULL;  /* to avoid warnings */
  CallInfo *ci = L->ci;
+
  const char *name = NULL;  /* to avoid warnings */
  const char *kind = NULL;
  if (isLua(ci)) {
    kind = getupvalname(ci, o, &name);  /* check whether 'o' is an upvalue */
@@ -687,13 +712,40 @@ static const char *varinfo (lua_State *L, const TValue *o) {
      kind = getobjname(ci_func(ci)->p, currentpc(ci),
                        cast_int(cast(StkId, o) - (ci->func + 1)), &name);
  }
-
  return (kind) ? luaO_pushfstring(L, " (%s '%s')", kind, name) : "";
+
  return formatvarinfo(L, kind, name);
}


-
l_noret luaG_typeerror (lua_State *L, const TValue *o, const char *op) {
+
/*
+
** Raise a type error
+
*/
+
static l_noret typeerror (lua_State *L, const TValue *o, const char *op,
+
                          const char *extra) {
  const char *t = luaT_objtypename(L, o);
-
  luaG_runerror(L, "attempt to %s a %s value%s", op, t, varinfo(L, o));
+
  luaG_runerror(L, "attempt to %s a %s value%s", op, t, extra);
+
}
+

+

+
/*
+
** Raise a type error with "standard" information about the faulty
+
** object 'o' (using 'varinfo').
+
*/
+
l_noret luaG_typeerror (lua_State *L, const TValue *o, const char *op) {
+
  typeerror(L, o, op, varinfo(L, o));
+
}
+

+

+
/*
+
** Raise an error for calling a non-callable object. Try to find a name
+
** for the object based on how it was called ('funcnamefromcall'); if it
+
** cannot get a name there, try 'varinfo'.
+
*/
+
l_noret luaG_callerror (lua_State *L, const TValue *o) {
+
  CallInfo *ci = L->ci;
+
  const char *name = NULL;  /* to avoid warnings */
+
  const char *kind = funcnamefromcall(L, ci, &name);
+
  const char *extra = kind ? formatvarinfo(L, kind, name) : varinfo(L, o);
+
  typeerror(L, o, "call", extra);
}


@@ -722,7 +774,7 @@ l_noret luaG_opinterror (lua_State *L, const TValue *p1,
*/
l_noret luaG_tointerror (lua_State *L, const TValue *p1, const TValue *p2) {
  lua_Integer temp;
-
  if (!tointegerns(p1, &temp))
+
  if (!luaV_tointegerns(p1, &temp, LUA_FLOORN2I))
    p2 = p1;
  luaG_runerror(L, "number%s has no integer representation", varinfo(L, p2));
}
@@ -780,16 +832,30 @@ l_noret luaG_runerror (lua_State *L, const char *fmt, ...) {

/*
** Check whether new instruction 'newpc' is in a different line from
-
** previous instruction 'oldpc'.
+
** previous instruction 'oldpc'. More often than not, 'newpc' is only
+
** one or a few instructions after 'oldpc' (it must be after, see
+
** caller), so try to avoid calling 'luaG_getfuncline'. If they are
+
** too far apart, there is a good chance of a ABSLINEINFO in the way,
+
** so it goes directly to 'luaG_getfuncline'.
*/
static int changedline (const Proto *p, int oldpc, int newpc) {
  if (p->lineinfo == NULL)  /* no debug information? */
    return 0;
-
  while (oldpc++ < newpc) {
-
    if (p->lineinfo[oldpc] != 0)
-
      return (luaG_getfuncline(p, oldpc - 1) != luaG_getfuncline(p, newpc));
+
  if (newpc - oldpc < MAXIWTHABS / 2) {  /* not too far apart? */
+
    int delta = 0;  /* line diference */
+
    int pc = oldpc;
+
    for (;;) {
+
      int lineinfo = p->lineinfo[++pc];
+
      if (lineinfo == ABSLINEINFO)
+
        break;  /* cannot compute delta; fall through */
+
      delta += lineinfo;
+
      if (pc == newpc)
+
        return (delta != 0);  /* delta computed successfully */
+
    }
  }
-
  return 0;  /* no line changes between positions */
+
  /* either instructions are too far apart or there is an absolute line
+
     info in the way; compute line difference explicitly */
+
  return (luaG_getfuncline(p, oldpc) != luaG_getfuncline(p, newpc));
}


@@ -797,20 +863,19 @@ static int changedline (const Proto *p, int oldpc, int newpc) {
** Traces the execution of a Lua function. Called before the execution
** of each opcode, when debug is on. 'L->oldpc' stores the last
** instruction traced, to detect line changes. When entering a new
-
** function, 'npci' will be zero and will test as a new line without
-
** the need for 'oldpc'; so, 'oldpc' does not need to be initialized
-
** before. Some exceptional conditions may return to a function without
-
** updating 'oldpc'. In that case, 'oldpc' may be invalid; if so, it is
-
** reset to zero.  (A wrong but valid 'oldpc' at most causes an extra
-
** call to a line hook.)
+
** function, 'npci' will be zero and will test as a new line whatever
+
** the value of 'oldpc'.  Some exceptional conditions may return to
+
** a function without setting 'oldpc'. In that case, 'oldpc' may be
+
** invalid; if so, use zero as a valid value. (A wrong but valid 'oldpc'
+
** at most causes an extra call to a line hook.)
+
** This function is not "Protected" when called, so it should correct
+
** 'L->top' before calling anything that can run the GC.
*/
int luaG_traceexec (lua_State *L, const Instruction *pc) {
  CallInfo *ci = L->ci;
  lu_byte mask = L->hookmask;
  const Proto *p = ci_func(ci)->p;
  int counthook;
-
  /* 'L->oldpc' may be invalid; reset it in this case */
-
  int oldpc = (L->oldpc < p->sizecode) ? L->oldpc : 0;
  if (!(mask & (LUA_MASKLINE | LUA_MASKCOUNT))) {  /* no hooks? */
    ci->u.l.trap = 0;  /* don't need to stop again */
    return 0;  /* turn off 'trap' */
@@ -826,15 +891,16 @@ int luaG_traceexec (lua_State *L, const Instruction *pc) {
    ci->callstatus &= ~CIST_HOOKYIELD;  /* erase mark */
    return 1;  /* do not call hook again (VM yielded, so it did not move) */
  }
-
  if (!isIT(*(ci->u.l.savedpc - 1)))
-
    L->top = ci->top;  /* prepare top */
+
  if (!isIT(*(ci->u.l.savedpc - 1)))  /* top not being used? */
+
    L->top = ci->top;  /* correct top */
  if (counthook)
    luaD_hook(L, LUA_HOOKCOUNT, -1, 0, 0);  /* call count hook */
  if (mask & LUA_MASKLINE) {
+
    /* 'L->oldpc' may be invalid; use zero in this case */
+
    int oldpc = (L->oldpc < p->sizecode) ? L->oldpc : 0;
    int npci = pcRel(pc, p);
-
    if (npci == 0 ||  /* call linehook when enter a new function, */
-
        pc <= invpcRel(oldpc, p) ||  /* when jump back (loop), or when */
-
        changedline(p, oldpc, npci)) {  /* enter new line */
+
    if (npci <= oldpc ||  /* call hook when jump back (loop), */
+
        changedline(p, oldpc, npci)) {  /* or when enter new line */
      int newline = luaG_getfuncline(p, npci);
      luaD_hook(L, LUA_HOOKLINE, newline, 0, 0);  /* call line hook */
    }
modified external/lua/src/ldebug.h
@@ -26,11 +26,22 @@
*/
#define ABSLINEINFO	(-0x80)

+

+
/*
+
** MAXimum number of successive Instructions WiTHout ABSolute line
+
** information. (A power of two allows fast divisions.)
+
*/
+
#if !defined(MAXIWTHABS)
+
#define MAXIWTHABS	128
+
#endif
+

+

LUAI_FUNC int luaG_getfuncline (const Proto *f, int pc);
LUAI_FUNC const char *luaG_findlocal (lua_State *L, CallInfo *ci, int n,
                                                    StkId *pos);
LUAI_FUNC l_noret luaG_typeerror (lua_State *L, const TValue *o,
                                                const char *opname);
+
LUAI_FUNC l_noret luaG_callerror (lua_State *L, const TValue *o);
LUAI_FUNC l_noret luaG_forerror (lua_State *L, const TValue *o,
                                               const char *what);
LUAI_FUNC l_noret luaG_concaterror (lua_State *L, const TValue *p1,
modified external/lua/src/ldo.c
@@ -98,11 +98,12 @@ void luaD_seterrorobj (lua_State *L, int errcode, StkId oldtop) {
      setsvalue2s(L, oldtop, luaS_newliteral(L, "error in error handling"));
      break;
    }
-
    case CLOSEPROTECT: {
+
    case LUA_OK: {  /* special case only for closing upvalues */
      setnilvalue(s2v(oldtop));  /* no error message */
      break;
    }
    default: {
+
      lua_assert(errorstatus(errcode));  /* real error */
      setobjs2s(L, oldtop, L->top - 1);  /* error message on current top */
      break;
    }
@@ -118,17 +119,13 @@ l_noret luaD_throw (lua_State *L, int errcode) {
  }
  else {  /* thread has no error handler */
    global_State *g = G(L);
-
    errcode = luaF_close(L, L->stack, errcode);  /* close all upvalues */
-
    L->status = cast_byte(errcode);  /* mark it as dead */
+
    errcode = luaE_resetthread(L, errcode);  /* close all upvalues */
    if (g->mainthread->errorJmp) {  /* main thread has a handler? */
      setobjs2s(L, g->mainthread->top++, L->top - 1);  /* copy error obj. */
      luaD_throw(g->mainthread, errcode);  /* re-throw in main thread */
    }
    else {  /* no handler at all; abort */
      if (g->panic) {  /* panic function? */
-
        luaD_seterrorobj(L, errcode, L->top);  /* assume EXTRA_STACK */
-
        if (L->ci->top < L->top)
-
          L->ci->top = L->top;  /* pushing msg. can break this invariant */
        lua_unlock(L);
        g->panic(L);  /* call panic function (last chance to jump out) */
      }
@@ -163,9 +160,8 @@ int luaD_rawrunprotected (lua_State *L, Pfunc f, void *ud) {
static void correctstack (lua_State *L, StkId oldstack, StkId newstack) {
  CallInfo *ci;
  UpVal *up;
-
  if (oldstack == newstack)
-
    return;  /* stack address did not change */
  L->top = (L->top - oldstack) + newstack;
+
  L->tbclist = (L->tbclist - oldstack) + newstack;
  for (up = L->openupval; up != NULL; up = up->u.open.next)
    up->v = s2v((uplevel(up) - oldstack) + newstack);
  for (ci = L->ci; ci != NULL; ci = ci->previous) {
@@ -181,19 +177,35 @@ static void correctstack (lua_State *L, StkId oldstack, StkId newstack) {
#define ERRORSTACKSIZE	(LUAI_MAXSTACK + 200)


+
/*
+
** Reallocate the stack to a new size, correcting all pointers into
+
** it. (There are pointers to a stack from its upvalues, from its list
+
** of call infos, plus a few individual pointers.) The reallocation is
+
** done in two steps (allocation + free) because the correction must be
+
** done while both addresses (the old stack and the new one) are valid.
+
** (In ISO C, any pointer use after the pointer has been deallocated is
+
** undefined behavior.)
+
** In case of allocation error, raise an error or return false according
+
** to 'raiseerror'.
+
*/
int luaD_reallocstack (lua_State *L, int newsize, int raiseerror) {
-
  int lim = stacksize(L);
-
  StkId newstack = luaM_reallocvector(L, L->stack,
-
                      lim + EXTRA_STACK, newsize + EXTRA_STACK, StackValue);
+
  int oldsize = stacksize(L);
+
  int i;
+
  StkId newstack = luaM_reallocvector(L, NULL, 0,
+
                                      newsize + EXTRA_STACK, StackValue);
  lua_assert(newsize <= LUAI_MAXSTACK || newsize == ERRORSTACKSIZE);
-
  if (unlikely(newstack == NULL)) {  /* reallocation failed? */
+
  if (l_unlikely(newstack == NULL)) {  /* reallocation failed? */
    if (raiseerror)
      luaM_error(L);
    else return 0;  /* do not raise an error */
  }
-
  for (; lim < newsize; lim++)
-
    setnilvalue(s2v(newstack + lim + EXTRA_STACK)); /* erase new segment */
+
  /* number of elements to be copied to the new stack */
+
  i = ((oldsize <= newsize) ? oldsize : newsize) + EXTRA_STACK;
+
  memcpy(newstack, L->stack, i * sizeof(StackValue));
+
  for (; i < newsize + EXTRA_STACK; i++)
+
    setnilvalue(s2v(newstack + i)); /* erase new segment */
  correctstack(L, L->stack, newstack);
+
  luaM_freearray(L, L->stack, oldsize + EXTRA_STACK);
  L->stack = newstack;
  L->stack_last = L->stack + newsize;
  return 1;
@@ -206,7 +218,7 @@ int luaD_reallocstack (lua_State *L, int newsize, int raiseerror) {
*/
int luaD_growstack (lua_State *L, int n, int raiseerror) {
  int size = stacksize(L);
-
  if (unlikely(size > LUAI_MAXSTACK)) {
+
  if (l_unlikely(size > LUAI_MAXSTACK)) {
    /* if stack is larger than maximum, thread is already using the
       extra space reserved for errors, that is, thread is handling
       a stack error; cannot grow further than that. */
@@ -222,7 +234,7 @@ int luaD_growstack (lua_State *L, int n, int raiseerror) {
      newsize = LUAI_MAXSTACK;
    if (newsize < needed)  /* but must respect what was asked for */
      newsize = needed;
-
    if (likely(newsize <= LUAI_MAXSTACK))
+
    if (l_likely(newsize <= LUAI_MAXSTACK))
      return luaD_reallocstack(L, newsize, raiseerror);
    else {  /* stack overflow */
      /* add extra size to be able to handle the error message */
@@ -297,8 +309,8 @@ void luaD_hook (lua_State *L, int event, int line,
  if (hook && L->allowhook) {  /* make sure there is a hook */
    int mask = CIST_HOOKED;
    CallInfo *ci = L->ci;
-
    ptrdiff_t top = savestack(L, L->top);
-
    ptrdiff_t ci_top = savestack(L, ci->top);
+
    ptrdiff_t top = savestack(L, L->top);  /* preserve original 'top' */
+
    ptrdiff_t ci_top = savestack(L, ci->top);  /* idem for 'ci->top' */
    lua_Debug ar;
    ar.event = event;
    ar.currentline = line;
@@ -308,8 +320,10 @@ void luaD_hook (lua_State *L, int event, int line,
      ci->u2.transferinfo.ftransfer = ftransfer;
      ci->u2.transferinfo.ntransfer = ntransfer;
    }
+
    if (isLua(ci) && L->top < ci->top)
+
      L->top = ci->top;  /* protect entire activation register */
    luaD_checkstack(L, LUA_MINSTACK);  /* ensure minimum stack size */
-
    if (L->top + LUA_MINSTACK > ci->top)
+
    if (ci->top < L->top + LUA_MINSTACK)
      ci->top = L->top + LUA_MINSTACK;
    L->allowhook = 0;  /* cannot call hooks inside a hook */
    ci->callstatus |= mask;
@@ -331,38 +345,40 @@ void luaD_hook (lua_State *L, int event, int line,
** active.
*/
void luaD_hookcall (lua_State *L, CallInfo *ci) {
-
  int hook = (ci->callstatus & CIST_TAIL) ? LUA_HOOKTAILCALL : LUA_HOOKCALL;
-
  Proto *p;
-
  if (!(L->hookmask & LUA_MASKCALL))  /* some other hook? */
-
    return;  /* don't call hook */
-
  p = clLvalue(s2v(ci->func))->p;
-
  L->top = ci->top;  /* prepare top */
-
  ci->u.l.savedpc++;  /* hooks assume 'pc' is already incremented */
-
  luaD_hook(L, hook, -1, 1, p->numparams);
-
  ci->u.l.savedpc--;  /* correct 'pc' */
-
}
-

-

-
static StkId rethook (lua_State *L, CallInfo *ci, StkId firstres, int nres) {
-
  ptrdiff_t oldtop = savestack(L, L->top);  /* hook may change top */
-
  int delta = 0;
-
  if (isLuacode(ci)) {
+
  L->oldpc = 0;  /* set 'oldpc' for new function */
+
  if (L->hookmask & LUA_MASKCALL) {  /* is call hook on? */
+
    int event = (ci->callstatus & CIST_TAIL) ? LUA_HOOKTAILCALL
+
                                             : LUA_HOOKCALL;
    Proto *p = ci_func(ci)->p;
-
    if (p->is_vararg)
-
      delta = ci->u.l.nextraargs + p->numparams + 1;
-
    if (L->top < ci->top)
-
      L->top = ci->top;  /* correct top to run hook */
+
    ci->u.l.savedpc++;  /* hooks assume 'pc' is already incremented */
+
    luaD_hook(L, event, -1, 1, p->numparams);
+
    ci->u.l.savedpc--;  /* correct 'pc' */
  }
+
}
+

+

+
/*
+
** Executes a return hook for Lua and C functions and sets/corrects
+
** 'oldpc'. (Note that this correction is needed by the line hook, so it
+
** is done even when return hooks are off.)
+
*/
+
static void rethook (lua_State *L, CallInfo *ci, int nres) {
  if (L->hookmask & LUA_MASKRET) {  /* is return hook on? */
+
    StkId firstres = L->top - nres;  /* index of first result */
+
    int delta = 0;  /* correction for vararg functions */
    int ftransfer;
+
    if (isLua(ci)) {
+
      Proto *p = ci_func(ci)->p;
+
      if (p->is_vararg)
+
        delta = ci->u.l.nextraargs + p->numparams + 1;
+
    }
    ci->func += delta;  /* if vararg, back to virtual 'func' */
    ftransfer = cast(unsigned short, firstres - ci->func);
    luaD_hook(L, LUA_HOOKRET, -1, ftransfer, nres);  /* call it */
    ci->func -= delta;
  }
  if (isLua(ci = ci->previous))
-
    L->oldpc = pcRel(ci->u.l.savedpc, ci_func(ci)->p);  /* update 'oldpc' */
-
  return restorestack(L, oldtop);
+
    L->oldpc = pcRel(ci->u.l.savedpc, ci_func(ci)->p);  /* set 'oldpc' */
}


@@ -371,15 +387,18 @@ static StkId rethook (lua_State *L, CallInfo *ci, StkId firstres, int nres) {
** stack, below original 'func', so that 'luaD_precall' can call it. Raise
** an error if there is no '__call' metafield.
*/
-
void luaD_tryfuncTM (lua_State *L, StkId func) {
-
  const TValue *tm = luaT_gettmbyobj(L, s2v(func), TM_CALL);
+
StkId luaD_tryfuncTM (lua_State *L, StkId func) {
+
  const TValue *tm;
  StkId p;
-
  if (unlikely(ttisnil(tm)))
-
    luaG_typeerror(L, s2v(func), "call");  /* nothing to call */
+
  checkstackGCp(L, 1, func);  /* space for metamethod */
+
  tm = luaT_gettmbyobj(L, s2v(func), TM_CALL);  /* (after previous GC) */
+
  if (l_unlikely(ttisnil(tm)))
+
    luaG_callerror(L, s2v(func));  /* nothing to call */
  for (p = L->top; p > func; p--)  /* open space for metamethod */
    setobjs2s(L, p, p-1);
  L->top++;  /* stack space pre-allocated by the caller */
  setobj2s(L, func, tm);  /* metamethod is the new function to be called */
+
  return func;
}


@@ -389,7 +408,7 @@ void luaD_tryfuncTM (lua_State *L, StkId func) {
** expressions, multiple results for tail calls/single parameters)
** separated.
*/
-
static void moveresults (lua_State *L, StkId res, int nres, int wanted) {
+
l_sinline void moveresults (lua_State *L, StkId res, int nres, int wanted) {
  StkId firstresult;
  int i;
  switch (wanted) {  /* handle typical cases separately */
@@ -399,27 +418,34 @@ static void moveresults (lua_State *L, StkId res, int nres, int wanted) {
    case 1:  /* one value needed */
      if (nres == 0)   /* no results? */
        setnilvalue(s2v(res));  /* adjust with nil */
-
      else
+
      else  /* at least one result */
        setobjs2s(L, res, L->top - nres);  /* move it to proper place */
      L->top = res + 1;
      return;
    case LUA_MULTRET:
      wanted = nres;  /* we want all results */
      break;
-
    default:  /* multiple results (or to-be-closed variables) */
+
    default:  /* two/more results and/or to-be-closed variables */
      if (hastocloseCfunc(wanted)) {  /* to-be-closed variables? */
        ptrdiff_t savedres = savestack(L, res);
-
        luaF_close(L, res, LUA_OK);  /* may change the stack */
-
        res = restorestack(L, savedres);
-
        wanted = codeNresults(wanted);  /* correct value */
+
        L->ci->callstatus |= CIST_CLSRET;  /* in case of yields */
+
        L->ci->u2.nres = nres;
+
        luaF_close(L, res, CLOSEKTOP, 1);
+
        L->ci->callstatus &= ~CIST_CLSRET;
+
        if (L->hookmask)  /* if needed, call hook after '__close's */
+
          rethook(L, L->ci, nres);
+
        res = restorestack(L, savedres);  /* close and hook can move stack */
+
        wanted = decodeNresults(wanted);
        if (wanted == LUA_MULTRET)
-
          wanted = nres;
+
          wanted = nres;  /* we want all results */
      }
      break;
  }
+
  /* generic case */
  firstresult = L->top - nres;  /* index of first result */
-
  /* move all results to correct place */
-
  for (i = 0; i < nres && i < wanted; i++)
+
  if (nres > wanted)  /* extra results? */
+
    nres = wanted;  /* don't need them */
+
  for (i = 0; i < nres; i++)  /* move all results to correct place */
    setobjs2s(L, res + i, firstresult + i);
  for (; i < wanted; i++)  /* complete wanted number of results */
    setnilvalue(s2v(res + i));
@@ -428,15 +454,21 @@ static void moveresults (lua_State *L, StkId res, int nres, int wanted) {


/*
-
** Finishes a function call: calls hook if necessary, removes CallInfo,
-
** moves current number of results to proper place.
+
** Finishes a function call: calls hook if necessary, moves current
+
** number of results to proper place, and returns to previous call
+
** info. If function has to close variables, hook must be called after
+
** that.
*/
void luaD_poscall (lua_State *L, CallInfo *ci, int nres) {
-
  if (L->hookmask)
-
    L->top = rethook(L, ci, L->top - nres, nres);
-
  L->ci = ci->previous;  /* back to caller */
+
  int wanted = ci->nresults;
+
  if (l_unlikely(L->hookmask && !hastocloseCfunc(wanted)))
+
    rethook(L, ci, nres);
  /* move results to proper place */
-
  moveresults(L, ci->func, nres, ci->nresults);
+
  moveresults(L, ci->func, nres, wanted);
+
  /* function cannot be in any of these cases when returning */
+
  lua_assert(!(ci->callstatus &
+
        (CIST_HOOKED | CIST_YPCALL | CIST_FIN | CIST_TRAN | CIST_CLSRET)));
+
  L->ci = ci->previous;  /* back to caller (after closing variables) */
}


@@ -444,27 +476,81 @@ void luaD_poscall (lua_State *L, CallInfo *ci, int nres) {
#define next_ci(L)  (L->ci->next ? L->ci->next : luaE_extendCI(L))


+
l_sinline CallInfo *prepCallInfo (lua_State *L, StkId func, int nret,
+
                                                int mask, StkId top) {
+
  CallInfo *ci = L->ci = next_ci(L);  /* new frame */
+
  ci->func = func;
+
  ci->nresults = nret;
+
  ci->callstatus = mask;
+
  ci->top = top;
+
  return ci;
+
}
+

+

+
/*
+
** precall for C functions
+
*/
+
l_sinline int precallC (lua_State *L, StkId func, int nresults,
+
                                            lua_CFunction f) {
+
  int n;  /* number of returns */
+
  CallInfo *ci;
+
  checkstackGCp(L, LUA_MINSTACK, func);  /* ensure minimum stack size */
+
  L->ci = ci = prepCallInfo(L, func, nresults, CIST_C,
+
                               L->top + LUA_MINSTACK);
+
  lua_assert(ci->top <= L->stack_last);
+
  if (l_unlikely(L->hookmask & LUA_MASKCALL)) {
+
    int narg = cast_int(L->top - func) - 1;
+
    luaD_hook(L, LUA_HOOKCALL, -1, 1, narg);
+
  }
+
  lua_unlock(L);
+
  n = (*f)(L);  /* do the actual call */
+
  lua_lock(L);
+
  api_checknelems(L, n);
+
  luaD_poscall(L, ci, n);
+
  return n;
+
}
+

+

/*
** Prepare a function for a tail call, building its call info on top
** of the current call info. 'narg1' is the number of arguments plus 1
-
** (so that it includes the function itself).
+
** (so that it includes the function itself). Return the number of
+
** results, if it was a C function, or -1 for a Lua function.
*/
-
void luaD_pretailcall (lua_State *L, CallInfo *ci, StkId func, int narg1) {
-
  Proto *p = clLvalue(s2v(func))->p;
-
  int fsize = p->maxstacksize;  /* frame size */
-
  int nfixparams = p->numparams;
-
  int i;
-
  for (i = 0; i < narg1; i++)  /* move down function and arguments */
-
    setobjs2s(L, ci->func + i, func + i);
-
  checkstackGC(L, fsize);
-
  func = ci->func;  /* moved-down function */
-
  for (; narg1 <= nfixparams; narg1++)
-
    setnilvalue(s2v(func + narg1));  /* complete missing arguments */
-
  ci->top = func + 1 + fsize;  /* top for new function */
-
  lua_assert(ci->top <= L->stack_last);
-
  ci->u.l.savedpc = p->code;  /* starting point */
-
  ci->callstatus |= CIST_TAIL;
-
  L->top = func + narg1;  /* set top */
+
int luaD_pretailcall (lua_State *L, CallInfo *ci, StkId func,
+
                                    int narg1, int delta) {
+
 retry:
+
  switch (ttypetag(s2v(func))) {
+
    case LUA_VCCL:  /* C closure */
+
      return precallC(L, func, LUA_MULTRET, clCvalue(s2v(func))->f);
+
    case LUA_VLCF:  /* light C function */
+
      return precallC(L, func, LUA_MULTRET, fvalue(s2v(func)));
+
    case LUA_VLCL: {  /* Lua function */
+
      Proto *p = clLvalue(s2v(func))->p;
+
      int fsize = p->maxstacksize;  /* frame size */
+
      int nfixparams = p->numparams;
+
      int i;
+
      checkstackGCp(L, fsize - delta, func);
+
      ci->func -= delta;  /* restore 'func' (if vararg) */
+
      for (i = 0; i < narg1; i++)  /* move down function and arguments */
+
        setobjs2s(L, ci->func + i, func + i);
+
      func = ci->func;  /* moved-down function */
+
      for (; narg1 <= nfixparams; narg1++)
+
        setnilvalue(s2v(func + narg1));  /* complete missing arguments */
+
      ci->top = func + 1 + fsize;  /* top for new function */
+
      lua_assert(ci->top <= L->stack_last);
+
      ci->u.l.savedpc = p->code;  /* starting point */
+
      ci->callstatus |= CIST_TAIL;
+
      L->top = func + narg1;  /* set top */
+
      return -1;
+
    }
+
    default: {  /* not a function */
+
      func = luaD_tryfuncTM(L, func);  /* try to get '__call' metamethod */
+
      /* return luaD_pretailcall(L, ci, func, narg1 + 1, delta); */
+
      narg1++;
+
      goto retry;  /* try again */
+
    }
+
  }
}


@@ -477,35 +563,14 @@ void luaD_pretailcall (lua_State *L, CallInfo *ci, StkId func, int narg1) {
** original function position.
*/
CallInfo *luaD_precall (lua_State *L, StkId func, int nresults) {
-
  lua_CFunction f;
 retry:
  switch (ttypetag(s2v(func))) {
    case LUA_VCCL:  /* C closure */
-
      f = clCvalue(s2v(func))->f;
-
      goto Cfunc;
+
      precallC(L, func, nresults, clCvalue(s2v(func))->f);
+
      return NULL;
    case LUA_VLCF:  /* light C function */
-
      f = fvalue(s2v(func));
-
     Cfunc: {
-
      int n;  /* number of returns */
-
      CallInfo *ci;
-
      checkstackGCp(L, LUA_MINSTACK, func);  /* ensure minimum stack size */
-
      L->ci = ci = next_ci(L);
-
      ci->nresults = nresults;
-
      ci->callstatus = CIST_C;
-
      ci->top = L->top + LUA_MINSTACK;
-
      ci->func = func;
-
      lua_assert(ci->top <= L->stack_last);
-
      if (L->hookmask & LUA_MASKCALL) {
-
        int narg = cast_int(L->top - func) - 1;
-
        luaD_hook(L, LUA_HOOKCALL, -1, 1, narg);
-
      }
-
      lua_unlock(L);
-
      n = (*f)(L);  /* do the actual call */
-
      lua_lock(L);
-
      api_checknelems(L, n);
-
      luaD_poscall(L, ci, n);
+
      precallC(L, func, nresults, fvalue(s2v(func)));
      return NULL;
-
    }
    case LUA_VLCL: {  /* Lua function */
      CallInfo *ci;
      Proto *p = clLvalue(s2v(func))->p;
@@ -513,20 +578,16 @@ CallInfo *luaD_precall (lua_State *L, StkId func, int nresults) {
      int nfixparams = p->numparams;
      int fsize = p->maxstacksize;  /* frame size */
      checkstackGCp(L, fsize, func);
-
      L->ci = ci = next_ci(L);
-
      ci->nresults = nresults;
+
      L->ci = ci = prepCallInfo(L, func, nresults, 0, func + 1 + fsize);
      ci->u.l.savedpc = p->code;  /* starting point */
-
      ci->top = func + 1 + fsize;
-
      ci->func = func;
-
      L->ci = ci;
      for (; narg < nfixparams; narg++)
        setnilvalue(s2v(L->top++));  /* complete missing arguments */
      lua_assert(ci->top <= L->stack_last);
      return ci;
    }
    default: {  /* not a function */
-
      checkstackGCp(L, 1, func);  /* space for metamethod */
-
      luaD_tryfuncTM(L, func);  /* try to get '__call' metamethod */
+
      func = luaD_tryfuncTM(L, func);  /* try to get '__call' metamethod */
+
      /* return luaD_precall(L, func, nresults); */
      goto retry;  /* try again with metamethod */
    }
  }
@@ -538,10 +599,10 @@ CallInfo *luaD_precall (lua_State *L, StkId func, int nresults) {
** number of recursive invocations in the C stack) or nyci (the same
** plus increment number of non-yieldable calls).
*/
-
static void ccall (lua_State *L, StkId func, int nResults, int inc) {
+
l_sinline void ccall (lua_State *L, StkId func, int nResults, int inc) {
  CallInfo *ci;
  L->nCcalls += inc;
-
  if (unlikely(getCcalls(L) >= LUAI_MAXCCALLS))
+
  if (l_unlikely(getCcalls(L) >= LUAI_MAXCCALLS))
    luaE_checkcstack(L);
  if ((ci = luaD_precall(L, func, nResults)) != NULL) {  /* Lua function? */
    ci->callstatus = CIST_FRESH;  /* mark that it is a "fresh" execute */
@@ -568,27 +629,74 @@ void luaD_callnoyield (lua_State *L, StkId func, int nResults) {


/*
-
** Completes the execution of an interrupted C function, calling its
-
** continuation function.
+
** Finish the job of 'lua_pcallk' after it was interrupted by an yield.
+
** (The caller, 'finishCcall', does the final call to 'adjustresults'.)
+
** The main job is to complete the 'luaD_pcall' called by 'lua_pcallk'.
+
** If a '__close' method yields here, eventually control will be back
+
** to 'finishCcall' (when that '__close' method finally returns) and
+
** 'finishpcallk' will run again and close any still pending '__close'
+
** methods. Similarly, if a '__close' method errs, 'precover' calls
+
** 'unroll' which calls ''finishCcall' and we are back here again, to
+
** close any pending '__close' methods.
+
** Note that, up to the call to 'luaF_close', the corresponding
+
** 'CallInfo' is not modified, so that this repeated run works like the
+
** first one (except that it has at least one less '__close' to do). In
+
** particular, field CIST_RECST preserves the error status across these
+
** multiple runs, changing only if there is a new error.
*/
-
static void finishCcall (lua_State *L, int status) {
-
  CallInfo *ci = L->ci;
-
  int n;
-
  /* must have a continuation and must be able to call it */
-
  lua_assert(ci->u.c.k != NULL && yieldable(L));
-
  /* error status can only happen in a protected call */
-
  lua_assert((ci->callstatus & CIST_YPCALL) || status == LUA_YIELD);
-
  if (ci->callstatus & CIST_YPCALL) {  /* was inside a pcall? */
-
    ci->callstatus &= ~CIST_YPCALL;  /* continuation is also inside it */
-
    L->errfunc = ci->u.c.old_errfunc;  /* with the same error function */
-
  }
-
  /* finish 'lua_callk'/'lua_pcall'; CIST_YPCALL and 'errfunc' already
-
     handled */
-
  adjustresults(L, ci->nresults);
-
  lua_unlock(L);
-
  n = (*ci->u.c.k)(L, status, ci->u.c.ctx);  /* call continuation function */
-
  lua_lock(L);
-
  api_checknelems(L, n);
+
static int finishpcallk (lua_State *L,  CallInfo *ci) {
+
  int status = getcistrecst(ci);  /* get original status */
+
  if (l_likely(status == LUA_OK))  /* no error? */
+
    status = LUA_YIELD;  /* was interrupted by an yield */
+
  else {  /* error */
+
    StkId func = restorestack(L, ci->u2.funcidx);
+
    L->allowhook = getoah(ci->callstatus);  /* restore 'allowhook' */
+
    luaF_close(L, func, status, 1);  /* can yield or raise an error */
+
    func = restorestack(L, ci->u2.funcidx);  /* stack may be moved */
+
    luaD_seterrorobj(L, status, func);
+
    luaD_shrinkstack(L);   /* restore stack size in case of overflow */
+
    setcistrecst(ci, LUA_OK);  /* clear original status */
+
  }
+
  ci->callstatus &= ~CIST_YPCALL;
+
  L->errfunc = ci->u.c.old_errfunc;
+
  /* if it is here, there were errors or yields; unlike 'lua_pcallk',
+
     do not change status */
+
  return status;
+
}
+

+

+
/*
+
** Completes the execution of a C function interrupted by an yield.
+
** The interruption must have happened while the function was either
+
** closing its tbc variables in 'moveresults' or executing
+
** 'lua_callk'/'lua_pcallk'. In the first case, it just redoes
+
** 'luaD_poscall'. In the second case, the call to 'finishpcallk'
+
** finishes the interrupted execution of 'lua_pcallk'.  After that, it
+
** calls the continuation of the interrupted function and finally it
+
** completes the job of the 'luaD_call' that called the function.  In
+
** the call to 'adjustresults', we do not know the number of results
+
** of the function called by 'lua_callk'/'lua_pcallk', so we are
+
** conservative and use LUA_MULTRET (always adjust).
+
*/
+
static void finishCcall (lua_State *L, CallInfo *ci) {
+
  int n;  /* actual number of results from C function */
+
  if (ci->callstatus & CIST_CLSRET) {  /* was returning? */
+
    lua_assert(hastocloseCfunc(ci->nresults));
+
    n = ci->u2.nres;  /* just redo 'luaD_poscall' */
+
    /* don't need to reset CIST_CLSRET, as it will be set again anyway */
+
  }
+
  else {
+
    int status = LUA_YIELD;  /* default if there were no errors */
+
    /* must have a continuation and must be able to call it */
+
    lua_assert(ci->u.c.k != NULL && yieldable(L));
+
    if (ci->callstatus & CIST_YPCALL)   /* was inside a 'lua_pcallk'? */
+
      status = finishpcallk(L, ci);  /* finish it */
+
    adjustresults(L, LUA_MULTRET);  /* finish 'lua_callk' */
+
    lua_unlock(L);
+
    n = (*ci->u.c.k)(L, status, ci->u.c.ctx);  /* call continuation */
+
    lua_lock(L);
+
    api_checknelems(L, n);
+
  }
  luaD_poscall(L, ci, n);  /* finish 'luaD_call' */
}

@@ -596,18 +704,14 @@ static void finishCcall (lua_State *L, int status) {
/*
** Executes "full continuation" (everything in the stack) of a
** previously interrupted coroutine until the stack is empty (or another
-
** interruption long-jumps out of the loop). If the coroutine is
-
** recovering from an error, 'ud' points to the error status, which must
-
** be passed to the first continuation function (otherwise the default
-
** status is LUA_YIELD).
+
** interruption long-jumps out of the loop).
*/
static void unroll (lua_State *L, void *ud) {
  CallInfo *ci;
-
  if (ud != NULL)  /* error status? */
-
    finishCcall(L, *(int *)ud);  /* finish 'lua_pcallk' callee */
+
  UNUSED(ud);
  while ((ci = L->ci) != &L->base_ci) {  /* something in the stack */
    if (!isLua(ci))  /* C function? */
-
      finishCcall(L, LUA_YIELD);  /* complete its execution */
+
      finishCcall(L, ci);  /* complete its execution */
    else {  /* Lua function */
      luaV_finishOp(L);  /* finish interrupted instruction */
      luaV_execute(L, ci);  /* execute down to higher C 'boundary' */
@@ -631,28 +735,6 @@ static CallInfo *findpcall (lua_State *L) {


/*
-
** Recovers from an error in a coroutine. Finds a recover point (if
-
** there is one) and completes the execution of the interrupted
-
** 'luaD_pcall'. If there is no recover point, returns zero.
-
*/
-
static int recover (lua_State *L, int status) {
-
  StkId oldtop;
-
  CallInfo *ci = findpcall(L);
-
  if (ci == NULL) return 0;  /* no recovery point */
-
  /* "finish" luaD_pcall */
-
  oldtop = restorestack(L, ci->u2.funcidx);
-
  L->ci = ci;
-
  L->allowhook = getoah(ci->callstatus);  /* restore original 'allowhook' */
-
  status = luaF_close(L, oldtop, status);  /* may change the stack */
-
  oldtop = restorestack(L, ci->u2.funcidx);
-
  luaD_seterrorobj(L, status, oldtop);
-
  luaD_shrinkstack(L);   /* restore stack size in case of overflow */
-
  L->errfunc = ci->u.c.old_errfunc;
-
  return 1;  /* continue running the coroutine */
-
}
-

-

-
/*
** Signal an error in the call to 'lua_resume', not in the execution
** of the coroutine itself. (Such errors should not be handled by any
** coroutine error handler and should not kill the coroutine.)
@@ -678,13 +760,14 @@ static void resume (lua_State *L, void *ud) {
  StkId firstArg = L->top - n;  /* first argument */
  CallInfo *ci = L->ci;
  if (L->status == LUA_OK)  /* starting a coroutine? */
-
    ccall(L, firstArg - 1, LUA_MULTRET, 1);  /* just call its body */
+
    ccall(L, firstArg - 1, LUA_MULTRET, 0);  /* just call its body */
  else {  /* resuming from previous yield */
    lua_assert(L->status == LUA_YIELD);
    L->status = LUA_OK;  /* mark that it is running (again) */
-
    luaE_incCstack(L);  /* control the C stack */
-
    if (isLua(ci))  /* yielded inside a hook? */
+
    if (isLua(ci)) {  /* yielded inside a hook? */
+
      L->top = firstArg;  /* discard arguments */
      luaV_execute(L, ci);  /* just continue running Lua code */
+
    }
    else {  /* 'common' yield */
      if (ci->u.c.k != NULL) {  /* does it have a continuation function? */
        lua_unlock(L);
@@ -698,6 +781,26 @@ static void resume (lua_State *L, void *ud) {
  }
}

+

+
/*
+
** Unrolls a coroutine in protected mode while there are recoverable
+
** errors, that is, errors inside a protected call. (Any error
+
** interrupts 'unroll', and this loop protects it again so it can
+
** continue.) Stops with a normal end (status == LUA_OK), an yield
+
** (status == LUA_YIELD), or an unprotected error ('findpcall' doesn't
+
** find a recover point).
+
*/
+
static int precover (lua_State *L, int status) {
+
  CallInfo *ci;
+
  while (errorstatus(status) && (ci = findpcall(L)) != NULL) {
+
    L->ci = ci;  /* go down to recovery functions */
+
    setcistrecst(ci, status);  /* status to finish 'pcall' */
+
    status = luaD_rawrunprotected(L, unroll, NULL);
+
  }
+
  return status;
+
}
+

+

LUA_API int lua_resume (lua_State *L, lua_State *from, int nargs,
                                      int *nresults) {
  int status;
@@ -711,15 +814,15 @@ LUA_API int lua_resume (lua_State *L, lua_State *from, int nargs,
  else if (L->status != LUA_YIELD)  /* ended with errors? */
    return resume_error(L, "cannot resume dead coroutine", nargs);
  L->nCcalls = (from) ? getCcalls(from) : 0;
+
  if (getCcalls(L) >= LUAI_MAXCCALLS)
+
    return resume_error(L, "C stack overflow", nargs);
+
  L->nCcalls++;
  luai_userstateresume(L, nargs);
  api_checknelems(L, (L->status == LUA_OK) ? nargs + 1 : nargs);
  status = luaD_rawrunprotected(L, resume, &nargs);
   /* continue running after recoverable errors */
-
  while (errorstatus(status) && recover(L, status)) {
-
    /* unroll continuation */
-
    status = luaD_rawrunprotected(L, unroll, &status);
-
  }
-
  if (likely(!errorstatus(status)))
+
  status = precover(L, status);
+
  if (l_likely(!errorstatus(status)))
    lua_assert(status == L->status);  /* normal end or yield */
  else {  /* unrecoverable error */
    L->status = cast_byte(status);  /* mark thread as 'dead' */
@@ -745,22 +848,22 @@ LUA_API int lua_yieldk (lua_State *L, int nresults, lua_KContext ctx,
  lua_lock(L);
  ci = L->ci;
  api_checknelems(L, nresults);
-
  if (unlikely(!yieldable(L))) {
+
  if (l_unlikely(!yieldable(L))) {
    if (L != G(L)->mainthread)
      luaG_runerror(L, "attempt to yield across a C-call boundary");
    else
      luaG_runerror(L, "attempt to yield from outside a coroutine");
  }
  L->status = LUA_YIELD;
+
  ci->u2.nyield = nresults;  /* save number of results */
  if (isLua(ci)) {  /* inside a hook? */
    lua_assert(!isLuacode(ci));
+
    api_check(L, nresults == 0, "hooks cannot yield values");
    api_check(L, k == NULL, "hooks cannot continue after yielding");
-
    ci->u2.nyield = 0;  /* no results */
  }
  else {
    if ((ci->u.c.k = k) != NULL)  /* is there a continuation? */
      ci->u.c.ctx = ctx;  /* save context */
-
    ci->u2.nyield = nresults;  /* save number of results */
    luaD_throw(L, LUA_YIELD);
  }
  lua_assert(ci->callstatus & CIST_HOOKED);  /* must be inside a hook */
@@ -770,6 +873,45 @@ LUA_API int lua_yieldk (lua_State *L, int nresults, lua_KContext ctx,


/*
+
** Auxiliary structure to call 'luaF_close' in protected mode.
+
*/
+
struct CloseP {
+
  StkId level;
+
  int status;
+
};
+

+

+
/*
+
** Auxiliary function to call 'luaF_close' in protected mode.
+
*/
+
static void closepaux (lua_State *L, void *ud) {
+
  struct CloseP *pcl = cast(struct CloseP *, ud);
+
  luaF_close(L, pcl->level, pcl->status, 0);
+
}
+

+

+
/*
+
** Calls 'luaF_close' in protected mode. Return the original status
+
** or, in case of errors, the new status.
+
*/
+
int luaD_closeprotected (lua_State *L, ptrdiff_t level, int status) {
+
  CallInfo *old_ci = L->ci;
+
  lu_byte old_allowhooks = L->allowhook;
+
  for (;;) {  /* keep closing upvalues until no more errors */
+
    struct CloseP pcl;
+
    pcl.level = restorestack(L, level); pcl.status = status;
+
    status = luaD_rawrunprotected(L, &closepaux, &pcl);
+
    if (l_likely(status == LUA_OK))  /* no more errors? */
+
      return pcl.status;
+
    else {  /* an error occurred; restore saved state and repeat */
+
      L->ci = old_ci;
+
      L->allowhook = old_allowhooks;
+
    }
+
  }
+
}
+

+

+
/*
** Call the C function 'func' in protected mode, restoring basic
** thread information ('allowhook', etc.) and in particular
** its stack level in case of errors.
@@ -782,13 +924,11 @@ int luaD_pcall (lua_State *L, Pfunc func, void *u,
  ptrdiff_t old_errfunc = L->errfunc;
  L->errfunc = ef;
  status = luaD_rawrunprotected(L, func, u);
-
  if (unlikely(status != LUA_OK)) {  /* an error occurred? */
-
    StkId oldtop = restorestack(L, old_top);
+
  if (l_unlikely(status != LUA_OK)) {  /* an error occurred? */
    L->ci = old_ci;
    L->allowhook = old_allowhooks;
-
    status = luaF_close(L, oldtop, status);
-
    oldtop = restorestack(L, old_top);  /* previous call may change stack */
-
    luaD_seterrorobj(L, status, oldtop);
+
    status = luaD_closeprotected(L, old_top, status);
+
    luaD_seterrorobj(L, status, restorestack(L, old_top));
    luaD_shrinkstack(L);   /* restore stack size in case of overflow */
  }
  L->errfunc = old_errfunc;
modified external/lua/src/ldo.h
@@ -23,7 +23,7 @@
** at every check.
*/
#define luaD_checkstackaux(L,n,pre,pos)  \
-
	if (L->stack_last - L->top <= (n)) \
+
	if (l_unlikely(L->stack_last - L->top <= (n))) \
	  { pre; luaD_growstack(L, n, 1); pos; } \
        else { condmovestack(L,pre,pos); }

@@ -58,11 +58,12 @@ LUAI_FUNC int luaD_protectedparser (lua_State *L, ZIO *z, const char *name,
LUAI_FUNC void luaD_hook (lua_State *L, int event, int line,
                                        int fTransfer, int nTransfer);
LUAI_FUNC void luaD_hookcall (lua_State *L, CallInfo *ci);
-
LUAI_FUNC void luaD_pretailcall (lua_State *L, CallInfo *ci, StkId func, int n);
+
LUAI_FUNC int luaD_pretailcall (lua_State *L, CallInfo *ci, StkId func,                                                    int narg1, int delta);
LUAI_FUNC CallInfo *luaD_precall (lua_State *L, StkId func, int nResults);
LUAI_FUNC void luaD_call (lua_State *L, StkId func, int nResults);
LUAI_FUNC void luaD_callnoyield (lua_State *L, StkId func, int nResults);
-
LUAI_FUNC void luaD_tryfuncTM (lua_State *L, StkId func);
+
LUAI_FUNC StkId luaD_tryfuncTM (lua_State *L, StkId func);
+
LUAI_FUNC int luaD_closeprotected (lua_State *L, ptrdiff_t level, int status);
LUAI_FUNC int luaD_pcall (lua_State *L, Pfunc func, void *u,
                                        ptrdiff_t oldtop, ptrdiff_t ef);
LUAI_FUNC void luaD_poscall (lua_State *L, CallInfo *ci, int nres);
modified external/lua/src/lfunc.c
@@ -100,115 +100,83 @@ UpVal *luaF_findupval (lua_State *L, StkId level) {
}


-
static void callclose (lua_State *L, void *ud) {
-
  UNUSED(ud);
-
  luaD_callnoyield(L, L->top - 3, 0);
-
}
-

-

/*
-
** Prepare closing method plus its arguments for object 'obj' with
-
** error message 'err'. (This function assumes EXTRA_STACK.)
+
** Call closing method for object 'obj' with error message 'err'. The
+
** boolean 'yy' controls whether the call is yieldable.
+
** (This function assumes EXTRA_STACK.)
*/
-
static int prepclosingmethod (lua_State *L, TValue *obj, TValue *err) {
+
static void callclosemethod (lua_State *L, TValue *obj, TValue *err, int yy) {
  StkId top = L->top;
  const TValue *tm = luaT_gettmbyobj(L, obj, TM_CLOSE);
-
  if (ttisnil(tm))  /* no metamethod? */
-
    return 0;  /* nothing to call */
  setobj2s(L, top, tm);  /* will call metamethod... */
  setobj2s(L, top + 1, obj);  /* with 'self' as the 1st argument */
  setobj2s(L, top + 2, err);  /* and error msg. as 2nd argument */
  L->top = top + 3;  /* add function and arguments */
-
  return 1;
+
  if (yy)
+
    luaD_call(L, top, 0);
+
  else
+
    luaD_callnoyield(L, top, 0);
}


/*
-
** Raise an error with message 'msg', inserting the name of the
-
** local variable at position 'level' in the stack.
+
** Check whether object at given level has a close metamethod and raise
+
** an error if not.
*/
-
static void varerror (lua_State *L, StkId level, const char *msg) {
-
  int idx = cast_int(level - L->ci->func);
-
  const char *vname = luaG_findlocal(L, L->ci, idx, NULL);
-
  if (vname == NULL) vname = "?";
-
  luaG_runerror(L, msg, vname);
+
static void checkclosemth (lua_State *L, StkId level) {
+
  const TValue *tm = luaT_gettmbyobj(L, s2v(level), TM_CLOSE);
+
  if (ttisnil(tm)) {  /* no metamethod? */
+
    int idx = cast_int(level - L->ci->func);  /* variable index */
+
    const char *vname = luaG_findlocal(L, L->ci, idx, NULL);
+
    if (vname == NULL) vname = "?";
+
    luaG_runerror(L, "variable '%s' got a non-closable value", vname);
+
  }
}


/*
-
** Prepare and call a closing method. If status is OK, code is still
-
** inside the original protected call, and so any error will be handled
-
** there. Otherwise, a previous error already activated the original
-
** protected call, and so the call to the closing method must be
-
** protected here. (A status == CLOSEPROTECT behaves like a previous
-
** error, to also run the closing method in protected mode).
-
** If status is OK, the call to the closing method will be pushed
-
** at the top of the stack. Otherwise, values are pushed after
-
** the 'level' of the upvalue being closed, as everything after
-
** that won't be used again.
+
** Prepare and call a closing method.
+
** If status is CLOSEKTOP, the call to the closing method will be pushed
+
** at the top of the stack. Otherwise, values can be pushed right after
+
** the 'level' of the upvalue being closed, as everything after that
+
** won't be used again.
*/
-
static int callclosemth (lua_State *L, StkId level, int status) {
+
static void prepcallclosemth (lua_State *L, StkId level, int status, int yy) {
  TValue *uv = s2v(level);  /* value being closed */
-
  if (likely(status == LUA_OK)) {
-
    if (prepclosingmethod(L, uv, &G(L)->nilvalue))  /* something to call? */
-
      callclose(L, NULL);  /* call closing method */
-
    else if (!l_isfalse(uv))  /* non-closable non-false value? */
-
      varerror(L, level, "attempt to close non-closable variable '%s'");
-
  }
-
  else {  /* must close the object in protected mode */
-
    ptrdiff_t oldtop;
-
    level++;  /* space for error message */
-
    oldtop = savestack(L, level + 1);  /* top will be after that */
-
    luaD_seterrorobj(L, status, level);  /* set error message */
-
    if (prepclosingmethod(L, uv, s2v(level))) {  /* something to call? */
-
      int newstatus = luaD_pcall(L, callclose, NULL, oldtop, 0);
-
      if (newstatus != LUA_OK && status == CLOSEPROTECT)  /* first error? */
-
        status = newstatus;  /* this will be the new error */
-
      else {
-
        if (newstatus != LUA_OK)  /* suppressed error? */
-
          luaE_warnerror(L, "__close metamethod");
-
        /* leave original error (or nil) on top */
-
        L->top = restorestack(L, oldtop);
-
      }
-
    }
-
    /* else no metamethod; ignore this case and keep original error */
+
  TValue *errobj;
+
  if (status == CLOSEKTOP)
+
    errobj = &G(L)->nilvalue;  /* error object is nil */
+
  else {  /* 'luaD_seterrorobj' will set top to level + 2 */
+
    errobj = s2v(level + 1);  /* error object goes after 'uv' */
+
    luaD_seterrorobj(L, status, level + 1);  /* set error object */
  }
-
  return status;
+
  callclosemethod(L, uv, errobj, yy);
}


/*
-
** Try to create a to-be-closed upvalue
-
** (can raise a memory-allocation error)
+
** Maximum value for deltas in 'tbclist', dependent on the type
+
** of delta. (This macro assumes that an 'L' is in scope where it
+
** is used.)
*/
-
static void trynewtbcupval (lua_State *L, void *ud) {
-
  newupval(L, 1, cast(StkId, ud), &L->openupval);
-
}
+
#define MAXDELTA  \
+
	((256ul << ((sizeof(L->stack->tbclist.delta) - 1) * 8)) - 1)


/*
-
** Create a to-be-closed upvalue. If there is a memory error
-
** when creating the upvalue, the closing method must be called here,
-
** as there is no upvalue to call it later.
+
** Insert a variable in the list of to-be-closed variables.
*/
void luaF_newtbcupval (lua_State *L, StkId level) {
-
  TValue *obj = s2v(level);
-
  lua_assert(L->openupval == NULL || uplevel(L->openupval) < level);
-
  if (!l_isfalse(obj)) {  /* false doesn't need to be closed */
-
    int status;
-
    const TValue *tm = luaT_gettmbyobj(L, obj, TM_CLOSE);
-
    if (ttisnil(tm))  /* no metamethod? */
-
      varerror(L, level, "variable '%s' got a non-closable value");
-
    status = luaD_rawrunprotected(L, trynewtbcupval, level);
-
    if (unlikely(status != LUA_OK)) {  /* memory error creating upvalue? */
-
      lua_assert(status == LUA_ERRMEM);
-
      luaD_seterrorobj(L, LUA_ERRMEM, level + 1);  /* save error message */
-
      /* next call must succeed, as object is closable */
-
      prepclosingmethod(L, s2v(level), s2v(level + 1));
-
      callclose(L, NULL);  /* call closing method */
-
      luaD_throw(L, LUA_ERRMEM);  /* throw memory error */
-
    }
+
  lua_assert(level > L->tbclist);
+
  if (l_isfalse(s2v(level)))
+
    return;  /* false doesn't need to be closed */
+
  checkclosemth(L, level);  /* value must have a close method */
+
  while (cast_uint(level - L->tbclist) > MAXDELTA) {
+
    L->tbclist += MAXDELTA;  /* create a dummy node at maximum delta */
+
    L->tbclist->tbclist.delta = 0;
  }
+
  level->tbclist.delta = cast(unsigned short, level - L->tbclist);
+
  L->tbclist = level;
}


@@ -220,18 +188,16 @@ void luaF_unlinkupval (UpVal *uv) {
}


-
int luaF_close (lua_State *L, StkId level, int status) {
+
/*
+
** Close all upvalues up to the given stack level.
+
*/
+
void luaF_closeupval (lua_State *L, StkId level) {
  UpVal *uv;
-
  while ((uv = L->openupval) != NULL && uplevel(uv) >= level) {
+
  StkId upl;  /* stack index pointed by 'uv' */
+
  while ((uv = L->openupval) != NULL && (upl = uplevel(uv)) >= level) {
    TValue *slot = &uv->u.value;  /* new position for value */
    lua_assert(uplevel(uv) < L->top);
-
    if (uv->tbc && status != NOCLOSINGMETH) {
-
      /* must run closing method, which may change the stack */
-
      ptrdiff_t levelrel = savestack(L, level);
-
      status = callclosemth(L, uplevel(uv), status);
-
      level = restorestack(L, levelrel);
-
    }
-
    luaF_unlinkupval(uv);
+
    luaF_unlinkupval(uv);  /* remove upvalue from 'openupval' list */
    setobj(L, slot, uv->v);  /* move value to upvalue slot */
    uv->v = slot;  /* now current value lives here */
    if (!iswhite(uv)) {  /* neither white nor dead? */
@@ -239,7 +205,35 @@ int luaF_close (lua_State *L, StkId level, int status) {
      luaC_barrier(L, uv, slot);
    }
  }
-
  return status;
+
}
+

+

+
/*
+
** Remove firt element from the tbclist plus its dummy nodes.
+
*/
+
static void poptbclist (lua_State *L) {
+
  StkId tbc = L->tbclist;
+
  lua_assert(tbc->tbclist.delta > 0);  /* first element cannot be dummy */
+
  tbc -= tbc->tbclist.delta;
+
  while (tbc > L->stack && tbc->tbclist.delta == 0)
+
    tbc -= MAXDELTA;  /* remove dummy nodes */
+
  L->tbclist = tbc;
+
}
+

+

+
/*
+
** Close all upvalues and to-be-closed variables up to the given stack
+
** level.
+
*/
+
void luaF_close (lua_State *L, StkId level, int status, int yy) {
+
  ptrdiff_t levelrel = savestack(L, level);
+
  luaF_closeupval(L, level);  /* first, close the upvalues */
+
  while (L->tbclist >= level) {  /* traverse tbc's down to that level */
+
    StkId tbc = L->tbclist;  /* get variable index */
+
    poptbclist(L);  /* remove it from list */
+
    prepcallclosemth(L, tbc, status, yy);  /* close variable */
+
    level = restorestack(L, levelrel);
+
  }
}


modified external/lua/src/lfunc.h
@@ -42,15 +42,9 @@
#define MAXMISS		10


-
/*
-
** Special "status" for 'luaF_close'
-
*/
-

-
/* close upvalues without running their closing methods */
-
#define NOCLOSINGMETH	(-1)

-
/* close upvalues running all closing methods in protected mode */
-
#define CLOSEPROTECT	(-2)
+
/* special status to close upvalues preserving the top of the stack */
+
#define CLOSEKTOP	(-1)


LUAI_FUNC Proto *luaF_newproto (lua_State *L);
@@ -59,7 +53,8 @@ LUAI_FUNC LClosure *luaF_newLclosure (lua_State *L, int nupvals);
LUAI_FUNC void luaF_initupvals (lua_State *L, LClosure *cl);
LUAI_FUNC UpVal *luaF_findupval (lua_State *L, StkId level);
LUAI_FUNC void luaF_newtbcupval (lua_State *L, StkId level);
-
LUAI_FUNC int luaF_close (lua_State *L, StkId level, int status);
+
LUAI_FUNC void luaF_closeupval (lua_State *L, StkId level);
+
LUAI_FUNC void luaF_close (lua_State *L, StkId level, int status, int yy);
LUAI_FUNC void luaF_unlinkupval (UpVal *uv);
LUAI_FUNC void luaF_freeproto (lua_State *L, Proto *f);
LUAI_FUNC const char *luaF_getlocalname (const Proto *func, int local_number,
modified external/lua/src/lgc.c
@@ -906,18 +906,18 @@ static void GCTM (lua_State *L) {
  if (!notm(tm)) {  /* is there a finalizer? */
    int status;
    lu_byte oldah = L->allowhook;
-
    int running  = g->gcrunning;
+
    int oldgcstp  = g->gcstp;
+
    g->gcstp |= GCSTPGC;  /* avoid GC steps */
    L->allowhook = 0;  /* stop debug hooks during GC metamethod */
-
    g->gcrunning = 0;  /* avoid GC steps */
    setobj2s(L, L->top++, tm);  /* push finalizer... */
    setobj2s(L, L->top++, &v);  /* ... and its argument */
    L->ci->callstatus |= CIST_FIN;  /* will run a finalizer */
    status = luaD_pcall(L, dothecall, NULL, savestack(L, L->top - 2), 0);
    L->ci->callstatus &= ~CIST_FIN;  /* not running a finalizer anymore */
    L->allowhook = oldah;  /* restore hooks */
-
    g->gcrunning = running;  /* restore state */
-
    if (unlikely(status != LUA_OK)) {  /* error while running __gc? */
-
      luaE_warnerror(L, "__gc metamethod");
+
    g->gcstp = oldgcstp;  /* restore state */
+
    if (l_unlikely(status != LUA_OK)) {  /* error while running __gc? */
+
      luaE_warnerror(L, "__gc");
      L->top--;  /* pops error object */
    }
  }
@@ -1011,7 +1011,8 @@ static void correctpointers (global_State *g, GCObject *o) {
void luaC_checkfinalizer (lua_State *L, GCObject *o, Table *mt) {
  global_State *g = G(L);
  if (tofinalize(o) ||                 /* obj. is already marked... */
-
      gfasttm(g, mt, TM_GC) == NULL)   /* or has no finalizer? */
+
      gfasttm(g, mt, TM_GC) == NULL ||    /* or has no finalizer... */
+
      (g->gcstp & GCSTPCLS))                   /* or closing state? */
    return;  /* nothing to be done */
  else {  /* move 'o' to 'finobj' list */
    GCObject **p;
@@ -1502,12 +1503,13 @@ static void deletelist (lua_State *L, GCObject *p, GCObject *limit) {
*/
void luaC_freeallobjects (lua_State *L) {
  global_State *g = G(L);
+
  g->gcstp = GCSTPCLS;  /* no extra finalizers after here */
  luaC_changemode(L, KGC_INC);
  separatetobefnz(g, 1);  /* separate all objects with finalizers */
  lua_assert(g->finobj == NULL);
  callallpendingfinalizers(L);
  deletelist(L, g->allgc, obj2gco(g->mainthread));
-
  deletelist(L, g->finobj, NULL);
+
  lua_assert(g->finobj == NULL);  /* no new finalizers */
  deletelist(L, g->fixedgc, NULL);  /* collect fixed objects */
  lua_assert(g->strt.nuse == 0);
}
@@ -1575,52 +1577,64 @@ static int sweepstep (lua_State *L, global_State *g,

static lu_mem singlestep (lua_State *L) {
  global_State *g = G(L);
+
  lu_mem work;
+
  lua_assert(!g->gcstopem);  /* collector is not reentrant */
+
  g->gcstopem = 1;  /* no emergency collections while collecting */
  switch (g->gcstate) {
    case GCSpause: {
      restartcollection(g);
      g->gcstate = GCSpropagate;
-
      return 1;
+
      work = 1;
+
      break;
    }
    case GCSpropagate: {
      if (g->gray == NULL) {  /* no more gray objects? */
        g->gcstate = GCSenteratomic;  /* finish propagate phase */
-
        return 0;
+
        work = 0;
      }
      else
-
        return propagatemark(g);  /* traverse one gray object */
+
        work = propagatemark(g);  /* traverse one gray object */
+
      break;
    }
    case GCSenteratomic: {
-
      lu_mem work = atomic(L);  /* work is what was traversed by 'atomic' */
+
      work = atomic(L);  /* work is what was traversed by 'atomic' */
      entersweep(L);
      g->GCestimate = gettotalbytes(g);  /* first estimate */;
-
      return work;
+
      break;
    }
    case GCSswpallgc: {  /* sweep "regular" objects */
-
      return sweepstep(L, g, GCSswpfinobj, &g->finobj);
+
      work = sweepstep(L, g, GCSswpfinobj, &g->finobj);
+
      break;
    }
    case GCSswpfinobj: {  /* sweep objects with finalizers */
-
      return sweepstep(L, g, GCSswptobefnz, &g->tobefnz);
+
      work = sweepstep(L, g, GCSswptobefnz, &g->tobefnz);
+
      break;
    }
    case GCSswptobefnz: {  /* sweep objects to be finalized */
-
      return sweepstep(L, g, GCSswpend, NULL);
+
      work = sweepstep(L, g, GCSswpend, NULL);
+
      break;
    }
    case GCSswpend: {  /* finish sweeps */
      checkSizes(L, g);
      g->gcstate = GCScallfin;
-
      return 0;
+
      work = 0;
+
      break;
    }
    case GCScallfin: {  /* call remaining finalizers */
      if (g->tobefnz && !g->gcemergency) {
-
        int n = runafewfinalizers(L, GCFINMAX);
-
        return n * GCFINALIZECOST;
+
        g->gcstopem = 0;  /* ok collections during finalizers */
+
        work = runafewfinalizers(L, GCFINMAX) * GCFINALIZECOST;
      }
      else {  /* emergency mode or no more finalizers */
        g->gcstate = GCSpause;  /* finish collection */
-
        return 0;
+
        work = 0;
      }
+
      break;
    }
    default: lua_assert(0); return 0;
  }
+
  g->gcstopem = 0;
+
  return work;
}


@@ -1635,6 +1649,7 @@ void luaC_runtilstate (lua_State *L, int statesmask) {
}


+

/*
** Performs a basic incremental step. The debt and step size are
** converted from bytes to "units of work"; then the function loops
@@ -1666,7 +1681,7 @@ static void incstep (lua_State *L, global_State *g) {
void luaC_step (lua_State *L) {
  global_State *g = G(L);
  lua_assert(!g->gcemergency);
-
  if (g->gcrunning) {  /* running? */
+
  if (gcrunning(g)) {  /* running? */
    if(isdecGCmodegen(g))
      genstep(L, g);
    else
modified external/lua/src/lgc.h
@@ -148,6 +148,16 @@
*/
#define isdecGCmodegen(g)	(g->gckind == KGC_GEN || g->lastatomic != 0)

+

+
/*
+
** Control when GC is running:
+
*/
+
#define GCSTPUSR	1  /* bit true when GC stopped by user */
+
#define GCSTPGC		2  /* bit true when GC stopped by itself */
+
#define GCSTPCLS	4  /* bit true when closing Lua state */
+
#define gcrunning(g)	((g)->gcstp == 0)
+

+

/*
** Does one step of collection when debt becomes positive. 'pre'/'pos'
** allows some adjustments to be done only when needed. macro
modified external/lua/src/liolib.c
@@ -52,12 +52,6 @@ static int l_checkmode (const char *mode) {
** =======================================================
*/

-
#if !defined(l_checkmodep)
-
/* By default, Lua accepts only "r" or "w" as mode */
-
#define l_checkmodep(m)	((m[0] == 'r' || m[0] == 'w') && m[1] == '\0')
-
#endif
-

-

#if !defined(l_popen)		/* { */

#if defined(LUA_USE_POSIX)	/* { */
@@ -70,6 +64,12 @@ static int l_checkmode (const char *mode) {
#define l_popen(L,c,m)		(_popen(c,m))
#define l_pclose(L,file)	(_pclose(file))

+
#if !defined(l_checkmodep)
+
/* Windows accepts "[rw][bt]?" as valid modes */
+
#define l_checkmodep(m)	((m[0] == 'r' || m[0] == 'w') && \
+
  (m[1] == '\0' || ((m[1] == 'b' || m[1] == 't') && m[2] == '\0')))
+
#endif
+

#else				/* }{ */

/* ISO C definitions */
@@ -83,6 +83,12 @@ static int l_checkmode (const char *mode) {

#endif				/* } */

+

+
#if !defined(l_checkmodep)
+
/* By default, Lua accepts only "r" or "w" as valid modes */
+
#define l_checkmodep(m)        ((m[0] == 'r' || m[0] == 'w') && m[1] == '\0')
+
#endif
+

/* }====================================================== */


@@ -180,7 +186,7 @@ static int f_tostring (lua_State *L) {

static FILE *tofile (lua_State *L) {
  LStream *p = tolstream(L);
-
  if (isclosed(p))
+
  if (l_unlikely(isclosed(p)))
    luaL_error(L, "attempt to use a closed file");
  lua_assert(p->f);
  return p->f;
@@ -255,7 +261,7 @@ static LStream *newfile (lua_State *L) {
static void opencheck (lua_State *L, const char *fname, const char *mode) {
  LStream *p = newfile(L);
  p->f = fopen(fname, mode);
-
  if (p->f == NULL)
+
  if (l_unlikely(p->f == NULL))
    luaL_error(L, "cannot open file '%s' (%s)", fname, strerror(errno));
}

@@ -303,7 +309,7 @@ static FILE *getiofile (lua_State *L, const char *findex) {
  LStream *p;
  lua_getfield(L, LUA_REGISTRYINDEX, findex);
  p = (LStream *)lua_touserdata(L, -1);
-
  if (isclosed(p))
+
  if (l_unlikely(isclosed(p)))
    luaL_error(L, "default %s file is closed", findex + IOPREF_LEN);
  return p->f;
}
@@ -430,7 +436,7 @@ typedef struct {
** Add current char to buffer (if not out of space) and read next one
*/
static int nextc (RN *rn) {
-
  if (rn->n >= L_MAXLENNUM) {  /* buffer overflow? */
+
  if (l_unlikely(rn->n >= L_MAXLENNUM)) {  /* buffer overflow? */
    rn->buff[0] = '\0';  /* invalidate result */
    return 0;  /* fail */
  }
@@ -493,8 +499,8 @@ static int read_number (lua_State *L, FILE *f) {
  ungetc(rn.c, rn.f);  /* unread look-ahead char */
  l_unlockfile(rn.f);
  rn.buff[rn.n] = '\0';  /* finish string */
-
  if (lua_stringtonumber(L, rn.buff))  /* is this a valid number? */
-
    return 1;  /* ok */
+
  if (l_likely(lua_stringtonumber(L, rn.buff)))
+
    return 1;  /* ok, it is a valid number */
  else {  /* invalid format */
   lua_pushnil(L);  /* "result" to be removed */
   return 0;  /* read fails */
@@ -670,7 +676,8 @@ static int g_write (lua_State *L, FILE *f, int arg) {
      status = status && (fwrite(s, sizeof(char), l, f) == l);
    }
  }
-
  if (status) return 1;  /* file handle already on stack top */
+
  if (l_likely(status))
+
    return 1;  /* file handle already on stack top */
  else return luaL_fileresult(L, status, NULL);
}

@@ -697,7 +704,7 @@ static int f_seek (lua_State *L) {
  luaL_argcheck(L, (lua_Integer)offset == p3, 3,
                  "not an integer in proper range");
  op = l_fseek(f, offset, mode[op]);
-
  if (op)
+
  if (l_unlikely(op))
    return luaL_fileresult(L, 0, NULL);  /* error */
  else {
    lua_pushinteger(L, (lua_Integer)l_ftell(f));
modified external/lua/src/llex.c
@@ -122,26 +122,29 @@ l_noret luaX_syntaxerror (LexState *ls, const char *msg) {


/*
-
** creates a new string and anchors it in scanner's table so that
-
** it will not be collected until the end of the compilation
-
** (by that time it should be anchored somewhere)
+
** Creates a new string and anchors it in scanner's table so that it
+
** will not be collected until the end of the compilation; by that time
+
** it should be anchored somewhere. It also internalizes long strings,
+
** ensuring there is only one copy of each unique string.  The table
+
** here is used as a set: the string enters as the key, while its value
+
** is irrelevant. We use the string itself as the value only because it
+
** is a TValue readly available. Later, the code generation can change
+
** this value.
*/
TString *luaX_newstring (LexState *ls, const char *str, size_t l) {
  lua_State *L = ls->L;
-
  TValue *o;  /* entry for 'str' */
  TString *ts = luaS_newlstr(L, str, l);  /* create new string */
-
  setsvalue2s(L, L->top++, ts);  /* temporarily anchor it in stack */
-
  o = luaH_set(L, ls->h, s2v(L->top - 1));
-
  if (isempty(o)) {  /* not in use yet? */
-
    /* boolean value does not need GC barrier;
-
       table is not a metatable, so it does not need to invalidate cache */
-
    setbtvalue(o);  /* t[string] = true */
+
  const TValue *o = luaH_getstr(ls->h, ts);
+
  if (!ttisnil(o))  /* string already present? */
+
    ts = keystrval(nodefromval(o));  /* get saved copy */
+
  else {  /* not in use yet */
+
    TValue *stv = s2v(L->top++);  /* reserve stack space for string */
+
    setsvalue(L, stv, ts);  /* temporarily anchor the string */
+
    luaH_finishset(L, ls->h, stv, o, stv);  /* t[string] = string */
+
    /* table is not a metatable, so it does not need to invalidate cache */
    luaC_checkGC(L);
+
    L->top--;  /* remove string from stack */
  }
-
  else {  /* string already present */
-
    ts = keystrval(nodefromval(o));  /* re-use value previously stored */
-
  }
-
  L->top--;  /* remove string from stack */
  return ts;
}

modified external/lua/src/llimits.h
@@ -150,35 +150,33 @@ typedef LUAI_UACINT l_uacInt;


/*
-
** macros to improve jump prediction (used mainly for error handling)
+
** non-return type
*/
-
#if !defined(likely)
+
#if !defined(l_noret)

#if defined(__GNUC__)
-
#define likely(x)	(__builtin_expect(((x) != 0), 1))
-
#define unlikely(x)	(__builtin_expect(((x) != 0), 0))
+
#define l_noret		void __attribute__((noreturn))
+
#elif defined(_MSC_VER) && _MSC_VER >= 1200
+
#define l_noret		void __declspec(noreturn)
#else
-
#define likely(x)	(x)
-
#define unlikely(x)	(x)
+
#define l_noret		void
#endif

#endif


/*
-
** non-return type
+
** Inline functions
*/
-
#if !defined(l_noret)
-

-
#if defined(__GNUC__)
-
#define l_noret		void __attribute__((noreturn))
-
#elif defined(_MSC_VER) && _MSC_VER >= 1200
-
#define l_noret		void __declspec(noreturn)
+
#if !defined(LUA_USE_C89)
+
#define l_inline	inline
+
#elif defined(__GNUC__)
+
#define l_inline	__inline__
#else
-
#define l_noret		void
+
#define l_inline	/* empty */
#endif

-
#endif
+
#define l_sinline	static l_inline


/*
@@ -363,7 +361,7 @@ typedef l_uint32 Instruction;
#define condchangemem(L,pre,pos)	((void)0)
#else
#define condchangemem(L,pre,pos)  \
-
	{ if (G(L)->gcrunning) { pre; luaC_fullgc(L, 0); pos; } }
+
	{ if (gcrunning(G(L))) { pre; luaC_fullgc(L, 0); pos; } }
#endif

#endif
modified external/lua/src/lmathlib.c
@@ -73,7 +73,7 @@ static int math_atan (lua_State *L) {
static int math_toint (lua_State *L) {
  int valid;
  lua_Integer n = lua_tointegerx(L, 1, &valid);
-
  if (valid)
+
  if (l_likely(valid))
    lua_pushinteger(L, n);
  else {
    luaL_checkany(L, 1);
@@ -175,7 +175,8 @@ static int math_log (lua_State *L) {
    lua_Number base = luaL_checknumber(L, 2);
#if !defined(LUA_USE_C89)
    if (base == l_mathop(2.0))
-
      res = l_mathop(log2)(x); else
+
      res = l_mathop(log2)(x);
+
    else
#endif
    if (base == l_mathop(10.0))
      res = l_mathop(log10)(x);
@@ -474,7 +475,7 @@ static lua_Number I2d (Rand64 x) {

/* 2^(-FIGS) = 1.0 / 2^30 / 2^3 / 2^(FIGS-33) */
#define scaleFIG  \
-
	((lua_Number)1.0 / (UONE << 30) / 8.0 / (UONE << (FIGS - 33)))
+
    (l_mathop(1.0) / (UONE << 30) / l_mathop(8.0) / (UONE << (FIGS - 33)))

/*
** use FIGS - 32 bits from lower half, throwing out the other
@@ -485,7 +486,7 @@ static lua_Number I2d (Rand64 x) {
/*
** higher 32 bits go after those (FIGS - 32) bits: shiftHI = 2^(FIGS - 32)
*/
-
#define shiftHI		((lua_Number)(UONE << (FIGS - 33)) * 2.0)
+
#define shiftHI		((lua_Number)(UONE << (FIGS - 33)) * l_mathop(2.0))


static lua_Number I2d (Rand64 x) {
modified external/lua/src/lmem.c
@@ -24,12 +24,12 @@

#if defined(EMERGENCYGCTESTS)
/*
-
** First allocation will fail whenever not building initial state
-
** and not shrinking a block. (This fail will trigger 'tryagain' and
-
** a full GC cycle at every allocation.)
+
** First allocation will fail whenever not building initial state.
+
** (This fail will trigger 'tryagain' and a full GC cycle at every
+
** allocation.)
*/
static void *firsttry (global_State *g, void *block, size_t os, size_t ns) {
-
  if (ttisnil(&g->nilvalue) && ns > os)
+
  if (completestate(g) && ns > 0)  /* frees never fail */
    return NULL;  /* fail */
  else  /* normal allocation */
    return (*g->frealloc)(g->ud, block, os, ns);
@@ -83,7 +83,7 @@ void *luaM_growaux_ (lua_State *L, void *block, int nelems, int *psize,
  if (nelems + 1 <= size)  /* does one extra element still fit? */
    return block;  /* nothing to be done */
  if (size >= limit / 2) {  /* cannot double it? */
-
    if (unlikely(size >= limit))  /* cannot grow even a little? */
+
    if (l_unlikely(size >= limit))  /* cannot grow even a little? */
      luaG_runerror(L, "too many %s (limit is %d)", what, limit);
    size = limit;  /* still have at least one free place */
  }
@@ -138,15 +138,17 @@ void luaM_free_ (lua_State *L, void *block, size_t osize) {


/*
-
** In case of allocation fail, this function will call the GC to try
-
** to free some memory and then try the allocation again.
-
** (It should not be called when shrinking a block, because then the
-
** interpreter may be in the middle of a collection step.)
+
** In case of allocation fail, this function will do an emergency
+
** collection to free some memory and then try the allocation again.
+
** The GC should not be called while state is not fully built, as the
+
** collector is not yet fully initialized. Also, it should not be called
+
** when 'gcstopem' is true, because then the interpreter is in the
+
** middle of a collection step.
*/
static void *tryagain (lua_State *L, void *block,
                       size_t osize, size_t nsize) {
  global_State *g = G(L);
-
  if (ttisnil(&g->nilvalue)) {  /* is state fully build? */
+
  if (completestate(g) && !g->gcstopem) {
    luaC_fullgc(L, 1);  /* try to free some memory... */
    return (*g->frealloc)(g->ud, block, osize, nsize);  /* try again */
  }
@@ -156,17 +158,14 @@ static void *tryagain (lua_State *L, void *block,

/*
** Generic allocation routine.
-
** If allocation fails while shrinking a block, do not try again; the
-
** GC shrinks some blocks and it is not reentrant.
*/
void *luaM_realloc_ (lua_State *L, void *block, size_t osize, size_t nsize) {
  void *newblock;
  global_State *g = G(L);
  lua_assert((osize == 0) == (block == NULL));
  newblock = firsttry(g, block, osize, nsize);
-
  if (unlikely(newblock == NULL && nsize > 0)) {
-
    if (nsize > osize)  /* not shrinking a block? */
-
      newblock = tryagain(L, block, osize, nsize);
+
  if (l_unlikely(newblock == NULL && nsize > 0)) {
+
    newblock = tryagain(L, block, osize, nsize);
    if (newblock == NULL)  /* still no memory? */
      return NULL;  /* do not update 'GCdebt' */
  }
@@ -179,7 +178,7 @@ void *luaM_realloc_ (lua_State *L, void *block, size_t osize, size_t nsize) {
void *luaM_saferealloc_ (lua_State *L, void *block, size_t osize,
                                                    size_t nsize) {
  void *newblock = luaM_realloc_(L, block, osize, nsize);
-
  if (unlikely(newblock == NULL && nsize > 0))  /* allocation failed? */
+
  if (l_unlikely(newblock == NULL && nsize > 0))  /* allocation failed? */
    luaM_error(L);
  return newblock;
}
@@ -191,7 +190,7 @@ void *luaM_malloc_ (lua_State *L, size_t size, int tag) {
  else {
    global_State *g = G(L);
    void *newblock = firsttry(g, NULL, tag, size);
-
    if (unlikely(newblock == NULL)) {
+
    if (l_unlikely(newblock == NULL)) {
      newblock = tryagain(L, NULL, tag, size);
      if (newblock == NULL)
        luaM_error(L);
modified external/lua/src/loadlib.c
@@ -132,14 +132,16 @@ static void lsys_unloadlib (void *lib) {

static void *lsys_load (lua_State *L, const char *path, int seeglb) {
  void *lib = dlopen(path, RTLD_NOW | (seeglb ? RTLD_GLOBAL : RTLD_LOCAL));
-
  if (lib == NULL) lua_pushstring(L, dlerror());
+
  if (l_unlikely(lib == NULL))
+
    lua_pushstring(L, dlerror());
  return lib;
}


static lua_CFunction lsys_sym (lua_State *L, void *lib, const char *sym) {
  lua_CFunction f = cast_func(dlsym(lib, sym));
-
  if (f == NULL) lua_pushstring(L, dlerror());
+
  if (l_unlikely(f == NULL))
+
    lua_pushstring(L, dlerror());
  return f;
}

@@ -410,7 +412,7 @@ static int ll_loadlib (lua_State *L) {
  const char *path = luaL_checkstring(L, 1);
  const char *init = luaL_checkstring(L, 2);
  int stat = lookforfunc(L, path, init);
-
  if (stat == 0)  /* no errors? */
+
  if (l_likely(stat == 0))  /* no errors? */
    return 1;  /* return the loaded function */
  else {  /* error; error message is on stack top */
    luaL_pushfail(L);
@@ -523,14 +525,14 @@ static const char *findfile (lua_State *L, const char *name,
  const char *path;
  lua_getfield(L, lua_upvalueindex(1), pname);
  path = lua_tostring(L, -1);
-
  if (path == NULL)
+
  if (l_unlikely(path == NULL))
    luaL_error(L, "'package.%s' must be a string", pname);
  return searchpath(L, name, path, ".", dirsep);
}


static int checkload (lua_State *L, int stat, const char *filename) {
-
  if (stat) {  /* module loaded successfully? */
+
  if (l_likely(stat)) {  /* module loaded successfully? */
    lua_pushstring(L, filename);  /* will be 2nd argument to module */
    return 2;  /* return open function and file name */
  }
@@ -623,13 +625,14 @@ static void findloader (lua_State *L, const char *name) {
  int i;
  luaL_Buffer msg;  /* to build error message */
  /* push 'package.searchers' to index 3 in the stack */
-
  if (lua_getfield(L, lua_upvalueindex(1), "searchers") != LUA_TTABLE)
+
  if (l_unlikely(lua_getfield(L, lua_upvalueindex(1), "searchers")
+
                 != LUA_TTABLE))
    luaL_error(L, "'package.searchers' must be a table");
  luaL_buffinit(L, &msg);
  /*  iterate over available searchers to find a loader */
  for (i = 1; ; i++) {
    luaL_addstring(&msg, "\n\t");  /* error-message prefix */
-
    if (lua_rawgeti(L, 3, i) == LUA_TNIL) {  /* no more searchers? */
+
    if (l_unlikely(lua_rawgeti(L, 3, i) == LUA_TNIL)) {  /* no more searchers? */
      lua_pop(L, 1);  /* remove nil */
      luaL_buffsub(&msg, 2);  /* remove prefix */
      luaL_pushresult(&msg);  /* create error message */
modified external/lua/src/lobject.c
@@ -164,7 +164,7 @@ static int isneg (const char **s) {
*/
static lua_Number lua_strx2number (const char *s, char **endptr) {
  int dot = lua_getlocaledecpoint();
-
  lua_Number r = 0.0;  /* result (accumulator) */
+
  lua_Number r = l_mathop(0.0);  /* result (accumulator) */
  int sigdig = 0;  /* number of significant digits */
  int nosigdig = 0;  /* number of non-significant digits */
  int e = 0;  /* exponent correction */
@@ -174,7 +174,7 @@ static lua_Number lua_strx2number (const char *s, char **endptr) {
  while (lisspace(cast_uchar(*s))) s++;  /* skip initial spaces */
  neg = isneg(&s);  /* check sign */
  if (!(*s == '0' && (*(s + 1) == 'x' || *(s + 1) == 'X')))  /* check '0x' */
-
    return 0.0;  /* invalid format (no '0x') */
+
    return l_mathop(0.0);  /* invalid format (no '0x') */
  for (s += 2; ; s++) {  /* skip '0x' and read numeral */
    if (*s == dot) {
      if (hasdot) break;  /* second dot? stop loop */
@@ -184,14 +184,14 @@ static lua_Number lua_strx2number (const char *s, char **endptr) {
      if (sigdig == 0 && *s == '0')  /* non-significant digit (zero)? */
        nosigdig++;
      else if (++sigdig <= MAXSIGDIG)  /* can read it without overflow? */
-
          r = (r * cast_num(16.0)) + luaO_hexavalue(*s);
+
          r = (r * l_mathop(16.0)) + luaO_hexavalue(*s);
      else e++; /* too many digits; ignore, but still count for exponent */
      if (hasdot) e--;  /* decimal digit? correct exponent */
    }
    else break;  /* neither a dot nor a digit */
  }
  if (nosigdig + sigdig == 0)  /* no digits? */
-
    return 0.0;  /* invalid format */
+
    return l_mathop(0.0);  /* invalid format */
  *endptr = cast_charp(s);  /* valid up to here */
  e *= 4;  /* each digit multiplies/divides value by 2^4 */
  if (*s == 'p' || *s == 'P') {  /* exponent part? */
@@ -200,7 +200,7 @@ static lua_Number lua_strx2number (const char *s, char **endptr) {
    s++;  /* skip 'p' */
    neg1 = isneg(&s);  /* sign */
    if (!lisdigit(cast_uchar(*s)))
-
      return 0.0;  /* invalid; must have at least one digit */
+
      return l_mathop(0.0);  /* invalid; must have at least one digit */
    while (lisdigit(cast_uchar(*s)))  /* read exponent */
      exp1 = exp1 * 10 + *(s++) - '0';
    if (neg1) exp1 = -exp1;
modified external/lua/src/lobject.h
@@ -68,7 +68,7 @@ typedef struct TValue {


#define val_(o)		((o)->value_)
-
#define valraw(o)	(&val_(o))
+
#define valraw(o)	(val_(o))


/* raw type tag of a TValue */
@@ -112,7 +112,7 @@ typedef struct TValue {
#define settt_(o,t)	((o)->tt_=(t))


-
/* main macro to copy values (from 'obj1' to 'obj2') */
+
/* main macro to copy values (from 'obj2' to 'obj1') */
#define setobj(L,obj1,obj2) \
	{ TValue *io1=(obj1); const TValue *io2=(obj2); \
          io1->value_ = io2->value_; settt_(io1, io2->tt_); \
@@ -136,10 +136,19 @@ typedef struct TValue {


/*
-
** Entries in the Lua stack
+
** Entries in a Lua stack. Field 'tbclist' forms a list of all
+
** to-be-closed variables active in this stack. Dummy entries are
+
** used when the distance between two tbc variables does not fit
+
** in an unsigned short. They are represented by delta==0, and
+
** their real delta is always the maximum value that fits in
+
** that field.
*/
typedef union StackValue {
  TValue val;
+
  struct {
+
    TValuefields;
+
    unsigned short delta;
+
  } tbclist;
} StackValue;


@@ -570,10 +579,11 @@ typedef struct Proto {
#define LUA_VCCL	makevariant(LUA_TFUNCTION, 2)  /* C closure */

#define ttisfunction(o)		checktype(o, LUA_TFUNCTION)
-
#define ttisclosure(o)		((rawtt(o) & 0x1F) == LUA_VLCL)
#define ttisLclosure(o)		checktag((o), ctb(LUA_VLCL))
#define ttislcf(o)		checktag((o), LUA_VLCF)
#define ttisCclosure(o)		checktag((o), ctb(LUA_VCCL))
+
#define ttisclosure(o)         (ttisLclosure(o) || ttisCclosure(o))
+


#define isLfunction(o)	ttisLclosure(o)

modified external/lua/src/lopcodes.h
@@ -190,7 +190,8 @@ enum OpMode {iABC, iABx, iAsBx, iAx, isJ}; /* basic instruction formats */


/*
-
** grep "ORDER OP" if you change these enums
+
** Grep "ORDER OP" if you change these enums. Opcodes marked with a (*)
+
** has extra descriptions in the notes after the enumeration.
*/

typedef enum {
@@ -203,7 +204,7 @@ OP_LOADF,/* A sBx R[A] := (lua_Number)sBx */
OP_LOADK,/*	A Bx	R[A] := K[Bx]					*/
OP_LOADKX,/*	A	R[A] := K[extra arg]				*/
OP_LOADFALSE,/*	A	R[A] := false					*/
-
OP_LFALSESKIP,/*A	R[A] := false; pc++				*/
+
OP_LFALSESKIP,/*A	R[A] := false; pc++	(*)			*/
OP_LOADTRUE,/*	A	R[A] := true					*/
OP_LOADNIL,/*	A B	R[A], R[A+1], ..., R[A+B] := nil		*/
OP_GETUPVAL,/*	A B	R[A] := UpValue[B]				*/
@@ -225,13 +226,13 @@ OP_SELF,/* A B C R[A+1] := R[B]; R[A] := R[B][RK(C):string] */

OP_ADDI,/*	A B sC	R[A] := R[B] + sC				*/

-
OP_ADDK,/*	A B C	R[A] := R[B] + K[C]				*/
-
OP_SUBK,/*	A B C	R[A] := R[B] - K[C]				*/
-
OP_MULK,/*	A B C	R[A] := R[B] * K[C]				*/
-
OP_MODK,/*	A B C	R[A] := R[B] % K[C]				*/
-
OP_POWK,/*	A B C	R[A] := R[B] ^ K[C]				*/
-
OP_DIVK,/*	A B C	R[A] := R[B] / K[C]				*/
-
OP_IDIVK,/*	A B C	R[A] := R[B] // K[C]				*/
+
OP_ADDK,/*	A B C	R[A] := R[B] + K[C]:number			*/
+
OP_SUBK,/*	A B C	R[A] := R[B] - K[C]:number			*/
+
OP_MULK,/*	A B C	R[A] := R[B] * K[C]:number			*/
+
OP_MODK,/*	A B C	R[A] := R[B] % K[C]:number			*/
+
OP_POWK,/*	A B C	R[A] := R[B] ^ K[C]:number			*/
+
OP_DIVK,/*	A B C	R[A] := R[B] / K[C]:number			*/
+
OP_IDIVK,/*	A B C	R[A] := R[B] // K[C]:number			*/

OP_BANDK,/*	A B C	R[A] := R[B] & K[C]:integer			*/
OP_BORK,/*	A B C	R[A] := R[B] | K[C]:integer			*/
@@ -254,7 +255,7 @@ OP_BXOR,/* A B C R[A] := R[B] ~ R[C] */
OP_SHL,/*	A B C	R[A] := R[B] << R[C]				*/
OP_SHR,/*	A B C	R[A] := R[B] >> R[C]				*/

-
OP_MMBIN,/*	A B C	call C metamethod over R[A] and R[B]		*/
+
OP_MMBIN,/*	A B C	call C metamethod over R[A] and R[B]	(*)	*/
OP_MMBINI,/*	A sB C k	call C metamethod over R[A] and sB	*/
OP_MMBINK,/*	A B C k		call C metamethod over R[A] and K[B]	*/

@@ -280,7 +281,7 @@ OP_GTI,/* A sB k if ((R[A] > sB) ~= k) then pc++ */
OP_GEI,/*	A sB k	if ((R[A] >= sB) ~= k) then pc++		*/

OP_TEST,/*	A k	if (not R[A] == k) then pc++			*/
-
OP_TESTSET,/*	A B k	if (not R[B] == k) then pc++ else R[A] := R[B]	*/
+
OP_TESTSET,/*	A B k	if (not R[B] == k) then pc++ else R[A] := R[B] (*) */

OP_CALL,/*	A B C	R[A], ... ,R[A+C-2] := R[A](R[A+1], ... ,R[A+B-1]) */
OP_TAILCALL,/*	A B C k	return R[A](R[A+1], ... ,R[A+B-1])		*/
@@ -315,6 +316,18 @@ OP_EXTRAARG/* Ax extra (larger) argument for previous opcode */

/*===========================================================================
  Notes:
+

+
  (*) Opcode OP_LFALSESKIP is used to convert a condition to a boolean
+
  value, in a code equivalent to (not cond ? false : true).  (It
+
  produces false and skips the next instruction producing true.)
+

+
  (*) Opcodes OP_MMBIN and variants follow each arithmetic and
+
  bitwise opcode. If the operation succeeds, it skips this next
+
  opcode. Otherwise, this opcode calls the corresponding metamethod.
+

+
  (*) Opcode OP_TESTSET is used in short-circuit expressions that need
+
  both to jump and to produce a value, such as (a = b or c).
+

  (*) In OP_CALL, if (B == 0) then B = top - A. If (C == 0), then
  'top' is set to last_result+1, so next open instruction (OP_CALL,
  OP_RETURN*, OP_SETLIST) may use 'top'.
modified external/lua/src/loslib.c
@@ -170,7 +170,7 @@ static int os_tmpname (lua_State *L) {
  char buff[LUA_TMPNAMBUFSIZE];
  int err;
  lua_tmpnam(buff, err);
-
  if (err)
+
  if (l_unlikely(err))
    return luaL_error(L, "unable to generate a unique filename");
  lua_pushstring(L, buff);
  return 1;
@@ -208,7 +208,7 @@ static int os_clock (lua_State *L) {
*/
static void setfield (lua_State *L, const char *key, int value, int delta) {
  #if (defined(LUA_NUMTIME) && LUA_MAXINTEGER <= INT_MAX)
-
    if (value > LUA_MAXINTEGER - delta)
+
    if (l_unlikely(value > LUA_MAXINTEGER - delta))
      luaL_error(L, "field '%s' is out-of-bound", key);
  #endif
  lua_pushinteger(L, (lua_Integer)value + delta);
@@ -253,9 +253,9 @@ static int getfield (lua_State *L, const char *key, int d, int delta) {
  int t = lua_getfield(L, -1, key);  /* get field and its type */
  lua_Integer res = lua_tointegerx(L, -1, &isnum);
  if (!isnum) {  /* field is not an integer? */
-
    if (t != LUA_TNIL)  /* some other value? */
+
    if (l_unlikely(t != LUA_TNIL))  /* some other value? */
      return luaL_error(L, "field '%s' is not an integer", key);
-
    else if (d < 0)  /* absent field; no default? */
+
    else if (l_unlikely(d < 0))  /* absent field; no default? */
      return luaL_error(L, "field '%s' missing in date table", key);
    res = d;
  }
modified external/lua/src/lparser.c
@@ -128,7 +128,7 @@ static void checknext (LexState *ls, int c) {
** in line 'where' (if that is not the current line).
*/
static void check_match (LexState *ls, int what, int who, int where) {
-
  if (unlikely(!testnext(ls, what))) {
+
  if (l_unlikely(!testnext(ls, what))) {
    if (where == ls->linenumber)  /* all in the same line? */
      error_expected(ls, what);  /* do not need a complex message */
    else {
@@ -222,26 +222,26 @@ static Vardesc *getlocalvardesc (FuncState *fs, int vidx) {


/*
-
** Convert 'nvar', a compiler index level, to it corresponding
-
** stack index level. For that, search for the highest variable
-
** below that level that is in the stack and uses its stack
-
** index ('sidx').
+
** Convert 'nvar', a compiler index level, to its corresponding
+
** register. For that, search for the highest variable below that level
+
** that is in a register and uses its register index ('ridx') plus one.
*/
-
static int stacklevel (FuncState *fs, int nvar) {
+
static int reglevel (FuncState *fs, int nvar) {
  while (nvar-- > 0) {
-
    Vardesc *vd = getlocalvardesc(fs, nvar);  /* get variable */
-
    if (vd->vd.kind != RDKCTC)  /* is in the stack? */
-
      return vd->vd.sidx + 1;
+
    Vardesc *vd = getlocalvardesc(fs, nvar);  /* get previous variable */
+
    if (vd->vd.kind != RDKCTC)  /* is in a register? */
+
      return vd->vd.ridx + 1;
  }
-
  return 0;  /* no variables in the stack */
+
  return 0;  /* no variables in registers */
}


/*
-
** Return the number of variables in the stack for function 'fs'
+
** Return the number of variables in the register stack for the given
+
** function.
*/
int luaY_nvarstack (FuncState *fs) {
-
  return stacklevel(fs, fs->nactvar);
+
  return reglevel(fs, fs->nactvar);
}


@@ -267,7 +267,7 @@ static void init_var (FuncState *fs, expdesc *e, int vidx) {
  e->f = e->t = NO_JUMP;
  e->k = VLOCAL;
  e->u.var.vidx = vidx;
-
  e->u.var.sidx = getlocalvardesc(fs, vidx)->vd.sidx;
+
  e->u.var.ridx = getlocalvardesc(fs, vidx)->vd.ridx;
}


@@ -310,12 +310,12 @@ static void check_readonly (LexState *ls, expdesc *e) {
*/
static void adjustlocalvars (LexState *ls, int nvars) {
  FuncState *fs = ls->fs;
-
  int stklevel = luaY_nvarstack(fs);
+
  int reglevel = luaY_nvarstack(fs);
  int i;
  for (i = 0; i < nvars; i++) {
    int vidx = fs->nactvar++;
    Vardesc *var = getlocalvardesc(fs, vidx);
-
    var->vd.sidx = stklevel++;
+
    var->vd.ridx = reglevel++;
    var->vd.pidx = registerlocalvar(ls, fs, var->vd.name);
  }
}
@@ -366,7 +366,7 @@ static int newupvalue (FuncState *fs, TString *name, expdesc *v) {
  FuncState *prev = fs->prev;
  if (v->k == VLOCAL) {
    up->instack = 1;
-
    up->idx = v->u.var.sidx;
+
    up->idx = v->u.var.ridx;
    up->kind = getlocalvardesc(prev, v->u.var.vidx)->vd.kind;
    lua_assert(eqstr(name, getlocalvardesc(prev, v->u.var.vidx)->vd.name));
  }
@@ -417,6 +417,17 @@ static void markupval (FuncState *fs, int level) {


/*
+
** Mark that current block has a to-be-closed variable.
+
*/
+
static void marktobeclosed (FuncState *fs) {
+
  BlockCnt *bl = fs->bl;
+
  bl->upval = 1;
+
  bl->insidetbc = 1;
+
  fs->needclose = 1;
+
}
+

+

+
/*
** Find a variable with the given name 'n'. If it is an upvalue, add
** this upvalue into all intermediate functions. If it is a global, set
** 'var' as 'void' as a flag.
@@ -517,7 +528,7 @@ static void solvegoto (LexState *ls, int g, Labeldesc *label) {
  Labellist *gl = &ls->dyd->gt;  /* list of goto's */
  Labeldesc *gt = &gl->arr[g];  /* goto to be resolved */
  lua_assert(eqstr(gt->name, label->name));
-
  if (unlikely(gt->nactvar < label->nactvar))  /* enter some scope? */
+
  if (l_unlikely(gt->nactvar < label->nactvar))  /* enter some scope? */
    jumpscopeerror(ls, gt);
  luaK_patchlist(ls->fs, gt->pc, label->pc);
  for (i = g; i < gl->n - 1; i++)  /* remove goto from pending list */
@@ -620,7 +631,7 @@ static void movegotosout (FuncState *fs, BlockCnt *bl) {
  for (i = bl->firstgoto; i < gl->n; i++) {  /* for each pending goto */
    Labeldesc *gt = &gl->arr[i];
    /* leaving a variable scope? */
-
    if (stacklevel(fs, gt->nactvar) > stacklevel(fs, bl->nactvar))
+
    if (reglevel(fs, gt->nactvar) > reglevel(fs, bl->nactvar))
      gt->close |= bl->upval;  /* jump may need a close */
    gt->nactvar = bl->nactvar;  /* update goto level */
  }
@@ -661,7 +672,7 @@ static void leaveblock (FuncState *fs) {
  BlockCnt *bl = fs->bl;
  LexState *ls = fs->ls;
  int hasclose = 0;
-
  int stklevel = stacklevel(fs, bl->nactvar);  /* level outside the block */
+
  int stklevel = reglevel(fs, bl->nactvar);  /* level outside the block */
  if (bl->isloop)  /* fix pending breaks? */
    hasclose = createlabel(ls, luaS_newliteral(ls->L, "break"), 0, 0);
  if (!hasclose && bl->previous && bl->upval)
@@ -1330,13 +1341,13 @@ static void check_conflict (LexState *ls, struct LHS_assign *lh, expdesc *v) {
        }
      }
      else {  /* table is a register */
-
        if (v->k == VLOCAL && lh->v.u.ind.t == v->u.var.sidx) {
+
        if (v->k == VLOCAL && lh->v.u.ind.t == v->u.var.ridx) {
          conflict = 1;  /* table is the local being assigned now */
          lh->v.u.ind.t = extra;  /* assignment will use safe copy */
        }
        /* is index the local being assigned? */
        if (lh->v.k == VINDEXED && v->k == VLOCAL &&
-
            lh->v.u.ind.idx == v->u.var.sidx) {
+
            lh->v.u.ind.idx == v->u.var.ridx) {
          conflict = 1;
          lh->v.u.ind.idx = extra;  /* previous assignment will use safe copy */
        }
@@ -1346,7 +1357,7 @@ static void check_conflict (LexState *ls, struct LHS_assign *lh, expdesc *v) {
  if (conflict) {
    /* copy upvalue/local value to a temporary (in position 'extra') */
    if (v->k == VLOCAL)
-
      luaK_codeABC(fs, OP_MOVE, extra, v->u.var.sidx, 0);
+
      luaK_codeABC(fs, OP_MOVE, extra, v->u.var.ridx, 0);
    else
      luaK_codeABC(fs, OP_GETUPVAL, extra, v->u.info, 0);
    luaK_reserveregs(fs, 1);
@@ -1411,7 +1422,7 @@ static void gotostat (LexState *ls) {
    newgotoentry(ls, name, line, luaK_jump(fs));
  else {  /* found a label */
    /* backward jump; will be resolved here */
-
    int lblevel = stacklevel(fs, lb->nactvar);  /* label level */
+
    int lblevel = reglevel(fs, lb->nactvar);  /* label level */
    if (luaY_nvarstack(fs) > lblevel)  /* leaving the scope of a variable? */
      luaK_codeABC(fs, OP_CLOSE, lblevel, 0, 0);
    /* create jump and link it to the label */
@@ -1435,7 +1446,7 @@ static void breakstat (LexState *ls) {
*/
static void checkrepeated (LexState *ls, TString *name) {
  Labeldesc *lb = findlabel(ls, name);
-
  if (unlikely(lb != NULL)) {  /* already defined? */
+
  if (l_unlikely(lb != NULL)) {  /* already defined? */
    const char *msg = "label '%s' already defined on line %d";
    msg = luaO_pushfstring(ls->L, msg, getstr(name), lb->line);
    luaK_semerror(ls, msg);  /* error */
@@ -1488,7 +1499,7 @@ static void repeatstat (LexState *ls, int line) {
  if (bl2.upval) {  /* upvalues? */
    int exit = luaK_jump(fs);  /* normal exit must jump over fix */
    luaK_patchtohere(fs, condexit);  /* repetition must close upvalues */
-
    luaK_codeABC(fs, OP_CLOSE, stacklevel(fs, bl2.nactvar), 0, 0);
+
    luaK_codeABC(fs, OP_CLOSE, reglevel(fs, bl2.nactvar), 0, 0);
    condexit = luaK_jump(fs);  /* repeat after closing upvalues */
    luaK_patchtohere(fs, exit);  /* normal exit comes to here */
  }
@@ -1520,7 +1531,7 @@ static void fixforjump (FuncState *fs, int pc, int dest, int back) {
  int offset = dest - (pc + 1);
  if (back)
    offset = -offset;
-
  if (unlikely(offset > MAXARG_Bx))
+
  if (l_unlikely(offset > MAXARG_Bx))
    luaX_syntaxerror(fs->ls, "control structure too long");
  SETARG_Bx(*jmp, offset);
}
@@ -1599,7 +1610,7 @@ static void forlist (LexState *ls, TString *indexname) {
  line = ls->linenumber;
  adjust_assign(ls, 4, explist(ls, &e), &e);
  adjustlocalvars(ls, 4);  /* control variables */
-
  markupval(fs, fs->nactvar);  /* last control var. must be closed */
+
  marktobeclosed(fs);  /* last control var. must be closed */
  luaK_checkstack(fs, 3);  /* extra space to call generator */
  forbody(ls, base, line, nvars - 4, 1);
}
@@ -1703,12 +1714,10 @@ static int getlocalattribute (LexState *ls) {
}


-
static void checktoclose (LexState *ls, int level) {
+
static void checktoclose (FuncState *fs, int level) {
  if (level != -1) {  /* is there a to-be-closed variable? */
-
    FuncState *fs = ls->fs;
-
    markupval(fs, level + 1);
-
    fs->bl->insidetbc = 1;  /* in the scope of a to-be-closed variable */
-
    luaK_codeABC(fs, OP_TBC, stacklevel(fs, level), 0, 0);
+
    marktobeclosed(fs);
+
    luaK_codeABC(fs, OP_TBC, reglevel(fs, level), 0, 0);
  }
}

@@ -1751,7 +1760,7 @@ static void localstat (LexState *ls) {
    adjust_assign(ls, nvars, nexps, &e);
    adjustlocalvars(ls, nvars);
  }
-
  checktoclose(ls, toclose);
+
  checktoclose(fs, toclose);
}


@@ -1776,6 +1785,7 @@ static void funcstat (LexState *ls, int line) {
  luaX_next(ls);  /* skip FUNCTION */
  ismethod = funcname(ls, &v);
  body(ls, &b, ismethod, line);
+
  check_readonly(ls, &v);
  luaK_storevar(ls->fs, &v, &b);
  luaK_fixline(ls->fs, line);  /* definition "happens" in the first line */
}
modified external/lua/src/lparser.h
@@ -35,7 +35,7 @@ typedef enum {
             (string is fixed by the lexer) */
  VNONRELOC,  /* expression has its value in a fixed register;
                 info = result register */
-
  VLOCAL,  /* local variable; var.sidx = stack index (local register);
+
  VLOCAL,  /* local variable; var.ridx = register index;
              var.vidx = relative index in 'actvar.arr'  */
  VUPVAL,  /* upvalue variable; info = index of upvalue in 'upvalues' */
  VCONST,  /* compile-time <const> variable;
@@ -77,7 +77,7 @@ typedef struct expdesc {
      lu_byte t;  /* table (register or upvalue) */
    } ind;
    struct {  /* for local variables */
-
      lu_byte sidx;  /* index in the stack */
+
      lu_byte ridx;  /* register holding the variable */
      unsigned short vidx;  /* compiler index (in 'actvar.arr')  */
    } var;
  } u;
@@ -97,7 +97,7 @@ typedef union Vardesc {
  struct {
    TValuefields;  /* constant value (if it is a compile-time constant) */
    lu_byte kind;
-
    lu_byte sidx;  /* index of the variable in the stack */
+
    lu_byte ridx;  /* register holding the variable */
    short pidx;  /* index of the variable in the Proto's 'locvars' array */
    TString *name;  /* variable name */
  } vd;
modified external/lua/src/lstate.c
@@ -166,13 +166,13 @@ void luaE_checkcstack (lua_State *L) {
  if (getCcalls(L) == LUAI_MAXCCALLS)
    luaG_runerror(L, "C stack overflow");
  else if (getCcalls(L) >= (LUAI_MAXCCALLS / 10 * 11))
-
    luaD_throw(L, LUA_ERRERR);  /* error while handing stack error */
+
    luaD_throw(L, LUA_ERRERR);  /* error while handling stack error */
}


LUAI_FUNC void luaE_incCstack (lua_State *L) {
  L->nCcalls++;
-
  if (unlikely(getCcalls(L) >= LUAI_MAXCCALLS))
+
  if (l_unlikely(getCcalls(L) >= LUAI_MAXCCALLS))
    luaE_checkcstack(L);
}

@@ -181,6 +181,7 @@ static void stack_init (lua_State *L1, lua_State *L) {
  int i; CallInfo *ci;
  /* initialize stack array */
  L1->stack = luaM_newvector(L, BASIC_STACK_SIZE + EXTRA_STACK, StackValue);
+
  L1->tbclist = L1->stack;
  for (i = 0; i < BASIC_STACK_SIZE + EXTRA_STACK; i++)
    setnilvalue(s2v(L1->stack + i));  /* erase new stack */
  L1->top = L1->stack;
@@ -213,24 +214,19 @@ static void freestack (lua_State *L) {
** Create registry table and its predefined values
*/
static void init_registry (lua_State *L, global_State *g) {
-
  TValue temp;
  /* create registry */
  Table *registry = luaH_new(L);
  sethvalue(L, &g->l_registry, registry);
  luaH_resize(L, registry, LUA_RIDX_LAST, 0);
  /* registry[LUA_RIDX_MAINTHREAD] = L */
-
  setthvalue(L, &temp, L);  /* temp = L */
-
  luaH_setint(L, registry, LUA_RIDX_MAINTHREAD, &temp);
-
  /* registry[LUA_RIDX_GLOBALS] = table of globals */
-
  sethvalue(L, &temp, luaH_new(L));  /* temp = new table (global table) */
-
  luaH_setint(L, registry, LUA_RIDX_GLOBALS, &temp);
+
  setthvalue(L, &registry->array[LUA_RIDX_MAINTHREAD - 1], L);
+
  /* registry[LUA_RIDX_GLOBALS] = new table (table of globals) */
+
  sethvalue(L, &registry->array[LUA_RIDX_GLOBALS - 1], luaH_new(L));
}


/*
** open parts of the state that may cause memory-allocation errors.
-
** ('g->nilvalue' being a nil value flags that the state was completely
-
** build.)
*/
static void f_luaopen (lua_State *L, void *ud) {
  global_State *g = G(L);
@@ -240,8 +236,8 @@ static void f_luaopen (lua_State *L, void *ud) {
  luaS_init(L);
  luaT_init(L);
  luaX_init(L);
-
  g->gcrunning = 1;  /* allow gc */
-
  setnilvalue(&g->nilvalue);
+
  g->gcstp = 0;  /* allow gc */
+
  setnilvalue(&g->nilvalue);  /* now state is complete */
  luai_userstateopen(L);
}

@@ -256,6 +252,7 @@ static void preinit_thread (lua_State *L, global_State *g) {
  L->ci = NULL;
  L->nci = 0;
  L->twups = L;  /* thread has no upvalues */
+
  L->nCcalls = 0;
  L->errorJmp = NULL;
  L->hook = NULL;
  L->hookmask = 0;
@@ -271,10 +268,14 @@ static void preinit_thread (lua_State *L, global_State *g) {

static void close_state (lua_State *L) {
  global_State *g = G(L);
-
  luaF_close(L, L->stack, CLOSEPROTECT);  /* close all upvalues */
-
  luaC_freeallobjects(L);  /* collect all objects */
-
  if (ttisnil(&g->nilvalue))  /* closing a fully built state? */
+
  if (!completestate(g))  /* closing a partially built state? */
+
    luaC_freeallobjects(L);  /* just collect its objects */
+
  else {  /* closing a fully built state */
+
    L->ci = &L->base_ci;  /* unwind CallInfo list */
+
    luaD_closeprotected(L, 1, LUA_OK);  /* close all upvalues */
+
    luaC_freeallobjects(L);  /* collect all objects */
    luai_userstateclose(L);
+
  }
  luaM_freearray(L, G(L)->strt.hash, G(L)->strt.size);
  freestack(L);
  lua_assert(gettotalbytes(g) == sizeof(LG));
@@ -299,7 +300,6 @@ LUA_API lua_State *lua_newthread (lua_State *L) {
  setthvalue2s(L, L->top, L1);
  api_incr_top(L);
  preinit_thread(L1, g);
-
  L1->nCcalls = 0;
  L1->hookmask = L->hookmask;
  L1->basehookcount = L->basehookcount;
  L1->hook = L->hook;
@@ -316,7 +316,7 @@ LUA_API lua_State *lua_newthread (lua_State *L) {

void luaE_freethread (lua_State *L, lua_State *L1) {
  LX *l = fromstate(L1);
-
  luaF_close(L1, L1->stack, NOCLOSINGMETH);  /* close all upvalues */
+
  luaF_closeupval(L1, L1->stack);  /* close all upvalues */
  lua_assert(L1->openupval == NULL);
  luai_userstatefree(L, L1);
  freestack(L1);
@@ -324,23 +324,29 @@ void luaE_freethread (lua_State *L, lua_State *L1) {
}


-
int lua_resetthread (lua_State *L) {
-
  CallInfo *ci;
-
  int status;
-
  lua_lock(L);
-
  L->ci = ci = &L->base_ci;  /* unwind CallInfo list */
+
int luaE_resetthread (lua_State *L, int status) {
+
  CallInfo *ci = L->ci = &L->base_ci;  /* unwind CallInfo list */
  setnilvalue(s2v(L->stack));  /* 'function' entry for basic 'ci' */
  ci->func = L->stack;
  ci->callstatus = CIST_C;
-
  status = luaF_close(L, L->stack, CLOSEPROTECT);
-
  if (status != CLOSEPROTECT)  /* real errors? */
-
    luaD_seterrorobj(L, status, L->stack + 1);
-
  else {
+
  if (status == LUA_YIELD)
    status = LUA_OK;
+
  L->status = LUA_OK;  /* so it can run __close metamethods */
+
  status = luaD_closeprotected(L, 1, status);
+
  if (status != LUA_OK)  /* errors? */
+
    luaD_seterrorobj(L, status, L->stack + 1);
+
  else
    L->top = L->stack + 1;
-
  }
  ci->top = L->top + LUA_MINSTACK;
-
  L->status = status;
+
  luaD_reallocstack(L, cast_int(ci->top - L->stack), 0);
+
  return status;
+
}
+

+

+
LUA_API int lua_resetthread (lua_State *L) {
+
  int status;
+
  lua_lock(L);
+
  status = luaE_resetthread(L, L->status);
  lua_unlock(L);
  return status;
}
@@ -360,7 +366,6 @@ LUA_API lua_State *lua_newstate (lua_Alloc f, void *ud) {
  preinit_thread(L, g);
  g->allgc = obj2gco(L);  /* by now, only object is the main thread */
  L->next = NULL;
-
  L->nCcalls = 0;
  incnny(L);  /* main thread is always non yieldable */
  g->frealloc = f;
  g->ud = ud;
@@ -368,13 +373,14 @@ LUA_API lua_State *lua_newstate (lua_Alloc f, void *ud) {
  g->ud_warn = NULL;
  g->mainthread = L;
  g->seed = luai_makeseed(L);
-
  g->gcrunning = 0;  /* no GC while building state */
+
  g->gcstp = GCSTPGC;  /* no GC while building state */
  g->strt.size = g->strt.nuse = 0;
  g->strt.hash = NULL;
  setnilvalue(&g->l_registry);
  g->panic = NULL;
  g->gcstate = GCSpause;
  g->gckind = KGC_INC;
+
  g->gcstopem = 0;
  g->gcemergency = 0;
  g->finobj = g->tobefnz = g->fixedgc = NULL;
  g->firstold1 = g->survival = g->old1 = g->reallyold = NULL;
modified external/lua/src/lstate.h
@@ -156,6 +156,18 @@ typedef struct stringtable {

/*
** Information about a call.
+
** About union 'u':
+
** - field 'l' is used only for Lua functions;
+
** - field 'c' is used only for C functions.
+
** About union 'u2':
+
** - field 'funcidx' is used only by C functions while doing a
+
** protected call;
+
** - field 'nyield' is used only while a function is "doing" an
+
** yield (from the yield until the next resume);
+
** - field 'nres' is used only while closing tbc variables when
+
** returning from a function;
+
** - field 'transferinfo' is used only during call/returnhooks,
+
** before the function starts or after it ends.
*/
typedef struct CallInfo {
  StkId func;  /* function index in the stack */
@@ -176,6 +188,7 @@ typedef struct CallInfo {
  union {
    int funcidx;  /* called-function index */
    int nyield;  /* number of values yielded */
+
    int nres;  /* number of values returned */
    struct {  /* info about transferred values (for call/return hooks) */
      unsigned short ftransfer;  /* offset of first value transferred */
      unsigned short ntransfer;  /* number of values transferred */
@@ -191,17 +204,34 @@ typedef struct CallInfo {
*/
#define CIST_OAH	(1<<0)	/* original value of 'allowhook' */
#define CIST_C		(1<<1)	/* call is running a C function */
-
#define CIST_FRESH	(1<<2)  /* call is on a fresh "luaV_execute" frame */
+
#define CIST_FRESH	(1<<2)	/* call is on a fresh "luaV_execute" frame */
#define CIST_HOOKED	(1<<3)	/* call is running a debug hook */
-
#define CIST_YPCALL	(1<<4)	/* call is a yieldable protected call */
+
#define CIST_YPCALL	(1<<4)	/* doing a yieldable protected call */
#define CIST_TAIL	(1<<5)	/* call was tail called */
#define CIST_HOOKYIELD	(1<<6)	/* last hook called yielded */
-
#define CIST_FIN	(1<<7)  /* call is running a finalizer */
+
#define CIST_FIN	(1<<7)	/* function "called" a finalizer */
#define CIST_TRAN	(1<<8)	/* 'ci' has transfer information */
+
#define CIST_CLSRET	(1<<9)  /* function is closing tbc variables */
+
/* Bits 10-12 are used for CIST_RECST (see below) */
+
#define CIST_RECST	10
#if defined(LUA_COMPAT_LT_LE)
-
#define CIST_LEQ	(1<<9)  /* using __lt for __le */
+
#define CIST_LEQ	(1<<13)  /* using __lt for __le */
#endif

+

+
/*
+
** Field CIST_RECST stores the "recover status", used to keep the error
+
** status while closing to-be-closed variables in coroutines, so that
+
** Lua can correctly resume after an yield from a __close method called
+
** because of an error.  (Three bits are enough for error status.)
+
*/
+
#define getcistrecst(ci)     (((ci)->callstatus >> CIST_RECST) & 7)
+
#define setcistrecst(ci,st)  \
+
  check_exp(((st) & 7) == (st),   /* status must fit in three bits */  \
+
            ((ci)->callstatus = ((ci)->callstatus & ~(7 << CIST_RECST))  \
+
                                                  | ((st) << CIST_RECST)))
+

+

/* active function is a Lua function */
#define isLua(ci)	(!((ci)->callstatus & CIST_C))

@@ -230,9 +260,10 @@ typedef struct global_State {
  lu_byte currentwhite;
  lu_byte gcstate;  /* state of garbage collector */
  lu_byte gckind;  /* kind of GC running */
+
  lu_byte gcstopem;  /* stops emergency collections */
  lu_byte genminormul;  /* control for minor generational collections */
  lu_byte genmajormul;  /* control for major generational collections */
-
  lu_byte gcrunning;  /* true if GC is running */
+
  lu_byte gcstp;  /* control whether GC is running */
  lu_byte gcemergency;  /* true if this is an emergency collection */
  lu_byte gcpause;  /* size of pause between successive GCs */
  lu_byte gcstepmul;  /* GC "speed" */
@@ -281,6 +312,7 @@ struct lua_State {
  StkId stack_last;  /* end of stack (last element + 1) */
  StkId stack;  /* stack base */
  UpVal *openupval;  /* list of open upvalues in this stack */
+
  StkId tbclist;  /* list of to-be-closed variables */
  GCObject *gclist;
  struct lua_State *twups;  /* list of threads with open upvalues */
  struct lua_longjmp *errorJmp;  /* current error recover point */
@@ -297,6 +329,12 @@ struct lua_State {

#define G(L)	(L->l_G)

+
/*
+
** 'g->nilvalue' being a nil value flags that the state was completely
+
** build.
+
*/
+
#define completestate(g)	ttisnil(&g->nilvalue)
+


/*
** Union of all collectable objects (only for conversions)
@@ -359,6 +397,7 @@ LUAI_FUNC void luaE_checkcstack (lua_State *L);
LUAI_FUNC void luaE_incCstack (lua_State *L);
LUAI_FUNC void luaE_warning (lua_State *L, const char *msg, int tocont);
LUAI_FUNC void luaE_warnerror (lua_State *L, const char *where);
+
LUAI_FUNC int luaE_resetthread (lua_State *L, int status);


#endif
modified external/lua/src/lstring.c
@@ -89,7 +89,7 @@ void luaS_resize (lua_State *L, int nsize) {
  if (nsize < osize)  /* shrinking table? */
    tablerehash(tb->hash, osize, nsize);  /* depopulate shrinking part */
  newvect = luaM_reallocvector(L, tb->hash, osize, nsize, TString*);
-
  if (unlikely(newvect == NULL)) {  /* reallocation failed? */
+
  if (l_unlikely(newvect == NULL)) {  /* reallocation failed? */
    if (nsize < osize)  /* was it shrinking table? */
      tablerehash(tb->hash, nsize, osize);  /* restore to original size */
    /* leave table as it was */
@@ -172,7 +172,7 @@ void luaS_remove (lua_State *L, TString *ts) {


static void growstrtab (lua_State *L, stringtable *tb) {
-
  if (unlikely(tb->nuse == MAX_INT)) {  /* too many strings? */
+
  if (l_unlikely(tb->nuse == MAX_INT)) {  /* too many strings? */
    luaC_fullgc(L, 1);  /* try to free some... */
    if (tb->nuse == MAX_INT)  /* still too many? */
      luaM_error(L);  /* cannot even create a message... */
@@ -223,7 +223,7 @@ TString *luaS_newlstr (lua_State *L, const char *str, size_t l) {
    return internshrstr(L, str, l);
  else {
    TString *ts;
-
    if (unlikely(l >= (MAX_SIZE - sizeof(TString))/sizeof(char)))
+
    if (l_unlikely(l >= (MAX_SIZE - sizeof(TString))/sizeof(char)))
      luaM_toobig(L);
    ts = luaS_createlngstrobj(L, l);
    memcpy(getstr(ts), str, l * sizeof(char));
@@ -259,7 +259,7 @@ Udata *luaS_newudata (lua_State *L, size_t s, int nuvalue) {
  Udata *u;
  int i;
  GCObject *o;
-
  if (unlikely(s > MAX_SIZE - udatamemoffset(nuvalue)))
+
  if (l_unlikely(s > MAX_SIZE - udatamemoffset(nuvalue)))
    luaM_toobig(L);
  o = luaC_newobj(L, LUA_VUSERDATA, sizeudata(nuvalue, s));
  u = gco2u(o);
modified external/lua/src/lstrlib.c
@@ -152,8 +152,9 @@ static int str_rep (lua_State *L) {
  const char *s = luaL_checklstring(L, 1, &l);
  lua_Integer n = luaL_checkinteger(L, 2);
  const char *sep = luaL_optlstring(L, 3, "", &lsep);
-
  if (n <= 0) lua_pushliteral(L, "");
-
  else if (l + lsep < l || l + lsep > MAXSIZE / n)  /* may overflow? */
+
  if (n <= 0)
+
    lua_pushliteral(L, "");
+
  else if (l_unlikely(l + lsep < l || l + lsep > MAXSIZE / n))
    return luaL_error(L, "resulting string too large");
  else {
    size_t totallen = (size_t)n * l + (size_t)(n - 1) * lsep;
@@ -181,7 +182,7 @@ static int str_byte (lua_State *L) {
  size_t pose = getendpos(L, 3, pi, l);
  int n, i;
  if (posi > pose) return 0;  /* empty interval; return no values */
-
  if (pose - posi >= (size_t)INT_MAX)  /* arithmetic overflow? */
+
  if (l_unlikely(pose - posi >= (size_t)INT_MAX))  /* arithmetic overflow? */
    return luaL_error(L, "string slice too long");
  n = (int)(pose -  posi) + 1;
  luaL_checkstack(L, n, "string slice too long");
@@ -235,7 +236,7 @@ static int str_dump (lua_State *L) {
  luaL_checktype(L, 1, LUA_TFUNCTION);
  lua_settop(L, 1);  /* ensure function is on the top of the stack */
  state.init = 0;
-
  if (lua_dump(L, writer, &state, strip) != 0)
+
  if (l_unlikely(lua_dump(L, writer, &state, strip) != 0))
    return luaL_error(L, "unable to dump given function");
  luaL_pushresult(&state.B);
  return 1;
@@ -275,7 +276,8 @@ static int tonum (lua_State *L, int arg) {

static void trymt (lua_State *L, const char *mtname) {
  lua_settop(L, 2);  /* back to the original arguments */
-
  if (lua_type(L, 2) == LUA_TSTRING || !luaL_getmetafield(L, 2, mtname))
+
  if (l_unlikely(lua_type(L, 2) == LUA_TSTRING ||
+
                 !luaL_getmetafield(L, 2, mtname)))
    luaL_error(L, "attempt to %s a '%s' with a '%s'", mtname + 2,
                  luaL_typename(L, -2), luaL_typename(L, -1));
  lua_insert(L, -3);  /* put metamethod before arguments */
@@ -383,7 +385,8 @@ static const char *match (MatchState *ms, const char *s, const char *p);

static int check_capture (MatchState *ms, int l) {
  l -= '1';
-
  if (l < 0 || l >= ms->level || ms->capture[l].len == CAP_UNFINISHED)
+
  if (l_unlikely(l < 0 || l >= ms->level ||
+
                 ms->capture[l].len == CAP_UNFINISHED))
    return luaL_error(ms->L, "invalid capture index %%%d", l + 1);
  return l;
}
@@ -400,14 +403,14 @@ static int capture_to_close (MatchState *ms) {
static const char *classend (MatchState *ms, const char *p) {
  switch (*p++) {
    case L_ESC: {
-
      if (p == ms->p_end)
+
      if (l_unlikely(p == ms->p_end))
        luaL_error(ms->L, "malformed pattern (ends with '%%')");
      return p+1;
    }
    case '[': {
      if (*p == '^') p++;
      do {  /* look for a ']' */
-
        if (p == ms->p_end)
+
        if (l_unlikely(p == ms->p_end))
          luaL_error(ms->L, "malformed pattern (missing ']')");
        if (*(p++) == L_ESC && p < ms->p_end)
          p++;  /* skip escapes (e.g. '%]') */
@@ -482,7 +485,7 @@ static int singlematch (MatchState *ms, const char *s, const char *p,

static const char *matchbalance (MatchState *ms, const char *s,
                                   const char *p) {
-
  if (p >= ms->p_end - 1)
+
  if (l_unlikely(p >= ms->p_end - 1))
    luaL_error(ms->L, "malformed pattern (missing arguments to '%%b')");
  if (*s != *p) return NULL;
  else {
@@ -565,7 +568,7 @@ static const char *match_capture (MatchState *ms, const char *s, int l) {


static const char *match (MatchState *ms, const char *s, const char *p) {
-
  if (ms->matchdepth-- == 0)
+
  if (l_unlikely(ms->matchdepth-- == 0))
    luaL_error(ms->L, "pattern too complex");
  init: /* using goto's to optimize tail recursion */
  if (p != ms->p_end) {  /* end of pattern? */
@@ -599,7 +602,7 @@ static const char *match (MatchState *ms, const char *s, const char *p) {
          case 'f': {  /* frontier? */
            const char *ep; char previous;
            p += 2;
-
            if (*p != '[')
+
            if (l_unlikely(*p != '['))
              luaL_error(ms->L, "missing '[' after '%%f' in pattern");
            ep = classend(ms, p);  /* points to what is next */
            previous = (s == ms->src_init) ? '\0' : *(s - 1);
@@ -699,7 +702,7 @@ static const char *lmemfind (const char *s1, size_t l1,
static size_t get_onecapture (MatchState *ms, int i, const char *s,
                              const char *e, const char **cap) {
  if (i >= ms->level) {
-
    if (i != 0)
+
    if (l_unlikely(i != 0))
      luaL_error(ms->L, "invalid capture index %%%d", i + 1);
    *cap = s;
    return e - s;
@@ -707,7 +710,7 @@ static size_t get_onecapture (MatchState *ms, int i, const char *s,
  else {
    ptrdiff_t capl = ms->capture[i].len;
    *cap = ms->capture[i].init;
-
    if (capl == CAP_UNFINISHED)
+
    if (l_unlikely(capl == CAP_UNFINISHED))
      luaL_error(ms->L, "unfinished capture");
    else if (capl == CAP_POSITION)
      lua_pushinteger(ms->L, (ms->capture[i].init - ms->src_init) + 1);
@@ -926,7 +929,7 @@ static int add_value (MatchState *ms, luaL_Buffer *b, const char *s,
    luaL_addlstring(b, s, e - s);  /* keep original text */
    return 0;  /* no changes */
  }
-
  else if (!lua_isstring(L, -1))
+
  else if (l_unlikely(!lua_isstring(L, -1)))
    return luaL_error(L, "invalid replacement value (a %s)",
                         luaL_typename(L, -1));
  else {
@@ -1058,7 +1061,7 @@ static int lua_number2strx (lua_State *L, char *buff, int sz,
    for (i = 0; i < n; i++)
      buff[i] = toupper(uchar(buff[i]));
  }
-
  else if (fmt[SIZELENMOD] != 'a')
+
  else if (l_unlikely(fmt[SIZELENMOD] != 'a'))
    return luaL_error(L, "modifiers for format '%%a'/'%%A' not implemented");
  return n;
}
@@ -1087,13 +1090,31 @@ static int lua_number2strx (lua_State *L, char *buff, int sz,


/* valid flags in a format specification */
-
#if !defined(L_FMTFLAGS)
-
#define L_FMTFLAGS	"-+ #0"
+
#if !defined(L_FMTFLAGSF)
+

+
/* valid flags for a, A, e, E, f, F, g, and G conversions */
+
#define L_FMTFLAGSF	"-+#0 "
+

+
/* valid flags for o, x, and X conversions */
+
#define L_FMTFLAGSX	"-#0"
+

+
/* valid flags for d and i conversions */
+
#define L_FMTFLAGSI	"-+0 "
+

+
/* valid flags for u conversions */
+
#define L_FMTFLAGSU	"-0"
+

+
/* valid flags for c, p, and s conversions */
+
#define L_FMTFLAGSC	"-"
+

#endif


/*
-
** maximum size of each format specification (such as "%-099.99d")
+
** Maximum size of each format specification (such as "%-099.99d"):
+
** Initial '%', flags (up to 5), width (2), period, precision (2),
+
** length modifier (8), conversion specifier, and final '\0', plus some
+
** extra.
*/
#define MAX_FORMAT	32

@@ -1186,25 +1207,53 @@ static void addliteral (lua_State *L, luaL_Buffer *b, int arg) {
}


-
static const char *scanformat (lua_State *L, const char *strfrmt, char *form) {
-
  const char *p = strfrmt;
-
  while (*p != '\0' && strchr(L_FMTFLAGS, *p) != NULL) p++;  /* skip flags */
-
  if ((size_t)(p - strfrmt) >= sizeof(L_FMTFLAGS)/sizeof(char))
-
    luaL_error(L, "invalid format (repeated flags)");
-
  if (isdigit(uchar(*p))) p++;  /* skip width */
-
  if (isdigit(uchar(*p))) p++;  /* (2 digits at most) */
-
  if (*p == '.') {
-
    p++;
-
    if (isdigit(uchar(*p))) p++;  /* skip precision */
-
    if (isdigit(uchar(*p))) p++;  /* (2 digits at most) */
+
static const char *get2digits (const char *s) {
+
  if (isdigit(uchar(*s))) {
+
    s++;
+
    if (isdigit(uchar(*s))) s++;  /* (2 digits at most) */
+
  }
+
  return s;
+
}
+

+

+
/*
+
** Check whether a conversion specification is valid. When called,
+
** first character in 'form' must be '%' and last character must
+
** be a valid conversion specifier. 'flags' are the accepted flags;
+
** 'precision' signals whether to accept a precision.
+
*/
+
static void checkformat (lua_State *L, const char *form, const char *flags,
+
                                       int precision) {
+
  const char *spec = form + 1;  /* skip '%' */
+
  spec += strspn(spec, flags);  /* skip flags */
+
  if (*spec != '0') {  /* a width cannot start with '0' */
+
    spec = get2digits(spec);  /* skip width */
+
    if (*spec == '.' && precision) {
+
      spec++;
+
      spec = get2digits(spec);  /* skip precision */
+
    }
  }
-
  if (isdigit(uchar(*p)))
-
    luaL_error(L, "invalid format (width or precision too long)");
+
  if (!isalpha(uchar(*spec)))  /* did not go to the end? */
+
    luaL_error(L, "invalid conversion specification: '%s'", form);
+
}
+

+

+
/*
+
** Get a conversion specification and copy it to 'form'.
+
** Return the address of its last character.
+
*/
+
static const char *getformat (lua_State *L, const char *strfrmt,
+
                                            char *form) {
+
  /* spans flags, width, and precision ('0' is included as a flag) */
+
  size_t len = strspn(strfrmt, L_FMTFLAGSF "123456789.");
+
  len++;  /* adds following character (should be the specifier) */
+
  /* still needs space for '%', '\0', plus a length modifier */
+
  if (len >= MAX_FORMAT - 10)
+
    luaL_error(L, "invalid format (too long)");
  *(form++) = '%';
-
  memcpy(form, strfrmt, ((p - strfrmt) + 1) * sizeof(char));
-
  form += (p - strfrmt) + 1;
-
  *form = '\0';
-
  return p;
+
  memcpy(form, strfrmt, len * sizeof(char));
+
  *(form + len) = '\0';
+
  return strfrmt + len - 1;
}


@@ -1227,6 +1276,7 @@ static int str_format (lua_State *L) {
  size_t sfl;
  const char *strfrmt = luaL_checklstring(L, arg, &sfl);
  const char *strfrmt_end = strfrmt+sfl;
+
  const char *flags;
  luaL_Buffer b;
  luaL_buffinit(L, &b);
  while (strfrmt < strfrmt_end) {
@@ -1236,25 +1286,35 @@ static int str_format (lua_State *L) {
      luaL_addchar(&b, *strfrmt++);  /* %% */
    else { /* format item */
      char form[MAX_FORMAT];  /* to store the format ('%...') */
-
      int maxitem = MAX_ITEM;
-
      char *buff = luaL_prepbuffsize(&b, maxitem);  /* to put formatted item */
-
      int nb = 0;  /* number of bytes in added item */
+
      int maxitem = MAX_ITEM;  /* maximum length for the result */
+
      char *buff = luaL_prepbuffsize(&b, maxitem);  /* to put result */
+
      int nb = 0;  /* number of bytes in result */
      if (++arg > top)
        return luaL_argerror(L, arg, "no value");
-
      strfrmt = scanformat(L, strfrmt, form);
+
      strfrmt = getformat(L, strfrmt, form);
      switch (*strfrmt++) {
        case 'c': {
+
          checkformat(L, form, L_FMTFLAGSC, 0);
          nb = l_sprintf(buff, maxitem, form, (int)luaL_checkinteger(L, arg));
          break;
        }
        case 'd': case 'i':
-
        case 'o': case 'u': case 'x': case 'X': {
+
          flags = L_FMTFLAGSI;
+
          goto intcase;
+
        case 'u':
+
          flags = L_FMTFLAGSU;
+
          goto intcase;
+
        case 'o': case 'x': case 'X':
+
          flags = L_FMTFLAGSX;
+
         intcase: {
          lua_Integer n = luaL_checkinteger(L, arg);
+
          checkformat(L, form, flags, 1);
          addlenmod(form, LUA_INTEGER_FRMLEN);
          nb = l_sprintf(buff, maxitem, form, (LUAI_UACINT)n);
          break;
        }
        case 'a': case 'A':
+
          checkformat(L, form, L_FMTFLAGSF, 1);
          addlenmod(form, LUA_NUMBER_FRMLEN);
          nb = lua_number2strx(L, buff, maxitem, form,
                                  luaL_checknumber(L, arg));
@@ -1265,12 +1325,14 @@ static int str_format (lua_State *L) {
          /* FALLTHROUGH */
        case 'e': case 'E': case 'g': case 'G': {
          lua_Number n = luaL_checknumber(L, arg);
+
          checkformat(L, form, L_FMTFLAGSF, 1);
          addlenmod(form, LUA_NUMBER_FRMLEN);
          nb = l_sprintf(buff, maxitem, form, (LUAI_UACNUMBER)n);
          break;
        }
        case 'p': {
          const void *p = lua_topointer(L, arg);
+
          checkformat(L, form, L_FMTFLAGSC, 0);
          if (p == NULL) {  /* avoid calling 'printf' with argument NULL */
            p = "(null)";  /* result */
            form[strlen(form) - 1] = 's';  /* format it as a string */
@@ -1291,7 +1353,8 @@ static int str_format (lua_State *L) {
            luaL_addvalue(&b);  /* keep entire string */
          else {
            luaL_argcheck(L, l == strlen(s), arg, "string contains zeros");
-
            if (!strchr(form, '.') && l >= 100) {
+
            checkformat(L, form, L_FMTFLAGSC, 1);
+
            if (strchr(form, '.') == NULL && l >= 100) {
              /* no precision and string is too long to be formatted */
              luaL_addvalue(&b);  /* keep entire string */
            }
@@ -1349,25 +1412,6 @@ static const union {
} nativeendian = {1};


-
/* dummy structure to get native alignment requirements */
-
struct cD {
-
  char c;
-
  union { double d; void *p; lua_Integer i; lua_Number n; } u;
-
};
-

-
#define MAXALIGN	(offsetof(struct cD, u))
-

-

-
/*
-
** Union for serializing floats
-
*/
-
typedef union Ftypes {
-
  float f;
-
  double d;
-
  lua_Number n;
-
} Ftypes;
-

-

/*
** information to pack/unpack stuff
*/
@@ -1384,7 +1428,9 @@ typedef struct Header {
typedef enum KOption {
  Kint,		/* signed integers */
  Kuint,	/* unsigned integers */
-
  Kfloat,	/* floating-point numbers */
+
  Kfloat,	/* single-precision floating-point numbers */
+
  Knumber,	/* Lua "native" floating-point numbers */
+
  Kdouble,	/* double-precision floating-point numbers */
  Kchar,	/* fixed-length strings */
  Kstring,	/* strings with prefixed length */
  Kzstr,	/* zero-terminated strings */
@@ -1419,7 +1465,7 @@ static int getnum (const char **fmt, int df) {
*/
static int getnumlimit (Header *h, const char **fmt, int df) {
  int sz = getnum(fmt, df);
-
  if (sz > MAXINTSIZE || sz <= 0)
+
  if (l_unlikely(sz > MAXINTSIZE || sz <= 0))
    return luaL_error(h->L, "integral size (%d) out of limits [1,%d]",
                            sz, MAXINTSIZE);
  return sz;
@@ -1440,6 +1486,8 @@ static void initheader (lua_State *L, Header *h) {
** Read and classify next option. 'size' is filled with option's size.
*/
static KOption getoption (Header *h, const char **fmt, int *size) {
+
  /* dummy structure to get native alignment requirements */
+
  struct cD { char c; union { LUAI_MAXALIGN; } u; };
  int opt = *((*fmt)++);
  *size = 0;  /* default */
  switch (opt) {
@@ -1453,14 +1501,14 @@ static KOption getoption (Header *h, const char **fmt, int *size) {
    case 'J': *size = sizeof(lua_Integer); return Kuint;
    case 'T': *size = sizeof(size_t); return Kuint;
    case 'f': *size = sizeof(float); return Kfloat;
-
    case 'd': *size = sizeof(double); return Kfloat;
-
    case 'n': *size = sizeof(lua_Number); return Kfloat;
+
    case 'n': *size = sizeof(lua_Number); return Knumber;
+
    case 'd': *size = sizeof(double); return Kdouble;
    case 'i': *size = getnumlimit(h, fmt, sizeof(int)); return Kint;
    case 'I': *size = getnumlimit(h, fmt, sizeof(int)); return Kuint;
    case 's': *size = getnumlimit(h, fmt, sizeof(size_t)); return Kstring;
    case 'c':
      *size = getnum(fmt, -1);
-
      if (*size == -1)
+
      if (l_unlikely(*size == -1))
        luaL_error(h->L, "missing size for format option 'c'");
      return Kchar;
    case 'z': return Kzstr;
@@ -1470,7 +1518,11 @@ static KOption getoption (Header *h, const char **fmt, int *size) {
    case '<': h->islittle = 1; break;
    case '>': h->islittle = 0; break;
    case '=': h->islittle = nativeendian.little; break;
-
    case '!': h->maxalign = getnumlimit(h, fmt, MAXALIGN); break;
+
    case '!': {
+
      const int maxalign = offsetof(struct cD, u);
+
      h->maxalign = getnumlimit(h, fmt, maxalign);
+
      break;
+
    }
    default: luaL_error(h->L, "invalid format option '%c'", opt);
  }
  return Knop;
@@ -1499,7 +1551,7 @@ static KOption getdetails (Header *h, size_t totalsize,
  else {
    if (align > h->maxalign)  /* enforce maximum alignment */
      align = h->maxalign;
-
    if ((align & (align - 1)) != 0)  /* is 'align' not a power of 2? */
+
    if (l_unlikely((align & (align - 1)) != 0))  /* not a power of 2? */
      luaL_argerror(h->L, 1, "format asks for alignment not power of 2");
    *ntoalign = (align - (int)(totalsize & (align - 1))) & (align - 1);
  }
@@ -1580,15 +1632,27 @@ static int str_pack (lua_State *L) {
        packint(&b, (lua_Unsigned)n, h.islittle, size, 0);
        break;
      }
-
      case Kfloat: {  /* floating-point options */
-
        Ftypes u;
-
        char *buff = luaL_prepbuffsize(&b, size);
-
        lua_Number n = luaL_checknumber(L, arg);  /* get argument */
-
        if (size == sizeof(u.f)) u.f = (float)n;  /* copy it into 'u' */
-
        else if (size == sizeof(u.d)) u.d = (double)n;
-
        else u.n = n;
-
        /* move 'u' to final result, correcting endianness if needed */
-
        copywithendian(buff, (char *)&u, size, h.islittle);
+
      case Kfloat: {  /* C float */
+
        float f = (float)luaL_checknumber(L, arg);  /* get argument */
+
        char *buff = luaL_prepbuffsize(&b, sizeof(f));
+
        /* move 'f' to final result, correcting endianness if needed */
+
        copywithendian(buff, (char *)&f, sizeof(f), h.islittle);
+
        luaL_addsize(&b, size);
+
        break;
+
      }
+
      case Knumber: {  /* Lua float */
+
        lua_Number f = luaL_checknumber(L, arg);  /* get argument */
+
        char *buff = luaL_prepbuffsize(&b, sizeof(f));
+
        /* move 'f' to final result, correcting endianness if needed */
+
        copywithendian(buff, (char *)&f, sizeof(f), h.islittle);
+
        luaL_addsize(&b, size);
+
        break;
+
      }
+
      case Kdouble: {  /* C double */
+
        double f = (double)luaL_checknumber(L, arg);  /* get argument */
+
        char *buff = luaL_prepbuffsize(&b, sizeof(f));
+
        /* move 'f' to final result, correcting endianness if needed */
+
        copywithendian(buff, (char *)&f, sizeof(f), h.islittle);
        luaL_addsize(&b, size);
        break;
      }
@@ -1679,7 +1743,7 @@ static lua_Integer unpackint (lua_State *L, const char *str,
  else if (size > SZINT) {  /* must check unread bytes */
    int mask = (!issigned || (lua_Integer)res >= 0) ? 0 : MC;
    for (i = limit; i < size; i++) {
-
      if ((unsigned char)str[islittle ? i : size - 1 - i] != mask)
+
      if (l_unlikely((unsigned char)str[islittle ? i : size - 1 - i] != mask))
        luaL_error(L, "%d-byte integer does not fit into Lua Integer", size);
    }
  }
@@ -1714,13 +1778,21 @@ static int str_unpack (lua_State *L) {
        break;
      }
      case Kfloat: {
-
        Ftypes u;
-
        lua_Number num;
-
        copywithendian((char *)&u, data + pos, size, h.islittle);
-
        if (size == sizeof(u.f)) num = (lua_Number)u.f;
-
        else if (size == sizeof(u.d)) num = (lua_Number)u.d;
-
        else num = u.n;
-
        lua_pushnumber(L, num);
+
        float f;
+
        copywithendian((char *)&f, data + pos, sizeof(f), h.islittle);
+
        lua_pushnumber(L, (lua_Number)f);
+
        break;
+
      }
+
      case Knumber: {
+
        lua_Number f;
+
        copywithendian((char *)&f, data + pos, sizeof(f), h.islittle);
+
        lua_pushnumber(L, f);
+
        break;
+
      }
+
      case Kdouble: {
+
        double f;
+
        copywithendian((char *)&f, data + pos, sizeof(f), h.islittle);
+
        lua_pushnumber(L, (lua_Number)f);
        break;
      }
      case Kchar: {
modified external/lua/src/ltable.c
@@ -68,20 +68,23 @@
#define MAXHSIZE	luaM_limitN(1u << MAXHBITS, Node)


+
/*
+
** When the original hash value is good, hashing by a power of 2
+
** avoids the cost of '%'.
+
*/
#define hashpow2(t,n)		(gnode(t, lmod((n), sizenode(t))))

-
#define hashstr(t,str)		hashpow2(t, (str)->hash)
-
#define hashboolean(t,p)	hashpow2(t, p)
-
#define hashint(t,i)		hashpow2(t, i)
-

-

/*
-
** for some types, it is better to avoid modulus by power of 2, as
-
** they tend to have many 2 factors.
+
** for other types, it is better to avoid modulo by power of 2, as
+
** they can have many 2 factors.
*/
#define hashmod(t,n)	(gnode(t, ((n) % ((sizenode(t)-1)|1))))


+
#define hashstr(t,str)		hashpow2(t, (str)->hash)
+
#define hashboolean(t,p)	hashpow2(t, p)
+

+

#define hashpointer(t,p)	hashmod(t, point2uint(p))


@@ -96,6 +99,20 @@ static const Node dummynode_ = {
static const TValue absentkey = {ABSTKEYCONSTANT};


+
/*
+
** Hash for integers. To allow a good hash, use the remainder operator
+
** ('%'). If integer fits as a non-negative int, compute an int
+
** remainder, which is faster. Otherwise, use an unsigned-integer
+
** remainder, which uses all bits and ensures a non-negative result.
+
*/
+
static Node *hashint (const Table *t, lua_Integer i) {
+
  lua_Unsigned ui = l_castS2U(i);
+
  if (ui <= (unsigned int)INT_MAX)
+
    return hashmod(t, cast_int(ui));
+
  else
+
    return hashmod(t, ui);
+
}
+


/*
** Hash for floating-point numbers.
@@ -129,39 +146,50 @@ static int l_hashfloat (lua_Number n) {

/*
** returns the 'main' position of an element in a table (that is,
-
** the index of its hash value). The key comes broken (tag in 'ktt'
-
** and value in 'vkl') so that we can call it on keys inserted into
-
** nodes.
+
** the index of its hash value).
*/
-
static Node *mainposition (const Table *t, int ktt, const Value *kvl) {
-
  switch (withvariant(ktt)) {
-
    case LUA_VNUMINT:
-
      return hashint(t, ivalueraw(*kvl));
-
    case LUA_VNUMFLT:
-
      return hashmod(t, l_hashfloat(fltvalueraw(*kvl)));
-
    case LUA_VSHRSTR:
-
      return hashstr(t, tsvalueraw(*kvl));
-
    case LUA_VLNGSTR:
-
      return hashpow2(t, luaS_hashlongstr(tsvalueraw(*kvl)));
+
static Node *mainpositionTV (const Table *t, const TValue *key) {
+
  switch (ttypetag(key)) {
+
    case LUA_VNUMINT: {
+
      lua_Integer i = ivalue(key);
+
      return hashint(t, i);
+
    }
+
    case LUA_VNUMFLT: {
+
      lua_Number n = fltvalue(key);
+
      return hashmod(t, l_hashfloat(n));
+
    }
+
    case LUA_VSHRSTR: {
+
      TString *ts = tsvalue(key);
+
      return hashstr(t, ts);
+
    }
+
    case LUA_VLNGSTR: {
+
      TString *ts = tsvalue(key);
+
      return hashpow2(t, luaS_hashlongstr(ts));
+
    }
    case LUA_VFALSE:
      return hashboolean(t, 0);
    case LUA_VTRUE:
      return hashboolean(t, 1);
-
    case LUA_VLIGHTUSERDATA:
-
      return hashpointer(t, pvalueraw(*kvl));
-
    case LUA_VLCF:
-
      return hashpointer(t, fvalueraw(*kvl));
-
    default:
-
      return hashpointer(t, gcvalueraw(*kvl));
+
    case LUA_VLIGHTUSERDATA: {
+
      void *p = pvalue(key);
+
      return hashpointer(t, p);
+
    }
+
    case LUA_VLCF: {
+
      lua_CFunction f = fvalue(key);
+
      return hashpointer(t, f);
+
    }
+
    default: {
+
      GCObject *o = gcvalue(key);
+
      return hashpointer(t, o);
+
    }
  }
}


-
/*
-
** Returns the main position of an element given as a 'TValue'
-
*/
-
static Node *mainpositionTV (const Table *t, const TValue *key) {
-
  return mainposition(t, rawtt(key), valraw(key));
+
l_sinline Node *mainpositionfromnode (const Table *t, Node *nd) {
+
  TValue key;
+
  getnodekey(cast(lua_State *, NULL), &key, nd);
+
  return mainpositionTV(t, &key);
}


@@ -307,7 +335,7 @@ static unsigned int findindex (lua_State *L, Table *t, TValue *key,
    return i;  /* yes; that's the index */
  else {
    const TValue *n = getgeneric(t, key, 1);
-
    if (unlikely(isabstkey(n)))
+
    if (l_unlikely(isabstkey(n)))
      luaG_runerror(L, "invalid key to 'next'");  /* key not found */
    i = cast_int(nodefromval(n) - gnode(t, 0));  /* key index in hash table */
    /* hash elements are numbered after array ones */
@@ -485,7 +513,7 @@ static void reinsert (lua_State *L, Table *ot, Table *t) {
         already present in the table */
      TValue k;
      getnodekey(L, &k, old);
-
      setobjt2t(L, luaH_set(L, t, &k), gval(old));
+
      luaH_set(L, t, &k, gval(old));
    }
  }
}
@@ -541,7 +569,7 @@ void luaH_resize (lua_State *L, Table *t, unsigned int newasize,
  }
  /* allocate new array */
  newarray = luaM_reallocvector(L, t->array, oldasize, newasize, TValue);
-
  if (unlikely(newarray == NULL && newasize > 0)) {  /* allocation failed? */
+
  if (l_unlikely(newarray == NULL && newasize > 0)) {  /* allocation failed? */
    freehash(L, &newt);  /* release new hash part */
    luaM_error(L);  /* raise error (with array unchanged) */
  }
@@ -632,10 +660,10 @@ static Node *getfreepos (Table *t) {
** put new key in its main position; otherwise (colliding node is in its main
** position), new key goes to an empty position.
*/
-
TValue *luaH_newkey (lua_State *L, Table *t, const TValue *key) {
+
void luaH_newkey (lua_State *L, Table *t, const TValue *key, TValue *value) {
  Node *mp;
  TValue aux;
-
  if (unlikely(ttisnil(key)))
+
  if (l_unlikely(ttisnil(key)))
    luaG_runerror(L, "table index is nil");
  else if (ttisfloat(key)) {
    lua_Number f = fltvalue(key);
@@ -644,9 +672,11 @@ TValue *luaH_newkey (lua_State *L, Table *t, const TValue *key) {
      setivalue(&aux, k);
      key = &aux;  /* insert it as an integer */
    }
-
    else if (unlikely(luai_numisnan(f)))
+
    else if (l_unlikely(luai_numisnan(f)))
      luaG_runerror(L, "table index is NaN");
  }
+
  if (ttisnil(value))
+
    return;  /* do not insert nil values */
  mp = mainpositionTV(t, key);
  if (!isempty(gval(mp)) || isdummy(t)) {  /* main position is taken? */
    Node *othern;
@@ -654,10 +684,11 @@ TValue *luaH_newkey (lua_State *L, Table *t, const TValue *key) {
    if (f == NULL) {  /* cannot find a free place? */
      rehash(L, t, key);  /* grow table */
      /* whatever called 'newkey' takes care of TM cache */
-
      return luaH_set(L, t, key);  /* insert key into grown table */
+
      luaH_set(L, t, key, value);  /* insert key into grown table */
+
      return;
    }
    lua_assert(!isdummy(t));
-
    othern = mainposition(t, keytt(mp), &keyval(mp));
+
    othern = mainpositionfromnode(t, mp);
    if (othern != mp) {  /* is colliding node out of its main position? */
      /* yes; move colliding node into free position */
      while (othern + gnext(othern) != mp)  /* find previous */
@@ -682,7 +713,7 @@ TValue *luaH_newkey (lua_State *L, Table *t, const TValue *key) {
  setnodekey(L, mp, key);
  luaC_barrierback(L, obj2gco(t), key);
  lua_assert(isempty(gval(mp)));
-
  return gval(mp);
+
  setobj2t(L, gval(mp), value);
}


@@ -770,28 +801,39 @@ const TValue *luaH_get (Table *t, const TValue *key) {


/*
+
** Finish a raw "set table" operation, where 'slot' is where the value
+
** should have been (the result of a previous "get table").
+
** Beware: when using this function you probably need to check a GC
+
** barrier and invalidate the TM cache.
+
*/
+
void luaH_finishset (lua_State *L, Table *t, const TValue *key,
+
                                   const TValue *slot, TValue *value) {
+
  if (isabstkey(slot))
+
    luaH_newkey(L, t, key, value);
+
  else
+
    setobj2t(L, cast(TValue *, slot), value);
+
}
+

+

+
/*
** beware: when using this function you probably need to check a GC
** barrier and invalidate the TM cache.
*/
-
TValue *luaH_set (lua_State *L, Table *t, const TValue *key) {
-
  const TValue *p = luaH_get(t, key);
-
  if (!isabstkey(p))
-
    return cast(TValue *, p);
-
  else return luaH_newkey(L, t, key);
+
void luaH_set (lua_State *L, Table *t, const TValue *key, TValue *value) {
+
  const TValue *slot = luaH_get(t, key);
+
  luaH_finishset(L, t, key, slot, value);
}


void luaH_setint (lua_State *L, Table *t, lua_Integer key, TValue *value) {
  const TValue *p = luaH_getint(t, key);
-
  TValue *cell;
-
  if (!isabstkey(p))
-
    cell = cast(TValue *, p);
-
  else {
+
  if (isabstkey(p)) {
    TValue k;
    setivalue(&k, key);
-
    cell = luaH_newkey(L, t, &k);
+
    luaH_newkey(L, t, &k, value);
  }
-
  setobj2t(L, cell, value);
+
  else
+
    setobj2t(L, cast(TValue *, p), value);
}


modified external/lua/src/ltable.h
@@ -41,8 +41,12 @@ LUAI_FUNC void luaH_setint (lua_State *L, Table *t, lua_Integer key,
LUAI_FUNC const TValue *luaH_getshortstr (Table *t, TString *key);
LUAI_FUNC const TValue *luaH_getstr (Table *t, TString *key);
LUAI_FUNC const TValue *luaH_get (Table *t, const TValue *key);
-
LUAI_FUNC TValue *luaH_newkey (lua_State *L, Table *t, const TValue *key);
-
LUAI_FUNC TValue *luaH_set (lua_State *L, Table *t, const TValue *key);
+
LUAI_FUNC void luaH_newkey (lua_State *L, Table *t, const TValue *key,
+
                                                    TValue *value);
+
LUAI_FUNC void luaH_set (lua_State *L, Table *t, const TValue *key,
+
                                                 TValue *value);
+
LUAI_FUNC void luaH_finishset (lua_State *L, Table *t, const TValue *key,
+
                                       const TValue *slot, TValue *value);
LUAI_FUNC Table *luaH_new (lua_State *L);
LUAI_FUNC void luaH_resize (lua_State *L, Table *t, unsigned int nasize,
                                                    unsigned int nhsize);
modified external/lua/src/ltablib.c
@@ -59,8 +59,9 @@ static void checktab (lua_State *L, int arg, int what) {


static int tinsert (lua_State *L) {
-
  lua_Integer e = aux_getn(L, 1, TAB_RW) + 1;  /* first empty element */
  lua_Integer pos;  /* where to insert new element */
+
  lua_Integer e = aux_getn(L, 1, TAB_RW);
+
  e = luaL_intop(+, e, 1);  /* first empty element */
  switch (lua_gettop(L)) {
    case 2: {  /* called with only 2 arguments */
      pos = e;  /* insert new element at the end */
@@ -145,9 +146,9 @@ static int tmove (lua_State *L) {

static void addfield (lua_State *L, luaL_Buffer *b, lua_Integer i) {
  lua_geti(L, 1, i);
-
  if (!lua_isstring(L, -1))
-
    luaL_error(L, "invalid value (%s) at index %d in table for 'concat'",
-
                  luaL_typename(L, -1), i);
+
  if (l_unlikely(!lua_isstring(L, -1)))
+
    luaL_error(L, "invalid value (%s) at index %I in table for 'concat'",
+
                  luaL_typename(L, -1), (LUAI_UACINT)i);
  luaL_addvalue(b);
}

@@ -196,7 +197,8 @@ static int tunpack (lua_State *L) {
  lua_Integer e = luaL_opt(L, luaL_checkinteger, 3, luaL_len(L, 1));
  if (i > e) return 0;  /* empty range */
  n = (lua_Unsigned)e - i;  /* number of elements minus 1 (avoid overflows) */
-
  if (n >= (unsigned int)INT_MAX  || !lua_checkstack(L, (int)(++n)))
+
  if (l_unlikely(n >= (unsigned int)INT_MAX  ||
+
                 !lua_checkstack(L, (int)(++n))))
    return luaL_error(L, "too many results to unpack");
  for (; i < e; i++) {  /* push arg[i..e - 1] (to avoid overflows) */
    lua_geti(L, 1, i);
@@ -300,14 +302,14 @@ static IdxT partition (lua_State *L, IdxT lo, IdxT up) {
  for (;;) {
    /* next loop: repeat ++i while a[i] < P */
    while ((void)lua_geti(L, 1, ++i), sort_comp(L, -1, -2)) {
-
      if (i == up - 1)  /* a[i] < P  but a[up - 1] == P  ?? */
+
      if (l_unlikely(i == up - 1))  /* a[i] < P  but a[up - 1] == P  ?? */
        luaL_error(L, "invalid order function for sorting");
      lua_pop(L, 1);  /* remove a[i] */
    }
    /* after the loop, a[i] >= P and a[lo .. i - 1] < P */
    /* next loop: repeat --j while P < a[j] */
    while ((void)lua_geti(L, 1, --j), sort_comp(L, -3, -1)) {
-
      if (j < i)  /* j < i  but  a[j] > P ?? */
+
      if (l_unlikely(j < i))  /* j < i  but  a[j] > P ?? */
        luaL_error(L, "invalid order function for sorting");
      lua_pop(L, 1);  /* remove a[j] */
    }
modified external/lua/src/ltm.c
@@ -147,7 +147,7 @@ static int callbinTM (lua_State *L, const TValue *p1, const TValue *p2,

void luaT_trybinTM (lua_State *L, const TValue *p1, const TValue *p2,
                    StkId res, TMS event) {
-
  if (!callbinTM(L, p1, p2, res, event)) {
+
  if (l_unlikely(!callbinTM(L, p1, p2, res, event))) {
    switch (event) {
      case TM_BAND: case TM_BOR: case TM_BXOR:
      case TM_SHL: case TM_SHR: case TM_BNOT: {
@@ -166,7 +166,8 @@ void luaT_trybinTM (lua_State *L, const TValue *p1, const TValue *p2,

void luaT_tryconcatTM (lua_State *L) {
  StkId top = L->top;
-
  if (!callbinTM(L, s2v(top - 2), s2v(top - 1), top - 2, TM_CONCAT))
+
  if (l_unlikely(!callbinTM(L, s2v(top - 2), s2v(top - 1), top - 2,
+
                               TM_CONCAT)))
    luaG_concaterror(L, s2v(top - 2), s2v(top - 1));
}

modified external/lua/src/lua.c
@@ -37,6 +37,26 @@ static lua_State *globalL = NULL;
static const char *progname = LUA_PROGNAME;


+
#if defined(LUA_USE_POSIX)   /* { */
+

+
/*
+
** Use 'sigaction' when available.
+
*/
+
static void setsignal (int sig, void (*handler)(int)) {
+
  struct sigaction sa;
+
  sa.sa_handler = handler;
+
  sa.sa_flags = 0;
+
  sigemptyset(&sa.sa_mask);  /* do not mask any signal */
+
  sigaction(sig, &sa, NULL);
+
}
+

+
#else           /* }{ */
+

+
#define setsignal            signal
+

+
#endif                               /* } */
+

+

/*
** Hook set by signal function to stop the interpreter.
*/
@@ -55,7 +75,7 @@ static void lstop (lua_State *L, lua_Debug *ar) {
*/
static void laction (int i) {
  int flag = LUA_MASKCALL | LUA_MASKRET | LUA_MASKLINE | LUA_MASKCOUNT;
-
  signal(i, SIG_DFL); /* if another SIGINT happens, terminate process */
+
  setsignal(i, SIG_DFL); /* if another SIGINT happens, terminate process */
  lua_sethook(globalL, lstop, flag, 1);
}

@@ -69,14 +89,15 @@ static void print_usage (const char *badoption) {
  lua_writestringerror(
  "usage: %s [options] [script [args]]\n"
  "Available options are:\n"
-
  "  -e stat  execute string 'stat'\n"
-
  "  -i       enter interactive mode after executing 'script'\n"
-
  "  -l name  require library 'name' into global 'name'\n"
-
  "  -v       show version information\n"
-
  "  -E       ignore environment variables\n"
-
  "  -W       turn warnings on\n"
-
  "  --       stop handling options\n"
-
  "  -        stop handling options and execute stdin\n"
+
  "  -e stat   execute string 'stat'\n"
+
  "  -i        enter interactive mode after executing 'script'\n"
+
  "  -l mod    require library 'mod' into global 'mod'\n"
+
  "  -l g=mod  require library 'mod' into global 'g'\n"
+
  "  -v        show version information\n"
+
  "  -E        ignore environment variables\n"
+
  "  -W        turn warnings on\n"
+
  "  --        stop handling options\n"
+
  "  -         stop handling options and execute stdin\n"
  ,
  progname);
}
@@ -135,9 +156,9 @@ static int docall (lua_State *L, int narg, int nres) {
  lua_pushcfunction(L, msghandler);  /* push message handler */
  lua_insert(L, base);  /* put it under function and args */
  globalL = L;  /* to be available to 'laction' */
-
  signal(SIGINT, laction);  /* set C-signal handler */
+
  setsignal(SIGINT, laction);  /* set C-signal handler */
  status = lua_pcall(L, narg, nres, base);
-
  signal(SIGINT, SIG_DFL); /* reset C-signal handler */
+
  setsignal(SIGINT, SIG_DFL); /* reset C-signal handler */
  lua_remove(L, base);  /* remove message handler from the stack */
  return status;
}
@@ -187,16 +208,22 @@ static int dostring (lua_State *L, const char *s, const char *name) {


/*
-
** Calls 'require(name)' and stores the result in a global variable
-
** with the given name.
+
** Receives 'globname[=modname]' and runs 'globname = require(modname)'.
*/
-
static int dolibrary (lua_State *L, const char *name) {
+
static int dolibrary (lua_State *L, char *globname) {
  int status;
+
  char *modname = strchr(globname, '=');
+
  if (modname == NULL)  /* no explicit name? */
+
    modname = globname;  /* module name is equal to global name */
+
  else {
+
    *modname = '\0';  /* global name ends here */
+
    modname++;  /* module name starts after the '=' */
+
  }
  lua_getglobal(L, "require");
-
  lua_pushstring(L, name);
-
  status = docall(L, 1, 1);  /* call 'require(name)' */
+
  lua_pushstring(L, modname);
+
  status = docall(L, 1, 1);  /* call 'require(modname)' */
  if (status == LUA_OK)
-
    lua_setglobal(L, name);  /* global[name] = require return */
+
    lua_setglobal(L, globname);  /* globname = require(modname) */
  return report(L, status);
}

@@ -307,7 +334,7 @@ static int runargs (lua_State *L, char **argv, int n) {
    switch (option) {
      case 'e':  case 'l': {
        int status;
-
        const char *extra = argv[i] + 2;  /* both options need an argument */
+
        char *extra = argv[i] + 2;  /* both options need an argument */
        if (*extra == '\0') extra = argv[++i];
        lua_assert(extra != NULL);
        status = (option == 'e')
modified external/lua/src/lua.h
@@ -18,14 +18,14 @@

#define LUA_VERSION_MAJOR	"5"
#define LUA_VERSION_MINOR	"4"
-
#define LUA_VERSION_RELEASE	"2"
+
#define LUA_VERSION_RELEASE	"4"

#define LUA_VERSION_NUM			504
-
#define LUA_VERSION_RELEASE_NUM		(LUA_VERSION_NUM * 100 + 0)
+
#define LUA_VERSION_RELEASE_NUM		(LUA_VERSION_NUM * 100 + 4)

#define LUA_VERSION	"Lua " LUA_VERSION_MAJOR "." LUA_VERSION_MINOR
#define LUA_RELEASE	LUA_VERSION "." LUA_VERSION_RELEASE
-
#define LUA_COPYRIGHT	LUA_RELEASE "  Copyright (C) 1994-2020 Lua.org, PUC-Rio"
+
#define LUA_COPYRIGHT	LUA_RELEASE "  Copyright (C) 1994-2022 Lua.org, PUC-Rio"
#define LUA_AUTHORS	"R. Ierusalimschy, L. H. de Figueiredo, W. Celes"


@@ -347,7 +347,8 @@ LUA_API size_t (lua_stringtonumber) (lua_State *L, const char *s);
LUA_API lua_Alloc (lua_getallocf) (lua_State *L, void **ud);
LUA_API void      (lua_setallocf) (lua_State *L, lua_Alloc f, void *ud);

-
LUA_API void  (lua_toclose) (lua_State *L, int idx);
+
LUA_API void (lua_toclose) (lua_State *L, int idx);
+
LUA_API void (lua_closeslot) (lua_State *L, int idx);


/*
@@ -491,7 +492,7 @@ struct lua_Debug {


/******************************************************************************
-
* Copyright (C) 1994-2020 Lua.org, PUC-Rio.
+
* Copyright (C) 1994-2022 Lua.org, PUC-Rio.
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
modified external/lua/src/luac.c
@@ -155,6 +155,7 @@ static const Proto* combine(lua_State* L, int n)
   f->p[i]=toproto(L,i-n-1);
   if (f->p[i]->sizeupvalues>0) f->p[i]->upvalues[0].instack=0;
  }
+
  luaM_freearray(L,f->lineinfo,f->sizelineinfo);
  f->sizelineinfo=0;
  return f;
 }
@@ -600,11 +601,11 @@ static void PrintCode(const Proto* f)
	if (c==0) printf("all out"); else printf("%d out",c-1);
	break;
   case OP_TAILCALL:
-
	printf("%d %d %d",a,b,c);
+
	printf("%d %d %d%s",a,b,c,ISK);
	printf(COMMENT "%d in",b-1);
	break;
   case OP_RETURN:
-
	printf("%d %d %d",a,b,c);
+
	printf("%d %d %d%s",a,b,c,ISK);
	printf(COMMENT);
	if (b==0) printf("all out"); else printf("%d out",b-1);
	break;
@@ -619,7 +620,7 @@ static void PrintCode(const Proto* f)
	break;
   case OP_FORPREP:
	printf("%d %d",a,bx);
-
	printf(COMMENT "to %d",pc+bx+2);
+
	printf(COMMENT "exit to %d",pc+bx+3);
	break;
   case OP_TFORPREP:
	printf("%d %d",a,bx);
modified external/lua/src/luaconf.h
@@ -16,13 +16,13 @@
** ===================================================================
** General Configuration File for Lua
**
-
** Some definitions here can be changed externally, through the
-
** compiler (e.g., with '-D' options). Those are protected by
-
** '#if !defined' guards. However, several other definitions should
-
** be changed directly here, either because they affect the Lua
-
** ABI (by making the changes here, you ensure that all software
-
** connected to Lua, such as C libraries, will be compiled with the
-
** same configuration); or because they are seldom changed.
+
** Some definitions here can be changed externally, through the compiler
+
** (e.g., with '-D' options): They are commented out or protected
+
** by '#if !defined' guards. However, several other definitions
+
** should be changed directly here, either because they affect the
+
** Lua ABI (by making the changes here, you ensure that all software
+
** connected to Lua, such as C libraries, will be compiled with the same
+
** configuration); or because they are seldom changed.
**
** Search for "@@" to find all configurable definitions.
** ===================================================================
@@ -81,27 +81,13 @@

/*
** {==================================================================
-
** Configuration for Number types.
+
** Configuration for Number types. These options should not be
+
** set externally, because any other code connected to Lua must
+
** use the same configuration.
** ===================================================================
*/

/*
-
@@ LUA_32BITS enables Lua with 32-bit integers and 32-bit floats.
-
*/
-
/* #define LUA_32BITS */
-

-

-
/*
-
@@ LUA_C89_NUMBERS ensures that Lua uses the largest types available for
-
** C89 ('long' and 'double'); Windows always has '__int64', so it does
-
** not need to use this case.
-
*/
-
#if defined(LUA_USE_C89) && !defined(LUA_USE_WINDOWS)
-
#define LUA_C89_NUMBERS
-
#endif
-

-

-
/*
@@ LUA_INT_TYPE defines the type for Lua integers.
@@ LUA_FLOAT_TYPE defines the type for Lua floats.
** Lua should work fine with any mix of these options supported
@@ -121,7 +107,31 @@
#define LUA_FLOAT_DOUBLE	2
#define LUA_FLOAT_LONGDOUBLE	3

-
#if defined(LUA_32BITS)		/* { */
+

+
/* Default configuration ('long long' and 'double', for 64-bit Lua) */
+
#define LUA_INT_DEFAULT		LUA_INT_LONGLONG
+
#define LUA_FLOAT_DEFAULT	LUA_FLOAT_DOUBLE
+

+

+
/*
+
@@ LUA_32BITS enables Lua with 32-bit integers and 32-bit floats.
+
*/
+
#define LUA_32BITS	0
+

+

+
/*
+
@@ LUA_C89_NUMBERS ensures that Lua uses the largest types available for
+
** C89 ('long' and 'double'); Windows always has '__int64', so it does
+
** not need to use this case.
+
*/
+
#if defined(LUA_USE_C89) && !defined(LUA_USE_WINDOWS)
+
#define LUA_C89_NUMBERS		1
+
#else
+
#define LUA_C89_NUMBERS		0
+
#endif
+

+

+
#if LUA_32BITS		/* { */
/*
** 32-bit integers and 'float'
*/
@@ -132,26 +142,21 @@
#endif
#define LUA_FLOAT_TYPE	LUA_FLOAT_FLOAT

-
#elif defined(LUA_C89_NUMBERS)	/* }{ */
+
#elif LUA_C89_NUMBERS	/* }{ */
/*
** largest types available for C89 ('long' and 'double')
*/
#define LUA_INT_TYPE	LUA_INT_LONG
#define LUA_FLOAT_TYPE	LUA_FLOAT_DOUBLE

-
#endif				/* } */
+
#else		/* }{ */
+
/* use defaults */

+
#define LUA_INT_TYPE	LUA_INT_DEFAULT
+
#define LUA_FLOAT_TYPE	LUA_FLOAT_DEFAULT

-
/*
-
** default configuration for 64-bit Lua ('long long' and 'double')
-
*/
-
#if !defined(LUA_INT_TYPE)
-
#define LUA_INT_TYPE	LUA_INT_LONGLONG
-
#endif
+
#endif				/* } */

-
#if !defined(LUA_FLOAT_TYPE)
-
#define LUA_FLOAT_TYPE	LUA_FLOAT_DOUBLE
-
#endif

/* }================================================================== */

@@ -373,14 +378,13 @@

/*
** {==================================================================
-
** Configuration for Numbers.
+
** Configuration for Numbers (low-level part).
** Change these definitions if no predefined LUA_FLOAT_* / LUA_INT_*
** satisfy your needs.
** ===================================================================
*/

/*
-
@@ LUA_NUMBER is the floating-point type used by Lua.
@@ LUAI_UACNUMBER is the result of a 'default argument promotion'
@@ over a floating number.
@@ l_floatatt(x) corrects float attribute 'x' to the proper float type
@@ -473,10 +477,7 @@


/*
-
@@ LUA_INTEGER is the integer type used by Lua.
-
**
@@ LUA_UNSIGNED is the unsigned version of LUA_INTEGER.
-
**
@@ LUAI_UACINT is the result of a 'default argument promotion'
@@ over a LUA_INTEGER.
@@ LUA_INTEGER_FRMLEN is the length modifier for reading/writing integers.
@@ -484,7 +485,6 @@
@@ LUA_MAXINTEGER is the maximum value for a LUA_INTEGER.
@@ LUA_MININTEGER is the minimum value for a LUA_INTEGER.
@@ LUA_MAXUNSIGNED is the maximum value for a LUA_UNSIGNED.
-
@@ LUA_UNSIGNEDBITS is the number of bits in a LUA_UNSIGNED.
@@ lua_integer2str converts an integer to a string.
*/

@@ -505,9 +505,6 @@
#define LUA_UNSIGNED		unsigned LUAI_UACINT


-
#define LUA_UNSIGNEDBITS	(sizeof(LUA_UNSIGNED) * CHAR_BIT)
-

-

/* now the variable definitions */

#if LUA_INT_TYPE == LUA_INT_INT		/* { int */
@@ -659,6 +656,34 @@
#define lua_getlocaledecpoint()		(localeconv()->decimal_point[0])
#endif

+

+
/*
+
** macros to improve jump prediction, used mostly for error handling
+
** and debug facilities. (Some macros in the Lua API use these macros.
+
** Define LUA_NOBUILTIN if you do not want '__builtin_expect' in your
+
** code.)
+
*/
+
#if !defined(luai_likely)
+

+
#if defined(__GNUC__) && !defined(LUA_NOBUILTIN)
+
#define luai_likely(x)		(__builtin_expect(((x) != 0), 1))
+
#define luai_unlikely(x)	(__builtin_expect(((x) != 0), 0))
+
#else
+
#define luai_likely(x)		(x)
+
#define luai_unlikely(x)	(x)
+
#endif
+

+
#endif
+

+

+
#if defined(LUA_CORE) || defined(LUA_LIB)
+
/* shorter names for Lua's own use */
+
#define l_likely(x)	luai_likely(x)
+
#define l_unlikely(x)	luai_unlikely(x)
+
#endif
+

+

+

/* }================================================================== */


modified external/lua/src/lualib.h
@@ -49,10 +49,4 @@ LUAMOD_API int (luaopen_package) (lua_State *L);
LUALIB_API void (luaL_openlibs) (lua_State *L);


-

-
#if !defined(lua_assert)
-
#define lua_assert(x)	((void)0)
-
#endif
-

-

#endif
modified external/lua/src/lutf8lib.c
@@ -224,14 +224,11 @@ static int byteoffset (lua_State *L) {
static int iter_aux (lua_State *L, int strict) {
  size_t len;
  const char *s = luaL_checklstring(L, 1, &len);
-
  lua_Integer n = lua_tointeger(L, 2) - 1;
-
  if (n < 0)  /* first iteration? */
-
    n = 0;  /* start from here */
-
  else if (n < (lua_Integer)len) {
-
    n++;  /* skip current byte */
-
    while (iscont(s + n)) n++;  /* and its continuations */
+
  lua_Unsigned n = (lua_Unsigned)lua_tointeger(L, 2);
+
  if (n < len) {
+
    while (iscont(s + n)) n++;  /* skip continuation bytes */
  }
-
  if (n >= (lua_Integer)len)
+
  if (n >= len)  /* (also handles original 'n' being negative) */
    return 0;  /* no more codepoints */
  else {
    utfint code;
modified external/lua/src/lvm.c
@@ -235,11 +235,11 @@ static int forprep (lua_State *L, StkId ra) {
  }
  else {  /* try making all values floats */
    lua_Number init; lua_Number limit; lua_Number step;
-
    if (unlikely(!tonumber(plimit, &limit)))
+
    if (l_unlikely(!tonumber(plimit, &limit)))
      luaG_forerror(L, plimit, "limit");
-
    if (unlikely(!tonumber(pstep, &step)))
+
    if (l_unlikely(!tonumber(pstep, &step)))
      luaG_forerror(L, pstep, "step");
-
    if (unlikely(!tonumber(pinit, &init)))
+
    if (l_unlikely(!tonumber(pinit, &init)))
      luaG_forerror(L, pinit, "initial value");
    if (step == 0)
      luaG_runerror(L, "'for' step is zero");
@@ -292,7 +292,7 @@ void luaV_finishget (lua_State *L, const TValue *t, TValue *key, StkId val,
    if (slot == NULL) {  /* 't' is not a table? */
      lua_assert(!ttistable(t));
      tm = luaT_gettmbyobj(L, t, TM_INDEX);
-
      if (unlikely(notm(tm)))
+
      if (l_unlikely(notm(tm)))
        luaG_typeerror(L, t, "index");  /* no metamethod */
      /* else will try the metamethod */
    }
@@ -337,10 +337,7 @@ void luaV_finishset (lua_State *L, const TValue *t, TValue *key,
      lua_assert(isempty(slot));  /* slot must be empty */
      tm = fasttm(L, h->metatable, TM_NEWINDEX);  /* get metamethod */
      if (tm == NULL) {  /* no metamethod? */
-
        if (isabstkey(slot))  /* no previous entry? */
-
          slot = luaH_newkey(L, h, key);  /* create one */
-
        /* no metamethod and (now) there is an entry with given key */
-
        setobj2t(L, cast(TValue *, slot), val);  /* set its new value */
+
        luaH_finishset(L, h, key, slot, val);  /* set new value */
        invalidateTMcache(h);
        luaC_barrierback(L, obj2gco(h), val);
        return;
@@ -349,7 +346,7 @@ void luaV_finishset (lua_State *L, const TValue *t, TValue *key,
    }
    else {  /* not a table; check metamethod */
      tm = luaT_gettmbyobj(L, t, TM_NEWINDEX);
-
      if (unlikely(notm(tm)))
+
      if (l_unlikely(notm(tm)))
        luaG_typeerror(L, t, "index");
    }
    /* try the metamethod */
@@ -409,7 +406,7 @@ static int l_strcmp (const TString *ls, const TString *rs) {
** from float to int.)
** When 'f' is NaN, comparisons must result in false.
*/
-
static int LTintfloat (lua_Integer i, lua_Number f) {
+
l_sinline int LTintfloat (lua_Integer i, lua_Number f) {
  if (l_intfitsf(i))
    return luai_numlt(cast_num(i), f);  /* compare them as floats */
  else {  /* i < f <=> i < ceil(f) */
@@ -426,7 +423,7 @@ static int LTintfloat (lua_Integer i, lua_Number f) {
** Check whether integer 'i' is less than or equal to float 'f'.
** See comments on previous function.
*/
-
static int LEintfloat (lua_Integer i, lua_Number f) {
+
l_sinline int LEintfloat (lua_Integer i, lua_Number f) {
  if (l_intfitsf(i))
    return luai_numle(cast_num(i), f);  /* compare them as floats */
  else {  /* i <= f <=> i <= floor(f) */
@@ -443,7 +440,7 @@ static int LEintfloat (lua_Integer i, lua_Number f) {
** Check whether float 'f' is less than integer 'i'.
** See comments on previous function.
*/
-
static int LTfloatint (lua_Number f, lua_Integer i) {
+
l_sinline int LTfloatint (lua_Number f, lua_Integer i) {
  if (l_intfitsf(i))
    return luai_numlt(f, cast_num(i));  /* compare them as floats */
  else {  /* f < i <=> floor(f) < i */
@@ -460,7 +457,7 @@ static int LTfloatint (lua_Number f, lua_Integer i) {
** Check whether float 'f' is less than or equal to integer 'i'.
** See comments on previous function.
*/
-
static int LEfloatint (lua_Number f, lua_Integer i) {
+
l_sinline int LEfloatint (lua_Number f, lua_Integer i) {
  if (l_intfitsf(i))
    return luai_numle(f, cast_num(i));  /* compare them as floats */
  else {  /* f <= i <=> ceil(f) <= i */
@@ -476,7 +473,7 @@ static int LEfloatint (lua_Number f, lua_Integer i) {
/*
** Return 'l < r', for numbers.
*/
-
static int LTnum (const TValue *l, const TValue *r) {
+
l_sinline int LTnum (const TValue *l, const TValue *r) {
  lua_assert(ttisnumber(l) && ttisnumber(r));
  if (ttisinteger(l)) {
    lua_Integer li = ivalue(l);
@@ -498,7 +495,7 @@ static int LTnum (const TValue *l, const TValue *r) {
/*
** Return 'l <= r', for numbers.
*/
-
static int LEnum (const TValue *l, const TValue *r) {
+
l_sinline int LEnum (const TValue *l, const TValue *r) {
  lua_assert(ttisnumber(l) && ttisnumber(r));
  if (ttisinteger(l)) {
    lua_Integer li = ivalue(l);
@@ -571,8 +568,13 @@ int luaV_equalobj (lua_State *L, const TValue *t1, const TValue *t2) {
    if (ttype(t1) != ttype(t2) || ttype(t1) != LUA_TNUMBER)
      return 0;  /* only numbers can be equal with different variants */
    else {  /* two numbers with different variants */
-
      lua_Integer i1, i2;  /* compare them as integers */
-
      return (tointegerns(t1, &i1) && tointegerns(t2, &i2) && i1 == i2);
+
      /* One of them is an integer. If the other does not have an
+
         integer value, they cannot be equal; otherwise, compare their
+
         integer values. */
+
      lua_Integer i1, i2;
+
      return (luaV_tointegerns(t1, &i1, F2Ieq) &&
+
              luaV_tointegerns(t2, &i2, F2Ieq) &&
+
              i1 == i2);
    }
  }
  /* values have same type and same variant */
@@ -654,7 +656,7 @@ void luaV_concat (lua_State *L, int total) {
      /* collect total length and number of strings */
      for (n = 1; n < total && tostring(L, s2v(top - n - 1)); n++) {
        size_t l = vslen(s2v(top - n - 1));
-
        if (unlikely(l >= (MAX_SIZE/sizeof(char)) - tl))
+
        if (l_unlikely(l >= (MAX_SIZE/sizeof(char)) - tl))
          luaG_runerror(L, "string length overflow");
        tl += l;
      }
@@ -698,7 +700,7 @@ void luaV_objlen (lua_State *L, StkId ra, const TValue *rb) {
    }
    default: {  /* try metamethod */
      tm = luaT_gettmbyobj(L, rb, TM_LEN);
-
      if (unlikely(notm(tm)))  /* no metamethod? */
+
      if (l_unlikely(notm(tm)))  /* no metamethod? */
        luaG_typeerror(L, rb, "get length of");
      break;
    }
@@ -714,7 +716,7 @@ void luaV_objlen (lua_State *L, StkId ra, const TValue *rb) {
** otherwise 'floor(q) == trunc(q) - 1'.
*/
lua_Integer luaV_idiv (lua_State *L, lua_Integer m, lua_Integer n) {
-
  if (unlikely(l_castS2U(n) + 1u <= 1u)) {  /* special cases: -1 or 0 */
+
  if (l_unlikely(l_castS2U(n) + 1u <= 1u)) {  /* special cases: -1 or 0 */
    if (n == 0)
      luaG_runerror(L, "attempt to divide by zero");
    return intop(-, 0, m);   /* n==-1; avoid overflow with 0x80000...//-1 */
@@ -734,7 +736,7 @@ lua_Integer luaV_idiv (lua_State *L, lua_Integer m, lua_Integer n) {
** about luaV_idiv.)
*/
lua_Integer luaV_mod (lua_State *L, lua_Integer m, lua_Integer n) {
-
  if (unlikely(l_castS2U(n) + 1u <= 1u)) {  /* special cases: -1 or 0 */
+
  if (l_unlikely(l_castS2U(n) + 1u <= 1u)) {  /* special cases: -1 or 0 */
    if (n == 0)
      luaG_runerror(L, "attempt to perform 'n%%0'");
    return 0;   /* m % -1 == 0; avoid overflow with 0x80000...%-1 */
@@ -764,7 +766,8 @@ lua_Number luaV_modf (lua_State *L, lua_Number m, lua_Number n) {
/*
** Shift left operation. (Shift right just negates 'y'.)
*/
-
#define luaV_shiftr(x,y)	luaV_shiftl(x,-(y))
+
#define luaV_shiftr(x,y)	luaV_shiftl(x,intop(-, 0, y))
+


lua_Integer luaV_shiftl (lua_Integer x, lua_Integer y) {
  if (y < 0) {  /* shift right? */
@@ -845,6 +848,19 @@ void luaV_finishOp (lua_State *L) {
      luaV_concat(L, total);  /* concat them (may yield again) */
      break;
    }
+
    case OP_CLOSE: {  /* yielded closing variables */
+
      ci->u.l.savedpc--;  /* repeat instruction to close other vars. */
+
      break;
+
    }
+
    case OP_RETURN: {  /* yielded closing variables */
+
      StkId ra = base + GETARG_A(inst);
+
      /* adjust top to signal correct number of returns, in case the
+
         return is "up to top" ('isIT') */
+
      L->top = ra + ci->u2.nres;
+
      /* repeat instruction to close other vars. and complete the return */
+
      ci->u.l.savedpc--;
+
      break;
+
    }
    default: {
      /* only these other opcodes can yield */
      lua_assert(op == OP_TFORCALL || op == OP_CALL ||
@@ -920,7 +936,7 @@ void luaV_finishOp (lua_State *L) {
*/
#define op_arithfK(L,fop) {  \
  TValue *v1 = vRB(i);  \
-
  TValue *v2 = KC(i);  \
+
  TValue *v2 = KC(i); lua_assert(ttisnumber(v2));  \
  op_arithf_aux(L, v1, v2, fop); }


@@ -949,7 +965,7 @@ void luaV_finishOp (lua_State *L) {
*/
#define op_arithK(L,iop,fop) {  \
  TValue *v1 = vRB(i);  \
-
  TValue *v2 = KC(i);  \
+
  TValue *v2 = KC(i); lua_assert(ttisnumber(v2));  \
  op_arith_aux(L, v1, v2, iop, fop); }


@@ -1048,7 +1064,8 @@ void luaV_finishOp (lua_State *L) {
#define updatebase(ci)	(base = ci->func + 1)


-
#define updatestack(ci) { if (trap) { updatebase(ci); ra = RA(i); } }
+
#define updatestack(ci)  \
+
	{ if (l_unlikely(trap)) { updatebase(ci); ra = RA(i); } }


/*
@@ -1092,7 +1109,7 @@ void luaV_finishOp (lua_State *L) {
#define ProtectNT(exp)  (savepc(L), (exp), updatetrap(ci))

/*
-
** Protect code that can only raise errors. (That is, it cannnot change
+
** Protect code that can only raise errors. (That is, it cannot change
** the stack or hooks.)
*/
#define halfProtect(exp)  (savestate(L,ci), (exp))
@@ -1106,7 +1123,7 @@ void luaV_finishOp (lua_State *L) {

/* fetch an instruction and prepare its execution */
#define vmfetch()	{ \
-
  if (trap) {  /* stack reallocation or hooks? */ \
+
  if (l_unlikely(trap)) {  /* stack reallocation or hooks? */ \
    trap = luaG_traceexec(L, pc);  /* handle hooks */ \
    updatebase(ci);  /* correct stack */ \
  } \
@@ -1134,7 +1151,7 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
  cl = clLvalue(s2v(ci->func));
  k = cl->p->k;
  pc = ci->u.l.savedpc;
-
  if (trap) {
+
  if (l_unlikely(trap)) {
    if (pc == cl->p->code) {  /* first instruction (not resuming)? */
      if (cl->p->is_vararg)
        trap = 0;  /* hooks will start after VARARGPREP instruction */
@@ -1149,6 +1166,10 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
    Instruction i;  /* instruction being executed */
    StkId ra;  /* instruction's A register */
    vmfetch();
+
    #if 0
+
      /* low-level line tracing for debugging Lua */
+
      printf("line: %d\n", luaG_getfuncline(cl->p, pcRel(pc, cl->p)));
+
    #endif
    lua_assert(base == ci->func + 1);
    lua_assert(base <= L->top && L->top < L->stack_last);
    /* invalidate top for instructions not expecting it */
@@ -1527,7 +1548,7 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
        vmbreak;
      }
      vmcase(OP_CLOSE) {
-
        Protect(luaF_close(L, ra, LUA_OK));
+
        Protect(luaF_close(L, ra, LUA_OK, 1));
        vmbreak;
      }
      vmcase(OP_TBC) {
@@ -1616,13 +1637,13 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
          updatetrap(ci);  /* C call; nothing else to be done */
        else {  /* Lua call: run function in this same C frame */
          ci = newci;
-
          ci->callstatus = 0;  /* call re-uses 'luaV_execute' */
          goto startfunc;
        }
        vmbreak;
      }
      vmcase(OP_TAILCALL) {
        int b = GETARG_B(i);  /* number of arguments + 1 (function) */
+
        int n;  /* number of results when calling a C function */
        int nparams1 = GETARG_C(i);
        /* delta is virtual 'func' - real 'func' (vararg functions) */
        int delta = (nparams1) ? ci->u.l.nextraargs + nparams1 : 0;
@@ -1632,29 +1653,18 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
          b = cast_int(L->top - ra);
        savepc(ci);  /* several calls here can raise errors */
        if (TESTARG_k(i)) {
-
          /* close upvalues from current call; the compiler ensures
-
             that there are no to-be-closed variables here, so this
-
             call cannot change the stack */
-
          luaF_close(L, base, NOCLOSINGMETH);
+
          luaF_closeupval(L, base);  /* close upvalues from current call */
+
          lua_assert(L->tbclist < base);  /* no pending tbc variables */
          lua_assert(base == ci->func + 1);
        }
-
        while (!ttisfunction(s2v(ra))) {  /* not a function? */
-
          luaD_tryfuncTM(L, ra);  /* try '__call' metamethod */
-
          b++;  /* there is now one extra argument */
-
          checkstackGCp(L, 1, ra);
-
        }
-
        if (!ttisLclosure(s2v(ra))) {  /* C function? */
-
          luaD_precall(L, ra, LUA_MULTRET);  /* call it */
-
          updatetrap(ci);
-
          updatestack(ci);  /* stack may have been relocated */
+
        if ((n = luaD_pretailcall(L, ci, ra, b, delta)) < 0)  /* Lua function? */
+
          goto startfunc;  /* execute the callee */
+
        else {  /* C function? */
          ci->func -= delta;  /* restore 'func' (if vararg) */
-
          luaD_poscall(L, ci, cast_int(L->top - ra));  /* finish caller */
+
          luaD_poscall(L, ci, n);  /* finish caller */
          updatetrap(ci);  /* 'luaD_poscall' can change hooks */
          goto ret;  /* caller returns after the tail call */
        }
-
        ci->func -= delta;  /* restore 'func' (if vararg) */
-
        luaD_pretailcall(L, ci, ra, b);  /* prepare call frame */
-
        goto startfunc;  /* execute the callee */
      }
      vmcase(OP_RETURN) {
        int n = GETARG_B(i) - 1;  /* number of results */
@@ -1663,9 +1673,10 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
          n = cast_int(L->top - ra);  /* get what is available */
        savepc(ci);
        if (TESTARG_k(i)) {  /* may there be open upvalues? */
+
          ci->u2.nres = n;  /* save number of returns */
          if (L->top < ci->top)
            L->top = ci->top;
-
          luaF_close(L, base, LUA_OK);
+
          luaF_close(L, base, CLOSEKTOP, 1);
          updatetrap(ci);
          updatestack(ci);
        }
@@ -1677,23 +1688,23 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
        goto ret;
      }
      vmcase(OP_RETURN0) {
-
        if (L->hookmask) {
+
        if (l_unlikely(L->hookmask)) {
          L->top = ra;
          savepc(ci);
          luaD_poscall(L, ci, 0);  /* no hurry... */
          trap = 1;
        }
        else {  /* do the 'poscall' here */
-
          int nres = ci->nresults;
+
          int nres;
          L->ci = ci->previous;  /* back to caller */
          L->top = base - 1;
-
          while (nres-- > 0)
+
          for (nres = ci->nresults; l_unlikely(nres > 0); nres--)
            setnilvalue(s2v(L->top++));  /* all results are nil */
        }
        goto ret;
      }
      vmcase(OP_RETURN1) {
-
        if (L->hookmask) {
+
        if (l_unlikely(L->hookmask)) {
          L->top = ra + 1;
          savepc(ci);
          luaD_poscall(L, ci, 1);  /* no hurry... */
@@ -1707,8 +1718,8 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
          else {
            setobjs2s(L, base - 1, ra);  /* at least this result */
            L->top = base;
-
            while (--nres > 0)  /* complete missing results */
-
              setnilvalue(s2v(L->top++));
+
            for (; l_unlikely(nres > 1); nres--)
+
              setnilvalue(s2v(L->top++));  /* complete missing results */
          }
        }
       ret:  /* return from a Lua function */
@@ -1811,7 +1822,7 @@ void luaV_execute (lua_State *L, CallInfo *ci) {
      }
      vmcase(OP_VARARGPREP) {
        ProtectNT(luaT_adjustvarargs(L, GETARG_A(i), ci, cl->p));
-
        if (trap) {
+
        if (l_unlikely(trap)) {  /* previous "Protect" updated trap */
          luaD_hookcall(L, ci);
          L->oldpc = 1;  /* next opcode will be seen as a "new" line */
        }
modified external/lua/src/lvm.h
@@ -60,12 +60,14 @@ typedef enum {

/* convert an object to an integer (including string coercion) */
#define tointeger(o,i) \
-
  (ttisinteger(o) ? (*(i) = ivalue(o), 1) : luaV_tointeger(o,i,LUA_FLOORN2I))
+
  (l_likely(ttisinteger(o)) ? (*(i) = ivalue(o), 1) \
+
                          : luaV_tointeger(o,i,LUA_FLOORN2I))


/* convert an object to an integer (without string coercion) */
#define tointegerns(o,i) \
-
  (ttisinteger(o) ? (*(i) = ivalue(o), 1) : luaV_tointegerns(o,i,LUA_FLOORN2I))
+
  (l_likely(ttisinteger(o)) ? (*(i) = ivalue(o), 1) \
+
                          : luaV_tointegerns(o,i,LUA_FLOORN2I))


#define intop(op,v1,v2) l_castU2S(l_castS2U(v1) op l_castS2U(v2))
modified external/sqlite/shell.c
@@ -34,6 +34,8 @@
/* This needs to come before any includes for MSVC compiler */
#define _CRT_SECURE_NO_WARNINGS
#endif
+
typedef unsigned int u32;
+
typedef unsigned short int u16;

/*
** Optionally #include a user-defined header, whereby compilation options
@@ -56,6 +58,15 @@
#endif

/*
+
** If SQLITE_SHELL_FIDDLE is defined then the shell is modified
+
** somewhat for use as a WASM module in a web browser. This flag
+
** should only be used when building the "fiddle" web application, as
+
** the browser-mode build has much different user input requirements
+
** and this build mode rewires the user input subsystem to account for
+
** that.
+
*/
+

+
/*
** Warning pragmas copied from msvc.h in the core.
*/
#if defined(_MSC_VER)
@@ -94,6 +105,14 @@
# define _LARGEFILE_SOURCE 1
#endif

+
#if defined(SQLITE_SHELL_FIDDLE) && !defined(_POSIX_SOURCE)
+
/*
+
** emcc requires _POSIX_SOURCE (or one of several similar defines)
+
** to expose strdup().
+
*/
+
# define _POSIX_SOURCE
+
#endif
+

#include <stdlib.h>
#include <string.h>
#include <stdio.h>
@@ -247,10 +266,21 @@ static void setTextMode(FILE *file, int isOutput){
# define setTextMode(X,Y)
#endif

-

/* True if the timer is enabled */
static int enableTimer = 0;

+
/* A version of strcmp() that works with NULL values */
+
static int cli_strcmp(const char *a, const char *b){
+
  if( a==0 ) a = "";
+
  if( b==0 ) b = "";
+
  return strcmp(a,b);
+
}
+
static int cli_strncmp(const char *a, const char *b, size_t n){
+
  if( a==0 ) a = "";
+
  if( b==0 ) b = "";
+
  return strncmp(a,b,n);
+
}
+

/* Return the current wall-clock time */
static sqlite3_int64 timeOfDay(void){
  static sqlite3_vfs *clockVfs = 0;
@@ -539,6 +569,7 @@ static void utf8_width_print(FILE *pOut, int w, const char *zUtf){
  int i;
  int n;
  int aw = w<0 ? -w : w;
+
  if( zUtf==0 ) zUtf = "";
  for(i=n=0; zUtf[i]; i++){
    if( (zUtf[i]&0xc0)!=0x80 ){
      n++;
@@ -682,7 +713,7 @@ static char *local_getline(char *zLine, FILE *in){
  if( stdin_is_interactive && in==stdin ){
    char *zTrans = sqlite3_win32_mbcs_to_utf8_v2(zLine, 0);
    if( zTrans ){
-
      int nTrans = strlen30(zTrans)+1;
+
      i64 nTrans = strlen(zTrans)+1;
      if( nTrans>nLine ){
        zLine = realloc(zLine, nTrans);
        shell_check_oom(zLine);
@@ -709,6 +740,7 @@ static char *local_getline(char *zLine, FILE *in){
** be freed by the caller or else passed back into this routine via the
** zPrior argument for reuse.
*/
+
#ifndef SQLITE_SHELL_FIDDLE
static char *one_input_line(FILE *in, char *zPrior, int isContinuation){
  char *zPrompt;
  char *zResult;
@@ -728,7 +760,7 @@ static char *one_input_line(FILE *in, char *zPrior, int isContinuation){
  }
  return zResult;
}
-

+
#endif /* !SQLITE_SHELL_FIDDLE */

/*
** Return the value of a hexadecimal digit.  Return -1 if the input
@@ -816,10 +848,10 @@ static void freeText(ShellText *p){
** If the third argument, quote, is not '\0', then it is used as a
** quote character for zAppend.
*/
-
static void appendText(ShellText *p, char const *zAppend, char quote){
-
  int len;
-
  int i;
-
  int nAppend = strlen30(zAppend);
+
static void appendText(ShellText *p, const char *zAppend, char quote){
+
  i64 len;
+
  i64 i;
+
  i64 nAppend = strlen30(zAppend);

  len = nAppend+p->n+1;
  if( quote ){
@@ -978,10 +1010,10 @@ static void shellAddSchemaName(
  const char *zName = (const char*)sqlite3_value_text(apVal[2]);
  sqlite3 *db = sqlite3_context_db_handle(pCtx);
  UNUSED_PARAMETER(nVal);
-
  if( zIn!=0 && strncmp(zIn, "CREATE ", 7)==0 ){
+
  if( zIn!=0 && cli_strncmp(zIn, "CREATE ", 7)==0 ){
    for(i=0; i<ArraySize(aPrefix); i++){
      int n = strlen30(aPrefix[i]);
-
      if( strncmp(zIn+7, aPrefix[i], n)==0 && zIn[n+7]==' ' ){
+
      if( cli_strncmp(zIn+7, aPrefix[i], n)==0 && zIn[n+7]==' ' ){
        char *z = 0;
        char *zFake = 0;
        if( zSchema ){
@@ -1381,6 +1413,117 @@ INT closedir(
/************************* End test_windirent.c ********************/
#define dirent DIRENT
#endif
+
/************************* Begin ../ext/misc/memtrace.c ******************/
+
/*
+
** 2019-01-21
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
*************************************************************************
+
**
+
** This file implements an extension that uses the SQLITE_CONFIG_MALLOC
+
** mechanism to add a tracing layer on top of SQLite.  If this extension
+
** is registered prior to sqlite3_initialize(), it will cause all memory
+
** allocation activities to be logged on standard output, or to some other
+
** FILE specified by the initializer.
+
**
+
** This file needs to be compiled into the application that uses it.
+
**
+
** This extension is used to implement the --memtrace option of the
+
** command-line shell.
+
*/
+
#include <assert.h>
+
#include <string.h>
+
#include <stdio.h>
+

+
/* The original memory allocation routines */
+
static sqlite3_mem_methods memtraceBase;
+
static FILE *memtraceOut;
+

+
/* Methods that trace memory allocations */
+
static void *memtraceMalloc(int n){
+
  if( memtraceOut ){
+
    fprintf(memtraceOut, "MEMTRACE: allocate %d bytes\n", 
+
            memtraceBase.xRoundup(n));
+
  }
+
  return memtraceBase.xMalloc(n);
+
}
+
static void memtraceFree(void *p){
+
  if( p==0 ) return;
+
  if( memtraceOut ){
+
    fprintf(memtraceOut, "MEMTRACE: free %d bytes\n", memtraceBase.xSize(p));
+
  }
+
  memtraceBase.xFree(p);
+
}
+
static void *memtraceRealloc(void *p, int n){
+
  if( p==0 ) return memtraceMalloc(n);
+
  if( n==0 ){
+
    memtraceFree(p);
+
    return 0;
+
  }
+
  if( memtraceOut ){
+
    fprintf(memtraceOut, "MEMTRACE: resize %d -> %d bytes\n",
+
            memtraceBase.xSize(p), memtraceBase.xRoundup(n));
+
  }
+
  return memtraceBase.xRealloc(p, n);
+
}
+
static int memtraceSize(void *p){
+
  return memtraceBase.xSize(p);
+
}
+
static int memtraceRoundup(int n){
+
  return memtraceBase.xRoundup(n);
+
}
+
static int memtraceInit(void *p){
+
  return memtraceBase.xInit(p);
+
}
+
static void memtraceShutdown(void *p){
+
  memtraceBase.xShutdown(p);
+
}
+

+
/* The substitute memory allocator */
+
static sqlite3_mem_methods ersaztMethods = {
+
  memtraceMalloc,
+
  memtraceFree,
+
  memtraceRealloc,
+
  memtraceSize,
+
  memtraceRoundup,
+
  memtraceInit,
+
  memtraceShutdown,
+
  0
+
};
+

+
/* Begin tracing memory allocations to out. */
+
int sqlite3MemTraceActivate(FILE *out){
+
  int rc = SQLITE_OK;
+
  if( memtraceBase.xMalloc==0 ){
+
    rc = sqlite3_config(SQLITE_CONFIG_GETMALLOC, &memtraceBase);
+
    if( rc==SQLITE_OK ){
+
      rc = sqlite3_config(SQLITE_CONFIG_MALLOC, &ersaztMethods);
+
    }
+
  }
+
  memtraceOut = out;
+
  return rc;
+
}
+

+
/* Deactivate memory tracing */
+
int sqlite3MemTraceDeactivate(void){
+
  int rc = SQLITE_OK;
+
  if( memtraceBase.xMalloc!=0 ){
+
    rc = sqlite3_config(SQLITE_CONFIG_MALLOC, &memtraceBase);
+
    if( rc==SQLITE_OK ){
+
      memset(&memtraceBase, 0, sizeof(memtraceBase));
+
    }
+
  }
+
  memtraceOut = 0;
+
  return rc;
+
}
+

+
/************************* End ../ext/misc/memtrace.c ********************/
/************************* Begin ../ext/misc/shathree.c ******************/
/*
** 2017-03-08
@@ -1403,7 +1546,7 @@ INT closedir(
** The sha3(X) function computes the SHA3 hash of the input X, or NULL if
** X is NULL.
**
-
** The sha3_query(Y) function evalutes all queries in the SQL statements of Y
+
** The sha3_query(Y) function evaluates all queries in the SQL statements of Y
** and returns a hash of their results.
**
** The SIZE argument is optional.  If omitted, the SHA3-256 hash algorithm
@@ -2108,9 +2251,9 @@ int sqlite3_shathree_init(
}

/************************* End ../ext/misc/shathree.c ********************/
-
/************************* Begin ../ext/misc/fileio.c ******************/
+
/************************* Begin ../ext/misc/uint.c ******************/
/*
-
** 2014-06-13
+
** 2020-04-14
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
@@ -2121,1029 +2264,729 @@ int sqlite3_shathree_init(
**
******************************************************************************
**
-
** This SQLite extension implements SQL functions readfile() and
-
** writefile(), and eponymous virtual type "fsdir".
-
**
-
** WRITEFILE(FILE, DATA [, MODE [, MTIME]]):
-
**
-
**   If neither of the optional arguments is present, then this UDF
-
**   function writes blob DATA to file FILE. If successful, the number
-
**   of bytes written is returned. If an error occurs, NULL is returned.
-
**
-
**   If the first option argument - MODE - is present, then it must
-
**   be passed an integer value that corresponds to a POSIX mode
-
**   value (file type + permissions, as returned in the stat.st_mode
-
**   field by the stat() system call). Three types of files may
-
**   be written/created:
-
**
-
**     regular files:  (mode & 0170000)==0100000
-
**     symbolic links: (mode & 0170000)==0120000
-
**     directories:    (mode & 0170000)==0040000
-
**
-
**   For a directory, the DATA is ignored. For a symbolic link, it is
-
**   interpreted as text and used as the target of the link. For a
-
**   regular file, it is interpreted as a blob and written into the
-
**   named file. Regardless of the type of file, its permissions are
-
**   set to (mode & 0777) before returning.
-
**
-
**   If the optional MTIME argument is present, then it is interpreted
-
**   as an integer - the number of seconds since the unix epoch. The
-
**   modification-time of the target file is set to this value before
-
**   returning.
-
**
-
**   If three or more arguments are passed to this function and an
-
**   error is encountered, an exception is raised.
-
**
-
** READFILE(FILE):
-
**
-
**   Read and return the contents of file FILE (type blob) from disk.
-
**
-
** FSDIR:
-
**
-
**   Used as follows:
-
**
-
**     SELECT * FROM fsdir($path [, $dir]);
-
**
-
**   Parameter $path is an absolute or relative pathname. If the file that it
-
**   refers to does not exist, it is an error. If the path refers to a regular
-
**   file or symbolic link, it returns a single row. Or, if the path refers
-
**   to a directory, it returns one row for the directory, and one row for each
-
**   file within the hierarchy rooted at $path.
+
** This SQLite extension implements the UINT collating sequence.
**
-
**   Each row has the following columns:
+
** UINT works like BINARY for text, except that embedded strings
+
** of digits compare in numeric order.
**
-
**     name:  Path to file or directory (text value).
-
**     mode:  Value of stat.st_mode for directory entry (an integer).
-
**     mtime: Value of stat.st_mtime for directory entry (an integer).
-
**     data:  For a regular file, a blob containing the file data. For a
-
**            symlink, a text value containing the text of the link. For a
-
**            directory, NULL.
+
**     *   Leading zeros are handled properly, in the sense that
+
**         they do not mess of the maginitude comparison of embedded
+
**         strings of digits.  "x00123y" is equal to "x123y".
**
-
**   If a non-NULL value is specified for the optional $dir parameter and
-
**   $path is a relative path, then $path is interpreted relative to $dir. 
-
**   And the paths returned in the "name" column of the table are also 
-
**   relative to directory $dir.
+
**     *   Only unsigned integers are recognized.  Plus and minus
+
**         signs are ignored.  Decimal points and exponential notation
+
**         are ignored.
**
-
** Notes on building this extension for Windows:
-
**   Unless linked statically with the SQLite library, a preprocessor
-
**   symbol, FILEIO_WIN32_DLL, must be #define'd to create a stand-alone
-
**   DLL form of this extension for WIN32. See its use below for details.
+
**     *   Embedded integers can be of arbitrary length.  Comparison
+
**         is *not* limited integers that can be expressed as a
+
**         64-bit machine integer.
*/
/* #include "sqlite3ext.h" */
SQLITE_EXTENSION_INIT1
-
#include <stdio.h>
-
#include <string.h>
#include <assert.h>
-

-
#include <sys/types.h>
-
#include <sys/stat.h>
-
#include <fcntl.h>
-
#if !defined(_WIN32) && !defined(WIN32)
-
#  include <unistd.h>
-
#  include <dirent.h>
-
#  include <utime.h>
-
#  include <sys/time.h>
-
#else
-
#  include "windows.h"
-
#  include <io.h>
-
#  include <direct.h>
-
/* #  include "test_windirent.h" */
-
#  define dirent DIRENT
-
#  ifndef chmod
-
#    define chmod _chmod
-
#  endif
-
#  ifndef stat
-
#    define stat _stat
-
#  endif
-
#  define mkdir(path,mode) _mkdir(path)
-
#  define lstat(path,buf) stat(path,buf)
-
#endif
-
#include <time.h>
-
#include <errno.h>
-

+
#include <string.h>
+
#include <ctype.h>

/*
-
** Structure of the fsdir() table-valued function
+
** Compare text in lexicographic order, except strings of digits
+
** compare in numeric order.
*/
-
                 /*    0    1    2     3    4           5             */
-
#define FSDIR_SCHEMA "(name,mode,mtime,data,path HIDDEN,dir HIDDEN)"
-
#define FSDIR_COLUMN_NAME     0     /* Name of the file */
-
#define FSDIR_COLUMN_MODE     1     /* Access mode */
-
#define FSDIR_COLUMN_MTIME    2     /* Last modification time */
-
#define FSDIR_COLUMN_DATA     3     /* File content */
-
#define FSDIR_COLUMN_PATH     4     /* Path to top of search */
-
#define FSDIR_COLUMN_DIR      5     /* Path is relative to this directory */
-

-

-
/*
-
** Set the result stored by context ctx to a blob containing the 
-
** contents of file zName.  Or, leave the result unchanged (NULL)
-
** if the file does not exist or is unreadable.
-
**
-
** If the file exceeds the SQLite blob size limit, through an
-
** SQLITE_TOOBIG error.
-
**
-
** Throw an SQLITE_IOERR if there are difficulties pulling the file
-
** off of disk.
-
*/
-
static void readFileContents(sqlite3_context *ctx, const char *zName){
-
  FILE *in;
-
  sqlite3_int64 nIn;
-
  void *pBuf;
-
  sqlite3 *db;
-
  int mxBlob;
-

-
  in = fopen(zName, "rb");
-
  if( in==0 ){
-
    /* File does not exist or is unreadable. Leave the result set to NULL. */
-
    return;
-
  }
-
  fseek(in, 0, SEEK_END);
-
  nIn = ftell(in);
-
  rewind(in);
-
  db = sqlite3_context_db_handle(ctx);
-
  mxBlob = sqlite3_limit(db, SQLITE_LIMIT_LENGTH, -1);
-
  if( nIn>mxBlob ){
-
    sqlite3_result_error_code(ctx, SQLITE_TOOBIG);
-
    fclose(in);
-
    return;
-
  }
-
  pBuf = sqlite3_malloc64( nIn ? nIn : 1 );
-
  if( pBuf==0 ){
-
    sqlite3_result_error_nomem(ctx);
-
    fclose(in);
-
    return;
-
  }
-
  if( nIn==(sqlite3_int64)fread(pBuf, 1, (size_t)nIn, in) ){
-
    sqlite3_result_blob64(ctx, pBuf, nIn, sqlite3_free);
-
  }else{
-
    sqlite3_result_error_code(ctx, SQLITE_IOERR);
-
    sqlite3_free(pBuf);
+
static int uintCollFunc(
+
  void *notUsed,
+
  int nKey1, const void *pKey1,
+
  int nKey2, const void *pKey2
+
){
+
  const unsigned char *zA = (const unsigned char*)pKey1;
+
  const unsigned char *zB = (const unsigned char*)pKey2;
+
  int i=0, j=0, x;
+
  (void)notUsed;
+
  while( i<nKey1 && j<nKey2 ){
+
    x = zA[i] - zB[j];
+
    if( isdigit(zA[i]) ){
+
      int k;
+
      if( !isdigit(zB[j]) ) return x;
+
      while( i<nKey1 && zA[i]=='0' ){ i++; }
+
      while( j<nKey2 && zB[j]=='0' ){ j++; }
+
      k = 0;
+
      while( i+k<nKey1 && isdigit(zA[i+k])
+
             && j+k<nKey2 && isdigit(zB[j+k]) ){
+
        k++;
+
      }
+
      if( i+k<nKey1 && isdigit(zA[i+k]) ){
+
        return +1;
+
      }else if( j+k<nKey2 && isdigit(zB[j+k]) ){
+
        return -1;
+
      }else{
+
        x = memcmp(zA+i, zB+j, k);
+
        if( x ) return x;
+
        i += k;
+
        j += k;
+
      }
+
    }else if( x ){
+
      return x;
+
    }else{
+
      i++;
+
      j++;
+
    }
  }
-
  fclose(in);
+
  return (nKey1 - i) - (nKey2 - j);
}

-
/*
-
** Implementation of the "readfile(X)" SQL function.  The entire content
-
** of the file named X is read and returned as a BLOB.  NULL is returned
-
** if the file does not exist or is unreadable.
-
*/
-
static void readfileFunc(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  const char *zName;
-
  (void)(argc);  /* Unused parameter */
-
  zName = (const char*)sqlite3_value_text(argv[0]);
-
  if( zName==0 ) return;
-
  readFileContents(context, zName);
-
}
+
#ifdef _WIN32

-
/*
-
** Set the error message contained in context ctx to the results of
-
** vprintf(zFmt, ...).
-
*/
-
static void ctxErrorMsg(sqlite3_context *ctx, const char *zFmt, ...){
-
  char *zMsg = 0;
-
  va_list ap;
-
  va_start(ap, zFmt);
-
  zMsg = sqlite3_vmprintf(zFmt, ap);
-
  sqlite3_result_error(ctx, zMsg, -1);
-
  sqlite3_free(zMsg);
-
  va_end(ap);
+
#endif
+
int sqlite3_uint_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
+
){
+
  SQLITE_EXTENSION_INIT2(pApi);
+
  (void)pzErrMsg;  /* Unused parameter */
+
  return sqlite3_create_collation(db, "uint", SQLITE_UTF8, 0, uintCollFunc);
}

-
#if defined(_WIN32)
+
/************************* End ../ext/misc/uint.c ********************/
+
/************************* Begin ../ext/misc/decimal.c ******************/
/*
-
** This function is designed to convert a Win32 FILETIME structure into the
-
** number of seconds since the Unix Epoch (1970-01-01 00:00:00 UTC).
+
** 2020-06-22
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** Routines to implement arbitrary-precision decimal math.
+
**
+
** The focus here is on simplicity and correctness, not performance.
*/
-
static sqlite3_uint64 fileTimeToUnixTime(
-
  LPFILETIME pFileTime
-
){
-
  SYSTEMTIME epochSystemTime;
-
  ULARGE_INTEGER epochIntervals;
-
  FILETIME epochFileTime;
-
  ULARGE_INTEGER fileIntervals;
-

-
  memset(&epochSystemTime, 0, sizeof(SYSTEMTIME));
-
  epochSystemTime.wYear = 1970;
-
  epochSystemTime.wMonth = 1;
-
  epochSystemTime.wDay = 1;
-
  SystemTimeToFileTime(&epochSystemTime, &epochFileTime);
-
  epochIntervals.LowPart = epochFileTime.dwLowDateTime;
-
  epochIntervals.HighPart = epochFileTime.dwHighDateTime;
-

-
  fileIntervals.LowPart = pFileTime->dwLowDateTime;
-
  fileIntervals.HighPart = pFileTime->dwHighDateTime;
+
/* #include "sqlite3ext.h" */
+
SQLITE_EXTENSION_INIT1
+
#include <assert.h>
+
#include <string.h>
+
#include <ctype.h>
+
#include <stdlib.h>

-
  return (fileIntervals.QuadPart - epochIntervals.QuadPart) / 10000000;
-
}
+
/* Mark a function parameter as unused, to suppress nuisance compiler
+
** warnings. */
+
#ifndef UNUSED_PARAMETER
+
# define UNUSED_PARAMETER(X)  (void)(X)
+
#endif


-
#if defined(FILEIO_WIN32_DLL) && (defined(_WIN32) || defined(WIN32))
-
#  /* To allow a standalone DLL, use this next replacement function: */
-
#  undef sqlite3_win32_utf8_to_unicode
-
#  define sqlite3_win32_utf8_to_unicode utf8_to_utf16
-
#
-
LPWSTR utf8_to_utf16(const char *z){
-
  int nAllot = MultiByteToWideChar(CP_UTF8, 0, z, -1, NULL, 0);
-
  LPWSTR rv = sqlite3_malloc(nAllot * sizeof(WCHAR));
-
  if( rv!=0 && 0 < MultiByteToWideChar(CP_UTF8, 0, z, -1, rv, nAllot) )
-
    return rv;
-
  sqlite3_free(rv);
-
  return 0;
-
}
-
#endif
+
/* A decimal object */
+
typedef struct Decimal Decimal;
+
struct Decimal {
+
  char sign;        /* 0 for positive, 1 for negative */
+
  char oom;         /* True if an OOM is encountered */
+
  char isNull;      /* True if holds a NULL rather than a number */
+
  char isInit;      /* True upon initialization */
+
  int nDigit;       /* Total number of digits */
+
  int nFrac;        /* Number of digits to the right of the decimal point */
+
  signed char *a;   /* Array of digits.  Most significant first. */
+
};

/*
-
** This function attempts to normalize the time values found in the stat()
-
** buffer to UTC.  This is necessary on Win32, where the runtime library
-
** appears to return these values as local times.
+
** Release memory held by a Decimal, but do not free the object itself.
*/
-
static void statTimesToUtc(
-
  const char *zPath,
-
  struct stat *pStatBuf
-
){
-
  HANDLE hFindFile;
-
  WIN32_FIND_DATAW fd;
-
  LPWSTR zUnicodeName;
-
  extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);
-
  zUnicodeName = sqlite3_win32_utf8_to_unicode(zPath);
-
  if( zUnicodeName ){
-
    memset(&fd, 0, sizeof(WIN32_FIND_DATAW));
-
    hFindFile = FindFirstFileW(zUnicodeName, &fd);
-
    if( hFindFile!=NULL ){
-
      pStatBuf->st_ctime = (time_t)fileTimeToUnixTime(&fd.ftCreationTime);
-
      pStatBuf->st_atime = (time_t)fileTimeToUnixTime(&fd.ftLastAccessTime);
-
      pStatBuf->st_mtime = (time_t)fileTimeToUnixTime(&fd.ftLastWriteTime);
-
      FindClose(hFindFile);
-
    }
-
    sqlite3_free(zUnicodeName);
-
  }
+
static void decimal_clear(Decimal *p){
+
  sqlite3_free(p->a);
}
-
#endif

/*
-
** This function is used in place of stat().  On Windows, special handling
-
** is required in order for the included time to be returned as UTC.  On all
-
** other systems, this function simply calls stat().
+
** Destroy a Decimal object
*/
-
static int fileStat(
-
  const char *zPath,
-
  struct stat *pStatBuf
-
){
-
#if defined(_WIN32)
-
  int rc = stat(zPath, pStatBuf);
-
  if( rc==0 ) statTimesToUtc(zPath, pStatBuf);
-
  return rc;
-
#else
-
  return stat(zPath, pStatBuf);
-
#endif
+
static void decimal_free(Decimal *p){
+
  if( p ){
+
    decimal_clear(p);
+
    sqlite3_free(p);
+
  }
}

/*
-
** This function is used in place of lstat().  On Windows, special handling
-
** is required in order for the included time to be returned as UTC.  On all
-
** other systems, this function simply calls lstat().
+
** Allocate a new Decimal object.  Initialize it to the number given
+
** by the input string.
*/
-
static int fileLinkStat(
-
  const char *zPath,
-
  struct stat *pStatBuf
+
static Decimal *decimal_new(
+
  sqlite3_context *pCtx,
+
  sqlite3_value *pIn,
+
  int nAlt,
+
  const unsigned char *zAlt
){
-
#if defined(_WIN32)
-
  int rc = lstat(zPath, pStatBuf);
-
  if( rc==0 ) statTimesToUtc(zPath, pStatBuf);
-
  return rc;
-
#else
-
  return lstat(zPath, pStatBuf);
-
#endif
-
}
-

-
/*
-
** Argument zFile is the name of a file that will be created and/or written
-
** by SQL function writefile(). This function ensures that the directory
-
** zFile will be written to exists, creating it if required. The permissions
-
** for any path components created by this function are set in accordance
-
** with the current umask.
-
**
-
** If an OOM condition is encountered, SQLITE_NOMEM is returned. Otherwise,
-
** SQLITE_OK is returned if the directory is successfully created, or
-
** SQLITE_ERROR otherwise.
-
*/
-
static int makeDirectory(
-
  const char *zFile
-
){
-
  char *zCopy = sqlite3_mprintf("%s", zFile);
-
  int rc = SQLITE_OK;
-

-
  if( zCopy==0 ){
-
    rc = SQLITE_NOMEM;
+
  Decimal *p;
+
  int n, i;
+
  const unsigned char *zIn;
+
  int iExp = 0;
+
  p = sqlite3_malloc( sizeof(*p) );
+
  if( p==0 ) goto new_no_mem;
+
  p->sign = 0;
+
  p->oom = 0;
+
  p->isInit = 1;
+
  p->isNull = 0;
+
  p->nDigit = 0;
+
  p->nFrac = 0;
+
  if( zAlt ){
+
    n = nAlt,
+
    zIn = zAlt;
  }else{
-
    int nCopy = (int)strlen(zCopy);
-
    int i = 1;
-

-
    while( rc==SQLITE_OK ){
-
      struct stat sStat;
-
      int rc2;
-

-
      for(; zCopy[i]!='/' && i<nCopy; i++);
-
      if( i==nCopy ) break;
-
      zCopy[i] = '\0';
-

-
      rc2 = fileStat(zCopy, &sStat);
-
      if( rc2!=0 ){
-
        if( mkdir(zCopy, 0777) ) rc = SQLITE_ERROR;
-
      }else{
-
        if( !S_ISDIR(sStat.st_mode) ) rc = SQLITE_ERROR;
-
      }
-
      zCopy[i] = '/';
-
      i++;
+
    if( sqlite3_value_type(pIn)==SQLITE_NULL ){
+
      p->a = 0;
+
      p->isNull = 1;
+
      return p;
    }
-

-
    sqlite3_free(zCopy);
+
    n = sqlite3_value_bytes(pIn);
+
    zIn = sqlite3_value_text(pIn);
  }
-

-
  return rc;
-
}
-

-
/*
-
** This function does the work for the writefile() UDF. Refer to 
-
** header comments at the top of this file for details.
-
*/
-
static int writeFile(
-
  sqlite3_context *pCtx,          /* Context to return bytes written in */
-
  const char *zFile,              /* File to write */
-
  sqlite3_value *pData,           /* Data to write */
-
  mode_t mode,                    /* MODE parameter passed to writefile() */
-
  sqlite3_int64 mtime             /* MTIME parameter (or -1 to not set time) */
-
){
-
  if( zFile==0 ) return 1;
-
#if !defined(_WIN32) && !defined(WIN32)
-
  if( S_ISLNK(mode) ){
-
    const char *zTo = (const char*)sqlite3_value_text(pData);
-
    if( zTo==0 || symlink(zTo, zFile)<0 ) return 1;
-
  }else
-
#endif
-
  {
-
    if( S_ISDIR(mode) ){
-
      if( mkdir(zFile, mode) ){
-
        /* The mkdir() call to create the directory failed. This might not
-
        ** be an error though - if there is already a directory at the same
-
        ** path and either the permissions already match or can be changed
-
        ** to do so using chmod(), it is not an error.  */
-
        struct stat sStat;
-
        if( errno!=EEXIST
-
         || 0!=fileStat(zFile, &sStat)
-
         || !S_ISDIR(sStat.st_mode)
-
         || ((sStat.st_mode&0777)!=(mode&0777) && 0!=chmod(zFile, mode&0777))
-
        ){
-
          return 1;
-
        }
+
  p->a = sqlite3_malloc64( n+1 );
+
  if( p->a==0 ) goto new_no_mem;
+
  for(i=0; isspace(zIn[i]); i++){}
+
  if( zIn[i]=='-' ){
+
    p->sign = 1;
+
    i++;
+
  }else if( zIn[i]=='+' ){
+
    i++;
+
  }
+
  while( i<n && zIn[i]=='0' ) i++;
+
  while( i<n ){
+
    char c = zIn[i];
+
    if( c>='0' && c<='9' ){
+
      p->a[p->nDigit++] = c - '0';
+
    }else if( c=='.' ){
+
      p->nFrac = p->nDigit + 1;
+
    }else if( c=='e' || c=='E' ){
+
      int j = i+1;
+
      int neg = 0;
+
      if( j>=n ) break;
+
      if( zIn[j]=='-' ){
+
        neg = 1;
+
        j++;
+
      }else if( zIn[j]=='+' ){
+
        j++;
      }
-
    }else{
-
      sqlite3_int64 nWrite = 0;
-
      const char *z;
-
      int rc = 0;
-
      FILE *out = fopen(zFile, "wb");
-
      if( out==0 ) return 1;
-
      z = (const char*)sqlite3_value_blob(pData);
-
      if( z ){
-
        sqlite3_int64 n = fwrite(z, 1, sqlite3_value_bytes(pData), out);
-
        nWrite = sqlite3_value_bytes(pData);
-
        if( nWrite!=n ){
-
          rc = 1;
+
      while( j<n && iExp<1000000 ){
+
        if( zIn[j]>='0' && zIn[j]<='9' ){
+
          iExp = iExp*10 + zIn[j] - '0';
        }
+
        j++;
      }
-
      fclose(out);
-
      if( rc==0 && mode && chmod(zFile, mode & 0777) ){
-
        rc = 1;
-
      }
-
      if( rc ) return 2;
-
      sqlite3_result_int64(pCtx, nWrite);
+
      if( neg ) iExp = -iExp;
+
      break;
    }
+
    i++;
  }
-

-
  if( mtime>=0 ){
-
#if defined(_WIN32)
-
#if !SQLITE_OS_WINRT
-
    /* Windows */
-
    FILETIME lastAccess;
-
    FILETIME lastWrite;
-
    SYSTEMTIME currentTime;
-
    LONGLONG intervals;
-
    HANDLE hFile;
-
    LPWSTR zUnicodeName;
-
    extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);
-

-
    GetSystemTime(&currentTime);
-
    SystemTimeToFileTime(&currentTime, &lastAccess);
-
    intervals = Int32x32To64(mtime, 10000000) + 116444736000000000;
-
    lastWrite.dwLowDateTime = (DWORD)intervals;
-
    lastWrite.dwHighDateTime = intervals >> 32;
-
    zUnicodeName = sqlite3_win32_utf8_to_unicode(zFile);
-
    if( zUnicodeName==0 ){
-
      return 1;
+
  if( p->nFrac ){
+
    p->nFrac = p->nDigit - (p->nFrac - 1);
+
  }
+
  if( iExp>0 ){
+
    if( p->nFrac>0 ){
+
      if( iExp<=p->nFrac ){
+
        p->nFrac -= iExp;
+
        iExp = 0;
+
      }else{
+
        iExp -= p->nFrac;
+
        p->nFrac = 0;
+
      }
    }
-
    hFile = CreateFileW(
-
      zUnicodeName, FILE_WRITE_ATTRIBUTES, 0, NULL, OPEN_EXISTING,
-
      FILE_FLAG_BACKUP_SEMANTICS, NULL
-
    );
-
    sqlite3_free(zUnicodeName);
-
    if( hFile!=INVALID_HANDLE_VALUE ){
-
      BOOL bResult = SetFileTime(hFile, NULL, &lastAccess, &lastWrite);
-
      CloseHandle(hFile);
-
      return !bResult;
-
    }else{
-
      return 1;
+
    if( iExp>0 ){   
+
      p->a = sqlite3_realloc64(p->a, p->nDigit + iExp + 1 );
+
      if( p->a==0 ) goto new_no_mem;
+
      memset(p->a+p->nDigit, 0, iExp);
+
      p->nDigit += iExp;
    }
-
#endif
-
#elif defined(AT_FDCWD) && 0 /* utimensat() is not universally available */
-
    /* Recent unix */
-
    struct timespec times[2];
-
    times[0].tv_nsec = times[1].tv_nsec = 0;
-
    times[0].tv_sec = time(0);
-
    times[1].tv_sec = mtime;
-
    if( utimensat(AT_FDCWD, zFile, times, AT_SYMLINK_NOFOLLOW) ){
-
      return 1;
+
  }else if( iExp<0 ){
+
    int nExtra;
+
    iExp = -iExp;
+
    nExtra = p->nDigit - p->nFrac - 1;
+
    if( nExtra ){
+
      if( nExtra>=iExp ){
+
        p->nFrac += iExp;
+
        iExp  = 0;
+
      }else{
+
        iExp -= nExtra;
+
        p->nFrac = p->nDigit - 1;
+
      }
    }
-
#else
-
    /* Legacy unix */
-
    struct timeval times[2];
-
    times[0].tv_usec = times[1].tv_usec = 0;
-
    times[0].tv_sec = time(0);
-
    times[1].tv_sec = mtime;
-
    if( utimes(zFile, times) ){
-
      return 1;
+
    if( iExp>0 ){
+
      p->a = sqlite3_realloc64(p->a, p->nDigit + iExp + 1 );
+
      if( p->a==0 ) goto new_no_mem;
+
      memmove(p->a+iExp, p->a, p->nDigit);
+
      memset(p->a, 0, iExp);
+
      p->nDigit += iExp;
+
      p->nFrac += iExp;
    }
-
#endif
  }
+
  return p;

+
new_no_mem:
+
  if( pCtx ) sqlite3_result_error_nomem(pCtx);
+
  sqlite3_free(p);
  return 0;
}

/*
-
** Implementation of the "writefile(W,X[,Y[,Z]]])" SQL function.  
-
** Refer to header comments at the top of this file for details.
+
** Make the given Decimal the result.
*/
-
static void writefileFunc(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  const char *zFile;
-
  mode_t mode = 0;
-
  int res;
-
  sqlite3_int64 mtime = -1;
-

-
  if( argc<2 || argc>4 ){
-
    sqlite3_result_error(context, 
-
        "wrong number of arguments to function writefile()", -1
-
    );
-
    return;
+
static void decimal_result(sqlite3_context *pCtx, Decimal *p){
+
  char *z;
+
  int i, j;
+
  int n;
+
  if( p==0 || p->oom ){
+
    sqlite3_result_error_nomem(pCtx);
+
    return;
  }
-

-
  zFile = (const char*)sqlite3_value_text(argv[0]);
-
  if( zFile==0 ) return;
-
  if( argc>=3 ){
-
    mode = (mode_t)sqlite3_value_int(argv[2]);
+
  if( p->isNull ){
+
    sqlite3_result_null(pCtx);
+
    return;
  }
-
  if( argc==4 ){
-
    mtime = sqlite3_value_int64(argv[3]);
+
  z = sqlite3_malloc( p->nDigit+4 );
+
  if( z==0 ){
+
    sqlite3_result_error_nomem(pCtx);
+
    return;
  }
-

-
  res = writeFile(context, zFile, argv[1], mode, mtime);
-
  if( res==1 && errno==ENOENT ){
-
    if( makeDirectory(zFile)==SQLITE_OK ){
-
      res = writeFile(context, zFile, argv[1], mode, mtime);
-
    }
+
  i = 0;
+
  if( p->nDigit==0 || (p->nDigit==1 && p->a[0]==0) ){
+
    p->sign = 0;
  }
-

-
  if( argc>2 && res!=0 ){
-
    if( S_ISLNK(mode) ){
-
      ctxErrorMsg(context, "failed to create symlink: %s", zFile);
-
    }else if( S_ISDIR(mode) ){
-
      ctxErrorMsg(context, "failed to create directory: %s", zFile);
-
    }else{
-
      ctxErrorMsg(context, "failed to write file: %s", zFile);
-
    }
+
  if( p->sign ){
+
    z[0] = '-';
+
    i = 1;
+
  }
+
  n = p->nDigit - p->nFrac;
+
  if( n<=0 ){
+
    z[i++] = '0';
+
  }
+
  j = 0;
+
  while( n>1 && p->a[j]==0 ){
+
    j++;
+
    n--;
+
  }
+
  while( n>0  ){
+
    z[i++] = p->a[j] + '0';
+
    j++;
+
    n--;
  }
+
  if( p->nFrac ){
+
    z[i++] = '.';
+
    do{
+
      z[i++] = p->a[j] + '0';
+
      j++;
+
    }while( j<p->nDigit );
+
  }
+
  z[i] = 0;
+
  sqlite3_result_text(pCtx, z, i, sqlite3_free);
}

/*
-
** SQL function:   lsmode(MODE)
+
** SQL Function:   decimal(X)
**
-
** Given a numberic st_mode from stat(), convert it into a human-readable
-
** text string in the style of "ls -l".
+
** Convert input X into decimal and then back into text
*/
-
static void lsModeFunc(
+
static void decimalFunc(
  sqlite3_context *context,
  int argc,
  sqlite3_value **argv
){
-
  int i;
-
  int iMode = sqlite3_value_int(argv[0]);
-
  char z[16];
-
  (void)argc;
-
  if( S_ISLNK(iMode) ){
-
    z[0] = 'l';
-
  }else if( S_ISREG(iMode) ){
-
    z[0] = '-';
-
  }else if( S_ISDIR(iMode) ){
-
    z[0] = 'd';
-
  }else{
-
    z[0] = '?';
-
  }
-
  for(i=0; i<3; i++){
-
    int m = (iMode >> ((2-i)*3));
-
    char *a = &z[1 + i*3];
-
    a[0] = (m & 0x4) ? 'r' : '-';
-
    a[1] = (m & 0x2) ? 'w' : '-';
-
    a[2] = (m & 0x1) ? 'x' : '-';
-
  }
-
  z[10] = '\0';
-
  sqlite3_result_text(context, z, -1, SQLITE_TRANSIENT);
+
  Decimal *p = decimal_new(context, argv[0], 0, 0);
+
  UNUSED_PARAMETER(argc);
+
  decimal_result(context, p);
+
  decimal_free(p);
}

-
#ifndef SQLITE_OMIT_VIRTUALTABLE
-

-
/* 
-
** Cursor type for recursively iterating through a directory structure.
-
*/
-
typedef struct fsdir_cursor fsdir_cursor;
-
typedef struct FsdirLevel FsdirLevel;
-

-
struct FsdirLevel {
-
  DIR *pDir;                 /* From opendir() */
-
  char *zDir;                /* Name of directory (nul-terminated) */
-
};
-

-
struct fsdir_cursor {
-
  sqlite3_vtab_cursor base;  /* Base class - must be first */
-

-
  int nLvl;                  /* Number of entries in aLvl[] array */
-
  int iLvl;                  /* Index of current entry */
-
  FsdirLevel *aLvl;          /* Hierarchy of directories being traversed */
-

-
  const char *zBase;
-
  int nBase;
-

-
  struct stat sStat;         /* Current lstat() results */
-
  char *zPath;               /* Path to current entry */
-
  sqlite3_int64 iRowid;      /* Current rowid */
-
};
-

-
typedef struct fsdir_tab fsdir_tab;
-
struct fsdir_tab {
-
  sqlite3_vtab base;         /* Base class - must be first */
-
};
-

/*
-
** Construct a new fsdir virtual table object.
+
** Compare to Decimal objects.  Return negative, 0, or positive if the
+
** first object is less than, equal to, or greater than the second.
+
**
+
** Preconditions for this routine:
+
**
+
**    pA!=0
+
**    pA->isNull==0
+
**    pB!=0
+
**    pB->isNull==0
*/
-
static int fsdirConnect(
-
  sqlite3 *db,
-
  void *pAux,
-
  int argc, const char *const*argv,
-
  sqlite3_vtab **ppVtab,
-
  char **pzErr
-
){
-
  fsdir_tab *pNew = 0;
-
  int rc;
-
  (void)pAux;
-
  (void)argc;
-
  (void)argv;
-
  (void)pzErr;
-
  rc = sqlite3_declare_vtab(db, "CREATE TABLE x" FSDIR_SCHEMA);
-
  if( rc==SQLITE_OK ){
-
    pNew = (fsdir_tab*)sqlite3_malloc( sizeof(*pNew) );
-
    if( pNew==0 ) return SQLITE_NOMEM;
-
    memset(pNew, 0, sizeof(*pNew));
-
    sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY);
+
static int decimal_cmp(const Decimal *pA, const Decimal *pB){
+
  int nASig, nBSig, rc, n;
+
  if( pA->sign!=pB->sign ){
+
    return pA->sign ? -1 : +1;
+
  }
+
  if( pA->sign ){
+
    const Decimal *pTemp = pA;
+
    pA = pB;
+
    pB = pTemp;
+
  }
+
  nASig = pA->nDigit - pA->nFrac;
+
  nBSig = pB->nDigit - pB->nFrac;
+
  if( nASig!=nBSig ){
+
    return nASig - nBSig;
+
  }
+
  n = pA->nDigit;
+
  if( n>pB->nDigit ) n = pB->nDigit;
+
  rc = memcmp(pA->a, pB->a, n);
+
  if( rc==0 ){
+
    rc = pA->nDigit - pB->nDigit;
  }
-
  *ppVtab = (sqlite3_vtab*)pNew;
  return rc;
}

/*
-
** This method is the destructor for fsdir vtab objects.
+
** SQL Function:   decimal_cmp(X, Y)
+
**
+
** Return negative, zero, or positive if X is less then, equal to, or
+
** greater than Y.
*/
-
static int fsdirDisconnect(sqlite3_vtab *pVtab){
-
  sqlite3_free(pVtab);
-
  return SQLITE_OK;
+
static void decimalCmpFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  Decimal *pA = 0, *pB = 0;
+
  int rc;
+

+
  UNUSED_PARAMETER(argc);
+
  pA = decimal_new(context, argv[0], 0, 0);
+
  if( pA==0 || pA->isNull ) goto cmp_done;
+
  pB = decimal_new(context, argv[1], 0, 0);
+
  if( pB==0 || pB->isNull ) goto cmp_done;
+
  rc = decimal_cmp(pA, pB);
+
  if( rc<0 ) rc = -1;
+
  else if( rc>0 ) rc = +1;
+
  sqlite3_result_int(context, rc);
+
cmp_done:
+
  decimal_free(pA);
+
  decimal_free(pB);
}

/*
-
** Constructor for a new fsdir_cursor object.
+
** Expand the Decimal so that it has a least nDigit digits and nFrac
+
** digits to the right of the decimal point.
*/
-
static int fsdirOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
-
  fsdir_cursor *pCur;
-
  (void)p;
-
  pCur = sqlite3_malloc( sizeof(*pCur) );
-
  if( pCur==0 ) return SQLITE_NOMEM;
-
  memset(pCur, 0, sizeof(*pCur));
-
  pCur->iLvl = -1;
-
  *ppCursor = &pCur->base;
-
  return SQLITE_OK;
+
static void decimal_expand(Decimal *p, int nDigit, int nFrac){
+
  int nAddSig;
+
  int nAddFrac;
+
  if( p==0 ) return;
+
  nAddFrac = nFrac - p->nFrac;
+
  nAddSig = (nDigit - p->nDigit) - nAddFrac;
+
  if( nAddFrac==0 && nAddSig==0 ) return;
+
  p->a = sqlite3_realloc64(p->a, nDigit+1);
+
  if( p->a==0 ){
+
    p->oom = 1;
+
    return;
+
  }
+
  if( nAddSig ){
+
    memmove(p->a+nAddSig, p->a, p->nDigit);
+
    memset(p->a, 0, nAddSig);
+
    p->nDigit += nAddSig;
+
  }
+
  if( nAddFrac ){
+
    memset(p->a+p->nDigit, 0, nAddFrac);
+
    p->nDigit += nAddFrac;
+
    p->nFrac += nAddFrac;
+
  }
}

/*
-
** Reset a cursor back to the state it was in when first returned
-
** by fsdirOpen().
+
** Add the value pB into pA.
+
**
+
** Both pA and pB might become denormalized by this routine.
*/
-
static void fsdirResetCursor(fsdir_cursor *pCur){
-
  int i;
-
  for(i=0; i<=pCur->iLvl; i++){
-
    FsdirLevel *pLvl = &pCur->aLvl[i];
-
    if( pLvl->pDir ) closedir(pLvl->pDir);
-
    sqlite3_free(pLvl->zDir);
-
  }
-
  sqlite3_free(pCur->zPath);
-
  sqlite3_free(pCur->aLvl);
-
  pCur->aLvl = 0;
-
  pCur->zPath = 0;
-
  pCur->zBase = 0;
-
  pCur->nBase = 0;
-
  pCur->nLvl = 0;
-
  pCur->iLvl = -1;
-
  pCur->iRowid = 1;
-
}
-

-
/*
-
** Destructor for an fsdir_cursor.
-
*/
-
static int fsdirClose(sqlite3_vtab_cursor *cur){
-
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
-

-
  fsdirResetCursor(pCur);
-
  sqlite3_free(pCur);
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Set the error message for the virtual table associated with cursor
-
** pCur to the results of vprintf(zFmt, ...).
-
*/
-
static void fsdirSetErrmsg(fsdir_cursor *pCur, const char *zFmt, ...){
-
  va_list ap;
-
  va_start(ap, zFmt);
-
  pCur->base.pVtab->zErrMsg = sqlite3_vmprintf(zFmt, ap);
-
  va_end(ap);
-
}
-

-

-
/*
-
** Advance an fsdir_cursor to its next row of output.
-
*/
-
static int fsdirNext(sqlite3_vtab_cursor *cur){
-
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
-
  mode_t m = pCur->sStat.st_mode;
-

-
  pCur->iRowid++;
-
  if( S_ISDIR(m) ){
-
    /* Descend into this directory */
-
    int iNew = pCur->iLvl + 1;
-
    FsdirLevel *pLvl;
-
    if( iNew>=pCur->nLvl ){
-
      int nNew = iNew+1;
-
      sqlite3_int64 nByte = nNew*sizeof(FsdirLevel);
-
      FsdirLevel *aNew = (FsdirLevel*)sqlite3_realloc64(pCur->aLvl, nByte);
-
      if( aNew==0 ) return SQLITE_NOMEM;
-
      memset(&aNew[pCur->nLvl], 0, sizeof(FsdirLevel)*(nNew-pCur->nLvl));
-
      pCur->aLvl = aNew;
-
      pCur->nLvl = nNew;
-
    }
-
    pCur->iLvl = iNew;
-
    pLvl = &pCur->aLvl[iNew];
-
    
-
    pLvl->zDir = pCur->zPath;
-
    pCur->zPath = 0;
-
    pLvl->pDir = opendir(pLvl->zDir);
-
    if( pLvl->pDir==0 ){
-
      fsdirSetErrmsg(pCur, "cannot read directory: %s", pCur->zPath);
-
      return SQLITE_ERROR;
-
    }
+
static void decimal_add(Decimal *pA, Decimal *pB){
+
  int nSig, nFrac, nDigit;
+
  int i, rc;
+
  if( pA==0 ){
+
    return;
  }
-

-
  while( pCur->iLvl>=0 ){
-
    FsdirLevel *pLvl = &pCur->aLvl[pCur->iLvl];
-
    struct dirent *pEntry = readdir(pLvl->pDir);
-
    if( pEntry ){
-
      if( pEntry->d_name[0]=='.' ){
-
       if( pEntry->d_name[1]=='.' && pEntry->d_name[2]=='\0' ) continue;
-
       if( pEntry->d_name[1]=='\0' ) continue;
-
      }
-
      sqlite3_free(pCur->zPath);
-
      pCur->zPath = sqlite3_mprintf("%s/%s", pLvl->zDir, pEntry->d_name);
-
      if( pCur->zPath==0 ) return SQLITE_NOMEM;
-
      if( fileLinkStat(pCur->zPath, &pCur->sStat) ){
-
        fsdirSetErrmsg(pCur, "cannot stat file: %s", pCur->zPath);
-
        return SQLITE_ERROR;
-
      }
-
      return SQLITE_OK;
-
    }
-
    closedir(pLvl->pDir);
-
    sqlite3_free(pLvl->zDir);
-
    pLvl->pDir = 0;
-
    pLvl->zDir = 0;
-
    pCur->iLvl--;
+
  if( pA->oom || pB==0 || pB->oom ){
+
    pA->oom = 1;
+
    return;
  }
-

-
  /* EOF */
-
  sqlite3_free(pCur->zPath);
-
  pCur->zPath = 0;
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Return values of columns for the row at which the series_cursor
-
** is currently pointing.
-
*/
-
static int fsdirColumn(
-
  sqlite3_vtab_cursor *cur,   /* The cursor */
-
  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
-
  int i                       /* Which column to return */
-
){
-
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
-
  switch( i ){
-
    case FSDIR_COLUMN_NAME: {
-
      sqlite3_result_text(ctx, &pCur->zPath[pCur->nBase], -1, SQLITE_TRANSIENT);
-
      break;
-
    }
-

-
    case FSDIR_COLUMN_MODE:
-
      sqlite3_result_int64(ctx, pCur->sStat.st_mode);
-
      break;
-

-
    case FSDIR_COLUMN_MTIME:
-
      sqlite3_result_int64(ctx, pCur->sStat.st_mtime);
-
      break;
-

-
    case FSDIR_COLUMN_DATA: {
-
      mode_t m = pCur->sStat.st_mode;
-
      if( S_ISDIR(m) ){
-
        sqlite3_result_null(ctx);
-
#if !defined(_WIN32) && !defined(WIN32)
-
      }else if( S_ISLNK(m) ){
-
        char aStatic[64];
-
        char *aBuf = aStatic;
-
        sqlite3_int64 nBuf = 64;
-
        int n;
-

-
        while( 1 ){
-
          n = readlink(pCur->zPath, aBuf, nBuf);
-
          if( n<nBuf ) break;
-
          if( aBuf!=aStatic ) sqlite3_free(aBuf);
-
          nBuf = nBuf*2;
-
          aBuf = sqlite3_malloc64(nBuf);
-
          if( aBuf==0 ){
-
            sqlite3_result_error_nomem(ctx);
-
            return SQLITE_NOMEM;
-
          }
+
  if( pA->isNull || pB->isNull ){
+
    pA->isNull = 1;
+
    return;
+
  }
+
  nSig = pA->nDigit - pA->nFrac;
+
  if( nSig && pA->a[0]==0 ) nSig--;
+
  if( nSig<pB->nDigit-pB->nFrac ){
+
    nSig = pB->nDigit - pB->nFrac;
+
  }
+
  nFrac = pA->nFrac;
+
  if( nFrac<pB->nFrac ) nFrac = pB->nFrac;
+
  nDigit = nSig + nFrac + 1;
+
  decimal_expand(pA, nDigit, nFrac);
+
  decimal_expand(pB, nDigit, nFrac);
+
  if( pA->oom || pB->oom ){
+
    pA->oom = 1;
+
  }else{
+
    if( pA->sign==pB->sign ){
+
      int carry = 0;
+
      for(i=nDigit-1; i>=0; i--){
+
        int x = pA->a[i] + pB->a[i] + carry;
+
        if( x>=10 ){
+
          carry = 1;
+
          pA->a[i] = x - 10;
+
        }else{
+
          carry = 0;
+
          pA->a[i] = x;
        }
-

-
        sqlite3_result_text(ctx, aBuf, n, SQLITE_TRANSIENT);
-
        if( aBuf!=aStatic ) sqlite3_free(aBuf);
-
#endif
+
      }
+
    }else{
+
      signed char *aA, *aB;
+
      int borrow = 0;
+
      rc = memcmp(pA->a, pB->a, nDigit);
+
      if( rc<0 ){
+
        aA = pB->a;
+
        aB = pA->a;
+
        pA->sign = !pA->sign;
      }else{
-
        readFileContents(ctx, pCur->zPath);
+
        aA = pA->a;
+
        aB = pB->a;
+
      }
+
      for(i=nDigit-1; i>=0; i--){
+
        int x = aA[i] - aB[i] - borrow;
+
        if( x<0 ){
+
          pA->a[i] = x+10;
+
          borrow = 1;
+
        }else{
+
          pA->a[i] = x;
+
          borrow = 0;
+
        }
      }
-
    }
-
    case FSDIR_COLUMN_PATH:
-
    default: {
-
      /* The FSDIR_COLUMN_PATH and FSDIR_COLUMN_DIR are input parameters.
-
      ** always return their values as NULL */
-
      break;
    }
  }
-
  return SQLITE_OK;
}

/*
-
** Return the rowid for the current row. In this implementation, the
-
** first row returned is assigned rowid value 1, and each subsequent
-
** row a value 1 more than that of the previous.
+
** Compare text in decimal order.
*/
-
static int fsdirRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
-
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
-
  *pRowid = pCur->iRowid;
-
  return SQLITE_OK;
+
static int decimalCollFunc(
+
  void *notUsed,
+
  int nKey1, const void *pKey1,
+
  int nKey2, const void *pKey2
+
){
+
  const unsigned char *zA = (const unsigned char*)pKey1;
+
  const unsigned char *zB = (const unsigned char*)pKey2;
+
  Decimal *pA = decimal_new(0, 0, nKey1, zA);
+
  Decimal *pB = decimal_new(0, 0, nKey2, zB);
+
  int rc;
+
  UNUSED_PARAMETER(notUsed);
+
  if( pA==0 || pB==0 ){
+
    rc = 0;
+
  }else{
+
    rc = decimal_cmp(pA, pB);
+
  }
+
  decimal_free(pA);
+
  decimal_free(pB);
+
  return rc;
}

-
/*
-
** Return TRUE if the cursor has been moved off of the last
-
** row of output.
-
*/
-
static int fsdirEof(sqlite3_vtab_cursor *cur){
-
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
-
  return (pCur->zPath==0);
-
}

/*
-
** xFilter callback.
+
** SQL Function:   decimal_add(X, Y)
+
**                 decimal_sub(X, Y)
**
-
** idxNum==1   PATH parameter only
-
** idxNum==2   Both PATH and DIR supplied
+
** Return the sum or difference of X and Y.
*/
-
static int fsdirFilter(
-
  sqlite3_vtab_cursor *cur, 
-
  int idxNum, const char *idxStr,
-
  int argc, sqlite3_value **argv
+
static void decimalAddFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
){
-
  const char *zDir = 0;
-
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
-
  (void)idxStr;
-
  fsdirResetCursor(pCur);
-

-
  if( idxNum==0 ){
-
    fsdirSetErrmsg(pCur, "table function fsdir requires an argument");
-
    return SQLITE_ERROR;
-
  }
-

-
  assert( argc==idxNum && (argc==1 || argc==2) );
-
  zDir = (const char*)sqlite3_value_text(argv[0]);
-
  if( zDir==0 ){
-
    fsdirSetErrmsg(pCur, "table function fsdir requires a non-NULL argument");
-
    return SQLITE_ERROR;
-
  }
-
  if( argc==2 ){
-
    pCur->zBase = (const char*)sqlite3_value_text(argv[1]);
-
  }
-
  if( pCur->zBase ){
-
    pCur->nBase = (int)strlen(pCur->zBase)+1;
-
    pCur->zPath = sqlite3_mprintf("%s/%s", pCur->zBase, zDir);
-
  }else{
-
    pCur->zPath = sqlite3_mprintf("%s", zDir);
+
  Decimal *pA = decimal_new(context, argv[0], 0, 0);
+
  Decimal *pB = decimal_new(context, argv[1], 0, 0);
+
  UNUSED_PARAMETER(argc);
+
  decimal_add(pA, pB);
+
  decimal_result(context, pA);
+
  decimal_free(pA);
+
  decimal_free(pB);
+
}
+
static void decimalSubFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  Decimal *pA = decimal_new(context, argv[0], 0, 0);
+
  Decimal *pB = decimal_new(context, argv[1], 0, 0);
+
  UNUSED_PARAMETER(argc);
+
  if( pB ){
+
    pB->sign = !pB->sign;
+
    decimal_add(pA, pB);
+
    decimal_result(context, pA);
  }
+
  decimal_free(pA);
+
  decimal_free(pB);
+
}

-
  if( pCur->zPath==0 ){
-
    return SQLITE_NOMEM;
-
  }
-
  if( fileLinkStat(pCur->zPath, &pCur->sStat) ){
-
    fsdirSetErrmsg(pCur, "cannot stat file: %s", pCur->zPath);
-
    return SQLITE_ERROR;
+
/* Aggregate funcion:   decimal_sum(X)
+
**
+
** Works like sum() except that it uses decimal arithmetic for unlimited
+
** precision.
+
*/
+
static void decimalSumStep(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  Decimal *p;
+
  Decimal *pArg;
+
  UNUSED_PARAMETER(argc);
+
  p = sqlite3_aggregate_context(context, sizeof(*p));
+
  if( p==0 ) return;
+
  if( !p->isInit ){
+
    p->isInit = 1;
+
    p->a = sqlite3_malloc(2);
+
    if( p->a==0 ){
+
      p->oom = 1;
+
    }else{
+
      p->a[0] = 0;
+
    }
+
    p->nDigit = 1;
+
    p->nFrac = 0;
  }
-

-
  return SQLITE_OK;
+
  if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return;
+
  pArg = decimal_new(context, argv[0], 0, 0);
+
  decimal_add(p, pArg);
+
  decimal_free(pArg);
+
}
+
static void decimalSumInverse(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  Decimal *p;
+
  Decimal *pArg;
+
  UNUSED_PARAMETER(argc);
+
  p = sqlite3_aggregate_context(context, sizeof(*p));
+
  if( p==0 ) return;
+
  if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return;
+
  pArg = decimal_new(context, argv[0], 0, 0);
+
  if( pArg ) pArg->sign = !pArg->sign;
+
  decimal_add(p, pArg);
+
  decimal_free(pArg);
+
}
+
static void decimalSumValue(sqlite3_context *context){
+
  Decimal *p = sqlite3_aggregate_context(context, 0);
+
  if( p==0 ) return;
+
  decimal_result(context, p);
+
}
+
static void decimalSumFinalize(sqlite3_context *context){
+
  Decimal *p = sqlite3_aggregate_context(context, 0);
+
  if( p==0 ) return;
+
  decimal_result(context, p);
+
  decimal_clear(p);
}

/*
-
** SQLite will invoke this method one or more times while planning a query
-
** that uses the generate_series virtual table.  This routine needs to create
-
** a query plan for each invocation and compute an estimated cost for that
-
** plan.
-
**
-
** In this implementation idxNum is used to represent the
-
** query plan.  idxStr is unused.
+
** SQL Function:   decimal_mul(X, Y)
**
-
** The query plan is represented by values of idxNum:
+
** Return the product of X and Y.
**
-
**  (1)  The path value is supplied by argv[0]
-
**  (2)  Path is in argv[0] and dir is in argv[1]
+
** All significant digits after the decimal point are retained.
+
** Trailing zeros after the decimal point are omitted as long as
+
** the number of digits after the decimal point is no less than
+
** either the number of digits in either input.
*/
-
static int fsdirBestIndex(
-
  sqlite3_vtab *tab,
-
  sqlite3_index_info *pIdxInfo
+
static void decimalMulFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
){
-
  int i;                 /* Loop over constraints */
-
  int idxPath = -1;      /* Index in pIdxInfo->aConstraint of PATH= */
-
  int idxDir = -1;       /* Index in pIdxInfo->aConstraint of DIR= */
-
  int seenPath = 0;      /* True if an unusable PATH= constraint is seen */
-
  int seenDir = 0;       /* True if an unusable DIR= constraint is seen */
-
  const struct sqlite3_index_constraint *pConstraint;
-

-
  (void)tab;
-
  pConstraint = pIdxInfo->aConstraint;
-
  for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){
-
    if( pConstraint->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue;
-
    switch( pConstraint->iColumn ){
-
      case FSDIR_COLUMN_PATH: {
-
        if( pConstraint->usable ){
-
          idxPath = i;
-
          seenPath = 0;
-
        }else if( idxPath<0 ){
-
          seenPath = 1;
-
        }
-
        break;
-
      }
-
      case FSDIR_COLUMN_DIR: {
-
        if( pConstraint->usable ){
-
          idxDir = i;
-
          seenDir = 0;
-
        }else if( idxDir<0 ){
-
          seenDir = 1;
-
        }
-
        break;
-
      }
-
    } 
+
  Decimal *pA = decimal_new(context, argv[0], 0, 0);
+
  Decimal *pB = decimal_new(context, argv[1], 0, 0);
+
  signed char *acc = 0;
+
  int i, j, k;
+
  int minFrac;
+
  UNUSED_PARAMETER(argc);
+
  if( pA==0 || pA->oom || pA->isNull
+
   || pB==0 || pB->oom || pB->isNull 
+
  ){
+
    goto mul_end;
  }
-
  if( seenPath || seenDir ){
-
    /* If input parameters are unusable, disallow this plan */
-
    return SQLITE_CONSTRAINT;
+
  acc = sqlite3_malloc64( pA->nDigit + pB->nDigit + 2 );
+
  if( acc==0 ){
+
    sqlite3_result_error_nomem(context);
+
    goto mul_end;
  }
-

-
  if( idxPath<0 ){
-
    pIdxInfo->idxNum = 0;
-
    /* The pIdxInfo->estimatedCost should have been initialized to a huge
-
    ** number.  Leave it unchanged. */
-
    pIdxInfo->estimatedRows = 0x7fffffff;
-
  }else{
-
    pIdxInfo->aConstraintUsage[idxPath].omit = 1;
-
    pIdxInfo->aConstraintUsage[idxPath].argvIndex = 1;
-
    if( idxDir>=0 ){
-
      pIdxInfo->aConstraintUsage[idxDir].omit = 1;
-
      pIdxInfo->aConstraintUsage[idxDir].argvIndex = 2;
-
      pIdxInfo->idxNum = 2;
-
      pIdxInfo->estimatedCost = 10.0;
-
    }else{
-
      pIdxInfo->idxNum = 1;
-
      pIdxInfo->estimatedCost = 100.0;
+
  memset(acc, 0, pA->nDigit + pB->nDigit + 2);
+
  minFrac = pA->nFrac;
+
  if( pB->nFrac<minFrac ) minFrac = pB->nFrac;
+
  for(i=pA->nDigit-1; i>=0; i--){
+
    signed char f = pA->a[i];
+
    int carry = 0, x;
+
    for(j=pB->nDigit-1, k=i+j+3; j>=0; j--, k--){
+
      x = acc[k] + f*pB->a[j] + carry;
+
      acc[k] = x%10;
+
      carry = x/10;
    }
+
    x = acc[k] + carry;
+
    acc[k] = x%10;
+
    acc[k-1] += x/10;
+
  }
+
  sqlite3_free(pA->a);
+
  pA->a = acc;
+
  acc = 0;
+
  pA->nDigit += pB->nDigit + 2;
+
  pA->nFrac += pB->nFrac;
+
  pA->sign ^= pB->sign;
+
  while( pA->nFrac>minFrac && pA->a[pA->nDigit-1]==0 ){
+
    pA->nFrac--;
+
    pA->nDigit--;
  }
+
  decimal_result(context, pA);

-
  return SQLITE_OK;
+
mul_end:
+
  sqlite3_free(acc);
+
  decimal_free(pA);
+
  decimal_free(pB);
}

-
/*
-
** Register the "fsdir" virtual table.
-
*/
-
static int fsdirRegister(sqlite3 *db){
-
  static sqlite3_module fsdirModule = {
-
    0,                         /* iVersion */
-
    0,                         /* xCreate */
-
    fsdirConnect,              /* xConnect */
-
    fsdirBestIndex,            /* xBestIndex */
-
    fsdirDisconnect,           /* xDisconnect */
-
    0,                         /* xDestroy */
-
    fsdirOpen,                 /* xOpen - open a cursor */
-
    fsdirClose,                /* xClose - close a cursor */
-
    fsdirFilter,               /* xFilter - configure scan constraints */
-
    fsdirNext,                 /* xNext - advance a cursor */
-
    fsdirEof,                  /* xEof - check for end of scan */
-
    fsdirColumn,               /* xColumn - read data */
-
    fsdirRowid,                /* xRowid - read data */
-
    0,                         /* xUpdate */
-
    0,                         /* xBegin */
-
    0,                         /* xSync */
-
    0,                         /* xCommit */
-
    0,                         /* xRollback */
-
    0,                         /* xFindMethod */
-
    0,                         /* xRename */
-
    0,                         /* xSavepoint */
-
    0,                         /* xRelease */
-
    0,                         /* xRollbackTo */
-
    0,                         /* xShadowName */
-
  };
-

-
  int rc = sqlite3_create_module(db, "fsdir", &fsdirModule, 0);
-
  return rc;
-
}
-
#else         /* SQLITE_OMIT_VIRTUALTABLE */
-
# define fsdirRegister(x) SQLITE_OK
-
#endif
-

#ifdef _WIN32

#endif
-
int sqlite3_fileio_init(
+
int sqlite3_decimal_init(
  sqlite3 *db, 
  char **pzErrMsg, 
  const sqlite3_api_routines *pApi
){
  int rc = SQLITE_OK;
-
  SQLITE_EXTENSION_INIT2(pApi);
+
  static const struct {
+
    const char *zFuncName;
+
    int nArg;
+
    void (*xFunc)(sqlite3_context*,int,sqlite3_value**);
+
  } aFunc[] = {
+
    { "decimal",       1,   decimalFunc        },
+
    { "decimal_cmp",   2,   decimalCmpFunc     },
+
    { "decimal_add",   2,   decimalAddFunc     },
+
    { "decimal_sub",   2,   decimalSubFunc     },
+
    { "decimal_mul",   2,   decimalMulFunc     },
+
  };
+
  unsigned int i;
  (void)pzErrMsg;  /* Unused parameter */
-
  rc = sqlite3_create_function(db, "readfile", 1, 
-
                               SQLITE_UTF8|SQLITE_DIRECTONLY, 0,
-
                               readfileFunc, 0, 0);
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_function(db, "writefile", -1,
-
                                 SQLITE_UTF8|SQLITE_DIRECTONLY, 0,
-
                                 writefileFunc, 0, 0);
+

+
  SQLITE_EXTENSION_INIT2(pApi);
+

+
  for(i=0; i<sizeof(aFunc)/sizeof(aFunc[0]) && rc==SQLITE_OK; i++){
+
    rc = sqlite3_create_function(db, aFunc[i].zFuncName, aFunc[i].nArg,
+
                   SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC,
+
                   0, aFunc[i].xFunc, 0, 0);
  }
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_function(db, "lsmode", 1, SQLITE_UTF8, 0,
-
                                 lsModeFunc, 0, 0);
+
    rc = sqlite3_create_window_function(db, "decimal_sum", 1,
+
                   SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0,
+
                   decimalSumStep, decimalSumFinalize,
+
                   decimalSumValue, decimalSumInverse, 0);
  }
  if( rc==SQLITE_OK ){
-
    rc = fsdirRegister(db);
+
    rc = sqlite3_create_collation(db, "decimal", SQLITE_UTF8,
+
                                  0, decimalCollFunc);
  }
  return rc;
}

-
#if defined(FILEIO_WIN32_DLL) && (defined(_WIN32) || defined(WIN32))
-
/* To allow a standalone DLL, make test_windirent.c use the same
-
 * redefined SQLite API calls as the above extension code does.
-
 * Just pull in this .c to accomplish this. As a beneficial side
-
 * effect, this extension becomes a single translation unit. */
-
#  include "test_windirent.c"
-
#endif
-

-
/************************* End ../ext/misc/fileio.c ********************/
-
/************************* Begin ../ext/misc/completion.c ******************/
+
/************************* End ../ext/misc/decimal.c ********************/
+
/************************* Begin ../ext/misc/ieee754.c ******************/
/*
-
** 2017-07-10
+
** 2013-04-17
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
@@ -3152,502 +2995,294 @@ int sqlite3_fileio_init(
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
-
*************************************************************************
+
******************************************************************************
**
-
** This file implements an eponymous virtual table that returns suggested
-
** completions for a partial SQL input.
+
** This SQLite extension implements functions for the exact display
+
** and input of IEEE754 Binary64 floating-point numbers.
**
-
** Suggested usage:
+
**   ieee754(X)
+
**   ieee754(Y,Z)
**
-
**     SELECT DISTINCT candidate COLLATE nocase
-
**       FROM completion($prefix,$wholeline)
-
**      ORDER BY 1;
+
** In the first form, the value X should be a floating-point number.
+
** The function will return a string of the form 'ieee754(Y,Z)' where
+
** Y and Z are integers such that X==Y*pow(2,Z).
**
-
** The two query parameters are optional.  $prefix is the text of the
-
** current word being typed and that is to be completed.  $wholeline is
-
** the complete input line, used for context.
+
** In the second form, Y and Z are integers which are the mantissa and
+
** base-2 exponent of a new floating point number.  The function returns
+
** a floating-point value equal to Y*pow(2,Z).
**
-
** The raw completion() table might return the same candidate multiple
-
** times, for example if the same column name is used to two or more
-
** tables.  And the candidates are returned in an arbitrary order.  Hence,
-
** the DISTINCT and ORDER BY are recommended.
+
** Examples:
**
-
** This virtual table operates at the speed of human typing, and so there
-
** is no attempt to make it fast.  Even a slow implementation will be much
-
** faster than any human can type.
+
**     ieee754(2.0)             ->     'ieee754(2,0)'
+
**     ieee754(45.25)           ->     'ieee754(181,-2)'
+
**     ieee754(2, 0)            ->     2.0
+
**     ieee754(181, -2)         ->     45.25
+
**
+
** Two additional functions break apart the one-argument ieee754()
+
** result into separate integer values:
+
**
+
**     ieee754_mantissa(45.25)  ->     181
+
**     ieee754_exponent(45.25)  ->     -2
+
**
+
** These functions convert binary64 numbers into blobs and back again.
+
**
+
**     ieee754_from_blob(x'3ff0000000000000')  ->  1.0
+
**     ieee754_to_blob(1.0)                    ->  x'3ff0000000000000'
+
**
+
** In all single-argument functions, if the argument is an 8-byte blob
+
** then that blob is interpreted as a big-endian binary64 value.
+
**
+
**
+
** EXACT DECIMAL REPRESENTATION OF BINARY64 VALUES
+
** -----------------------------------------------
+
**
+
** This extension in combination with the separate 'decimal' extension
+
** can be used to compute the exact decimal representation of binary64
+
** values.  To begin, first compute a table of exponent values:
+
**
+
**    CREATE TABLE pow2(x INTEGER PRIMARY KEY, v TEXT);
+
**    WITH RECURSIVE c(x,v) AS (
+
**      VALUES(0,'1')
+
**      UNION ALL
+
**      SELECT x+1, decimal_mul(v,'2') FROM c WHERE x+1<=971
+
**    ) INSERT INTO pow2(x,v) SELECT x, v FROM c;
+
**    WITH RECURSIVE c(x,v) AS (
+
**      VALUES(-1,'0.5')
+
**      UNION ALL
+
**      SELECT x-1, decimal_mul(v,'0.5') FROM c WHERE x-1>=-1075
+
**    ) INSERT INTO pow2(x,v) SELECT x, v FROM c;
+
**
+
** Then, to compute the exact decimal representation of a floating
+
** point value (the value 47.49 is used in the example) do:
+
**
+
**    WITH c(n) AS (VALUES(47.49))
+
**          ---------------^^^^^---- Replace with whatever you want
+
**    SELECT decimal_mul(ieee754_mantissa(c.n),pow2.v)
+
**      FROM pow2, c WHERE pow2.x=ieee754_exponent(c.n);
+
**
+
** Here is a query to show various boundry values for the binary64
+
** number format:
+
**
+
**    WITH c(name,bin) AS (VALUES
+
**       ('minimum positive value',        x'0000000000000001'),
+
**       ('maximum subnormal value',       x'000fffffffffffff'),
+
**       ('mininum positive nornal value', x'0010000000000000'),
+
**       ('maximum value',                 x'7fefffffffffffff'))
+
**    SELECT c.name, decimal_mul(ieee754_mantissa(c.bin),pow2.v)
+
**      FROM pow2, c WHERE pow2.x=ieee754_exponent(c.bin);
**
*/
/* #include "sqlite3ext.h" */
SQLITE_EXTENSION_INIT1
#include <assert.h>
#include <string.h>
-
#include <ctype.h>
-

-
#ifndef SQLITE_OMIT_VIRTUALTABLE
-

-
/* completion_vtab is a subclass of sqlite3_vtab which will
-
** serve as the underlying representation of a completion virtual table
-
*/
-
typedef struct completion_vtab completion_vtab;
-
struct completion_vtab {
-
  sqlite3_vtab base;  /* Base class - must be first */
-
  sqlite3 *db;        /* Database connection for this completion vtab */
-
};

-
/* completion_cursor is a subclass of sqlite3_vtab_cursor which will
-
** serve as the underlying representation of a cursor that scans
-
** over rows of the result
-
*/
-
typedef struct completion_cursor completion_cursor;
-
struct completion_cursor {
-
  sqlite3_vtab_cursor base;  /* Base class - must be first */
-
  sqlite3 *db;               /* Database connection for this cursor */
-
  int nPrefix, nLine;        /* Number of bytes in zPrefix and zLine */
-
  char *zPrefix;             /* The prefix for the word we want to complete */
-
  char *zLine;               /* The whole that we want to complete */
-
  const char *zCurrentRow;   /* Current output row */
-
  int szRow;                 /* Length of the zCurrentRow string */
-
  sqlite3_stmt *pStmt;       /* Current statement */
-
  sqlite3_int64 iRowid;      /* The rowid */
-
  int ePhase;                /* Current phase */
-
  int j;                     /* inter-phase counter */
-
};
-

-
/* Values for ePhase:
-
*/
-
#define COMPLETION_FIRST_PHASE   1
-
#define COMPLETION_KEYWORDS      1
-
#define COMPLETION_PRAGMAS       2
-
#define COMPLETION_FUNCTIONS     3
-
#define COMPLETION_COLLATIONS    4
-
#define COMPLETION_INDEXES       5
-
#define COMPLETION_TRIGGERS      6
-
#define COMPLETION_DATABASES     7
-
#define COMPLETION_TABLES        8    /* Also VIEWs and TRIGGERs */
-
#define COMPLETION_COLUMNS       9
-
#define COMPLETION_MODULES       10
-
#define COMPLETION_EOF           11
+
/* Mark a function parameter as unused, to suppress nuisance compiler
+
** warnings. */
+
#ifndef UNUSED_PARAMETER
+
# define UNUSED_PARAMETER(X)  (void)(X)
+
#endif

/*
-
** The completionConnect() method is invoked to create a new
-
** completion_vtab that describes the completion virtual table.
-
**
-
** Think of this routine as the constructor for completion_vtab objects.
-
**
-
** All this routine needs to do is:
-
**
-
**    (1) Allocate the completion_vtab object and initialize all fields.
-
**
-
**    (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the
-
**        result set of queries against completion will look like.
+
** Implementation of the ieee754() function
*/
-
static int completionConnect(
-
  sqlite3 *db,
-
  void *pAux,
-
  int argc, const char *const*argv,
-
  sqlite3_vtab **ppVtab,
-
  char **pzErr
+
static void ieee754func(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
){
-
  completion_vtab *pNew;
-
  int rc;
-

-
  (void)(pAux);    /* Unused parameter */
-
  (void)(argc);    /* Unused parameter */
-
  (void)(argv);    /* Unused parameter */
-
  (void)(pzErr);   /* Unused parameter */
-

-
/* Column numbers */
-
#define COMPLETION_COLUMN_CANDIDATE 0  /* Suggested completion of the input */
-
#define COMPLETION_COLUMN_PREFIX    1  /* Prefix of the word to be completed */
-
#define COMPLETION_COLUMN_WHOLELINE 2  /* Entire line seen so far */
-
#define COMPLETION_COLUMN_PHASE     3  /* ePhase - used for debugging only */
-

-
  sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
-
  rc = sqlite3_declare_vtab(db,
-
      "CREATE TABLE x("
-
      "  candidate TEXT,"
-
      "  prefix TEXT HIDDEN,"
-
      "  wholeline TEXT HIDDEN,"
-
      "  phase INT HIDDEN"        /* Used for debugging only */
-
      ")");
-
  if( rc==SQLITE_OK ){
-
    pNew = sqlite3_malloc( sizeof(*pNew) );
-
    *ppVtab = (sqlite3_vtab*)pNew;
-
    if( pNew==0 ) return SQLITE_NOMEM;
-
    memset(pNew, 0, sizeof(*pNew));
-
    pNew->db = db;
-
  }
-
  return rc;
-
}
-

-
/*
-
** This method is the destructor for completion_cursor objects.
-
*/
-
static int completionDisconnect(sqlite3_vtab *pVtab){
-
  sqlite3_free(pVtab);
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Constructor for a new completion_cursor object.
-
*/
-
static int completionOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
-
  completion_cursor *pCur;
-
  pCur = sqlite3_malloc( sizeof(*pCur) );
-
  if( pCur==0 ) return SQLITE_NOMEM;
-
  memset(pCur, 0, sizeof(*pCur));
-
  pCur->db = ((completion_vtab*)p)->db;
-
  *ppCursor = &pCur->base;
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Reset the completion_cursor.
-
*/
-
static void completionCursorReset(completion_cursor *pCur){
-
  sqlite3_free(pCur->zPrefix);   pCur->zPrefix = 0;  pCur->nPrefix = 0;
-
  sqlite3_free(pCur->zLine);     pCur->zLine = 0;    pCur->nLine = 0;
-
  sqlite3_finalize(pCur->pStmt); pCur->pStmt = 0;
-
  pCur->j = 0;
-
}
-

-
/*
-
** Destructor for a completion_cursor.
-
*/
-
static int completionClose(sqlite3_vtab_cursor *cur){
-
  completionCursorReset((completion_cursor*)cur);
-
  sqlite3_free(cur);
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Advance a completion_cursor to its next row of output.
-
**
-
** The ->ePhase, ->j, and ->pStmt fields of the completion_cursor object
-
** record the current state of the scan.  This routine sets ->zCurrentRow
-
** to the current row of output and then returns.  If no more rows remain,
-
** then ->ePhase is set to COMPLETION_EOF which will signal the virtual
-
** table that has reached the end of its scan.
-
**
-
** The current implementation just lists potential identifiers and
-
** keywords and filters them by zPrefix.  Future enhancements should
-
** take zLine into account to try to restrict the set of identifiers and
-
** keywords based on what would be legal at the current point of input.
-
*/
-
static int completionNext(sqlite3_vtab_cursor *cur){
-
  completion_cursor *pCur = (completion_cursor*)cur;
-
  int eNextPhase = 0;  /* Next phase to try if current phase reaches end */
-
  int iCol = -1;       /* If >=0, step pCur->pStmt and use the i-th column */
-
  pCur->iRowid++;
-
  while( pCur->ePhase!=COMPLETION_EOF ){
-
    switch( pCur->ePhase ){
-
      case COMPLETION_KEYWORDS: {
-
        if( pCur->j >= sqlite3_keyword_count() ){
-
          pCur->zCurrentRow = 0;
-
          pCur->ePhase = COMPLETION_DATABASES;
-
        }else{
-
          sqlite3_keyword_name(pCur->j++, &pCur->zCurrentRow, &pCur->szRow);
-
        }
-
        iCol = -1;
-
        break;
+
  if( argc==1 ){
+
    sqlite3_int64 m, a;
+
    double r;
+
    int e;
+
    int isNeg;
+
    char zResult[100];
+
    assert( sizeof(m)==sizeof(r) );
+
    if( sqlite3_value_type(argv[0])==SQLITE_BLOB
+
     && sqlite3_value_bytes(argv[0])==sizeof(r)
+
    ){
+
      const unsigned char *x = sqlite3_value_blob(argv[0]);
+
      unsigned int i;
+
      sqlite3_uint64 v = 0;
+
      for(i=0; i<sizeof(r); i++){
+
        v = (v<<8) | x[i];
      }
-
      case COMPLETION_DATABASES: {
-
        if( pCur->pStmt==0 ){
-
          sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1,
-
                             &pCur->pStmt, 0);
-
        }
-
        iCol = 1;
-
        eNextPhase = COMPLETION_TABLES;
-
        break;
+
      memcpy(&r, &v, sizeof(r));
+
    }else{
+
      r = sqlite3_value_double(argv[0]);
+
    }
+
    if( r<0.0 ){
+
      isNeg = 1;
+
      r = -r;
+
    }else{
+
      isNeg = 0;
+
    }
+
    memcpy(&a,&r,sizeof(a));
+
    if( a==0 ){
+
      e = 0;
+
      m = 0;
+
    }else{
+
      e = a>>52;
+
      m = a & ((((sqlite3_int64)1)<<52)-1);
+
      if( e==0 ){
+
        m <<= 1;
+
      }else{
+
        m |= ((sqlite3_int64)1)<<52;
      }
-
      case COMPLETION_TABLES: {
-
        if( pCur->pStmt==0 ){
-
          sqlite3_stmt *pS2;
-
          char *zSql = 0;
-
          const char *zSep = "";
-
          sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0);
-
          while( sqlite3_step(pS2)==SQLITE_ROW ){
-
            const char *zDb = (const char*)sqlite3_column_text(pS2, 1);
-
            zSql = sqlite3_mprintf(
-
               "%z%s"
-
               "SELECT name FROM \"%w\".sqlite_schema",
-
               zSql, zSep, zDb
-
            );
-
            if( zSql==0 ) return SQLITE_NOMEM;
-
            zSep = " UNION ";
-
          }
-
          sqlite3_finalize(pS2);
-
          sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
-
          sqlite3_free(zSql);
-
        }
-
        iCol = 0;
-
        eNextPhase = COMPLETION_COLUMNS;
-
        break;
+
      while( e<1075 && m>0 && (m&1)==0 ){
+
        m >>= 1;
+
        e++;
      }
-
      case COMPLETION_COLUMNS: {
-
        if( pCur->pStmt==0 ){
-
          sqlite3_stmt *pS2;
-
          char *zSql = 0;
-
          const char *zSep = "";
-
          sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0);
-
          while( sqlite3_step(pS2)==SQLITE_ROW ){
-
            const char *zDb = (const char*)sqlite3_column_text(pS2, 1);
-
            zSql = sqlite3_mprintf(
-
               "%z%s"
-
               "SELECT pti.name FROM \"%w\".sqlite_schema AS sm"
-
                       " JOIN pragma_table_info(sm.name,%Q) AS pti"
-
               " WHERE sm.type='table'",
-
               zSql, zSep, zDb, zDb
-
            );
-
            if( zSql==0 ) return SQLITE_NOMEM;
-
            zSep = " UNION ";
-
          }
-
          sqlite3_finalize(pS2);
-
          sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
-
          sqlite3_free(zSql);
-
        }
-
        iCol = 0;
-
        eNextPhase = COMPLETION_EOF;
+
      if( isNeg ) m = -m;
+
    }
+
    switch( *(int*)sqlite3_user_data(context) ){
+
      case 0:
+
        sqlite3_snprintf(sizeof(zResult), zResult, "ieee754(%lld,%d)",
+
                         m, e-1075);
+
        sqlite3_result_text(context, zResult, -1, SQLITE_TRANSIENT);
+
        break;
+
      case 1:
+
        sqlite3_result_int64(context, m);
+
        break;
+
      case 2:
+
        sqlite3_result_int(context, e-1075);
        break;
-
      }
    }
-
    if( iCol<0 ){
-
      /* This case is when the phase presets zCurrentRow */
-
      if( pCur->zCurrentRow==0 ) continue;
-
    }else{
-
      if( sqlite3_step(pCur->pStmt)==SQLITE_ROW ){
-
        /* Extract the next row of content */
-
        pCur->zCurrentRow = (const char*)sqlite3_column_text(pCur->pStmt, iCol);
-
        pCur->szRow = sqlite3_column_bytes(pCur->pStmt, iCol);
+
  }else{
+
    sqlite3_int64 m, e, a;
+
    double r;
+
    int isNeg = 0;
+
    m = sqlite3_value_int64(argv[0]);
+
    e = sqlite3_value_int64(argv[1]);
+

+
    /* Limit the range of e.  Ticket 22dea1cfdb9151e4 2021-03-02 */
+
    if( e>10000 ){
+
      e = 10000;
+
    }else if( e<-10000 ){
+
      e = -10000;
+
    }
+

+
    if( m<0 ){
+
      isNeg = 1;
+
      m = -m;
+
      if( m<0 ) return;
+
    }else if( m==0 && e>-1000 && e<1000 ){
+
      sqlite3_result_double(context, 0.0);
+
      return;
+
    }
+
    while( (m>>32)&0xffe00000 ){
+
      m >>= 1;
+
      e++;
+
    }
+
    while( m!=0 && ((m>>32)&0xfff00000)==0 ){
+
      m <<= 1;
+
      e--;
+
    }
+
    e += 1075;
+
    if( e<=0 ){
+
      /* Subnormal */
+
      if( 1-e >= 64 ){
+
        m = 0;
      }else{
-
        /* When all rows are finished, advance to the next phase */
-
        sqlite3_finalize(pCur->pStmt);
-
        pCur->pStmt = 0;
-
        pCur->ePhase = eNextPhase;
-
        continue;
+
        m >>= 1-e;
      }
+
      e = 0;
+
    }else if( e>0x7ff ){
+
      e = 0x7ff;
    }
-
    if( pCur->nPrefix==0 ) break;
-
    if( pCur->nPrefix<=pCur->szRow
-
     && sqlite3_strnicmp(pCur->zPrefix, pCur->zCurrentRow, pCur->nPrefix)==0
-
    ){
-
      break;
-
    }
+
    a = m & ((((sqlite3_int64)1)<<52)-1);
+
    a |= e<<52;
+
    if( isNeg ) a |= ((sqlite3_uint64)1)<<63;
+
    memcpy(&r, &a, sizeof(r));
+
    sqlite3_result_double(context, r);
  }
-

-
  return SQLITE_OK;
}

/*
-
** Return values of columns for the row at which the completion_cursor
-
** is currently pointing.
+
** Functions to convert between blobs and floats.
*/
-
static int completionColumn(
-
  sqlite3_vtab_cursor *cur,   /* The cursor */
-
  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
-
  int i                       /* Which column to return */
+
static void ieee754func_from_blob(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
){
-
  completion_cursor *pCur = (completion_cursor*)cur;
-
  switch( i ){
-
    case COMPLETION_COLUMN_CANDIDATE: {
-
      sqlite3_result_text(ctx, pCur->zCurrentRow, pCur->szRow,SQLITE_TRANSIENT);
-
      break;
-
    }
-
    case COMPLETION_COLUMN_PREFIX: {
-
      sqlite3_result_text(ctx, pCur->zPrefix, -1, SQLITE_TRANSIENT);
-
      break;
-
    }
-
    case COMPLETION_COLUMN_WHOLELINE: {
-
      sqlite3_result_text(ctx, pCur->zLine, -1, SQLITE_TRANSIENT);
-
      break;
-
    }
-
    case COMPLETION_COLUMN_PHASE: {
-
      sqlite3_result_int(ctx, pCur->ePhase);
-
      break;
-
    }
-
  }
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Return the rowid for the current row.  In this implementation, the
-
** rowid is the same as the output value.
-
*/
-
static int completionRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
-
  completion_cursor *pCur = (completion_cursor*)cur;
-
  *pRowid = pCur->iRowid;
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Return TRUE if the cursor has been moved off of the last
-
** row of output.
-
*/
-
static int completionEof(sqlite3_vtab_cursor *cur){
-
  completion_cursor *pCur = (completion_cursor*)cur;
-
  return pCur->ePhase >= COMPLETION_EOF;
-
}
-

-
/*
-
** This method is called to "rewind" the completion_cursor object back
-
** to the first row of output.  This method is always called at least
-
** once prior to any call to completionColumn() or completionRowid() or 
-
** completionEof().
-
*/
-
static int completionFilter(
-
  sqlite3_vtab_cursor *pVtabCursor, 
-
  int idxNum, const char *idxStr,
-
  int argc, sqlite3_value **argv
-
){
-
  completion_cursor *pCur = (completion_cursor *)pVtabCursor;
-
  int iArg = 0;
-
  (void)(idxStr);   /* Unused parameter */
-
  (void)(argc);     /* Unused parameter */
-
  completionCursorReset(pCur);
-
  if( idxNum & 1 ){
-
    pCur->nPrefix = sqlite3_value_bytes(argv[iArg]);
-
    if( pCur->nPrefix>0 ){
-
      pCur->zPrefix = sqlite3_mprintf("%s", sqlite3_value_text(argv[iArg]));
-
      if( pCur->zPrefix==0 ) return SQLITE_NOMEM;
-
    }
-
    iArg = 1;
-
  }
-
  if( idxNum & 2 ){
-
    pCur->nLine = sqlite3_value_bytes(argv[iArg]);
-
    if( pCur->nLine>0 ){
-
      pCur->zLine = sqlite3_mprintf("%s", sqlite3_value_text(argv[iArg]));
-
      if( pCur->zLine==0 ) return SQLITE_NOMEM;
-
    }
-
  }
-
  if( pCur->zLine!=0 && pCur->zPrefix==0 ){
-
    int i = pCur->nLine;
-
    while( i>0 && (isalnum(pCur->zLine[i-1]) || pCur->zLine[i-1]=='_') ){
-
      i--;
-
    }
-
    pCur->nPrefix = pCur->nLine - i;
-
    if( pCur->nPrefix>0 ){
-
      pCur->zPrefix = sqlite3_mprintf("%.*s", pCur->nPrefix, pCur->zLine + i);
-
      if( pCur->zPrefix==0 ) return SQLITE_NOMEM;
+
  UNUSED_PARAMETER(argc);
+
  if( sqlite3_value_type(argv[0])==SQLITE_BLOB
+
   && sqlite3_value_bytes(argv[0])==sizeof(double)
+
  ){
+
    double r;
+
    const unsigned char *x = sqlite3_value_blob(argv[0]);
+
    unsigned int i;
+
    sqlite3_uint64 v = 0;
+
    for(i=0; i<sizeof(r); i++){
+
      v = (v<<8) | x[i];
    }
+
    memcpy(&r, &v, sizeof(r));
+
    sqlite3_result_double(context, r);
  }
-
  pCur->iRowid = 0;
-
  pCur->ePhase = COMPLETION_FIRST_PHASE;
-
  return completionNext(pVtabCursor);
}
-

-
/*
-
** SQLite will invoke this method one or more times while planning a query
-
** that uses the completion virtual table.  This routine needs to create
-
** a query plan for each invocation and compute an estimated cost for that
-
** plan.
-
**
-
** There are two hidden parameters that act as arguments to the table-valued
-
** function:  "prefix" and "wholeline".  Bit 0 of idxNum is set if "prefix"
-
** is available and bit 1 is set if "wholeline" is available.
-
*/
-
static int completionBestIndex(
-
  sqlite3_vtab *tab,
-
  sqlite3_index_info *pIdxInfo
+
static void ieee754func_to_blob(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
){
-
  int i;                 /* Loop over constraints */
-
  int idxNum = 0;        /* The query plan bitmask */
-
  int prefixIdx = -1;    /* Index of the start= constraint, or -1 if none */
-
  int wholelineIdx = -1; /* Index of the stop= constraint, or -1 if none */
-
  int nArg = 0;          /* Number of arguments that completeFilter() expects */
-
  const struct sqlite3_index_constraint *pConstraint;
-

-
  (void)(tab);    /* Unused parameter */
-
  pConstraint = pIdxInfo->aConstraint;
-
  for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){
-
    if( pConstraint->usable==0 ) continue;
-
    if( pConstraint->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue;
-
    switch( pConstraint->iColumn ){
-
      case COMPLETION_COLUMN_PREFIX:
-
        prefixIdx = i;
-
        idxNum |= 1;
-
        break;
-
      case COMPLETION_COLUMN_WHOLELINE:
-
        wholelineIdx = i;
-
        idxNum |= 2;
-
        break;
+
  UNUSED_PARAMETER(argc);
+
  if( sqlite3_value_type(argv[0])==SQLITE_FLOAT
+
   || sqlite3_value_type(argv[0])==SQLITE_INTEGER
+
  ){
+
    double r = sqlite3_value_double(argv[0]);
+
    sqlite3_uint64 v;
+
    unsigned char a[sizeof(r)];
+
    unsigned int i;
+
    memcpy(&v, &r, sizeof(r));
+
    for(i=1; i<=sizeof(r); i++){
+
      a[sizeof(r)-i] = v&0xff;
+
      v >>= 8;
    }
+
    sqlite3_result_blob(context, a, sizeof(r), SQLITE_TRANSIENT);
  }
-
  if( prefixIdx>=0 ){
-
    pIdxInfo->aConstraintUsage[prefixIdx].argvIndex = ++nArg;
-
    pIdxInfo->aConstraintUsage[prefixIdx].omit = 1;
-
  }
-
  if( wholelineIdx>=0 ){
-
    pIdxInfo->aConstraintUsage[wholelineIdx].argvIndex = ++nArg;
-
    pIdxInfo->aConstraintUsage[wholelineIdx].omit = 1;
-
  }
-
  pIdxInfo->idxNum = idxNum;
-
  pIdxInfo->estimatedCost = (double)5000 - 1000*nArg;
-
  pIdxInfo->estimatedRows = 500 - 100*nArg;
-
  return SQLITE_OK;
}

-
/*
-
** This following structure defines all the methods for the 
-
** completion virtual table.
-
*/
-
static sqlite3_module completionModule = {
-
  0,                         /* iVersion */
-
  0,                         /* xCreate */
-
  completionConnect,         /* xConnect */
-
  completionBestIndex,       /* xBestIndex */
-
  completionDisconnect,      /* xDisconnect */
-
  0,                         /* xDestroy */
-
  completionOpen,            /* xOpen - open a cursor */
-
  completionClose,           /* xClose - close a cursor */
-
  completionFilter,          /* xFilter - configure scan constraints */
-
  completionNext,            /* xNext - advance a cursor */
-
  completionEof,             /* xEof - check for end of scan */
-
  completionColumn,          /* xColumn - read data */
-
  completionRowid,           /* xRowid - read data */
-
  0,                         /* xUpdate */
-
  0,                         /* xBegin */
-
  0,                         /* xSync */
-
  0,                         /* xCommit */
-
  0,                         /* xRollback */
-
  0,                         /* xFindMethod */
-
  0,                         /* xRename */
-
  0,                         /* xSavepoint */
-
  0,                         /* xRelease */
-
  0,                         /* xRollbackTo */
-
  0                          /* xShadowName */
-
};
-

-
#endif /* SQLITE_OMIT_VIRTUALTABLE */
-

-
int sqlite3CompletionVtabInit(sqlite3 *db){
-
  int rc = SQLITE_OK;
-
#ifndef SQLITE_OMIT_VIRTUALTABLE
-
  rc = sqlite3_create_module(db, "completion", &completionModule, 0);
-
#endif
-
  return rc;
-
}

#ifdef _WIN32

#endif
-
int sqlite3_completion_init(
+
int sqlite3_ieee_init(
  sqlite3 *db, 
  char **pzErrMsg, 
  const sqlite3_api_routines *pApi
){
+
  static const struct {
+
    char *zFName;
+
    int nArg;
+
    int iAux;
+
    void (*xFunc)(sqlite3_context*,int,sqlite3_value**);
+
  } aFunc[] = {
+
    { "ieee754",           1,   0, ieee754func },
+
    { "ieee754",           2,   0, ieee754func },
+
    { "ieee754_mantissa",  1,   1, ieee754func },
+
    { "ieee754_exponent",  1,   2, ieee754func },
+
    { "ieee754_to_blob",   1,   0, ieee754func_to_blob },
+
    { "ieee754_from_blob", 1,   0, ieee754func_from_blob },
+

+
  };
+
  unsigned int i;
  int rc = SQLITE_OK;
  SQLITE_EXTENSION_INIT2(pApi);
-
  (void)(pzErrMsg);  /* Unused parameter */
-
#ifndef SQLITE_OMIT_VIRTUALTABLE
-
  rc = sqlite3CompletionVtabInit(db);
-
#endif
+
  (void)pzErrMsg;  /* Unused parameter */
+
  for(i=0; i<sizeof(aFunc)/sizeof(aFunc[0]) && rc==SQLITE_OK; i++){
+
    rc = sqlite3_create_function(db, aFunc[i].zFName, aFunc[i].nArg,
+
                               SQLITE_UTF8|SQLITE_INNOCUOUS,
+
                               (void*)&aFunc[i].iAux,
+
                               aFunc[i].xFunc, 0, 0);
+
  }
  return rc;
}

-
/************************* End ../ext/misc/completion.c ********************/
-
/************************* Begin ../ext/misc/appendvfs.c ******************/
+
/************************* End ../ext/misc/ieee754.c ********************/
+
/************************* Begin ../ext/misc/series.c ******************/
/*
-
** 2017-10-20
+
** 2015-08-18
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
@@ -3656,589 +3291,3472 @@ int sqlite3_completion_init(
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
-
******************************************************************************
+
*************************************************************************
**
-
** This file implements a VFS shim that allows an SQLite database to be
-
** appended onto the end of some other file, such as an executable.
+
** This file demonstrates how to create a table-valued-function using
+
** a virtual table.  This demo implements the generate_series() function
+
** which gives similar results to the eponymous function in PostgreSQL.
+
** Examples:
**
-
** A special record must appear at the end of the file that identifies the
-
** file as an appended database and provides the offset to the first page
-
** of the exposed content. (Or, it is the length of the content prefix.)
-
** For best performance page 1 should be located at a disk page boundary,
-
** though that is not required.
+
**      SELECT * FROM generate_series(0,100,5);
**
-
** When opening a database using this VFS, the connection might treat
-
** the file as an ordinary SQLite database, or it might treat it as a
-
** database appended onto some other file.  The decision is made by
-
** applying the following rules in order:
+
** The query above returns integers from 0 through 100 counting by steps
+
** of 5.
**
-
**  (1)  An empty file is an ordinary database.
+
**      SELECT * FROM generate_series(0,100);
**
-
**  (2)  If the file ends with the appendvfs trailer string
-
**       "Start-Of-SQLite3-NNNNNNNN" that file is an appended database.
+
** Integers from 0 through 100 with a step size of 1.
**
-
**  (3)  If the file begins with the standard SQLite prefix string
-
**       "SQLite format 3", that file is an ordinary database.
+
**      SELECT * FROM generate_series(20) LIMIT 10;
**
-
**  (4)  If none of the above apply and the SQLITE_OPEN_CREATE flag is
-
**       set, then a new database is appended to the already existing file.
+
** Integers 20 through 29.
**
-
**  (5)  Otherwise, SQLITE_CANTOPEN is returned.
+
** HOW IT WORKS
**
-
** To avoid unnecessary complications with the PENDING_BYTE, the size of
-
** the file containing the database is limited to 1GiB. (1073741824 bytes)
-
** This VFS will not read or write past the 1GiB mark.  This restriction
-
** might be lifted in future versions.  For now, if you need a larger
-
** database, then keep it in a separate file.
+
** The generate_series "function" is really a virtual table with the
+
** following schema:
**
-
** If the file being opened is a plain database (not an appended one), then
-
** this shim is a pass-through into the default underlying VFS. (rule 3)
-
**/
+
**     CREATE TABLE generate_series(
+
**       value,
+
**       start HIDDEN,
+
**       stop HIDDEN,
+
**       step HIDDEN
+
**     );
+
**
+
** Function arguments in queries against this virtual table are translated
+
** into equality constraints against successive hidden columns.  In other
+
** words, the following pairs of queries are equivalent to each other:
+
**
+
**    SELECT * FROM generate_series(0,100,5);
+
**    SELECT * FROM generate_series WHERE start=0 AND stop=100 AND step=5;
+
**
+
**    SELECT * FROM generate_series(0,100);
+
**    SELECT * FROM generate_series WHERE start=0 AND stop=100;
+
**
+
**    SELECT * FROM generate_series(20) LIMIT 10;
+
**    SELECT * FROM generate_series WHERE start=20 LIMIT 10;
+
**
+
** The generate_series virtual table implementation leaves the xCreate method
+
** set to NULL.  This means that it is not possible to do a CREATE VIRTUAL
+
** TABLE command with "generate_series" as the USING argument.  Instead, there
+
** is a single generate_series virtual table that is always available without
+
** having to be created first.
+
**
+
** The xBestIndex method looks for equality constraints against the hidden
+
** start, stop, and step columns, and if present, it uses those constraints
+
** to bound the sequence of generated values.  If the equality constraints
+
** are missing, it uses 0 for start, 4294967295 for stop, and 1 for step.
+
** xBestIndex returns a small cost when both start and stop are available,
+
** and a very large cost if either start or stop are unavailable.  This
+
** encourages the query planner to order joins such that the bounds of the
+
** series are well-defined.
+
*/
/* #include "sqlite3ext.h" */
SQLITE_EXTENSION_INIT1
-
#include <string.h>
#include <assert.h>
+
#include <string.h>

-
/* The append mark at the end of the database is:
+
#ifndef SQLITE_OMIT_VIRTUALTABLE
+

+

+
/* series_cursor is a subclass of sqlite3_vtab_cursor which will
+
** serve as the underlying representation of a cursor that scans
+
** over rows of the result
+
*/
+
typedef struct series_cursor series_cursor;
+
struct series_cursor {
+
  sqlite3_vtab_cursor base;  /* Base class - must be first */
+
  int isDesc;                /* True to count down rather than up */
+
  sqlite3_int64 iRowid;      /* The rowid */
+
  sqlite3_int64 iValue;      /* Current value ("value") */
+
  sqlite3_int64 mnValue;     /* Mimimum value ("start") */
+
  sqlite3_int64 mxValue;     /* Maximum value ("stop") */
+
  sqlite3_int64 iStep;       /* Increment ("step") */
+
};
+

+
/*
+
** The seriesConnect() method is invoked to create a new
+
** series_vtab that describes the generate_series virtual table.
**
-
**     Start-Of-SQLite3-NNNNNNNN
-
**     123456789 123456789 12345
+
** Think of this routine as the constructor for series_vtab objects.
**
-
** The NNNNNNNN represents a 64-bit big-endian unsigned integer which is
-
** the offset to page 1, and also the length of the prefix content.
+
** All this routine needs to do is:
+
**
+
**    (1) Allocate the series_vtab object and initialize all fields.
+
**
+
**    (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the
+
**        result set of queries against generate_series will look like.
*/
-
#define APND_MARK_PREFIX     "Start-Of-SQLite3-"
-
#define APND_MARK_PREFIX_SZ  17
-
#define APND_MARK_FOS_SZ      8
-
#define APND_MARK_SIZE       (APND_MARK_PREFIX_SZ+APND_MARK_FOS_SZ)
+
static int seriesConnect(
+
  sqlite3 *db,
+
  void *pUnused,
+
  int argcUnused, const char *const*argvUnused,
+
  sqlite3_vtab **ppVtab,
+
  char **pzErrUnused
+
){
+
  sqlite3_vtab *pNew;
+
  int rc;
+

+
/* Column numbers */
+
#define SERIES_COLUMN_VALUE 0
+
#define SERIES_COLUMN_START 1
+
#define SERIES_COLUMN_STOP  2
+
#define SERIES_COLUMN_STEP  3
+

+
  (void)pUnused;
+
  (void)argcUnused;
+
  (void)argvUnused;
+
  (void)pzErrUnused;
+
  rc = sqlite3_declare_vtab(db,
+
     "CREATE TABLE x(value,start hidden,stop hidden,step hidden)");
+
  if( rc==SQLITE_OK ){
+
    pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) );
+
    if( pNew==0 ) return SQLITE_NOMEM;
+
    memset(pNew, 0, sizeof(*pNew));
+
    sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
+
  }
+
  return rc;
+
}

/*
-
** Maximum size of the combined prefix + database + append-mark.  This
-
** must be less than 0x40000000 to avoid locking issues on Windows.
+
** This method is the destructor for series_cursor objects.
*/
-
#define APND_MAX_SIZE  (0x40000000)
+
static int seriesDisconnect(sqlite3_vtab *pVtab){
+
  sqlite3_free(pVtab);
+
  return SQLITE_OK;
+
}

/*
-
** Try to align the database to an even multiple of APND_ROUNDUP bytes.
+
** Constructor for a new series_cursor object.
*/
-
#ifndef APND_ROUNDUP
-
#define APND_ROUNDUP 4096
-
#endif
-
#define APND_ALIGN_MASK         ((sqlite3_int64)(APND_ROUNDUP-1))
-
#define APND_START_ROUNDUP(fsz) (((fsz)+APND_ALIGN_MASK) & ~APND_ALIGN_MASK)
+
static int seriesOpen(sqlite3_vtab *pUnused, sqlite3_vtab_cursor **ppCursor){
+
  series_cursor *pCur;
+
  (void)pUnused;
+
  pCur = sqlite3_malloc( sizeof(*pCur) );
+
  if( pCur==0 ) return SQLITE_NOMEM;
+
  memset(pCur, 0, sizeof(*pCur));
+
  *ppCursor = &pCur->base;
+
  return SQLITE_OK;
+
}

/*
-
** Forward declaration of objects used by this utility
+
** Destructor for a series_cursor.
*/
-
typedef struct sqlite3_vfs ApndVfs;
-
typedef struct ApndFile ApndFile;
+
static int seriesClose(sqlite3_vtab_cursor *cur){
+
  sqlite3_free(cur);
+
  return SQLITE_OK;
+
}

-
/* Access to a lower-level VFS that (might) implement dynamic loading,
-
** access to randomness, etc.
+

+
/*
+
** Advance a series_cursor to its next row of output.
*/
-
#define ORIGVFS(p)  ((sqlite3_vfs*)((p)->pAppData))
-
#define ORIGFILE(p) ((sqlite3_file*)(((ApndFile*)(p))+1))
+
static int seriesNext(sqlite3_vtab_cursor *cur){
+
  series_cursor *pCur = (series_cursor*)cur;
+
  if( pCur->isDesc ){
+
    pCur->iValue -= pCur->iStep;
+
  }else{
+
    pCur->iValue += pCur->iStep;
+
  }
+
  pCur->iRowid++;
+
  return SQLITE_OK;
+
}

-
/* An open appendvfs file
-
**
-
** An instance of this structure describes the appended database file.
-
** A separate sqlite3_file object is always appended. The appended
-
** sqlite3_file object (which can be accessed using ORIGFILE()) describes
-
** the entire file, including the prefix, the database, and the
-
** append-mark.
-
**
-
** The structure of an AppendVFS database is like this:
-
**
-
**   +-------------+---------+----------+-------------+
-
**   | prefix-file | padding | database | append-mark |
-
**   +-------------+---------+----------+-------------+
-
**                           ^          ^
-
**                           |          |
-
**                         iPgOne      iMark
-
**
-
**
-
** "prefix file" -  file onto which the database has been appended.
-
** "padding"     -  zero or more bytes inserted so that "database"
-
**                  starts on an APND_ROUNDUP boundary
-
** "database"    -  The SQLite database file
-
** "append-mark" -  The 25-byte "Start-Of-SQLite3-NNNNNNNN" that indicates
-
**                  the offset from the start of prefix-file to the start
-
**                  of "database".
-
**
-
** The size of the database is iMark - iPgOne.
-
**
-
** The NNNNNNNN in the "Start-Of-SQLite3-NNNNNNNN" suffix is the value
-
** of iPgOne stored as a big-ending 64-bit integer.
-
**
-
** iMark will be the size of the underlying file minus 25 (APND_MARKSIZE).
-
** Or, iMark is -1 to indicate that it has not yet been written.
+
/*
+
** Return values of columns for the row at which the series_cursor
+
** is currently pointing.
*/
-
struct ApndFile {
-
  sqlite3_file base;        /* Subclass.  MUST BE FIRST! */
-
  sqlite3_int64 iPgOne;     /* Offset to the start of the database */
-
  sqlite3_int64 iMark;      /* Offset of the append mark.  -1 if unwritten */
-
  /* Always followed by another sqlite3_file that describes the whole file */
-
};
+
static int seriesColumn(
+
  sqlite3_vtab_cursor *cur,   /* The cursor */
+
  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
+
  int i                       /* Which column to return */
+
){
+
  series_cursor *pCur = (series_cursor*)cur;
+
  sqlite3_int64 x = 0;
+
  switch( i ){
+
    case SERIES_COLUMN_START:  x = pCur->mnValue; break;
+
    case SERIES_COLUMN_STOP:   x = pCur->mxValue; break;
+
    case SERIES_COLUMN_STEP:   x = pCur->iStep;   break;
+
    default:                   x = pCur->iValue;  break;
+
  }
+
  sqlite3_result_int64(ctx, x);
+
  return SQLITE_OK;
+
}

/*
-
** Methods for ApndFile
+
** Return the rowid for the current row. In this implementation, the
+
** first row returned is assigned rowid value 1, and each subsequent
+
** row a value 1 more than that of the previous.
*/
-
static int apndClose(sqlite3_file*);
-
static int apndRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
-
static int apndWrite(sqlite3_file*,const void*,int iAmt, sqlite3_int64 iOfst);
-
static int apndTruncate(sqlite3_file*, sqlite3_int64 size);
-
static int apndSync(sqlite3_file*, int flags);
-
static int apndFileSize(sqlite3_file*, sqlite3_int64 *pSize);
-
static int apndLock(sqlite3_file*, int);
-
static int apndUnlock(sqlite3_file*, int);
-
static int apndCheckReservedLock(sqlite3_file*, int *pResOut);
-
static int apndFileControl(sqlite3_file*, int op, void *pArg);
-
static int apndSectorSize(sqlite3_file*);
-
static int apndDeviceCharacteristics(sqlite3_file*);
-
static int apndShmMap(sqlite3_file*, int iPg, int pgsz, int, void volatile**);
-
static int apndShmLock(sqlite3_file*, int offset, int n, int flags);
-
static void apndShmBarrier(sqlite3_file*);
-
static int apndShmUnmap(sqlite3_file*, int deleteFlag);
-
static int apndFetch(sqlite3_file*, sqlite3_int64 iOfst, int iAmt, void **pp);
-
static int apndUnfetch(sqlite3_file*, sqlite3_int64 iOfst, void *p);
+
static int seriesRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
+
  series_cursor *pCur = (series_cursor*)cur;
+
  *pRowid = pCur->iRowid;
+
  return SQLITE_OK;
+
}

/*
-
** Methods for ApndVfs
+
** Return TRUE if the cursor has been moved off of the last
+
** row of output.
*/
-
static int apndOpen(sqlite3_vfs*, const char *, sqlite3_file*, int , int *);
-
static int apndDelete(sqlite3_vfs*, const char *zName, int syncDir);
-
static int apndAccess(sqlite3_vfs*, const char *zName, int flags, int *);
-
static int apndFullPathname(sqlite3_vfs*, const char *zName, int, char *zOut);
-
static void *apndDlOpen(sqlite3_vfs*, const char *zFilename);
-
static void apndDlError(sqlite3_vfs*, int nByte, char *zErrMsg);
-
static void (*apndDlSym(sqlite3_vfs *pVfs, void *p, const char*zSym))(void);
-
static void apndDlClose(sqlite3_vfs*, void*);
-
static int apndRandomness(sqlite3_vfs*, int nByte, char *zOut);
-
static int apndSleep(sqlite3_vfs*, int microseconds);
-
static int apndCurrentTime(sqlite3_vfs*, double*);
-
static int apndGetLastError(sqlite3_vfs*, int, char *);
-
static int apndCurrentTimeInt64(sqlite3_vfs*, sqlite3_int64*);
-
static int apndSetSystemCall(sqlite3_vfs*, const char*,sqlite3_syscall_ptr);
-
static sqlite3_syscall_ptr apndGetSystemCall(sqlite3_vfs*, const char *z);
-
static const char *apndNextSystemCall(sqlite3_vfs*, const char *zName);
-

-
static sqlite3_vfs apnd_vfs = {
-
  3,                            /* iVersion (set when registered) */
-
  0,                            /* szOsFile (set when registered) */
-
  1024,                         /* mxPathname */
-
  0,                            /* pNext */
-
  "apndvfs",                    /* zName */
-
  0,                            /* pAppData (set when registered) */ 
-
  apndOpen,                     /* xOpen */
-
  apndDelete,                   /* xDelete */
-
  apndAccess,                   /* xAccess */
-
  apndFullPathname,             /* xFullPathname */
-
  apndDlOpen,                   /* xDlOpen */
-
  apndDlError,                  /* xDlError */
-
  apndDlSym,                    /* xDlSym */
-
  apndDlClose,                  /* xDlClose */
-
  apndRandomness,               /* xRandomness */
-
  apndSleep,                    /* xSleep */
-
  apndCurrentTime,              /* xCurrentTime */
-
  apndGetLastError,             /* xGetLastError */
-
  apndCurrentTimeInt64,         /* xCurrentTimeInt64 */
-
  apndSetSystemCall,            /* xSetSystemCall */
-
  apndGetSystemCall,            /* xGetSystemCall */
-
  apndNextSystemCall            /* xNextSystemCall */
-
};
-

-
static const sqlite3_io_methods apnd_io_methods = {
-
  3,                              /* iVersion */
-
  apndClose,                      /* xClose */
-
  apndRead,                       /* xRead */
-
  apndWrite,                      /* xWrite */
-
  apndTruncate,                   /* xTruncate */
-
  apndSync,                       /* xSync */
-
  apndFileSize,                   /* xFileSize */
-
  apndLock,                       /* xLock */
-
  apndUnlock,                     /* xUnlock */
-
  apndCheckReservedLock,          /* xCheckReservedLock */
-
  apndFileControl,                /* xFileControl */
-
  apndSectorSize,                 /* xSectorSize */
-
  apndDeviceCharacteristics,      /* xDeviceCharacteristics */
-
  apndShmMap,                     /* xShmMap */
-
  apndShmLock,                    /* xShmLock */
-
  apndShmBarrier,                 /* xShmBarrier */
-
  apndShmUnmap,                   /* xShmUnmap */
-
  apndFetch,                      /* xFetch */
-
  apndUnfetch                     /* xUnfetch */
-
};
-

-
/*
-
** Close an apnd-file.
-
*/
-
static int apndClose(sqlite3_file *pFile){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xClose(pFile);
+
static int seriesEof(sqlite3_vtab_cursor *cur){
+
  series_cursor *pCur = (series_cursor*)cur;
+
  if( pCur->isDesc ){
+
    return pCur->iValue < pCur->mnValue;
+
  }else{
+
    return pCur->iValue > pCur->mxValue;
+
  }
}

-
/*
-
** Read data from an apnd-file.
+
/* True to cause run-time checking of the start=, stop=, and/or step= 
+
** parameters.  The only reason to do this is for testing the
+
** constraint checking logic for virtual tables in the SQLite core.
*/
-
static int apndRead(
-
  sqlite3_file *pFile, 
-
  void *zBuf, 
-
  int iAmt, 
-
  sqlite_int64 iOfst
-
){
-
  ApndFile *paf = (ApndFile *)pFile;
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xRead(pFile, zBuf, iAmt, paf->iPgOne+iOfst);
-
}
+
#ifndef SQLITE_SERIES_CONSTRAINT_VERIFY
+
# define SQLITE_SERIES_CONSTRAINT_VERIFY 0
+
#endif

/*
-
** Add the append-mark onto what should become the end of the file.
-
*  If and only if this succeeds, internal ApndFile.iMark is updated.
-
*  Parameter iWriteEnd is the appendvfs-relative offset of the new mark.
+
** This method is called to "rewind" the series_cursor object back
+
** to the first row of output.  This method is always called at least
+
** once prior to any call to seriesColumn() or seriesRowid() or 
+
** seriesEof().
+
**
+
** The query plan selected by seriesBestIndex is passed in the idxNum
+
** parameter.  (idxStr is not used in this implementation.)  idxNum
+
** is a bitmask showing which constraints are available:
+
**
+
**    1:    start=VALUE
+
**    2:    stop=VALUE
+
**    4:    step=VALUE
+
**
+
** Also, if bit 8 is set, that means that the series should be output
+
** in descending order rather than in ascending order.  If bit 16 is
+
** set, then output must appear in ascending order.
+
**
+
** This routine should initialize the cursor and position it so that it
+
** is pointing at the first row, or pointing off the end of the table
+
** (so that seriesEof() will return true) if the table is empty.
*/
-
static int apndWriteMark(
-
  ApndFile *paf,
-
  sqlite3_file *pFile,
-
  sqlite_int64 iWriteEnd
+
static int seriesFilter(
+
  sqlite3_vtab_cursor *pVtabCursor, 
+
  int idxNum, const char *idxStrUnused,
+
  int argc, sqlite3_value **argv
){
-
  sqlite_int64 iPgOne = paf->iPgOne;
-
  unsigned char a[APND_MARK_SIZE];
-
  int i = APND_MARK_FOS_SZ;
-
  int rc;
-
  assert(pFile == ORIGFILE(paf));
-
  memcpy(a, APND_MARK_PREFIX, APND_MARK_PREFIX_SZ);
-
  while( --i >= 0 ){
-
    a[APND_MARK_PREFIX_SZ+i] = (unsigned char)(iPgOne & 0xff);
-
    iPgOne >>= 8;
+
  series_cursor *pCur = (series_cursor *)pVtabCursor;
+
  int i = 0;
+
  (void)idxStrUnused;
+
  if( idxNum & 1 ){
+
    pCur->mnValue = sqlite3_value_int64(argv[i++]);
+
  }else{
+
    pCur->mnValue = 0;
  }
-
  iWriteEnd += paf->iPgOne;
-
  if( SQLITE_OK==(rc = pFile->pMethods->xWrite
-
                  (pFile, a, APND_MARK_SIZE, iWriteEnd)) ){
-
    paf->iMark = iWriteEnd;
+
  if( idxNum & 2 ){
+
    pCur->mxValue = sqlite3_value_int64(argv[i++]);
+
  }else{
+
    pCur->mxValue = 0xffffffff;
  }
-
  return rc;
-
}
-

-
/*
-
** Write data to an apnd-file.
-
*/
-
static int apndWrite(
-
  sqlite3_file *pFile,
-
  const void *zBuf,
-
  int iAmt,
-
  sqlite_int64 iOfst
-
){
-
  ApndFile *paf = (ApndFile *)pFile;
-
  sqlite_int64 iWriteEnd = iOfst + iAmt;
-
  if( iWriteEnd>=APND_MAX_SIZE ) return SQLITE_FULL;
-
  pFile = ORIGFILE(pFile);
-
  /* If append-mark is absent or will be overwritten, write it. */
-
  if( paf->iMark < 0 || paf->iPgOne + iWriteEnd > paf->iMark ){
-
    int rc = apndWriteMark(paf, pFile, iWriteEnd);
-
    if( SQLITE_OK!=rc ) return rc;
+
  if( idxNum & 4 ){
+
    pCur->iStep = sqlite3_value_int64(argv[i++]);
+
    if( pCur->iStep==0 ){
+
      pCur->iStep = 1;
+
    }else if( pCur->iStep<0 ){
+
      pCur->iStep = -pCur->iStep;
+
      if( (idxNum & 16)==0 ) idxNum |= 8;
+
    }
+
  }else{
+
    pCur->iStep = 1;
  }
-
  return pFile->pMethods->xWrite(pFile, zBuf, iAmt, paf->iPgOne+iOfst);
+
  for(i=0; i<argc; i++){
+
    if( sqlite3_value_type(argv[i])==SQLITE_NULL ){
+
      /* If any of the constraints have a NULL value, then return no rows.
+
      ** See ticket https://www.sqlite.org/src/info/fac496b61722daf2 */
+
      pCur->mnValue = 1;
+
      pCur->mxValue = 0;
+
      break;
+
    }
+
  }
+
  if( idxNum & 8 ){
+
    pCur->isDesc = 1;
+
    pCur->iValue = pCur->mxValue;
+
    if( pCur->iStep>0 ){
+
      pCur->iValue -= (pCur->mxValue - pCur->mnValue)%pCur->iStep;
+
    }
+
  }else{
+
    pCur->isDesc = 0;
+
    pCur->iValue = pCur->mnValue;
+
  }
+
  pCur->iRowid = 1;
+
  return SQLITE_OK;
}

/*
-
** Truncate an apnd-file.
+
** SQLite will invoke this method one or more times while planning a query
+
** that uses the generate_series virtual table.  This routine needs to create
+
** a query plan for each invocation and compute an estimated cost for that
+
** plan.
+
**
+
** In this implementation idxNum is used to represent the
+
** query plan.  idxStr is unused.
+
**
+
** The query plan is represented by bits in idxNum:
+
**
+
**  (1)  start = $value  -- constraint exists
+
**  (2)  stop = $value   -- constraint exists
+
**  (4)  step = $value   -- constraint exists
+
**  (8)  output in descending order
*/
-
static int apndTruncate(sqlite3_file *pFile, sqlite_int64 size){
-
  ApndFile *paf = (ApndFile *)pFile;
-
  pFile = ORIGFILE(pFile);
-
  /* The append mark goes out first so truncate failure does not lose it. */
-
  if( SQLITE_OK!=apndWriteMark(paf, pFile, size) ) return SQLITE_IOERR;
-
  /* Truncate underlying file just past append mark */
-
  return pFile->pMethods->xTruncate(pFile, paf->iMark+APND_MARK_SIZE);
-
}
+
static int seriesBestIndex(
+
  sqlite3_vtab *pVTab,
+
  sqlite3_index_info *pIdxInfo
+
){
+
  int i, j;              /* Loop over constraints */
+
  int idxNum = 0;        /* The query plan bitmask */
+
  int bStartSeen = 0;    /* EQ constraint seen on the START column */
+
  int unusableMask = 0;  /* Mask of unusable constraints */
+
  int nArg = 0;          /* Number of arguments that seriesFilter() expects */
+
  int aIdx[3];           /* Constraints on start, stop, and step */
+
  const struct sqlite3_index_constraint *pConstraint;

-
/*
-
** Sync an apnd-file.
-
*/
-
static int apndSync(sqlite3_file *pFile, int flags){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xSync(pFile, flags);
-
}
+
  /* This implementation assumes that the start, stop, and step columns
+
  ** are the last three columns in the virtual table. */
+
  assert( SERIES_COLUMN_STOP == SERIES_COLUMN_START+1 );
+
  assert( SERIES_COLUMN_STEP == SERIES_COLUMN_START+2 );

-
/*
-
** Return the current file-size of an apnd-file.
-
** If the append mark is not yet there, the file-size is 0.
-
*/
-
static int apndFileSize(sqlite3_file *pFile, sqlite_int64 *pSize){
-
  ApndFile *paf = (ApndFile *)pFile;
-
  *pSize = ( paf->iMark >= 0 )? (paf->iMark - paf->iPgOne) : 0;
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Lock an apnd-file.
-
*/
-
static int apndLock(sqlite3_file *pFile, int eLock){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xLock(pFile, eLock);
+
  aIdx[0] = aIdx[1] = aIdx[2] = -1;
+
  pConstraint = pIdxInfo->aConstraint;
+
  for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){
+
    int iCol;    /* 0 for start, 1 for stop, 2 for step */
+
    int iMask;   /* bitmask for those column */
+
    if( pConstraint->iColumn<SERIES_COLUMN_START ) continue;
+
    iCol = pConstraint->iColumn - SERIES_COLUMN_START;
+
    assert( iCol>=0 && iCol<=2 );
+
    iMask = 1 << iCol;
+
    if( iCol==0 ) bStartSeen = 1;
+
    if( pConstraint->usable==0 ){
+
      unusableMask |=  iMask;
+
      continue;
+
    }else if( pConstraint->op==SQLITE_INDEX_CONSTRAINT_EQ ){
+
      idxNum |= iMask;
+
      aIdx[iCol] = i;
+
    }
+
  }
+
  for(i=0; i<3; i++){
+
    if( (j = aIdx[i])>=0 ){
+
      pIdxInfo->aConstraintUsage[j].argvIndex = ++nArg;
+
      pIdxInfo->aConstraintUsage[j].omit = !SQLITE_SERIES_CONSTRAINT_VERIFY;
+
    }
+
  }
+
  /* The current generate_column() implementation requires at least one
+
  ** argument (the START value).  Legacy versions assumed START=0 if the
+
  ** first argument was omitted.  Compile with -DZERO_ARGUMENT_GENERATE_SERIES
+
  ** to obtain the legacy behavior */
+
#ifndef ZERO_ARGUMENT_GENERATE_SERIES
+
  if( !bStartSeen ){
+
    sqlite3_free(pVTab->zErrMsg);
+
    pVTab->zErrMsg = sqlite3_mprintf(
+
        "first argument to \"generate_series()\" missing or unusable");
+
    return SQLITE_ERROR;
+
  }
+
#endif
+
  if( (unusableMask & ~idxNum)!=0 ){
+
    /* The start, stop, and step columns are inputs.  Therefore if there
+
    ** are unusable constraints on any of start, stop, or step then
+
    ** this plan is unusable */
+
    return SQLITE_CONSTRAINT;
+
  }
+
  if( (idxNum & 3)==3 ){
+
    /* Both start= and stop= boundaries are available.  This is the 
+
    ** the preferred case */
+
    pIdxInfo->estimatedCost = (double)(2 - ((idxNum&4)!=0));
+
    pIdxInfo->estimatedRows = 1000;
+
    if( pIdxInfo->nOrderBy>=1 && pIdxInfo->aOrderBy[0].iColumn==0 ){
+
      if( pIdxInfo->aOrderBy[0].desc ){
+
        idxNum |= 8;
+
      }else{
+
        idxNum |= 16;
+
      }
+
      pIdxInfo->orderByConsumed = 1;
+
    }
+
  }else{
+
    /* If either boundary is missing, we have to generate a huge span
+
    ** of numbers.  Make this case very expensive so that the query
+
    ** planner will work hard to avoid it. */
+
    pIdxInfo->estimatedRows = 2147483647;
+
  }
+
  pIdxInfo->idxNum = idxNum;
+
  return SQLITE_OK;
}

/*
-
** Unlock an apnd-file.
+
** This following structure defines all the methods for the 
+
** generate_series virtual table.
*/
-
static int apndUnlock(sqlite3_file *pFile, int eLock){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xUnlock(pFile, eLock);
-
}
+
static sqlite3_module seriesModule = {
+
  0,                         /* iVersion */
+
  0,                         /* xCreate */
+
  seriesConnect,             /* xConnect */
+
  seriesBestIndex,           /* xBestIndex */
+
  seriesDisconnect,          /* xDisconnect */
+
  0,                         /* xDestroy */
+
  seriesOpen,                /* xOpen - open a cursor */
+
  seriesClose,               /* xClose - close a cursor */
+
  seriesFilter,              /* xFilter - configure scan constraints */
+
  seriesNext,                /* xNext - advance a cursor */
+
  seriesEof,                 /* xEof - check for end of scan */
+
  seriesColumn,              /* xColumn - read data */
+
  seriesRowid,               /* xRowid - read data */
+
  0,                         /* xUpdate */
+
  0,                         /* xBegin */
+
  0,                         /* xSync */
+
  0,                         /* xCommit */
+
  0,                         /* xRollback */
+
  0,                         /* xFindMethod */
+
  0,                         /* xRename */
+
  0,                         /* xSavepoint */
+
  0,                         /* xRelease */
+
  0,                         /* xRollbackTo */
+
  0                          /* xShadowName */
+
};

-
/*
-
** Check if another file-handle holds a RESERVED lock on an apnd-file.
-
*/
-
static int apndCheckReservedLock(sqlite3_file *pFile, int *pResOut){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xCheckReservedLock(pFile, pResOut);
-
}
+
#endif /* SQLITE_OMIT_VIRTUALTABLE */

-
/*
-
** File control method. For custom operations on an apnd-file.
-
*/
-
static int apndFileControl(sqlite3_file *pFile, int op, void *pArg){
-
  ApndFile *paf = (ApndFile *)pFile;
-
  int rc;
-
  pFile = ORIGFILE(pFile);
-
  if( op==SQLITE_FCNTL_SIZE_HINT ) *(sqlite3_int64*)pArg += paf->iPgOne;
-
  rc = pFile->pMethods->xFileControl(pFile, op, pArg);
-
  if( rc==SQLITE_OK && op==SQLITE_FCNTL_VFSNAME ){
-
    *(char**)pArg = sqlite3_mprintf("apnd(%lld)/%z", paf->iPgOne,*(char**)pArg);
+
#ifdef _WIN32
+

+
#endif
+
int sqlite3_series_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
+
){
+
  int rc = SQLITE_OK;
+
  SQLITE_EXTENSION_INIT2(pApi);
+
#ifndef SQLITE_OMIT_VIRTUALTABLE
+
  if( sqlite3_libversion_number()<3008012 && pzErrMsg!=0 ){
+
    *pzErrMsg = sqlite3_mprintf(
+
        "generate_series() requires SQLite 3.8.12 or later");
+
    return SQLITE_ERROR;
  }
+
  rc = sqlite3_create_module(db, "generate_series", &seriesModule, 0);
+
#endif
  return rc;
}

+
/************************* End ../ext/misc/series.c ********************/
+
/************************* Begin ../ext/misc/regexp.c ******************/
/*
-
** Return the sector-size in bytes for an apnd-file.
+
** 2012-11-13
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** The code in this file implements a compact but reasonably
+
** efficient regular-expression matcher for posix extended regular
+
** expressions against UTF8 text.
+
**
+
** This file is an SQLite extension.  It registers a single function
+
** named "regexp(A,B)" where A is the regular expression and B is the
+
** string to be matched.  By registering this function, SQLite will also
+
** then implement the "B regexp A" operator.  Note that with the function
+
** the regular expression comes first, but with the operator it comes
+
** second.
+
**
+
**  The following regular expression syntax is supported:
+
**
+
**     X*      zero or more occurrences of X
+
**     X+      one or more occurrences of X
+
**     X?      zero or one occurrences of X
+
**     X{p,q}  between p and q occurrences of X
+
**     (X)     match X
+
**     X|Y     X or Y
+
**     ^X      X occurring at the beginning of the string
+
**     X$      X occurring at the end of the string
+
**     .       Match any single character
+
**     \c      Character c where c is one of \{}()[]|*+?.
+
**     \c      C-language escapes for c in afnrtv.  ex: \t or \n
+
**     \uXXXX  Where XXXX is exactly 4 hex digits, unicode value XXXX
+
**     \xXX    Where XX is exactly 2 hex digits, unicode value XX
+
**     [abc]   Any single character from the set abc
+
**     [^abc]  Any single character not in the set abc
+
**     [a-z]   Any single character in the range a-z
+
**     [^a-z]  Any single character not in the range a-z
+
**     \b      Word boundary
+
**     \w      Word character.  [A-Za-z0-9_]
+
**     \W      Non-word character
+
**     \d      Digit
+
**     \D      Non-digit
+
**     \s      Whitespace character
+
**     \S      Non-whitespace character
+
**
+
** A nondeterministic finite automaton (NFA) is used for matching, so the
+
** performance is bounded by O(N*M) where N is the size of the regular
+
** expression and M is the size of the input string.  The matcher never
+
** exhibits exponential behavior.  Note that the X{p,q} operator expands
+
** to p copies of X following by q-p copies of X? and that the size of the
+
** regular expression in the O(N*M) performance bound is computed after
+
** this expansion.
*/
-
static int apndSectorSize(sqlite3_file *pFile){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xSectorSize(pFile);
-
}
+
#include <string.h>
+
#include <stdlib.h>
+
/* #include "sqlite3ext.h" */
+
SQLITE_EXTENSION_INIT1

/*
-
** Return the device characteristic flags supported by an apnd-file.
+
** The following #defines change the names of some functions implemented in
+
** this file to prevent name collisions with C-library functions of the
+
** same name.
*/
-
static int apndDeviceCharacteristics(sqlite3_file *pFile){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xDeviceCharacteristics(pFile);
-
}
+
#define re_match   sqlite3re_match
+
#define re_compile sqlite3re_compile
+
#define re_free    sqlite3re_free

-
/* Create a shared memory file mapping */
-
static int apndShmMap(
-
  sqlite3_file *pFile,
-
  int iPg,
-
  int pgsz,
-
  int bExtend,
-
  void volatile **pp
-
){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xShmMap(pFile,iPg,pgsz,bExtend,pp);
-
}
+
/* The end-of-input character */
+
#define RE_EOF            0    /* End of input */
+
#define RE_START  0xfffffff    /* Start of input - larger than an UTF-8 */

-
/* Perform locking on a shared-memory segment */
-
static int apndShmLock(sqlite3_file *pFile, int offset, int n, int flags){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xShmLock(pFile,offset,n,flags);
-
}
+
/* The NFA is implemented as sequence of opcodes taken from the following
+
** set.  Each opcode has a single integer argument.
+
*/
+
#define RE_OP_MATCH       1    /* Match the one character in the argument */
+
#define RE_OP_ANY         2    /* Match any one character.  (Implements ".") */
+
#define RE_OP_ANYSTAR     3    /* Special optimized version of .* */
+
#define RE_OP_FORK        4    /* Continue to both next and opcode at iArg */
+
#define RE_OP_GOTO        5    /* Jump to opcode at iArg */
+
#define RE_OP_ACCEPT      6    /* Halt and indicate a successful match */
+
#define RE_OP_CC_INC      7    /* Beginning of a [...] character class */
+
#define RE_OP_CC_EXC      8    /* Beginning of a [^...] character class */
+
#define RE_OP_CC_VALUE    9    /* Single value in a character class */
+
#define RE_OP_CC_RANGE   10    /* Range of values in a character class */
+
#define RE_OP_WORD       11    /* Perl word character [A-Za-z0-9_] */
+
#define RE_OP_NOTWORD    12    /* Not a perl word character */
+
#define RE_OP_DIGIT      13    /* digit:  [0-9] */
+
#define RE_OP_NOTDIGIT   14    /* Not a digit */
+
#define RE_OP_SPACE      15    /* space:  [ \t\n\r\v\f] */
+
#define RE_OP_NOTSPACE   16    /* Not a digit */
+
#define RE_OP_BOUNDARY   17    /* Boundary between word and non-word */
+
#define RE_OP_ATSTART    18    /* Currently at the start of the string */
+

+
#if defined(SQLITE_DEBUG)
+
/* Opcode names used for symbolic debugging */
+
static const char *ReOpName[] = {
+
  "EOF",
+
  "MATCH",
+
  "ANY",
+
  "ANYSTAR",
+
  "FORK",
+
  "GOTO",
+
  "ACCEPT",
+
  "CC_INC",
+
  "CC_EXC",
+
  "CC_VALUE",
+
  "CC_RANGE",
+
  "WORD",
+
  "NOTWORD",
+
  "DIGIT",
+
  "NOTDIGIT",
+
  "SPACE",
+
  "NOTSPACE",
+
  "BOUNDARY",
+
  "ATSTART",
+
};
+
#endif /* SQLITE_DEBUG */

-
/* Memory barrier operation on shared memory */
-
static void apndShmBarrier(sqlite3_file *pFile){
-
  pFile = ORIGFILE(pFile);
-
  pFile->pMethods->xShmBarrier(pFile);
-
}

-
/* Unmap a shared memory segment */
-
static int apndShmUnmap(sqlite3_file *pFile, int deleteFlag){
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xShmUnmap(pFile,deleteFlag);
-
}
+
/* Each opcode is a "state" in the NFA */
+
typedef unsigned short ReStateNumber;

-
/* Fetch a page of a memory-mapped file */
-
static int apndFetch(
-
  sqlite3_file *pFile,
-
  sqlite3_int64 iOfst,
-
  int iAmt,
-
  void **pp
-
){
-
  ApndFile *p = (ApndFile *)pFile;
-
  if( p->iMark < 0 || iOfst+iAmt > p->iMark ){
-
    return SQLITE_IOERR; /* Cannot read what is not yet there. */
-
  }
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xFetch(pFile, iOfst+p->iPgOne, iAmt, pp);
-
}
+
/* Because this is an NFA and not a DFA, multiple states can be active at
+
** once.  An instance of the following object records all active states in
+
** the NFA.  The implementation is optimized for the common case where the
+
** number of actives states is small.
+
*/
+
typedef struct ReStateSet {
+
  unsigned nState;            /* Number of current states */
+
  ReStateNumber *aState;      /* Current states */
+
} ReStateSet;

-
/* Release a memory-mapped page */
-
static int apndUnfetch(sqlite3_file *pFile, sqlite3_int64 iOfst, void *pPage){
-
  ApndFile *p = (ApndFile *)pFile;
-
  pFile = ORIGFILE(pFile);
-
  return pFile->pMethods->xUnfetch(pFile, iOfst+p->iPgOne, pPage);
-
}
+
/* An input string read one character at a time.
+
*/
+
typedef struct ReInput ReInput;
+
struct ReInput {
+
  const unsigned char *z;  /* All text */
+
  int i;                   /* Next byte to read */
+
  int mx;                  /* EOF when i>=mx */
+
};

-
/*
-
** Try to read the append-mark off the end of a file.  Return the
-
** start of the appended database if the append-mark is present.
-
** If there is no valid append-mark, return -1;
-
**
-
** An append-mark is only valid if the NNNNNNNN start-of-database offset
-
** indicates that the appended database contains at least one page.  The
-
** start-of-database value must be a multiple of 512.
+
/* A compiled NFA (or an NFA that is in the process of being compiled) is
+
** an instance of the following object.
*/
-
static sqlite3_int64 apndReadMark(sqlite3_int64 sz, sqlite3_file *pFile){
-
  int rc, i;
-
  sqlite3_int64 iMark;
-
  int msbs = 8 * (APND_MARK_FOS_SZ-1);
-
  unsigned char a[APND_MARK_SIZE];
+
typedef struct ReCompiled ReCompiled;
+
struct ReCompiled {
+
  ReInput sIn;                /* Regular expression text */
+
  const char *zErr;           /* Error message to return */
+
  char *aOp;                  /* Operators for the virtual machine */
+
  int *aArg;                  /* Arguments to each operator */
+
  unsigned (*xNextChar)(ReInput*);  /* Next character function */
+
  unsigned char zInit[12];    /* Initial text to match */
+
  int nInit;                  /* Number of bytes in zInit */
+
  unsigned nState;            /* Number of entries in aOp[] and aArg[] */
+
  unsigned nAlloc;            /* Slots allocated for aOp[] and aArg[] */
+
};

-
  if( APND_MARK_SIZE!=(sz & 0x1ff) ) return -1;
-
  rc = pFile->pMethods->xRead(pFile, a, APND_MARK_SIZE, sz-APND_MARK_SIZE);
-
  if( rc ) return -1;
-
  if( memcmp(a, APND_MARK_PREFIX, APND_MARK_PREFIX_SZ)!=0 ) return -1;
-
  iMark = ((sqlite3_int64)(a[APND_MARK_PREFIX_SZ] & 0x7f)) << msbs;
-
  for(i=1; i<8; i++){
-
    msbs -= 8;
-
    iMark |= (sqlite3_int64)a[APND_MARK_PREFIX_SZ+i]<<msbs;
-
  }
-
  if( iMark > (sz - APND_MARK_SIZE - 512) ) return -1;
-
  if( iMark & 0x1ff ) return -1;
-
  return iMark;
+
/* Add a state to the given state set if it is not already there */
+
static void re_add_state(ReStateSet *pSet, int newState){
+
  unsigned i;
+
  for(i=0; i<pSet->nState; i++) if( pSet->aState[i]==newState ) return;
+
  pSet->aState[pSet->nState++] = (ReStateNumber)newState;
}

-
static const char apvfsSqliteHdr[] = "SQLite format 3";
-
/*
-
** Check to see if the file is an appendvfs SQLite database file.
-
** Return true iff it is such. Parameter sz is the file's size.
+
/* Extract the next unicode character from *pzIn and return it.  Advance
+
** *pzIn to the first byte past the end of the character returned.  To
+
** be clear:  this routine converts utf8 to unicode.  This routine is 
+
** optimized for the common case where the next character is a single byte.
*/
-
static int apndIsAppendvfsDatabase(sqlite3_int64 sz, sqlite3_file *pFile){
-
  int rc;
-
  char zHdr[16];
-
  sqlite3_int64 iMark = apndReadMark(sz, pFile);
-
  if( iMark>=0 ){
-
    /* If file has the correct end-marker, the expected odd size, and the
-
    ** SQLite DB type marker where the end-marker puts it, then it
-
    ** is an appendvfs database.
-
    */
-
    rc = pFile->pMethods->xRead(pFile, zHdr, sizeof(zHdr), iMark);
-
    if( SQLITE_OK==rc
-
     && memcmp(zHdr, apvfsSqliteHdr, sizeof(zHdr))==0
-
     && (sz & 0x1ff) == APND_MARK_SIZE
-
     && sz>=512+APND_MARK_SIZE
-
    ){
-
      return 1; /* It's an appendvfs database */
+
static unsigned re_next_char(ReInput *p){
+
  unsigned c;
+
  if( p->i>=p->mx ) return 0;
+
  c = p->z[p->i++];
+
  if( c>=0x80 ){
+
    if( (c&0xe0)==0xc0 && p->i<p->mx && (p->z[p->i]&0xc0)==0x80 ){
+
      c = (c&0x1f)<<6 | (p->z[p->i++]&0x3f);
+
      if( c<0x80 ) c = 0xfffd;
+
    }else if( (c&0xf0)==0xe0 && p->i+1<p->mx && (p->z[p->i]&0xc0)==0x80
+
           && (p->z[p->i+1]&0xc0)==0x80 ){
+
      c = (c&0x0f)<<12 | ((p->z[p->i]&0x3f)<<6) | (p->z[p->i+1]&0x3f);
+
      p->i += 2;
+
      if( c<=0x7ff || (c>=0xd800 && c<=0xdfff) ) c = 0xfffd;
+
    }else if( (c&0xf8)==0xf0 && p->i+3<p->mx && (p->z[p->i]&0xc0)==0x80
+
           && (p->z[p->i+1]&0xc0)==0x80 && (p->z[p->i+2]&0xc0)==0x80 ){
+
      c = (c&0x07)<<18 | ((p->z[p->i]&0x3f)<<12) | ((p->z[p->i+1]&0x3f)<<6)
+
                       | (p->z[p->i+2]&0x3f);
+
      p->i += 3;
+
      if( c<=0xffff || c>0x10ffff ) c = 0xfffd;
+
    }else{
+
      c = 0xfffd;
    }
  }
-
  return 0;
+
  return c;
}
-

-
/*
-
** Check to see if the file is an ordinary SQLite database file.
-
** Return true iff so. Parameter sz is the file's size.
-
*/
-
static int apndIsOrdinaryDatabaseFile(sqlite3_int64 sz, sqlite3_file *pFile){
-
  char zHdr[16];
-
  if( apndIsAppendvfsDatabase(sz, pFile) /* rule 2 */
-
   || (sz & 0x1ff) != 0
-
   || SQLITE_OK!=pFile->pMethods->xRead(pFile, zHdr, sizeof(zHdr), 0)
-
   || memcmp(zHdr, apvfsSqliteHdr, sizeof(zHdr))!=0
-
  ){
-
    return 0;
-
  }else{
-
    return 1;
-
  }
+
static unsigned re_next_char_nocase(ReInput *p){
+
  unsigned c = re_next_char(p);
+
  if( c>='A' && c<='Z' ) c += 'a' - 'A';
+
  return c;
}

-
/*
-
** Open an apnd file handle.
-
*/
-
static int apndOpen(
-
  sqlite3_vfs *pApndVfs,
-
  const char *zName,
-
  sqlite3_file *pFile,
-
  int flags,
-
  int *pOutFlags
-
){
-
  ApndFile *pApndFile = (ApndFile*)pFile;
-
  sqlite3_file *pBaseFile = ORIGFILE(pFile);
-
  sqlite3_vfs *pBaseVfs = ORIGVFS(pApndVfs);
-
  int rc;
-
  sqlite3_int64 sz = 0;
-
  if( (flags & SQLITE_OPEN_MAIN_DB)==0 ){
-
    /* The appendvfs is not to be used for transient or temporary databases.
-
    ** Just use the base VFS open to initialize the given file object and
-
    ** open the underlying file. (Appendvfs is then unused for this file.)
-
    */
-
    return pBaseVfs->xOpen(pBaseVfs, zName, pFile, flags, pOutFlags);
-
  }
-
  memset(pApndFile, 0, sizeof(ApndFile));
-
  pFile->pMethods = &apnd_io_methods;
-
  pApndFile->iMark = -1;    /* Append mark not yet written */
+
/* Return true if c is a perl "word" character:  [A-Za-z0-9_] */
+
static int re_word_char(int c){
+
  return (c>='0' && c<='9') || (c>='a' && c<='z')
+
      || (c>='A' && c<='Z') || c=='_';
+
}

-
  rc = pBaseVfs->xOpen(pBaseVfs, zName, pBaseFile, flags, pOutFlags);
-
  if( rc==SQLITE_OK ){
-
    rc = pBaseFile->pMethods->xFileSize(pBaseFile, &sz);
-
    if( rc ){
-
      pBaseFile->pMethods->xClose(pBaseFile);
-
    }
-
  }
-
  if( rc ){
-
    pFile->pMethods = 0;
-
    return rc;
-
  }
-
  if( apndIsOrdinaryDatabaseFile(sz, pBaseFile) ){
-
    /* The file being opened appears to be just an ordinary DB. Copy
-
    ** the base dispatch-table so this instance mimics the base VFS. 
-
    */
-
    memmove(pApndFile, pBaseFile, pBaseVfs->szOsFile);
-
    return SQLITE_OK;
-
  }
-
  pApndFile->iPgOne = apndReadMark(sz, pFile);
-
  if( pApndFile->iPgOne>=0 ){
-
    pApndFile->iMark = sz - APND_MARK_SIZE; /* Append mark found */
-
    return SQLITE_OK;
-
  }
-
  if( (flags & SQLITE_OPEN_CREATE)==0 ){
-
    pBaseFile->pMethods->xClose(pBaseFile);
-
    rc = SQLITE_CANTOPEN;
-
    pFile->pMethods = 0;
-
  }else{
-
    /* Round newly added appendvfs location to #define'd page boundary. 
-
    ** Note that nothing has yet been written to the underlying file.
-
    ** The append mark will be written along with first content write.
-
    ** Until then, paf->iMark value indicates it is not yet written.
-
    */
-
    pApndFile->iPgOne = APND_START_ROUNDUP(sz);
-
  }
-
  return rc;
+
/* Return true if c is a "digit" character:  [0-9] */
+
static int re_digit_char(int c){
+
  return (c>='0' && c<='9');
}

-
/*
-
** Delete an apnd file.
-
** For an appendvfs, this could mean delete the appendvfs portion,
-
** leaving the appendee as it was before it gained an appendvfs.
-
** For now, this code deletes the underlying file too.
-
*/
-
static int apndDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){
-
  return ORIGVFS(pVfs)->xDelete(ORIGVFS(pVfs), zPath, dirSync);
+
/* Return true if c is a perl "space" character:  [ \t\r\n\v\f] */
+
static int re_space_char(int c){
+
  return c==' ' || c=='\t' || c=='\n' || c=='\r' || c=='\v' || c=='\f';
}

-
/*
-
** All other VFS methods are pass-thrus.
+
/* Run a compiled regular expression on the zero-terminated input
+
** string zIn[].  Return true on a match and false if there is no match.
*/
-
static int apndAccess(
-
  sqlite3_vfs *pVfs, 
-
  const char *zPath, 
-
  int flags, 
-
  int *pResOut
-
){
-
  return ORIGVFS(pVfs)->xAccess(ORIGVFS(pVfs), zPath, flags, pResOut);
-
}
-
static int apndFullPathname(
-
  sqlite3_vfs *pVfs, 
-
  const char *zPath, 
-
  int nOut, 
-
  char *zOut
+
static int re_match(ReCompiled *pRe, const unsigned char *zIn, int nIn){
+
  ReStateSet aStateSet[2], *pThis, *pNext;
+
  ReStateNumber aSpace[100];
+
  ReStateNumber *pToFree;
+
  unsigned int i = 0;
+
  unsigned int iSwap = 0;
+
  int c = RE_START;
+
  int cPrev = 0;
+
  int rc = 0;
+
  ReInput in;
+

+
  in.z = zIn;
+
  in.i = 0;
+
  in.mx = nIn>=0 ? nIn : (int)strlen((char const*)zIn);
+

+
  /* Look for the initial prefix match, if there is one. */
+
  if( pRe->nInit ){
+
    unsigned char x = pRe->zInit[0];
+
    while( in.i+pRe->nInit<=in.mx 
+
     && (zIn[in.i]!=x ||
+
         strncmp((const char*)zIn+in.i, (const char*)pRe->zInit, pRe->nInit)!=0)
+
    ){
+
      in.i++;
+
    }
+
    if( in.i+pRe->nInit>in.mx ) return 0;
+
    c = RE_START-1;
+
  }
+

+
  if( pRe->nState<=(sizeof(aSpace)/(sizeof(aSpace[0])*2)) ){
+
    pToFree = 0;
+
    aStateSet[0].aState = aSpace;
+
  }else{
+
    pToFree = sqlite3_malloc64( sizeof(ReStateNumber)*2*pRe->nState );
+
    if( pToFree==0 ) return -1;
+
    aStateSet[0].aState = pToFree;
+
  }
+
  aStateSet[1].aState = &aStateSet[0].aState[pRe->nState];
+
  pNext = &aStateSet[1];
+
  pNext->nState = 0;
+
  re_add_state(pNext, 0);
+
  while( c!=RE_EOF && pNext->nState>0 ){
+
    cPrev = c;
+
    c = pRe->xNextChar(&in);
+
    pThis = pNext;
+
    pNext = &aStateSet[iSwap];
+
    iSwap = 1 - iSwap;
+
    pNext->nState = 0;
+
    for(i=0; i<pThis->nState; i++){
+
      int x = pThis->aState[i];
+
      switch( pRe->aOp[x] ){
+
        case RE_OP_MATCH: {
+
          if( pRe->aArg[x]==c ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_ATSTART: {
+
          if( cPrev==RE_START ) re_add_state(pThis, x+1);
+
          break;
+
        }
+
        case RE_OP_ANY: {
+
          if( c!=0 ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_WORD: {
+
          if( re_word_char(c) ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_NOTWORD: {
+
          if( !re_word_char(c) && c!=0 ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_DIGIT: {
+
          if( re_digit_char(c) ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_NOTDIGIT: {
+
          if( !re_digit_char(c) && c!=0 ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_SPACE: {
+
          if( re_space_char(c) ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_NOTSPACE: {
+
          if( !re_space_char(c) && c!=0 ) re_add_state(pNext, x+1);
+
          break;
+
        }
+
        case RE_OP_BOUNDARY: {
+
          if( re_word_char(c)!=re_word_char(cPrev) ) re_add_state(pThis, x+1);
+
          break;
+
        }
+
        case RE_OP_ANYSTAR: {
+
          re_add_state(pNext, x);
+
          re_add_state(pThis, x+1);
+
          break;
+
        }
+
        case RE_OP_FORK: {
+
          re_add_state(pThis, x+pRe->aArg[x]);
+
          re_add_state(pThis, x+1);
+
          break;
+
        }
+
        case RE_OP_GOTO: {
+
          re_add_state(pThis, x+pRe->aArg[x]);
+
          break;
+
        }
+
        case RE_OP_ACCEPT: {
+
          rc = 1;
+
          goto re_match_end;
+
        }
+
        case RE_OP_CC_EXC: {
+
          if( c==0 ) break;
+
          /* fall-through */ goto re_op_cc_inc;
+
        }
+
        case RE_OP_CC_INC: re_op_cc_inc: {
+
          int j = 1;
+
          int n = pRe->aArg[x];
+
          int hit = 0;
+
          for(j=1; j>0 && j<n; j++){
+
            if( pRe->aOp[x+j]==RE_OP_CC_VALUE ){
+
              if( pRe->aArg[x+j]==c ){
+
                hit = 1;
+
                j = -1;
+
              }
+
            }else{
+
              if( pRe->aArg[x+j]<=c && pRe->aArg[x+j+1]>=c ){
+
                hit = 1;
+
                j = -1;
+
              }else{
+
                j++;
+
              }
+
            }
+
          }
+
          if( pRe->aOp[x]==RE_OP_CC_EXC ) hit = !hit;
+
          if( hit ) re_add_state(pNext, x+n);
+
          break;
+
        }
+
      }
+
    }
+
  }
+
  for(i=0; i<pNext->nState; i++){
+
    int x = pNext->aState[i];
+
    while( pRe->aOp[x]==RE_OP_GOTO ) x += pRe->aArg[x];
+
    if( pRe->aOp[x]==RE_OP_ACCEPT ){ rc = 1; break; }
+
  }
+
re_match_end:
+
  sqlite3_free(pToFree);
+
  return rc;
+
}
+

+
/* Resize the opcode and argument arrays for an RE under construction.
+
*/
+
static int re_resize(ReCompiled *p, int N){
+
  char *aOp;
+
  int *aArg;
+
  aOp = sqlite3_realloc64(p->aOp, N*sizeof(p->aOp[0]));
+
  if( aOp==0 ) return 1;
+
  p->aOp = aOp;
+
  aArg = sqlite3_realloc64(p->aArg, N*sizeof(p->aArg[0]));
+
  if( aArg==0 ) return 1;
+
  p->aArg = aArg;
+
  p->nAlloc = N;
+
  return 0;
+
}
+

+
/* Insert a new opcode and argument into an RE under construction.  The
+
** insertion point is just prior to existing opcode iBefore.
+
*/
+
static int re_insert(ReCompiled *p, int iBefore, int op, int arg){
+
  int i;
+
  if( p->nAlloc<=p->nState && re_resize(p, p->nAlloc*2) ) return 0;
+
  for(i=p->nState; i>iBefore; i--){
+
    p->aOp[i] = p->aOp[i-1];
+
    p->aArg[i] = p->aArg[i-1];
+
  }
+
  p->nState++;
+
  p->aOp[iBefore] = (char)op;
+
  p->aArg[iBefore] = arg;
+
  return iBefore;
+
}
+

+
/* Append a new opcode and argument to the end of the RE under construction.
+
*/
+
static int re_append(ReCompiled *p, int op, int arg){
+
  return re_insert(p, p->nState, op, arg);
+
}
+

+
/* Make a copy of N opcodes starting at iStart onto the end of the RE
+
** under construction.
+
*/
+
static void re_copy(ReCompiled *p, int iStart, int N){
+
  if( p->nState+N>=p->nAlloc && re_resize(p, p->nAlloc*2+N) ) return;
+
  memcpy(&p->aOp[p->nState], &p->aOp[iStart], N*sizeof(p->aOp[0]));
+
  memcpy(&p->aArg[p->nState], &p->aArg[iStart], N*sizeof(p->aArg[0]));
+
  p->nState += N;
+
}
+

+
/* Return true if c is a hexadecimal digit character:  [0-9a-fA-F]
+
** If c is a hex digit, also set *pV = (*pV)*16 + valueof(c).  If
+
** c is not a hex digit *pV is unchanged.
+
*/
+
static int re_hex(int c, int *pV){
+
  if( c>='0' && c<='9' ){
+
    c -= '0';
+
  }else if( c>='a' && c<='f' ){
+
    c -= 'a' - 10;
+
  }else if( c>='A' && c<='F' ){
+
    c -= 'A' - 10;
+
  }else{
+
    return 0;
+
  }
+
  *pV = (*pV)*16 + (c & 0xff);
+
  return 1;
+
}
+

+
/* A backslash character has been seen, read the next character and
+
** return its interpretation.
+
*/
+
static unsigned re_esc_char(ReCompiled *p){
+
  static const char zEsc[] = "afnrtv\\()*.+?[$^{|}]";
+
  static const char zTrans[] = "\a\f\n\r\t\v";
+
  int i, v = 0;
+
  char c;
+
  if( p->sIn.i>=p->sIn.mx ) return 0;
+
  c = p->sIn.z[p->sIn.i];
+
  if( c=='u' && p->sIn.i+4<p->sIn.mx ){
+
    const unsigned char *zIn = p->sIn.z + p->sIn.i;
+
    if( re_hex(zIn[1],&v)
+
     && re_hex(zIn[2],&v)
+
     && re_hex(zIn[3],&v)
+
     && re_hex(zIn[4],&v)
+
    ){
+
      p->sIn.i += 5;
+
      return v;
+
    }
+
  }
+
  if( c=='x' && p->sIn.i+2<p->sIn.mx ){
+
    const unsigned char *zIn = p->sIn.z + p->sIn.i;
+
    if( re_hex(zIn[1],&v)
+
     && re_hex(zIn[2],&v)
+
    ){
+
      p->sIn.i += 3;
+
      return v;
+
    }
+
  }
+
  for(i=0; zEsc[i] && zEsc[i]!=c; i++){}
+
  if( zEsc[i] ){
+
    if( i<6 ) c = zTrans[i];
+
    p->sIn.i++;
+
  }else{
+
    p->zErr = "unknown \\ escape";
+
  }
+
  return c;
+
}
+

+
/* Forward declaration */
+
static const char *re_subcompile_string(ReCompiled*);
+

+
/* Peek at the next byte of input */
+
static unsigned char rePeek(ReCompiled *p){
+
  return p->sIn.i<p->sIn.mx ? p->sIn.z[p->sIn.i] : 0;
+
}
+

+
/* Compile RE text into a sequence of opcodes.  Continue up to the
+
** first unmatched ")" character, then return.  If an error is found,
+
** return a pointer to the error message string.
+
*/
+
static const char *re_subcompile_re(ReCompiled *p){
+
  const char *zErr;
+
  int iStart, iEnd, iGoto;
+
  iStart = p->nState;
+
  zErr = re_subcompile_string(p);
+
  if( zErr ) return zErr;
+
  while( rePeek(p)=='|' ){
+
    iEnd = p->nState;
+
    re_insert(p, iStart, RE_OP_FORK, iEnd + 2 - iStart);
+
    iGoto = re_append(p, RE_OP_GOTO, 0);
+
    p->sIn.i++;
+
    zErr = re_subcompile_string(p);
+
    if( zErr ) return zErr;
+
    p->aArg[iGoto] = p->nState - iGoto;
+
  }
+
  return 0;
+
}
+

+
/* Compile an element of regular expression text (anything that can be
+
** an operand to the "|" operator).  Return NULL on success or a pointer
+
** to the error message if there is a problem.
+
*/
+
static const char *re_subcompile_string(ReCompiled *p){
+
  int iPrev = -1;
+
  int iStart;
+
  unsigned c;
+
  const char *zErr;
+
  while( (c = p->xNextChar(&p->sIn))!=0 ){
+
    iStart = p->nState;
+
    switch( c ){
+
      case '|':
+
      case ')': {
+
        p->sIn.i--;
+
        return 0;
+
      }
+
      case '(': {
+
        zErr = re_subcompile_re(p);
+
        if( zErr ) return zErr;
+
        if( rePeek(p)!=')' ) return "unmatched '('";
+
        p->sIn.i++;
+
        break;
+
      }
+
      case '.': {
+
        if( rePeek(p)=='*' ){
+
          re_append(p, RE_OP_ANYSTAR, 0);
+
          p->sIn.i++;
+
        }else{
+
          re_append(p, RE_OP_ANY, 0);
+
        }
+
        break;
+
      }
+
      case '*': {
+
        if( iPrev<0 ) return "'*' without operand";
+
        re_insert(p, iPrev, RE_OP_GOTO, p->nState - iPrev + 1);
+
        re_append(p, RE_OP_FORK, iPrev - p->nState + 1);
+
        break;
+
      }
+
      case '+': {
+
        if( iPrev<0 ) return "'+' without operand";
+
        re_append(p, RE_OP_FORK, iPrev - p->nState);
+
        break;
+
      }
+
      case '?': {
+
        if( iPrev<0 ) return "'?' without operand";
+
        re_insert(p, iPrev, RE_OP_FORK, p->nState - iPrev+1);
+
        break;
+
      }
+
      case '$': {
+
        re_append(p, RE_OP_MATCH, RE_EOF);
+
        break;
+
      }
+
      case '^': {
+
        re_append(p, RE_OP_ATSTART, 0);
+
        break;
+
      }
+
      case '{': {
+
        int m = 0, n = 0;
+
        int sz, j;
+
        if( iPrev<0 ) return "'{m,n}' without operand";
+
        while( (c=rePeek(p))>='0' && c<='9' ){ m = m*10 + c - '0'; p->sIn.i++; }
+
        n = m;
+
        if( c==',' ){
+
          p->sIn.i++;
+
          n = 0;
+
          while( (c=rePeek(p))>='0' && c<='9' ){ n = n*10 + c-'0'; p->sIn.i++; }
+
        }
+
        if( c!='}' ) return "unmatched '{'";
+
        if( n>0 && n<m ) return "n less than m in '{m,n}'";
+
        p->sIn.i++;
+
        sz = p->nState - iPrev;
+
        if( m==0 ){
+
          if( n==0 ) return "both m and n are zero in '{m,n}'";
+
          re_insert(p, iPrev, RE_OP_FORK, sz+1);
+
          iPrev++;
+
          n--;
+
        }else{
+
          for(j=1; j<m; j++) re_copy(p, iPrev, sz);
+
        }
+
        for(j=m; j<n; j++){
+
          re_append(p, RE_OP_FORK, sz+1);
+
          re_copy(p, iPrev, sz);
+
        }
+
        if( n==0 && m>0 ){
+
          re_append(p, RE_OP_FORK, -sz);
+
        }
+
        break;
+
      }
+
      case '[': {
+
        int iFirst = p->nState;
+
        if( rePeek(p)=='^' ){
+
          re_append(p, RE_OP_CC_EXC, 0);
+
          p->sIn.i++;
+
        }else{
+
          re_append(p, RE_OP_CC_INC, 0);
+
        }
+
        while( (c = p->xNextChar(&p->sIn))!=0 ){
+
          if( c=='[' && rePeek(p)==':' ){
+
            return "POSIX character classes not supported";
+
          }
+
          if( c=='\\' ) c = re_esc_char(p);
+
          if( rePeek(p)=='-' ){
+
            re_append(p, RE_OP_CC_RANGE, c);
+
            p->sIn.i++;
+
            c = p->xNextChar(&p->sIn);
+
            if( c=='\\' ) c = re_esc_char(p);
+
            re_append(p, RE_OP_CC_RANGE, c);
+
          }else{
+
            re_append(p, RE_OP_CC_VALUE, c);
+
          }
+
          if( rePeek(p)==']' ){ p->sIn.i++; break; }
+
        }
+
        if( c==0 ) return "unclosed '['";
+
        p->aArg[iFirst] = p->nState - iFirst;
+
        break;
+
      }
+
      case '\\': {
+
        int specialOp = 0;
+
        switch( rePeek(p) ){
+
          case 'b': specialOp = RE_OP_BOUNDARY;   break;
+
          case 'd': specialOp = RE_OP_DIGIT;      break;
+
          case 'D': specialOp = RE_OP_NOTDIGIT;   break;
+
          case 's': specialOp = RE_OP_SPACE;      break;
+
          case 'S': specialOp = RE_OP_NOTSPACE;   break;
+
          case 'w': specialOp = RE_OP_WORD;       break;
+
          case 'W': specialOp = RE_OP_NOTWORD;    break;
+
        }
+
        if( specialOp ){
+
          p->sIn.i++;
+
          re_append(p, specialOp, 0);
+
        }else{
+
          c = re_esc_char(p);
+
          re_append(p, RE_OP_MATCH, c);
+
        }
+
        break;
+
      }
+
      default: {
+
        re_append(p, RE_OP_MATCH, c);
+
        break;
+
      }
+
    }
+
    iPrev = iStart;
+
  }
+
  return 0;
+
}
+

+
/* Free and reclaim all the memory used by a previously compiled
+
** regular expression.  Applications should invoke this routine once
+
** for every call to re_compile() to avoid memory leaks.
+
*/
+
static void re_free(ReCompiled *pRe){
+
  if( pRe ){
+
    sqlite3_free(pRe->aOp);
+
    sqlite3_free(pRe->aArg);
+
    sqlite3_free(pRe);
+
  }
+
}
+

+
/*
+
** Compile a textual regular expression in zIn[] into a compiled regular
+
** expression suitable for us by re_match() and return a pointer to the
+
** compiled regular expression in *ppRe.  Return NULL on success or an
+
** error message if something goes wrong.
+
*/
+
static const char *re_compile(ReCompiled **ppRe, const char *zIn, int noCase){
+
  ReCompiled *pRe;
+
  const char *zErr;
+
  int i, j;
+

+
  *ppRe = 0;
+
  pRe = sqlite3_malloc( sizeof(*pRe) );
+
  if( pRe==0 ){
+
    return "out of memory";
+
  }
+
  memset(pRe, 0, sizeof(*pRe));
+
  pRe->xNextChar = noCase ? re_next_char_nocase : re_next_char;
+
  if( re_resize(pRe, 30) ){
+
    re_free(pRe);
+
    return "out of memory";
+
  }
+
  if( zIn[0]=='^' ){
+
    zIn++;
+
  }else{
+
    re_append(pRe, RE_OP_ANYSTAR, 0);
+
  }
+
  pRe->sIn.z = (unsigned char*)zIn;
+
  pRe->sIn.i = 0;
+
  pRe->sIn.mx = (int)strlen(zIn);
+
  zErr = re_subcompile_re(pRe);
+
  if( zErr ){
+
    re_free(pRe);
+
    return zErr;
+
  }
+
  if( pRe->sIn.i>=pRe->sIn.mx ){
+
    re_append(pRe, RE_OP_ACCEPT, 0);
+
    *ppRe = pRe;
+
  }else{
+
    re_free(pRe);
+
    return "unrecognized character";
+
  }
+

+
  /* The following is a performance optimization.  If the regex begins with
+
  ** ".*" (if the input regex lacks an initial "^") and afterwards there are
+
  ** one or more matching characters, enter those matching characters into
+
  ** zInit[].  The re_match() routine can then search ahead in the input 
+
  ** string looking for the initial match without having to run the whole
+
  ** regex engine over the string.  Do not worry able trying to match
+
  ** unicode characters beyond plane 0 - those are very rare and this is
+
  ** just an optimization. */
+
  if( pRe->aOp[0]==RE_OP_ANYSTAR && !noCase ){
+
    for(j=0, i=1; j<(int)sizeof(pRe->zInit)-2 && pRe->aOp[i]==RE_OP_MATCH; i++){
+
      unsigned x = pRe->aArg[i];
+
      if( x<=127 ){
+
        pRe->zInit[j++] = (unsigned char)x;
+
      }else if( x<=0xfff ){
+
        pRe->zInit[j++] = (unsigned char)(0xc0 | (x>>6));
+
        pRe->zInit[j++] = 0x80 | (x&0x3f);
+
      }else if( x<=0xffff ){
+
        pRe->zInit[j++] = (unsigned char)(0xe0 | (x>>12));
+
        pRe->zInit[j++] = 0x80 | ((x>>6)&0x3f);
+
        pRe->zInit[j++] = 0x80 | (x&0x3f);
+
      }else{
+
        break;
+
      }
+
    }
+
    if( j>0 && pRe->zInit[j-1]==0 ) j--;
+
    pRe->nInit = j;
+
  }
+
  return pRe->zErr;
+
}
+

+
/*
+
** Implementation of the regexp() SQL function.  This function implements
+
** the build-in REGEXP operator.  The first argument to the function is the
+
** pattern and the second argument is the string.  So, the SQL statements:
+
**
+
**       A REGEXP B
+
**
+
** is implemented as regexp(B,A).
+
*/
+
static void re_sql_func(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  ReCompiled *pRe;          /* Compiled regular expression */
+
  const char *zPattern;     /* The regular expression */
+
  const unsigned char *zStr;/* String being searched */
+
  const char *zErr;         /* Compile error message */
+
  int setAux = 0;           /* True to invoke sqlite3_set_auxdata() */
+

+
  (void)argc;  /* Unused */
+
  pRe = sqlite3_get_auxdata(context, 0);
+
  if( pRe==0 ){
+
    zPattern = (const char*)sqlite3_value_text(argv[0]);
+
    if( zPattern==0 ) return;
+
    zErr = re_compile(&pRe, zPattern, sqlite3_user_data(context)!=0);
+
    if( zErr ){
+
      re_free(pRe);
+
      sqlite3_result_error(context, zErr, -1);
+
      return;
+
    }
+
    if( pRe==0 ){
+
      sqlite3_result_error_nomem(context);
+
      return;
+
    }
+
    setAux = 1;
+
  }
+
  zStr = (const unsigned char*)sqlite3_value_text(argv[1]);
+
  if( zStr!=0 ){
+
    sqlite3_result_int(context, re_match(pRe, zStr, -1));
+
  }
+
  if( setAux ){
+
    sqlite3_set_auxdata(context, 0, pRe, (void(*)(void*))re_free);
+
  }
+
}
+

+
#if defined(SQLITE_DEBUG)
+
/*
+
** This function is used for testing and debugging only.  It is only available
+
** if the SQLITE_DEBUG compile-time option is used.
+
**
+
** Compile a regular expression and then convert the compiled expression into
+
** text and return that text.
+
*/
+
static void re_bytecode_func(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  const char *zPattern;
+
  const char *zErr;
+
  ReCompiled *pRe;
+
  sqlite3_str *pStr;
+
  int i;
+
  int n;
+
  char *z;
+

+
  zPattern = (const char*)sqlite3_value_text(argv[0]);
+
  if( zPattern==0 ) return;
+
  zErr = re_compile(&pRe, zPattern, sqlite3_user_data(context)!=0);
+
  if( zErr ){
+
    re_free(pRe);
+
    sqlite3_result_error(context, zErr, -1);
+
    return;
+
  }
+
  if( pRe==0 ){
+
    sqlite3_result_error_nomem(context);
+
    return;
+
  }
+
  pStr = sqlite3_str_new(0);
+
  if( pStr==0 ) goto re_bytecode_func_err;
+
  if( pRe->nInit>0 ){
+
    sqlite3_str_appendf(pStr, "INIT     ");
+
    for(i=0; i<pRe->nInit; i++){
+
      sqlite3_str_appendf(pStr, "%02x", pRe->zInit[i]);
+
    }
+
    sqlite3_str_appendf(pStr, "\n");
+
  }
+
  for(i=0; (unsigned)i<pRe->nState; i++){
+
    sqlite3_str_appendf(pStr, "%-8s %4d\n",
+
         ReOpName[(unsigned char)pRe->aOp[i]], pRe->aArg[i]);
+
  }
+
  n = sqlite3_str_length(pStr);
+
  z = sqlite3_str_finish(pStr);
+
  if( n==0 ){
+
    sqlite3_free(z);
+
  }else{
+
    sqlite3_result_text(context, z, n-1, sqlite3_free);
+
  }
+

+
re_bytecode_func_err:
+
  re_free(pRe);
+
}
+

+
#endif /* SQLITE_DEBUG */
+

+

+
/*
+
** Invoke this routine to register the regexp() function with the
+
** SQLite database connection.
+
*/
+
#ifdef _WIN32
+

+
#endif
+
int sqlite3_regexp_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
+
){
+
  int rc = SQLITE_OK;
+
  SQLITE_EXTENSION_INIT2(pApi);
+
  (void)pzErrMsg;  /* Unused */
+
  rc = sqlite3_create_function(db, "regexp", 2, 
+
                            SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC,
+
                            0, re_sql_func, 0, 0);
+
  if( rc==SQLITE_OK ){
+
    /* The regexpi(PATTERN,STRING) function is a case-insensitive version
+
    ** of regexp(PATTERN,STRING). */
+
    rc = sqlite3_create_function(db, "regexpi", 2,
+
                            SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC,
+
                            (void*)db, re_sql_func, 0, 0);
+
#if defined(SQLITE_DEBUG)
+
    if( rc==SQLITE_OK ){
+
      rc = sqlite3_create_function(db, "regexp_bytecode", 1,
+
                            SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC,
+
                            0, re_bytecode_func, 0, 0);
+
    }
+
#endif /* SQLITE_DEBUG */
+
  }
+
  return rc;
+
}
+

+
/************************* End ../ext/misc/regexp.c ********************/
+
#ifndef SQLITE_SHELL_FIDDLE
+
/************************* Begin ../ext/misc/fileio.c ******************/
+
/*
+
** 2014-06-13
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** This SQLite extension implements SQL functions readfile() and
+
** writefile(), and eponymous virtual type "fsdir".
+
**
+
** WRITEFILE(FILE, DATA [, MODE [, MTIME]]):
+
**
+
**   If neither of the optional arguments is present, then this UDF
+
**   function writes blob DATA to file FILE. If successful, the number
+
**   of bytes written is returned. If an error occurs, NULL is returned.
+
**
+
**   If the first option argument - MODE - is present, then it must
+
**   be passed an integer value that corresponds to a POSIX mode
+
**   value (file type + permissions, as returned in the stat.st_mode
+
**   field by the stat() system call). Three types of files may
+
**   be written/created:
+
**
+
**     regular files:  (mode & 0170000)==0100000
+
**     symbolic links: (mode & 0170000)==0120000
+
**     directories:    (mode & 0170000)==0040000
+
**
+
**   For a directory, the DATA is ignored. For a symbolic link, it is
+
**   interpreted as text and used as the target of the link. For a
+
**   regular file, it is interpreted as a blob and written into the
+
**   named file. Regardless of the type of file, its permissions are
+
**   set to (mode & 0777) before returning.
+
**
+
**   If the optional MTIME argument is present, then it is interpreted
+
**   as an integer - the number of seconds since the unix epoch. The
+
**   modification-time of the target file is set to this value before
+
**   returning.
+
**
+
**   If three or more arguments are passed to this function and an
+
**   error is encountered, an exception is raised.
+
**
+
** READFILE(FILE):
+
**
+
**   Read and return the contents of file FILE (type blob) from disk.
+
**
+
** FSDIR:
+
**
+
**   Used as follows:
+
**
+
**     SELECT * FROM fsdir($path [, $dir]);
+
**
+
**   Parameter $path is an absolute or relative pathname. If the file that it
+
**   refers to does not exist, it is an error. If the path refers to a regular
+
**   file or symbolic link, it returns a single row. Or, if the path refers
+
**   to a directory, it returns one row for the directory, and one row for each
+
**   file within the hierarchy rooted at $path.
+
**
+
**   Each row has the following columns:
+
**
+
**     name:  Path to file or directory (text value).
+
**     mode:  Value of stat.st_mode for directory entry (an integer).
+
**     mtime: Value of stat.st_mtime for directory entry (an integer).
+
**     data:  For a regular file, a blob containing the file data. For a
+
**            symlink, a text value containing the text of the link. For a
+
**            directory, NULL.
+
**
+
**   If a non-NULL value is specified for the optional $dir parameter and
+
**   $path is a relative path, then $path is interpreted relative to $dir. 
+
**   And the paths returned in the "name" column of the table are also 
+
**   relative to directory $dir.
+
**
+
** Notes on building this extension for Windows:
+
**   Unless linked statically with the SQLite library, a preprocessor
+
**   symbol, FILEIO_WIN32_DLL, must be #define'd to create a stand-alone
+
**   DLL form of this extension for WIN32. See its use below for details.
+
*/
+
/* #include "sqlite3ext.h" */
+
SQLITE_EXTENSION_INIT1
+
#include <stdio.h>
+
#include <string.h>
+
#include <assert.h>
+

+
#include <sys/types.h>
+
#include <sys/stat.h>
+
#include <fcntl.h>
+
#if !defined(_WIN32) && !defined(WIN32)
+
#  include <unistd.h>
+
#  include <dirent.h>
+
#  include <utime.h>
+
#  include <sys/time.h>
+
#else
+
#  include "windows.h"
+
#  include <io.h>
+
#  include <direct.h>
+
/* #  include "test_windirent.h" */
+
#  define dirent DIRENT
+
#  ifndef chmod
+
#    define chmod _chmod
+
#  endif
+
#  ifndef stat
+
#    define stat _stat
+
#  endif
+
#  define mkdir(path,mode) _mkdir(path)
+
#  define lstat(path,buf) stat(path,buf)
+
#endif
+
#include <time.h>
+
#include <errno.h>
+

+

+
/*
+
** Structure of the fsdir() table-valued function
+
*/
+
                 /*    0    1    2     3    4           5             */
+
#define FSDIR_SCHEMA "(name,mode,mtime,data,path HIDDEN,dir HIDDEN)"
+
#define FSDIR_COLUMN_NAME     0     /* Name of the file */
+
#define FSDIR_COLUMN_MODE     1     /* Access mode */
+
#define FSDIR_COLUMN_MTIME    2     /* Last modification time */
+
#define FSDIR_COLUMN_DATA     3     /* File content */
+
#define FSDIR_COLUMN_PATH     4     /* Path to top of search */
+
#define FSDIR_COLUMN_DIR      5     /* Path is relative to this directory */
+

+

+
/*
+
** Set the result stored by context ctx to a blob containing the 
+
** contents of file zName.  Or, leave the result unchanged (NULL)
+
** if the file does not exist or is unreadable.
+
**
+
** If the file exceeds the SQLite blob size limit, through an
+
** SQLITE_TOOBIG error.
+
**
+
** Throw an SQLITE_IOERR if there are difficulties pulling the file
+
** off of disk.
+
*/
+
static void readFileContents(sqlite3_context *ctx, const char *zName){
+
  FILE *in;
+
  sqlite3_int64 nIn;
+
  void *pBuf;
+
  sqlite3 *db;
+
  int mxBlob;
+

+
  in = fopen(zName, "rb");
+
  if( in==0 ){
+
    /* File does not exist or is unreadable. Leave the result set to NULL. */
+
    return;
+
  }
+
  fseek(in, 0, SEEK_END);
+
  nIn = ftell(in);
+
  rewind(in);
+
  db = sqlite3_context_db_handle(ctx);
+
  mxBlob = sqlite3_limit(db, SQLITE_LIMIT_LENGTH, -1);
+
  if( nIn>mxBlob ){
+
    sqlite3_result_error_code(ctx, SQLITE_TOOBIG);
+
    fclose(in);
+
    return;
+
  }
+
  pBuf = sqlite3_malloc64( nIn ? nIn : 1 );
+
  if( pBuf==0 ){
+
    sqlite3_result_error_nomem(ctx);
+
    fclose(in);
+
    return;
+
  }
+
  if( nIn==(sqlite3_int64)fread(pBuf, 1, (size_t)nIn, in) ){
+
    sqlite3_result_blob64(ctx, pBuf, nIn, sqlite3_free);
+
  }else{
+
    sqlite3_result_error_code(ctx, SQLITE_IOERR);
+
    sqlite3_free(pBuf);
+
  }
+
  fclose(in);
+
}
+

+
/*
+
** Implementation of the "readfile(X)" SQL function.  The entire content
+
** of the file named X is read and returned as a BLOB.  NULL is returned
+
** if the file does not exist or is unreadable.
+
*/
+
static void readfileFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  const char *zName;
+
  (void)(argc);  /* Unused parameter */
+
  zName = (const char*)sqlite3_value_text(argv[0]);
+
  if( zName==0 ) return;
+
  readFileContents(context, zName);
+
}
+

+
/*
+
** Set the error message contained in context ctx to the results of
+
** vprintf(zFmt, ...).
+
*/
+
static void ctxErrorMsg(sqlite3_context *ctx, const char *zFmt, ...){
+
  char *zMsg = 0;
+
  va_list ap;
+
  va_start(ap, zFmt);
+
  zMsg = sqlite3_vmprintf(zFmt, ap);
+
  sqlite3_result_error(ctx, zMsg, -1);
+
  sqlite3_free(zMsg);
+
  va_end(ap);
+
}
+

+
#if defined(_WIN32)
+
/*
+
** This function is designed to convert a Win32 FILETIME structure into the
+
** number of seconds since the Unix Epoch (1970-01-01 00:00:00 UTC).
+
*/
+
static sqlite3_uint64 fileTimeToUnixTime(
+
  LPFILETIME pFileTime
+
){
+
  SYSTEMTIME epochSystemTime;
+
  ULARGE_INTEGER epochIntervals;
+
  FILETIME epochFileTime;
+
  ULARGE_INTEGER fileIntervals;
+

+
  memset(&epochSystemTime, 0, sizeof(SYSTEMTIME));
+
  epochSystemTime.wYear = 1970;
+
  epochSystemTime.wMonth = 1;
+
  epochSystemTime.wDay = 1;
+
  SystemTimeToFileTime(&epochSystemTime, &epochFileTime);
+
  epochIntervals.LowPart = epochFileTime.dwLowDateTime;
+
  epochIntervals.HighPart = epochFileTime.dwHighDateTime;
+

+
  fileIntervals.LowPart = pFileTime->dwLowDateTime;
+
  fileIntervals.HighPart = pFileTime->dwHighDateTime;
+

+
  return (fileIntervals.QuadPart - epochIntervals.QuadPart) / 10000000;
+
}
+

+

+
#if defined(FILEIO_WIN32_DLL) && (defined(_WIN32) || defined(WIN32))
+
#  /* To allow a standalone DLL, use this next replacement function: */
+
#  undef sqlite3_win32_utf8_to_unicode
+
#  define sqlite3_win32_utf8_to_unicode utf8_to_utf16
+
#
+
LPWSTR utf8_to_utf16(const char *z){
+
  int nAllot = MultiByteToWideChar(CP_UTF8, 0, z, -1, NULL, 0);
+
  LPWSTR rv = sqlite3_malloc(nAllot * sizeof(WCHAR));
+
  if( rv!=0 && 0 < MultiByteToWideChar(CP_UTF8, 0, z, -1, rv, nAllot) )
+
    return rv;
+
  sqlite3_free(rv);
+
  return 0;
+
}
+
#endif
+

+
/*
+
** This function attempts to normalize the time values found in the stat()
+
** buffer to UTC.  This is necessary on Win32, where the runtime library
+
** appears to return these values as local times.
+
*/
+
static void statTimesToUtc(
+
  const char *zPath,
+
  struct stat *pStatBuf
+
){
+
  HANDLE hFindFile;
+
  WIN32_FIND_DATAW fd;
+
  LPWSTR zUnicodeName;
+
  extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);
+
  zUnicodeName = sqlite3_win32_utf8_to_unicode(zPath);
+
  if( zUnicodeName ){
+
    memset(&fd, 0, sizeof(WIN32_FIND_DATAW));
+
    hFindFile = FindFirstFileW(zUnicodeName, &fd);
+
    if( hFindFile!=NULL ){
+
      pStatBuf->st_ctime = (time_t)fileTimeToUnixTime(&fd.ftCreationTime);
+
      pStatBuf->st_atime = (time_t)fileTimeToUnixTime(&fd.ftLastAccessTime);
+
      pStatBuf->st_mtime = (time_t)fileTimeToUnixTime(&fd.ftLastWriteTime);
+
      FindClose(hFindFile);
+
    }
+
    sqlite3_free(zUnicodeName);
+
  }
+
}
+
#endif
+

+
/*
+
** This function is used in place of stat().  On Windows, special handling
+
** is required in order for the included time to be returned as UTC.  On all
+
** other systems, this function simply calls stat().
+
*/
+
static int fileStat(
+
  const char *zPath,
+
  struct stat *pStatBuf
+
){
+
#if defined(_WIN32)
+
  int rc = stat(zPath, pStatBuf);
+
  if( rc==0 ) statTimesToUtc(zPath, pStatBuf);
+
  return rc;
+
#else
+
  return stat(zPath, pStatBuf);
+
#endif
+
}
+

+
/*
+
** This function is used in place of lstat().  On Windows, special handling
+
** is required in order for the included time to be returned as UTC.  On all
+
** other systems, this function simply calls lstat().
+
*/
+
static int fileLinkStat(
+
  const char *zPath,
+
  struct stat *pStatBuf
+
){
+
#if defined(_WIN32)
+
  int rc = lstat(zPath, pStatBuf);
+
  if( rc==0 ) statTimesToUtc(zPath, pStatBuf);
+
  return rc;
+
#else
+
  return lstat(zPath, pStatBuf);
+
#endif
+
}
+

+
/*
+
** Argument zFile is the name of a file that will be created and/or written
+
** by SQL function writefile(). This function ensures that the directory
+
** zFile will be written to exists, creating it if required. The permissions
+
** for any path components created by this function are set in accordance
+
** with the current umask.
+
**
+
** If an OOM condition is encountered, SQLITE_NOMEM is returned. Otherwise,
+
** SQLITE_OK is returned if the directory is successfully created, or
+
** SQLITE_ERROR otherwise.
+
*/
+
static int makeDirectory(
+
  const char *zFile
+
){
+
  char *zCopy = sqlite3_mprintf("%s", zFile);
+
  int rc = SQLITE_OK;
+

+
  if( zCopy==0 ){
+
    rc = SQLITE_NOMEM;
+
  }else{
+
    int nCopy = (int)strlen(zCopy);
+
    int i = 1;
+

+
    while( rc==SQLITE_OK ){
+
      struct stat sStat;
+
      int rc2;
+

+
      for(; zCopy[i]!='/' && i<nCopy; i++);
+
      if( i==nCopy ) break;
+
      zCopy[i] = '\0';
+

+
      rc2 = fileStat(zCopy, &sStat);
+
      if( rc2!=0 ){
+
        if( mkdir(zCopy, 0777) ) rc = SQLITE_ERROR;
+
      }else{
+
        if( !S_ISDIR(sStat.st_mode) ) rc = SQLITE_ERROR;
+
      }
+
      zCopy[i] = '/';
+
      i++;
+
    }
+

+
    sqlite3_free(zCopy);
+
  }
+

+
  return rc;
+
}
+

+
/*
+
** This function does the work for the writefile() UDF. Refer to 
+
** header comments at the top of this file for details.
+
*/
+
static int writeFile(
+
  sqlite3_context *pCtx,          /* Context to return bytes written in */
+
  const char *zFile,              /* File to write */
+
  sqlite3_value *pData,           /* Data to write */
+
  mode_t mode,                    /* MODE parameter passed to writefile() */
+
  sqlite3_int64 mtime             /* MTIME parameter (or -1 to not set time) */
+
){
+
  if( zFile==0 ) return 1;
+
#if !defined(_WIN32) && !defined(WIN32)
+
  if( S_ISLNK(mode) ){
+
    const char *zTo = (const char*)sqlite3_value_text(pData);
+
    if( zTo==0 || symlink(zTo, zFile)<0 ) return 1;
+
  }else
+
#endif
+
  {
+
    if( S_ISDIR(mode) ){
+
      if( mkdir(zFile, mode) ){
+
        /* The mkdir() call to create the directory failed. This might not
+
        ** be an error though - if there is already a directory at the same
+
        ** path and either the permissions already match or can be changed
+
        ** to do so using chmod(), it is not an error.  */
+
        struct stat sStat;
+
        if( errno!=EEXIST
+
         || 0!=fileStat(zFile, &sStat)
+
         || !S_ISDIR(sStat.st_mode)
+
         || ((sStat.st_mode&0777)!=(mode&0777) && 0!=chmod(zFile, mode&0777))
+
        ){
+
          return 1;
+
        }
+
      }
+
    }else{
+
      sqlite3_int64 nWrite = 0;
+
      const char *z;
+
      int rc = 0;
+
      FILE *out = fopen(zFile, "wb");
+
      if( out==0 ) return 1;
+
      z = (const char*)sqlite3_value_blob(pData);
+
      if( z ){
+
        sqlite3_int64 n = fwrite(z, 1, sqlite3_value_bytes(pData), out);
+
        nWrite = sqlite3_value_bytes(pData);
+
        if( nWrite!=n ){
+
          rc = 1;
+
        }
+
      }
+
      fclose(out);
+
      if( rc==0 && mode && chmod(zFile, mode & 0777) ){
+
        rc = 1;
+
      }
+
      if( rc ) return 2;
+
      sqlite3_result_int64(pCtx, nWrite);
+
    }
+
  }
+

+
  if( mtime>=0 ){
+
#if defined(_WIN32)
+
#if !SQLITE_OS_WINRT
+
    /* Windows */
+
    FILETIME lastAccess;
+
    FILETIME lastWrite;
+
    SYSTEMTIME currentTime;
+
    LONGLONG intervals;
+
    HANDLE hFile;
+
    LPWSTR zUnicodeName;
+
    extern LPWSTR sqlite3_win32_utf8_to_unicode(const char*);
+

+
    GetSystemTime(&currentTime);
+
    SystemTimeToFileTime(&currentTime, &lastAccess);
+
    intervals = Int32x32To64(mtime, 10000000) + 116444736000000000;
+
    lastWrite.dwLowDateTime = (DWORD)intervals;
+
    lastWrite.dwHighDateTime = intervals >> 32;
+
    zUnicodeName = sqlite3_win32_utf8_to_unicode(zFile);
+
    if( zUnicodeName==0 ){
+
      return 1;
+
    }
+
    hFile = CreateFileW(
+
      zUnicodeName, FILE_WRITE_ATTRIBUTES, 0, NULL, OPEN_EXISTING,
+
      FILE_FLAG_BACKUP_SEMANTICS, NULL
+
    );
+
    sqlite3_free(zUnicodeName);
+
    if( hFile!=INVALID_HANDLE_VALUE ){
+
      BOOL bResult = SetFileTime(hFile, NULL, &lastAccess, &lastWrite);
+
      CloseHandle(hFile);
+
      return !bResult;
+
    }else{
+
      return 1;
+
    }
+
#endif
+
#elif defined(AT_FDCWD) && 0 /* utimensat() is not universally available */
+
    /* Recent unix */
+
    struct timespec times[2];
+
    times[0].tv_nsec = times[1].tv_nsec = 0;
+
    times[0].tv_sec = time(0);
+
    times[1].tv_sec = mtime;
+
    if( utimensat(AT_FDCWD, zFile, times, AT_SYMLINK_NOFOLLOW) ){
+
      return 1;
+
    }
+
#else
+
    /* Legacy unix */
+
    struct timeval times[2];
+
    times[0].tv_usec = times[1].tv_usec = 0;
+
    times[0].tv_sec = time(0);
+
    times[1].tv_sec = mtime;
+
    if( utimes(zFile, times) ){
+
      return 1;
+
    }
+
#endif
+
  }
+

+
  return 0;
+
}
+

+
/*
+
** Implementation of the "writefile(W,X[,Y[,Z]]])" SQL function.  
+
** Refer to header comments at the top of this file for details.
+
*/
+
static void writefileFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  const char *zFile;
+
  mode_t mode = 0;
+
  int res;
+
  sqlite3_int64 mtime = -1;
+

+
  if( argc<2 || argc>4 ){
+
    sqlite3_result_error(context, 
+
        "wrong number of arguments to function writefile()", -1
+
    );
+
    return;
+
  }
+

+
  zFile = (const char*)sqlite3_value_text(argv[0]);
+
  if( zFile==0 ) return;
+
  if( argc>=3 ){
+
    mode = (mode_t)sqlite3_value_int(argv[2]);
+
  }
+
  if( argc==4 ){
+
    mtime = sqlite3_value_int64(argv[3]);
+
  }
+

+
  res = writeFile(context, zFile, argv[1], mode, mtime);
+
  if( res==1 && errno==ENOENT ){
+
    if( makeDirectory(zFile)==SQLITE_OK ){
+
      res = writeFile(context, zFile, argv[1], mode, mtime);
+
    }
+
  }
+

+
  if( argc>2 && res!=0 ){
+
    if( S_ISLNK(mode) ){
+
      ctxErrorMsg(context, "failed to create symlink: %s", zFile);
+
    }else if( S_ISDIR(mode) ){
+
      ctxErrorMsg(context, "failed to create directory: %s", zFile);
+
    }else{
+
      ctxErrorMsg(context, "failed to write file: %s", zFile);
+
    }
+
  }
+
}
+

+
/*
+
** SQL function:   lsmode(MODE)
+
**
+
** Given a numberic st_mode from stat(), convert it into a human-readable
+
** text string in the style of "ls -l".
+
*/
+
static void lsModeFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  int i;
+
  int iMode = sqlite3_value_int(argv[0]);
+
  char z[16];
+
  (void)argc;
+
  if( S_ISLNK(iMode) ){
+
    z[0] = 'l';
+
  }else if( S_ISREG(iMode) ){
+
    z[0] = '-';
+
  }else if( S_ISDIR(iMode) ){
+
    z[0] = 'd';
+
  }else{
+
    z[0] = '?';
+
  }
+
  for(i=0; i<3; i++){
+
    int m = (iMode >> ((2-i)*3));
+
    char *a = &z[1 + i*3];
+
    a[0] = (m & 0x4) ? 'r' : '-';
+
    a[1] = (m & 0x2) ? 'w' : '-';
+
    a[2] = (m & 0x1) ? 'x' : '-';
+
  }
+
  z[10] = '\0';
+
  sqlite3_result_text(context, z, -1, SQLITE_TRANSIENT);
+
}
+

+
#ifndef SQLITE_OMIT_VIRTUALTABLE
+

+
/* 
+
** Cursor type for recursively iterating through a directory structure.
+
*/
+
typedef struct fsdir_cursor fsdir_cursor;
+
typedef struct FsdirLevel FsdirLevel;
+

+
struct FsdirLevel {
+
  DIR *pDir;                 /* From opendir() */
+
  char *zDir;                /* Name of directory (nul-terminated) */
+
};
+

+
struct fsdir_cursor {
+
  sqlite3_vtab_cursor base;  /* Base class - must be first */
+

+
  int nLvl;                  /* Number of entries in aLvl[] array */
+
  int iLvl;                  /* Index of current entry */
+
  FsdirLevel *aLvl;          /* Hierarchy of directories being traversed */
+

+
  const char *zBase;
+
  int nBase;
+

+
  struct stat sStat;         /* Current lstat() results */
+
  char *zPath;               /* Path to current entry */
+
  sqlite3_int64 iRowid;      /* Current rowid */
+
};
+

+
typedef struct fsdir_tab fsdir_tab;
+
struct fsdir_tab {
+
  sqlite3_vtab base;         /* Base class - must be first */
+
};
+

+
/*
+
** Construct a new fsdir virtual table object.
+
*/
+
static int fsdirConnect(
+
  sqlite3 *db,
+
  void *pAux,
+
  int argc, const char *const*argv,
+
  sqlite3_vtab **ppVtab,
+
  char **pzErr
+
){
+
  fsdir_tab *pNew = 0;
+
  int rc;
+
  (void)pAux;
+
  (void)argc;
+
  (void)argv;
+
  (void)pzErr;
+
  rc = sqlite3_declare_vtab(db, "CREATE TABLE x" FSDIR_SCHEMA);
+
  if( rc==SQLITE_OK ){
+
    pNew = (fsdir_tab*)sqlite3_malloc( sizeof(*pNew) );
+
    if( pNew==0 ) return SQLITE_NOMEM;
+
    memset(pNew, 0, sizeof(*pNew));
+
    sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY);
+
  }
+
  *ppVtab = (sqlite3_vtab*)pNew;
+
  return rc;
+
}
+

+
/*
+
** This method is the destructor for fsdir vtab objects.
+
*/
+
static int fsdirDisconnect(sqlite3_vtab *pVtab){
+
  sqlite3_free(pVtab);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Constructor for a new fsdir_cursor object.
+
*/
+
static int fsdirOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
+
  fsdir_cursor *pCur;
+
  (void)p;
+
  pCur = sqlite3_malloc( sizeof(*pCur) );
+
  if( pCur==0 ) return SQLITE_NOMEM;
+
  memset(pCur, 0, sizeof(*pCur));
+
  pCur->iLvl = -1;
+
  *ppCursor = &pCur->base;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Reset a cursor back to the state it was in when first returned
+
** by fsdirOpen().
+
*/
+
static void fsdirResetCursor(fsdir_cursor *pCur){
+
  int i;
+
  for(i=0; i<=pCur->iLvl; i++){
+
    FsdirLevel *pLvl = &pCur->aLvl[i];
+
    if( pLvl->pDir ) closedir(pLvl->pDir);
+
    sqlite3_free(pLvl->zDir);
+
  }
+
  sqlite3_free(pCur->zPath);
+
  sqlite3_free(pCur->aLvl);
+
  pCur->aLvl = 0;
+
  pCur->zPath = 0;
+
  pCur->zBase = 0;
+
  pCur->nBase = 0;
+
  pCur->nLvl = 0;
+
  pCur->iLvl = -1;
+
  pCur->iRowid = 1;
+
}
+

+
/*
+
** Destructor for an fsdir_cursor.
+
*/
+
static int fsdirClose(sqlite3_vtab_cursor *cur){
+
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
+

+
  fsdirResetCursor(pCur);
+
  sqlite3_free(pCur);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Set the error message for the virtual table associated with cursor
+
** pCur to the results of vprintf(zFmt, ...).
+
*/
+
static void fsdirSetErrmsg(fsdir_cursor *pCur, const char *zFmt, ...){
+
  va_list ap;
+
  va_start(ap, zFmt);
+
  pCur->base.pVtab->zErrMsg = sqlite3_vmprintf(zFmt, ap);
+
  va_end(ap);
+
}
+

+

+
/*
+
** Advance an fsdir_cursor to its next row of output.
+
*/
+
static int fsdirNext(sqlite3_vtab_cursor *cur){
+
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
+
  mode_t m = pCur->sStat.st_mode;
+

+
  pCur->iRowid++;
+
  if( S_ISDIR(m) ){
+
    /* Descend into this directory */
+
    int iNew = pCur->iLvl + 1;
+
    FsdirLevel *pLvl;
+
    if( iNew>=pCur->nLvl ){
+
      int nNew = iNew+1;
+
      sqlite3_int64 nByte = nNew*sizeof(FsdirLevel);
+
      FsdirLevel *aNew = (FsdirLevel*)sqlite3_realloc64(pCur->aLvl, nByte);
+
      if( aNew==0 ) return SQLITE_NOMEM;
+
      memset(&aNew[pCur->nLvl], 0, sizeof(FsdirLevel)*(nNew-pCur->nLvl));
+
      pCur->aLvl = aNew;
+
      pCur->nLvl = nNew;
+
    }
+
    pCur->iLvl = iNew;
+
    pLvl = &pCur->aLvl[iNew];
+
    
+
    pLvl->zDir = pCur->zPath;
+
    pCur->zPath = 0;
+
    pLvl->pDir = opendir(pLvl->zDir);
+
    if( pLvl->pDir==0 ){
+
      fsdirSetErrmsg(pCur, "cannot read directory: %s", pCur->zPath);
+
      return SQLITE_ERROR;
+
    }
+
  }
+

+
  while( pCur->iLvl>=0 ){
+
    FsdirLevel *pLvl = &pCur->aLvl[pCur->iLvl];
+
    struct dirent *pEntry = readdir(pLvl->pDir);
+
    if( pEntry ){
+
      if( pEntry->d_name[0]=='.' ){
+
       if( pEntry->d_name[1]=='.' && pEntry->d_name[2]=='\0' ) continue;
+
       if( pEntry->d_name[1]=='\0' ) continue;
+
      }
+
      sqlite3_free(pCur->zPath);
+
      pCur->zPath = sqlite3_mprintf("%s/%s", pLvl->zDir, pEntry->d_name);
+
      if( pCur->zPath==0 ) return SQLITE_NOMEM;
+
      if( fileLinkStat(pCur->zPath, &pCur->sStat) ){
+
        fsdirSetErrmsg(pCur, "cannot stat file: %s", pCur->zPath);
+
        return SQLITE_ERROR;
+
      }
+
      return SQLITE_OK;
+
    }
+
    closedir(pLvl->pDir);
+
    sqlite3_free(pLvl->zDir);
+
    pLvl->pDir = 0;
+
    pLvl->zDir = 0;
+
    pCur->iLvl--;
+
  }
+

+
  /* EOF */
+
  sqlite3_free(pCur->zPath);
+
  pCur->zPath = 0;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return values of columns for the row at which the series_cursor
+
** is currently pointing.
+
*/
+
static int fsdirColumn(
+
  sqlite3_vtab_cursor *cur,   /* The cursor */
+
  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
+
  int i                       /* Which column to return */
+
){
+
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
+
  switch( i ){
+
    case FSDIR_COLUMN_NAME: {
+
      sqlite3_result_text(ctx, &pCur->zPath[pCur->nBase], -1, SQLITE_TRANSIENT);
+
      break;
+
    }
+

+
    case FSDIR_COLUMN_MODE:
+
      sqlite3_result_int64(ctx, pCur->sStat.st_mode);
+
      break;
+

+
    case FSDIR_COLUMN_MTIME:
+
      sqlite3_result_int64(ctx, pCur->sStat.st_mtime);
+
      break;
+

+
    case FSDIR_COLUMN_DATA: {
+
      mode_t m = pCur->sStat.st_mode;
+
      if( S_ISDIR(m) ){
+
        sqlite3_result_null(ctx);
+
#if !defined(_WIN32) && !defined(WIN32)
+
      }else if( S_ISLNK(m) ){
+
        char aStatic[64];
+
        char *aBuf = aStatic;
+
        sqlite3_int64 nBuf = 64;
+
        int n;
+

+
        while( 1 ){
+
          n = readlink(pCur->zPath, aBuf, nBuf);
+
          if( n<nBuf ) break;
+
          if( aBuf!=aStatic ) sqlite3_free(aBuf);
+
          nBuf = nBuf*2;
+
          aBuf = sqlite3_malloc64(nBuf);
+
          if( aBuf==0 ){
+
            sqlite3_result_error_nomem(ctx);
+
            return SQLITE_NOMEM;
+
          }
+
        }
+

+
        sqlite3_result_text(ctx, aBuf, n, SQLITE_TRANSIENT);
+
        if( aBuf!=aStatic ) sqlite3_free(aBuf);
+
#endif
+
      }else{
+
        readFileContents(ctx, pCur->zPath);
+
      }
+
    }
+
    case FSDIR_COLUMN_PATH:
+
    default: {
+
      /* The FSDIR_COLUMN_PATH and FSDIR_COLUMN_DIR are input parameters.
+
      ** always return their values as NULL */
+
      break;
+
    }
+
  }
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return the rowid for the current row. In this implementation, the
+
** first row returned is assigned rowid value 1, and each subsequent
+
** row a value 1 more than that of the previous.
+
*/
+
static int fsdirRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
+
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
+
  *pRowid = pCur->iRowid;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return TRUE if the cursor has been moved off of the last
+
** row of output.
+
*/
+
static int fsdirEof(sqlite3_vtab_cursor *cur){
+
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
+
  return (pCur->zPath==0);
+
}
+

+
/*
+
** xFilter callback.
+
**
+
** idxNum==1   PATH parameter only
+
** idxNum==2   Both PATH and DIR supplied
+
*/
+
static int fsdirFilter(
+
  sqlite3_vtab_cursor *cur, 
+
  int idxNum, const char *idxStr,
+
  int argc, sqlite3_value **argv
+
){
+
  const char *zDir = 0;
+
  fsdir_cursor *pCur = (fsdir_cursor*)cur;
+
  (void)idxStr;
+
  fsdirResetCursor(pCur);
+

+
  if( idxNum==0 ){
+
    fsdirSetErrmsg(pCur, "table function fsdir requires an argument");
+
    return SQLITE_ERROR;
+
  }
+

+
  assert( argc==idxNum && (argc==1 || argc==2) );
+
  zDir = (const char*)sqlite3_value_text(argv[0]);
+
  if( zDir==0 ){
+
    fsdirSetErrmsg(pCur, "table function fsdir requires a non-NULL argument");
+
    return SQLITE_ERROR;
+
  }
+
  if( argc==2 ){
+
    pCur->zBase = (const char*)sqlite3_value_text(argv[1]);
+
  }
+
  if( pCur->zBase ){
+
    pCur->nBase = (int)strlen(pCur->zBase)+1;
+
    pCur->zPath = sqlite3_mprintf("%s/%s", pCur->zBase, zDir);
+
  }else{
+
    pCur->zPath = sqlite3_mprintf("%s", zDir);
+
  }
+

+
  if( pCur->zPath==0 ){
+
    return SQLITE_NOMEM;
+
  }
+
  if( fileLinkStat(pCur->zPath, &pCur->sStat) ){
+
    fsdirSetErrmsg(pCur, "cannot stat file: %s", pCur->zPath);
+
    return SQLITE_ERROR;
+
  }
+

+
  return SQLITE_OK;
+
}
+

+
/*
+
** SQLite will invoke this method one or more times while planning a query
+
** that uses the generate_series virtual table.  This routine needs to create
+
** a query plan for each invocation and compute an estimated cost for that
+
** plan.
+
**
+
** In this implementation idxNum is used to represent the
+
** query plan.  idxStr is unused.
+
**
+
** The query plan is represented by values of idxNum:
+
**
+
**  (1)  The path value is supplied by argv[0]
+
**  (2)  Path is in argv[0] and dir is in argv[1]
+
*/
+
static int fsdirBestIndex(
+
  sqlite3_vtab *tab,
+
  sqlite3_index_info *pIdxInfo
+
){
+
  int i;                 /* Loop over constraints */
+
  int idxPath = -1;      /* Index in pIdxInfo->aConstraint of PATH= */
+
  int idxDir = -1;       /* Index in pIdxInfo->aConstraint of DIR= */
+
  int seenPath = 0;      /* True if an unusable PATH= constraint is seen */
+
  int seenDir = 0;       /* True if an unusable DIR= constraint is seen */
+
  const struct sqlite3_index_constraint *pConstraint;
+

+
  (void)tab;
+
  pConstraint = pIdxInfo->aConstraint;
+
  for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){
+
    if( pConstraint->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue;
+
    switch( pConstraint->iColumn ){
+
      case FSDIR_COLUMN_PATH: {
+
        if( pConstraint->usable ){
+
          idxPath = i;
+
          seenPath = 0;
+
        }else if( idxPath<0 ){
+
          seenPath = 1;
+
        }
+
        break;
+
      }
+
      case FSDIR_COLUMN_DIR: {
+
        if( pConstraint->usable ){
+
          idxDir = i;
+
          seenDir = 0;
+
        }else if( idxDir<0 ){
+
          seenDir = 1;
+
        }
+
        break;
+
      }
+
    } 
+
  }
+
  if( seenPath || seenDir ){
+
    /* If input parameters are unusable, disallow this plan */
+
    return SQLITE_CONSTRAINT;
+
  }
+

+
  if( idxPath<0 ){
+
    pIdxInfo->idxNum = 0;
+
    /* The pIdxInfo->estimatedCost should have been initialized to a huge
+
    ** number.  Leave it unchanged. */
+
    pIdxInfo->estimatedRows = 0x7fffffff;
+
  }else{
+
    pIdxInfo->aConstraintUsage[idxPath].omit = 1;
+
    pIdxInfo->aConstraintUsage[idxPath].argvIndex = 1;
+
    if( idxDir>=0 ){
+
      pIdxInfo->aConstraintUsage[idxDir].omit = 1;
+
      pIdxInfo->aConstraintUsage[idxDir].argvIndex = 2;
+
      pIdxInfo->idxNum = 2;
+
      pIdxInfo->estimatedCost = 10.0;
+
    }else{
+
      pIdxInfo->idxNum = 1;
+
      pIdxInfo->estimatedCost = 100.0;
+
    }
+
  }
+

+
  return SQLITE_OK;
+
}
+

+
/*
+
** Register the "fsdir" virtual table.
+
*/
+
static int fsdirRegister(sqlite3 *db){
+
  static sqlite3_module fsdirModule = {
+
    0,                         /* iVersion */
+
    0,                         /* xCreate */
+
    fsdirConnect,              /* xConnect */
+
    fsdirBestIndex,            /* xBestIndex */
+
    fsdirDisconnect,           /* xDisconnect */
+
    0,                         /* xDestroy */
+
    fsdirOpen,                 /* xOpen - open a cursor */
+
    fsdirClose,                /* xClose - close a cursor */
+
    fsdirFilter,               /* xFilter - configure scan constraints */
+
    fsdirNext,                 /* xNext - advance a cursor */
+
    fsdirEof,                  /* xEof - check for end of scan */
+
    fsdirColumn,               /* xColumn - read data */
+
    fsdirRowid,                /* xRowid - read data */
+
    0,                         /* xUpdate */
+
    0,                         /* xBegin */
+
    0,                         /* xSync */
+
    0,                         /* xCommit */
+
    0,                         /* xRollback */
+
    0,                         /* xFindMethod */
+
    0,                         /* xRename */
+
    0,                         /* xSavepoint */
+
    0,                         /* xRelease */
+
    0,                         /* xRollbackTo */
+
    0,                         /* xShadowName */
+
  };
+

+
  int rc = sqlite3_create_module(db, "fsdir", &fsdirModule, 0);
+
  return rc;
+
}
+
#else         /* SQLITE_OMIT_VIRTUALTABLE */
+
# define fsdirRegister(x) SQLITE_OK
+
#endif
+

+
#ifdef _WIN32
+

+
#endif
+
int sqlite3_fileio_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
+
){
+
  int rc = SQLITE_OK;
+
  SQLITE_EXTENSION_INIT2(pApi);
+
  (void)pzErrMsg;  /* Unused parameter */
+
  rc = sqlite3_create_function(db, "readfile", 1, 
+
                               SQLITE_UTF8|SQLITE_DIRECTONLY, 0,
+
                               readfileFunc, 0, 0);
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_create_function(db, "writefile", -1,
+
                                 SQLITE_UTF8|SQLITE_DIRECTONLY, 0,
+
                                 writefileFunc, 0, 0);
+
  }
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_create_function(db, "lsmode", 1, SQLITE_UTF8, 0,
+
                                 lsModeFunc, 0, 0);
+
  }
+
  if( rc==SQLITE_OK ){
+
    rc = fsdirRegister(db);
+
  }
+
  return rc;
+
}
+

+
#if defined(FILEIO_WIN32_DLL) && (defined(_WIN32) || defined(WIN32))
+
/* To allow a standalone DLL, make test_windirent.c use the same
+
 * redefined SQLite API calls as the above extension code does.
+
 * Just pull in this .c to accomplish this. As a beneficial side
+
 * effect, this extension becomes a single translation unit. */
+
#  include "test_windirent.c"
+
#endif
+

+
/************************* End ../ext/misc/fileio.c ********************/
+
/************************* Begin ../ext/misc/completion.c ******************/
+
/*
+
** 2017-07-10
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
*************************************************************************
+
**
+
** This file implements an eponymous virtual table that returns suggested
+
** completions for a partial SQL input.
+
**
+
** Suggested usage:
+
**
+
**     SELECT DISTINCT candidate COLLATE nocase
+
**       FROM completion($prefix,$wholeline)
+
**      ORDER BY 1;
+
**
+
** The two query parameters are optional.  $prefix is the text of the
+
** current word being typed and that is to be completed.  $wholeline is
+
** the complete input line, used for context.
+
**
+
** The raw completion() table might return the same candidate multiple
+
** times, for example if the same column name is used to two or more
+
** tables.  And the candidates are returned in an arbitrary order.  Hence,
+
** the DISTINCT and ORDER BY are recommended.
+
**
+
** This virtual table operates at the speed of human typing, and so there
+
** is no attempt to make it fast.  Even a slow implementation will be much
+
** faster than any human can type.
+
**
+
*/
+
/* #include "sqlite3ext.h" */
+
SQLITE_EXTENSION_INIT1
+
#include <assert.h>
+
#include <string.h>
+
#include <ctype.h>
+

+
#ifndef SQLITE_OMIT_VIRTUALTABLE
+

+
/* completion_vtab is a subclass of sqlite3_vtab which will
+
** serve as the underlying representation of a completion virtual table
+
*/
+
typedef struct completion_vtab completion_vtab;
+
struct completion_vtab {
+
  sqlite3_vtab base;  /* Base class - must be first */
+
  sqlite3 *db;        /* Database connection for this completion vtab */
+
};
+

+
/* completion_cursor is a subclass of sqlite3_vtab_cursor which will
+
** serve as the underlying representation of a cursor that scans
+
** over rows of the result
+
*/
+
typedef struct completion_cursor completion_cursor;
+
struct completion_cursor {
+
  sqlite3_vtab_cursor base;  /* Base class - must be first */
+
  sqlite3 *db;               /* Database connection for this cursor */
+
  int nPrefix, nLine;        /* Number of bytes in zPrefix and zLine */
+
  char *zPrefix;             /* The prefix for the word we want to complete */
+
  char *zLine;               /* The whole that we want to complete */
+
  const char *zCurrentRow;   /* Current output row */
+
  int szRow;                 /* Length of the zCurrentRow string */
+
  sqlite3_stmt *pStmt;       /* Current statement */
+
  sqlite3_int64 iRowid;      /* The rowid */
+
  int ePhase;                /* Current phase */
+
  int j;                     /* inter-phase counter */
+
};
+

+
/* Values for ePhase:
+
*/
+
#define COMPLETION_FIRST_PHASE   1
+
#define COMPLETION_KEYWORDS      1
+
#define COMPLETION_PRAGMAS       2
+
#define COMPLETION_FUNCTIONS     3
+
#define COMPLETION_COLLATIONS    4
+
#define COMPLETION_INDEXES       5
+
#define COMPLETION_TRIGGERS      6
+
#define COMPLETION_DATABASES     7
+
#define COMPLETION_TABLES        8    /* Also VIEWs and TRIGGERs */
+
#define COMPLETION_COLUMNS       9
+
#define COMPLETION_MODULES       10
+
#define COMPLETION_EOF           11
+

+
/*
+
** The completionConnect() method is invoked to create a new
+
** completion_vtab that describes the completion virtual table.
+
**
+
** Think of this routine as the constructor for completion_vtab objects.
+
**
+
** All this routine needs to do is:
+
**
+
**    (1) Allocate the completion_vtab object and initialize all fields.
+
**
+
**    (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the
+
**        result set of queries against completion will look like.
+
*/
+
static int completionConnect(
+
  sqlite3 *db,
+
  void *pAux,
+
  int argc, const char *const*argv,
+
  sqlite3_vtab **ppVtab,
+
  char **pzErr
+
){
+
  completion_vtab *pNew;
+
  int rc;
+

+
  (void)(pAux);    /* Unused parameter */
+
  (void)(argc);    /* Unused parameter */
+
  (void)(argv);    /* Unused parameter */
+
  (void)(pzErr);   /* Unused parameter */
+

+
/* Column numbers */
+
#define COMPLETION_COLUMN_CANDIDATE 0  /* Suggested completion of the input */
+
#define COMPLETION_COLUMN_PREFIX    1  /* Prefix of the word to be completed */
+
#define COMPLETION_COLUMN_WHOLELINE 2  /* Entire line seen so far */
+
#define COMPLETION_COLUMN_PHASE     3  /* ePhase - used for debugging only */
+

+
  sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
+
  rc = sqlite3_declare_vtab(db,
+
      "CREATE TABLE x("
+
      "  candidate TEXT,"
+
      "  prefix TEXT HIDDEN,"
+
      "  wholeline TEXT HIDDEN,"
+
      "  phase INT HIDDEN"        /* Used for debugging only */
+
      ")");
+
  if( rc==SQLITE_OK ){
+
    pNew = sqlite3_malloc( sizeof(*pNew) );
+
    *ppVtab = (sqlite3_vtab*)pNew;
+
    if( pNew==0 ) return SQLITE_NOMEM;
+
    memset(pNew, 0, sizeof(*pNew));
+
    pNew->db = db;
+
  }
+
  return rc;
+
}
+

+
/*
+
** This method is the destructor for completion_cursor objects.
+
*/
+
static int completionDisconnect(sqlite3_vtab *pVtab){
+
  sqlite3_free(pVtab);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Constructor for a new completion_cursor object.
+
*/
+
static int completionOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
+
  completion_cursor *pCur;
+
  pCur = sqlite3_malloc( sizeof(*pCur) );
+
  if( pCur==0 ) return SQLITE_NOMEM;
+
  memset(pCur, 0, sizeof(*pCur));
+
  pCur->db = ((completion_vtab*)p)->db;
+
  *ppCursor = &pCur->base;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Reset the completion_cursor.
+
*/
+
static void completionCursorReset(completion_cursor *pCur){
+
  sqlite3_free(pCur->zPrefix);   pCur->zPrefix = 0;  pCur->nPrefix = 0;
+
  sqlite3_free(pCur->zLine);     pCur->zLine = 0;    pCur->nLine = 0;
+
  sqlite3_finalize(pCur->pStmt); pCur->pStmt = 0;
+
  pCur->j = 0;
+
}
+

+
/*
+
** Destructor for a completion_cursor.
+
*/
+
static int completionClose(sqlite3_vtab_cursor *cur){
+
  completionCursorReset((completion_cursor*)cur);
+
  sqlite3_free(cur);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Advance a completion_cursor to its next row of output.
+
**
+
** The ->ePhase, ->j, and ->pStmt fields of the completion_cursor object
+
** record the current state of the scan.  This routine sets ->zCurrentRow
+
** to the current row of output and then returns.  If no more rows remain,
+
** then ->ePhase is set to COMPLETION_EOF which will signal the virtual
+
** table that has reached the end of its scan.
+
**
+
** The current implementation just lists potential identifiers and
+
** keywords and filters them by zPrefix.  Future enhancements should
+
** take zLine into account to try to restrict the set of identifiers and
+
** keywords based on what would be legal at the current point of input.
+
*/
+
static int completionNext(sqlite3_vtab_cursor *cur){
+
  completion_cursor *pCur = (completion_cursor*)cur;
+
  int eNextPhase = 0;  /* Next phase to try if current phase reaches end */
+
  int iCol = -1;       /* If >=0, step pCur->pStmt and use the i-th column */
+
  pCur->iRowid++;
+
  while( pCur->ePhase!=COMPLETION_EOF ){
+
    switch( pCur->ePhase ){
+
      case COMPLETION_KEYWORDS: {
+
        if( pCur->j >= sqlite3_keyword_count() ){
+
          pCur->zCurrentRow = 0;
+
          pCur->ePhase = COMPLETION_DATABASES;
+
        }else{
+
          sqlite3_keyword_name(pCur->j++, &pCur->zCurrentRow, &pCur->szRow);
+
        }
+
        iCol = -1;
+
        break;
+
      }
+
      case COMPLETION_DATABASES: {
+
        if( pCur->pStmt==0 ){
+
          sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1,
+
                             &pCur->pStmt, 0);
+
        }
+
        iCol = 1;
+
        eNextPhase = COMPLETION_TABLES;
+
        break;
+
      }
+
      case COMPLETION_TABLES: {
+
        if( pCur->pStmt==0 ){
+
          sqlite3_stmt *pS2;
+
          char *zSql = 0;
+
          const char *zSep = "";
+
          sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0);
+
          while( sqlite3_step(pS2)==SQLITE_ROW ){
+
            const char *zDb = (const char*)sqlite3_column_text(pS2, 1);
+
            zSql = sqlite3_mprintf(
+
               "%z%s"
+
               "SELECT name FROM \"%w\".sqlite_schema",
+
               zSql, zSep, zDb
+
            );
+
            if( zSql==0 ) return SQLITE_NOMEM;
+
            zSep = " UNION ";
+
          }
+
          sqlite3_finalize(pS2);
+
          sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
+
          sqlite3_free(zSql);
+
        }
+
        iCol = 0;
+
        eNextPhase = COMPLETION_COLUMNS;
+
        break;
+
      }
+
      case COMPLETION_COLUMNS: {
+
        if( pCur->pStmt==0 ){
+
          sqlite3_stmt *pS2;
+
          char *zSql = 0;
+
          const char *zSep = "";
+
          sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0);
+
          while( sqlite3_step(pS2)==SQLITE_ROW ){
+
            const char *zDb = (const char*)sqlite3_column_text(pS2, 1);
+
            zSql = sqlite3_mprintf(
+
               "%z%s"
+
               "SELECT pti.name FROM \"%w\".sqlite_schema AS sm"
+
                       " JOIN pragma_table_info(sm.name,%Q) AS pti"
+
               " WHERE sm.type='table'",
+
               zSql, zSep, zDb, zDb
+
            );
+
            if( zSql==0 ) return SQLITE_NOMEM;
+
            zSep = " UNION ";
+
          }
+
          sqlite3_finalize(pS2);
+
          sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0);
+
          sqlite3_free(zSql);
+
        }
+
        iCol = 0;
+
        eNextPhase = COMPLETION_EOF;
+
        break;
+
      }
+
    }
+
    if( iCol<0 ){
+
      /* This case is when the phase presets zCurrentRow */
+
      if( pCur->zCurrentRow==0 ) continue;
+
    }else{
+
      if( sqlite3_step(pCur->pStmt)==SQLITE_ROW ){
+
        /* Extract the next row of content */
+
        pCur->zCurrentRow = (const char*)sqlite3_column_text(pCur->pStmt, iCol);
+
        pCur->szRow = sqlite3_column_bytes(pCur->pStmt, iCol);
+
      }else{
+
        /* When all rows are finished, advance to the next phase */
+
        sqlite3_finalize(pCur->pStmt);
+
        pCur->pStmt = 0;
+
        pCur->ePhase = eNextPhase;
+
        continue;
+
      }
+
    }
+
    if( pCur->nPrefix==0 ) break;
+
    if( pCur->nPrefix<=pCur->szRow
+
     && sqlite3_strnicmp(pCur->zPrefix, pCur->zCurrentRow, pCur->nPrefix)==0
+
    ){
+
      break;
+
    }
+
  }
+

+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return values of columns for the row at which the completion_cursor
+
** is currently pointing.
+
*/
+
static int completionColumn(
+
  sqlite3_vtab_cursor *cur,   /* The cursor */
+
  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
+
  int i                       /* Which column to return */
+
){
+
  completion_cursor *pCur = (completion_cursor*)cur;
+
  switch( i ){
+
    case COMPLETION_COLUMN_CANDIDATE: {
+
      sqlite3_result_text(ctx, pCur->zCurrentRow, pCur->szRow,SQLITE_TRANSIENT);
+
      break;
+
    }
+
    case COMPLETION_COLUMN_PREFIX: {
+
      sqlite3_result_text(ctx, pCur->zPrefix, -1, SQLITE_TRANSIENT);
+
      break;
+
    }
+
    case COMPLETION_COLUMN_WHOLELINE: {
+
      sqlite3_result_text(ctx, pCur->zLine, -1, SQLITE_TRANSIENT);
+
      break;
+
    }
+
    case COMPLETION_COLUMN_PHASE: {
+
      sqlite3_result_int(ctx, pCur->ePhase);
+
      break;
+
    }
+
  }
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return the rowid for the current row.  In this implementation, the
+
** rowid is the same as the output value.
+
*/
+
static int completionRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
+
  completion_cursor *pCur = (completion_cursor*)cur;
+
  *pRowid = pCur->iRowid;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return TRUE if the cursor has been moved off of the last
+
** row of output.
+
*/
+
static int completionEof(sqlite3_vtab_cursor *cur){
+
  completion_cursor *pCur = (completion_cursor*)cur;
+
  return pCur->ePhase >= COMPLETION_EOF;
+
}
+

+
/*
+
** This method is called to "rewind" the completion_cursor object back
+
** to the first row of output.  This method is always called at least
+
** once prior to any call to completionColumn() or completionRowid() or 
+
** completionEof().
+
*/
+
static int completionFilter(
+
  sqlite3_vtab_cursor *pVtabCursor, 
+
  int idxNum, const char *idxStr,
+
  int argc, sqlite3_value **argv
+
){
+
  completion_cursor *pCur = (completion_cursor *)pVtabCursor;
+
  int iArg = 0;
+
  (void)(idxStr);   /* Unused parameter */
+
  (void)(argc);     /* Unused parameter */
+
  completionCursorReset(pCur);
+
  if( idxNum & 1 ){
+
    pCur->nPrefix = sqlite3_value_bytes(argv[iArg]);
+
    if( pCur->nPrefix>0 ){
+
      pCur->zPrefix = sqlite3_mprintf("%s", sqlite3_value_text(argv[iArg]));
+
      if( pCur->zPrefix==0 ) return SQLITE_NOMEM;
+
    }
+
    iArg = 1;
+
  }
+
  if( idxNum & 2 ){
+
    pCur->nLine = sqlite3_value_bytes(argv[iArg]);
+
    if( pCur->nLine>0 ){
+
      pCur->zLine = sqlite3_mprintf("%s", sqlite3_value_text(argv[iArg]));
+
      if( pCur->zLine==0 ) return SQLITE_NOMEM;
+
    }
+
  }
+
  if( pCur->zLine!=0 && pCur->zPrefix==0 ){
+
    int i = pCur->nLine;
+
    while( i>0 && (isalnum(pCur->zLine[i-1]) || pCur->zLine[i-1]=='_') ){
+
      i--;
+
    }
+
    pCur->nPrefix = pCur->nLine - i;
+
    if( pCur->nPrefix>0 ){
+
      pCur->zPrefix = sqlite3_mprintf("%.*s", pCur->nPrefix, pCur->zLine + i);
+
      if( pCur->zPrefix==0 ) return SQLITE_NOMEM;
+
    }
+
  }
+
  pCur->iRowid = 0;
+
  pCur->ePhase = COMPLETION_FIRST_PHASE;
+
  return completionNext(pVtabCursor);
+
}
+

+
/*
+
** SQLite will invoke this method one or more times while planning a query
+
** that uses the completion virtual table.  This routine needs to create
+
** a query plan for each invocation and compute an estimated cost for that
+
** plan.
+
**
+
** There are two hidden parameters that act as arguments to the table-valued
+
** function:  "prefix" and "wholeline".  Bit 0 of idxNum is set if "prefix"
+
** is available and bit 1 is set if "wholeline" is available.
+
*/
+
static int completionBestIndex(
+
  sqlite3_vtab *tab,
+
  sqlite3_index_info *pIdxInfo
+
){
+
  int i;                 /* Loop over constraints */
+
  int idxNum = 0;        /* The query plan bitmask */
+
  int prefixIdx = -1;    /* Index of the start= constraint, or -1 if none */
+
  int wholelineIdx = -1; /* Index of the stop= constraint, or -1 if none */
+
  int nArg = 0;          /* Number of arguments that completeFilter() expects */
+
  const struct sqlite3_index_constraint *pConstraint;
+

+
  (void)(tab);    /* Unused parameter */
+
  pConstraint = pIdxInfo->aConstraint;
+
  for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){
+
    if( pConstraint->usable==0 ) continue;
+
    if( pConstraint->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue;
+
    switch( pConstraint->iColumn ){
+
      case COMPLETION_COLUMN_PREFIX:
+
        prefixIdx = i;
+
        idxNum |= 1;
+
        break;
+
      case COMPLETION_COLUMN_WHOLELINE:
+
        wholelineIdx = i;
+
        idxNum |= 2;
+
        break;
+
    }
+
  }
+
  if( prefixIdx>=0 ){
+
    pIdxInfo->aConstraintUsage[prefixIdx].argvIndex = ++nArg;
+
    pIdxInfo->aConstraintUsage[prefixIdx].omit = 1;
+
  }
+
  if( wholelineIdx>=0 ){
+
    pIdxInfo->aConstraintUsage[wholelineIdx].argvIndex = ++nArg;
+
    pIdxInfo->aConstraintUsage[wholelineIdx].omit = 1;
+
  }
+
  pIdxInfo->idxNum = idxNum;
+
  pIdxInfo->estimatedCost = (double)5000 - 1000*nArg;
+
  pIdxInfo->estimatedRows = 500 - 100*nArg;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** This following structure defines all the methods for the 
+
** completion virtual table.
+
*/
+
static sqlite3_module completionModule = {
+
  0,                         /* iVersion */
+
  0,                         /* xCreate */
+
  completionConnect,         /* xConnect */
+
  completionBestIndex,       /* xBestIndex */
+
  completionDisconnect,      /* xDisconnect */
+
  0,                         /* xDestroy */
+
  completionOpen,            /* xOpen - open a cursor */
+
  completionClose,           /* xClose - close a cursor */
+
  completionFilter,          /* xFilter - configure scan constraints */
+
  completionNext,            /* xNext - advance a cursor */
+
  completionEof,             /* xEof - check for end of scan */
+
  completionColumn,          /* xColumn - read data */
+
  completionRowid,           /* xRowid - read data */
+
  0,                         /* xUpdate */
+
  0,                         /* xBegin */
+
  0,                         /* xSync */
+
  0,                         /* xCommit */
+
  0,                         /* xRollback */
+
  0,                         /* xFindMethod */
+
  0,                         /* xRename */
+
  0,                         /* xSavepoint */
+
  0,                         /* xRelease */
+
  0,                         /* xRollbackTo */
+
  0                          /* xShadowName */
+
};
+

+
#endif /* SQLITE_OMIT_VIRTUALTABLE */
+

+
int sqlite3CompletionVtabInit(sqlite3 *db){
+
  int rc = SQLITE_OK;
+
#ifndef SQLITE_OMIT_VIRTUALTABLE
+
  rc = sqlite3_create_module(db, "completion", &completionModule, 0);
+
#endif
+
  return rc;
+
}
+

+
#ifdef _WIN32
+

+
#endif
+
int sqlite3_completion_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
+
){
+
  int rc = SQLITE_OK;
+
  SQLITE_EXTENSION_INIT2(pApi);
+
  (void)(pzErrMsg);  /* Unused parameter */
+
#ifndef SQLITE_OMIT_VIRTUALTABLE
+
  rc = sqlite3CompletionVtabInit(db);
+
#endif
+
  return rc;
+
}
+

+
/************************* End ../ext/misc/completion.c ********************/
+
/************************* Begin ../ext/misc/appendvfs.c ******************/
+
/*
+
** 2017-10-20
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** This file implements a VFS shim that allows an SQLite database to be
+
** appended onto the end of some other file, such as an executable.
+
**
+
** A special record must appear at the end of the file that identifies the
+
** file as an appended database and provides the offset to the first page
+
** of the exposed content. (Or, it is the length of the content prefix.)
+
** For best performance page 1 should be located at a disk page boundary,
+
** though that is not required.
+
**
+
** When opening a database using this VFS, the connection might treat
+
** the file as an ordinary SQLite database, or it might treat it as a
+
** database appended onto some other file.  The decision is made by
+
** applying the following rules in order:
+
**
+
**  (1)  An empty file is an ordinary database.
+
**
+
**  (2)  If the file ends with the appendvfs trailer string
+
**       "Start-Of-SQLite3-NNNNNNNN" that file is an appended database.
+
**
+
**  (3)  If the file begins with the standard SQLite prefix string
+
**       "SQLite format 3", that file is an ordinary database.
+
**
+
**  (4)  If none of the above apply and the SQLITE_OPEN_CREATE flag is
+
**       set, then a new database is appended to the already existing file.
+
**
+
**  (5)  Otherwise, SQLITE_CANTOPEN is returned.
+
**
+
** To avoid unnecessary complications with the PENDING_BYTE, the size of
+
** the file containing the database is limited to 1GiB. (1073741824 bytes)
+
** This VFS will not read or write past the 1GiB mark.  This restriction
+
** might be lifted in future versions.  For now, if you need a larger
+
** database, then keep it in a separate file.
+
**
+
** If the file being opened is a plain database (not an appended one), then
+
** this shim is a pass-through into the default underlying VFS. (rule 3)
+
**/
+
/* #include "sqlite3ext.h" */
+
SQLITE_EXTENSION_INIT1
+
#include <string.h>
+
#include <assert.h>
+

+
/* The append mark at the end of the database is:
+
**
+
**     Start-Of-SQLite3-NNNNNNNN
+
**     123456789 123456789 12345
+
**
+
** The NNNNNNNN represents a 64-bit big-endian unsigned integer which is
+
** the offset to page 1, and also the length of the prefix content.
+
*/
+
#define APND_MARK_PREFIX     "Start-Of-SQLite3-"
+
#define APND_MARK_PREFIX_SZ  17
+
#define APND_MARK_FOS_SZ      8
+
#define APND_MARK_SIZE       (APND_MARK_PREFIX_SZ+APND_MARK_FOS_SZ)
+

+
/*
+
** Maximum size of the combined prefix + database + append-mark.  This
+
** must be less than 0x40000000 to avoid locking issues on Windows.
+
*/
+
#define APND_MAX_SIZE  (0x40000000)
+

+
/*
+
** Try to align the database to an even multiple of APND_ROUNDUP bytes.
+
*/
+
#ifndef APND_ROUNDUP
+
#define APND_ROUNDUP 4096
+
#endif
+
#define APND_ALIGN_MASK         ((sqlite3_int64)(APND_ROUNDUP-1))
+
#define APND_START_ROUNDUP(fsz) (((fsz)+APND_ALIGN_MASK) & ~APND_ALIGN_MASK)
+

+
/*
+
** Forward declaration of objects used by this utility
+
*/
+
typedef struct sqlite3_vfs ApndVfs;
+
typedef struct ApndFile ApndFile;
+

+
/* Access to a lower-level VFS that (might) implement dynamic loading,
+
** access to randomness, etc.
+
*/
+
#define ORIGVFS(p)  ((sqlite3_vfs*)((p)->pAppData))
+
#define ORIGFILE(p) ((sqlite3_file*)(((ApndFile*)(p))+1))
+

+
/* An open appendvfs file
+
**
+
** An instance of this structure describes the appended database file.
+
** A separate sqlite3_file object is always appended. The appended
+
** sqlite3_file object (which can be accessed using ORIGFILE()) describes
+
** the entire file, including the prefix, the database, and the
+
** append-mark.
+
**
+
** The structure of an AppendVFS database is like this:
+
**
+
**   +-------------+---------+----------+-------------+
+
**   | prefix-file | padding | database | append-mark |
+
**   +-------------+---------+----------+-------------+
+
**                           ^          ^
+
**                           |          |
+
**                         iPgOne      iMark
+
**
+
**
+
** "prefix file" -  file onto which the database has been appended.
+
** "padding"     -  zero or more bytes inserted so that "database"
+
**                  starts on an APND_ROUNDUP boundary
+
** "database"    -  The SQLite database file
+
** "append-mark" -  The 25-byte "Start-Of-SQLite3-NNNNNNNN" that indicates
+
**                  the offset from the start of prefix-file to the start
+
**                  of "database".
+
**
+
** The size of the database is iMark - iPgOne.
+
**
+
** The NNNNNNNN in the "Start-Of-SQLite3-NNNNNNNN" suffix is the value
+
** of iPgOne stored as a big-ending 64-bit integer.
+
**
+
** iMark will be the size of the underlying file minus 25 (APND_MARKSIZE).
+
** Or, iMark is -1 to indicate that it has not yet been written.
+
*/
+
struct ApndFile {
+
  sqlite3_file base;        /* Subclass.  MUST BE FIRST! */
+
  sqlite3_int64 iPgOne;     /* Offset to the start of the database */
+
  sqlite3_int64 iMark;      /* Offset of the append mark.  -1 if unwritten */
+
  /* Always followed by another sqlite3_file that describes the whole file */
+
};
+

+
/*
+
** Methods for ApndFile
+
*/
+
static int apndClose(sqlite3_file*);
+
static int apndRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
+
static int apndWrite(sqlite3_file*,const void*,int iAmt, sqlite3_int64 iOfst);
+
static int apndTruncate(sqlite3_file*, sqlite3_int64 size);
+
static int apndSync(sqlite3_file*, int flags);
+
static int apndFileSize(sqlite3_file*, sqlite3_int64 *pSize);
+
static int apndLock(sqlite3_file*, int);
+
static int apndUnlock(sqlite3_file*, int);
+
static int apndCheckReservedLock(sqlite3_file*, int *pResOut);
+
static int apndFileControl(sqlite3_file*, int op, void *pArg);
+
static int apndSectorSize(sqlite3_file*);
+
static int apndDeviceCharacteristics(sqlite3_file*);
+
static int apndShmMap(sqlite3_file*, int iPg, int pgsz, int, void volatile**);
+
static int apndShmLock(sqlite3_file*, int offset, int n, int flags);
+
static void apndShmBarrier(sqlite3_file*);
+
static int apndShmUnmap(sqlite3_file*, int deleteFlag);
+
static int apndFetch(sqlite3_file*, sqlite3_int64 iOfst, int iAmt, void **pp);
+
static int apndUnfetch(sqlite3_file*, sqlite3_int64 iOfst, void *p);
+

+
/*
+
** Methods for ApndVfs
+
*/
+
static int apndOpen(sqlite3_vfs*, const char *, sqlite3_file*, int , int *);
+
static int apndDelete(sqlite3_vfs*, const char *zName, int syncDir);
+
static int apndAccess(sqlite3_vfs*, const char *zName, int flags, int *);
+
static int apndFullPathname(sqlite3_vfs*, const char *zName, int, char *zOut);
+
static void *apndDlOpen(sqlite3_vfs*, const char *zFilename);
+
static void apndDlError(sqlite3_vfs*, int nByte, char *zErrMsg);
+
static void (*apndDlSym(sqlite3_vfs *pVfs, void *p, const char*zSym))(void);
+
static void apndDlClose(sqlite3_vfs*, void*);
+
static int apndRandomness(sqlite3_vfs*, int nByte, char *zOut);
+
static int apndSleep(sqlite3_vfs*, int microseconds);
+
static int apndCurrentTime(sqlite3_vfs*, double*);
+
static int apndGetLastError(sqlite3_vfs*, int, char *);
+
static int apndCurrentTimeInt64(sqlite3_vfs*, sqlite3_int64*);
+
static int apndSetSystemCall(sqlite3_vfs*, const char*,sqlite3_syscall_ptr);
+
static sqlite3_syscall_ptr apndGetSystemCall(sqlite3_vfs*, const char *z);
+
static const char *apndNextSystemCall(sqlite3_vfs*, const char *zName);
+

+
static sqlite3_vfs apnd_vfs = {
+
  3,                            /* iVersion (set when registered) */
+
  0,                            /* szOsFile (set when registered) */
+
  1024,                         /* mxPathname */
+
  0,                            /* pNext */
+
  "apndvfs",                    /* zName */
+
  0,                            /* pAppData (set when registered) */ 
+
  apndOpen,                     /* xOpen */
+
  apndDelete,                   /* xDelete */
+
  apndAccess,                   /* xAccess */
+
  apndFullPathname,             /* xFullPathname */
+
  apndDlOpen,                   /* xDlOpen */
+
  apndDlError,                  /* xDlError */
+
  apndDlSym,                    /* xDlSym */
+
  apndDlClose,                  /* xDlClose */
+
  apndRandomness,               /* xRandomness */
+
  apndSleep,                    /* xSleep */
+
  apndCurrentTime,              /* xCurrentTime */
+
  apndGetLastError,             /* xGetLastError */
+
  apndCurrentTimeInt64,         /* xCurrentTimeInt64 */
+
  apndSetSystemCall,            /* xSetSystemCall */
+
  apndGetSystemCall,            /* xGetSystemCall */
+
  apndNextSystemCall            /* xNextSystemCall */
+
};
+

+
static const sqlite3_io_methods apnd_io_methods = {
+
  3,                              /* iVersion */
+
  apndClose,                      /* xClose */
+
  apndRead,                       /* xRead */
+
  apndWrite,                      /* xWrite */
+
  apndTruncate,                   /* xTruncate */
+
  apndSync,                       /* xSync */
+
  apndFileSize,                   /* xFileSize */
+
  apndLock,                       /* xLock */
+
  apndUnlock,                     /* xUnlock */
+
  apndCheckReservedLock,          /* xCheckReservedLock */
+
  apndFileControl,                /* xFileControl */
+
  apndSectorSize,                 /* xSectorSize */
+
  apndDeviceCharacteristics,      /* xDeviceCharacteristics */
+
  apndShmMap,                     /* xShmMap */
+
  apndShmLock,                    /* xShmLock */
+
  apndShmBarrier,                 /* xShmBarrier */
+
  apndShmUnmap,                   /* xShmUnmap */
+
  apndFetch,                      /* xFetch */
+
  apndUnfetch                     /* xUnfetch */
+
};
+

+
/*
+
** Close an apnd-file.
+
*/
+
static int apndClose(sqlite3_file *pFile){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xClose(pFile);
+
}
+

+
/*
+
** Read data from an apnd-file.
+
*/
+
static int apndRead(
+
  sqlite3_file *pFile, 
+
  void *zBuf, 
+
  int iAmt, 
+
  sqlite_int64 iOfst
+
){
+
  ApndFile *paf = (ApndFile *)pFile;
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xRead(pFile, zBuf, iAmt, paf->iPgOne+iOfst);
+
}
+

+
/*
+
** Add the append-mark onto what should become the end of the file.
+
*  If and only if this succeeds, internal ApndFile.iMark is updated.
+
*  Parameter iWriteEnd is the appendvfs-relative offset of the new mark.
+
*/
+
static int apndWriteMark(
+
  ApndFile *paf,
+
  sqlite3_file *pFile,
+
  sqlite_int64 iWriteEnd
+
){
+
  sqlite_int64 iPgOne = paf->iPgOne;
+
  unsigned char a[APND_MARK_SIZE];
+
  int i = APND_MARK_FOS_SZ;
+
  int rc;
+
  assert(pFile == ORIGFILE(paf));
+
  memcpy(a, APND_MARK_PREFIX, APND_MARK_PREFIX_SZ);
+
  while( --i >= 0 ){
+
    a[APND_MARK_PREFIX_SZ+i] = (unsigned char)(iPgOne & 0xff);
+
    iPgOne >>= 8;
+
  }
+
  iWriteEnd += paf->iPgOne;
+
  if( SQLITE_OK==(rc = pFile->pMethods->xWrite
+
                  (pFile, a, APND_MARK_SIZE, iWriteEnd)) ){
+
    paf->iMark = iWriteEnd;
+
  }
+
  return rc;
+
}
+

+
/*
+
** Write data to an apnd-file.
+
*/
+
static int apndWrite(
+
  sqlite3_file *pFile,
+
  const void *zBuf,
+
  int iAmt,
+
  sqlite_int64 iOfst
+
){
+
  ApndFile *paf = (ApndFile *)pFile;
+
  sqlite_int64 iWriteEnd = iOfst + iAmt;
+
  if( iWriteEnd>=APND_MAX_SIZE ) return SQLITE_FULL;
+
  pFile = ORIGFILE(pFile);
+
  /* If append-mark is absent or will be overwritten, write it. */
+
  if( paf->iMark < 0 || paf->iPgOne + iWriteEnd > paf->iMark ){
+
    int rc = apndWriteMark(paf, pFile, iWriteEnd);
+
    if( SQLITE_OK!=rc ) return rc;
+
  }
+
  return pFile->pMethods->xWrite(pFile, zBuf, iAmt, paf->iPgOne+iOfst);
+
}
+

+
/*
+
** Truncate an apnd-file.
+
*/
+
static int apndTruncate(sqlite3_file *pFile, sqlite_int64 size){
+
  ApndFile *paf = (ApndFile *)pFile;
+
  pFile = ORIGFILE(pFile);
+
  /* The append mark goes out first so truncate failure does not lose it. */
+
  if( SQLITE_OK!=apndWriteMark(paf, pFile, size) ) return SQLITE_IOERR;
+
  /* Truncate underlying file just past append mark */
+
  return pFile->pMethods->xTruncate(pFile, paf->iMark+APND_MARK_SIZE);
+
}
+

+
/*
+
** Sync an apnd-file.
+
*/
+
static int apndSync(sqlite3_file *pFile, int flags){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xSync(pFile, flags);
+
}
+

+
/*
+
** Return the current file-size of an apnd-file.
+
** If the append mark is not yet there, the file-size is 0.
+
*/
+
static int apndFileSize(sqlite3_file *pFile, sqlite_int64 *pSize){
+
  ApndFile *paf = (ApndFile *)pFile;
+
  *pSize = ( paf->iMark >= 0 )? (paf->iMark - paf->iPgOne) : 0;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Lock an apnd-file.
+
*/
+
static int apndLock(sqlite3_file *pFile, int eLock){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xLock(pFile, eLock);
+
}
+

+
/*
+
** Unlock an apnd-file.
+
*/
+
static int apndUnlock(sqlite3_file *pFile, int eLock){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xUnlock(pFile, eLock);
+
}
+

+
/*
+
** Check if another file-handle holds a RESERVED lock on an apnd-file.
+
*/
+
static int apndCheckReservedLock(sqlite3_file *pFile, int *pResOut){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xCheckReservedLock(pFile, pResOut);
+
}
+

+
/*
+
** File control method. For custom operations on an apnd-file.
+
*/
+
static int apndFileControl(sqlite3_file *pFile, int op, void *pArg){
+
  ApndFile *paf = (ApndFile *)pFile;
+
  int rc;
+
  pFile = ORIGFILE(pFile);
+
  if( op==SQLITE_FCNTL_SIZE_HINT ) *(sqlite3_int64*)pArg += paf->iPgOne;
+
  rc = pFile->pMethods->xFileControl(pFile, op, pArg);
+
  if( rc==SQLITE_OK && op==SQLITE_FCNTL_VFSNAME ){
+
    *(char**)pArg = sqlite3_mprintf("apnd(%lld)/%z", paf->iPgOne,*(char**)pArg);
+
  }
+
  return rc;
+
}
+

+
/*
+
** Return the sector-size in bytes for an apnd-file.
+
*/
+
static int apndSectorSize(sqlite3_file *pFile){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xSectorSize(pFile);
+
}
+

+
/*
+
** Return the device characteristic flags supported by an apnd-file.
+
*/
+
static int apndDeviceCharacteristics(sqlite3_file *pFile){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xDeviceCharacteristics(pFile);
+
}
+

+
/* Create a shared memory file mapping */
+
static int apndShmMap(
+
  sqlite3_file *pFile,
+
  int iPg,
+
  int pgsz,
+
  int bExtend,
+
  void volatile **pp
+
){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xShmMap(pFile,iPg,pgsz,bExtend,pp);
+
}
+

+
/* Perform locking on a shared-memory segment */
+
static int apndShmLock(sqlite3_file *pFile, int offset, int n, int flags){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xShmLock(pFile,offset,n,flags);
+
}
+

+
/* Memory barrier operation on shared memory */
+
static void apndShmBarrier(sqlite3_file *pFile){
+
  pFile = ORIGFILE(pFile);
+
  pFile->pMethods->xShmBarrier(pFile);
+
}
+

+
/* Unmap a shared memory segment */
+
static int apndShmUnmap(sqlite3_file *pFile, int deleteFlag){
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xShmUnmap(pFile,deleteFlag);
+
}
+

+
/* Fetch a page of a memory-mapped file */
+
static int apndFetch(
+
  sqlite3_file *pFile,
+
  sqlite3_int64 iOfst,
+
  int iAmt,
+
  void **pp
+
){
+
  ApndFile *p = (ApndFile *)pFile;
+
  if( p->iMark < 0 || iOfst+iAmt > p->iMark ){
+
    return SQLITE_IOERR; /* Cannot read what is not yet there. */
+
  }
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xFetch(pFile, iOfst+p->iPgOne, iAmt, pp);
+
}
+

+
/* Release a memory-mapped page */
+
static int apndUnfetch(sqlite3_file *pFile, sqlite3_int64 iOfst, void *pPage){
+
  ApndFile *p = (ApndFile *)pFile;
+
  pFile = ORIGFILE(pFile);
+
  return pFile->pMethods->xUnfetch(pFile, iOfst+p->iPgOne, pPage);
+
}
+

+
/*
+
** Try to read the append-mark off the end of a file.  Return the
+
** start of the appended database if the append-mark is present.
+
** If there is no valid append-mark, return -1;
+
**
+
** An append-mark is only valid if the NNNNNNNN start-of-database offset
+
** indicates that the appended database contains at least one page.  The
+
** start-of-database value must be a multiple of 512.
+
*/
+
static sqlite3_int64 apndReadMark(sqlite3_int64 sz, sqlite3_file *pFile){
+
  int rc, i;
+
  sqlite3_int64 iMark;
+
  int msbs = 8 * (APND_MARK_FOS_SZ-1);
+
  unsigned char a[APND_MARK_SIZE];
+

+
  if( APND_MARK_SIZE!=(sz & 0x1ff) ) return -1;
+
  rc = pFile->pMethods->xRead(pFile, a, APND_MARK_SIZE, sz-APND_MARK_SIZE);
+
  if( rc ) return -1;
+
  if( memcmp(a, APND_MARK_PREFIX, APND_MARK_PREFIX_SZ)!=0 ) return -1;
+
  iMark = ((sqlite3_int64)(a[APND_MARK_PREFIX_SZ] & 0x7f)) << msbs;
+
  for(i=1; i<8; i++){
+
    msbs -= 8;
+
    iMark |= (sqlite3_int64)a[APND_MARK_PREFIX_SZ+i]<<msbs;
+
  }
+
  if( iMark > (sz - APND_MARK_SIZE - 512) ) return -1;
+
  if( iMark & 0x1ff ) return -1;
+
  return iMark;
+
}
+

+
static const char apvfsSqliteHdr[] = "SQLite format 3";
+
/*
+
** Check to see if the file is an appendvfs SQLite database file.
+
** Return true iff it is such. Parameter sz is the file's size.
+
*/
+
static int apndIsAppendvfsDatabase(sqlite3_int64 sz, sqlite3_file *pFile){
+
  int rc;
+
  char zHdr[16];
+
  sqlite3_int64 iMark = apndReadMark(sz, pFile);
+
  if( iMark>=0 ){
+
    /* If file has the correct end-marker, the expected odd size, and the
+
    ** SQLite DB type marker where the end-marker puts it, then it
+
    ** is an appendvfs database.
+
    */
+
    rc = pFile->pMethods->xRead(pFile, zHdr, sizeof(zHdr), iMark);
+
    if( SQLITE_OK==rc
+
     && memcmp(zHdr, apvfsSqliteHdr, sizeof(zHdr))==0
+
     && (sz & 0x1ff) == APND_MARK_SIZE
+
     && sz>=512+APND_MARK_SIZE
+
    ){
+
      return 1; /* It's an appendvfs database */
+
    }
+
  }
+
  return 0;
+
}
+

+
/*
+
** Check to see if the file is an ordinary SQLite database file.
+
** Return true iff so. Parameter sz is the file's size.
+
*/
+
static int apndIsOrdinaryDatabaseFile(sqlite3_int64 sz, sqlite3_file *pFile){
+
  char zHdr[16];
+
  if( apndIsAppendvfsDatabase(sz, pFile) /* rule 2 */
+
   || (sz & 0x1ff) != 0
+
   || SQLITE_OK!=pFile->pMethods->xRead(pFile, zHdr, sizeof(zHdr), 0)
+
   || memcmp(zHdr, apvfsSqliteHdr, sizeof(zHdr))!=0
+
  ){
+
    return 0;
+
  }else{
+
    return 1;
+
  }
+
}
+

+
/*
+
** Open an apnd file handle.
+
*/
+
static int apndOpen(
+
  sqlite3_vfs *pApndVfs,
+
  const char *zName,
+
  sqlite3_file *pFile,
+
  int flags,
+
  int *pOutFlags
+
){
+
  ApndFile *pApndFile = (ApndFile*)pFile;
+
  sqlite3_file *pBaseFile = ORIGFILE(pFile);
+
  sqlite3_vfs *pBaseVfs = ORIGVFS(pApndVfs);
+
  int rc;
+
  sqlite3_int64 sz = 0;
+
  if( (flags & SQLITE_OPEN_MAIN_DB)==0 ){
+
    /* The appendvfs is not to be used for transient or temporary databases.
+
    ** Just use the base VFS open to initialize the given file object and
+
    ** open the underlying file. (Appendvfs is then unused for this file.)
+
    */
+
    return pBaseVfs->xOpen(pBaseVfs, zName, pFile, flags, pOutFlags);
+
  }
+
  memset(pApndFile, 0, sizeof(ApndFile));
+
  pFile->pMethods = &apnd_io_methods;
+
  pApndFile->iMark = -1;    /* Append mark not yet written */
+

+
  rc = pBaseVfs->xOpen(pBaseVfs, zName, pBaseFile, flags, pOutFlags);
+
  if( rc==SQLITE_OK ){
+
    rc = pBaseFile->pMethods->xFileSize(pBaseFile, &sz);
+
    if( rc ){
+
      pBaseFile->pMethods->xClose(pBaseFile);
+
    }
+
  }
+
  if( rc ){
+
    pFile->pMethods = 0;
+
    return rc;
+
  }
+
  if( apndIsOrdinaryDatabaseFile(sz, pBaseFile) ){
+
    /* The file being opened appears to be just an ordinary DB. Copy
+
    ** the base dispatch-table so this instance mimics the base VFS. 
+
    */
+
    memmove(pApndFile, pBaseFile, pBaseVfs->szOsFile);
+
    return SQLITE_OK;
+
  }
+
  pApndFile->iPgOne = apndReadMark(sz, pFile);
+
  if( pApndFile->iPgOne>=0 ){
+
    pApndFile->iMark = sz - APND_MARK_SIZE; /* Append mark found */
+
    return SQLITE_OK;
+
  }
+
  if( (flags & SQLITE_OPEN_CREATE)==0 ){
+
    pBaseFile->pMethods->xClose(pBaseFile);
+
    rc = SQLITE_CANTOPEN;
+
    pFile->pMethods = 0;
+
  }else{
+
    /* Round newly added appendvfs location to #define'd page boundary. 
+
    ** Note that nothing has yet been written to the underlying file.
+
    ** The append mark will be written along with first content write.
+
    ** Until then, paf->iMark value indicates it is not yet written.
+
    */
+
    pApndFile->iPgOne = APND_START_ROUNDUP(sz);
+
  }
+
  return rc;
+
}
+

+
/*
+
** Delete an apnd file.
+
** For an appendvfs, this could mean delete the appendvfs portion,
+
** leaving the appendee as it was before it gained an appendvfs.
+
** For now, this code deletes the underlying file too.
+
*/
+
static int apndDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){
+
  return ORIGVFS(pVfs)->xDelete(ORIGVFS(pVfs), zPath, dirSync);
+
}
+

+
/*
+
** All other VFS methods are pass-thrus.
+
*/
+
static int apndAccess(
+
  sqlite3_vfs *pVfs, 
+
  const char *zPath, 
+
  int flags, 
+
  int *pResOut
+
){
+
  return ORIGVFS(pVfs)->xAccess(ORIGVFS(pVfs), zPath, flags, pResOut);
+
}
+
static int apndFullPathname(
+
  sqlite3_vfs *pVfs, 
+
  const char *zPath, 
+
  int nOut, 
+
  char *zOut
){
  return ORIGVFS(pVfs)->xFullPathname(ORIGVFS(pVfs),zPath,nOut,zOut);
}
@@ -4263,272 +6781,68 @@ static int apndSleep(sqlite3_vfs *pVfs, int nMicro){
static int apndCurrentTime(sqlite3_vfs *pVfs, double *pTimeOut){
  return ORIGVFS(pVfs)->xCurrentTime(ORIGVFS(pVfs), pTimeOut);
}
-
static int apndGetLastError(sqlite3_vfs *pVfs, int a, char *b){
-
  return ORIGVFS(pVfs)->xGetLastError(ORIGVFS(pVfs), a, b);
-
}
-
static int apndCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *p){
-
  return ORIGVFS(pVfs)->xCurrentTimeInt64(ORIGVFS(pVfs), p);
-
}
-
static int apndSetSystemCall(
-
  sqlite3_vfs *pVfs,
-
  const char *zName,
-
  sqlite3_syscall_ptr pCall
-
){
-
  return ORIGVFS(pVfs)->xSetSystemCall(ORIGVFS(pVfs),zName,pCall);
-
}
-
static sqlite3_syscall_ptr apndGetSystemCall(
-
  sqlite3_vfs *pVfs,
-
  const char *zName
-
){
-
  return ORIGVFS(pVfs)->xGetSystemCall(ORIGVFS(pVfs),zName);
-
}
-
static const char *apndNextSystemCall(sqlite3_vfs *pVfs, const char *zName){
-
  return ORIGVFS(pVfs)->xNextSystemCall(ORIGVFS(pVfs), zName);
-
}
-

-
  
-
#ifdef _WIN32
-

-
#endif
-
/* 
-
** This routine is called when the extension is loaded.
-
** Register the new VFS.
-
*/
-
int sqlite3_appendvfs_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  int rc = SQLITE_OK;
-
  sqlite3_vfs *pOrig;
-
  SQLITE_EXTENSION_INIT2(pApi);
-
  (void)pzErrMsg;
-
  (void)db;
-
  pOrig = sqlite3_vfs_find(0);
-
  if( pOrig==0 ) return SQLITE_ERROR;
-
  apnd_vfs.iVersion = pOrig->iVersion;
-
  apnd_vfs.pAppData = pOrig;
-
  apnd_vfs.szOsFile = pOrig->szOsFile + sizeof(ApndFile);
-
  rc = sqlite3_vfs_register(&apnd_vfs, 0);
-
#ifdef APPENDVFS_TEST
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_auto_extension((void(*)(void))apndvfsRegister);
-
  }
-
#endif
-
  if( rc==SQLITE_OK ) rc = SQLITE_OK_LOAD_PERMANENTLY;
-
  return rc;
-
}
-

-
/************************* End ../ext/misc/appendvfs.c ********************/
-
/************************* Begin ../ext/misc/memtrace.c ******************/
-
/*
-
** 2019-01-21
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
*************************************************************************
-
**
-
** This file implements an extension that uses the SQLITE_CONFIG_MALLOC
-
** mechanism to add a tracing layer on top of SQLite.  If this extension
-
** is registered prior to sqlite3_initialize(), it will cause all memory
-
** allocation activities to be logged on standard output, or to some other
-
** FILE specified by the initializer.
-
**
-
** This file needs to be compiled into the application that uses it.
-
**
-
** This extension is used to implement the --memtrace option of the
-
** command-line shell.
-
*/
-
#include <assert.h>
-
#include <string.h>
-
#include <stdio.h>
-

-
/* The original memory allocation routines */
-
static sqlite3_mem_methods memtraceBase;
-
static FILE *memtraceOut;
-

-
/* Methods that trace memory allocations */
-
static void *memtraceMalloc(int n){
-
  if( memtraceOut ){
-
    fprintf(memtraceOut, "MEMTRACE: allocate %d bytes\n", 
-
            memtraceBase.xRoundup(n));
-
  }
-
  return memtraceBase.xMalloc(n);
-
}
-
static void memtraceFree(void *p){
-
  if( p==0 ) return;
-
  if( memtraceOut ){
-
    fprintf(memtraceOut, "MEMTRACE: free %d bytes\n", memtraceBase.xSize(p));
-
  }
-
  memtraceBase.xFree(p);
-
}
-
static void *memtraceRealloc(void *p, int n){
-
  if( p==0 ) return memtraceMalloc(n);
-
  if( n==0 ){
-
    memtraceFree(p);
-
    return 0;
-
  }
-
  if( memtraceOut ){
-
    fprintf(memtraceOut, "MEMTRACE: resize %d -> %d bytes\n",
-
            memtraceBase.xSize(p), memtraceBase.xRoundup(n));
-
  }
-
  return memtraceBase.xRealloc(p, n);
-
}
-
static int memtraceSize(void *p){
-
  return memtraceBase.xSize(p);
-
}
-
static int memtraceRoundup(int n){
-
  return memtraceBase.xRoundup(n);
-
}
-
static int memtraceInit(void *p){
-
  return memtraceBase.xInit(p);
-
}
-
static void memtraceShutdown(void *p){
-
  memtraceBase.xShutdown(p);
-
}
-

-
/* The substitute memory allocator */
-
static sqlite3_mem_methods ersaztMethods = {
-
  memtraceMalloc,
-
  memtraceFree,
-
  memtraceRealloc,
-
  memtraceSize,
-
  memtraceRoundup,
-
  memtraceInit,
-
  memtraceShutdown,
-
  0
-
};
-

-
/* Begin tracing memory allocations to out. */
-
int sqlite3MemTraceActivate(FILE *out){
-
  int rc = SQLITE_OK;
-
  if( memtraceBase.xMalloc==0 ){
-
    rc = sqlite3_config(SQLITE_CONFIG_GETMALLOC, &memtraceBase);
-
    if( rc==SQLITE_OK ){
-
      rc = sqlite3_config(SQLITE_CONFIG_MALLOC, &ersaztMethods);
-
    }
-
  }
-
  memtraceOut = out;
-
  return rc;
-
}
-

-
/* Deactivate memory tracing */
-
int sqlite3MemTraceDeactivate(void){
-
  int rc = SQLITE_OK;
-
  if( memtraceBase.xMalloc!=0 ){
-
    rc = sqlite3_config(SQLITE_CONFIG_MALLOC, &memtraceBase);
-
    if( rc==SQLITE_OK ){
-
      memset(&memtraceBase, 0, sizeof(memtraceBase));
-
    }
-
  }
-
  memtraceOut = 0;
-
  return rc;
-
}
-

-
/************************* End ../ext/misc/memtrace.c ********************/
-
/************************* Begin ../ext/misc/uint.c ******************/
-
/*
-
** 2020-04-14
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** This SQLite extension implements the UINT collating sequence.
-
**
-
** UINT works like BINARY for text, except that embedded strings
-
** of digits compare in numeric order.
-
**
-
**     *   Leading zeros are handled properly, in the sense that
-
**         they do not mess of the maginitude comparison of embedded
-
**         strings of digits.  "x00123y" is equal to "x123y".
-
**
-
**     *   Only unsigned integers are recognized.  Plus and minus
-
**         signs are ignored.  Decimal points and exponential notation
-
**         are ignored.
-
**
-
**     *   Embedded integers can be of arbitrary length.  Comparison
-
**         is *not* limited integers that can be expressed as a
-
**         64-bit machine integer.
-
*/
-
/* #include "sqlite3ext.h" */
-
SQLITE_EXTENSION_INIT1
-
#include <assert.h>
-
#include <string.h>
-
#include <ctype.h>
-

-
/*
-
** Compare text in lexicographic order, except strings of digits
-
** compare in numeric order.
-
*/
-
static int uintCollFunc(
-
  void *notUsed,
-
  int nKey1, const void *pKey1,
-
  int nKey2, const void *pKey2
-
){
-
  const unsigned char *zA = (const unsigned char*)pKey1;
-
  const unsigned char *zB = (const unsigned char*)pKey2;
-
  int i=0, j=0, x;
-
  (void)notUsed;
-
  while( i<nKey1 && j<nKey2 ){
-
    x = zA[i] - zB[j];
-
    if( isdigit(zA[i]) ){
-
      int k;
-
      if( !isdigit(zB[j]) ) return x;
-
      while( i<nKey1 && zA[i]=='0' ){ i++; }
-
      while( j<nKey2 && zB[j]=='0' ){ j++; }
-
      k = 0;
-
      while( i+k<nKey1 && isdigit(zA[i+k])
-
             && j+k<nKey2 && isdigit(zB[j+k]) ){
-
        k++;
-
      }
-
      if( i+k<nKey1 && isdigit(zA[i+k]) ){
-
        return +1;
-
      }else if( j+k<nKey2 && isdigit(zB[j+k]) ){
-
        return -1;
-
      }else{
-
        x = memcmp(zA+i, zB+j, k);
-
        if( x ) return x;
-
        i += k;
-
        j += k;
-
      }
-
    }else if( x ){
-
      return x;
-
    }else{
-
      i++;
-
      j++;
-
    }
-
  }
-
  return (nKey1 - i) - (nKey2 - j);
+
static int apndGetLastError(sqlite3_vfs *pVfs, int a, char *b){
+
  return ORIGVFS(pVfs)->xGetLastError(ORIGVFS(pVfs), a, b);
+
}
+
static int apndCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *p){
+
  return ORIGVFS(pVfs)->xCurrentTimeInt64(ORIGVFS(pVfs), p);
+
}
+
static int apndSetSystemCall(
+
  sqlite3_vfs *pVfs,
+
  const char *zName,
+
  sqlite3_syscall_ptr pCall
+
){
+
  return ORIGVFS(pVfs)->xSetSystemCall(ORIGVFS(pVfs),zName,pCall);
+
}
+
static sqlite3_syscall_ptr apndGetSystemCall(
+
  sqlite3_vfs *pVfs,
+
  const char *zName
+
){
+
  return ORIGVFS(pVfs)->xGetSystemCall(ORIGVFS(pVfs),zName);
+
}
+
static const char *apndNextSystemCall(sqlite3_vfs *pVfs, const char *zName){
+
  return ORIGVFS(pVfs)->xNextSystemCall(ORIGVFS(pVfs), zName);
}

+
  
#ifdef _WIN32

#endif
-
int sqlite3_uint_init(
+
/* 
+
** This routine is called when the extension is loaded.
+
** Register the new VFS.
+
*/
+
int sqlite3_appendvfs_init(
  sqlite3 *db, 
  char **pzErrMsg, 
  const sqlite3_api_routines *pApi
){
+
  int rc = SQLITE_OK;
+
  sqlite3_vfs *pOrig;
  SQLITE_EXTENSION_INIT2(pApi);
-
  (void)pzErrMsg;  /* Unused parameter */
-
  return sqlite3_create_collation(db, "uint", SQLITE_UTF8, 0, uintCollFunc);
+
  (void)pzErrMsg;
+
  (void)db;
+
  pOrig = sqlite3_vfs_find(0);
+
  if( pOrig==0 ) return SQLITE_ERROR;
+
  apnd_vfs.iVersion = pOrig->iVersion;
+
  apnd_vfs.pAppData = pOrig;
+
  apnd_vfs.szOsFile = pOrig->szOsFile + sizeof(ApndFile);
+
  rc = sqlite3_vfs_register(&apnd_vfs, 0);
+
#ifdef APPENDVFS_TEST
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_auto_extension((void(*)(void))apndvfsRegister);
+
  }
+
#endif
+
  if( rc==SQLITE_OK ) rc = SQLITE_OK_LOAD_PERMANENTLY;
+
  return rc;
}

-
/************************* End ../ext/misc/uint.c ********************/
-
/************************* Begin ../ext/misc/decimal.c ******************/
+
/************************* End ../ext/misc/appendvfs.c ********************/
+
#endif
+
#ifdef SQLITE_HAVE_ZLIB
+
/************************* Begin ../ext/misc/zipfile.c ******************/
/*
-
** 2020-06-22
+
** 2017-12-26
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
@@ -4539,7579 +6853,8622 @@ int sqlite3_uint_init(
**
******************************************************************************
**
-
** Routines to implement arbitrary-precision decimal math.
+
** This file implements a virtual table for reading and writing ZIP archive
+
** files.
**
-
** The focus here is on simplicity and correctness, not performance.
+
** Usage example:
+
**
+
**     SELECT name, sz, datetime(mtime,'unixepoch') FROM zipfile($filename);
+
**
+
** Current limitations:
+
**
+
**    *  No support for encryption
+
**    *  No support for ZIP archives spanning multiple files
+
**    *  No support for zip64 extensions
+
**    *  Only the "inflate/deflate" (zlib) compression method is supported
*/
/* #include "sqlite3ext.h" */
SQLITE_EXTENSION_INIT1
-
#include <assert.h>
+
#include <stdio.h>
#include <string.h>
-
#include <ctype.h>
-
#include <stdlib.h>
-

-
/* Mark a function parameter as unused, to suppress nuisance compiler
-
** warnings. */
-
#ifndef UNUSED_PARAMETER
-
# define UNUSED_PARAMETER(X)  (void)(X)
-
#endif
-

+
#include <assert.h>

-
/* A decimal object */
-
typedef struct Decimal Decimal;
-
struct Decimal {
-
  char sign;        /* 0 for positive, 1 for negative */
-
  char oom;         /* True if an OOM is encountered */
-
  char isNull;      /* True if holds a NULL rather than a number */
-
  char isInit;      /* True upon initialization */
-
  int nDigit;       /* Total number of digits */
-
  int nFrac;        /* Number of digits to the right of the decimal point */
-
  signed char *a;   /* Array of digits.  Most significant first. */
-
};
+
#include <zlib.h>

-
/*
-
** Release memory held by a Decimal, but do not free the object itself.
-
*/
-
static void decimal_clear(Decimal *p){
-
  sqlite3_free(p->a);
-
}
+
#ifndef SQLITE_OMIT_VIRTUALTABLE

-
/*
-
** Destroy a Decimal object
-
*/
-
static void decimal_free(Decimal *p){
-
  if( p ){
-
    decimal_clear(p);
-
    sqlite3_free(p);
-
  }
-
}
+
#ifndef SQLITE_AMALGAMATION

-
/*
-
** Allocate a new Decimal object.  Initialize it to the number given
-
** by the input string.
-
*/
-
static Decimal *decimal_new(
-
  sqlite3_context *pCtx,
-
  sqlite3_value *pIn,
-
  int nAlt,
-
  const unsigned char *zAlt
-
){
-
  Decimal *p;
-
  int n, i;
-
  const unsigned char *zIn;
-
  int iExp = 0;
-
  p = sqlite3_malloc( sizeof(*p) );
-
  if( p==0 ) goto new_no_mem;
-
  p->sign = 0;
-
  p->oom = 0;
-
  p->isInit = 1;
-
  p->isNull = 0;
-
  p->nDigit = 0;
-
  p->nFrac = 0;
-
  if( zAlt ){
-
    n = nAlt,
-
    zIn = zAlt;
-
  }else{
-
    if( sqlite3_value_type(pIn)==SQLITE_NULL ){
-
      p->a = 0;
-
      p->isNull = 1;
-
      return p;
-
    }
-
    n = sqlite3_value_bytes(pIn);
-
    zIn = sqlite3_value_text(pIn);
-
  }
-
  p->a = sqlite3_malloc64( n+1 );
-
  if( p->a==0 ) goto new_no_mem;
-
  for(i=0; isspace(zIn[i]); i++){}
-
  if( zIn[i]=='-' ){
-
    p->sign = 1;
-
    i++;
-
  }else if( zIn[i]=='+' ){
-
    i++;
-
  }
-
  while( i<n && zIn[i]=='0' ) i++;
-
  while( i<n ){
-
    char c = zIn[i];
-
    if( c>='0' && c<='9' ){
-
      p->a[p->nDigit++] = c - '0';
-
    }else if( c=='.' ){
-
      p->nFrac = p->nDigit + 1;
-
    }else if( c=='e' || c=='E' ){
-
      int j = i+1;
-
      int neg = 0;
-
      if( j>=n ) break;
-
      if( zIn[j]=='-' ){
-
        neg = 1;
-
        j++;
-
      }else if( zIn[j]=='+' ){
-
        j++;
-
      }
-
      while( j<n && iExp<1000000 ){
-
        if( zIn[j]>='0' && zIn[j]<='9' ){
-
          iExp = iExp*10 + zIn[j] - '0';
-
        }
-
        j++;
-
      }
-
      if( neg ) iExp = -iExp;
-
      break;
-
    }
-
    i++;
-
  }
-
  if( p->nFrac ){
-
    p->nFrac = p->nDigit - (p->nFrac - 1);
-
  }
-
  if( iExp>0 ){
-
    if( p->nFrac>0 ){
-
      if( iExp<=p->nFrac ){
-
        p->nFrac -= iExp;
-
        iExp = 0;
-
      }else{
-
        iExp -= p->nFrac;
-
        p->nFrac = 0;
-
      }
-
    }
-
    if( iExp>0 ){   
-
      p->a = sqlite3_realloc64(p->a, p->nDigit + iExp + 1 );
-
      if( p->a==0 ) goto new_no_mem;
-
      memset(p->a+p->nDigit, 0, iExp);
-
      p->nDigit += iExp;
-
    }
-
  }else if( iExp<0 ){
-
    int nExtra;
-
    iExp = -iExp;
-
    nExtra = p->nDigit - p->nFrac - 1;
-
    if( nExtra ){
-
      if( nExtra>=iExp ){
-
        p->nFrac += iExp;
-
        iExp  = 0;
-
      }else{
-
        iExp -= nExtra;
-
        p->nFrac = p->nDigit - 1;
-
      }
-
    }
-
    if( iExp>0 ){
-
      p->a = sqlite3_realloc64(p->a, p->nDigit + iExp + 1 );
-
      if( p->a==0 ) goto new_no_mem;
-
      memmove(p->a+iExp, p->a, p->nDigit);
-
      memset(p->a, 0, iExp);
-
      p->nDigit += iExp;
-
      p->nFrac += iExp;
-
    }
-
  }
-
  return p;
+
#ifndef UINT32_TYPE
+
# ifdef HAVE_UINT32_T
+
#  define UINT32_TYPE uint32_t
+
# else
+
#  define UINT32_TYPE unsigned int
+
# endif
+
#endif
+
#ifndef UINT16_TYPE
+
# ifdef HAVE_UINT16_T
+
#  define UINT16_TYPE uint16_t
+
# else
+
#  define UINT16_TYPE unsigned short int
+
# endif
+
#endif
+
/* typedef sqlite3_int64 i64; */
+
/* typedef unsigned char u8; */
+
/* typedef UINT32_TYPE u32;           // 4-byte unsigned integer // */
+
/* typedef UINT16_TYPE u16;           // 2-byte unsigned integer // */
+
#define MIN(a,b) ((a)<(b) ? (a) : (b))

-
new_no_mem:
-
  if( pCtx ) sqlite3_result_error_nomem(pCtx);
-
  sqlite3_free(p);
-
  return 0;
-
}
+
#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
+
# define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1
+
#endif
+
#if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS)
+
# define ALWAYS(X)      (1)
+
# define NEVER(X)       (0)
+
#elif !defined(NDEBUG)
+
# define ALWAYS(X)      ((X)?1:(assert(0),0))
+
# define NEVER(X)       ((X)?(assert(0),1):0)
+
#else
+
# define ALWAYS(X)      (X)
+
# define NEVER(X)       (X)
+
#endif

-
/*
-
** Make the given Decimal the result.
-
*/
-
static void decimal_result(sqlite3_context *pCtx, Decimal *p){
-
  char *z;
-
  int i, j;
-
  int n;
-
  if( p==0 || p->oom ){
-
    sqlite3_result_error_nomem(pCtx);
-
    return;
-
  }
-
  if( p->isNull ){
-
    sqlite3_result_null(pCtx);
-
    return;
-
  }
-
  z = sqlite3_malloc( p->nDigit+4 );
-
  if( z==0 ){
-
    sqlite3_result_error_nomem(pCtx);
-
    return;
-
  }
-
  i = 0;
-
  if( p->nDigit==0 || (p->nDigit==1 && p->a[0]==0) ){
-
    p->sign = 0;
-
  }
-
  if( p->sign ){
-
    z[0] = '-';
-
    i = 1;
-
  }
-
  n = p->nDigit - p->nFrac;
-
  if( n<=0 ){
-
    z[i++] = '0';
-
  }
-
  j = 0;
-
  while( n>1 && p->a[j]==0 ){
-
    j++;
-
    n--;
-
  }
-
  while( n>0  ){
-
    z[i++] = p->a[j] + '0';
-
    j++;
-
    n--;
-
  }
-
  if( p->nFrac ){
-
    z[i++] = '.';
-
    do{
-
      z[i++] = p->a[j] + '0';
-
      j++;
-
    }while( j<p->nDigit );
-
  }
-
  z[i] = 0;
-
  sqlite3_result_text(pCtx, z, i, sqlite3_free);
-
}
+
#endif   /* SQLITE_AMALGAMATION */

/*
-
** SQL Function:   decimal(X)
+
** Definitions for mode bitmasks S_IFDIR, S_IFREG and S_IFLNK.
**
-
** Convert input X into decimal and then back into text
+
** In some ways it would be better to obtain these values from system 
+
** header files. But, the dependency is undesirable and (a) these
+
** have been stable for decades, (b) the values are part of POSIX and
+
** are also made explicit in [man stat], and (c) are part of the 
+
** file format for zip archives.
*/
-
static void decimalFunc(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  Decimal *p = decimal_new(context, argv[0], 0, 0);
-
  UNUSED_PARAMETER(argc);
-
  decimal_result(context, p);
-
  decimal_free(p);
-
}
+
#ifndef S_IFDIR
+
# define S_IFDIR 0040000
+
#endif
+
#ifndef S_IFREG
+
# define S_IFREG 0100000
+
#endif
+
#ifndef S_IFLNK
+
# define S_IFLNK 0120000
+
#endif
+

+
static const char ZIPFILE_SCHEMA[] = 
+
  "CREATE TABLE y("
+
    "name PRIMARY KEY,"  /* 0: Name of file in zip archive */
+
    "mode,"              /* 1: POSIX mode for file */
+
    "mtime,"             /* 2: Last modification time (secs since 1970)*/
+
    "sz,"                /* 3: Size of object */
+
    "rawdata,"           /* 4: Raw data */
+
    "data,"              /* 5: Uncompressed data */
+
    "method,"            /* 6: Compression method (integer) */
+
    "z HIDDEN"           /* 7: Name of zip file */
+
  ") WITHOUT ROWID;";
+

+
#define ZIPFILE_F_COLUMN_IDX 7    /* Index of column "file" in the above */
+
#define ZIPFILE_BUFFER_SIZE (64*1024)
+


/*
-
** Compare to Decimal objects.  Return negative, 0, or positive if the
-
** first object is less than, equal to, or greater than the second.
+
** Magic numbers used to read and write zip files.
**
-
** Preconditions for this routine:
+
** ZIPFILE_NEWENTRY_MADEBY:
+
**   Use this value for the "version-made-by" field in new zip file
+
**   entries. The upper byte indicates "unix", and the lower byte 
+
**   indicates that the zip file matches pkzip specification 3.0. 
+
**   This is what info-zip seems to do.
**
-
**    pA!=0
-
**    pA->isNull==0
-
**    pB!=0
-
**    pB->isNull==0
+
** ZIPFILE_NEWENTRY_REQUIRED:
+
**   Value for "version-required-to-extract" field of new entries.
+
**   Version 2.0 is required to support folders and deflate compression.
+
**
+
** ZIPFILE_NEWENTRY_FLAGS:
+
**   Value for "general-purpose-bit-flags" field of new entries. Bit
+
**   11 means "utf-8 filename and comment".
+
**
+
** ZIPFILE_SIGNATURE_CDS:
+
**   First 4 bytes of a valid CDS record.
+
**
+
** ZIPFILE_SIGNATURE_LFH:
+
**   First 4 bytes of a valid LFH record.
+
**
+
** ZIPFILE_SIGNATURE_EOCD
+
**   First 4 bytes of a valid EOCD record.
*/
-
static int decimal_cmp(const Decimal *pA, const Decimal *pB){
-
  int nASig, nBSig, rc, n;
-
  if( pA->sign!=pB->sign ){
-
    return pA->sign ? -1 : +1;
-
  }
-
  if( pA->sign ){
-
    const Decimal *pTemp = pA;
-
    pA = pB;
-
    pB = pTemp;
-
  }
-
  nASig = pA->nDigit - pA->nFrac;
-
  nBSig = pB->nDigit - pB->nFrac;
-
  if( nASig!=nBSig ){
-
    return nASig - nBSig;
-
  }
-
  n = pA->nDigit;
-
  if( n>pB->nDigit ) n = pB->nDigit;
-
  rc = memcmp(pA->a, pB->a, n);
-
  if( rc==0 ){
-
    rc = pA->nDigit - pB->nDigit;
-
  }
-
  return rc;
-
}
+
#define ZIPFILE_EXTRA_TIMESTAMP   0x5455
+
#define ZIPFILE_NEWENTRY_MADEBY   ((3<<8) + 30)
+
#define ZIPFILE_NEWENTRY_REQUIRED 20
+
#define ZIPFILE_NEWENTRY_FLAGS    0x800
+
#define ZIPFILE_SIGNATURE_CDS     0x02014b50
+
#define ZIPFILE_SIGNATURE_LFH     0x04034b50
+
#define ZIPFILE_SIGNATURE_EOCD    0x06054b50
+

+
/*
+
** The sizes of the fixed-size part of each of the three main data 
+
** structures in a zip archive.
+
*/
+
#define ZIPFILE_LFH_FIXED_SZ      30
+
#define ZIPFILE_EOCD_FIXED_SZ     22
+
#define ZIPFILE_CDS_FIXED_SZ      46
+

+
/*
+
*** 4.3.16  End of central directory record:
+
***
+
***   end of central dir signature    4 bytes  (0x06054b50)
+
***   number of this disk             2 bytes
+
***   number of the disk with the
+
***   start of the central directory  2 bytes
+
***   total number of entries in the
+
***   central directory on this disk  2 bytes
+
***   total number of entries in
+
***   the central directory           2 bytes
+
***   size of the central directory   4 bytes
+
***   offset of start of central
+
***   directory with respect to
+
***   the starting disk number        4 bytes
+
***   .ZIP file comment length        2 bytes
+
***   .ZIP file comment       (variable size)
+
*/
+
typedef struct ZipfileEOCD ZipfileEOCD;
+
struct ZipfileEOCD {
+
  u16 iDisk;
+
  u16 iFirstDisk;
+
  u16 nEntry;
+
  u16 nEntryTotal;
+
  u32 nSize;
+
  u32 iOffset;
+
};
+

+
/*
+
*** 4.3.12  Central directory structure:
+
***
+
*** ...
+
***
+
***   central file header signature   4 bytes  (0x02014b50)
+
***   version made by                 2 bytes
+
***   version needed to extract       2 bytes
+
***   general purpose bit flag        2 bytes
+
***   compression method              2 bytes
+
***   last mod file time              2 bytes
+
***   last mod file date              2 bytes
+
***   crc-32                          4 bytes
+
***   compressed size                 4 bytes
+
***   uncompressed size               4 bytes
+
***   file name length                2 bytes
+
***   extra field length              2 bytes
+
***   file comment length             2 bytes
+
***   disk number start               2 bytes
+
***   internal file attributes        2 bytes
+
***   external file attributes        4 bytes
+
***   relative offset of local header 4 bytes
+
*/
+
typedef struct ZipfileCDS ZipfileCDS;
+
struct ZipfileCDS {
+
  u16 iVersionMadeBy;
+
  u16 iVersionExtract;
+
  u16 flags;
+
  u16 iCompression;
+
  u16 mTime;
+
  u16 mDate;
+
  u32 crc32;
+
  u32 szCompressed;
+
  u32 szUncompressed;
+
  u16 nFile;
+
  u16 nExtra;
+
  u16 nComment;
+
  u16 iDiskStart;
+
  u16 iInternalAttr;
+
  u32 iExternalAttr;
+
  u32 iOffset;
+
  char *zFile;                    /* Filename (sqlite3_malloc()) */
+
};
+

+
/*
+
*** 4.3.7  Local file header:
+
***
+
***   local file header signature     4 bytes  (0x04034b50)
+
***   version needed to extract       2 bytes
+
***   general purpose bit flag        2 bytes
+
***   compression method              2 bytes
+
***   last mod file time              2 bytes
+
***   last mod file date              2 bytes
+
***   crc-32                          4 bytes
+
***   compressed size                 4 bytes
+
***   uncompressed size               4 bytes
+
***   file name length                2 bytes
+
***   extra field length              2 bytes
+
***   
+
*/
+
typedef struct ZipfileLFH ZipfileLFH;
+
struct ZipfileLFH {
+
  u16 iVersionExtract;
+
  u16 flags;
+
  u16 iCompression;
+
  u16 mTime;
+
  u16 mDate;
+
  u32 crc32;
+
  u32 szCompressed;
+
  u32 szUncompressed;
+
  u16 nFile;
+
  u16 nExtra;
+
};

-
/*
-
** SQL Function:   decimal_cmp(X, Y)
-
**
-
** Return negative, zero, or positive if X is less then, equal to, or
-
** greater than Y.
+
typedef struct ZipfileEntry ZipfileEntry;
+
struct ZipfileEntry {
+
  ZipfileCDS cds;            /* Parsed CDS record */
+
  u32 mUnixTime;             /* Modification time, in UNIX format */
+
  u8 *aExtra;                /* cds.nExtra+cds.nComment bytes of extra data */
+
  i64 iDataOff;              /* Offset to data in file (if aData==0) */
+
  u8 *aData;                 /* cds.szCompressed bytes of compressed data */
+
  ZipfileEntry *pNext;       /* Next element in in-memory CDS */
+
};
+

+
/* 
+
** Cursor type for zipfile tables.
*/
-
static void decimalCmpFunc(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  Decimal *pA = 0, *pB = 0;
-
  int rc;
+
typedef struct ZipfileCsr ZipfileCsr;
+
struct ZipfileCsr {
+
  sqlite3_vtab_cursor base;  /* Base class - must be first */
+
  i64 iId;                   /* Cursor ID */
+
  u8 bEof;                   /* True when at EOF */
+
  u8 bNoop;                  /* If next xNext() call is no-op */

-
  UNUSED_PARAMETER(argc);
-
  pA = decimal_new(context, argv[0], 0, 0);
-
  if( pA==0 || pA->isNull ) goto cmp_done;
-
  pB = decimal_new(context, argv[1], 0, 0);
-
  if( pB==0 || pB->isNull ) goto cmp_done;
-
  rc = decimal_cmp(pA, pB);
-
  if( rc<0 ) rc = -1;
-
  else if( rc>0 ) rc = +1;
-
  sqlite3_result_int(context, rc);
-
cmp_done:
-
  decimal_free(pA);
-
  decimal_free(pB);
-
}
+
  /* Used outside of write transactions */
+
  FILE *pFile;               /* Zip file */
+
  i64 iNextOff;              /* Offset of next record in central directory */
+
  ZipfileEOCD eocd;          /* Parse of central directory record */
+

+
  ZipfileEntry *pFreeEntry;  /* Free this list when cursor is closed or reset */
+
  ZipfileEntry *pCurrent;    /* Current entry */
+
  ZipfileCsr *pCsrNext;      /* Next cursor on same virtual table */
+
};
+

+
typedef struct ZipfileTab ZipfileTab;
+
struct ZipfileTab {
+
  sqlite3_vtab base;         /* Base class - must be first */
+
  char *zFile;               /* Zip file this table accesses (may be NULL) */
+
  sqlite3 *db;               /* Host database connection */
+
  u8 *aBuffer;               /* Temporary buffer used for various tasks */
+

+
  ZipfileCsr *pCsrList;      /* List of cursors */
+
  i64 iNextCsrid;
+

+
  /* The following are used by write transactions only */
+
  ZipfileEntry *pFirstEntry; /* Linked list of all files (if pWriteFd!=0) */
+
  ZipfileEntry *pLastEntry;  /* Last element in pFirstEntry list */
+
  FILE *pWriteFd;            /* File handle open on zip archive */
+
  i64 szCurrent;             /* Current size of zip archive */
+
  i64 szOrig;                /* Size of archive at start of transaction */
+
};

/*
-
** Expand the Decimal so that it has a least nDigit digits and nFrac
-
** digits to the right of the decimal point.
+
** Set the error message contained in context ctx to the results of
+
** vprintf(zFmt, ...).
*/
-
static void decimal_expand(Decimal *p, int nDigit, int nFrac){
-
  int nAddSig;
-
  int nAddFrac;
-
  if( p==0 ) return;
-
  nAddFrac = nFrac - p->nFrac;
-
  nAddSig = (nDigit - p->nDigit) - nAddFrac;
-
  if( nAddFrac==0 && nAddSig==0 ) return;
-
  p->a = sqlite3_realloc64(p->a, nDigit+1);
-
  if( p->a==0 ){
-
    p->oom = 1;
-
    return;
-
  }
-
  if( nAddSig ){
-
    memmove(p->a+nAddSig, p->a, p->nDigit);
-
    memset(p->a, 0, nAddSig);
-
    p->nDigit += nAddSig;
-
  }
-
  if( nAddFrac ){
-
    memset(p->a+p->nDigit, 0, nAddFrac);
-
    p->nDigit += nAddFrac;
-
    p->nFrac += nAddFrac;
-
  }
+
static void zipfileCtxErrorMsg(sqlite3_context *ctx, const char *zFmt, ...){
+
  char *zMsg = 0;
+
  va_list ap;
+
  va_start(ap, zFmt);
+
  zMsg = sqlite3_vmprintf(zFmt, ap);
+
  sqlite3_result_error(ctx, zMsg, -1);
+
  sqlite3_free(zMsg);
+
  va_end(ap);
}

/*
-
** Add the value pB into pA.
-
**
-
** Both pA and pB might become denormalized by this routine.
+
** If string zIn is quoted, dequote it in place. Otherwise, if the string
+
** is not quoted, do nothing.
*/
-
static void decimal_add(Decimal *pA, Decimal *pB){
-
  int nSig, nFrac, nDigit;
-
  int i, rc;
-
  if( pA==0 ){
-
    return;
-
  }
-
  if( pA->oom || pB==0 || pB->oom ){
-
    pA->oom = 1;
-
    return;
-
  }
-
  if( pA->isNull || pB->isNull ){
-
    pA->isNull = 1;
-
    return;
-
  }
-
  nSig = pA->nDigit - pA->nFrac;
-
  if( nSig && pA->a[0]==0 ) nSig--;
-
  if( nSig<pB->nDigit-pB->nFrac ){
-
    nSig = pB->nDigit - pB->nFrac;
-
  }
-
  nFrac = pA->nFrac;
-
  if( nFrac<pB->nFrac ) nFrac = pB->nFrac;
-
  nDigit = nSig + nFrac + 1;
-
  decimal_expand(pA, nDigit, nFrac);
-
  decimal_expand(pB, nDigit, nFrac);
-
  if( pA->oom || pB->oom ){
-
    pA->oom = 1;
-
  }else{
-
    if( pA->sign==pB->sign ){
-
      int carry = 0;
-
      for(i=nDigit-1; i>=0; i--){
-
        int x = pA->a[i] + pB->a[i] + carry;
-
        if( x>=10 ){
-
          carry = 1;
-
          pA->a[i] = x - 10;
-
        }else{
-
          carry = 0;
-
          pA->a[i] = x;
-
        }
-
      }
-
    }else{
-
      signed char *aA, *aB;
-
      int borrow = 0;
-
      rc = memcmp(pA->a, pB->a, nDigit);
-
      if( rc<0 ){
-
        aA = pB->a;
-
        aB = pA->a;
-
        pA->sign = !pA->sign;
-
      }else{
-
        aA = pA->a;
-
        aB = pB->a;
-
      }
-
      for(i=nDigit-1; i>=0; i--){
-
        int x = aA[i] - aB[i] - borrow;
-
        if( x<0 ){
-
          pA->a[i] = x+10;
-
          borrow = 1;
-
        }else{
-
          pA->a[i] = x;
-
          borrow = 0;
-
        }
-
      }
+
static void zipfileDequote(char *zIn){
+
  char q = zIn[0];
+
  if( q=='"' || q=='\'' || q=='`' || q=='[' ){
+
    int iIn = 1;
+
    int iOut = 0;
+
    if( q=='[' ) q = ']';
+
    while( ALWAYS(zIn[iIn]) ){
+
      char c = zIn[iIn++];
+
      if( c==q && zIn[iIn++]!=q ) break;
+
      zIn[iOut++] = c;
    }
+
    zIn[iOut] = '\0';
  }
}

/*
-
** Compare text in decimal order.
+
** Construct a new ZipfileTab virtual table object.
+
** 
+
**   argv[0]   -> module name  ("zipfile")
+
**   argv[1]   -> database name
+
**   argv[2]   -> table name
+
**   argv[...] -> "column name" and other module argument fields.
*/
-
static int decimalCollFunc(
-
  void *notUsed,
-
  int nKey1, const void *pKey1,
-
  int nKey2, const void *pKey2
+
static int zipfileConnect(
+
  sqlite3 *db,
+
  void *pAux,
+
  int argc, const char *const*argv,
+
  sqlite3_vtab **ppVtab,
+
  char **pzErr
){
-
  const unsigned char *zA = (const unsigned char*)pKey1;
-
  const unsigned char *zB = (const unsigned char*)pKey2;
-
  Decimal *pA = decimal_new(0, 0, nKey1, zA);
-
  Decimal *pB = decimal_new(0, 0, nKey2, zB);
+
  int nByte = sizeof(ZipfileTab) + ZIPFILE_BUFFER_SIZE;
+
  int nFile = 0;
+
  const char *zFile = 0;
+
  ZipfileTab *pNew = 0;
  int rc;
-
  UNUSED_PARAMETER(notUsed);
-
  if( pA==0 || pB==0 ){
-
    rc = 0;
-
  }else{
-
    rc = decimal_cmp(pA, pB);
+

+
  /* If the table name is not "zipfile", require that the argument be
+
  ** specified. This stops zipfile tables from being created as:
+
  **
+
  **   CREATE VIRTUAL TABLE zzz USING zipfile();
+
  **
+
  ** It does not prevent:
+
  **
+
  **   CREATE VIRTUAL TABLE zipfile USING zipfile();
+
  */
+
  assert( 0==sqlite3_stricmp(argv[0], "zipfile") );
+
  if( (0!=sqlite3_stricmp(argv[2], "zipfile") && argc<4) || argc>4 ){
+
    *pzErr = sqlite3_mprintf("zipfile constructor requires one argument");
+
    return SQLITE_ERROR;
  }
-
  decimal_free(pA);
-
  decimal_free(pB);
+

+
  if( argc>3 ){
+
    zFile = argv[3];
+
    nFile = (int)strlen(zFile)+1;
+
  }
+

+
  rc = sqlite3_declare_vtab(db, ZIPFILE_SCHEMA);
+
  if( rc==SQLITE_OK ){
+
    pNew = (ZipfileTab*)sqlite3_malloc64((sqlite3_int64)nByte+nFile);
+
    if( pNew==0 ) return SQLITE_NOMEM;
+
    memset(pNew, 0, nByte+nFile);
+
    pNew->db = db;
+
    pNew->aBuffer = (u8*)&pNew[1];
+
    if( zFile ){
+
      pNew->zFile = (char*)&pNew->aBuffer[ZIPFILE_BUFFER_SIZE];
+
      memcpy(pNew->zFile, zFile, nFile);
+
      zipfileDequote(pNew->zFile);
+
    }
+
  }
+
  sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY);
+
  *ppVtab = (sqlite3_vtab*)pNew;
  return rc;
}

-

/*
-
** SQL Function:   decimal_add(X, Y)
-
**                 decimal_sub(X, Y)
-
**
-
** Return the sum or difference of X and Y.
+
** Free the ZipfileEntry structure indicated by the only argument.
*/
-
static void decimalAddFunc(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  Decimal *pA = decimal_new(context, argv[0], 0, 0);
-
  Decimal *pB = decimal_new(context, argv[1], 0, 0);
-
  UNUSED_PARAMETER(argc);
-
  decimal_add(pA, pB);
-
  decimal_result(context, pA);
-
  decimal_free(pA);
-
  decimal_free(pB);
-
}
-
static void decimalSubFunc(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  Decimal *pA = decimal_new(context, argv[0], 0, 0);
-
  Decimal *pB = decimal_new(context, argv[1], 0, 0);
-
  UNUSED_PARAMETER(argc);
-
  if( pB ){
-
    pB->sign = !pB->sign;
-
    decimal_add(pA, pB);
-
    decimal_result(context, pA);
+
static void zipfileEntryFree(ZipfileEntry *p){
+
  if( p ){
+
    sqlite3_free(p->cds.zFile);
+
    sqlite3_free(p);
  }
-
  decimal_free(pA);
-
  decimal_free(pB);
}

-
/* Aggregate funcion:   decimal_sum(X)
-
**
-
** Works like sum() except that it uses decimal arithmetic for unlimited
-
** precision.
+
/*
+
** Release resources that should be freed at the end of a write 
+
** transaction.
*/
-
static void decimalSumStep(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  Decimal *p;
-
  Decimal *pArg;
-
  UNUSED_PARAMETER(argc);
-
  p = sqlite3_aggregate_context(context, sizeof(*p));
-
  if( p==0 ) return;
-
  if( !p->isInit ){
-
    p->isInit = 1;
-
    p->a = sqlite3_malloc(2);
-
    if( p->a==0 ){
-
      p->oom = 1;
-
    }else{
-
      p->a[0] = 0;
-
    }
-
    p->nDigit = 1;
-
    p->nFrac = 0;
+
static void zipfileCleanupTransaction(ZipfileTab *pTab){
+
  ZipfileEntry *pEntry;
+
  ZipfileEntry *pNext;
+

+
  if( pTab->pWriteFd ){
+
    fclose(pTab->pWriteFd);
+
    pTab->pWriteFd = 0;
  }
-
  if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return;
-
  pArg = decimal_new(context, argv[0], 0, 0);
-
  decimal_add(p, pArg);
-
  decimal_free(pArg);
-
}
-
static void decimalSumInverse(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  Decimal *p;
-
  Decimal *pArg;
-
  UNUSED_PARAMETER(argc);
-
  p = sqlite3_aggregate_context(context, sizeof(*p));
-
  if( p==0 ) return;
-
  if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return;
-
  pArg = decimal_new(context, argv[0], 0, 0);
-
  if( pArg ) pArg->sign = !pArg->sign;
-
  decimal_add(p, pArg);
-
  decimal_free(pArg);
-
}
-
static void decimalSumValue(sqlite3_context *context){
-
  Decimal *p = sqlite3_aggregate_context(context, 0);
-
  if( p==0 ) return;
-
  decimal_result(context, p);
+
  for(pEntry=pTab->pFirstEntry; pEntry; pEntry=pNext){
+
    pNext = pEntry->pNext;
+
    zipfileEntryFree(pEntry);
+
  }
+
  pTab->pFirstEntry = 0;
+
  pTab->pLastEntry = 0;
+
  pTab->szCurrent = 0;
+
  pTab->szOrig = 0;
}
-
static void decimalSumFinalize(sqlite3_context *context){
-
  Decimal *p = sqlite3_aggregate_context(context, 0);
-
  if( p==0 ) return;
-
  decimal_result(context, p);
-
  decimal_clear(p);
+

+
/*
+
** This method is the destructor for zipfile vtab objects.
+
*/
+
static int zipfileDisconnect(sqlite3_vtab *pVtab){
+
  zipfileCleanupTransaction((ZipfileTab*)pVtab);
+
  sqlite3_free(pVtab);
+
  return SQLITE_OK;
}

/*
-
** SQL Function:   decimal_mul(X, Y)
-
**
-
** Return the product of X and Y.
-
**
-
** All significant digits after the decimal point are retained.
-
** Trailing zeros after the decimal point are omitted as long as
-
** the number of digits after the decimal point is no less than
-
** either the number of digits in either input.
+
** Constructor for a new ZipfileCsr object.
*/
-
static void decimalMulFunc(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  Decimal *pA = decimal_new(context, argv[0], 0, 0);
-
  Decimal *pB = decimal_new(context, argv[1], 0, 0);
-
  signed char *acc = 0;
-
  int i, j, k;
-
  int minFrac;
-
  UNUSED_PARAMETER(argc);
-
  if( pA==0 || pA->oom || pA->isNull
-
   || pB==0 || pB->oom || pB->isNull 
-
  ){
-
    goto mul_end;
-
  }
-
  acc = sqlite3_malloc64( pA->nDigit + pB->nDigit + 2 );
-
  if( acc==0 ){
-
    sqlite3_result_error_nomem(context);
-
    goto mul_end;
-
  }
-
  memset(acc, 0, pA->nDigit + pB->nDigit + 2);
-
  minFrac = pA->nFrac;
-
  if( pB->nFrac<minFrac ) minFrac = pB->nFrac;
-
  for(i=pA->nDigit-1; i>=0; i--){
-
    signed char f = pA->a[i];
-
    int carry = 0, x;
-
    for(j=pB->nDigit-1, k=i+j+3; j>=0; j--, k--){
-
      x = acc[k] + f*pB->a[j] + carry;
-
      acc[k] = x%10;
-
      carry = x/10;
-
    }
-
    x = acc[k] + carry;
-
    acc[k] = x%10;
-
    acc[k-1] += x/10;
-
  }
-
  sqlite3_free(pA->a);
-
  pA->a = acc;
-
  acc = 0;
-
  pA->nDigit += pB->nDigit + 2;
-
  pA->nFrac += pB->nFrac;
-
  pA->sign ^= pB->sign;
-
  while( pA->nFrac>minFrac && pA->a[pA->nDigit-1]==0 ){
-
    pA->nFrac--;
-
    pA->nDigit--;
+
static int zipfileOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCsr){
+
  ZipfileTab *pTab = (ZipfileTab*)p;
+
  ZipfileCsr *pCsr;
+
  pCsr = sqlite3_malloc(sizeof(*pCsr));
+
  *ppCsr = (sqlite3_vtab_cursor*)pCsr;
+
  if( pCsr==0 ){
+
    return SQLITE_NOMEM;
  }
-
  decimal_result(context, pA);
-

-
mul_end:
-
  sqlite3_free(acc);
-
  decimal_free(pA);
-
  decimal_free(pB);
+
  memset(pCsr, 0, sizeof(*pCsr));
+
  pCsr->iId = ++pTab->iNextCsrid;
+
  pCsr->pCsrNext = pTab->pCsrList;
+
  pTab->pCsrList = pCsr;
+
  return SQLITE_OK;
}

-
#ifdef _WIN32
-

-
#endif
-
int sqlite3_decimal_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  int rc = SQLITE_OK;
-
  static const struct {
-
    const char *zFuncName;
-
    int nArg;
-
    void (*xFunc)(sqlite3_context*,int,sqlite3_value**);
-
  } aFunc[] = {
-
    { "decimal",       1,   decimalFunc        },
-
    { "decimal_cmp",   2,   decimalCmpFunc     },
-
    { "decimal_add",   2,   decimalAddFunc     },
-
    { "decimal_sub",   2,   decimalSubFunc     },
-
    { "decimal_mul",   2,   decimalMulFunc     },
-
  };
-
  unsigned int i;
-
  (void)pzErrMsg;  /* Unused parameter */
-

-
  SQLITE_EXTENSION_INIT2(pApi);
+
/*
+
** Reset a cursor back to the state it was in when first returned
+
** by zipfileOpen().
+
*/
+
static void zipfileResetCursor(ZipfileCsr *pCsr){
+
  ZipfileEntry *p;
+
  ZipfileEntry *pNext;

-
  for(i=0; i<sizeof(aFunc)/sizeof(aFunc[0]) && rc==SQLITE_OK; i++){
-
    rc = sqlite3_create_function(db, aFunc[i].zFuncName, aFunc[i].nArg,
-
                   SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC,
-
                   0, aFunc[i].xFunc, 0, 0);
-
  }
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_window_function(db, "decimal_sum", 1,
-
                   SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0,
-
                   decimalSumStep, decimalSumFinalize,
-
                   decimalSumValue, decimalSumInverse, 0);
+
  pCsr->bEof = 0;
+
  if( pCsr->pFile ){
+
    fclose(pCsr->pFile);
+
    pCsr->pFile = 0;
+
    zipfileEntryFree(pCsr->pCurrent);
+
    pCsr->pCurrent = 0;
  }
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_collation(db, "decimal", SQLITE_UTF8,
-
                                  0, decimalCollFunc);
+

+
  for(p=pCsr->pFreeEntry; p; p=pNext){
+
    pNext = p->pNext;
+
    zipfileEntryFree(p);
  }
-
  return rc;
}

-
/************************* End ../ext/misc/decimal.c ********************/
-
/************************* Begin ../ext/misc/ieee754.c ******************/
/*
-
** 2013-04-17
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** This SQLite extension implements functions for the exact display
-
** and input of IEEE754 Binary64 floating-point numbers.
-
**
-
**   ieee754(X)
-
**   ieee754(Y,Z)
-
**
-
** In the first form, the value X should be a floating-point number.
-
** The function will return a string of the form 'ieee754(Y,Z)' where
-
** Y and Z are integers such that X==Y*pow(2,Z).
-
**
-
** In the second form, Y and Z are integers which are the mantissa and
-
** base-2 exponent of a new floating point number.  The function returns
-
** a floating-point value equal to Y*pow(2,Z).
-
**
-
** Examples:
-
**
-
**     ieee754(2.0)             ->     'ieee754(2,0)'
-
**     ieee754(45.25)           ->     'ieee754(181,-2)'
-
**     ieee754(2, 0)            ->     2.0
-
**     ieee754(181, -2)         ->     45.25
-
**
-
** Two additional functions break apart the one-argument ieee754()
-
** result into separate integer values:
-
**
-
**     ieee754_mantissa(45.25)  ->     181
-
**     ieee754_exponent(45.25)  ->     -2
-
**
-
** These functions convert binary64 numbers into blobs and back again.
-
**
-
**     ieee754_from_blob(x'3ff0000000000000')  ->  1.0
-
**     ieee754_to_blob(1.0)                    ->  x'3ff0000000000000'
-
**
-
** In all single-argument functions, if the argument is an 8-byte blob
-
** then that blob is interpreted as a big-endian binary64 value.
-
**
-
**
-
** EXACT DECIMAL REPRESENTATION OF BINARY64 VALUES
-
** -----------------------------------------------
-
**
-
** This extension in combination with the separate 'decimal' extension
-
** can be used to compute the exact decimal representation of binary64
-
** values.  To begin, first compute a table of exponent values:
-
**
-
**    CREATE TABLE pow2(x INTEGER PRIMARY KEY, v TEXT);
-
**    WITH RECURSIVE c(x,v) AS (
-
**      VALUES(0,'1')
-
**      UNION ALL
-
**      SELECT x+1, decimal_mul(v,'2') FROM c WHERE x+1<=971
-
**    ) INSERT INTO pow2(x,v) SELECT x, v FROM c;
-
**    WITH RECURSIVE c(x,v) AS (
-
**      VALUES(-1,'0.5')
-
**      UNION ALL
-
**      SELECT x-1, decimal_mul(v,'0.5') FROM c WHERE x-1>=-1075
-
**    ) INSERT INTO pow2(x,v) SELECT x, v FROM c;
-
**
-
** Then, to compute the exact decimal representation of a floating
-
** point value (the value 47.49 is used in the example) do:
-
**
-
**    WITH c(n) AS (VALUES(47.49))
-
**          ---------------^^^^^---- Replace with whatever you want
-
**    SELECT decimal_mul(ieee754_mantissa(c.n),pow2.v)
-
**      FROM pow2, c WHERE pow2.x=ieee754_exponent(c.n);
-
**
-
** Here is a query to show various boundry values for the binary64
-
** number format:
-
**
-
**    WITH c(name,bin) AS (VALUES
-
**       ('minimum positive value',        x'0000000000000001'),
-
**       ('maximum subnormal value',       x'000fffffffffffff'),
-
**       ('mininum positive nornal value', x'0010000000000000'),
-
**       ('maximum value',                 x'7fefffffffffffff'))
-
**    SELECT c.name, decimal_mul(ieee754_mantissa(c.bin),pow2.v)
-
**      FROM pow2, c WHERE pow2.x=ieee754_exponent(c.bin);
-
**
+
** Destructor for an ZipfileCsr.
+
*/
+
static int zipfileClose(sqlite3_vtab_cursor *cur){
+
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
+
  ZipfileTab *pTab = (ZipfileTab*)(pCsr->base.pVtab);
+
  ZipfileCsr **pp;
+
  zipfileResetCursor(pCsr);
+

+
  /* Remove this cursor from the ZipfileTab.pCsrList list. */
+
  for(pp=&pTab->pCsrList; *pp!=pCsr; pp=&((*pp)->pCsrNext));
+
  *pp = pCsr->pCsrNext;
+

+
  sqlite3_free(pCsr);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Set the error message for the virtual table associated with cursor
+
** pCsr to the results of vprintf(zFmt, ...).
*/
-
/* #include "sqlite3ext.h" */
-
SQLITE_EXTENSION_INIT1
-
#include <assert.h>
-
#include <string.h>
-

-
/* Mark a function parameter as unused, to suppress nuisance compiler
-
** warnings. */
-
#ifndef UNUSED_PARAMETER
-
# define UNUSED_PARAMETER(X)  (void)(X)
-
#endif
+
static void zipfileTableErr(ZipfileTab *pTab, const char *zFmt, ...){
+
  va_list ap;
+
  va_start(ap, zFmt);
+
  sqlite3_free(pTab->base.zErrMsg);
+
  pTab->base.zErrMsg = sqlite3_vmprintf(zFmt, ap);
+
  va_end(ap);
+
}
+
static void zipfileCursorErr(ZipfileCsr *pCsr, const char *zFmt, ...){
+
  va_list ap;
+
  va_start(ap, zFmt);
+
  sqlite3_free(pCsr->base.pVtab->zErrMsg);
+
  pCsr->base.pVtab->zErrMsg = sqlite3_vmprintf(zFmt, ap);
+
  va_end(ap);
+
}

/*
-
** Implementation of the ieee754() function
+
** Read nRead bytes of data from offset iOff of file pFile into buffer
+
** aRead[]. Return SQLITE_OK if successful, or an SQLite error code
+
** otherwise. 
+
**
+
** If an error does occur, output variable (*pzErrmsg) may be set to point
+
** to an English language error message. It is the responsibility of the
+
** caller to eventually free this buffer using
+
** sqlite3_free().
*/
-
static void ieee754func(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
+
static int zipfileReadData(
+
  FILE *pFile,                    /* Read from this file */
+
  u8 *aRead,                      /* Read into this buffer */
+
  int nRead,                      /* Number of bytes to read */
+
  i64 iOff,                       /* Offset to read from */
+
  char **pzErrmsg                 /* OUT: Error message (from sqlite3_malloc) */
){
-
  if( argc==1 ){
-
    sqlite3_int64 m, a;
-
    double r;
-
    int e;
-
    int isNeg;
-
    char zResult[100];
-
    assert( sizeof(m)==sizeof(r) );
-
    if( sqlite3_value_type(argv[0])==SQLITE_BLOB
-
     && sqlite3_value_bytes(argv[0])==sizeof(r)
-
    ){
-
      const unsigned char *x = sqlite3_value_blob(argv[0]);
-
      unsigned int i;
-
      sqlite3_uint64 v = 0;
-
      for(i=0; i<sizeof(r); i++){
-
        v = (v<<8) | x[i];
-
      }
-
      memcpy(&r, &v, sizeof(r));
-
    }else{
-
      r = sqlite3_value_double(argv[0]);
-
    }
-
    if( r<0.0 ){
-
      isNeg = 1;
-
      r = -r;
-
    }else{
-
      isNeg = 0;
-
    }
-
    memcpy(&a,&r,sizeof(a));
-
    if( a==0 ){
-
      e = 0;
-
      m = 0;
-
    }else{
-
      e = a>>52;
-
      m = a & ((((sqlite3_int64)1)<<52)-1);
-
      if( e==0 ){
-
        m <<= 1;
-
      }else{
-
        m |= ((sqlite3_int64)1)<<52;
-
      }
-
      while( e<1075 && m>0 && (m&1)==0 ){
-
        m >>= 1;
-
        e++;
-
      }
-
      if( isNeg ) m = -m;
-
    }
-
    switch( *(int*)sqlite3_user_data(context) ){
-
      case 0:
-
        sqlite3_snprintf(sizeof(zResult), zResult, "ieee754(%lld,%d)",
-
                         m, e-1075);
-
        sqlite3_result_text(context, zResult, -1, SQLITE_TRANSIENT);
-
        break;
-
      case 1:
-
        sqlite3_result_int64(context, m);
-
        break;
-
      case 2:
-
        sqlite3_result_int(context, e-1075);
-
        break;
-
    }
-
  }else{
-
    sqlite3_int64 m, e, a;
-
    double r;
-
    int isNeg = 0;
-
    m = sqlite3_value_int64(argv[0]);
-
    e = sqlite3_value_int64(argv[1]);
-

-
    /* Limit the range of e.  Ticket 22dea1cfdb9151e4 2021-03-02 */
-
    if( e>10000 ){
-
      e = 10000;
-
    }else if( e<-10000 ){
-
      e = -10000;
-
    }
+
  size_t n;
+
  fseek(pFile, (long)iOff, SEEK_SET);
+
  n = fread(aRead, 1, nRead, pFile);
+
  if( (int)n!=nRead ){
+
    *pzErrmsg = sqlite3_mprintf("error in fread()");
+
    return SQLITE_ERROR;
+
  }
+
  return SQLITE_OK;
+
}

-
    if( m<0 ){
-
      isNeg = 1;
-
      m = -m;
-
      if( m<0 ) return;
-
    }else if( m==0 && e>-1000 && e<1000 ){
-
      sqlite3_result_double(context, 0.0);
-
      return;
-
    }
-
    while( (m>>32)&0xffe00000 ){
-
      m >>= 1;
-
      e++;
-
    }
-
    while( m!=0 && ((m>>32)&0xfff00000)==0 ){
-
      m <<= 1;
-
      e--;
-
    }
-
    e += 1075;
-
    if( e<=0 ){
-
      /* Subnormal */
-
      if( 1-e >= 64 ){
-
        m = 0;
-
      }else{
-
        m >>= 1-e;
-
      }
-
      e = 0;
-
    }else if( e>0x7ff ){
-
      e = 0x7ff;
+
static int zipfileAppendData(
+
  ZipfileTab *pTab,
+
  const u8 *aWrite,
+
  int nWrite
+
){
+
  if( nWrite>0 ){
+
    size_t n = nWrite;
+
    fseek(pTab->pWriteFd, (long)pTab->szCurrent, SEEK_SET);
+
    n = fwrite(aWrite, 1, nWrite, pTab->pWriteFd);
+
    if( (int)n!=nWrite ){
+
      pTab->base.zErrMsg = sqlite3_mprintf("error in fwrite()");
+
      return SQLITE_ERROR;
    }
-
    a = m & ((((sqlite3_int64)1)<<52)-1);
-
    a |= e<<52;
-
    if( isNeg ) a |= ((sqlite3_uint64)1)<<63;
-
    memcpy(&r, &a, sizeof(r));
-
    sqlite3_result_double(context, r);
+
    pTab->szCurrent += nWrite;
  }
+
  return SQLITE_OK;
}

/*
-
** Functions to convert between blobs and floats.
+
** Read and return a 16-bit little-endian unsigned integer from buffer aBuf.
*/
-
static void ieee754func_from_blob(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  UNUSED_PARAMETER(argc);
-
  if( sqlite3_value_type(argv[0])==SQLITE_BLOB
-
   && sqlite3_value_bytes(argv[0])==sizeof(double)
-
  ){
-
    double r;
-
    const unsigned char *x = sqlite3_value_blob(argv[0]);
-
    unsigned int i;
-
    sqlite3_uint64 v = 0;
-
    for(i=0; i<sizeof(r); i++){
-
      v = (v<<8) | x[i];
-
    }
-
    memcpy(&r, &v, sizeof(r));
-
    sqlite3_result_double(context, r);
-
  }
+
static u16 zipfileGetU16(const u8 *aBuf){
+
  return (aBuf[1] << 8) + aBuf[0];
}
-
static void ieee754func_to_blob(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  UNUSED_PARAMETER(argc);
-
  if( sqlite3_value_type(argv[0])==SQLITE_FLOAT
-
   || sqlite3_value_type(argv[0])==SQLITE_INTEGER
-
  ){
-
    double r = sqlite3_value_double(argv[0]);
-
    sqlite3_uint64 v;
-
    unsigned char a[sizeof(r)];
-
    unsigned int i;
-
    memcpy(&v, &r, sizeof(r));
-
    for(i=1; i<=sizeof(r); i++){
-
      a[sizeof(r)-i] = v&0xff;
-
      v >>= 8;
-
    }
-
    sqlite3_result_blob(context, a, sizeof(r), SQLITE_TRANSIENT);
-
  }
+

+
/*
+
** Read and return a 32-bit little-endian unsigned integer from buffer aBuf.
+
*/
+
static u32 zipfileGetU32(const u8 *aBuf){
+
  if( aBuf==0 ) return 0;
+
  return ((u32)(aBuf[3]) << 24)
+
       + ((u32)(aBuf[2]) << 16)
+
       + ((u32)(aBuf[1]) <<  8)
+
       + ((u32)(aBuf[0]) <<  0);
+
}
+

+
/*
+
** Write a 16-bit little endiate integer into buffer aBuf.
+
*/
+
static void zipfilePutU16(u8 *aBuf, u16 val){
+
  aBuf[0] = val & 0xFF;
+
  aBuf[1] = (val>>8) & 0xFF;
}

+
/*
+
** Write a 32-bit little endiate integer into buffer aBuf.
+
*/
+
static void zipfilePutU32(u8 *aBuf, u32 val){
+
  aBuf[0] = val & 0xFF;
+
  aBuf[1] = (val>>8) & 0xFF;
+
  aBuf[2] = (val>>16) & 0xFF;
+
  aBuf[3] = (val>>24) & 0xFF;
+
}

-
#ifdef _WIN32
+
#define zipfileRead32(aBuf) ( aBuf+=4, zipfileGetU32(aBuf-4) )
+
#define zipfileRead16(aBuf) ( aBuf+=2, zipfileGetU16(aBuf-2) )

-
#endif
-
int sqlite3_ieee_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  static const struct {
-
    char *zFName;
-
    int nArg;
-
    int iAux;
-
    void (*xFunc)(sqlite3_context*,int,sqlite3_value**);
-
  } aFunc[] = {
-
    { "ieee754",           1,   0, ieee754func },
-
    { "ieee754",           2,   0, ieee754func },
-
    { "ieee754_mantissa",  1,   1, ieee754func },
-
    { "ieee754_exponent",  1,   2, ieee754func },
-
    { "ieee754_to_blob",   1,   0, ieee754func_to_blob },
-
    { "ieee754_from_blob", 1,   0, ieee754func_from_blob },
+
#define zipfileWrite32(aBuf,val) { zipfilePutU32(aBuf,val); aBuf+=4; }
+
#define zipfileWrite16(aBuf,val) { zipfilePutU16(aBuf,val); aBuf+=2; }

-
  };
-
  unsigned int i;
+
/*
+
** Magic numbers used to read CDS records.
+
*/
+
#define ZIPFILE_CDS_NFILE_OFF        28
+
#define ZIPFILE_CDS_SZCOMPRESSED_OFF 20
+

+
/*
+
** Decode the CDS record in buffer aBuf into (*pCDS). Return SQLITE_ERROR
+
** if the record is not well-formed, or SQLITE_OK otherwise.
+
*/
+
static int zipfileReadCDS(u8 *aBuf, ZipfileCDS *pCDS){
+
  u8 *aRead = aBuf;
+
  u32 sig = zipfileRead32(aRead);
  int rc = SQLITE_OK;
-
  SQLITE_EXTENSION_INIT2(pApi);
-
  (void)pzErrMsg;  /* Unused parameter */
-
  for(i=0; i<sizeof(aFunc)/sizeof(aFunc[0]) && rc==SQLITE_OK; i++){
-
    rc = sqlite3_create_function(db, aFunc[i].zFName, aFunc[i].nArg,
-
                               SQLITE_UTF8|SQLITE_INNOCUOUS,
-
                               (void*)&aFunc[i].iAux,
-
                               aFunc[i].xFunc, 0, 0);
+
  if( sig!=ZIPFILE_SIGNATURE_CDS ){
+
    rc = SQLITE_ERROR;
+
  }else{
+
    pCDS->iVersionMadeBy = zipfileRead16(aRead);
+
    pCDS->iVersionExtract = zipfileRead16(aRead);
+
    pCDS->flags = zipfileRead16(aRead);
+
    pCDS->iCompression = zipfileRead16(aRead);
+
    pCDS->mTime = zipfileRead16(aRead);
+
    pCDS->mDate = zipfileRead16(aRead);
+
    pCDS->crc32 = zipfileRead32(aRead);
+
    pCDS->szCompressed = zipfileRead32(aRead);
+
    pCDS->szUncompressed = zipfileRead32(aRead);
+
    assert( aRead==&aBuf[ZIPFILE_CDS_NFILE_OFF] );
+
    pCDS->nFile = zipfileRead16(aRead);
+
    pCDS->nExtra = zipfileRead16(aRead);
+
    pCDS->nComment = zipfileRead16(aRead);
+
    pCDS->iDiskStart = zipfileRead16(aRead);
+
    pCDS->iInternalAttr = zipfileRead16(aRead);
+
    pCDS->iExternalAttr = zipfileRead32(aRead);
+
    pCDS->iOffset = zipfileRead32(aRead);
+
    assert( aRead==&aBuf[ZIPFILE_CDS_FIXED_SZ] );
  }
+

  return rc;
}

-
/************************* End ../ext/misc/ieee754.c ********************/
-
/************************* Begin ../ext/misc/series.c ******************/
/*
-
** 2015-08-18
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
*************************************************************************
-
**
-
** This file demonstrates how to create a table-valued-function using
-
** a virtual table.  This demo implements the generate_series() function
-
** which gives similar results to the eponymous function in PostgreSQL.
-
** Examples:
-
**
-
**      SELECT * FROM generate_series(0,100,5);
-
**
-
** The query above returns integers from 0 through 100 counting by steps
-
** of 5.
-
**
-
**      SELECT * FROM generate_series(0,100);
-
**
-
** Integers from 0 through 100 with a step size of 1.
-
**
-
**      SELECT * FROM generate_series(20) LIMIT 10;
-
**
-
** Integers 20 through 29.
-
**
-
** HOW IT WORKS
-
**
-
** The generate_series "function" is really a virtual table with the
-
** following schema:
-
**
-
**     CREATE TABLE generate_series(
-
**       value,
-
**       start HIDDEN,
-
**       stop HIDDEN,
-
**       step HIDDEN
-
**     );
+
** Decode the LFH record in buffer aBuf into (*pLFH). Return SQLITE_ERROR
+
** if the record is not well-formed, or SQLITE_OK otherwise.
+
*/
+
static int zipfileReadLFH(
+
  u8 *aBuffer,
+
  ZipfileLFH *pLFH
+
){
+
  u8 *aRead = aBuffer;
+
  int rc = SQLITE_OK;
+

+
  u32 sig = zipfileRead32(aRead);
+
  if( sig!=ZIPFILE_SIGNATURE_LFH ){
+
    rc = SQLITE_ERROR;
+
  }else{
+
    pLFH->iVersionExtract = zipfileRead16(aRead);
+
    pLFH->flags = zipfileRead16(aRead);
+
    pLFH->iCompression = zipfileRead16(aRead);
+
    pLFH->mTime = zipfileRead16(aRead);
+
    pLFH->mDate = zipfileRead16(aRead);
+
    pLFH->crc32 = zipfileRead32(aRead);
+
    pLFH->szCompressed = zipfileRead32(aRead);
+
    pLFH->szUncompressed = zipfileRead32(aRead);
+
    pLFH->nFile = zipfileRead16(aRead);
+
    pLFH->nExtra = zipfileRead16(aRead);
+
  }
+
  return rc;
+
}
+

+

+
/*
+
** Buffer aExtra (size nExtra bytes) contains zip archive "extra" fields.
+
** Scan through this buffer to find an "extra-timestamp" field. If one
+
** exists, extract the 32-bit modification-timestamp from it and store
+
** the value in output parameter *pmTime.
**
-
** Function arguments in queries against this virtual table are translated
-
** into equality constraints against successive hidden columns.  In other
-
** words, the following pairs of queries are equivalent to each other:
+
** Zero is returned if no extra-timestamp record could be found (and so
+
** *pmTime is left unchanged), or non-zero otherwise.
**
-
**    SELECT * FROM generate_series(0,100,5);
-
**    SELECT * FROM generate_series WHERE start=0 AND stop=100 AND step=5;
+
** The general format of an extra field is:
**
-
**    SELECT * FROM generate_series(0,100);
-
**    SELECT * FROM generate_series WHERE start=0 AND stop=100;
+
**   Header ID    2 bytes
+
**   Data Size    2 bytes
+
**   Data         N bytes
+
*/
+
static int zipfileScanExtra(u8 *aExtra, int nExtra, u32 *pmTime){
+
  int ret = 0;
+
  u8 *p = aExtra;
+
  u8 *pEnd = &aExtra[nExtra];
+

+
  while( p<pEnd ){
+
    u16 id = zipfileRead16(p);
+
    u16 nByte = zipfileRead16(p);
+

+
    switch( id ){
+
      case ZIPFILE_EXTRA_TIMESTAMP: {
+
        u8 b = p[0];
+
        if( b & 0x01 ){     /* 0x01 -> modtime is present */
+
          *pmTime = zipfileGetU32(&p[1]);
+
          ret = 1;
+
        }
+
        break;
+
      }
+
    }
+

+
    p += nByte;
+
  }
+
  return ret;
+
}
+

+
/*
+
** Convert the standard MS-DOS timestamp stored in the mTime and mDate
+
** fields of the CDS structure passed as the only argument to a 32-bit
+
** UNIX seconds-since-the-epoch timestamp. Return the result.
**
-
**    SELECT * FROM generate_series(20) LIMIT 10;
-
**    SELECT * FROM generate_series WHERE start=20 LIMIT 10;
+
** "Standard" MS-DOS time format:
**
-
** The generate_series virtual table implementation leaves the xCreate method
-
** set to NULL.  This means that it is not possible to do a CREATE VIRTUAL
-
** TABLE command with "generate_series" as the USING argument.  Instead, there
-
** is a single generate_series virtual table that is always available without
-
** having to be created first.
+
**   File modification time:
+
**     Bits 00-04: seconds divided by 2
+
**     Bits 05-10: minute
+
**     Bits 11-15: hour
+
**   File modification date:
+
**     Bits 00-04: day
+
**     Bits 05-08: month (1-12)
+
**     Bits 09-15: years from 1980 
**
-
** The xBestIndex method looks for equality constraints against the hidden
-
** start, stop, and step columns, and if present, it uses those constraints
-
** to bound the sequence of generated values.  If the equality constraints
-
** are missing, it uses 0 for start, 4294967295 for stop, and 1 for step.
-
** xBestIndex returns a small cost when both start and stop are available,
-
** and a very large cost if either start or stop are unavailable.  This
-
** encourages the query planner to order joins such that the bounds of the
-
** series are well-defined.
+
** https://msdn.microsoft.com/en-us/library/9kkf9tah.aspx
*/
-
/* #include "sqlite3ext.h" */
-
SQLITE_EXTENSION_INIT1
-
#include <assert.h>
-
#include <string.h>
+
static u32 zipfileMtime(ZipfileCDS *pCDS){
+
  int Y,M,D,X1,X2,A,B,sec,min,hr;
+
  i64 JDsec;
+
  Y = (1980 + ((pCDS->mDate >> 9) & 0x7F));
+
  M = ((pCDS->mDate >> 5) & 0x0F);
+
  D = (pCDS->mDate & 0x1F);
+
  sec = (pCDS->mTime & 0x1F)*2;
+
  min = (pCDS->mTime >> 5) & 0x3F;
+
  hr = (pCDS->mTime >> 11) & 0x1F;
+
  if( M<=2 ){
+
    Y--;
+
    M += 12;
+
  }
+
  X1 = 36525*(Y+4716)/100;
+
  X2 = 306001*(M+1)/10000;
+
  A = Y/100;
+
  B = 2 - A + (A/4);
+
  JDsec = (i64)((X1 + X2 + D + B - 1524.5)*86400) + hr*3600 + min*60 + sec;
+
  return (u32)(JDsec - (i64)24405875*(i64)8640);
+
}

-
#ifndef SQLITE_OMIT_VIRTUALTABLE
+
/*
+
** The opposite of zipfileMtime(). This function populates the mTime and
+
** mDate fields of the CDS structure passed as the first argument according
+
** to the UNIX timestamp value passed as the second.
+
*/
+
static void zipfileMtimeToDos(ZipfileCDS *pCds, u32 mUnixTime){
+
  /* Convert unix timestamp to JD (2440588 is noon on 1/1/1970) */
+
  i64 JD = (i64)2440588 + mUnixTime / (24*60*60);

+
  int A, B, C, D, E;
+
  int yr, mon, day;
+
  int hr, min, sec;

-
/* series_cursor is a subclass of sqlite3_vtab_cursor which will
-
** serve as the underlying representation of a cursor that scans
-
** over rows of the result
-
*/
-
typedef struct series_cursor series_cursor;
-
struct series_cursor {
-
  sqlite3_vtab_cursor base;  /* Base class - must be first */
-
  int isDesc;                /* True to count down rather than up */
-
  sqlite3_int64 iRowid;      /* The rowid */
-
  sqlite3_int64 iValue;      /* Current value ("value") */
-
  sqlite3_int64 mnValue;     /* Mimimum value ("start") */
-
  sqlite3_int64 mxValue;     /* Maximum value ("stop") */
-
  sqlite3_int64 iStep;       /* Increment ("step") */
-
};
+
  A = (int)((JD - 1867216.25)/36524.25);
+
  A = (int)(JD + 1 + A - (A/4));
+
  B = A + 1524;
+
  C = (int)((B - 122.1)/365.25);
+
  D = (36525*(C&32767))/100;
+
  E = (int)((B-D)/30.6001);
+

+
  day = B - D - (int)(30.6001*E);
+
  mon = (E<14 ? E-1 : E-13);
+
  yr = mon>2 ? C-4716 : C-4715;
+

+
  hr = (mUnixTime % (24*60*60)) / (60*60);
+
  min = (mUnixTime % (60*60)) / 60;
+
  sec = (mUnixTime % 60);
+

+
  if( yr>=1980 ){
+
    pCds->mDate = (u16)(day + (mon << 5) + ((yr-1980) << 9));
+
    pCds->mTime = (u16)(sec/2 + (min<<5) + (hr<<11));
+
  }else{
+
    pCds->mDate = pCds->mTime = 0;
+
  }
+

+
  assert( mUnixTime<315507600 
+
       || mUnixTime==zipfileMtime(pCds) 
+
       || ((mUnixTime % 2) && mUnixTime-1==zipfileMtime(pCds)) 
+
       /* || (mUnixTime % 2) */
+
  );
+
}

/*
-
** The seriesConnect() method is invoked to create a new
-
** series_vtab that describes the generate_series virtual table.
-
**
-
** Think of this routine as the constructor for series_vtab objects.
-
**
-
** All this routine needs to do is:
-
**
-
**    (1) Allocate the series_vtab object and initialize all fields.
+
** If aBlob is not NULL, then it is a pointer to a buffer (nBlob bytes in
+
** size) containing an entire zip archive image. Or, if aBlob is NULL,
+
** then pFile is a file-handle open on a zip file. In either case, this
+
** function creates a ZipfileEntry object based on the zip archive entry
+
** for which the CDS record is at offset iOff.
**
-
**    (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the
-
**        result set of queries against generate_series will look like.
+
** If successful, SQLITE_OK is returned and (*ppEntry) set to point to
+
** the new object. Otherwise, an SQLite error code is returned and the
+
** final value of (*ppEntry) undefined.
*/
-
static int seriesConnect(
-
  sqlite3 *db,
-
  void *pUnused,
-
  int argcUnused, const char *const*argvUnused,
-
  sqlite3_vtab **ppVtab,
-
  char **pzErrUnused
+
static int zipfileGetEntry(
+
  ZipfileTab *pTab,               /* Store any error message here */
+
  const u8 *aBlob,                /* Pointer to in-memory file image */
+
  int nBlob,                      /* Size of aBlob[] in bytes */
+
  FILE *pFile,                    /* If aBlob==0, read from this file */
+
  i64 iOff,                       /* Offset of CDS record */
+
  ZipfileEntry **ppEntry          /* OUT: Pointer to new object */
){
-
  sqlite3_vtab *pNew;
-
  int rc;
+
  u8 *aRead;
+
  char **pzErr = &pTab->base.zErrMsg;
+
  int rc = SQLITE_OK;

-
/* Column numbers */
-
#define SERIES_COLUMN_VALUE 0
-
#define SERIES_COLUMN_START 1
-
#define SERIES_COLUMN_STOP  2
-
#define SERIES_COLUMN_STEP  3
+
  if( aBlob==0 ){
+
    aRead = pTab->aBuffer;
+
    rc = zipfileReadData(pFile, aRead, ZIPFILE_CDS_FIXED_SZ, iOff, pzErr);
+
  }else{
+
    aRead = (u8*)&aBlob[iOff];
+
  }

-
  (void)pUnused;
-
  (void)argcUnused;
-
  (void)argvUnused;
-
  (void)pzErrUnused;
-
  rc = sqlite3_declare_vtab(db,
-
     "CREATE TABLE x(value,start hidden,stop hidden,step hidden)");
  if( rc==SQLITE_OK ){
-
    pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) );
-
    if( pNew==0 ) return SQLITE_NOMEM;
-
    memset(pNew, 0, sizeof(*pNew));
-
    sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
+
    sqlite3_int64 nAlloc;
+
    ZipfileEntry *pNew;
+

+
    int nFile = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF]);
+
    int nExtra = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+2]);
+
    nExtra += zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+4]);
+

+
    nAlloc = sizeof(ZipfileEntry) + nExtra;
+
    if( aBlob ){
+
      nAlloc += zipfileGetU32(&aRead[ZIPFILE_CDS_SZCOMPRESSED_OFF]);
+
    }
+

+
    pNew = (ZipfileEntry*)sqlite3_malloc64(nAlloc);
+
    if( pNew==0 ){
+
      rc = SQLITE_NOMEM;
+
    }else{
+
      memset(pNew, 0, sizeof(ZipfileEntry));
+
      rc = zipfileReadCDS(aRead, &pNew->cds);
+
      if( rc!=SQLITE_OK ){
+
        *pzErr = sqlite3_mprintf("failed to read CDS at offset %lld", iOff);
+
      }else if( aBlob==0 ){
+
        rc = zipfileReadData(
+
            pFile, aRead, nExtra+nFile, iOff+ZIPFILE_CDS_FIXED_SZ, pzErr
+
        );
+
      }else{
+
        aRead = (u8*)&aBlob[iOff + ZIPFILE_CDS_FIXED_SZ];
+
      }
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      u32 *pt = &pNew->mUnixTime;
+
      pNew->cds.zFile = sqlite3_mprintf("%.*s", nFile, aRead); 
+
      pNew->aExtra = (u8*)&pNew[1];
+
      memcpy(pNew->aExtra, &aRead[nFile], nExtra);
+
      if( pNew->cds.zFile==0 ){
+
        rc = SQLITE_NOMEM;
+
      }else if( 0==zipfileScanExtra(&aRead[nFile], pNew->cds.nExtra, pt) ){
+
        pNew->mUnixTime = zipfileMtime(&pNew->cds);
+
      }
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      static const int szFix = ZIPFILE_LFH_FIXED_SZ;
+
      ZipfileLFH lfh;
+
      if( pFile ){
+
        rc = zipfileReadData(pFile, aRead, szFix, pNew->cds.iOffset, pzErr);
+
      }else{
+
        aRead = (u8*)&aBlob[pNew->cds.iOffset];
+
      }
+

+
      if( rc==SQLITE_OK ) rc = zipfileReadLFH(aRead, &lfh);
+
      if( rc==SQLITE_OK ){
+
        pNew->iDataOff =  pNew->cds.iOffset + ZIPFILE_LFH_FIXED_SZ;
+
        pNew->iDataOff += lfh.nFile + lfh.nExtra;
+
        if( aBlob && pNew->cds.szCompressed ){
+
          pNew->aData = &pNew->aExtra[nExtra];
+
          memcpy(pNew->aData, &aBlob[pNew->iDataOff], pNew->cds.szCompressed);
+
        }
+
      }else{
+
        *pzErr = sqlite3_mprintf("failed to read LFH at offset %d", 
+
            (int)pNew->cds.iOffset
+
        );
+
      }
+
    }
+

+
    if( rc!=SQLITE_OK ){
+
      zipfileEntryFree(pNew);
+
    }else{
+
      *ppEntry = pNew;
+
    }
+
  }
+

+
  return rc;
+
}
+

+
/*
+
** Advance an ZipfileCsr to its next row of output.
+
*/
+
static int zipfileNext(sqlite3_vtab_cursor *cur){
+
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
+
  int rc = SQLITE_OK;
+

+
  if( pCsr->pFile ){
+
    i64 iEof = pCsr->eocd.iOffset + pCsr->eocd.nSize;
+
    zipfileEntryFree(pCsr->pCurrent);
+
    pCsr->pCurrent = 0;
+
    if( pCsr->iNextOff>=iEof ){
+
      pCsr->bEof = 1;
+
    }else{
+
      ZipfileEntry *p = 0;
+
      ZipfileTab *pTab = (ZipfileTab*)(cur->pVtab);
+
      rc = zipfileGetEntry(pTab, 0, 0, pCsr->pFile, pCsr->iNextOff, &p);
+
      if( rc==SQLITE_OK ){
+
        pCsr->iNextOff += ZIPFILE_CDS_FIXED_SZ;
+
        pCsr->iNextOff += (int)p->cds.nExtra + p->cds.nFile + p->cds.nComment;
+
      }
+
      pCsr->pCurrent = p;
+
    }
+
  }else{
+
    if( !pCsr->bNoop ){
+
      pCsr->pCurrent = pCsr->pCurrent->pNext;
+
    }
+
    if( pCsr->pCurrent==0 ){
+
      pCsr->bEof = 1;
+
    }
  }
+

+
  pCsr->bNoop = 0;
  return rc;
}

-
/*
-
** This method is the destructor for series_cursor objects.
-
*/
-
static int seriesDisconnect(sqlite3_vtab *pVtab){
-
  sqlite3_free(pVtab);
-
  return SQLITE_OK;
+
static void zipfileFree(void *p) { 
+
  sqlite3_free(p); 
}

/*
-
** Constructor for a new series_cursor object.
+
** Buffer aIn (size nIn bytes) contains compressed data. Uncompressed, the
+
** size is nOut bytes. This function uncompresses the data and sets the
+
** return value in context pCtx to the result (a blob).
+
**
+
** If an error occurs, an error code is left in pCtx instead.
*/
-
static int seriesOpen(sqlite3_vtab *pUnused, sqlite3_vtab_cursor **ppCursor){
-
  series_cursor *pCur;
-
  (void)pUnused;
-
  pCur = sqlite3_malloc( sizeof(*pCur) );
-
  if( pCur==0 ) return SQLITE_NOMEM;
-
  memset(pCur, 0, sizeof(*pCur));
-
  *ppCursor = &pCur->base;
-
  return SQLITE_OK;
+
static void zipfileInflate(
+
  sqlite3_context *pCtx,          /* Store result here */
+
  const u8 *aIn,                  /* Compressed data */
+
  int nIn,                        /* Size of buffer aIn[] in bytes */
+
  int nOut                        /* Expected output size */
+
){
+
  u8 *aRes = sqlite3_malloc(nOut);
+
  if( aRes==0 ){
+
    sqlite3_result_error_nomem(pCtx);
+
  }else{
+
    int err;
+
    z_stream str;
+
    memset(&str, 0, sizeof(str));
+

+
    str.next_in = (Byte*)aIn;
+
    str.avail_in = nIn;
+
    str.next_out = (Byte*)aRes;
+
    str.avail_out = nOut;
+

+
    err = inflateInit2(&str, -15);
+
    if( err!=Z_OK ){
+
      zipfileCtxErrorMsg(pCtx, "inflateInit2() failed (%d)", err);
+
    }else{
+
      err = inflate(&str, Z_NO_FLUSH);
+
      if( err!=Z_STREAM_END ){
+
        zipfileCtxErrorMsg(pCtx, "inflate() failed (%d)", err);
+
      }else{
+
        sqlite3_result_blob(pCtx, aRes, nOut, zipfileFree);
+
        aRes = 0;
+
      }
+
    }
+
    sqlite3_free(aRes);
+
    inflateEnd(&str);
+
  }
}

/*
-
** Destructor for a series_cursor.
+
** Buffer aIn (size nIn bytes) contains uncompressed data. This function
+
** compresses it and sets (*ppOut) to point to a buffer containing the
+
** compressed data. The caller is responsible for eventually calling
+
** sqlite3_free() to release buffer (*ppOut). Before returning, (*pnOut) 
+
** is set to the size of buffer (*ppOut) in bytes.
+
**
+
** If no error occurs, SQLITE_OK is returned. Otherwise, an SQLite error
+
** code is returned and an error message left in virtual-table handle
+
** pTab. The values of (*ppOut) and (*pnOut) are left unchanged in this
+
** case.
*/
-
static int seriesClose(sqlite3_vtab_cursor *cur){
-
  sqlite3_free(cur);
-
  return SQLITE_OK;
-
}
+
static int zipfileDeflate(
+
  const u8 *aIn, int nIn,         /* Input */
+
  u8 **ppOut, int *pnOut,         /* Output */
+
  char **pzErr                    /* OUT: Error message */
+
){
+
  int rc = SQLITE_OK;
+
  sqlite3_int64 nAlloc;
+
  z_stream str;
+
  u8 *aOut;

+
  memset(&str, 0, sizeof(str));
+
  str.next_in = (Bytef*)aIn;
+
  str.avail_in = nIn;
+
  deflateInit2(&str, 9, Z_DEFLATED, -15, 8, Z_DEFAULT_STRATEGY);

-
/*
-
** Advance a series_cursor to its next row of output.
-
*/
-
static int seriesNext(sqlite3_vtab_cursor *cur){
-
  series_cursor *pCur = (series_cursor*)cur;
-
  if( pCur->isDesc ){
-
    pCur->iValue -= pCur->iStep;
+
  nAlloc = deflateBound(&str, nIn);
+
  aOut = (u8*)sqlite3_malloc64(nAlloc);
+
  if( aOut==0 ){
+
    rc = SQLITE_NOMEM;
  }else{
-
    pCur->iValue += pCur->iStep;
+
    int res;
+
    str.next_out = aOut;
+
    str.avail_out = nAlloc;
+
    res = deflate(&str, Z_FINISH);
+
    if( res==Z_STREAM_END ){
+
      *ppOut = aOut;
+
      *pnOut = (int)str.total_out;
+
    }else{
+
      sqlite3_free(aOut);
+
      *pzErr = sqlite3_mprintf("zipfile: deflate() error");
+
      rc = SQLITE_ERROR;
+
    }
+
    deflateEnd(&str);
  }
-
  pCur->iRowid++;
-
  return SQLITE_OK;
+

+
  return rc;
}

+

/*
** Return values of columns for the row at which the series_cursor
** is currently pointing.
*/
-
static int seriesColumn(
+
static int zipfileColumn(
  sqlite3_vtab_cursor *cur,   /* The cursor */
  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
  int i                       /* Which column to return */
){
-
  series_cursor *pCur = (series_cursor*)cur;
-
  sqlite3_int64 x = 0;
+
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
+
  ZipfileCDS *pCDS = &pCsr->pCurrent->cds;
+
  int rc = SQLITE_OK;
  switch( i ){
-
    case SERIES_COLUMN_START:  x = pCur->mnValue; break;
-
    case SERIES_COLUMN_STOP:   x = pCur->mxValue; break;
-
    case SERIES_COLUMN_STEP:   x = pCur->iStep;   break;
-
    default:                   x = pCur->iValue;  break;
-
  }
-
  sqlite3_result_int64(ctx, x);
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Return the rowid for the current row. In this implementation, the
-
** first row returned is assigned rowid value 1, and each subsequent
-
** row a value 1 more than that of the previous.
-
*/
-
static int seriesRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
-
  series_cursor *pCur = (series_cursor*)cur;
-
  *pRowid = pCur->iRowid;
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Return TRUE if the cursor has been moved off of the last
-
** row of output.
-
*/
-
static int seriesEof(sqlite3_vtab_cursor *cur){
-
  series_cursor *pCur = (series_cursor*)cur;
-
  if( pCur->isDesc ){
-
    return pCur->iValue < pCur->mnValue;
-
  }else{
-
    return pCur->iValue > pCur->mxValue;
-
  }
-
}
-

-
/* True to cause run-time checking of the start=, stop=, and/or step= 
-
** parameters.  The only reason to do this is for testing the
-
** constraint checking logic for virtual tables in the SQLite core.
-
*/
-
#ifndef SQLITE_SERIES_CONSTRAINT_VERIFY
-
# define SQLITE_SERIES_CONSTRAINT_VERIFY 0
-
#endif
-

-
/*
-
** This method is called to "rewind" the series_cursor object back
-
** to the first row of output.  This method is always called at least
-
** once prior to any call to seriesColumn() or seriesRowid() or 
-
** seriesEof().
-
**
-
** The query plan selected by seriesBestIndex is passed in the idxNum
-
** parameter.  (idxStr is not used in this implementation.)  idxNum
-
** is a bitmask showing which constraints are available:
-
**
-
**    1:    start=VALUE
-
**    2:    stop=VALUE
-
**    4:    step=VALUE
-
**
-
** Also, if bit 8 is set, that means that the series should be output
-
** in descending order rather than in ascending order.  If bit 16 is
-
** set, then output must appear in ascending order.
-
**
-
** This routine should initialize the cursor and position it so that it
-
** is pointing at the first row, or pointing off the end of the table
-
** (so that seriesEof() will return true) if the table is empty.
-
*/
-
static int seriesFilter(
-
  sqlite3_vtab_cursor *pVtabCursor, 
-
  int idxNum, const char *idxStrUnused,
-
  int argc, sqlite3_value **argv
-
){
-
  series_cursor *pCur = (series_cursor *)pVtabCursor;
-
  int i = 0;
-
  (void)idxStrUnused;
-
  if( idxNum & 1 ){
-
    pCur->mnValue = sqlite3_value_int64(argv[i++]);
-
  }else{
-
    pCur->mnValue = 0;
-
  }
-
  if( idxNum & 2 ){
-
    pCur->mxValue = sqlite3_value_int64(argv[i++]);
-
  }else{
-
    pCur->mxValue = 0xffffffff;
-
  }
-
  if( idxNum & 4 ){
-
    pCur->iStep = sqlite3_value_int64(argv[i++]);
-
    if( pCur->iStep==0 ){
-
      pCur->iStep = 1;
-
    }else if( pCur->iStep<0 ){
-
      pCur->iStep = -pCur->iStep;
-
      if( (idxNum & 16)==0 ) idxNum |= 8;
+
    case 0:   /* name */
+
      sqlite3_result_text(ctx, pCDS->zFile, -1, SQLITE_TRANSIENT);
+
      break;
+
    case 1:   /* mode */
+
      /* TODO: Whether or not the following is correct surely depends on
+
      ** the platform on which the archive was created.  */
+
      sqlite3_result_int(ctx, pCDS->iExternalAttr >> 16);
+
      break;
+
    case 2: { /* mtime */
+
      sqlite3_result_int64(ctx, pCsr->pCurrent->mUnixTime);
+
      break;
    }
-
  }else{
-
    pCur->iStep = 1;
-
  }
-
  for(i=0; i<argc; i++){
-
    if( sqlite3_value_type(argv[i])==SQLITE_NULL ){
-
      /* If any of the constraints have a NULL value, then return no rows.
-
      ** See ticket https://www.sqlite.org/src/info/fac496b61722daf2 */
-
      pCur->mnValue = 1;
-
      pCur->mxValue = 0;
+
    case 3: { /* sz */
+
      if( sqlite3_vtab_nochange(ctx)==0 ){
+
        sqlite3_result_int64(ctx, pCDS->szUncompressed);
+
      }
+
      break;
+
    }
+
    case 4:   /* rawdata */
+
      if( sqlite3_vtab_nochange(ctx) ) break;
+
    case 5: { /* data */
+
      if( i==4 || pCDS->iCompression==0 || pCDS->iCompression==8 ){
+
        int sz = pCDS->szCompressed;
+
        int szFinal = pCDS->szUncompressed;
+
        if( szFinal>0 ){
+
          u8 *aBuf;
+
          u8 *aFree = 0;
+
          if( pCsr->pCurrent->aData ){
+
            aBuf = pCsr->pCurrent->aData;
+
          }else{
+
            aBuf = aFree = sqlite3_malloc64(sz);
+
            if( aBuf==0 ){
+
              rc = SQLITE_NOMEM;
+
            }else{
+
              FILE *pFile = pCsr->pFile;
+
              if( pFile==0 ){
+
                pFile = ((ZipfileTab*)(pCsr->base.pVtab))->pWriteFd;
+
              }
+
              rc = zipfileReadData(pFile, aBuf, sz, pCsr->pCurrent->iDataOff,
+
                  &pCsr->base.pVtab->zErrMsg
+
              );
+
            }
+
          }
+
          if( rc==SQLITE_OK ){
+
            if( i==5 && pCDS->iCompression ){
+
              zipfileInflate(ctx, aBuf, sz, szFinal);
+
            }else{
+
              sqlite3_result_blob(ctx, aBuf, sz, SQLITE_TRANSIENT);
+
            }
+
          }
+
          sqlite3_free(aFree);
+
        }else{
+
          /* Figure out if this is a directory or a zero-sized file. Consider
+
          ** it to be a directory either if the mode suggests so, or if
+
          ** the final character in the name is '/'.  */
+
          u32 mode = pCDS->iExternalAttr >> 16;
+
          if( !(mode & S_IFDIR) && pCDS->zFile[pCDS->nFile-1]!='/' ){
+
            sqlite3_result_blob(ctx, "", 0, SQLITE_STATIC);
+
          }
+
        }
+
      }
      break;
    }
+
    case 6:   /* method */
+
      sqlite3_result_int(ctx, pCDS->iCompression);
+
      break;
+
    default:  /* z */
+
      assert( i==7 );
+
      sqlite3_result_int64(ctx, pCsr->iId);
+
      break;
  }
-
  if( idxNum & 8 ){
-
    pCur->isDesc = 1;
-
    pCur->iValue = pCur->mxValue;
-
    if( pCur->iStep>0 ){
-
      pCur->iValue -= (pCur->mxValue - pCur->mnValue)%pCur->iStep;
-
    }
-
  }else{
-
    pCur->isDesc = 0;
-
    pCur->iValue = pCur->mnValue;
-
  }
-
  pCur->iRowid = 1;
-
  return SQLITE_OK;
+

+
  return rc;
}

/*
-
** SQLite will invoke this method one or more times while planning a query
-
** that uses the generate_series virtual table.  This routine needs to create
-
** a query plan for each invocation and compute an estimated cost for that
-
** plan.
-
**
-
** In this implementation idxNum is used to represent the
-
** query plan.  idxStr is unused.
-
**
-
** The query plan is represented by bits in idxNum:
+
** Return TRUE if the cursor is at EOF.
+
*/
+
static int zipfileEof(sqlite3_vtab_cursor *cur){
+
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
+
  return pCsr->bEof;
+
}
+

+
/*
+
** If aBlob is not NULL, then it points to a buffer nBlob bytes in size
+
** containing an entire zip archive image. Or, if aBlob is NULL, then pFile
+
** is guaranteed to be a file-handle open on a zip file.
**
-
**  (1)  start = $value  -- constraint exists
-
**  (2)  stop = $value   -- constraint exists
-
**  (4)  step = $value   -- constraint exists
-
**  (8)  output in descending order
+
** This function attempts to locate the EOCD record within the zip archive
+
** and populate *pEOCD with the results of decoding it. SQLITE_OK is
+
** returned if successful. Otherwise, an SQLite error code is returned and
+
** an English language error message may be left in virtual-table pTab.
*/
-
static int seriesBestIndex(
-
  sqlite3_vtab *pVTab,
-
  sqlite3_index_info *pIdxInfo
+
static int zipfileReadEOCD(
+
  ZipfileTab *pTab,               /* Return errors here */
+
  const u8 *aBlob,                /* Pointer to in-memory file image */
+
  int nBlob,                      /* Size of aBlob[] in bytes */
+
  FILE *pFile,                    /* Read from this file if aBlob==0 */
+
  ZipfileEOCD *pEOCD              /* Object to populate */
){
-
  int i, j;              /* Loop over constraints */
-
  int idxNum = 0;        /* The query plan bitmask */
-
  int bStartSeen = 0;    /* EQ constraint seen on the START column */
-
  int unusableMask = 0;  /* Mask of unusable constraints */
-
  int nArg = 0;          /* Number of arguments that seriesFilter() expects */
-
  int aIdx[3];           /* Constraints on start, stop, and step */
-
  const struct sqlite3_index_constraint *pConstraint;
-

-
  /* This implementation assumes that the start, stop, and step columns
-
  ** are the last three columns in the virtual table. */
-
  assert( SERIES_COLUMN_STOP == SERIES_COLUMN_START+1 );
-
  assert( SERIES_COLUMN_STEP == SERIES_COLUMN_START+2 );
+
  u8 *aRead = pTab->aBuffer;      /* Temporary buffer */
+
  int nRead;                      /* Bytes to read from file */
+
  int rc = SQLITE_OK;

-
  aIdx[0] = aIdx[1] = aIdx[2] = -1;
-
  pConstraint = pIdxInfo->aConstraint;
-
  for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){
-
    int iCol;    /* 0 for start, 1 for stop, 2 for step */
-
    int iMask;   /* bitmask for those column */
-
    if( pConstraint->iColumn<SERIES_COLUMN_START ) continue;
-
    iCol = pConstraint->iColumn - SERIES_COLUMN_START;
-
    assert( iCol>=0 && iCol<=2 );
-
    iMask = 1 << iCol;
-
    if( iCol==0 ) bStartSeen = 1;
-
    if( pConstraint->usable==0 ){
-
      unusableMask |=  iMask;
-
      continue;
-
    }else if( pConstraint->op==SQLITE_INDEX_CONSTRAINT_EQ ){
-
      idxNum |= iMask;
-
      aIdx[iCol] = i;
-
    }
-
  }
-
  for(i=0; i<3; i++){
-
    if( (j = aIdx[i])>=0 ){
-
      pIdxInfo->aConstraintUsage[j].argvIndex = ++nArg;
-
      pIdxInfo->aConstraintUsage[j].omit = !SQLITE_SERIES_CONSTRAINT_VERIFY;
-
    }
-
  }
-
  /* The current generate_column() implementation requires at least one
-
  ** argument (the START value).  Legacy versions assumed START=0 if the
-
  ** first argument was omitted.  Compile with -DZERO_ARGUMENT_GENERATE_SERIES
-
  ** to obtain the legacy behavior */
-
#ifndef ZERO_ARGUMENT_GENERATE_SERIES
-
  if( !bStartSeen ){
-
    sqlite3_free(pVTab->zErrMsg);
-
    pVTab->zErrMsg = sqlite3_mprintf(
-
        "first argument to \"generate_series()\" missing or unusable");
-
    return SQLITE_ERROR;
-
  }
-
#endif
-
  if( (unusableMask & ~idxNum)!=0 ){
-
    /* The start, stop, and step columns are inputs.  Therefore if there
-
    ** are unusable constraints on any of start, stop, or step then
-
    ** this plan is unusable */
-
    return SQLITE_CONSTRAINT;
-
  }
-
  if( (idxNum & 3)==3 ){
-
    /* Both start= and stop= boundaries are available.  This is the 
-
    ** the preferred case */
-
    pIdxInfo->estimatedCost = (double)(2 - ((idxNum&4)!=0));
-
    pIdxInfo->estimatedRows = 1000;
-
    if( pIdxInfo->nOrderBy>=1 && pIdxInfo->aOrderBy[0].iColumn==0 ){
-
      if( pIdxInfo->aOrderBy[0].desc ){
-
        idxNum |= 8;
-
      }else{
-
        idxNum |= 16;
-
      }
-
      pIdxInfo->orderByConsumed = 1;
+
  memset(pEOCD, 0, sizeof(ZipfileEOCD));
+
  if( aBlob==0 ){
+
    i64 iOff;                     /* Offset to read from */
+
    i64 szFile;                   /* Total size of file in bytes */
+
    fseek(pFile, 0, SEEK_END);
+
    szFile = (i64)ftell(pFile);
+
    if( szFile==0 ){
+
      return SQLITE_OK;
    }
+
    nRead = (int)(MIN(szFile, ZIPFILE_BUFFER_SIZE));
+
    iOff = szFile - nRead;
+
    rc = zipfileReadData(pFile, aRead, nRead, iOff, &pTab->base.zErrMsg);
  }else{
-
    /* If either boundary is missing, we have to generate a huge span
-
    ** of numbers.  Make this case very expensive so that the query
-
    ** planner will work hard to avoid it. */
-
    pIdxInfo->estimatedRows = 2147483647;
+
    nRead = (int)(MIN(nBlob, ZIPFILE_BUFFER_SIZE));
+
    aRead = (u8*)&aBlob[nBlob-nRead];
  }
-
  pIdxInfo->idxNum = idxNum;
-
  return SQLITE_OK;
-
}
-

-
/*
-
** This following structure defines all the methods for the 
-
** generate_series virtual table.
-
*/
-
static sqlite3_module seriesModule = {
-
  0,                         /* iVersion */
-
  0,                         /* xCreate */
-
  seriesConnect,             /* xConnect */
-
  seriesBestIndex,           /* xBestIndex */
-
  seriesDisconnect,          /* xDisconnect */
-
  0,                         /* xDestroy */
-
  seriesOpen,                /* xOpen - open a cursor */
-
  seriesClose,               /* xClose - close a cursor */
-
  seriesFilter,              /* xFilter - configure scan constraints */
-
  seriesNext,                /* xNext - advance a cursor */
-
  seriesEof,                 /* xEof - check for end of scan */
-
  seriesColumn,              /* xColumn - read data */
-
  seriesRowid,               /* xRowid - read data */
-
  0,                         /* xUpdate */
-
  0,                         /* xBegin */
-
  0,                         /* xSync */
-
  0,                         /* xCommit */
-
  0,                         /* xRollback */
-
  0,                         /* xFindMethod */
-
  0,                         /* xRename */
-
  0,                         /* xSavepoint */
-
  0,                         /* xRelease */
-
  0,                         /* xRollbackTo */
-
  0                          /* xShadowName */
-
};

-
#endif /* SQLITE_OMIT_VIRTUALTABLE */
+
  if( rc==SQLITE_OK ){
+
    int i;

-
#ifdef _WIN32
+
    /* Scan backwards looking for the signature bytes */
+
    for(i=nRead-20; i>=0; i--){
+
      if( aRead[i]==0x50 && aRead[i+1]==0x4b 
+
       && aRead[i+2]==0x05 && aRead[i+3]==0x06 
+
      ){
+
        break;
+
      }
+
    }
+
    if( i<0 ){
+
      pTab->base.zErrMsg = sqlite3_mprintf(
+
          "cannot find end of central directory record"
+
      );
+
      return SQLITE_ERROR;
+
    }

-
#endif
-
int sqlite3_series_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  int rc = SQLITE_OK;
-
  SQLITE_EXTENSION_INIT2(pApi);
-
#ifndef SQLITE_OMIT_VIRTUALTABLE
-
  if( sqlite3_libversion_number()<3008012 && pzErrMsg!=0 ){
-
    *pzErrMsg = sqlite3_mprintf(
-
        "generate_series() requires SQLite 3.8.12 or later");
-
    return SQLITE_ERROR;
+
    aRead += i+4;
+
    pEOCD->iDisk = zipfileRead16(aRead);
+
    pEOCD->iFirstDisk = zipfileRead16(aRead);
+
    pEOCD->nEntry = zipfileRead16(aRead);
+
    pEOCD->nEntryTotal = zipfileRead16(aRead);
+
    pEOCD->nSize = zipfileRead32(aRead);
+
    pEOCD->iOffset = zipfileRead32(aRead);
  }
-
  rc = sqlite3_create_module(db, "generate_series", &seriesModule, 0);
-
#endif
+

  return rc;
}

-
/************************* End ../ext/misc/series.c ********************/
-
/************************* Begin ../ext/misc/regexp.c ******************/
/*
-
** 2012-11-13
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** The code in this file implements a compact but reasonably
-
** efficient regular-expression matcher for posix extended regular
-
** expressions against UTF8 text.
-
**
-
** This file is an SQLite extension.  It registers a single function
-
** named "regexp(A,B)" where A is the regular expression and B is the
-
** string to be matched.  By registering this function, SQLite will also
-
** then implement the "B regexp A" operator.  Note that with the function
-
** the regular expression comes first, but with the operator it comes
-
** second.
-
**
-
**  The following regular expression syntax is supported:
-
**
-
**     X*      zero or more occurrences of X
-
**     X+      one or more occurrences of X
-
**     X?      zero or one occurrences of X
-
**     X{p,q}  between p and q occurrences of X
-
**     (X)     match X
-
**     X|Y     X or Y
-
**     ^X      X occurring at the beginning of the string
-
**     X$      X occurring at the end of the string
-
**     .       Match any single character
-
**     \c      Character c where c is one of \{}()[]|*+?.
-
**     \c      C-language escapes for c in afnrtv.  ex: \t or \n
-
**     \uXXXX  Where XXXX is exactly 4 hex digits, unicode value XXXX
-
**     \xXX    Where XX is exactly 2 hex digits, unicode value XX
-
**     [abc]   Any single character from the set abc
-
**     [^abc]  Any single character not in the set abc
-
**     [a-z]   Any single character in the range a-z
-
**     [^a-z]  Any single character not in the range a-z
-
**     \b      Word boundary
-
**     \w      Word character.  [A-Za-z0-9_]
-
**     \W      Non-word character
-
**     \d      Digit
-
**     \D      Non-digit
-
**     \s      Whitespace character
-
**     \S      Non-whitespace character
-
**
-
** A nondeterministic finite automaton (NFA) is used for matching, so the
-
** performance is bounded by O(N*M) where N is the size of the regular
-
** expression and M is the size of the input string.  The matcher never
-
** exhibits exponential behavior.  Note that the X{p,q} operator expands
-
** to p copies of X following by q-p copies of X? and that the size of the
-
** regular expression in the O(N*M) performance bound is computed after
-
** this expansion.
+
** Add object pNew to the linked list that begins at ZipfileTab.pFirstEntry 
+
** and ends with pLastEntry. If argument pBefore is NULL, then pNew is added
+
** to the end of the list. Otherwise, it is added to the list immediately
+
** before pBefore (which is guaranteed to be a part of said list).
*/
-
#include <string.h>
-
#include <stdlib.h>
-
/* #include "sqlite3ext.h" */
-
SQLITE_EXTENSION_INIT1
+
static void zipfileAddEntry(
+
  ZipfileTab *pTab, 
+
  ZipfileEntry *pBefore, 
+
  ZipfileEntry *pNew
+
){
+
  assert( (pTab->pFirstEntry==0)==(pTab->pLastEntry==0) );
+
  assert( pNew->pNext==0 );
+
  if( pBefore==0 ){
+
    if( pTab->pFirstEntry==0 ){
+
      pTab->pFirstEntry = pTab->pLastEntry = pNew;
+
    }else{
+
      assert( pTab->pLastEntry->pNext==0 );
+
      pTab->pLastEntry->pNext = pNew;
+
      pTab->pLastEntry = pNew;
+
    }
+
  }else{
+
    ZipfileEntry **pp;
+
    for(pp=&pTab->pFirstEntry; *pp!=pBefore; pp=&((*pp)->pNext));
+
    pNew->pNext = pBefore;
+
    *pp = pNew;
+
  }
+
}

-
/*
-
** The following #defines change the names of some functions implemented in
-
** this file to prevent name collisions with C-library functions of the
-
** same name.
-
*/
-
#define re_match   sqlite3re_match
-
#define re_compile sqlite3re_compile
-
#define re_free    sqlite3re_free
+
static int zipfileLoadDirectory(ZipfileTab *pTab, const u8 *aBlob, int nBlob){
+
  ZipfileEOCD eocd;
+
  int rc;
+
  int i;
+
  i64 iOff;

-
/* The end-of-input character */
-
#define RE_EOF            0    /* End of input */
+
  rc = zipfileReadEOCD(pTab, aBlob, nBlob, pTab->pWriteFd, &eocd);
+
  iOff = eocd.iOffset;
+
  for(i=0; rc==SQLITE_OK && i<eocd.nEntry; i++){
+
    ZipfileEntry *pNew = 0;
+
    rc = zipfileGetEntry(pTab, aBlob, nBlob, pTab->pWriteFd, iOff, &pNew);

-
/* The NFA is implemented as sequence of opcodes taken from the following
-
** set.  Each opcode has a single integer argument.
+
    if( rc==SQLITE_OK ){
+
      zipfileAddEntry(pTab, 0, pNew);
+
      iOff += ZIPFILE_CDS_FIXED_SZ;
+
      iOff += (int)pNew->cds.nExtra + pNew->cds.nFile + pNew->cds.nComment;
+
    }
+
  }
+
  return rc;
+
}
+

+
/*
+
** xFilter callback.
*/
-
#define RE_OP_MATCH       1    /* Match the one character in the argument */
-
#define RE_OP_ANY         2    /* Match any one character.  (Implements ".") */
-
#define RE_OP_ANYSTAR     3    /* Special optimized version of .* */
-
#define RE_OP_FORK        4    /* Continue to both next and opcode at iArg */
-
#define RE_OP_GOTO        5    /* Jump to opcode at iArg */
-
#define RE_OP_ACCEPT      6    /* Halt and indicate a successful match */
-
#define RE_OP_CC_INC      7    /* Beginning of a [...] character class */
-
#define RE_OP_CC_EXC      8    /* Beginning of a [^...] character class */
-
#define RE_OP_CC_VALUE    9    /* Single value in a character class */
-
#define RE_OP_CC_RANGE   10    /* Range of values in a character class */
-
#define RE_OP_WORD       11    /* Perl word character [A-Za-z0-9_] */
-
#define RE_OP_NOTWORD    12    /* Not a perl word character */
-
#define RE_OP_DIGIT      13    /* digit:  [0-9] */
-
#define RE_OP_NOTDIGIT   14    /* Not a digit */
-
#define RE_OP_SPACE      15    /* space:  [ \t\n\r\v\f] */
-
#define RE_OP_NOTSPACE   16    /* Not a digit */
-
#define RE_OP_BOUNDARY   17    /* Boundary between word and non-word */
+
static int zipfileFilter(
+
  sqlite3_vtab_cursor *cur, 
+
  int idxNum, const char *idxStr,
+
  int argc, sqlite3_value **argv
+
){
+
  ZipfileTab *pTab = (ZipfileTab*)cur->pVtab;
+
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
+
  const char *zFile = 0;          /* Zip file to scan */
+
  int rc = SQLITE_OK;             /* Return Code */
+
  int bInMemory = 0;              /* True for an in-memory zipfile */

-
/* Each opcode is a "state" in the NFA */
-
typedef unsigned short ReStateNumber;
+
  zipfileResetCursor(pCsr);

-
/* Because this is an NFA and not a DFA, multiple states can be active at
-
** once.  An instance of the following object records all active states in
-
** the NFA.  The implementation is optimized for the common case where the
-
** number of actives states is small.
-
*/
-
typedef struct ReStateSet {
-
  unsigned nState;            /* Number of current states */
-
  ReStateNumber *aState;      /* Current states */
-
} ReStateSet;
+
  if( pTab->zFile ){
+
    zFile = pTab->zFile;
+
  }else if( idxNum==0 ){
+
    zipfileCursorErr(pCsr, "zipfile() function requires an argument");
+
    return SQLITE_ERROR;
+
  }else if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){
+
    static const u8 aEmptyBlob = 0;
+
    const u8 *aBlob = (const u8*)sqlite3_value_blob(argv[0]);
+
    int nBlob = sqlite3_value_bytes(argv[0]);
+
    assert( pTab->pFirstEntry==0 );
+
    if( aBlob==0 ){
+
      aBlob = &aEmptyBlob;
+
      nBlob = 0;
+
    }
+
    rc = zipfileLoadDirectory(pTab, aBlob, nBlob);
+
    pCsr->pFreeEntry = pTab->pFirstEntry;
+
    pTab->pFirstEntry = pTab->pLastEntry = 0;
+
    if( rc!=SQLITE_OK ) return rc;
+
    bInMemory = 1;
+
  }else{
+
    zFile = (const char*)sqlite3_value_text(argv[0]);
+
  }

-
/* An input string read one character at a time.
-
*/
-
typedef struct ReInput ReInput;
-
struct ReInput {
-
  const unsigned char *z;  /* All text */
-
  int i;                   /* Next byte to read */
-
  int mx;                  /* EOF when i>=mx */
-
};
+
  if( 0==pTab->pWriteFd && 0==bInMemory ){
+
    pCsr->pFile = fopen(zFile, "rb");
+
    if( pCsr->pFile==0 ){
+
      zipfileCursorErr(pCsr, "cannot open file: %s", zFile);
+
      rc = SQLITE_ERROR;
+
    }else{
+
      rc = zipfileReadEOCD(pTab, 0, 0, pCsr->pFile, &pCsr->eocd);
+
      if( rc==SQLITE_OK ){
+
        if( pCsr->eocd.nEntry==0 ){
+
          pCsr->bEof = 1;
+
        }else{
+
          pCsr->iNextOff = pCsr->eocd.iOffset;
+
          rc = zipfileNext(cur);
+
        }
+
      }
+
    }
+
  }else{
+
    pCsr->bNoop = 1;
+
    pCsr->pCurrent = pCsr->pFreeEntry ? pCsr->pFreeEntry : pTab->pFirstEntry;
+
    rc = zipfileNext(cur);
+
  }

-
/* A compiled NFA (or an NFA that is in the process of being compiled) is
-
** an instance of the following object.
+
  return rc;
+
}
+

+
/*
+
** xBestIndex callback.
*/
-
typedef struct ReCompiled ReCompiled;
-
struct ReCompiled {
-
  ReInput sIn;                /* Regular expression text */
-
  const char *zErr;           /* Error message to return */
-
  char *aOp;                  /* Operators for the virtual machine */
-
  int *aArg;                  /* Arguments to each operator */
-
  unsigned (*xNextChar)(ReInput*);  /* Next character function */
-
  unsigned char zInit[12];    /* Initial text to match */
-
  int nInit;                  /* Number of characters in zInit */
-
  unsigned nState;            /* Number of entries in aOp[] and aArg[] */
-
  unsigned nAlloc;            /* Slots allocated for aOp[] and aArg[] */
-
};
+
static int zipfileBestIndex(
+
  sqlite3_vtab *tab,
+
  sqlite3_index_info *pIdxInfo
+
){
+
  int i;
+
  int idx = -1;
+
  int unusable = 0;

-
/* Add a state to the given state set if it is not already there */
-
static void re_add_state(ReStateSet *pSet, int newState){
-
  unsigned i;
-
  for(i=0; i<pSet->nState; i++) if( pSet->aState[i]==newState ) return;
-
  pSet->aState[pSet->nState++] = (ReStateNumber)newState;
+
  for(i=0; i<pIdxInfo->nConstraint; i++){
+
    const struct sqlite3_index_constraint *pCons = &pIdxInfo->aConstraint[i];
+
    if( pCons->iColumn!=ZIPFILE_F_COLUMN_IDX ) continue;
+
    if( pCons->usable==0 ){
+
      unusable = 1;
+
    }else if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ ){
+
      idx = i;
+
    }
+
  }
+
  pIdxInfo->estimatedCost = 1000.0;
+
  if( idx>=0 ){
+
    pIdxInfo->aConstraintUsage[idx].argvIndex = 1;
+
    pIdxInfo->aConstraintUsage[idx].omit = 1;
+
    pIdxInfo->idxNum = 1;
+
  }else if( unusable ){
+
    return SQLITE_CONSTRAINT;
+
  }
+
  return SQLITE_OK;
}

-
/* Extract the next unicode character from *pzIn and return it.  Advance
-
** *pzIn to the first byte past the end of the character returned.  To
-
** be clear:  this routine converts utf8 to unicode.  This routine is 
-
** optimized for the common case where the next character is a single byte.
-
*/
-
static unsigned re_next_char(ReInput *p){
-
  unsigned c;
-
  if( p->i>=p->mx ) return 0;
-
  c = p->z[p->i++];
-
  if( c>=0x80 ){
-
    if( (c&0xe0)==0xc0 && p->i<p->mx && (p->z[p->i]&0xc0)==0x80 ){
-
      c = (c&0x1f)<<6 | (p->z[p->i++]&0x3f);
-
      if( c<0x80 ) c = 0xfffd;
-
    }else if( (c&0xf0)==0xe0 && p->i+1<p->mx && (p->z[p->i]&0xc0)==0x80
-
           && (p->z[p->i+1]&0xc0)==0x80 ){
-
      c = (c&0x0f)<<12 | ((p->z[p->i]&0x3f)<<6) | (p->z[p->i+1]&0x3f);
-
      p->i += 2;
-
      if( c<=0x7ff || (c>=0xd800 && c<=0xdfff) ) c = 0xfffd;
-
    }else if( (c&0xf8)==0xf0 && p->i+3<p->mx && (p->z[p->i]&0xc0)==0x80
-
           && (p->z[p->i+1]&0xc0)==0x80 && (p->z[p->i+2]&0xc0)==0x80 ){
-
      c = (c&0x07)<<18 | ((p->z[p->i]&0x3f)<<12) | ((p->z[p->i+1]&0x3f)<<6)
-
                       | (p->z[p->i+2]&0x3f);
-
      p->i += 3;
-
      if( c<=0xffff || c>0x10ffff ) c = 0xfffd;
-
    }else{
-
      c = 0xfffd;
+
static ZipfileEntry *zipfileNewEntry(const char *zPath){
+
  ZipfileEntry *pNew;
+
  pNew = sqlite3_malloc(sizeof(ZipfileEntry));
+
  if( pNew ){
+
    memset(pNew, 0, sizeof(ZipfileEntry));
+
    pNew->cds.zFile = sqlite3_mprintf("%s", zPath);
+
    if( pNew->cds.zFile==0 ){
+
      sqlite3_free(pNew);
+
      pNew = 0;
    }
  }
-
  return c;
-
}
-
static unsigned re_next_char_nocase(ReInput *p){
-
  unsigned c = re_next_char(p);
-
  if( c>='A' && c<='Z' ) c += 'a' - 'A';
-
  return c;
+
  return pNew;
}

-
/* Return true if c is a perl "word" character:  [A-Za-z0-9_] */
-
static int re_word_char(int c){
-
  return (c>='0' && c<='9') || (c>='a' && c<='z')
-
      || (c>='A' && c<='Z') || c=='_';
-
}
+
static int zipfileSerializeLFH(ZipfileEntry *pEntry, u8 *aBuf){
+
  ZipfileCDS *pCds = &pEntry->cds;
+
  u8 *a = aBuf;

-
/* Return true if c is a "digit" character:  [0-9] */
-
static int re_digit_char(int c){
-
  return (c>='0' && c<='9');
-
}
+
  pCds->nExtra = 9;

-
/* Return true if c is a perl "space" character:  [ \t\r\n\v\f] */
-
static int re_space_char(int c){
-
  return c==' ' || c=='\t' || c=='\n' || c=='\r' || c=='\v' || c=='\f';
-
}
+
  /* Write the LFH itself */
+
  zipfileWrite32(a, ZIPFILE_SIGNATURE_LFH);
+
  zipfileWrite16(a, pCds->iVersionExtract);
+
  zipfileWrite16(a, pCds->flags);
+
  zipfileWrite16(a, pCds->iCompression);
+
  zipfileWrite16(a, pCds->mTime);
+
  zipfileWrite16(a, pCds->mDate);
+
  zipfileWrite32(a, pCds->crc32);
+
  zipfileWrite32(a, pCds->szCompressed);
+
  zipfileWrite32(a, pCds->szUncompressed);
+
  zipfileWrite16(a, (u16)pCds->nFile);
+
  zipfileWrite16(a, pCds->nExtra);
+
  assert( a==&aBuf[ZIPFILE_LFH_FIXED_SZ] );

-
/* Run a compiled regular expression on the zero-terminated input
-
** string zIn[].  Return true on a match and false if there is no match.
-
*/
-
static int re_match(ReCompiled *pRe, const unsigned char *zIn, int nIn){
-
  ReStateSet aStateSet[2], *pThis, *pNext;
-
  ReStateNumber aSpace[100];
-
  ReStateNumber *pToFree;
-
  unsigned int i = 0;
-
  unsigned int iSwap = 0;
-
  int c = RE_EOF+1;
-
  int cPrev = 0;
-
  int rc = 0;
-
  ReInput in;
+
  /* Add the file name */
+
  memcpy(a, pCds->zFile, (int)pCds->nFile);
+
  a += (int)pCds->nFile;

-
  in.z = zIn;
-
  in.i = 0;
-
  in.mx = nIn>=0 ? nIn : (int)strlen((char const*)zIn);
+
  /* The "extra" data */
+
  zipfileWrite16(a, ZIPFILE_EXTRA_TIMESTAMP);
+
  zipfileWrite16(a, 5);
+
  *a++ = 0x01;
+
  zipfileWrite32(a, pEntry->mUnixTime);

-
  /* Look for the initial prefix match, if there is one. */
-
  if( pRe->nInit ){
-
    unsigned char x = pRe->zInit[0];
-
    while( in.i+pRe->nInit<=in.mx 
-
     && (zIn[in.i]!=x ||
-
         strncmp((const char*)zIn+in.i, (const char*)pRe->zInit, pRe->nInit)!=0)
-
    ){
-
      in.i++;
-
    }
-
    if( in.i+pRe->nInit>in.mx ) return 0;
+
  return a-aBuf;
+
}
+

+
static int zipfileAppendEntry(
+
  ZipfileTab *pTab,
+
  ZipfileEntry *pEntry,
+
  const u8 *pData,
+
  int nData
+
){
+
  u8 *aBuf = pTab->aBuffer;
+
  int nBuf;
+
  int rc;
+

+
  nBuf = zipfileSerializeLFH(pEntry, aBuf);
+
  rc = zipfileAppendData(pTab, aBuf, nBuf);
+
  if( rc==SQLITE_OK ){
+
    pEntry->iDataOff = pTab->szCurrent;
+
    rc = zipfileAppendData(pTab, pData, nData);
  }

-
  if( pRe->nState<=(sizeof(aSpace)/(sizeof(aSpace[0])*2)) ){
-
    pToFree = 0;
-
    aStateSet[0].aState = aSpace;
+
  return rc;
+
}
+

+
static int zipfileGetMode(
+
  sqlite3_value *pVal, 
+
  int bIsDir,                     /* If true, default to directory */
+
  u32 *pMode,                     /* OUT: Mode value */
+
  char **pzErr                    /* OUT: Error message */
+
){
+
  const char *z = (const char*)sqlite3_value_text(pVal);
+
  u32 mode = 0;
+
  if( z==0 ){
+
    mode = (bIsDir ? (S_IFDIR + 0755) : (S_IFREG + 0644));
+
  }else if( z[0]>='0' && z[0]<='9' ){
+
    mode = (unsigned int)sqlite3_value_int(pVal);
  }else{
-
    pToFree = sqlite3_malloc64( sizeof(ReStateNumber)*2*pRe->nState );
-
    if( pToFree==0 ) return -1;
-
    aStateSet[0].aState = pToFree;
-
  }
-
  aStateSet[1].aState = &aStateSet[0].aState[pRe->nState];
-
  pNext = &aStateSet[1];
-
  pNext->nState = 0;
-
  re_add_state(pNext, 0);
-
  while( c!=RE_EOF && pNext->nState>0 ){
-
    cPrev = c;
-
    c = pRe->xNextChar(&in);
-
    pThis = pNext;
-
    pNext = &aStateSet[iSwap];
-
    iSwap = 1 - iSwap;
-
    pNext->nState = 0;
-
    for(i=0; i<pThis->nState; i++){
-
      int x = pThis->aState[i];
-
      switch( pRe->aOp[x] ){
-
        case RE_OP_MATCH: {
-
          if( pRe->aArg[x]==c ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_ANY: {
-
          if( c!=0 ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_WORD: {
-
          if( re_word_char(c) ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_NOTWORD: {
-
          if( !re_word_char(c) && c!=0 ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_DIGIT: {
-
          if( re_digit_char(c) ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_NOTDIGIT: {
-
          if( !re_digit_char(c) && c!=0 ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_SPACE: {
-
          if( re_space_char(c) ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_NOTSPACE: {
-
          if( !re_space_char(c) && c!=0 ) re_add_state(pNext, x+1);
-
          break;
-
        }
-
        case RE_OP_BOUNDARY: {
-
          if( re_word_char(c)!=re_word_char(cPrev) ) re_add_state(pThis, x+1);
-
          break;
-
        }
-
        case RE_OP_ANYSTAR: {
-
          re_add_state(pNext, x);
-
          re_add_state(pThis, x+1);
-
          break;
-
        }
-
        case RE_OP_FORK: {
-
          re_add_state(pThis, x+pRe->aArg[x]);
-
          re_add_state(pThis, x+1);
-
          break;
-
        }
-
        case RE_OP_GOTO: {
-
          re_add_state(pThis, x+pRe->aArg[x]);
-
          break;
-
        }
-
        case RE_OP_ACCEPT: {
-
          rc = 1;
-
          goto re_match_end;
-
        }
-
        case RE_OP_CC_EXC: {
-
          if( c==0 ) break;
-
          /* fall-through */ goto re_op_cc_inc;
-
        }
-
        case RE_OP_CC_INC: re_op_cc_inc: {
-
          int j = 1;
-
          int n = pRe->aArg[x];
-
          int hit = 0;
-
          for(j=1; j>0 && j<n; j++){
-
            if( pRe->aOp[x+j]==RE_OP_CC_VALUE ){
-
              if( pRe->aArg[x+j]==c ){
-
                hit = 1;
-
                j = -1;
-
              }
-
            }else{
-
              if( pRe->aArg[x+j]<=c && pRe->aArg[x+j+1]>=c ){
-
                hit = 1;
-
                j = -1;
-
              }else{
-
                j++;
-
              }
-
            }
-
          }
-
          if( pRe->aOp[x]==RE_OP_CC_EXC ) hit = !hit;
-
          if( hit ) re_add_state(pNext, x+n);
-
          break;
-
        }
-
      }
+
    const char zTemplate[11] = "-rwxrwxrwx";
+
    int i;
+
    if( strlen(z)!=10 ) goto parse_error;
+
    switch( z[0] ){
+
      case '-': mode |= S_IFREG; break;
+
      case 'd': mode |= S_IFDIR; break;
+
      case 'l': mode |= S_IFLNK; break;
+
      default: goto parse_error;
+
    }
+
    for(i=1; i<10; i++){
+
      if( z[i]==zTemplate[i] ) mode |= 1 << (9-i);
+
      else if( z[i]!='-' ) goto parse_error;
    }
  }
-
  for(i=0; i<pNext->nState; i++){
-
    if( pRe->aOp[pNext->aState[i]]==RE_OP_ACCEPT ){ rc = 1; break; }
+
  if( ((mode & S_IFDIR)==0)==bIsDir ){
+
    /* The "mode" attribute is a directory, but data has been specified.
+
    ** Or vice-versa - no data but "mode" is a file or symlink.  */
+
    *pzErr = sqlite3_mprintf("zipfile: mode does not match data");
+
    return SQLITE_CONSTRAINT;
  }
-
re_match_end:
-
  sqlite3_free(pToFree);
-
  return rc;
-
}
+
  *pMode = mode;
+
  return SQLITE_OK;

-
/* Resize the opcode and argument arrays for an RE under construction.
-
*/
-
static int re_resize(ReCompiled *p, int N){
-
  char *aOp;
-
  int *aArg;
-
  aOp = sqlite3_realloc64(p->aOp, N*sizeof(p->aOp[0]));
-
  if( aOp==0 ) return 1;
-
  p->aOp = aOp;
-
  aArg = sqlite3_realloc64(p->aArg, N*sizeof(p->aArg[0]));
-
  if( aArg==0 ) return 1;
-
  p->aArg = aArg;
-
  p->nAlloc = N;
-
  return 0;
+
 parse_error:
+
  *pzErr = sqlite3_mprintf("zipfile: parse error in mode: %s", z);
+
  return SQLITE_ERROR;
}

-
/* Insert a new opcode and argument into an RE under construction.  The
-
** insertion point is just prior to existing opcode iBefore.
-
*/
-
static int re_insert(ReCompiled *p, int iBefore, int op, int arg){
-
  int i;
-
  if( p->nAlloc<=p->nState && re_resize(p, p->nAlloc*2) ) return 0;
-
  for(i=p->nState; i>iBefore; i--){
-
    p->aOp[i] = p->aOp[i-1];
-
    p->aArg[i] = p->aArg[i-1];
-
  }
-
  p->nState++;
-
  p->aOp[iBefore] = (char)op;
-
  p->aArg[iBefore] = arg;
-
  return iBefore;
+
/*
+
** Both (const char*) arguments point to nul-terminated strings. Argument
+
** nB is the value of strlen(zB). This function returns 0 if the strings are
+
** identical, ignoring any trailing '/' character in either path.  */
+
static int zipfileComparePath(const char *zA, const char *zB, int nB){
+
  int nA = (int)strlen(zA);
+
  if( nA>0 && zA[nA-1]=='/' ) nA--;
+
  if( nB>0 && zB[nB-1]=='/' ) nB--;
+
  if( nA==nB && memcmp(zA, zB, nA)==0 ) return 0;
+
  return 1;
}

-
/* Append a new opcode and argument to the end of the RE under construction.
-
*/
-
static int re_append(ReCompiled *p, int op, int arg){
-
  return re_insert(p, p->nState, op, arg);
-
}
+
static int zipfileBegin(sqlite3_vtab *pVtab){
+
  ZipfileTab *pTab = (ZipfileTab*)pVtab;
+
  int rc = SQLITE_OK;

-
/* Make a copy of N opcodes starting at iStart onto the end of the RE
-
** under construction.
-
*/
-
static void re_copy(ReCompiled *p, int iStart, int N){
-
  if( p->nState+N>=p->nAlloc && re_resize(p, p->nAlloc*2+N) ) return;
-
  memcpy(&p->aOp[p->nState], &p->aOp[iStart], N*sizeof(p->aOp[0]));
-
  memcpy(&p->aArg[p->nState], &p->aArg[iStart], N*sizeof(p->aArg[0]));
-
  p->nState += N;
-
}
+
  assert( pTab->pWriteFd==0 );
+
  if( pTab->zFile==0 || pTab->zFile[0]==0 ){
+
    pTab->base.zErrMsg = sqlite3_mprintf("zipfile: missing filename");
+
    return SQLITE_ERROR;
+
  }

-
/* Return true if c is a hexadecimal digit character:  [0-9a-fA-F]
-
** If c is a hex digit, also set *pV = (*pV)*16 + valueof(c).  If
-
** c is not a hex digit *pV is unchanged.
-
*/
-
static int re_hex(int c, int *pV){
-
  if( c>='0' && c<='9' ){
-
    c -= '0';
-
  }else if( c>='a' && c<='f' ){
-
    c -= 'a' - 10;
-
  }else if( c>='A' && c<='F' ){
-
    c -= 'A' - 10;
+
  /* Open a write fd on the file. Also load the entire central directory
+
  ** structure into memory. During the transaction any new file data is 
+
  ** appended to the archive file, but the central directory is accumulated
+
  ** in main-memory until the transaction is committed.  */
+
  pTab->pWriteFd = fopen(pTab->zFile, "ab+");
+
  if( pTab->pWriteFd==0 ){
+
    pTab->base.zErrMsg = sqlite3_mprintf(
+
        "zipfile: failed to open file %s for writing", pTab->zFile
+
        );
+
    rc = SQLITE_ERROR;
  }else{
-
    return 0;
+
    fseek(pTab->pWriteFd, 0, SEEK_END);
+
    pTab->szCurrent = pTab->szOrig = (i64)ftell(pTab->pWriteFd);
+
    rc = zipfileLoadDirectory(pTab, 0, 0);
  }
-
  *pV = (*pV)*16 + (c & 0xff);
-
  return 1;
+

+
  if( rc!=SQLITE_OK ){
+
    zipfileCleanupTransaction(pTab);
+
  }
+

+
  return rc;
}

-
/* A backslash character has been seen, read the next character and
-
** return its interpretation.
+
/*
+
** Return the current time as a 32-bit timestamp in UNIX epoch format (like
+
** time(2)).
*/
-
static unsigned re_esc_char(ReCompiled *p){
-
  static const char zEsc[] = "afnrtv\\()*.+?[$^{|}]";
-
  static const char zTrans[] = "\a\f\n\r\t\v";
-
  int i, v = 0;
-
  char c;
-
  if( p->sIn.i>=p->sIn.mx ) return 0;
-
  c = p->sIn.z[p->sIn.i];
-
  if( c=='u' && p->sIn.i+4<p->sIn.mx ){
-
    const unsigned char *zIn = p->sIn.z + p->sIn.i;
-
    if( re_hex(zIn[1],&v)
-
     && re_hex(zIn[2],&v)
-
     && re_hex(zIn[3],&v)
-
     && re_hex(zIn[4],&v)
-
    ){
-
      p->sIn.i += 5;
-
      return v;
-
    }
-
  }
-
  if( c=='x' && p->sIn.i+2<p->sIn.mx ){
-
    const unsigned char *zIn = p->sIn.z + p->sIn.i;
-
    if( re_hex(zIn[1],&v)
-
     && re_hex(zIn[2],&v)
-
    ){
-
      p->sIn.i += 3;
-
      return v;
-
    }
-
  }
-
  for(i=0; zEsc[i] && zEsc[i]!=c; i++){}
-
  if( zEsc[i] ){
-
    if( i<6 ) c = zTrans[i];
-
    p->sIn.i++;
+
static u32 zipfileTime(void){
+
  sqlite3_vfs *pVfs = sqlite3_vfs_find(0);
+
  u32 ret;
+
  if( pVfs==0 ) return 0;
+
  if( pVfs->iVersion>=2 && pVfs->xCurrentTimeInt64 ){
+
    i64 ms;
+
    pVfs->xCurrentTimeInt64(pVfs, &ms);
+
    ret = (u32)((ms/1000) - ((i64)24405875 * 8640));
  }else{
-
    p->zErr = "unknown \\ escape";
+
    double day;
+
    pVfs->xCurrentTime(pVfs, &day);
+
    ret = (u32)((day - 2440587.5) * 86400);
  }
-
  return c;
+
  return ret;
}

-
/* Forward declaration */
-
static const char *re_subcompile_string(ReCompiled*);
-

-
/* Peek at the next byte of input */
-
static unsigned char rePeek(ReCompiled *p){
-
  return p->sIn.i<p->sIn.mx ? p->sIn.z[p->sIn.i] : 0;
+
/*
+
** Return a 32-bit timestamp in UNIX epoch format.
+
**
+
** If the value passed as the only argument is either NULL or an SQL NULL,
+
** return the current time. Otherwise, return the value stored in (*pVal)
+
** cast to a 32-bit unsigned integer.
+
*/
+
static u32 zipfileGetTime(sqlite3_value *pVal){
+
  if( pVal==0 || sqlite3_value_type(pVal)==SQLITE_NULL ){
+
    return zipfileTime();
+
  }
+
  return (u32)sqlite3_value_int64(pVal);
}

-
/* Compile RE text into a sequence of opcodes.  Continue up to the
-
** first unmatched ")" character, then return.  If an error is found,
-
** return a pointer to the error message string.
+
/*
+
** Unless it is NULL, entry pOld is currently part of the pTab->pFirstEntry
+
** linked list.  Remove it from the list and free the object.
*/
-
static const char *re_subcompile_re(ReCompiled *p){
-
  const char *zErr;
-
  int iStart, iEnd, iGoto;
-
  iStart = p->nState;
-
  zErr = re_subcompile_string(p);
-
  if( zErr ) return zErr;
-
  while( rePeek(p)=='|' ){
-
    iEnd = p->nState;
-
    re_insert(p, iStart, RE_OP_FORK, iEnd + 2 - iStart);
-
    iGoto = re_append(p, RE_OP_GOTO, 0);
-
    p->sIn.i++;
-
    zErr = re_subcompile_string(p);
-
    if( zErr ) return zErr;
-
    p->aArg[iGoto] = p->nState - iGoto;
+
static void zipfileRemoveEntryFromList(ZipfileTab *pTab, ZipfileEntry *pOld){
+
  if( pOld ){
+
    ZipfileEntry **pp;
+
    for(pp=&pTab->pFirstEntry; (*pp)!=pOld; pp=&((*pp)->pNext));
+
    *pp = (*pp)->pNext;
+
    zipfileEntryFree(pOld);
  }
-
  return 0;
}

-
/* Compile an element of regular expression text (anything that can be
-
** an operand to the "|" operator).  Return NULL on success or a pointer
-
** to the error message if there is a problem.
+
/*
+
** xUpdate method.
*/
-
static const char *re_subcompile_string(ReCompiled *p){
-
  int iPrev = -1;
-
  int iStart;
-
  unsigned c;
-
  const char *zErr;
-
  while( (c = p->xNextChar(&p->sIn))!=0 ){
-
    iStart = p->nState;
-
    switch( c ){
-
      case '|':
-
      case '$':
-
      case ')': {
-
        p->sIn.i--;
-
        return 0;
+
static int zipfileUpdate(
+
  sqlite3_vtab *pVtab, 
+
  int nVal, 
+
  sqlite3_value **apVal, 
+
  sqlite_int64 *pRowid
+
){
+
  ZipfileTab *pTab = (ZipfileTab*)pVtab;
+
  int rc = SQLITE_OK;             /* Return Code */
+
  ZipfileEntry *pNew = 0;         /* New in-memory CDS entry */
+

+
  u32 mode = 0;                   /* Mode for new entry */
+
  u32 mTime = 0;                  /* Modification time for new entry */
+
  i64 sz = 0;                     /* Uncompressed size */
+
  const char *zPath = 0;          /* Path for new entry */
+
  int nPath = 0;                  /* strlen(zPath) */
+
  const u8 *pData = 0;            /* Pointer to buffer containing content */
+
  int nData = 0;                  /* Size of pData buffer in bytes */
+
  int iMethod = 0;                /* Compression method for new entry */
+
  u8 *pFree = 0;                  /* Free this */
+
  char *zFree = 0;                /* Also free this */
+
  ZipfileEntry *pOld = 0;
+
  ZipfileEntry *pOld2 = 0;
+
  int bUpdate = 0;                /* True for an update that modifies "name" */
+
  int bIsDir = 0;
+
  u32 iCrc32 = 0;
+

+
  if( pTab->pWriteFd==0 ){
+
    rc = zipfileBegin(pVtab);
+
    if( rc!=SQLITE_OK ) return rc;
+
  }
+

+
  /* If this is a DELETE or UPDATE, find the archive entry to delete. */
+
  if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
+
    const char *zDelete = (const char*)sqlite3_value_text(apVal[0]);
+
    int nDelete = (int)strlen(zDelete);
+
    if( nVal>1 ){
+
      const char *zUpdate = (const char*)sqlite3_value_text(apVal[1]);
+
      if( zUpdate && zipfileComparePath(zUpdate, zDelete, nDelete)!=0 ){
+
        bUpdate = 1;
      }
-
      case '(': {
-
        zErr = re_subcompile_re(p);
-
        if( zErr ) return zErr;
-
        if( rePeek(p)!=')' ) return "unmatched '('";
-
        p->sIn.i++;
+
    }
+
    for(pOld=pTab->pFirstEntry; 1; pOld=pOld->pNext){
+
      if( zipfileComparePath(pOld->cds.zFile, zDelete, nDelete)==0 ){
        break;
      }
-
      case '.': {
-
        if( rePeek(p)=='*' ){
-
          re_append(p, RE_OP_ANYSTAR, 0);
-
          p->sIn.i++;
+
      assert( pOld->pNext );
+
    }
+
  }
+

+
  if( nVal>1 ){
+
    /* Check that "sz" and "rawdata" are both NULL: */
+
    if( sqlite3_value_type(apVal[5])!=SQLITE_NULL ){
+
      zipfileTableErr(pTab, "sz must be NULL");
+
      rc = SQLITE_CONSTRAINT;
+
    }
+
    if( sqlite3_value_type(apVal[6])!=SQLITE_NULL ){
+
      zipfileTableErr(pTab, "rawdata must be NULL"); 
+
      rc = SQLITE_CONSTRAINT;
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      if( sqlite3_value_type(apVal[7])==SQLITE_NULL ){
+
        /* data=NULL. A directory */
+
        bIsDir = 1;
+
      }else{
+
        /* Value specified for "data", and possibly "method". This must be
+
        ** a regular file or a symlink. */
+
        const u8 *aIn = sqlite3_value_blob(apVal[7]);
+
        int nIn = sqlite3_value_bytes(apVal[7]);
+
        int bAuto = sqlite3_value_type(apVal[8])==SQLITE_NULL;
+

+
        iMethod = sqlite3_value_int(apVal[8]);
+
        sz = nIn;
+
        pData = aIn;
+
        nData = nIn;
+
        if( iMethod!=0 && iMethod!=8 ){
+
          zipfileTableErr(pTab, "unknown compression method: %d", iMethod);
+
          rc = SQLITE_CONSTRAINT;
        }else{
-
          re_append(p, RE_OP_ANY, 0);
+
          if( bAuto || iMethod ){
+
            int nCmp;
+
            rc = zipfileDeflate(aIn, nIn, &pFree, &nCmp, &pTab->base.zErrMsg);
+
            if( rc==SQLITE_OK ){
+
              if( iMethod || nCmp<nIn ){
+
                iMethod = 8;
+
                pData = pFree;
+
                nData = nCmp;
+
              }
+
            }
+
          }
+
          iCrc32 = crc32(0, aIn, nIn);
        }
-
        break;
-
      }
-
      case '*': {
-
        if( iPrev<0 ) return "'*' without operand";
-
        re_insert(p, iPrev, RE_OP_GOTO, p->nState - iPrev + 1);
-
        re_append(p, RE_OP_FORK, iPrev - p->nState + 1);
-
        break;
-
      }
-
      case '+': {
-
        if( iPrev<0 ) return "'+' without operand";
-
        re_append(p, RE_OP_FORK, iPrev - p->nState);
-
        break;
-
      }
-
      case '?': {
-
        if( iPrev<0 ) return "'?' without operand";
-
        re_insert(p, iPrev, RE_OP_FORK, p->nState - iPrev+1);
-
        break;
      }
-
      case '{': {
-
        int m = 0, n = 0;
-
        int sz, j;
-
        if( iPrev<0 ) return "'{m,n}' without operand";
-
        while( (c=rePeek(p))>='0' && c<='9' ){ m = m*10 + c - '0'; p->sIn.i++; }
-
        n = m;
-
        if( c==',' ){
-
          p->sIn.i++;
-
          n = 0;
-
          while( (c=rePeek(p))>='0' && c<='9' ){ n = n*10 + c-'0'; p->sIn.i++; }
-
        }
-
        if( c!='}' ) return "unmatched '{'";
-
        if( n>0 && n<m ) return "n less than m in '{m,n}'";
-
        p->sIn.i++;
-
        sz = p->nState - iPrev;
-
        if( m==0 ){
-
          if( n==0 ) return "both m and n are zero in '{m,n}'";
-
          re_insert(p, iPrev, RE_OP_FORK, sz+1);
-
          n--;
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      rc = zipfileGetMode(apVal[3], bIsDir, &mode, &pTab->base.zErrMsg);
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      zPath = (const char*)sqlite3_value_text(apVal[2]);
+
      if( zPath==0 ) zPath = "";
+
      nPath = (int)strlen(zPath);
+
      mTime = zipfileGetTime(apVal[4]);
+
    }
+

+
    if( rc==SQLITE_OK && bIsDir ){
+
      /* For a directory, check that the last character in the path is a
+
      ** '/'. This appears to be required for compatibility with info-zip
+
      ** (the unzip command on unix). It does not create directories
+
      ** otherwise.  */
+
      if( nPath<=0 || zPath[nPath-1]!='/' ){
+
        zFree = sqlite3_mprintf("%s/", zPath);
+
        zPath = (const char*)zFree;
+
        if( zFree==0 ){
+
          rc = SQLITE_NOMEM;
+
          nPath = 0;
        }else{
-
          for(j=1; j<m; j++) re_copy(p, iPrev, sz);
-
        }
-
        for(j=m; j<n; j++){
-
          re_append(p, RE_OP_FORK, sz+1);
-
          re_copy(p, iPrev, sz);
-
        }
-
        if( n==0 && m>0 ){
-
          re_append(p, RE_OP_FORK, -sz);
+
          nPath = (int)strlen(zPath);
        }
-
        break;
      }
-
      case '[': {
-
        int iFirst = p->nState;
-
        if( rePeek(p)=='^' ){
-
          re_append(p, RE_OP_CC_EXC, 0);
-
          p->sIn.i++;
-
        }else{
-
          re_append(p, RE_OP_CC_INC, 0);
-
        }
-
        while( (c = p->xNextChar(&p->sIn))!=0 ){
-
          if( c=='[' && rePeek(p)==':' ){
-
            return "POSIX character classes not supported";
-
          }
-
          if( c=='\\' ) c = re_esc_char(p);
-
          if( rePeek(p)=='-' ){
-
            re_append(p, RE_OP_CC_RANGE, c);
-
            p->sIn.i++;
-
            c = p->xNextChar(&p->sIn);
-
            if( c=='\\' ) c = re_esc_char(p);
-
            re_append(p, RE_OP_CC_RANGE, c);
-
          }else{
-
            re_append(p, RE_OP_CC_VALUE, c);
+
    }
+

+
    /* Check that we're not inserting a duplicate entry -OR- updating an
+
    ** entry with a path, thereby making it into a duplicate. */
+
    if( (pOld==0 || bUpdate) && rc==SQLITE_OK ){
+
      ZipfileEntry *p;
+
      for(p=pTab->pFirstEntry; p; p=p->pNext){
+
        if( zipfileComparePath(p->cds.zFile, zPath, nPath)==0 ){
+
          switch( sqlite3_vtab_on_conflict(pTab->db) ){
+
            case SQLITE_IGNORE: {
+
              goto zipfile_update_done;
+
            }
+
            case SQLITE_REPLACE: {
+
              pOld2 = p;
+
              break;
+
            }
+
            default: {
+
              zipfileTableErr(pTab, "duplicate name: \"%s\"", zPath);
+
              rc = SQLITE_CONSTRAINT;
+
              break;
+
            }
          }
-
          if( rePeek(p)==']' ){ p->sIn.i++; break; }
+
          break;
        }
-
        if( c==0 ) return "unclosed '['";
-
        p->aArg[iFirst] = p->nState - iFirst;
-
        break;
      }
-
      case '\\': {
-
        int specialOp = 0;
-
        switch( rePeek(p) ){
-
          case 'b': specialOp = RE_OP_BOUNDARY;   break;
-
          case 'd': specialOp = RE_OP_DIGIT;      break;
-
          case 'D': specialOp = RE_OP_NOTDIGIT;   break;
-
          case 's': specialOp = RE_OP_SPACE;      break;
-
          case 'S': specialOp = RE_OP_NOTSPACE;   break;
-
          case 'w': specialOp = RE_OP_WORD;       break;
-
          case 'W': specialOp = RE_OP_NOTWORD;    break;
-
        }
-
        if( specialOp ){
-
          p->sIn.i++;
-
          re_append(p, specialOp, 0);
-
        }else{
-
          c = re_esc_char(p);
-
          re_append(p, RE_OP_MATCH, c);
-
        }
-
        break;
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      /* Create the new CDS record. */
+
      pNew = zipfileNewEntry(zPath);
+
      if( pNew==0 ){
+
        rc = SQLITE_NOMEM;
+
      }else{
+
        pNew->cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
+
        pNew->cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
+
        pNew->cds.flags = ZIPFILE_NEWENTRY_FLAGS;
+
        pNew->cds.iCompression = (u16)iMethod;
+
        zipfileMtimeToDos(&pNew->cds, mTime);
+
        pNew->cds.crc32 = iCrc32;
+
        pNew->cds.szCompressed = nData;
+
        pNew->cds.szUncompressed = (u32)sz;
+
        pNew->cds.iExternalAttr = (mode<<16);
+
        pNew->cds.iOffset = (u32)pTab->szCurrent;
+
        pNew->cds.nFile = (u16)nPath;
+
        pNew->mUnixTime = (u32)mTime;
+
        rc = zipfileAppendEntry(pTab, pNew, pData, nData);
+
        zipfileAddEntry(pTab, pOld, pNew);
      }
-
      default: {
-
        re_append(p, RE_OP_MATCH, c);
-
        break;
+
    }
+
  }
+

+
  if( rc==SQLITE_OK && (pOld || pOld2) ){
+
    ZipfileCsr *pCsr;
+
    for(pCsr=pTab->pCsrList; pCsr; pCsr=pCsr->pCsrNext){
+
      if( pCsr->pCurrent && (pCsr->pCurrent==pOld || pCsr->pCurrent==pOld2) ){
+
        pCsr->pCurrent = pCsr->pCurrent->pNext;
+
        pCsr->bNoop = 1;
      }
    }
-
    iPrev = iStart;
+

+
    zipfileRemoveEntryFromList(pTab, pOld);
+
    zipfileRemoveEntryFromList(pTab, pOld2);
  }
-
  return 0;
+

+
zipfile_update_done:
+
  sqlite3_free(pFree);
+
  sqlite3_free(zFree);
+
  return rc;
+
}
+

+
static int zipfileSerializeEOCD(ZipfileEOCD *p, u8 *aBuf){
+
  u8 *a = aBuf;
+
  zipfileWrite32(a, ZIPFILE_SIGNATURE_EOCD);
+
  zipfileWrite16(a, p->iDisk);
+
  zipfileWrite16(a, p->iFirstDisk);
+
  zipfileWrite16(a, p->nEntry);
+
  zipfileWrite16(a, p->nEntryTotal);
+
  zipfileWrite32(a, p->nSize);
+
  zipfileWrite32(a, p->iOffset);
+
  zipfileWrite16(a, 0);        /* Size of trailing comment in bytes*/
+

+
  return a-aBuf;
}

-
/* Free and reclaim all the memory used by a previously compiled
-
** regular expression.  Applications should invoke this routine once
-
** for every call to re_compile() to avoid memory leaks.
-
*/
-
static void re_free(ReCompiled *pRe){
-
  if( pRe ){
-
    sqlite3_free(pRe->aOp);
-
    sqlite3_free(pRe->aArg);
-
    sqlite3_free(pRe);
-
  }
+
static int zipfileAppendEOCD(ZipfileTab *pTab, ZipfileEOCD *p){
+
  int nBuf = zipfileSerializeEOCD(p, pTab->aBuffer);
+
  assert( nBuf==ZIPFILE_EOCD_FIXED_SZ );
+
  return zipfileAppendData(pTab, pTab->aBuffer, nBuf);
}

/*
-
** Compile a textual regular expression in zIn[] into a compiled regular
-
** expression suitable for us by re_match() and return a pointer to the
-
** compiled regular expression in *ppRe.  Return NULL on success or an
-
** error message if something goes wrong.
+
** Serialize the CDS structure into buffer aBuf[]. Return the number
+
** of bytes written.
*/
-
static const char *re_compile(ReCompiled **ppRe, const char *zIn, int noCase){
-
  ReCompiled *pRe;
-
  const char *zErr;
-
  int i, j;
+
static int zipfileSerializeCDS(ZipfileEntry *pEntry, u8 *aBuf){
+
  u8 *a = aBuf;
+
  ZipfileCDS *pCDS = &pEntry->cds;

-
  *ppRe = 0;
-
  pRe = sqlite3_malloc( sizeof(*pRe) );
-
  if( pRe==0 ){
-
    return "out of memory";
-
  }
-
  memset(pRe, 0, sizeof(*pRe));
-
  pRe->xNextChar = noCase ? re_next_char_nocase : re_next_char;
-
  if( re_resize(pRe, 30) ){
-
    re_free(pRe);
-
    return "out of memory";
-
  }
-
  if( zIn[0]=='^' ){
-
    zIn++;
-
  }else{
-
    re_append(pRe, RE_OP_ANYSTAR, 0);
-
  }
-
  pRe->sIn.z = (unsigned char*)zIn;
-
  pRe->sIn.i = 0;
-
  pRe->sIn.mx = (int)strlen(zIn);
-
  zErr = re_subcompile_re(pRe);
-
  if( zErr ){
-
    re_free(pRe);
-
    return zErr;
+
  if( pEntry->aExtra==0 ){
+
    pCDS->nExtra = 9;
  }
-
  if( rePeek(pRe)=='$' && pRe->sIn.i+1>=pRe->sIn.mx ){
-
    re_append(pRe, RE_OP_MATCH, RE_EOF);
-
    re_append(pRe, RE_OP_ACCEPT, 0);
-
    *ppRe = pRe;
-
  }else if( pRe->sIn.i>=pRe->sIn.mx ){
-
    re_append(pRe, RE_OP_ACCEPT, 0);
-
    *ppRe = pRe;
+

+
  zipfileWrite32(a, ZIPFILE_SIGNATURE_CDS);
+
  zipfileWrite16(a, pCDS->iVersionMadeBy);
+
  zipfileWrite16(a, pCDS->iVersionExtract);
+
  zipfileWrite16(a, pCDS->flags);
+
  zipfileWrite16(a, pCDS->iCompression);
+
  zipfileWrite16(a, pCDS->mTime);
+
  zipfileWrite16(a, pCDS->mDate);
+
  zipfileWrite32(a, pCDS->crc32);
+
  zipfileWrite32(a, pCDS->szCompressed);
+
  zipfileWrite32(a, pCDS->szUncompressed);
+
  assert( a==&aBuf[ZIPFILE_CDS_NFILE_OFF] );
+
  zipfileWrite16(a, pCDS->nFile);
+
  zipfileWrite16(a, pCDS->nExtra);
+
  zipfileWrite16(a, pCDS->nComment);
+
  zipfileWrite16(a, pCDS->iDiskStart);
+
  zipfileWrite16(a, pCDS->iInternalAttr);
+
  zipfileWrite32(a, pCDS->iExternalAttr);
+
  zipfileWrite32(a, pCDS->iOffset);
+

+
  memcpy(a, pCDS->zFile, pCDS->nFile);
+
  a += pCDS->nFile;
+

+
  if( pEntry->aExtra ){
+
    int n = (int)pCDS->nExtra + (int)pCDS->nComment;
+
    memcpy(a, pEntry->aExtra, n);
+
    a += n;
  }else{
-
    re_free(pRe);
-
    return "unrecognized character";
+
    assert( pCDS->nExtra==9 );
+
    zipfileWrite16(a, ZIPFILE_EXTRA_TIMESTAMP);
+
    zipfileWrite16(a, 5);
+
    *a++ = 0x01;
+
    zipfileWrite32(a, pEntry->mUnixTime);
  }

-
  /* The following is a performance optimization.  If the regex begins with
-
  ** ".*" (if the input regex lacks an initial "^") and afterwards there are
-
  ** one or more matching characters, enter those matching characters into
-
  ** zInit[].  The re_match() routine can then search ahead in the input 
-
  ** string looking for the initial match without having to run the whole
-
  ** regex engine over the string.  Do not worry able trying to match
-
  ** unicode characters beyond plane 0 - those are very rare and this is
-
  ** just an optimization. */
-
  if( pRe->aOp[0]==RE_OP_ANYSTAR && !noCase ){
-
    for(j=0, i=1; j<(int)sizeof(pRe->zInit)-2 && pRe->aOp[i]==RE_OP_MATCH; i++){
-
      unsigned x = pRe->aArg[i];
-
      if( x<=127 ){
-
        pRe->zInit[j++] = (unsigned char)x;
-
      }else if( x<=0xfff ){
-
        pRe->zInit[j++] = (unsigned char)(0xc0 | (x>>6));
-
        pRe->zInit[j++] = 0x80 | (x&0x3f);
-
      }else if( x<=0xffff ){
-
        pRe->zInit[j++] = (unsigned char)(0xd0 | (x>>12));
-
        pRe->zInit[j++] = 0x80 | ((x>>6)&0x3f);
-
        pRe->zInit[j++] = 0x80 | (x&0x3f);
-
      }else{
-
        break;
-
      }
+
  return a-aBuf;
+
}
+

+
static int zipfileCommit(sqlite3_vtab *pVtab){
+
  ZipfileTab *pTab = (ZipfileTab*)pVtab;
+
  int rc = SQLITE_OK;
+
  if( pTab->pWriteFd ){
+
    i64 iOffset = pTab->szCurrent;
+
    ZipfileEntry *p;
+
    ZipfileEOCD eocd;
+
    int nEntry = 0;
+

+
    /* Write out all entries */
+
    for(p=pTab->pFirstEntry; rc==SQLITE_OK && p; p=p->pNext){
+
      int n = zipfileSerializeCDS(p, pTab->aBuffer);
+
      rc = zipfileAppendData(pTab, pTab->aBuffer, n);
+
      nEntry++;
    }
-
    if( j>0 && pRe->zInit[j-1]==0 ) j--;
-
    pRe->nInit = j;
+

+
    /* Write out the EOCD record */
+
    eocd.iDisk = 0;
+
    eocd.iFirstDisk = 0;
+
    eocd.nEntry = (u16)nEntry;
+
    eocd.nEntryTotal = (u16)nEntry;
+
    eocd.nSize = (u32)(pTab->szCurrent - iOffset);
+
    eocd.iOffset = (u32)iOffset;
+
    rc = zipfileAppendEOCD(pTab, &eocd);
+

+
    zipfileCleanupTransaction(pTab);
  }
-
  return pRe->zErr;
+
  return rc;
}

-
/*
-
** Implementation of the regexp() SQL function.  This function implements
-
** the build-in REGEXP operator.  The first argument to the function is the
-
** pattern and the second argument is the string.  So, the SQL statements:
-
**
-
**       A REGEXP B
-
**
-
** is implemented as regexp(B,A).
-
*/
-
static void re_sql_func(
+
static int zipfileRollback(sqlite3_vtab *pVtab){
+
  return zipfileCommit(pVtab);
+
}
+

+
static ZipfileCsr *zipfileFindCursor(ZipfileTab *pTab, i64 iId){
+
  ZipfileCsr *pCsr;
+
  for(pCsr=pTab->pCsrList; pCsr; pCsr=pCsr->pCsrNext){
+
    if( iId==pCsr->iId ) break;
+
  }
+
  return pCsr;
+
}
+

+
static void zipfileFunctionCds(
  sqlite3_context *context,
  int argc,
  sqlite3_value **argv
){
-
  ReCompiled *pRe;          /* Compiled regular expression */
-
  const char *zPattern;     /* The regular expression */
-
  const unsigned char *zStr;/* String being searched */
-
  const char *zErr;         /* Compile error message */
-
  int setAux = 0;           /* True to invoke sqlite3_set_auxdata() */
+
  ZipfileCsr *pCsr;
+
  ZipfileTab *pTab = (ZipfileTab*)sqlite3_user_data(context);
+
  assert( argc>0 );

-
  (void)argc;  /* Unused */
-
  pRe = sqlite3_get_auxdata(context, 0);
-
  if( pRe==0 ){
-
    zPattern = (const char*)sqlite3_value_text(argv[0]);
-
    if( zPattern==0 ) return;
-
    zErr = re_compile(&pRe, zPattern, sqlite3_user_data(context)!=0);
-
    if( zErr ){
-
      re_free(pRe);
-
      sqlite3_result_error(context, zErr, -1);
-
      return;
-
    }
-
    if( pRe==0 ){
+
  pCsr = zipfileFindCursor(pTab, sqlite3_value_int64(argv[0]));
+
  if( pCsr ){
+
    ZipfileCDS *p = &pCsr->pCurrent->cds;
+
    char *zRes = sqlite3_mprintf("{"
+
        "\"version-made-by\" : %u, "
+
        "\"version-to-extract\" : %u, "
+
        "\"flags\" : %u, "
+
        "\"compression\" : %u, "
+
        "\"time\" : %u, "
+
        "\"date\" : %u, "
+
        "\"crc32\" : %u, "
+
        "\"compressed-size\" : %u, "
+
        "\"uncompressed-size\" : %u, "
+
        "\"file-name-length\" : %u, "
+
        "\"extra-field-length\" : %u, "
+
        "\"file-comment-length\" : %u, "
+
        "\"disk-number-start\" : %u, "
+
        "\"internal-attr\" : %u, "
+
        "\"external-attr\" : %u, "
+
        "\"offset\" : %u }",
+
        (u32)p->iVersionMadeBy, (u32)p->iVersionExtract,
+
        (u32)p->flags, (u32)p->iCompression,
+
        (u32)p->mTime, (u32)p->mDate,
+
        (u32)p->crc32, (u32)p->szCompressed,
+
        (u32)p->szUncompressed, (u32)p->nFile,
+
        (u32)p->nExtra, (u32)p->nComment,
+
        (u32)p->iDiskStart, (u32)p->iInternalAttr,
+
        (u32)p->iExternalAttr, (u32)p->iOffset
+
    );
+

+
    if( zRes==0 ){
      sqlite3_result_error_nomem(context);
-
      return;
+
    }else{
+
      sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT);
+
      sqlite3_free(zRes);
    }
-
    setAux = 1;
-
  }
-
  zStr = (const unsigned char*)sqlite3_value_text(argv[1]);
-
  if( zStr!=0 ){
-
    sqlite3_result_int(context, re_match(pRe, zStr, -1));
  }
-
  if( setAux ){
-
    sqlite3_set_auxdata(context, 0, pRe, (void(*)(void*))re_free);
+
}
+

+
/*
+
** xFindFunction method.
+
*/
+
static int zipfileFindFunction(
+
  sqlite3_vtab *pVtab,            /* Virtual table handle */
+
  int nArg,                       /* Number of SQL function arguments */
+
  const char *zName,              /* Name of SQL function */
+
  void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */
+
  void **ppArg                    /* OUT: User data for *pxFunc */
+
){
+
  if( sqlite3_stricmp("zipfile_cds", zName)==0 ){
+
    *pxFunc = zipfileFunctionCds;
+
    *ppArg = (void*)pVtab;
+
    return 1;
  }
+
  return 0;
}

-
/*
-
** Invoke this routine to register the regexp() function with the
-
** SQLite database connection.
-
*/
-
#ifdef _WIN32
+
typedef struct ZipfileBuffer ZipfileBuffer;
+
struct ZipfileBuffer {
+
  u8 *a;                          /* Pointer to buffer */
+
  int n;                          /* Size of buffer in bytes */
+
  int nAlloc;                     /* Byte allocated at a[] */
+
};
+

+
typedef struct ZipfileCtx ZipfileCtx;
+
struct ZipfileCtx {
+
  int nEntry;
+
  ZipfileBuffer body;
+
  ZipfileBuffer cds;
+
};

-
#endif
-
int sqlite3_regexp_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  int rc = SQLITE_OK;
-
  SQLITE_EXTENSION_INIT2(pApi);
-
  (void)pzErrMsg;  /* Unused */
-
  rc = sqlite3_create_function(db, "regexp", 2, 
-
                            SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC,
-
                            0, re_sql_func, 0, 0);
-
  if( rc==SQLITE_OK ){
-
    /* The regexpi(PATTERN,STRING) function is a case-insensitive version
-
    ** of regexp(PATTERN,STRING). */
-
    rc = sqlite3_create_function(db, "regexpi", 2,
-
                            SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC,
-
                            (void*)db, re_sql_func, 0, 0);
+
static int zipfileBufferGrow(ZipfileBuffer *pBuf, int nByte){
+
  if( pBuf->n+nByte>pBuf->nAlloc ){
+
    u8 *aNew;
+
    sqlite3_int64 nNew = pBuf->n ? pBuf->n*2 : 512;
+
    int nReq = pBuf->n + nByte;
+

+
    while( nNew<nReq ) nNew = nNew*2;
+
    aNew = sqlite3_realloc64(pBuf->a, nNew);
+
    if( aNew==0 ) return SQLITE_NOMEM;
+
    pBuf->a = aNew;
+
    pBuf->nAlloc = (int)nNew;
  }
-
  return rc;
+
  return SQLITE_OK;
}

-
/************************* End ../ext/misc/regexp.c ********************/
-
#ifdef SQLITE_HAVE_ZLIB
-
/************************* Begin ../ext/misc/zipfile.c ******************/
/*
-
** 2017-12-26
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** This file implements a virtual table for reading and writing ZIP archive
-
** files.
-
**
-
** Usage example:
-
**
-
**     SELECT name, sz, datetime(mtime,'unixepoch') FROM zipfile($filename);
-
**
-
** Current limitations:
+
** xStep() callback for the zipfile() aggregate. This can be called in
+
** any of the following ways:
**
-
**    *  No support for encryption
-
**    *  No support for ZIP archives spanning multiple files
-
**    *  No support for zip64 extensions
-
**    *  Only the "inflate/deflate" (zlib) compression method is supported
+
**   SELECT zipfile(name,data) ...
+
**   SELECT zipfile(name,mode,mtime,data) ...
+
**   SELECT zipfile(name,mode,mtime,data,method) ...
*/
-
/* #include "sqlite3ext.h" */
-
SQLITE_EXTENSION_INIT1
-
#include <stdio.h>
-
#include <string.h>
-
#include <assert.h>
+
static void zipfileStep(sqlite3_context *pCtx, int nVal, sqlite3_value **apVal){
+
  ZipfileCtx *p;                  /* Aggregate function context */
+
  ZipfileEntry e;                 /* New entry to add to zip archive */

-
#include <zlib.h>
+
  sqlite3_value *pName = 0;
+
  sqlite3_value *pMode = 0;
+
  sqlite3_value *pMtime = 0;
+
  sqlite3_value *pData = 0;
+
  sqlite3_value *pMethod = 0;

-
#ifndef SQLITE_OMIT_VIRTUALTABLE
+
  int bIsDir = 0;
+
  u32 mode;
+
  int rc = SQLITE_OK;
+
  char *zErr = 0;

-
#ifndef SQLITE_AMALGAMATION
+
  int iMethod = -1;               /* Compression method to use (0 or 8) */

-
#ifndef UINT32_TYPE
-
# ifdef HAVE_UINT32_T
-
#  define UINT32_TYPE uint32_t
-
# else
-
#  define UINT32_TYPE unsigned int
-
# endif
-
#endif
-
#ifndef UINT16_TYPE
-
# ifdef HAVE_UINT16_T
-
#  define UINT16_TYPE uint16_t
-
# else
-
#  define UINT16_TYPE unsigned short int
-
# endif
-
#endif
-
/* typedef sqlite3_int64 i64; */
-
/* typedef unsigned char u8; */
-
typedef UINT32_TYPE u32;           /* 4-byte unsigned integer */
-
typedef UINT16_TYPE u16;           /* 2-byte unsigned integer */
-
#define MIN(a,b) ((a)<(b) ? (a) : (b))
+
  const u8 *aData = 0;            /* Possibly compressed data for new entry */
+
  int nData = 0;                  /* Size of aData[] in bytes */
+
  int szUncompressed = 0;         /* Size of data before compression */
+
  u8 *aFree = 0;                  /* Free this before returning */
+
  u32 iCrc32 = 0;                 /* crc32 of uncompressed data */

-
#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
-
# define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1
-
#endif
-
#if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS)
-
# define ALWAYS(X)      (1)
-
# define NEVER(X)       (0)
-
#elif !defined(NDEBUG)
-
# define ALWAYS(X)      ((X)?1:(assert(0),0))
-
# define NEVER(X)       ((X)?(assert(0),1):0)
-
#else
-
# define ALWAYS(X)      (X)
-
# define NEVER(X)       (X)
-
#endif
+
  char *zName = 0;                /* Path (name) of new entry */
+
  int nName = 0;                  /* Size of zName in bytes */
+
  char *zFree = 0;                /* Free this before returning */
+
  int nByte;

-
#endif   /* SQLITE_AMALGAMATION */
+
  memset(&e, 0, sizeof(e));
+
  p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx));
+
  if( p==0 ) return;

-
/*
-
** Definitions for mode bitmasks S_IFDIR, S_IFREG and S_IFLNK.
-
**
-
** In some ways it would be better to obtain these values from system 
-
** header files. But, the dependency is undesirable and (a) these
-
** have been stable for decades, (b) the values are part of POSIX and
-
** are also made explicit in [man stat], and (c) are part of the 
-
** file format for zip archives.
-
*/
-
#ifndef S_IFDIR
-
# define S_IFDIR 0040000
-
#endif
-
#ifndef S_IFREG
-
# define S_IFREG 0100000
-
#endif
-
#ifndef S_IFLNK
-
# define S_IFLNK 0120000
-
#endif
+
  /* Martial the arguments into stack variables */
+
  if( nVal!=2 && nVal!=4 && nVal!=5 ){
+
    zErr = sqlite3_mprintf("wrong number of arguments to function zipfile()");
+
    rc = SQLITE_ERROR;
+
    goto zipfile_step_out;
+
  }
+
  pName = apVal[0];
+
  if( nVal==2 ){
+
    pData = apVal[1];
+
  }else{
+
    pMode = apVal[1];
+
    pMtime = apVal[2];
+
    pData = apVal[3];
+
    if( nVal==5 ){
+
      pMethod = apVal[4];
+
    }
+
  }

-
static const char ZIPFILE_SCHEMA[] = 
-
  "CREATE TABLE y("
-
    "name PRIMARY KEY,"  /* 0: Name of file in zip archive */
-
    "mode,"              /* 1: POSIX mode for file */
-
    "mtime,"             /* 2: Last modification time (secs since 1970)*/
-
    "sz,"                /* 3: Size of object */
-
    "rawdata,"           /* 4: Raw data */
-
    "data,"              /* 5: Uncompressed data */
-
    "method,"            /* 6: Compression method (integer) */
-
    "z HIDDEN"           /* 7: Name of zip file */
-
  ") WITHOUT ROWID;";
+
  /* Check that the 'name' parameter looks ok. */
+
  zName = (char*)sqlite3_value_text(pName);
+
  nName = sqlite3_value_bytes(pName);
+
  if( zName==0 ){
+
    zErr = sqlite3_mprintf("first argument to zipfile() must be non-NULL");
+
    rc = SQLITE_ERROR;
+
    goto zipfile_step_out;
+
  }
+

+
  /* Inspect the 'method' parameter. This must be either 0 (store), 8 (use
+
  ** deflate compression) or NULL (choose automatically).  */
+
  if( pMethod && SQLITE_NULL!=sqlite3_value_type(pMethod) ){
+
    iMethod = (int)sqlite3_value_int64(pMethod);
+
    if( iMethod!=0 && iMethod!=8 ){
+
      zErr = sqlite3_mprintf("illegal method value: %d", iMethod);
+
      rc = SQLITE_ERROR;
+
      goto zipfile_step_out;
+
    }
+
  }
+

+
  /* Now inspect the data. If this is NULL, then the new entry must be a
+
  ** directory.  Otherwise, figure out whether or not the data should
+
  ** be deflated or simply stored in the zip archive. */
+
  if( sqlite3_value_type(pData)==SQLITE_NULL ){
+
    bIsDir = 1;
+
    iMethod = 0;
+
  }else{
+
    aData = sqlite3_value_blob(pData);
+
    szUncompressed = nData = sqlite3_value_bytes(pData);
+
    iCrc32 = crc32(0, aData, nData);
+
    if( iMethod<0 || iMethod==8 ){
+
      int nOut = 0;
+
      rc = zipfileDeflate(aData, nData, &aFree, &nOut, &zErr);
+
      if( rc!=SQLITE_OK ){
+
        goto zipfile_step_out;
+
      }
+
      if( iMethod==8 || nOut<nData ){
+
        aData = aFree;
+
        nData = nOut;
+
        iMethod = 8;
+
      }else{
+
        iMethod = 0;
+
      }
+
    }
+
  }
+

+
  /* Decode the "mode" argument. */
+
  rc = zipfileGetMode(pMode, bIsDir, &mode, &zErr);
+
  if( rc ) goto zipfile_step_out;
+

+
  /* Decode the "mtime" argument. */
+
  e.mUnixTime = zipfileGetTime(pMtime);
+

+
  /* If this is a directory entry, ensure that there is exactly one '/'
+
  ** at the end of the path. Or, if this is not a directory and the path
+
  ** ends in '/' it is an error. */
+
  if( bIsDir==0 ){
+
    if( nName>0 && zName[nName-1]=='/' ){
+
      zErr = sqlite3_mprintf("non-directory name must not end with /");
+
      rc = SQLITE_ERROR;
+
      goto zipfile_step_out;
+
    }
+
  }else{
+
    if( nName==0 || zName[nName-1]!='/' ){
+
      zName = zFree = sqlite3_mprintf("%s/", zName);
+
      if( zName==0 ){
+
        rc = SQLITE_NOMEM;
+
        goto zipfile_step_out;
+
      }
+
      nName = (int)strlen(zName);
+
    }else{
+
      while( nName>1 && zName[nName-2]=='/' ) nName--;
+
    }
+
  }

-
#define ZIPFILE_F_COLUMN_IDX 7    /* Index of column "file" in the above */
-
#define ZIPFILE_BUFFER_SIZE (64*1024)
+
  /* Assemble the ZipfileEntry object for the new zip archive entry */
+
  e.cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
+
  e.cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
+
  e.cds.flags = ZIPFILE_NEWENTRY_FLAGS;
+
  e.cds.iCompression = (u16)iMethod;
+
  zipfileMtimeToDos(&e.cds, (u32)e.mUnixTime);
+
  e.cds.crc32 = iCrc32;
+
  e.cds.szCompressed = nData;
+
  e.cds.szUncompressed = szUncompressed;
+
  e.cds.iExternalAttr = (mode<<16);
+
  e.cds.iOffset = p->body.n;
+
  e.cds.nFile = (u16)nName;
+
  e.cds.zFile = zName;

+
  /* Append the LFH to the body of the new archive */
+
  nByte = ZIPFILE_LFH_FIXED_SZ + e.cds.nFile + 9;
+
  if( (rc = zipfileBufferGrow(&p->body, nByte)) ) goto zipfile_step_out;
+
  p->body.n += zipfileSerializeLFH(&e, &p->body.a[p->body.n]);

-
/*
-
** Magic numbers used to read and write zip files.
-
**
-
** ZIPFILE_NEWENTRY_MADEBY:
-
**   Use this value for the "version-made-by" field in new zip file
-
**   entries. The upper byte indicates "unix", and the lower byte 
-
**   indicates that the zip file matches pkzip specification 3.0. 
-
**   This is what info-zip seems to do.
-
**
-
** ZIPFILE_NEWENTRY_REQUIRED:
-
**   Value for "version-required-to-extract" field of new entries.
-
**   Version 2.0 is required to support folders and deflate compression.
-
**
-
** ZIPFILE_NEWENTRY_FLAGS:
-
**   Value for "general-purpose-bit-flags" field of new entries. Bit
-
**   11 means "utf-8 filename and comment".
-
**
-
** ZIPFILE_SIGNATURE_CDS:
-
**   First 4 bytes of a valid CDS record.
-
**
-
** ZIPFILE_SIGNATURE_LFH:
-
**   First 4 bytes of a valid LFH record.
-
**
-
** ZIPFILE_SIGNATURE_EOCD
-
**   First 4 bytes of a valid EOCD record.
-
*/
-
#define ZIPFILE_EXTRA_TIMESTAMP   0x5455
-
#define ZIPFILE_NEWENTRY_MADEBY   ((3<<8) + 30)
-
#define ZIPFILE_NEWENTRY_REQUIRED 20
-
#define ZIPFILE_NEWENTRY_FLAGS    0x800
-
#define ZIPFILE_SIGNATURE_CDS     0x02014b50
-
#define ZIPFILE_SIGNATURE_LFH     0x04034b50
-
#define ZIPFILE_SIGNATURE_EOCD    0x06054b50
+
  /* Append the data to the body of the new archive */
+
  if( nData>0 ){
+
    if( (rc = zipfileBufferGrow(&p->body, nData)) ) goto zipfile_step_out;
+
    memcpy(&p->body.a[p->body.n], aData, nData);
+
    p->body.n += nData;
+
  }

-
/*
-
** The sizes of the fixed-size part of each of the three main data 
-
** structures in a zip archive.
-
*/
-
#define ZIPFILE_LFH_FIXED_SZ      30
-
#define ZIPFILE_EOCD_FIXED_SZ     22
-
#define ZIPFILE_CDS_FIXED_SZ      46
+
  /* Append the CDS record to the directory of the new archive */
+
  nByte = ZIPFILE_CDS_FIXED_SZ + e.cds.nFile + 9;
+
  if( (rc = zipfileBufferGrow(&p->cds, nByte)) ) goto zipfile_step_out;
+
  p->cds.n += zipfileSerializeCDS(&e, &p->cds.a[p->cds.n]);

-
/*
-
*** 4.3.16  End of central directory record:
-
***
-
***   end of central dir signature    4 bytes  (0x06054b50)
-
***   number of this disk             2 bytes
-
***   number of the disk with the
-
***   start of the central directory  2 bytes
-
***   total number of entries in the
-
***   central directory on this disk  2 bytes
-
***   total number of entries in
-
***   the central directory           2 bytes
-
***   size of the central directory   4 bytes
-
***   offset of start of central
-
***   directory with respect to
-
***   the starting disk number        4 bytes
-
***   .ZIP file comment length        2 bytes
-
***   .ZIP file comment       (variable size)
-
*/
-
typedef struct ZipfileEOCD ZipfileEOCD;
-
struct ZipfileEOCD {
-
  u16 iDisk;
-
  u16 iFirstDisk;
-
  u16 nEntry;
-
  u16 nEntryTotal;
-
  u32 nSize;
-
  u32 iOffset;
-
};
+
  /* Increment the count of entries in the archive */
+
  p->nEntry++;

-
/*
-
*** 4.3.12  Central directory structure:
-
***
-
*** ...
-
***
-
***   central file header signature   4 bytes  (0x02014b50)
-
***   version made by                 2 bytes
-
***   version needed to extract       2 bytes
-
***   general purpose bit flag        2 bytes
-
***   compression method              2 bytes
-
***   last mod file time              2 bytes
-
***   last mod file date              2 bytes
-
***   crc-32                          4 bytes
-
***   compressed size                 4 bytes
-
***   uncompressed size               4 bytes
-
***   file name length                2 bytes
-
***   extra field length              2 bytes
-
***   file comment length             2 bytes
-
***   disk number start               2 bytes
-
***   internal file attributes        2 bytes
-
***   external file attributes        4 bytes
-
***   relative offset of local header 4 bytes
-
*/
-
typedef struct ZipfileCDS ZipfileCDS;
-
struct ZipfileCDS {
-
  u16 iVersionMadeBy;
-
  u16 iVersionExtract;
-
  u16 flags;
-
  u16 iCompression;
-
  u16 mTime;
-
  u16 mDate;
-
  u32 crc32;
-
  u32 szCompressed;
-
  u32 szUncompressed;
-
  u16 nFile;
-
  u16 nExtra;
-
  u16 nComment;
-
  u16 iDiskStart;
-
  u16 iInternalAttr;
-
  u32 iExternalAttr;
-
  u32 iOffset;
-
  char *zFile;                    /* Filename (sqlite3_malloc()) */
-
};
+
 zipfile_step_out:
+
  sqlite3_free(aFree);
+
  sqlite3_free(zFree);
+
  if( rc ){
+
    if( zErr ){
+
      sqlite3_result_error(pCtx, zErr, -1);
+
    }else{
+
      sqlite3_result_error_code(pCtx, rc);
+
    }
+
  }
+
  sqlite3_free(zErr);
+
}

/*
-
*** 4.3.7  Local file header:
-
***
-
***   local file header signature     4 bytes  (0x04034b50)
-
***   version needed to extract       2 bytes
-
***   general purpose bit flag        2 bytes
-
***   compression method              2 bytes
-
***   last mod file time              2 bytes
-
***   last mod file date              2 bytes
-
***   crc-32                          4 bytes
-
***   compressed size                 4 bytes
-
***   uncompressed size               4 bytes
-
***   file name length                2 bytes
-
***   extra field length              2 bytes
-
***   
+
** xFinalize() callback for zipfile aggregate function.
*/
-
typedef struct ZipfileLFH ZipfileLFH;
-
struct ZipfileLFH {
-
  u16 iVersionExtract;
-
  u16 flags;
-
  u16 iCompression;
-
  u16 mTime;
-
  u16 mDate;
-
  u32 crc32;
-
  u32 szCompressed;
-
  u32 szUncompressed;
-
  u16 nFile;
-
  u16 nExtra;
-
};
+
static void zipfileFinal(sqlite3_context *pCtx){
+
  ZipfileCtx *p;
+
  ZipfileEOCD eocd;
+
  sqlite3_int64 nZip;
+
  u8 *aZip;

-
typedef struct ZipfileEntry ZipfileEntry;
-
struct ZipfileEntry {
-
  ZipfileCDS cds;            /* Parsed CDS record */
-
  u32 mUnixTime;             /* Modification time, in UNIX format */
-
  u8 *aExtra;                /* cds.nExtra+cds.nComment bytes of extra data */
-
  i64 iDataOff;              /* Offset to data in file (if aData==0) */
-
  u8 *aData;                 /* cds.szCompressed bytes of compressed data */
-
  ZipfileEntry *pNext;       /* Next element in in-memory CDS */
-
};
+
  p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx));
+
  if( p==0 ) return;
+
  if( p->nEntry>0 ){
+
    memset(&eocd, 0, sizeof(eocd));
+
    eocd.nEntry = (u16)p->nEntry;
+
    eocd.nEntryTotal = (u16)p->nEntry;
+
    eocd.nSize = p->cds.n;
+
    eocd.iOffset = p->body.n;

-
/* 
-
** Cursor type for zipfile tables.
-
*/
-
typedef struct ZipfileCsr ZipfileCsr;
-
struct ZipfileCsr {
-
  sqlite3_vtab_cursor base;  /* Base class - must be first */
-
  i64 iId;                   /* Cursor ID */
-
  u8 bEof;                   /* True when at EOF */
-
  u8 bNoop;                  /* If next xNext() call is no-op */
+
    nZip = p->body.n + p->cds.n + ZIPFILE_EOCD_FIXED_SZ;
+
    aZip = (u8*)sqlite3_malloc64(nZip);
+
    if( aZip==0 ){
+
      sqlite3_result_error_nomem(pCtx);
+
    }else{
+
      memcpy(aZip, p->body.a, p->body.n);
+
      memcpy(&aZip[p->body.n], p->cds.a, p->cds.n);
+
      zipfileSerializeEOCD(&eocd, &aZip[p->body.n + p->cds.n]);
+
      sqlite3_result_blob(pCtx, aZip, (int)nZip, zipfileFree);
+
    }
+
  }

-
  /* Used outside of write transactions */
-
  FILE *pFile;               /* Zip file */
-
  i64 iNextOff;              /* Offset of next record in central directory */
-
  ZipfileEOCD eocd;          /* Parse of central directory record */
+
  sqlite3_free(p->body.a);
+
  sqlite3_free(p->cds.a);
+
}

-
  ZipfileEntry *pFreeEntry;  /* Free this list when cursor is closed or reset */
-
  ZipfileEntry *pCurrent;    /* Current entry */
-
  ZipfileCsr *pCsrNext;      /* Next cursor on same virtual table */
-
};

-
typedef struct ZipfileTab ZipfileTab;
-
struct ZipfileTab {
-
  sqlite3_vtab base;         /* Base class - must be first */
-
  char *zFile;               /* Zip file this table accesses (may be NULL) */
-
  sqlite3 *db;               /* Host database connection */
-
  u8 *aBuffer;               /* Temporary buffer used for various tasks */
+
/*
+
** Register the "zipfile" virtual table.
+
*/
+
static int zipfileRegister(sqlite3 *db){
+
  static sqlite3_module zipfileModule = {
+
    1,                         /* iVersion */
+
    zipfileConnect,            /* xCreate */
+
    zipfileConnect,            /* xConnect */
+
    zipfileBestIndex,          /* xBestIndex */
+
    zipfileDisconnect,         /* xDisconnect */
+
    zipfileDisconnect,         /* xDestroy */
+
    zipfileOpen,               /* xOpen - open a cursor */
+
    zipfileClose,              /* xClose - close a cursor */
+
    zipfileFilter,             /* xFilter - configure scan constraints */
+
    zipfileNext,               /* xNext - advance a cursor */
+
    zipfileEof,                /* xEof - check for end of scan */
+
    zipfileColumn,             /* xColumn - read data */
+
    0,                         /* xRowid - read data */
+
    zipfileUpdate,             /* xUpdate */
+
    zipfileBegin,              /* xBegin */
+
    0,                         /* xSync */
+
    zipfileCommit,             /* xCommit */
+
    zipfileRollback,           /* xRollback */
+
    zipfileFindFunction,       /* xFindMethod */
+
    0,                         /* xRename */
+
    0,                         /* xSavepoint */
+
    0,                         /* xRelease */
+
    0,                         /* xRollback */
+
    0                          /* xShadowName */
+
  };
+

+
  int rc = sqlite3_create_module(db, "zipfile"  , &zipfileModule, 0);
+
  if( rc==SQLITE_OK ) rc = sqlite3_overload_function(db, "zipfile_cds", -1);
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_create_function(db, "zipfile", -1, SQLITE_UTF8, 0, 0, 
+
        zipfileStep, zipfileFinal
+
    );
+
  }
+
  assert( sizeof(i64)==8 );
+
  assert( sizeof(u32)==4 );
+
  assert( sizeof(u16)==2 );
+
  assert( sizeof(u8)==1 );
+
  return rc;
+
}
+
#else         /* SQLITE_OMIT_VIRTUALTABLE */
+
# define zipfileRegister(x) SQLITE_OK
+
#endif

-
  ZipfileCsr *pCsrList;      /* List of cursors */
-
  i64 iNextCsrid;
+
#ifdef _WIN32

-
  /* The following are used by write transactions only */
-
  ZipfileEntry *pFirstEntry; /* Linked list of all files (if pWriteFd!=0) */
-
  ZipfileEntry *pLastEntry;  /* Last element in pFirstEntry list */
-
  FILE *pWriteFd;            /* File handle open on zip archive */
-
  i64 szCurrent;             /* Current size of zip archive */
-
  i64 szOrig;                /* Size of archive at start of transaction */
-
};
+
#endif
+
int sqlite3_zipfile_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
+
){
+
  SQLITE_EXTENSION_INIT2(pApi);
+
  (void)pzErrMsg;  /* Unused parameter */
+
  return zipfileRegister(db);
+
}

+
/************************* End ../ext/misc/zipfile.c ********************/
+
/************************* Begin ../ext/misc/sqlar.c ******************/
/*
-
** Set the error message contained in context ctx to the results of
-
** vprintf(zFmt, ...).
+
** 2017-12-17
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** Utility functions sqlar_compress() and sqlar_uncompress(). Useful
+
** for working with sqlar archives and used by the shell tool's built-in
+
** sqlar support.
*/
-
static void zipfileCtxErrorMsg(sqlite3_context *ctx, const char *zFmt, ...){
-
  char *zMsg = 0;
-
  va_list ap;
-
  va_start(ap, zFmt);
-
  zMsg = sqlite3_vmprintf(zFmt, ap);
-
  sqlite3_result_error(ctx, zMsg, -1);
-
  sqlite3_free(zMsg);
-
  va_end(ap);
-
}
+
/* #include "sqlite3ext.h" */
+
SQLITE_EXTENSION_INIT1
+
#include <zlib.h>
+
#include <assert.h>

/*
-
** If string zIn is quoted, dequote it in place. Otherwise, if the string
-
** is not quoted, do nothing.
+
** Implementation of the "sqlar_compress(X)" SQL function.
+
**
+
** If the type of X is SQLITE_BLOB, and compressing that blob using
+
** zlib utility function compress() yields a smaller blob, return the
+
** compressed blob. Otherwise, return a copy of X.
+
**
+
** SQLar uses the "zlib format" for compressed content.  The zlib format
+
** contains a two-byte identification header and a four-byte checksum at
+
** the end.  This is different from ZIP which uses the raw deflate format.
+
**
+
** Future enhancements to SQLar might add support for new compression formats.
+
** If so, those new formats will be identified by alternative headers in the
+
** compressed data.
*/
-
static void zipfileDequote(char *zIn){
-
  char q = zIn[0];
-
  if( q=='"' || q=='\'' || q=='`' || q=='[' ){
-
    int iIn = 1;
-
    int iOut = 0;
-
    if( q=='[' ) q = ']';
-
    while( ALWAYS(zIn[iIn]) ){
-
      char c = zIn[iIn++];
-
      if( c==q && zIn[iIn++]!=q ) break;
-
      zIn[iOut++] = c;
+
static void sqlarCompressFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
+
){
+
  assert( argc==1 );
+
  if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){
+
    const Bytef *pData = sqlite3_value_blob(argv[0]);
+
    uLong nData = sqlite3_value_bytes(argv[0]);
+
    uLongf nOut = compressBound(nData);
+
    Bytef *pOut;
+

+
    pOut = (Bytef*)sqlite3_malloc(nOut);
+
    if( pOut==0 ){
+
      sqlite3_result_error_nomem(context);
+
      return;
+
    }else{
+
      if( Z_OK!=compress(pOut, &nOut, pData, nData) ){
+
        sqlite3_result_error(context, "error in compress()", -1);
+
      }else if( nOut<nData ){
+
        sqlite3_result_blob(context, pOut, nOut, SQLITE_TRANSIENT);
+
      }else{
+
        sqlite3_result_value(context, argv[0]);
+
      }
+
      sqlite3_free(pOut);
    }
-
    zIn[iOut] = '\0';
+
  }else{
+
    sqlite3_result_value(context, argv[0]);
  }
}

/*
-
** Construct a new ZipfileTab virtual table object.
-
** 
-
**   argv[0]   -> module name  ("zipfile")
-
**   argv[1]   -> database name
-
**   argv[2]   -> table name
-
**   argv[...] -> "column name" and other module argument fields.
+
** Implementation of the "sqlar_uncompress(X,SZ)" SQL function
+
**
+
** Parameter SZ is interpreted as an integer. If it is less than or
+
** equal to zero, then this function returns a copy of X. Or, if
+
** SZ is equal to the size of X when interpreted as a blob, also
+
** return a copy of X. Otherwise, decompress blob X using zlib
+
** utility function uncompress() and return the results (another
+
** blob).
*/
-
static int zipfileConnect(
-
  sqlite3 *db,
-
  void *pAux,
-
  int argc, const char *const*argv,
-
  sqlite3_vtab **ppVtab,
-
  char **pzErr
+
static void sqlarUncompressFunc(
+
  sqlite3_context *context,
+
  int argc,
+
  sqlite3_value **argv
){
-
  int nByte = sizeof(ZipfileTab) + ZIPFILE_BUFFER_SIZE;
-
  int nFile = 0;
-
  const char *zFile = 0;
-
  ZipfileTab *pNew = 0;
-
  int rc;
-

-
  /* If the table name is not "zipfile", require that the argument be
-
  ** specified. This stops zipfile tables from being created as:
-
  **
-
  **   CREATE VIRTUAL TABLE zzz USING zipfile();
-
  **
-
  ** It does not prevent:
-
  **
-
  **   CREATE VIRTUAL TABLE zipfile USING zipfile();
-
  */
-
  assert( 0==sqlite3_stricmp(argv[0], "zipfile") );
-
  if( (0!=sqlite3_stricmp(argv[2], "zipfile") && argc<4) || argc>4 ){
-
    *pzErr = sqlite3_mprintf("zipfile constructor requires one argument");
-
    return SQLITE_ERROR;
-
  }
+
  uLong nData;
+
  uLongf sz;

-
  if( argc>3 ){
-
    zFile = argv[3];
-
    nFile = (int)strlen(zFile)+1;
-
  }
+
  assert( argc==2 );
+
  sz = sqlite3_value_int(argv[1]);

-
  rc = sqlite3_declare_vtab(db, ZIPFILE_SCHEMA);
-
  if( rc==SQLITE_OK ){
-
    pNew = (ZipfileTab*)sqlite3_malloc64((sqlite3_int64)nByte+nFile);
-
    if( pNew==0 ) return SQLITE_NOMEM;
-
    memset(pNew, 0, nByte+nFile);
-
    pNew->db = db;
-
    pNew->aBuffer = (u8*)&pNew[1];
-
    if( zFile ){
-
      pNew->zFile = (char*)&pNew->aBuffer[ZIPFILE_BUFFER_SIZE];
-
      memcpy(pNew->zFile, zFile, nFile);
-
      zipfileDequote(pNew->zFile);
+
  if( sz<=0 || sz==(nData = sqlite3_value_bytes(argv[0])) ){
+
    sqlite3_result_value(context, argv[0]);
+
  }else{
+
    const Bytef *pData= sqlite3_value_blob(argv[0]);
+
    Bytef *pOut = sqlite3_malloc(sz);
+
    if( Z_OK!=uncompress(pOut, &sz, pData, nData) ){
+
      sqlite3_result_error(context, "error in uncompress()", -1);
+
    }else{
+
      sqlite3_result_blob(context, pOut, sz, SQLITE_TRANSIENT);
    }
+
    sqlite3_free(pOut);
  }
-
  sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY);
-
  *ppVtab = (sqlite3_vtab*)pNew;
-
  return rc;
}

-
/*
-
** Free the ZipfileEntry structure indicated by the only argument.
-
*/
-
static void zipfileEntryFree(ZipfileEntry *p){
-
  if( p ){
-
    sqlite3_free(p->cds.zFile);
-
    sqlite3_free(p);
-
  }
-
}

-
/*
-
** Release resources that should be freed at the end of a write 
-
** transaction.
-
*/
-
static void zipfileCleanupTransaction(ZipfileTab *pTab){
-
  ZipfileEntry *pEntry;
-
  ZipfileEntry *pNext;
+
#ifdef _WIN32

-
  if( pTab->pWriteFd ){
-
    fclose(pTab->pWriteFd);
-
    pTab->pWriteFd = 0;
-
  }
-
  for(pEntry=pTab->pFirstEntry; pEntry; pEntry=pNext){
-
    pNext = pEntry->pNext;
-
    zipfileEntryFree(pEntry);
+
#endif
+
int sqlite3_sqlar_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
+
){
+
  int rc = SQLITE_OK;
+
  SQLITE_EXTENSION_INIT2(pApi);
+
  (void)pzErrMsg;  /* Unused parameter */
+
  rc = sqlite3_create_function(db, "sqlar_compress", 1, 
+
                               SQLITE_UTF8|SQLITE_INNOCUOUS, 0,
+
                               sqlarCompressFunc, 0, 0);
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_create_function(db, "sqlar_uncompress", 2,
+
                                 SQLITE_UTF8|SQLITE_INNOCUOUS, 0,
+
                                 sqlarUncompressFunc, 0, 0);
  }
-
  pTab->pFirstEntry = 0;
-
  pTab->pLastEntry = 0;
-
  pTab->szCurrent = 0;
-
  pTab->szOrig = 0;
+
  return rc;
}

+
/************************* End ../ext/misc/sqlar.c ********************/
+
#endif
+
/************************* Begin ../ext/expert/sqlite3expert.h ******************/
/*
-
** This method is the destructor for zipfile vtab objects.
+
** 2017 April 07
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
*************************************************************************
*/
-
static int zipfileDisconnect(sqlite3_vtab *pVtab){
-
  zipfileCleanupTransaction((ZipfileTab*)pVtab);
-
  sqlite3_free(pVtab);
-
  return SQLITE_OK;
-
}
+
#if !defined(SQLITEEXPERT_H)
+
#define SQLITEEXPERT_H 1
+
/* #include "sqlite3.h" */
+

+
typedef struct sqlite3expert sqlite3expert;

/*
-
** Constructor for a new ZipfileCsr object.
+
** Create a new sqlite3expert object.
+
**
+
** If successful, a pointer to the new object is returned and (*pzErr) set
+
** to NULL. Or, if an error occurs, NULL is returned and (*pzErr) set to
+
** an English-language error message. In this case it is the responsibility
+
** of the caller to eventually free the error message buffer using
+
** sqlite3_free().
*/
-
static int zipfileOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCsr){
-
  ZipfileTab *pTab = (ZipfileTab*)p;
-
  ZipfileCsr *pCsr;
-
  pCsr = sqlite3_malloc(sizeof(*pCsr));
-
  *ppCsr = (sqlite3_vtab_cursor*)pCsr;
-
  if( pCsr==0 ){
-
    return SQLITE_NOMEM;
-
  }
-
  memset(pCsr, 0, sizeof(*pCsr));
-
  pCsr->iId = ++pTab->iNextCsrid;
-
  pCsr->pCsrNext = pTab->pCsrList;
-
  pTab->pCsrList = pCsr;
-
  return SQLITE_OK;
-
}
+
sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErr);

/*
-
** Reset a cursor back to the state it was in when first returned
-
** by zipfileOpen().
+
** Configure an sqlite3expert object.
+
**
+
** EXPERT_CONFIG_SAMPLE:
+
**   By default, sqlite3_expert_analyze() generates sqlite_stat1 data for
+
**   each candidate index. This involves scanning and sorting the entire
+
**   contents of each user database table once for each candidate index
+
**   associated with the table. For large databases, this can be 
+
**   prohibitively slow. This option allows the sqlite3expert object to
+
**   be configured so that sqlite_stat1 data is instead generated based on a
+
**   subset of each table, or so that no sqlite_stat1 data is used at all.
+
**
+
**   A single integer argument is passed to this option. If the value is less
+
**   than or equal to zero, then no sqlite_stat1 data is generated or used by
+
**   the analysis - indexes are recommended based on the database schema only.
+
**   Or, if the value is 100 or greater, complete sqlite_stat1 data is
+
**   generated for each candidate index (this is the default). Finally, if the
+
**   value falls between 0 and 100, then it represents the percentage of user
+
**   table rows that should be considered when generating sqlite_stat1 data.
+
**
+
**   Examples:
+
**
+
**     // Do not generate any sqlite_stat1 data
+
**     sqlite3_expert_config(pExpert, EXPERT_CONFIG_SAMPLE, 0);
+
**
+
**     // Generate sqlite_stat1 data based on 10% of the rows in each table.
+
**     sqlite3_expert_config(pExpert, EXPERT_CONFIG_SAMPLE, 10);
*/
-
static void zipfileResetCursor(ZipfileCsr *pCsr){
-
  ZipfileEntry *p;
-
  ZipfileEntry *pNext;
-

-
  pCsr->bEof = 0;
-
  if( pCsr->pFile ){
-
    fclose(pCsr->pFile);
-
    pCsr->pFile = 0;
-
    zipfileEntryFree(pCsr->pCurrent);
-
    pCsr->pCurrent = 0;
-
  }
+
int sqlite3_expert_config(sqlite3expert *p, int op, ...);

-
  for(p=pCsr->pFreeEntry; p; p=pNext){
-
    pNext = p->pNext;
-
    zipfileEntryFree(p);
-
  }
-
}
+
#define EXPERT_CONFIG_SAMPLE 1    /* int */

/*
-
** Destructor for an ZipfileCsr.
+
** Specify zero or more SQL statements to be included in the analysis.
+
**
+
** Buffer zSql must contain zero or more complete SQL statements. This
+
** function parses all statements contained in the buffer and adds them
+
** to the internal list of statements to analyze. If successful, SQLITE_OK
+
** is returned and (*pzErr) set to NULL. Or, if an error occurs - for example
+
** due to a error in the SQL - an SQLite error code is returned and (*pzErr)
+
** may be set to point to an English language error message. In this case
+
** the caller is responsible for eventually freeing the error message buffer
+
** using sqlite3_free().
+
**
+
** If an error does occur while processing one of the statements in the
+
** buffer passed as the second argument, none of the statements in the
+
** buffer are added to the analysis.
+
**
+
** This function must be called before sqlite3_expert_analyze(). If a call
+
** to this function is made on an sqlite3expert object that has already
+
** been passed to sqlite3_expert_analyze() SQLITE_MISUSE is returned
+
** immediately and no statements are added to the analysis.
*/
-
static int zipfileClose(sqlite3_vtab_cursor *cur){
-
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
-
  ZipfileTab *pTab = (ZipfileTab*)(pCsr->base.pVtab);
-
  ZipfileCsr **pp;
-
  zipfileResetCursor(pCsr);
-

-
  /* Remove this cursor from the ZipfileTab.pCsrList list. */
-
  for(pp=&pTab->pCsrList; *pp!=pCsr; pp=&((*pp)->pCsrNext));
-
  *pp = pCsr->pCsrNext;
+
int sqlite3_expert_sql(
+
  sqlite3expert *p,               /* From a successful sqlite3_expert_new() */
+
  const char *zSql,               /* SQL statement(s) to add */
+
  char **pzErr                    /* OUT: Error message (if any) */
+
);

-
  sqlite3_free(pCsr);
-
  return SQLITE_OK;
-
}

/*
-
** Set the error message for the virtual table associated with cursor
-
** pCsr to the results of vprintf(zFmt, ...).
+
** This function is called after the sqlite3expert object has been configured
+
** with all SQL statements using sqlite3_expert_sql() to actually perform
+
** the analysis. Once this function has been called, it is not possible to
+
** add further SQL statements to the analysis.
+
**
+
** If successful, SQLITE_OK is returned and (*pzErr) is set to NULL. Or, if
+
** an error occurs, an SQLite error code is returned and (*pzErr) set to 
+
** point to a buffer containing an English language error message. In this
+
** case it is the responsibility of the caller to eventually free the buffer
+
** using sqlite3_free().
+
**
+
** If an error does occur within this function, the sqlite3expert object
+
** is no longer useful for any purpose. At that point it is no longer
+
** possible to add further SQL statements to the object or to re-attempt
+
** the analysis. The sqlite3expert object must still be freed using a call
+
** sqlite3_expert_destroy().
*/
-
static void zipfileTableErr(ZipfileTab *pTab, const char *zFmt, ...){
-
  va_list ap;
-
  va_start(ap, zFmt);
-
  sqlite3_free(pTab->base.zErrMsg);
-
  pTab->base.zErrMsg = sqlite3_vmprintf(zFmt, ap);
-
  va_end(ap);
-
}
-
static void zipfileCursorErr(ZipfileCsr *pCsr, const char *zFmt, ...){
-
  va_list ap;
-
  va_start(ap, zFmt);
-
  sqlite3_free(pCsr->base.pVtab->zErrMsg);
-
  pCsr->base.pVtab->zErrMsg = sqlite3_vmprintf(zFmt, ap);
-
  va_end(ap);
-
}
+
int sqlite3_expert_analyze(sqlite3expert *p, char **pzErr);

/*
-
** Read nRead bytes of data from offset iOff of file pFile into buffer
-
** aRead[]. Return SQLITE_OK if successful, or an SQLite error code
-
** otherwise. 
-
**
-
** If an error does occur, output variable (*pzErrmsg) may be set to point
-
** to an English language error message. It is the responsibility of the
-
** caller to eventually free this buffer using
-
** sqlite3_free().
+
** Return the total number of statements loaded using sqlite3_expert_sql().
+
** The total number of SQL statements may be different from the total number
+
** to calls to sqlite3_expert_sql().
*/
-
static int zipfileReadData(
-
  FILE *pFile,                    /* Read from this file */
-
  u8 *aRead,                      /* Read into this buffer */
-
  int nRead,                      /* Number of bytes to read */
-
  i64 iOff,                       /* Offset to read from */
-
  char **pzErrmsg                 /* OUT: Error message (from sqlite3_malloc) */
-
){
-
  size_t n;
-
  fseek(pFile, (long)iOff, SEEK_SET);
-
  n = fread(aRead, 1, nRead, pFile);
-
  if( (int)n!=nRead ){
-
    *pzErrmsg = sqlite3_mprintf("error in fread()");
-
    return SQLITE_ERROR;
-
  }
-
  return SQLITE_OK;
-
}
-

-
static int zipfileAppendData(
-
  ZipfileTab *pTab,
-
  const u8 *aWrite,
-
  int nWrite
-
){
-
  if( nWrite>0 ){
-
    size_t n = nWrite;
-
    fseek(pTab->pWriteFd, (long)pTab->szCurrent, SEEK_SET);
-
    n = fwrite(aWrite, 1, nWrite, pTab->pWriteFd);
-
    if( (int)n!=nWrite ){
-
      pTab->base.zErrMsg = sqlite3_mprintf("error in fwrite()");
-
      return SQLITE_ERROR;
-
    }
-
    pTab->szCurrent += nWrite;
-
  }
-
  return SQLITE_OK;
-
}
+
int sqlite3_expert_count(sqlite3expert*);

/*
-
** Read and return a 16-bit little-endian unsigned integer from buffer aBuf.
+
** Return a component of the report.
+
**
+
** This function is called after sqlite3_expert_analyze() to extract the
+
** results of the analysis. Each call to this function returns either a
+
** NULL pointer or a pointer to a buffer containing a nul-terminated string.
+
** The value passed as the third argument must be one of the EXPERT_REPORT_*
+
** #define constants defined below.
+
**
+
** For some EXPERT_REPORT_* parameters, the buffer returned contains 
+
** information relating to a specific SQL statement. In these cases that
+
** SQL statement is identified by the value passed as the second argument.
+
** SQL statements are numbered from 0 in the order in which they are parsed.
+
** If an out-of-range value (less than zero or equal to or greater than the
+
** value returned by sqlite3_expert_count()) is passed as the second argument
+
** along with such an EXPERT_REPORT_* parameter, NULL is always returned.
+
**
+
** EXPERT_REPORT_SQL:
+
**   Return the text of SQL statement iStmt.
+
**
+
** EXPERT_REPORT_INDEXES:
+
**   Return a buffer containing the CREATE INDEX statements for all recommended
+
**   indexes for statement iStmt. If there are no new recommeded indexes, NULL 
+
**   is returned.
+
**
+
** EXPERT_REPORT_PLAN:
+
**   Return a buffer containing the EXPLAIN QUERY PLAN output for SQL query
+
**   iStmt after the proposed indexes have been added to the database schema.
+
**
+
** EXPERT_REPORT_CANDIDATES:
+
**   Return a pointer to a buffer containing the CREATE INDEX statements 
+
**   for all indexes that were tested (for all SQL statements). The iStmt
+
**   parameter is ignored for EXPERT_REPORT_CANDIDATES calls.
*/
-
static u16 zipfileGetU16(const u8 *aBuf){
-
  return (aBuf[1] << 8) + aBuf[0];
-
}
+
const char *sqlite3_expert_report(sqlite3expert*, int iStmt, int eReport);

/*
-
** Read and return a 32-bit little-endian unsigned integer from buffer aBuf.
-
*/
-
static u32 zipfileGetU32(const u8 *aBuf){
-
  if( aBuf==0 ) return 0;
-
  return ((u32)(aBuf[3]) << 24)
-
       + ((u32)(aBuf[2]) << 16)
-
       + ((u32)(aBuf[1]) <<  8)
-
       + ((u32)(aBuf[0]) <<  0);
-
}
+
** Values for the third argument passed to sqlite3_expert_report().
+
*/
+
#define EXPERT_REPORT_SQL        1
+
#define EXPERT_REPORT_INDEXES    2
+
#define EXPERT_REPORT_PLAN       3
+
#define EXPERT_REPORT_CANDIDATES 4

/*
-
** Write a 16-bit little endiate integer into buffer aBuf.
+
** Free an (sqlite3expert*) handle and all associated resources. There 
+
** should be one call to this function for each successful call to 
+
** sqlite3-expert_new().
*/
-
static void zipfilePutU16(u8 *aBuf, u16 val){
-
  aBuf[0] = val & 0xFF;
-
  aBuf[1] = (val>>8) & 0xFF;
-
}
+
void sqlite3_expert_destroy(sqlite3expert*);
+

+
#endif  /* !defined(SQLITEEXPERT_H) */

+
/************************* End ../ext/expert/sqlite3expert.h ********************/
+
/************************* Begin ../ext/expert/sqlite3expert.c ******************/
/*
-
** Write a 32-bit little endiate integer into buffer aBuf.
+
** 2017 April 09
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
*************************************************************************
*/
-
static void zipfilePutU32(u8 *aBuf, u32 val){
-
  aBuf[0] = val & 0xFF;
-
  aBuf[1] = (val>>8) & 0xFF;
-
  aBuf[2] = (val>>16) & 0xFF;
-
  aBuf[3] = (val>>24) & 0xFF;
-
}
+
/* #include "sqlite3expert.h" */
+
#include <assert.h>
+
#include <string.h>
+
#include <stdio.h>

-
#define zipfileRead32(aBuf) ( aBuf+=4, zipfileGetU32(aBuf-4) )
-
#define zipfileRead16(aBuf) ( aBuf+=2, zipfileGetU16(aBuf-2) )
+
#if !defined(SQLITE_AMALGAMATION)
+
#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
+
# define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1
+
#endif
+
#if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS)
+
# define ALWAYS(X)      (1)
+
# define NEVER(X)       (0)
+
#elif !defined(NDEBUG)
+
# define ALWAYS(X)      ((X)?1:(assert(0),0))
+
# define NEVER(X)       ((X)?(assert(0),1):0)
+
#else
+
# define ALWAYS(X)      (X)
+
# define NEVER(X)       (X)
+
#endif
+
#endif /* !defined(SQLITE_AMALGAMATION) */

-
#define zipfileWrite32(aBuf,val) { zipfilePutU32(aBuf,val); aBuf+=4; }
-
#define zipfileWrite16(aBuf,val) { zipfilePutU16(aBuf,val); aBuf+=2; }

-
/*
-
** Magic numbers used to read CDS records.
-
*/
-
#define ZIPFILE_CDS_NFILE_OFF        28
-
#define ZIPFILE_CDS_SZCOMPRESSED_OFF 20
+
#ifndef SQLITE_OMIT_VIRTUALTABLE 

-
/*
-
** Decode the CDS record in buffer aBuf into (*pCDS). Return SQLITE_ERROR
-
** if the record is not well-formed, or SQLITE_OK otherwise.
-
*/
-
static int zipfileReadCDS(u8 *aBuf, ZipfileCDS *pCDS){
-
  u8 *aRead = aBuf;
-
  u32 sig = zipfileRead32(aRead);
-
  int rc = SQLITE_OK;
-
  if( sig!=ZIPFILE_SIGNATURE_CDS ){
-
    rc = SQLITE_ERROR;
-
  }else{
-
    pCDS->iVersionMadeBy = zipfileRead16(aRead);
-
    pCDS->iVersionExtract = zipfileRead16(aRead);
-
    pCDS->flags = zipfileRead16(aRead);
-
    pCDS->iCompression = zipfileRead16(aRead);
-
    pCDS->mTime = zipfileRead16(aRead);
-
    pCDS->mDate = zipfileRead16(aRead);
-
    pCDS->crc32 = zipfileRead32(aRead);
-
    pCDS->szCompressed = zipfileRead32(aRead);
-
    pCDS->szUncompressed = zipfileRead32(aRead);
-
    assert( aRead==&aBuf[ZIPFILE_CDS_NFILE_OFF] );
-
    pCDS->nFile = zipfileRead16(aRead);
-
    pCDS->nExtra = zipfileRead16(aRead);
-
    pCDS->nComment = zipfileRead16(aRead);
-
    pCDS->iDiskStart = zipfileRead16(aRead);
-
    pCDS->iInternalAttr = zipfileRead16(aRead);
-
    pCDS->iExternalAttr = zipfileRead32(aRead);
-
    pCDS->iOffset = zipfileRead32(aRead);
-
    assert( aRead==&aBuf[ZIPFILE_CDS_FIXED_SZ] );
-
  }
+
/* typedef sqlite3_int64 i64; */
+
/* typedef sqlite3_uint64 u64; */

-
  return rc;
-
}
+
typedef struct IdxColumn IdxColumn;
+
typedef struct IdxConstraint IdxConstraint;
+
typedef struct IdxScan IdxScan;
+
typedef struct IdxStatement IdxStatement;
+
typedef struct IdxTable IdxTable;
+
typedef struct IdxWrite IdxWrite;
+

+
#define STRLEN  (int)strlen

/*
-
** Decode the LFH record in buffer aBuf into (*pLFH). Return SQLITE_ERROR
-
** if the record is not well-formed, or SQLITE_OK otherwise.
+
** A temp table name that we assume no user database will actually use.
+
** If this assumption proves incorrect triggers on the table with the
+
** conflicting name will be ignored.
*/
-
static int zipfileReadLFH(
-
  u8 *aBuffer,
-
  ZipfileLFH *pLFH
-
){
-
  u8 *aRead = aBuffer;
-
  int rc = SQLITE_OK;
-

-
  u32 sig = zipfileRead32(aRead);
-
  if( sig!=ZIPFILE_SIGNATURE_LFH ){
-
    rc = SQLITE_ERROR;
-
  }else{
-
    pLFH->iVersionExtract = zipfileRead16(aRead);
-
    pLFH->flags = zipfileRead16(aRead);
-
    pLFH->iCompression = zipfileRead16(aRead);
-
    pLFH->mTime = zipfileRead16(aRead);
-
    pLFH->mDate = zipfileRead16(aRead);
-
    pLFH->crc32 = zipfileRead32(aRead);
-
    pLFH->szCompressed = zipfileRead32(aRead);
-
    pLFH->szUncompressed = zipfileRead32(aRead);
-
    pLFH->nFile = zipfileRead16(aRead);
-
    pLFH->nExtra = zipfileRead16(aRead);
-
  }
-
  return rc;
-
}
-

+
#define UNIQUE_TABLE_NAME "t592690916721053953805701627921227776"

/*
-
** Buffer aExtra (size nExtra bytes) contains zip archive "extra" fields.
-
** Scan through this buffer to find an "extra-timestamp" field. If one
-
** exists, extract the 32-bit modification-timestamp from it and store
-
** the value in output parameter *pmTime.
-
**
-
** Zero is returned if no extra-timestamp record could be found (and so
-
** *pmTime is left unchanged), or non-zero otherwise.
-
**
-
** The general format of an extra field is:
+
** A single constraint. Equivalent to either "col = ?" or "col < ?" (or
+
** any other type of single-ended range constraint on a column).
**
-
**   Header ID    2 bytes
-
**   Data Size    2 bytes
-
**   Data         N bytes
+
** pLink:
+
**   Used to temporarily link IdxConstraint objects into lists while
+
**   creating candidate indexes.
*/
-
static int zipfileScanExtra(u8 *aExtra, int nExtra, u32 *pmTime){
-
  int ret = 0;
-
  u8 *p = aExtra;
-
  u8 *pEnd = &aExtra[nExtra];
-

-
  while( p<pEnd ){
-
    u16 id = zipfileRead16(p);
-
    u16 nByte = zipfileRead16(p);
+
struct IdxConstraint {
+
  char *zColl;                    /* Collation sequence */
+
  int bRange;                     /* True for range, false for eq */
+
  int iCol;                       /* Constrained table column */
+
  int bFlag;                      /* Used by idxFindCompatible() */
+
  int bDesc;                      /* True if ORDER BY <expr> DESC */
+
  IdxConstraint *pNext;           /* Next constraint in pEq or pRange list */
+
  IdxConstraint *pLink;           /* See above */
+
};

-
    switch( id ){
-
      case ZIPFILE_EXTRA_TIMESTAMP: {
-
        u8 b = p[0];
-
        if( b & 0x01 ){     /* 0x01 -> modtime is present */
-
          *pmTime = zipfileGetU32(&p[1]);
-
          ret = 1;
-
        }
-
        break;
-
      }
-
    }
+
/*
+
** A single scan of a single table.
+
*/
+
struct IdxScan {
+
  IdxTable *pTab;                 /* Associated table object */
+
  int iDb;                        /* Database containing table zTable */
+
  i64 covering;                   /* Mask of columns required for cov. index */
+
  IdxConstraint *pOrder;          /* ORDER BY columns */
+
  IdxConstraint *pEq;             /* List of == constraints */
+
  IdxConstraint *pRange;          /* List of < constraints */
+
  IdxScan *pNextScan;             /* Next IdxScan object for same analysis */
+
};

-
    p += nByte;
-
  }
-
  return ret;
-
}
+
/*
+
** Information regarding a single database table. Extracted from 
+
** "PRAGMA table_info" by function idxGetTableInfo().
+
*/
+
struct IdxColumn {
+
  char *zName;
+
  char *zColl;
+
  int iPk;
+
};
+
struct IdxTable {
+
  int nCol;
+
  char *zName;                    /* Table name */
+
  IdxColumn *aCol;
+
  IdxTable *pNext;                /* Next table in linked list of all tables */
+
};

/*
-
** Convert the standard MS-DOS timestamp stored in the mTime and mDate
-
** fields of the CDS structure passed as the only argument to a 32-bit
-
** UNIX seconds-since-the-epoch timestamp. Return the result.
-
**
-
** "Standard" MS-DOS time format:
-
**
-
**   File modification time:
-
**     Bits 00-04: seconds divided by 2
-
**     Bits 05-10: minute
-
**     Bits 11-15: hour
-
**   File modification date:
-
**     Bits 00-04: day
-
**     Bits 05-08: month (1-12)
-
**     Bits 09-15: years from 1980 
-
**
-
** https://msdn.microsoft.com/en-us/library/9kkf9tah.aspx
+
** An object of the following type is created for each unique table/write-op
+
** seen. The objects are stored in a singly-linked list beginning at
+
** sqlite3expert.pWrite.
*/
-
static u32 zipfileMtime(ZipfileCDS *pCDS){
-
  int Y,M,D,X1,X2,A,B,sec,min,hr;
-
  i64 JDsec;
-
  Y = (1980 + ((pCDS->mDate >> 9) & 0x7F));
-
  M = ((pCDS->mDate >> 5) & 0x0F);
-
  D = (pCDS->mDate & 0x1F);
-
  sec = (pCDS->mTime & 0x1F)*2;
-
  min = (pCDS->mTime >> 5) & 0x3F;
-
  hr = (pCDS->mTime >> 11) & 0x1F;
-
  if( M<=2 ){
-
    Y--;
-
    M += 12;
-
  }
-
  X1 = 36525*(Y+4716)/100;
-
  X2 = 306001*(M+1)/10000;
-
  A = Y/100;
-
  B = 2 - A + (A/4);
-
  JDsec = (i64)((X1 + X2 + D + B - 1524.5)*86400) + hr*3600 + min*60 + sec;
-
  return (u32)(JDsec - (i64)24405875*(i64)8640);
-
}
+
struct IdxWrite {
+
  IdxTable *pTab;
+
  int eOp;                        /* SQLITE_UPDATE, DELETE or INSERT */
+
  IdxWrite *pNext;
+
};

/*
-
** The opposite of zipfileMtime(). This function populates the mTime and
-
** mDate fields of the CDS structure passed as the first argument according
-
** to the UNIX timestamp value passed as the second.
+
** Each statement being analyzed is represented by an instance of this
+
** structure.
*/
-
static void zipfileMtimeToDos(ZipfileCDS *pCds, u32 mUnixTime){
-
  /* Convert unix timestamp to JD (2440588 is noon on 1/1/1970) */
-
  i64 JD = (i64)2440588 + mUnixTime / (24*60*60);
+
struct IdxStatement {
+
  int iId;                        /* Statement number */
+
  char *zSql;                     /* SQL statement */
+
  char *zIdx;                     /* Indexes */
+
  char *zEQP;                     /* Plan */
+
  IdxStatement *pNext;
+
};

-
  int A, B, C, D, E;
-
  int yr, mon, day;
-
  int hr, min, sec;

-
  A = (int)((JD - 1867216.25)/36524.25);
-
  A = (int)(JD + 1 + A - (A/4));
-
  B = A + 1524;
-
  C = (int)((B - 122.1)/365.25);
-
  D = (36525*(C&32767))/100;
-
  E = (int)((B-D)/30.6001);
+
/*
+
** A hash table for storing strings. With space for a payload string
+
** with each entry. Methods are:
+
**
+
**   idxHashInit()
+
**   idxHashClear()
+
**   idxHashAdd()
+
**   idxHashSearch()
+
*/
+
#define IDX_HASH_SIZE 1023
+
typedef struct IdxHashEntry IdxHashEntry;
+
typedef struct IdxHash IdxHash;
+
struct IdxHashEntry {
+
  char *zKey;                     /* nul-terminated key */
+
  char *zVal;                     /* nul-terminated value string */
+
  char *zVal2;                    /* nul-terminated value string 2 */
+
  IdxHashEntry *pHashNext;        /* Next entry in same hash bucket */
+
  IdxHashEntry *pNext;            /* Next entry in hash */
+
};
+
struct IdxHash {
+
  IdxHashEntry *pFirst;
+
  IdxHashEntry *aHash[IDX_HASH_SIZE];
+
};

-
  day = B - D - (int)(30.6001*E);
-
  mon = (E<14 ? E-1 : E-13);
-
  yr = mon>2 ? C-4716 : C-4715;
+
/*
+
** sqlite3expert object.
+
*/
+
struct sqlite3expert {
+
  int iSample;                    /* Percentage of tables to sample for stat1 */
+
  sqlite3 *db;                    /* User database */
+
  sqlite3 *dbm;                   /* In-memory db for this analysis */
+
  sqlite3 *dbv;                   /* Vtab schema for this analysis */
+
  IdxTable *pTable;               /* List of all IdxTable objects */
+
  IdxScan *pScan;                 /* List of scan objects */
+
  IdxWrite *pWrite;               /* List of write objects */
+
  IdxStatement *pStatement;       /* List of IdxStatement objects */
+
  int bRun;                       /* True once analysis has run */
+
  char **pzErrmsg;
+
  int rc;                         /* Error code from whereinfo hook */
+
  IdxHash hIdx;                   /* Hash containing all candidate indexes */
+
  char *zCandidates;              /* For EXPERT_REPORT_CANDIDATES */
+
};

-
  hr = (mUnixTime % (24*60*60)) / (60*60);
-
  min = (mUnixTime % (60*60)) / 60;
-
  sec = (mUnixTime % 60);

-
  if( yr>=1980 ){
-
    pCds->mDate = (u16)(day + (mon << 5) + ((yr-1980) << 9));
-
    pCds->mTime = (u16)(sec/2 + (min<<5) + (hr<<11));
+
/*
+
** Allocate and return nByte bytes of zeroed memory using sqlite3_malloc(). 
+
** If the allocation fails, set *pRc to SQLITE_NOMEM and return NULL.
+
*/
+
static void *idxMalloc(int *pRc, int nByte){
+
  void *pRet;
+
  assert( *pRc==SQLITE_OK );
+
  assert( nByte>0 );
+
  pRet = sqlite3_malloc(nByte);
+
  if( pRet ){
+
    memset(pRet, 0, nByte);
  }else{
-
    pCds->mDate = pCds->mTime = 0;
+
    *pRc = SQLITE_NOMEM;
  }
-

-
  assert( mUnixTime<315507600 
-
       || mUnixTime==zipfileMtime(pCds) 
-
       || ((mUnixTime % 2) && mUnixTime-1==zipfileMtime(pCds)) 
-
       /* || (mUnixTime % 2) */
-
  );
+
  return pRet;
}

/*
-
** If aBlob is not NULL, then it is a pointer to a buffer (nBlob bytes in
-
** size) containing an entire zip archive image. Or, if aBlob is NULL,
-
** then pFile is a file-handle open on a zip file. In either case, this
-
** function creates a ZipfileEntry object based on the zip archive entry
-
** for which the CDS record is at offset iOff.
-
**
-
** If successful, SQLITE_OK is returned and (*ppEntry) set to point to
-
** the new object. Otherwise, an SQLite error code is returned and the
-
** final value of (*ppEntry) undefined.
+
** Initialize an IdxHash hash table.
*/
-
static int zipfileGetEntry(
-
  ZipfileTab *pTab,               /* Store any error message here */
-
  const u8 *aBlob,                /* Pointer to in-memory file image */
-
  int nBlob,                      /* Size of aBlob[] in bytes */
-
  FILE *pFile,                    /* If aBlob==0, read from this file */
-
  i64 iOff,                       /* Offset of CDS record */
-
  ZipfileEntry **ppEntry          /* OUT: Pointer to new object */
-
){
-
  u8 *aRead;
-
  char **pzErr = &pTab->base.zErrMsg;
-
  int rc = SQLITE_OK;
+
static void idxHashInit(IdxHash *pHash){
+
  memset(pHash, 0, sizeof(IdxHash));
+
}

-
  if( aBlob==0 ){
-
    aRead = pTab->aBuffer;
-
    rc = zipfileReadData(pFile, aRead, ZIPFILE_CDS_FIXED_SZ, iOff, pzErr);
-
  }else{
-
    aRead = (u8*)&aBlob[iOff];
+
/*
+
** Reset an IdxHash hash table.
+
*/
+
static void idxHashClear(IdxHash *pHash){
+
  int i;
+
  for(i=0; i<IDX_HASH_SIZE; i++){
+
    IdxHashEntry *pEntry;
+
    IdxHashEntry *pNext;
+
    for(pEntry=pHash->aHash[i]; pEntry; pEntry=pNext){
+
      pNext = pEntry->pHashNext;
+
      sqlite3_free(pEntry->zVal2);
+
      sqlite3_free(pEntry);
+
    }
  }
+
  memset(pHash, 0, sizeof(IdxHash));
+
}

-
  if( rc==SQLITE_OK ){
-
    sqlite3_int64 nAlloc;
-
    ZipfileEntry *pNew;
-

-
    int nFile = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF]);
-
    int nExtra = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+2]);
-
    nExtra += zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+4]);
-

-
    nAlloc = sizeof(ZipfileEntry) + nExtra;
-
    if( aBlob ){
-
      nAlloc += zipfileGetU32(&aRead[ZIPFILE_CDS_SZCOMPRESSED_OFF]);
-
    }
+
/*
+
** Return the index of the hash bucket that the string specified by the
+
** arguments to this function belongs.
+
*/
+
static int idxHashString(const char *z, int n){
+
  unsigned int ret = 0;
+
  int i;
+
  for(i=0; i<n; i++){
+
    ret += (ret<<3) + (unsigned char)(z[i]);
+
  }
+
  return (int)(ret % IDX_HASH_SIZE);
+
}

-
    pNew = (ZipfileEntry*)sqlite3_malloc64(nAlloc);
-
    if( pNew==0 ){
-
      rc = SQLITE_NOMEM;
-
    }else{
-
      memset(pNew, 0, sizeof(ZipfileEntry));
-
      rc = zipfileReadCDS(aRead, &pNew->cds);
-
      if( rc!=SQLITE_OK ){
-
        *pzErr = sqlite3_mprintf("failed to read CDS at offset %lld", iOff);
-
      }else if( aBlob==0 ){
-
        rc = zipfileReadData(
-
            pFile, aRead, nExtra+nFile, iOff+ZIPFILE_CDS_FIXED_SZ, pzErr
-
        );
-
      }else{
-
        aRead = (u8*)&aBlob[iOff + ZIPFILE_CDS_FIXED_SZ];
-
      }
+
/*
+
** If zKey is already present in the hash table, return non-zero and do
+
** nothing. Otherwise, add an entry with key zKey and payload string zVal to
+
** the hash table passed as the second argument. 
+
*/
+
static int idxHashAdd(
+
  int *pRc, 
+
  IdxHash *pHash, 
+
  const char *zKey,
+
  const char *zVal
+
){
+
  int nKey = STRLEN(zKey);
+
  int iHash = idxHashString(zKey, nKey);
+
  int nVal = (zVal ? STRLEN(zVal) : 0);
+
  IdxHashEntry *pEntry;
+
  assert( iHash>=0 );
+
  for(pEntry=pHash->aHash[iHash]; pEntry; pEntry=pEntry->pHashNext){
+
    if( STRLEN(pEntry->zKey)==nKey && 0==memcmp(pEntry->zKey, zKey, nKey) ){
+
      return 1;
    }
-

-
    if( rc==SQLITE_OK ){
-
      u32 *pt = &pNew->mUnixTime;
-
      pNew->cds.zFile = sqlite3_mprintf("%.*s", nFile, aRead); 
-
      pNew->aExtra = (u8*)&pNew[1];
-
      memcpy(pNew->aExtra, &aRead[nFile], nExtra);
-
      if( pNew->cds.zFile==0 ){
-
        rc = SQLITE_NOMEM;
-
      }else if( 0==zipfileScanExtra(&aRead[nFile], pNew->cds.nExtra, pt) ){
-
        pNew->mUnixTime = zipfileMtime(&pNew->cds);
-
      }
+
  }
+
  pEntry = idxMalloc(pRc, sizeof(IdxHashEntry) + nKey+1 + nVal+1);
+
  if( pEntry ){
+
    pEntry->zKey = (char*)&pEntry[1];
+
    memcpy(pEntry->zKey, zKey, nKey);
+
    if( zVal ){
+
      pEntry->zVal = &pEntry->zKey[nKey+1];
+
      memcpy(pEntry->zVal, zVal, nVal);
    }
+
    pEntry->pHashNext = pHash->aHash[iHash];
+
    pHash->aHash[iHash] = pEntry;

-
    if( rc==SQLITE_OK ){
-
      static const int szFix = ZIPFILE_LFH_FIXED_SZ;
-
      ZipfileLFH lfh;
-
      if( pFile ){
-
        rc = zipfileReadData(pFile, aRead, szFix, pNew->cds.iOffset, pzErr);
-
      }else{
-
        aRead = (u8*)&aBlob[pNew->cds.iOffset];
-
      }
-

-
      if( rc==SQLITE_OK ) rc = zipfileReadLFH(aRead, &lfh);
-
      if( rc==SQLITE_OK ){
-
        pNew->iDataOff =  pNew->cds.iOffset + ZIPFILE_LFH_FIXED_SZ;
-
        pNew->iDataOff += lfh.nFile + lfh.nExtra;
-
        if( aBlob && pNew->cds.szCompressed ){
-
          pNew->aData = &pNew->aExtra[nExtra];
-
          memcpy(pNew->aData, &aBlob[pNew->iDataOff], pNew->cds.szCompressed);
-
        }
-
      }else{
-
        *pzErr = sqlite3_mprintf("failed to read LFH at offset %d", 
-
            (int)pNew->cds.iOffset
-
        );
-
      }
-
    }
+
    pEntry->pNext = pHash->pFirst;
+
    pHash->pFirst = pEntry;
+
  }
+
  return 0;
+
}

-
    if( rc!=SQLITE_OK ){
-
      zipfileEntryFree(pNew);
-
    }else{
-
      *ppEntry = pNew;
+
/*
+
** If zKey/nKey is present in the hash table, return a pointer to the 
+
** hash-entry object.
+
*/
+
static IdxHashEntry *idxHashFind(IdxHash *pHash, const char *zKey, int nKey){
+
  int iHash;
+
  IdxHashEntry *pEntry;
+
  if( nKey<0 ) nKey = STRLEN(zKey);
+
  iHash = idxHashString(zKey, nKey);
+
  assert( iHash>=0 );
+
  for(pEntry=pHash->aHash[iHash]; pEntry; pEntry=pEntry->pHashNext){
+
    if( STRLEN(pEntry->zKey)==nKey && 0==memcmp(pEntry->zKey, zKey, nKey) ){
+
      return pEntry;
    }
  }
+
  return 0;
+
}

-
  return rc;
+
/*
+
** If the hash table contains an entry with a key equal to the string
+
** passed as the final two arguments to this function, return a pointer
+
** to the payload string. Otherwise, if zKey/nKey is not present in the
+
** hash table, return NULL.
+
*/
+
static const char *idxHashSearch(IdxHash *pHash, const char *zKey, int nKey){
+
  IdxHashEntry *pEntry = idxHashFind(pHash, zKey, nKey);
+
  if( pEntry ) return pEntry->zVal;
+
  return 0;
}

/*
-
** Advance an ZipfileCsr to its next row of output.
+
** Allocate and return a new IdxConstraint object. Set the IdxConstraint.zColl
+
** variable to point to a copy of nul-terminated string zColl.
*/
-
static int zipfileNext(sqlite3_vtab_cursor *cur){
-
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
-
  int rc = SQLITE_OK;
+
static IdxConstraint *idxNewConstraint(int *pRc, const char *zColl){
+
  IdxConstraint *pNew;
+
  int nColl = STRLEN(zColl);

-
  if( pCsr->pFile ){
-
    i64 iEof = pCsr->eocd.iOffset + pCsr->eocd.nSize;
-
    zipfileEntryFree(pCsr->pCurrent);
-
    pCsr->pCurrent = 0;
-
    if( pCsr->iNextOff>=iEof ){
-
      pCsr->bEof = 1;
-
    }else{
-
      ZipfileEntry *p = 0;
-
      ZipfileTab *pTab = (ZipfileTab*)(cur->pVtab);
-
      rc = zipfileGetEntry(pTab, 0, 0, pCsr->pFile, pCsr->iNextOff, &p);
-
      if( rc==SQLITE_OK ){
-
        pCsr->iNextOff += ZIPFILE_CDS_FIXED_SZ;
-
        pCsr->iNextOff += (int)p->cds.nExtra + p->cds.nFile + p->cds.nComment;
-
      }
-
      pCsr->pCurrent = p;
-
    }
-
  }else{
-
    if( !pCsr->bNoop ){
-
      pCsr->pCurrent = pCsr->pCurrent->pNext;
-
    }
-
    if( pCsr->pCurrent==0 ){
-
      pCsr->bEof = 1;
-
    }
+
  assert( *pRc==SQLITE_OK );
+
  pNew = (IdxConstraint*)idxMalloc(pRc, sizeof(IdxConstraint) * nColl + 1);
+
  if( pNew ){
+
    pNew->zColl = (char*)&pNew[1];
+
    memcpy(pNew->zColl, zColl, nColl+1);
  }
-

-
  pCsr->bNoop = 0;
-
  return rc;
-
}
-

-
static void zipfileFree(void *p) { 
-
  sqlite3_free(p); 
+
  return pNew;
}

/*
-
** Buffer aIn (size nIn bytes) contains compressed data. Uncompressed, the
-
** size is nOut bytes. This function uncompresses the data and sets the
-
** return value in context pCtx to the result (a blob).
-
**
-
** If an error occurs, an error code is left in pCtx instead.
+
** An error associated with database handle db has just occurred. Pass
+
** the error message to callback function xOut.
*/
-
static void zipfileInflate(
-
  sqlite3_context *pCtx,          /* Store result here */
-
  const u8 *aIn,                  /* Compressed data */
-
  int nIn,                        /* Size of buffer aIn[] in bytes */
-
  int nOut                        /* Expected output size */
+
static void idxDatabaseError(
+
  sqlite3 *db,                    /* Database handle */
+
  char **pzErrmsg                 /* Write error here */
){
-
  u8 *aRes = sqlite3_malloc(nOut);
-
  if( aRes==0 ){
-
    sqlite3_result_error_nomem(pCtx);
-
  }else{
-
    int err;
-
    z_stream str;
-
    memset(&str, 0, sizeof(str));
-

-
    str.next_in = (Byte*)aIn;
-
    str.avail_in = nIn;
-
    str.next_out = (Byte*)aRes;
-
    str.avail_out = nOut;
-

-
    err = inflateInit2(&str, -15);
-
    if( err!=Z_OK ){
-
      zipfileCtxErrorMsg(pCtx, "inflateInit2() failed (%d)", err);
-
    }else{
-
      err = inflate(&str, Z_NO_FLUSH);
-
      if( err!=Z_STREAM_END ){
-
        zipfileCtxErrorMsg(pCtx, "inflate() failed (%d)", err);
-
      }else{
-
        sqlite3_result_blob(pCtx, aRes, nOut, zipfileFree);
-
        aRes = 0;
-
      }
-
    }
-
    sqlite3_free(aRes);
-
    inflateEnd(&str);
-
  }
+
  *pzErrmsg = sqlite3_mprintf("%s", sqlite3_errmsg(db));
}

/*
-
** Buffer aIn (size nIn bytes) contains uncompressed data. This function
-
** compresses it and sets (*ppOut) to point to a buffer containing the
-
** compressed data. The caller is responsible for eventually calling
-
** sqlite3_free() to release buffer (*ppOut). Before returning, (*pnOut) 
-
** is set to the size of buffer (*ppOut) in bytes.
-
**
-
** If no error occurs, SQLITE_OK is returned. Otherwise, an SQLite error
-
** code is returned and an error message left in virtual-table handle
-
** pTab. The values of (*ppOut) and (*pnOut) are left unchanged in this
-
** case.
+
** Prepare an SQL statement.
*/
-
static int zipfileDeflate(
-
  const u8 *aIn, int nIn,         /* Input */
-
  u8 **ppOut, int *pnOut,         /* Output */
-
  char **pzErr                    /* OUT: Error message */
+
static int idxPrepareStmt(
+
  sqlite3 *db,                    /* Database handle to compile against */
+
  sqlite3_stmt **ppStmt,          /* OUT: Compiled SQL statement */
+
  char **pzErrmsg,                /* OUT: sqlite3_malloc()ed error message */
+
  const char *zSql                /* SQL statement to compile */
){
-
  int rc = SQLITE_OK;
-
  sqlite3_int64 nAlloc;
-
  z_stream str;
-
  u8 *aOut;
-

-
  memset(&str, 0, sizeof(str));
-
  str.next_in = (Bytef*)aIn;
-
  str.avail_in = nIn;
-
  deflateInit2(&str, 9, Z_DEFLATED, -15, 8, Z_DEFAULT_STRATEGY);
-

-
  nAlloc = deflateBound(&str, nIn);
-
  aOut = (u8*)sqlite3_malloc64(nAlloc);
-
  if( aOut==0 ){
-
    rc = SQLITE_NOMEM;
-
  }else{
-
    int res;
-
    str.next_out = aOut;
-
    str.avail_out = nAlloc;
-
    res = deflate(&str, Z_FINISH);
-
    if( res==Z_STREAM_END ){
-
      *ppOut = aOut;
-
      *pnOut = (int)str.total_out;
-
    }else{
-
      sqlite3_free(aOut);
-
      *pzErr = sqlite3_mprintf("zipfile: deflate() error");
-
      rc = SQLITE_ERROR;
-
    }
-
    deflateEnd(&str);
+
  int rc = sqlite3_prepare_v2(db, zSql, -1, ppStmt, 0);
+
  if( rc!=SQLITE_OK ){
+
    *ppStmt = 0;
+
    idxDatabaseError(db, pzErrmsg);
  }
-

  return rc;
}

-

/*
-
** Return values of columns for the row at which the series_cursor
-
** is currently pointing.
+
** Prepare an SQL statement using the results of a printf() formatting.
*/
-
static int zipfileColumn(
-
  sqlite3_vtab_cursor *cur,   /* The cursor */
-
  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
-
  int i                       /* Which column to return */
+
static int idxPrintfPrepareStmt(
+
  sqlite3 *db,                    /* Database handle to compile against */
+
  sqlite3_stmt **ppStmt,          /* OUT: Compiled SQL statement */
+
  char **pzErrmsg,                /* OUT: sqlite3_malloc()ed error message */
+
  const char *zFmt,               /* printf() format of SQL statement */
+
  ...                             /* Trailing printf() arguments */
){
-
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
-
  ZipfileCDS *pCDS = &pCsr->pCurrent->cds;
-
  int rc = SQLITE_OK;
-
  switch( i ){
-
    case 0:   /* name */
-
      sqlite3_result_text(ctx, pCDS->zFile, -1, SQLITE_TRANSIENT);
-
      break;
-
    case 1:   /* mode */
-
      /* TODO: Whether or not the following is correct surely depends on
-
      ** the platform on which the archive was created.  */
-
      sqlite3_result_int(ctx, pCDS->iExternalAttr >> 16);
-
      break;
-
    case 2: { /* mtime */
-
      sqlite3_result_int64(ctx, pCsr->pCurrent->mUnixTime);
-
      break;
-
    }
-
    case 3: { /* sz */
-
      if( sqlite3_vtab_nochange(ctx)==0 ){
-
        sqlite3_result_int64(ctx, pCDS->szUncompressed);
-
      }
-
      break;
-
    }
-
    case 4:   /* rawdata */
-
      if( sqlite3_vtab_nochange(ctx) ) break;
-
    case 5: { /* data */
-
      if( i==4 || pCDS->iCompression==0 || pCDS->iCompression==8 ){
-
        int sz = pCDS->szCompressed;
-
        int szFinal = pCDS->szUncompressed;
-
        if( szFinal>0 ){
-
          u8 *aBuf;
-
          u8 *aFree = 0;
-
          if( pCsr->pCurrent->aData ){
-
            aBuf = pCsr->pCurrent->aData;
-
          }else{
-
            aBuf = aFree = sqlite3_malloc64(sz);
-
            if( aBuf==0 ){
-
              rc = SQLITE_NOMEM;
-
            }else{
-
              FILE *pFile = pCsr->pFile;
-
              if( pFile==0 ){
-
                pFile = ((ZipfileTab*)(pCsr->base.pVtab))->pWriteFd;
-
              }
-
              rc = zipfileReadData(pFile, aBuf, sz, pCsr->pCurrent->iDataOff,
-
                  &pCsr->base.pVtab->zErrMsg
-
              );
-
            }
-
          }
-
          if( rc==SQLITE_OK ){
-
            if( i==5 && pCDS->iCompression ){
-
              zipfileInflate(ctx, aBuf, sz, szFinal);
-
            }else{
-
              sqlite3_result_blob(ctx, aBuf, sz, SQLITE_TRANSIENT);
-
            }
-
          }
-
          sqlite3_free(aFree);
-
        }else{
-
          /* Figure out if this is a directory or a zero-sized file. Consider
-
          ** it to be a directory either if the mode suggests so, or if
-
          ** the final character in the name is '/'.  */
-
          u32 mode = pCDS->iExternalAttr >> 16;
-
          if( !(mode & S_IFDIR) && pCDS->zFile[pCDS->nFile-1]!='/' ){
-
            sqlite3_result_blob(ctx, "", 0, SQLITE_STATIC);
-
          }
-
        }
-
      }
-
      break;
-
    }
-
    case 6:   /* method */
-
      sqlite3_result_int(ctx, pCDS->iCompression);
-
      break;
-
    default:  /* z */
-
      assert( i==7 );
-
      sqlite3_result_int64(ctx, pCsr->iId);
-
      break;
+
  va_list ap;
+
  int rc;
+
  char *zSql;
+
  va_start(ap, zFmt);
+
  zSql = sqlite3_vmprintf(zFmt, ap);
+
  if( zSql==0 ){
+
    rc = SQLITE_NOMEM;
+
  }else{
+
    rc = idxPrepareStmt(db, ppStmt, pzErrmsg, zSql);
+
    sqlite3_free(zSql);
  }
-

+
  va_end(ap);
  return rc;
}

-
/*
-
** Return TRUE if the cursor is at EOF.
+

+
/*************************************************************************
+
** Beginning of virtual table implementation.
*/
-
static int zipfileEof(sqlite3_vtab_cursor *cur){
-
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
-
  return pCsr->bEof;
+
typedef struct ExpertVtab ExpertVtab;
+
struct ExpertVtab {
+
  sqlite3_vtab base;
+
  IdxTable *pTab;
+
  sqlite3expert *pExpert;
+
};
+

+
typedef struct ExpertCsr ExpertCsr;
+
struct ExpertCsr {
+
  sqlite3_vtab_cursor base;
+
  sqlite3_stmt *pData;
+
};
+

+
static char *expertDequote(const char *zIn){
+
  int n = STRLEN(zIn);
+
  char *zRet = sqlite3_malloc(n);
+

+
  assert( zIn[0]=='\'' );
+
  assert( zIn[n-1]=='\'' );
+

+
  if( zRet ){
+
    int iOut = 0;
+
    int iIn = 0;
+
    for(iIn=1; iIn<(n-1); iIn++){
+
      if( zIn[iIn]=='\'' ){
+
        assert( zIn[iIn+1]=='\'' );
+
        iIn++;
+
      }
+
      zRet[iOut++] = zIn[iIn];
+
    }
+
    zRet[iOut] = '\0';
+
  }
+

+
  return zRet;
}

-
/*
-
** If aBlob is not NULL, then it points to a buffer nBlob bytes in size
-
** containing an entire zip archive image. Or, if aBlob is NULL, then pFile
-
** is guaranteed to be a file-handle open on a zip file.
+
/* 
+
** This function is the implementation of both the xConnect and xCreate
+
** methods of the r-tree virtual table.
**
-
** This function attempts to locate the EOCD record within the zip archive
-
** and populate *pEOCD with the results of decoding it. SQLITE_OK is
-
** returned if successful. Otherwise, an SQLite error code is returned and
-
** an English language error message may be left in virtual-table pTab.
+
**   argv[0]   -> module name
+
**   argv[1]   -> database name
+
**   argv[2]   -> table name
+
**   argv[...] -> column names...
*/
-
static int zipfileReadEOCD(
-
  ZipfileTab *pTab,               /* Return errors here */
-
  const u8 *aBlob,                /* Pointer to in-memory file image */
-
  int nBlob,                      /* Size of aBlob[] in bytes */
-
  FILE *pFile,                    /* Read from this file if aBlob==0 */
-
  ZipfileEOCD *pEOCD              /* Object to populate */
+
static int expertConnect(
+
  sqlite3 *db,
+
  void *pAux,
+
  int argc, const char *const*argv,
+
  sqlite3_vtab **ppVtab,
+
  char **pzErr
){
-
  u8 *aRead = pTab->aBuffer;      /* Temporary buffer */
-
  int nRead;                      /* Bytes to read from file */
-
  int rc = SQLITE_OK;
+
  sqlite3expert *pExpert = (sqlite3expert*)pAux;
+
  ExpertVtab *p = 0;
+
  int rc;

-
  memset(pEOCD, 0, sizeof(ZipfileEOCD));
-
  if( aBlob==0 ){
-
    i64 iOff;                     /* Offset to read from */
-
    i64 szFile;                   /* Total size of file in bytes */
-
    fseek(pFile, 0, SEEK_END);
-
    szFile = (i64)ftell(pFile);
-
    if( szFile==0 ){
-
      return SQLITE_OK;
-
    }
-
    nRead = (int)(MIN(szFile, ZIPFILE_BUFFER_SIZE));
-
    iOff = szFile - nRead;
-
    rc = zipfileReadData(pFile, aRead, nRead, iOff, &pTab->base.zErrMsg);
+
  if( argc!=4 ){
+
    *pzErr = sqlite3_mprintf("internal error!");
+
    rc = SQLITE_ERROR;
  }else{
-
    nRead = (int)(MIN(nBlob, ZIPFILE_BUFFER_SIZE));
-
    aRead = (u8*)&aBlob[nBlob-nRead];
+
    char *zCreateTable = expertDequote(argv[3]);
+
    if( zCreateTable ){
+
      rc = sqlite3_declare_vtab(db, zCreateTable);
+
      if( rc==SQLITE_OK ){
+
        p = idxMalloc(&rc, sizeof(ExpertVtab));
+
      }
+
      if( rc==SQLITE_OK ){
+
        p->pExpert = pExpert;
+
        p->pTab = pExpert->pTable;
+
        assert( sqlite3_stricmp(p->pTab->zName, argv[2])==0 );
+
      }
+
      sqlite3_free(zCreateTable);
+
    }else{
+
      rc = SQLITE_NOMEM;
+
    }
  }

-
  if( rc==SQLITE_OK ){
+
  *ppVtab = (sqlite3_vtab*)p;
+
  return rc;
+
}
+

+
static int expertDisconnect(sqlite3_vtab *pVtab){
+
  ExpertVtab *p = (ExpertVtab*)pVtab;
+
  sqlite3_free(p);
+
  return SQLITE_OK;
+
}
+

+
static int expertBestIndex(sqlite3_vtab *pVtab, sqlite3_index_info *pIdxInfo){
+
  ExpertVtab *p = (ExpertVtab*)pVtab;
+
  int rc = SQLITE_OK;
+
  int n = 0;
+
  IdxScan *pScan;
+
  const int opmask = 
+
    SQLITE_INDEX_CONSTRAINT_EQ | SQLITE_INDEX_CONSTRAINT_GT |
+
    SQLITE_INDEX_CONSTRAINT_LT | SQLITE_INDEX_CONSTRAINT_GE |
+
    SQLITE_INDEX_CONSTRAINT_LE;
+

+
  pScan = idxMalloc(&rc, sizeof(IdxScan));
+
  if( pScan ){
    int i;

-
    /* Scan backwards looking for the signature bytes */
-
    for(i=nRead-20; i>=0; i--){
-
      if( aRead[i]==0x50 && aRead[i+1]==0x4b 
-
       && aRead[i+2]==0x05 && aRead[i+3]==0x06 
+
    /* Link the new scan object into the list */
+
    pScan->pTab = p->pTab;
+
    pScan->pNextScan = p->pExpert->pScan;
+
    p->pExpert->pScan = pScan;
+

+
    /* Add the constraints to the IdxScan object */
+
    for(i=0; i<pIdxInfo->nConstraint; i++){
+
      struct sqlite3_index_constraint *pCons = &pIdxInfo->aConstraint[i];
+
      if( pCons->usable 
+
       && pCons->iColumn>=0 
+
       && p->pTab->aCol[pCons->iColumn].iPk==0
+
       && (pCons->op & opmask) 
      ){
-
        break;
+
        IdxConstraint *pNew;
+
        const char *zColl = sqlite3_vtab_collation(pIdxInfo, i);
+
        pNew = idxNewConstraint(&rc, zColl);
+
        if( pNew ){
+
          pNew->iCol = pCons->iColumn;
+
          if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ ){
+
            pNew->pNext = pScan->pEq;
+
            pScan->pEq = pNew;
+
          }else{
+
            pNew->bRange = 1;
+
            pNew->pNext = pScan->pRange;
+
            pScan->pRange = pNew;
+
          }
+
        }
+
        n++;
+
        pIdxInfo->aConstraintUsage[i].argvIndex = n;
      }
    }
-
    if( i<0 ){
-
      pTab->base.zErrMsg = sqlite3_mprintf(
-
          "cannot find end of central directory record"
-
      );
-
      return SQLITE_ERROR;
-
    }

-
    aRead += i+4;
-
    pEOCD->iDisk = zipfileRead16(aRead);
-
    pEOCD->iFirstDisk = zipfileRead16(aRead);
-
    pEOCD->nEntry = zipfileRead16(aRead);
-
    pEOCD->nEntryTotal = zipfileRead16(aRead);
-
    pEOCD->nSize = zipfileRead32(aRead);
-
    pEOCD->iOffset = zipfileRead32(aRead);
+
    /* Add the ORDER BY to the IdxScan object */
+
    for(i=pIdxInfo->nOrderBy-1; i>=0; i--){
+
      int iCol = pIdxInfo->aOrderBy[i].iColumn;
+
      if( iCol>=0 ){
+
        IdxConstraint *pNew = idxNewConstraint(&rc, p->pTab->aCol[iCol].zColl);
+
        if( pNew ){
+
          pNew->iCol = iCol;
+
          pNew->bDesc = pIdxInfo->aOrderBy[i].desc;
+
          pNew->pNext = pScan->pOrder;
+
          pNew->pLink = pScan->pOrder;
+
          pScan->pOrder = pNew;
+
          n++;
+
        }
+
      }
+
    }
  }

+
  pIdxInfo->estimatedCost = 1000000.0 / (n+1);
  return rc;
}

-
/*
-
** Add object pNew to the linked list that begins at ZipfileTab.pFirstEntry 
-
** and ends with pLastEntry. If argument pBefore is NULL, then pNew is added
-
** to the end of the list. Otherwise, it is added to the list immediately
-
** before pBefore (which is guaranteed to be a part of said list).
-
*/
-
static void zipfileAddEntry(
-
  ZipfileTab *pTab, 
-
  ZipfileEntry *pBefore, 
-
  ZipfileEntry *pNew
+
static int expertUpdate(
+
  sqlite3_vtab *pVtab, 
+
  int nData, 
+
  sqlite3_value **azData, 
+
  sqlite_int64 *pRowid
){
-
  assert( (pTab->pFirstEntry==0)==(pTab->pLastEntry==0) );
-
  assert( pNew->pNext==0 );
-
  if( pBefore==0 ){
-
    if( pTab->pFirstEntry==0 ){
-
      pTab->pFirstEntry = pTab->pLastEntry = pNew;
-
    }else{
-
      assert( pTab->pLastEntry->pNext==0 );
-
      pTab->pLastEntry->pNext = pNew;
-
      pTab->pLastEntry = pNew;
-
    }
-
  }else{
-
    ZipfileEntry **pp;
-
    for(pp=&pTab->pFirstEntry; *pp!=pBefore; pp=&((*pp)->pNext));
-
    pNew->pNext = pBefore;
-
    *pp = pNew;
-
  }
+
  (void)pVtab;
+
  (void)nData;
+
  (void)azData;
+
  (void)pRowid;
+
  return SQLITE_OK;
}

-
static int zipfileLoadDirectory(ZipfileTab *pTab, const u8 *aBlob, int nBlob){
-
  ZipfileEOCD eocd;
-
  int rc;
-
  int i;
-
  i64 iOff;
-

-
  rc = zipfileReadEOCD(pTab, aBlob, nBlob, pTab->pWriteFd, &eocd);
-
  iOff = eocd.iOffset;
-
  for(i=0; rc==SQLITE_OK && i<eocd.nEntry; i++){
-
    ZipfileEntry *pNew = 0;
-
    rc = zipfileGetEntry(pTab, aBlob, nBlob, pTab->pWriteFd, iOff, &pNew);
-

-
    if( rc==SQLITE_OK ){
-
      zipfileAddEntry(pTab, 0, pNew);
-
      iOff += ZIPFILE_CDS_FIXED_SZ;
-
      iOff += (int)pNew->cds.nExtra + pNew->cds.nFile + pNew->cds.nComment;
-
    }
-
  }
+
/* 
+
** Virtual table module xOpen method.
+
*/
+
static int expertOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
+
  int rc = SQLITE_OK;
+
  ExpertCsr *pCsr;
+
  (void)pVTab;
+
  pCsr = idxMalloc(&rc, sizeof(ExpertCsr));
+
  *ppCursor = (sqlite3_vtab_cursor*)pCsr;
  return rc;
}

-
/*
-
** xFilter callback.
+
/* 
+
** Virtual table module xClose method.
*/
-
static int zipfileFilter(
-
  sqlite3_vtab_cursor *cur, 
-
  int idxNum, const char *idxStr,
-
  int argc, sqlite3_value **argv
-
){
-
  ZipfileTab *pTab = (ZipfileTab*)cur->pVtab;
-
  ZipfileCsr *pCsr = (ZipfileCsr*)cur;
-
  const char *zFile = 0;          /* Zip file to scan */
-
  int rc = SQLITE_OK;             /* Return Code */
-
  int bInMemory = 0;              /* True for an in-memory zipfile */
+
static int expertClose(sqlite3_vtab_cursor *cur){
+
  ExpertCsr *pCsr = (ExpertCsr*)cur;
+
  sqlite3_finalize(pCsr->pData);
+
  sqlite3_free(pCsr);
+
  return SQLITE_OK;
+
}

-
  zipfileResetCursor(pCsr);
+
/*
+
** Virtual table module xEof method.
+
**
+
** Return non-zero if the cursor does not currently point to a valid 
+
** record (i.e if the scan has finished), or zero otherwise.
+
*/
+
static int expertEof(sqlite3_vtab_cursor *cur){
+
  ExpertCsr *pCsr = (ExpertCsr*)cur;
+
  return pCsr->pData==0;
+
}

-
  if( pTab->zFile ){
-
    zFile = pTab->zFile;
-
  }else if( idxNum==0 ){
-
    zipfileCursorErr(pCsr, "zipfile() function requires an argument");
-
    return SQLITE_ERROR;
-
  }else if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){
-
    static const u8 aEmptyBlob = 0;
-
    const u8 *aBlob = (const u8*)sqlite3_value_blob(argv[0]);
-
    int nBlob = sqlite3_value_bytes(argv[0]);
-
    assert( pTab->pFirstEntry==0 );
-
    if( aBlob==0 ){
-
      aBlob = &aEmptyBlob;
-
      nBlob = 0;
-
    }
-
    rc = zipfileLoadDirectory(pTab, aBlob, nBlob);
-
    pCsr->pFreeEntry = pTab->pFirstEntry;
-
    pTab->pFirstEntry = pTab->pLastEntry = 0;
-
    if( rc!=SQLITE_OK ) return rc;
-
    bInMemory = 1;
-
  }else{
-
    zFile = (const char*)sqlite3_value_text(argv[0]);
-
  }
+
/* 
+
** Virtual table module xNext method.
+
*/
+
static int expertNext(sqlite3_vtab_cursor *cur){
+
  ExpertCsr *pCsr = (ExpertCsr*)cur;
+
  int rc = SQLITE_OK;

-
  if( 0==pTab->pWriteFd && 0==bInMemory ){
-
    pCsr->pFile = fopen(zFile, "rb");
-
    if( pCsr->pFile==0 ){
-
      zipfileCursorErr(pCsr, "cannot open file: %s", zFile);
-
      rc = SQLITE_ERROR;
-
    }else{
-
      rc = zipfileReadEOCD(pTab, 0, 0, pCsr->pFile, &pCsr->eocd);
-
      if( rc==SQLITE_OK ){
-
        if( pCsr->eocd.nEntry==0 ){
-
          pCsr->bEof = 1;
-
        }else{
-
          pCsr->iNextOff = pCsr->eocd.iOffset;
-
          rc = zipfileNext(cur);
-
        }
-
      }
-
    }
+
  assert( pCsr->pData );
+
  rc = sqlite3_step(pCsr->pData);
+
  if( rc!=SQLITE_ROW ){
+
    rc = sqlite3_finalize(pCsr->pData);
+
    pCsr->pData = 0;
  }else{
-
    pCsr->bNoop = 1;
-
    pCsr->pCurrent = pCsr->pFreeEntry ? pCsr->pFreeEntry : pTab->pFirstEntry;
-
    rc = zipfileNext(cur);
+
    rc = SQLITE_OK;
  }

  return rc;
}

-
/*
-
** xBestIndex callback.
+
/* 
+
** Virtual table module xRowid method.
*/
-
static int zipfileBestIndex(
-
  sqlite3_vtab *tab,
-
  sqlite3_index_info *pIdxInfo
-
){
-
  int i;
-
  int idx = -1;
-
  int unusable = 0;
-

-
  for(i=0; i<pIdxInfo->nConstraint; i++){
-
    const struct sqlite3_index_constraint *pCons = &pIdxInfo->aConstraint[i];
-
    if( pCons->iColumn!=ZIPFILE_F_COLUMN_IDX ) continue;
-
    if( pCons->usable==0 ){
-
      unusable = 1;
-
    }else if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ ){
-
      idx = i;
-
    }
-
  }
-
  pIdxInfo->estimatedCost = 1000.0;
-
  if( idx>=0 ){
-
    pIdxInfo->aConstraintUsage[idx].argvIndex = 1;
-
    pIdxInfo->aConstraintUsage[idx].omit = 1;
-
    pIdxInfo->idxNum = 1;
-
  }else if( unusable ){
-
    return SQLITE_CONSTRAINT;
-
  }
+
static int expertRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
+
  (void)cur;
+
  *pRowid = 0;
  return SQLITE_OK;
}

-
static ZipfileEntry *zipfileNewEntry(const char *zPath){
-
  ZipfileEntry *pNew;
-
  pNew = sqlite3_malloc(sizeof(ZipfileEntry));
-
  if( pNew ){
-
    memset(pNew, 0, sizeof(ZipfileEntry));
-
    pNew->cds.zFile = sqlite3_mprintf("%s", zPath);
-
    if( pNew->cds.zFile==0 ){
-
      sqlite3_free(pNew);
-
      pNew = 0;
-
    }
+
/* 
+
** Virtual table module xColumn method.
+
*/
+
static int expertColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i){
+
  ExpertCsr *pCsr = (ExpertCsr*)cur;
+
  sqlite3_value *pVal;
+
  pVal = sqlite3_column_value(pCsr->pData, i);
+
  if( pVal ){
+
    sqlite3_result_value(ctx, pVal);
  }
-
  return pNew;
+
  return SQLITE_OK;
}

-
static int zipfileSerializeLFH(ZipfileEntry *pEntry, u8 *aBuf){
-
  ZipfileCDS *pCds = &pEntry->cds;
-
  u8 *a = aBuf;
-

-
  pCds->nExtra = 9;
+
/* 
+
** Virtual table module xFilter method.
+
*/
+
static int expertFilter(
+
  sqlite3_vtab_cursor *cur, 
+
  int idxNum, const char *idxStr,
+
  int argc, sqlite3_value **argv
+
){
+
  ExpertCsr *pCsr = (ExpertCsr*)cur;
+
  ExpertVtab *pVtab = (ExpertVtab*)(cur->pVtab);
+
  sqlite3expert *pExpert = pVtab->pExpert;
+
  int rc;

-
  /* Write the LFH itself */
-
  zipfileWrite32(a, ZIPFILE_SIGNATURE_LFH);
-
  zipfileWrite16(a, pCds->iVersionExtract);
-
  zipfileWrite16(a, pCds->flags);
-
  zipfileWrite16(a, pCds->iCompression);
-
  zipfileWrite16(a, pCds->mTime);
-
  zipfileWrite16(a, pCds->mDate);
-
  zipfileWrite32(a, pCds->crc32);
-
  zipfileWrite32(a, pCds->szCompressed);
-
  zipfileWrite32(a, pCds->szUncompressed);
-
  zipfileWrite16(a, (u16)pCds->nFile);
-
  zipfileWrite16(a, pCds->nExtra);
-
  assert( a==&aBuf[ZIPFILE_LFH_FIXED_SZ] );
+
  (void)idxNum;
+
  (void)idxStr;
+
  (void)argc;
+
  (void)argv;
+
  rc = sqlite3_finalize(pCsr->pData);
+
  pCsr->pData = 0;
+
  if( rc==SQLITE_OK ){
+
    rc = idxPrintfPrepareStmt(pExpert->db, &pCsr->pData, &pVtab->base.zErrMsg,
+
        "SELECT * FROM main.%Q WHERE sample()", pVtab->pTab->zName
+
    );
+
  }

-
  /* Add the file name */
-
  memcpy(a, pCds->zFile, (int)pCds->nFile);
-
  a += (int)pCds->nFile;
+
  if( rc==SQLITE_OK ){
+
    rc = expertNext(cur);
+
  }
+
  return rc;
+
}

-
  /* The "extra" data */
-
  zipfileWrite16(a, ZIPFILE_EXTRA_TIMESTAMP);
-
  zipfileWrite16(a, 5);
-
  *a++ = 0x01;
-
  zipfileWrite32(a, pEntry->mUnixTime);
+
static int idxRegisterVtab(sqlite3expert *p){
+
  static sqlite3_module expertModule = {
+
    2,                            /* iVersion */
+
    expertConnect,                /* xCreate - create a table */
+
    expertConnect,                /* xConnect - connect to an existing table */
+
    expertBestIndex,              /* xBestIndex - Determine search strategy */
+
    expertDisconnect,             /* xDisconnect - Disconnect from a table */
+
    expertDisconnect,             /* xDestroy - Drop a table */
+
    expertOpen,                   /* xOpen - open a cursor */
+
    expertClose,                  /* xClose - close a cursor */
+
    expertFilter,                 /* xFilter - configure scan constraints */
+
    expertNext,                   /* xNext - advance a cursor */
+
    expertEof,                    /* xEof */
+
    expertColumn,                 /* xColumn - read data */
+
    expertRowid,                  /* xRowid - read data */
+
    expertUpdate,                 /* xUpdate - write data */
+
    0,                            /* xBegin - begin transaction */
+
    0,                            /* xSync - sync transaction */
+
    0,                            /* xCommit - commit transaction */
+
    0,                            /* xRollback - rollback transaction */
+
    0,                            /* xFindFunction - function overloading */
+
    0,                            /* xRename - rename the table */
+
    0,                            /* xSavepoint */
+
    0,                            /* xRelease */
+
    0,                            /* xRollbackTo */
+
    0,                            /* xShadowName */
+
  };

-
  return a-aBuf;
+
  return sqlite3_create_module(p->dbv, "expert", &expertModule, (void*)p);
+
}
+
/*
+
** End of virtual table implementation.
+
*************************************************************************/
+
/*
+
** Finalize SQL statement pStmt. If (*pRc) is SQLITE_OK when this function
+
** is called, set it to the return value of sqlite3_finalize() before
+
** returning. Otherwise, discard the sqlite3_finalize() return value.
+
*/
+
static void idxFinalize(int *pRc, sqlite3_stmt *pStmt){
+
  int rc = sqlite3_finalize(pStmt);
+
  if( *pRc==SQLITE_OK ) *pRc = rc;
}

-
static int zipfileAppendEntry(
-
  ZipfileTab *pTab,
-
  ZipfileEntry *pEntry,
-
  const u8 *pData,
-
  int nData
+
/*
+
** Attempt to allocate an IdxTable structure corresponding to table zTab
+
** in the main database of connection db. If successful, set (*ppOut) to
+
** point to the new object and return SQLITE_OK. Otherwise, return an
+
** SQLite error code and set (*ppOut) to NULL. In this case *pzErrmsg may be
+
** set to point to an error string.
+
**
+
** It is the responsibility of the caller to eventually free either the
+
** IdxTable object or error message using sqlite3_free().
+
*/
+
static int idxGetTableInfo(
+
  sqlite3 *db,                    /* Database connection to read details from */
+
  const char *zTab,               /* Table name */
+
  IdxTable **ppOut,               /* OUT: New object (if successful) */
+
  char **pzErrmsg                 /* OUT: Error message (if not) */
){
-
  u8 *aBuf = pTab->aBuffer;
-
  int nBuf;
-
  int rc;
+
  sqlite3_stmt *p1 = 0;
+
  int nCol = 0;
+
  int nTab;
+
  int nByte;
+
  IdxTable *pNew = 0;
+
  int rc, rc2;
+
  char *pCsr = 0;
+
  int nPk = 0;

-
  nBuf = zipfileSerializeLFH(pEntry, aBuf);
-
  rc = zipfileAppendData(pTab, aBuf, nBuf);
-
  if( rc==SQLITE_OK ){
-
    pEntry->iDataOff = pTab->szCurrent;
-
    rc = zipfileAppendData(pTab, pData, nData);
+
  *ppOut = 0;
+
  if( zTab==0 ) return SQLITE_ERROR;
+
  nTab = STRLEN(zTab);
+
  nByte = sizeof(IdxTable) + nTab + 1;
+
  rc = idxPrintfPrepareStmt(db, &p1, pzErrmsg, "PRAGMA table_xinfo=%Q", zTab);
+
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){
+
    const char *zCol = (const char*)sqlite3_column_text(p1, 1);
+
    const char *zColSeq = 0;
+
    if( zCol==0 ){
+
      rc = SQLITE_ERROR;
+
      break;
+
    }
+
    nByte += 1 + STRLEN(zCol);
+
    rc = sqlite3_table_column_metadata(
+
        db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0
+
    );
+
    if( zColSeq==0 ) zColSeq = "binary";
+
    nByte += 1 + STRLEN(zColSeq);
+
    nCol++;
+
    nPk += (sqlite3_column_int(p1, 5)>0);
  }
+
  rc2 = sqlite3_reset(p1);
+
  if( rc==SQLITE_OK ) rc = rc2;

-
  return rc;
-
}
-

-
static int zipfileGetMode(
-
  sqlite3_value *pVal, 
-
  int bIsDir,                     /* If true, default to directory */
-
  u32 *pMode,                     /* OUT: Mode value */
-
  char **pzErr                    /* OUT: Error message */
-
){
-
  const char *z = (const char*)sqlite3_value_text(pVal);
-
  u32 mode = 0;
-
  if( z==0 ){
-
    mode = (bIsDir ? (S_IFDIR + 0755) : (S_IFREG + 0644));
-
  }else if( z[0]>='0' && z[0]<='9' ){
-
    mode = (unsigned int)sqlite3_value_int(pVal);
-
  }else{
-
    const char zTemplate[11] = "-rwxrwxrwx";
-
    int i;
-
    if( strlen(z)!=10 ) goto parse_error;
-
    switch( z[0] ){
-
      case '-': mode |= S_IFREG; break;
-
      case 'd': mode |= S_IFDIR; break;
-
      case 'l': mode |= S_IFLNK; break;
-
      default: goto parse_error;
-
    }
-
    for(i=1; i<10; i++){
-
      if( z[i]==zTemplate[i] ) mode |= 1 << (9-i);
-
      else if( z[i]!='-' ) goto parse_error;
-
    }
+
  nByte += sizeof(IdxColumn) * nCol;
+
  if( rc==SQLITE_OK ){
+
    pNew = idxMalloc(&rc, nByte);
  }
-
  if( ((mode & S_IFDIR)==0)==bIsDir ){
-
    /* The "mode" attribute is a directory, but data has been specified.
-
    ** Or vice-versa - no data but "mode" is a file or symlink.  */
-
    *pzErr = sqlite3_mprintf("zipfile: mode does not match data");
-
    return SQLITE_CONSTRAINT;
+
  if( rc==SQLITE_OK ){
+
    pNew->aCol = (IdxColumn*)&pNew[1];
+
    pNew->nCol = nCol;
+
    pCsr = (char*)&pNew->aCol[nCol];
  }
-
  *pMode = mode;
-
  return SQLITE_OK;
-

-
 parse_error:
-
  *pzErr = sqlite3_mprintf("zipfile: parse error in mode: %s", z);
-
  return SQLITE_ERROR;
-
}
-

-
/*
-
** Both (const char*) arguments point to nul-terminated strings. Argument
-
** nB is the value of strlen(zB). This function returns 0 if the strings are
-
** identical, ignoring any trailing '/' character in either path.  */
-
static int zipfileComparePath(const char *zA, const char *zB, int nB){
-
  int nA = (int)strlen(zA);
-
  if( nA>0 && zA[nA-1]=='/' ) nA--;
-
  if( nB>0 && zB[nB-1]=='/' ) nB--;
-
  if( nA==nB && memcmp(zA, zB, nA)==0 ) return 0;
-
  return 1;
-
}

-
static int zipfileBegin(sqlite3_vtab *pVtab){
-
  ZipfileTab *pTab = (ZipfileTab*)pVtab;
-
  int rc = SQLITE_OK;
+
  nCol = 0;
+
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){
+
    const char *zCol = (const char*)sqlite3_column_text(p1, 1);
+
    const char *zColSeq = 0;
+
    int nCopy;
+
    if( zCol==0 ) continue;
+
    nCopy = STRLEN(zCol) + 1;
+
    pNew->aCol[nCol].zName = pCsr;
+
    pNew->aCol[nCol].iPk = (sqlite3_column_int(p1, 5)==1 && nPk==1);
+
    memcpy(pCsr, zCol, nCopy);
+
    pCsr += nCopy;

-
  assert( pTab->pWriteFd==0 );
-
  if( pTab->zFile==0 || pTab->zFile[0]==0 ){
-
    pTab->base.zErrMsg = sqlite3_mprintf("zipfile: missing filename");
-
    return SQLITE_ERROR;
-
  }
+
    rc = sqlite3_table_column_metadata(
+
        db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0
+
    );
+
    if( rc==SQLITE_OK ){
+
      if( zColSeq==0 ) zColSeq = "binary";
+
      nCopy = STRLEN(zColSeq) + 1;
+
      pNew->aCol[nCol].zColl = pCsr;
+
      memcpy(pCsr, zColSeq, nCopy);
+
      pCsr += nCopy;
+
    }

-
  /* Open a write fd on the file. Also load the entire central directory
-
  ** structure into memory. During the transaction any new file data is 
-
  ** appended to the archive file, but the central directory is accumulated
-
  ** in main-memory until the transaction is committed.  */
-
  pTab->pWriteFd = fopen(pTab->zFile, "ab+");
-
  if( pTab->pWriteFd==0 ){
-
    pTab->base.zErrMsg = sqlite3_mprintf(
-
        "zipfile: failed to open file %s for writing", pTab->zFile
-
        );
-
    rc = SQLITE_ERROR;
-
  }else{
-
    fseek(pTab->pWriteFd, 0, SEEK_END);
-
    pTab->szCurrent = pTab->szOrig = (i64)ftell(pTab->pWriteFd);
-
    rc = zipfileLoadDirectory(pTab, 0, 0);
+
    nCol++;
  }
+
  idxFinalize(&rc, p1);

  if( rc!=SQLITE_OK ){
-
    zipfileCleanupTransaction(pTab);
+
    sqlite3_free(pNew);
+
    pNew = 0;
+
  }else if( ALWAYS(pNew!=0) ){
+
    pNew->zName = pCsr;
+
    if( ALWAYS(pNew->zName!=0) ) memcpy(pNew->zName, zTab, nTab+1);
  }

+
  *ppOut = pNew;
  return rc;
}

/*
-
** Return the current time as a 32-bit timestamp in UNIX epoch format (like
-
** time(2)).
+
** This function is a no-op if *pRc is set to anything other than 
+
** SQLITE_OK when it is called.
+
**
+
** If *pRc is initially set to SQLITE_OK, then the text specified by
+
** the printf() style arguments is appended to zIn and the result returned
+
** in a buffer allocated by sqlite3_malloc(). sqlite3_free() is called on
+
** zIn before returning.
*/
-
static u32 zipfileTime(void){
-
  sqlite3_vfs *pVfs = sqlite3_vfs_find(0);
-
  u32 ret;
-
  if( pVfs==0 ) return 0;
-
  if( pVfs->iVersion>=2 && pVfs->xCurrentTimeInt64 ){
-
    i64 ms;
-
    pVfs->xCurrentTimeInt64(pVfs, &ms);
-
    ret = (u32)((ms/1000) - ((i64)24405875 * 8640));
-
  }else{
-
    double day;
-
    pVfs->xCurrentTime(pVfs, &day);
-
    ret = (u32)((day - 2440587.5) * 86400);
+
static char *idxAppendText(int *pRc, char *zIn, const char *zFmt, ...){
+
  va_list ap;
+
  char *zAppend = 0;
+
  char *zRet = 0;
+
  int nIn = zIn ? STRLEN(zIn) : 0;
+
  int nAppend = 0;
+
  va_start(ap, zFmt);
+
  if( *pRc==SQLITE_OK ){
+
    zAppend = sqlite3_vmprintf(zFmt, ap);
+
    if( zAppend ){
+
      nAppend = STRLEN(zAppend);
+
      zRet = (char*)sqlite3_malloc(nIn + nAppend + 1);
+
    }
+
    if( zAppend && zRet ){
+
      if( nIn ) memcpy(zRet, zIn, nIn);
+
      memcpy(&zRet[nIn], zAppend, nAppend+1);
+
    }else{
+
      sqlite3_free(zRet);
+
      zRet = 0;
+
      *pRc = SQLITE_NOMEM;
+
    }
+
    sqlite3_free(zAppend);
+
    sqlite3_free(zIn);
  }
-
  return ret;
+
  va_end(ap);
+
  return zRet;
}

/*
-
** Return a 32-bit timestamp in UNIX epoch format.
-
**
-
** If the value passed as the only argument is either NULL or an SQL NULL,
-
** return the current time. Otherwise, return the value stored in (*pVal)
-
** cast to a 32-bit unsigned integer.
+
** Return true if zId must be quoted in order to use it as an SQL
+
** identifier, or false otherwise.
*/
-
static u32 zipfileGetTime(sqlite3_value *pVal){
-
  if( pVal==0 || sqlite3_value_type(pVal)==SQLITE_NULL ){
-
    return zipfileTime();
+
static int idxIdentifierRequiresQuotes(const char *zId){
+
  int i;
+
  int nId = STRLEN(zId);
+
  
+
  if( sqlite3_keyword_check(zId, nId) ) return 1;
+

+
  for(i=0; zId[i]; i++){
+
    if( !(zId[i]=='_')
+
     && !(zId[i]>='0' && zId[i]<='9')
+
     && !(zId[i]>='a' && zId[i]<='z')
+
     && !(zId[i]>='A' && zId[i]<='Z')
+
    ){
+
      return 1;
+
    }
  }
-
  return (u32)sqlite3_value_int64(pVal);
+
  return 0;
}

/*
-
** Unless it is NULL, entry pOld is currently part of the pTab->pFirstEntry
-
** linked list.  Remove it from the list and free the object.
+
** This function appends an index column definition suitable for constraint
+
** pCons to the string passed as zIn and returns the result.
*/
-
static void zipfileRemoveEntryFromList(ZipfileTab *pTab, ZipfileEntry *pOld){
-
  if( pOld ){
-
    ZipfileEntry **pp;
-
    for(pp=&pTab->pFirstEntry; (*pp)!=pOld; pp=&((*pp)->pNext));
-
    *pp = (*pp)->pNext;
-
    zipfileEntryFree(pOld);
+
static char *idxAppendColDefn(
+
  int *pRc,                       /* IN/OUT: Error code */
+
  char *zIn,                      /* Column defn accumulated so far */
+
  IdxTable *pTab,                 /* Table index will be created on */
+
  IdxConstraint *pCons
+
){
+
  char *zRet = zIn;
+
  IdxColumn *p = &pTab->aCol[pCons->iCol];
+
  if( zRet ) zRet = idxAppendText(pRc, zRet, ", ");
+

+
  if( idxIdentifierRequiresQuotes(p->zName) ){
+
    zRet = idxAppendText(pRc, zRet, "%Q", p->zName);
+
  }else{
+
    zRet = idxAppendText(pRc, zRet, "%s", p->zName);
+
  }
+

+
  if( sqlite3_stricmp(p->zColl, pCons->zColl) ){
+
    if( idxIdentifierRequiresQuotes(pCons->zColl) ){
+
      zRet = idxAppendText(pRc, zRet, " COLLATE %Q", pCons->zColl);
+
    }else{
+
      zRet = idxAppendText(pRc, zRet, " COLLATE %s", pCons->zColl);
+
    }
+
  }
+

+
  if( pCons->bDesc ){
+
    zRet = idxAppendText(pRc, zRet, " DESC");
  }
+
  return zRet;
}

/*
-
** xUpdate method.
+
** Search database dbm for an index compatible with the one idxCreateFromCons()
+
** would create from arguments pScan, pEq and pTail. If no error occurs and 
+
** such an index is found, return non-zero. Or, if no such index is found,
+
** return zero.
+
**
+
** If an error occurs, set *pRc to an SQLite error code and return zero.
*/
-
static int zipfileUpdate(
-
  sqlite3_vtab *pVtab, 
-
  int nVal, 
-
  sqlite3_value **apVal, 
-
  sqlite_int64 *pRowid
+
static int idxFindCompatible(
+
  int *pRc,                       /* OUT: Error code */
+
  sqlite3* dbm,                   /* Database to search */
+
  IdxScan *pScan,                 /* Scan for table to search for index on */
+
  IdxConstraint *pEq,             /* List of == constraints */
+
  IdxConstraint *pTail            /* List of range constraints */
){
-
  ZipfileTab *pTab = (ZipfileTab*)pVtab;
-
  int rc = SQLITE_OK;             /* Return Code */
-
  ZipfileEntry *pNew = 0;         /* New in-memory CDS entry */
+
  const char *zTbl = pScan->pTab->zName;
+
  sqlite3_stmt *pIdxList = 0;
+
  IdxConstraint *pIter;
+
  int nEq = 0;                    /* Number of elements in pEq */
+
  int rc;

-
  u32 mode = 0;                   /* Mode for new entry */
-
  u32 mTime = 0;                  /* Modification time for new entry */
-
  i64 sz = 0;                     /* Uncompressed size */
-
  const char *zPath = 0;          /* Path for new entry */
-
  int nPath = 0;                  /* strlen(zPath) */
-
  const u8 *pData = 0;            /* Pointer to buffer containing content */
-
  int nData = 0;                  /* Size of pData buffer in bytes */
-
  int iMethod = 0;                /* Compression method for new entry */
-
  u8 *pFree = 0;                  /* Free this */
-
  char *zFree = 0;                /* Also free this */
-
  ZipfileEntry *pOld = 0;
-
  ZipfileEntry *pOld2 = 0;
-
  int bUpdate = 0;                /* True for an update that modifies "name" */
-
  int bIsDir = 0;
-
  u32 iCrc32 = 0;
+
  /* Count the elements in list pEq */
+
  for(pIter=pEq; pIter; pIter=pIter->pLink) nEq++;

-
  if( pTab->pWriteFd==0 ){
-
    rc = zipfileBegin(pVtab);
-
    if( rc!=SQLITE_OK ) return rc;
-
  }
+
  rc = idxPrintfPrepareStmt(dbm, &pIdxList, 0, "PRAGMA index_list=%Q", zTbl);
+
  while( rc==SQLITE_OK && sqlite3_step(pIdxList)==SQLITE_ROW ){
+
    int bMatch = 1;
+
    IdxConstraint *pT = pTail;
+
    sqlite3_stmt *pInfo = 0;
+
    const char *zIdx = (const char*)sqlite3_column_text(pIdxList, 1);
+
    if( zIdx==0 ) continue;

-
  /* If this is a DELETE or UPDATE, find the archive entry to delete. */
-
  if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
-
    const char *zDelete = (const char*)sqlite3_value_text(apVal[0]);
-
    int nDelete = (int)strlen(zDelete);
-
    if( nVal>1 ){
-
      const char *zUpdate = (const char*)sqlite3_value_text(apVal[1]);
-
      if( zUpdate && zipfileComparePath(zUpdate, zDelete, nDelete)!=0 ){
-
        bUpdate = 1;
-
      }
-
    }
-
    for(pOld=pTab->pFirstEntry; 1; pOld=pOld->pNext){
-
      if( zipfileComparePath(pOld->cds.zFile, zDelete, nDelete)==0 ){
-
        break;
-
      }
-
      assert( pOld->pNext );
-
    }
-
  }
+
    /* Zero the IdxConstraint.bFlag values in the pEq list */
+
    for(pIter=pEq; pIter; pIter=pIter->pLink) pIter->bFlag = 0;

-
  if( nVal>1 ){
-
    /* Check that "sz" and "rawdata" are both NULL: */
-
    if( sqlite3_value_type(apVal[5])!=SQLITE_NULL ){
-
      zipfileTableErr(pTab, "sz must be NULL");
-
      rc = SQLITE_CONSTRAINT;
-
    }
-
    if( sqlite3_value_type(apVal[6])!=SQLITE_NULL ){
-
      zipfileTableErr(pTab, "rawdata must be NULL"); 
-
      rc = SQLITE_CONSTRAINT;
-
    }
+
    rc = idxPrintfPrepareStmt(dbm, &pInfo, 0, "PRAGMA index_xInfo=%Q", zIdx);
+
    while( rc==SQLITE_OK && sqlite3_step(pInfo)==SQLITE_ROW ){
+
      int iIdx = sqlite3_column_int(pInfo, 0);
+
      int iCol = sqlite3_column_int(pInfo, 1);
+
      const char *zColl = (const char*)sqlite3_column_text(pInfo, 4);

-
    if( rc==SQLITE_OK ){
-
      if( sqlite3_value_type(apVal[7])==SQLITE_NULL ){
-
        /* data=NULL. A directory */
-
        bIsDir = 1;
+
      if( iIdx<nEq ){
+
        for(pIter=pEq; pIter; pIter=pIter->pLink){
+
          if( pIter->bFlag ) continue;
+
          if( pIter->iCol!=iCol ) continue;
+
          if( sqlite3_stricmp(pIter->zColl, zColl) ) continue;
+
          pIter->bFlag = 1;
+
          break;
+
        }
+
        if( pIter==0 ){
+
          bMatch = 0;
+
          break;
+
        }
      }else{
-
        /* Value specified for "data", and possibly "method". This must be
-
        ** a regular file or a symlink. */
-
        const u8 *aIn = sqlite3_value_blob(apVal[7]);
-
        int nIn = sqlite3_value_bytes(apVal[7]);
-
        int bAuto = sqlite3_value_type(apVal[8])==SQLITE_NULL;
-

-
        iMethod = sqlite3_value_int(apVal[8]);
-
        sz = nIn;
-
        pData = aIn;
-
        nData = nIn;
-
        if( iMethod!=0 && iMethod!=8 ){
-
          zipfileTableErr(pTab, "unknown compression method: %d", iMethod);
-
          rc = SQLITE_CONSTRAINT;
-
        }else{
-
          if( bAuto || iMethod ){
-
            int nCmp;
-
            rc = zipfileDeflate(aIn, nIn, &pFree, &nCmp, &pTab->base.zErrMsg);
-
            if( rc==SQLITE_OK ){
-
              if( iMethod || nCmp<nIn ){
-
                iMethod = 8;
-
                pData = pFree;
-
                nData = nCmp;
-
              }
-
            }
+
        if( pT ){
+
          if( pT->iCol!=iCol || sqlite3_stricmp(pT->zColl, zColl) ){
+
            bMatch = 0;
+
            break;
          }
-
          iCrc32 = crc32(0, aIn, nIn);
+
          pT = pT->pLink;
        }
      }
    }
+
    idxFinalize(&rc, pInfo);

-
    if( rc==SQLITE_OK ){
-
      rc = zipfileGetMode(apVal[3], bIsDir, &mode, &pTab->base.zErrMsg);
+
    if( rc==SQLITE_OK && bMatch ){
+
      sqlite3_finalize(pIdxList);
+
      return 1;
    }
+
  }
+
  idxFinalize(&rc, pIdxList);

-
    if( rc==SQLITE_OK ){
-
      zPath = (const char*)sqlite3_value_text(apVal[2]);
-
      if( zPath==0 ) zPath = "";
-
      nPath = (int)strlen(zPath);
-
      mTime = zipfileGetTime(apVal[4]);
-
    }
+
  *pRc = rc;
+
  return 0;
+
}

-
    if( rc==SQLITE_OK && bIsDir ){
-
      /* For a directory, check that the last character in the path is a
-
      ** '/'. This appears to be required for compatibility with info-zip
-
      ** (the unzip command on unix). It does not create directories
-
      ** otherwise.  */
-
      if( nPath<=0 || zPath[nPath-1]!='/' ){
-
        zFree = sqlite3_mprintf("%s/", zPath);
-
        zPath = (const char*)zFree;
-
        if( zFree==0 ){
-
          rc = SQLITE_NOMEM;
-
          nPath = 0;
-
        }else{
-
          nPath = (int)strlen(zPath);
-
        }
-
      }
-
    }
+
/* Callback for sqlite3_exec() with query with leading count(*) column.
+
 * The first argument is expected to be an int*, referent to be incremented
+
 * if that leading column is not exactly '0'.
+
 */
+
static int countNonzeros(void* pCount, int nc,
+
                         char* azResults[], char* azColumns[]){
+
  (void)azColumns;  /* Suppress unused parameter warning */
+
  if( nc>0 && (azResults[0][0]!='0' || azResults[0][1]!=0) ){
+
    *((int *)pCount) += 1;
+
  }
+
  return 0;
+
}

-
    /* Check that we're not inserting a duplicate entry -OR- updating an
-
    ** entry with a path, thereby making it into a duplicate. */
-
    if( (pOld==0 || bUpdate) && rc==SQLITE_OK ){
-
      ZipfileEntry *p;
-
      for(p=pTab->pFirstEntry; p; p=p->pNext){
-
        if( zipfileComparePath(p->cds.zFile, zPath, nPath)==0 ){
-
          switch( sqlite3_vtab_on_conflict(pTab->db) ){
-
            case SQLITE_IGNORE: {
-
              goto zipfile_update_done;
-
            }
-
            case SQLITE_REPLACE: {
-
              pOld2 = p;
-
              break;
-
            }
-
            default: {
-
              zipfileTableErr(pTab, "duplicate name: \"%s\"", zPath);
-
              rc = SQLITE_CONSTRAINT;
-
              break;
-
            }
-
          }
-
          break;
-
        }
-
      }
+
static int idxCreateFromCons(
+
  sqlite3expert *p,
+
  IdxScan *pScan,
+
  IdxConstraint *pEq, 
+
  IdxConstraint *pTail
+
){
+
  sqlite3 *dbm = p->dbm;
+
  int rc = SQLITE_OK;
+
  if( (pEq || pTail) && 0==idxFindCompatible(&rc, dbm, pScan, pEq, pTail) ){
+
    IdxTable *pTab = pScan->pTab;
+
    char *zCols = 0;
+
    char *zIdx = 0;
+
    IdxConstraint *pCons;
+
    unsigned int h = 0;
+
    const char *zFmt;
+

+
    for(pCons=pEq; pCons; pCons=pCons->pLink){
+
      zCols = idxAppendColDefn(&rc, zCols, pTab, pCons);
+
    }
+
    for(pCons=pTail; pCons; pCons=pCons->pLink){
+
      zCols = idxAppendColDefn(&rc, zCols, pTab, pCons);
    }

    if( rc==SQLITE_OK ){
-
      /* Create the new CDS record. */
-
      pNew = zipfileNewEntry(zPath);
-
      if( pNew==0 ){
+
      /* Hash the list of columns to come up with a name for the index */
+
      const char *zTable = pScan->pTab->zName;
+
      int quoteTable = idxIdentifierRequiresQuotes(zTable);
+
      char *zName = 0;          /* Index name */
+
      int collisions = 0;
+
      do{
+
        int i;
+
        char *zFind;
+
        for(i=0; zCols[i]; i++){
+
          h += ((h<<3) + zCols[i]);
+
        }
+
        sqlite3_free(zName);
+
        zName = sqlite3_mprintf("%s_idx_%08x", zTable, h);
+
        if( zName==0 ) break;
+
        /* Is is unique among table, view and index names? */
+
        zFmt = "SELECT count(*) FROM sqlite_schema WHERE name=%Q"
+
          " AND type in ('index','table','view')";
+
        zFind = sqlite3_mprintf(zFmt, zName);
+
        i = 0;
+
        rc = sqlite3_exec(dbm, zFind, countNonzeros, &i, 0);
+
        assert(rc==SQLITE_OK);
+
        sqlite3_free(zFind);
+
        if( i==0 ){
+
          collisions = 0;
+
          break;
+
        }
+
        ++collisions;
+
      }while( collisions<50 && zName!=0 );
+
      if( collisions ){
+
        /* This return means "Gave up trying to find a unique index name." */
+
        rc = SQLITE_BUSY_TIMEOUT;
+
      }else if( zName==0 ){
        rc = SQLITE_NOMEM;
      }else{
-
        pNew->cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
-
        pNew->cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
-
        pNew->cds.flags = ZIPFILE_NEWENTRY_FLAGS;
-
        pNew->cds.iCompression = (u16)iMethod;
-
        zipfileMtimeToDos(&pNew->cds, mTime);
-
        pNew->cds.crc32 = iCrc32;
-
        pNew->cds.szCompressed = nData;
-
        pNew->cds.szUncompressed = (u32)sz;
-
        pNew->cds.iExternalAttr = (mode<<16);
-
        pNew->cds.iOffset = (u32)pTab->szCurrent;
-
        pNew->cds.nFile = (u16)nPath;
-
        pNew->mUnixTime = (u32)mTime;
-
        rc = zipfileAppendEntry(pTab, pNew, pData, nData);
-
        zipfileAddEntry(pTab, pOld, pNew);
-
      }
-
    }
-
  }
-

-
  if( rc==SQLITE_OK && (pOld || pOld2) ){
-
    ZipfileCsr *pCsr;
-
    for(pCsr=pTab->pCsrList; pCsr; pCsr=pCsr->pCsrNext){
-
      if( pCsr->pCurrent && (pCsr->pCurrent==pOld || pCsr->pCurrent==pOld2) ){
-
        pCsr->pCurrent = pCsr->pCurrent->pNext;
-
        pCsr->bNoop = 1;
+
        if( quoteTable ){
+
          zFmt = "CREATE INDEX \"%w\" ON \"%w\"(%s)";
+
        }else{
+
          zFmt = "CREATE INDEX %s ON %s(%s)";
+
        }
+
        zIdx = sqlite3_mprintf(zFmt, zName, zTable, zCols);
+
        if( !zIdx ){
+
          rc = SQLITE_NOMEM;
+
        }else{
+
          rc = sqlite3_exec(dbm, zIdx, 0, 0, p->pzErrmsg);
+
          if( rc!=SQLITE_OK ){
+
            rc = SQLITE_BUSY_TIMEOUT;
+
          }else{
+
            idxHashAdd(&rc, &p->hIdx, zName, zIdx);
+
          }
+
        }
+
        sqlite3_free(zName);
+
        sqlite3_free(zIdx);
      }
    }

-
    zipfileRemoveEntryFromList(pTab, pOld);
-
    zipfileRemoveEntryFromList(pTab, pOld2);
+
    sqlite3_free(zCols);
  }
-

-
zipfile_update_done:
-
  sqlite3_free(pFree);
-
  sqlite3_free(zFree);
  return rc;
}

-
static int zipfileSerializeEOCD(ZipfileEOCD *p, u8 *aBuf){
-
  u8 *a = aBuf;
-
  zipfileWrite32(a, ZIPFILE_SIGNATURE_EOCD);
-
  zipfileWrite16(a, p->iDisk);
-
  zipfileWrite16(a, p->iFirstDisk);
-
  zipfileWrite16(a, p->nEntry);
-
  zipfileWrite16(a, p->nEntryTotal);
-
  zipfileWrite32(a, p->nSize);
-
  zipfileWrite32(a, p->iOffset);
-
  zipfileWrite16(a, 0);        /* Size of trailing comment in bytes*/
-

-
  return a-aBuf;
-
}
-

-
static int zipfileAppendEOCD(ZipfileTab *pTab, ZipfileEOCD *p){
-
  int nBuf = zipfileSerializeEOCD(p, pTab->aBuffer);
-
  assert( nBuf==ZIPFILE_EOCD_FIXED_SZ );
-
  return zipfileAppendData(pTab, pTab->aBuffer, nBuf);
-
}
-

/*
-
** Serialize the CDS structure into buffer aBuf[]. Return the number
-
** of bytes written.
+
** Return true if list pList (linked by IdxConstraint.pLink) contains
+
** a constraint compatible with *p. Otherwise return false.
*/
-
static int zipfileSerializeCDS(ZipfileEntry *pEntry, u8 *aBuf){
-
  u8 *a = aBuf;
-
  ZipfileCDS *pCDS = &pEntry->cds;
-

-
  if( pEntry->aExtra==0 ){
-
    pCDS->nExtra = 9;
+
static int idxFindConstraint(IdxConstraint *pList, IdxConstraint *p){
+
  IdxConstraint *pCmp;
+
  for(pCmp=pList; pCmp; pCmp=pCmp->pLink){
+
    if( p->iCol==pCmp->iCol ) return 1;
  }
+
  return 0;
+
}

-
  zipfileWrite32(a, ZIPFILE_SIGNATURE_CDS);
-
  zipfileWrite16(a, pCDS->iVersionMadeBy);
-
  zipfileWrite16(a, pCDS->iVersionExtract);
-
  zipfileWrite16(a, pCDS->flags);
-
  zipfileWrite16(a, pCDS->iCompression);
-
  zipfileWrite16(a, pCDS->mTime);
-
  zipfileWrite16(a, pCDS->mDate);
-
  zipfileWrite32(a, pCDS->crc32);
-
  zipfileWrite32(a, pCDS->szCompressed);
-
  zipfileWrite32(a, pCDS->szUncompressed);
-
  assert( a==&aBuf[ZIPFILE_CDS_NFILE_OFF] );
-
  zipfileWrite16(a, pCDS->nFile);
-
  zipfileWrite16(a, pCDS->nExtra);
-
  zipfileWrite16(a, pCDS->nComment);
-
  zipfileWrite16(a, pCDS->iDiskStart);
-
  zipfileWrite16(a, pCDS->iInternalAttr);
-
  zipfileWrite32(a, pCDS->iExternalAttr);
-
  zipfileWrite32(a, pCDS->iOffset);
+
static int idxCreateFromWhere(
+
  sqlite3expert *p, 
+
  IdxScan *pScan,                 /* Create indexes for this scan */
+
  IdxConstraint *pTail            /* range/ORDER BY constraints for inclusion */
+
){
+
  IdxConstraint *p1 = 0;
+
  IdxConstraint *pCon;
+
  int rc;

-
  memcpy(a, pCDS->zFile, pCDS->nFile);
-
  a += pCDS->nFile;
+
  /* Gather up all the == constraints. */
+
  for(pCon=pScan->pEq; pCon; pCon=pCon->pNext){
+
    if( !idxFindConstraint(p1, pCon) && !idxFindConstraint(pTail, pCon) ){
+
      pCon->pLink = p1;
+
      p1 = pCon;
+
    }
+
  }

-
  if( pEntry->aExtra ){
-
    int n = (int)pCDS->nExtra + (int)pCDS->nComment;
-
    memcpy(a, pEntry->aExtra, n);
-
    a += n;
-
  }else{
-
    assert( pCDS->nExtra==9 );
-
    zipfileWrite16(a, ZIPFILE_EXTRA_TIMESTAMP);
-
    zipfileWrite16(a, 5);
-
    *a++ = 0x01;
-
    zipfileWrite32(a, pEntry->mUnixTime);
+
  /* Create an index using the == constraints collected above. And the
+
  ** range constraint/ORDER BY terms passed in by the caller, if any. */
+
  rc = idxCreateFromCons(p, pScan, p1, pTail);
+

+
  /* If no range/ORDER BY passed by the caller, create a version of the
+
  ** index for each range constraint.  */
+
  if( pTail==0 ){
+
    for(pCon=pScan->pRange; rc==SQLITE_OK && pCon; pCon=pCon->pNext){
+
      assert( pCon->pLink==0 );
+
      if( !idxFindConstraint(p1, pCon) && !idxFindConstraint(pTail, pCon) ){
+
        rc = idxCreateFromCons(p, pScan, p1, pCon);
+
      }
+
    }
  }

-
  return a-aBuf;
+
  return rc;
}

-
static int zipfileCommit(sqlite3_vtab *pVtab){
-
  ZipfileTab *pTab = (ZipfileTab*)pVtab;
+
/*
+
** Create candidate indexes in database [dbm] based on the data in 
+
** linked-list pScan.
+
*/
+
static int idxCreateCandidates(sqlite3expert *p){
  int rc = SQLITE_OK;
-
  if( pTab->pWriteFd ){
-
    i64 iOffset = pTab->szCurrent;
-
    ZipfileEntry *p;
-
    ZipfileEOCD eocd;
-
    int nEntry = 0;
+
  IdxScan *pIter;

-
    /* Write out all entries */
-
    for(p=pTab->pFirstEntry; rc==SQLITE_OK && p; p=p->pNext){
-
      int n = zipfileSerializeCDS(p, pTab->aBuffer);
-
      rc = zipfileAppendData(pTab, pTab->aBuffer, n);
-
      nEntry++;
+
  for(pIter=p->pScan; pIter && rc==SQLITE_OK; pIter=pIter->pNextScan){
+
    rc = idxCreateFromWhere(p, pIter, 0);
+
    if( rc==SQLITE_OK && pIter->pOrder ){
+
      rc = idxCreateFromWhere(p, pIter, pIter->pOrder);
    }
-

-
    /* Write out the EOCD record */
-
    eocd.iDisk = 0;
-
    eocd.iFirstDisk = 0;
-
    eocd.nEntry = (u16)nEntry;
-
    eocd.nEntryTotal = (u16)nEntry;
-
    eocd.nSize = (u32)(pTab->szCurrent - iOffset);
-
    eocd.iOffset = (u32)iOffset;
-
    rc = zipfileAppendEOCD(pTab, &eocd);
-

-
    zipfileCleanupTransaction(pTab);
  }
+

  return rc;
}

-
static int zipfileRollback(sqlite3_vtab *pVtab){
-
  return zipfileCommit(pVtab);
-
}
+
/*
+
** Free all elements of the linked list starting at pConstraint.
+
*/
+
static void idxConstraintFree(IdxConstraint *pConstraint){
+
  IdxConstraint *pNext;
+
  IdxConstraint *p;

-
static ZipfileCsr *zipfileFindCursor(ZipfileTab *pTab, i64 iId){
-
  ZipfileCsr *pCsr;
-
  for(pCsr=pTab->pCsrList; pCsr; pCsr=pCsr->pCsrNext){
-
    if( iId==pCsr->iId ) break;
+
  for(p=pConstraint; p; p=pNext){
+
    pNext = p->pNext;
+
    sqlite3_free(p);
  }
-
  return pCsr;
}

-
static void zipfileFunctionCds(
-
  sqlite3_context *context,
-
  int argc,
-
  sqlite3_value **argv
-
){
-
  ZipfileCsr *pCsr;
-
  ZipfileTab *pTab = (ZipfileTab*)sqlite3_user_data(context);
-
  assert( argc>0 );
-

-
  pCsr = zipfileFindCursor(pTab, sqlite3_value_int64(argv[0]));
-
  if( pCsr ){
-
    ZipfileCDS *p = &pCsr->pCurrent->cds;
-
    char *zRes = sqlite3_mprintf("{"
-
        "\"version-made-by\" : %u, "
-
        "\"version-to-extract\" : %u, "
-
        "\"flags\" : %u, "
-
        "\"compression\" : %u, "
-
        "\"time\" : %u, "
-
        "\"date\" : %u, "
-
        "\"crc32\" : %u, "
-
        "\"compressed-size\" : %u, "
-
        "\"uncompressed-size\" : %u, "
-
        "\"file-name-length\" : %u, "
-
        "\"extra-field-length\" : %u, "
-
        "\"file-comment-length\" : %u, "
-
        "\"disk-number-start\" : %u, "
-
        "\"internal-attr\" : %u, "
-
        "\"external-attr\" : %u, "
-
        "\"offset\" : %u }",
-
        (u32)p->iVersionMadeBy, (u32)p->iVersionExtract,
-
        (u32)p->flags, (u32)p->iCompression,
-
        (u32)p->mTime, (u32)p->mDate,
-
        (u32)p->crc32, (u32)p->szCompressed,
-
        (u32)p->szUncompressed, (u32)p->nFile,
-
        (u32)p->nExtra, (u32)p->nComment,
-
        (u32)p->iDiskStart, (u32)p->iInternalAttr,
-
        (u32)p->iExternalAttr, (u32)p->iOffset
-
    );
-

-
    if( zRes==0 ){
-
      sqlite3_result_error_nomem(context);
-
    }else{
-
      sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT);
-
      sqlite3_free(zRes);
-
    }
+
/*
+
** Free all elements of the linked list starting from pScan up until pLast
+
** (pLast is not freed).
+
*/
+
static void idxScanFree(IdxScan *pScan, IdxScan *pLast){
+
  IdxScan *p;
+
  IdxScan *pNext;
+
  for(p=pScan; p!=pLast; p=pNext){
+
    pNext = p->pNextScan;
+
    idxConstraintFree(p->pOrder);
+
    idxConstraintFree(p->pEq);
+
    idxConstraintFree(p->pRange);
+
    sqlite3_free(p);
  }
}

/*
-
** xFindFunction method.
+
** Free all elements of the linked list starting from pStatement up 
+
** until pLast (pLast is not freed).
*/
-
static int zipfileFindFunction(
-
  sqlite3_vtab *pVtab,            /* Virtual table handle */
-
  int nArg,                       /* Number of SQL function arguments */
-
  const char *zName,              /* Name of SQL function */
-
  void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */
-
  void **ppArg                    /* OUT: User data for *pxFunc */
-
){
-
  if( sqlite3_stricmp("zipfile_cds", zName)==0 ){
-
    *pxFunc = zipfileFunctionCds;
-
    *ppArg = (void*)pVtab;
-
    return 1;
+
static void idxStatementFree(IdxStatement *pStatement, IdxStatement *pLast){
+
  IdxStatement *p;
+
  IdxStatement *pNext;
+
  for(p=pStatement; p!=pLast; p=pNext){
+
    pNext = p->pNext;
+
    sqlite3_free(p->zEQP);
+
    sqlite3_free(p->zIdx);
+
    sqlite3_free(p);
  }
-
  return 0;
}

-
typedef struct ZipfileBuffer ZipfileBuffer;
-
struct ZipfileBuffer {
-
  u8 *a;                          /* Pointer to buffer */
-
  int n;                          /* Size of buffer in bytes */
-
  int nAlloc;                     /* Byte allocated at a[] */
-
};
-

-
typedef struct ZipfileCtx ZipfileCtx;
-
struct ZipfileCtx {
-
  int nEntry;
-
  ZipfileBuffer body;
-
  ZipfileBuffer cds;
-
};
-

-
static int zipfileBufferGrow(ZipfileBuffer *pBuf, int nByte){
-
  if( pBuf->n+nByte>pBuf->nAlloc ){
-
    u8 *aNew;
-
    sqlite3_int64 nNew = pBuf->n ? pBuf->n*2 : 512;
-
    int nReq = pBuf->n + nByte;
-

-
    while( nNew<nReq ) nNew = nNew*2;
-
    aNew = sqlite3_realloc64(pBuf->a, nNew);
-
    if( aNew==0 ) return SQLITE_NOMEM;
-
    pBuf->a = aNew;
-
    pBuf->nAlloc = (int)nNew;
+
/*
+
** Free the linked list of IdxTable objects starting at pTab.
+
*/
+
static void idxTableFree(IdxTable *pTab){
+
  IdxTable *pIter;
+
  IdxTable *pNext;
+
  for(pIter=pTab; pIter; pIter=pNext){
+
    pNext = pIter->pNext;
+
    sqlite3_free(pIter);
  }
-
  return SQLITE_OK;
}

/*
-
** xStep() callback for the zipfile() aggregate. This can be called in
-
** any of the following ways:
-
**
-
**   SELECT zipfile(name,data) ...
-
**   SELECT zipfile(name,mode,mtime,data) ...
-
**   SELECT zipfile(name,mode,mtime,data,method) ...
+
** Free the linked list of IdxWrite objects starting at pTab.
*/
-
static void zipfileStep(sqlite3_context *pCtx, int nVal, sqlite3_value **apVal){
-
  ZipfileCtx *p;                  /* Aggregate function context */
-
  ZipfileEntry e;                 /* New entry to add to zip archive */
+
static void idxWriteFree(IdxWrite *pTab){
+
  IdxWrite *pIter;
+
  IdxWrite *pNext;
+
  for(pIter=pTab; pIter; pIter=pNext){
+
    pNext = pIter->pNext;
+
    sqlite3_free(pIter);
+
  }
+
}

-
  sqlite3_value *pName = 0;
-
  sqlite3_value *pMode = 0;
-
  sqlite3_value *pMtime = 0;
-
  sqlite3_value *pData = 0;
-
  sqlite3_value *pMethod = 0;

-
  int bIsDir = 0;
-
  u32 mode;
+

+
/*
+
** This function is called after candidate indexes have been created. It
+
** runs all the queries to see which indexes they prefer, and populates
+
** IdxStatement.zIdx and IdxStatement.zEQP with the results.
+
*/
+
static int idxFindIndexes(
+
  sqlite3expert *p,
+
  char **pzErr                         /* OUT: Error message (sqlite3_malloc) */
+
){
+
  IdxStatement *pStmt;
+
  sqlite3 *dbm = p->dbm;
  int rc = SQLITE_OK;
-
  char *zErr = 0;

-
  int iMethod = -1;               /* Compression method to use (0 or 8) */
+
  IdxHash hIdx;
+
  idxHashInit(&hIdx);

-
  const u8 *aData = 0;            /* Possibly compressed data for new entry */
-
  int nData = 0;                  /* Size of aData[] in bytes */
-
  int szUncompressed = 0;         /* Size of data before compression */
-
  u8 *aFree = 0;                  /* Free this before returning */
-
  u32 iCrc32 = 0;                 /* crc32 of uncompressed data */
+
  for(pStmt=p->pStatement; rc==SQLITE_OK && pStmt; pStmt=pStmt->pNext){
+
    IdxHashEntry *pEntry;
+
    sqlite3_stmt *pExplain = 0;
+
    idxHashClear(&hIdx);
+
    rc = idxPrintfPrepareStmt(dbm, &pExplain, pzErr,
+
        "EXPLAIN QUERY PLAN %s", pStmt->zSql
+
    );
+
    while( rc==SQLITE_OK && sqlite3_step(pExplain)==SQLITE_ROW ){
+
      /* int iId = sqlite3_column_int(pExplain, 0); */
+
      /* int iParent = sqlite3_column_int(pExplain, 1); */
+
      /* int iNotUsed = sqlite3_column_int(pExplain, 2); */
+
      const char *zDetail = (const char*)sqlite3_column_text(pExplain, 3);
+
      int nDetail;
+
      int i;

-
  char *zName = 0;                /* Path (name) of new entry */
-
  int nName = 0;                  /* Size of zName in bytes */
-
  char *zFree = 0;                /* Free this before returning */
-
  int nByte;
+
      if( !zDetail ) continue;
+
      nDetail = STRLEN(zDetail);

-
  memset(&e, 0, sizeof(e));
-
  p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx));
-
  if( p==0 ) return;
+
      for(i=0; i<nDetail; i++){
+
        const char *zIdx = 0;
+
        if( i+13<nDetail && memcmp(&zDetail[i], " USING INDEX ", 13)==0 ){
+
          zIdx = &zDetail[i+13];
+
        }else if( i+22<nDetail 
+
            && memcmp(&zDetail[i], " USING COVERING INDEX ", 22)==0 
+
        ){
+
          zIdx = &zDetail[i+22];
+
        }
+
        if( zIdx ){
+
          const char *zSql;
+
          int nIdx = 0;
+
          while( zIdx[nIdx]!='\0' && (zIdx[nIdx]!=' ' || zIdx[nIdx+1]!='(') ){
+
            nIdx++;
+
          }
+
          zSql = idxHashSearch(&p->hIdx, zIdx, nIdx);
+
          if( zSql ){
+
            idxHashAdd(&rc, &hIdx, zSql, 0);
+
            if( rc ) goto find_indexes_out;
+
          }
+
          break;
+
        }
+
      }

-
  /* Martial the arguments into stack variables */
-
  if( nVal!=2 && nVal!=4 && nVal!=5 ){
-
    zErr = sqlite3_mprintf("wrong number of arguments to function zipfile()");
-
    rc = SQLITE_ERROR;
-
    goto zipfile_step_out;
-
  }
-
  pName = apVal[0];
-
  if( nVal==2 ){
-
    pData = apVal[1];
-
  }else{
-
    pMode = apVal[1];
-
    pMtime = apVal[2];
-
    pData = apVal[3];
-
    if( nVal==5 ){
-
      pMethod = apVal[4];
+
      if( zDetail[0]!='-' ){
+
        pStmt->zEQP = idxAppendText(&rc, pStmt->zEQP, "%s\n", zDetail);
+
      }
    }
-
  }

-
  /* Check that the 'name' parameter looks ok. */
-
  zName = (char*)sqlite3_value_text(pName);
-
  nName = sqlite3_value_bytes(pName);
-
  if( zName==0 ){
-
    zErr = sqlite3_mprintf("first argument to zipfile() must be non-NULL");
-
    rc = SQLITE_ERROR;
-
    goto zipfile_step_out;
+
    for(pEntry=hIdx.pFirst; pEntry; pEntry=pEntry->pNext){
+
      pStmt->zIdx = idxAppendText(&rc, pStmt->zIdx, "%s;\n", pEntry->zKey);
+
    }
+

+
    idxFinalize(&rc, pExplain);
  }

-
  /* Inspect the 'method' parameter. This must be either 0 (store), 8 (use
-
  ** deflate compression) or NULL (choose automatically).  */
-
  if( pMethod && SQLITE_NULL!=sqlite3_value_type(pMethod) ){
-
    iMethod = (int)sqlite3_value_int64(pMethod);
-
    if( iMethod!=0 && iMethod!=8 ){
-
      zErr = sqlite3_mprintf("illegal method value: %d", iMethod);
-
      rc = SQLITE_ERROR;
-
      goto zipfile_step_out;
+
 find_indexes_out:
+
  idxHashClear(&hIdx);
+
  return rc;
+
}
+

+
static int idxAuthCallback(
+
  void *pCtx,
+
  int eOp,
+
  const char *z3,
+
  const char *z4,
+
  const char *zDb,
+
  const char *zTrigger
+
){
+
  int rc = SQLITE_OK;
+
  (void)z4;
+
  (void)zTrigger;
+
  if( eOp==SQLITE_INSERT || eOp==SQLITE_UPDATE || eOp==SQLITE_DELETE ){
+
    if( sqlite3_stricmp(zDb, "main")==0 ){
+
      sqlite3expert *p = (sqlite3expert*)pCtx;
+
      IdxTable *pTab;
+
      for(pTab=p->pTable; pTab; pTab=pTab->pNext){
+
        if( 0==sqlite3_stricmp(z3, pTab->zName) ) break;
+
      }
+
      if( pTab ){
+
        IdxWrite *pWrite;
+
        for(pWrite=p->pWrite; pWrite; pWrite=pWrite->pNext){
+
          if( pWrite->pTab==pTab && pWrite->eOp==eOp ) break;
+
        }
+
        if( pWrite==0 ){
+
          pWrite = idxMalloc(&rc, sizeof(IdxWrite));
+
          if( rc==SQLITE_OK ){
+
            pWrite->pTab = pTab;
+
            pWrite->eOp = eOp;
+
            pWrite->pNext = p->pWrite;
+
            p->pWrite = pWrite;
+
          }
+
        }
+
      }
    }
  }
+
  return rc;
+
}
+

+
static int idxProcessOneTrigger(
+
  sqlite3expert *p, 
+
  IdxWrite *pWrite, 
+
  char **pzErr
+
){
+
  static const char *zInt = UNIQUE_TABLE_NAME;
+
  static const char *zDrop = "DROP TABLE " UNIQUE_TABLE_NAME;
+
  IdxTable *pTab = pWrite->pTab;
+
  const char *zTab = pTab->zName;
+
  const char *zSql = 
+
    "SELECT 'CREATE TEMP' || substr(sql, 7) FROM sqlite_schema "
+
    "WHERE tbl_name = %Q AND type IN ('table', 'trigger') "
+
    "ORDER BY type;";
+
  sqlite3_stmt *pSelect = 0;
+
  int rc = SQLITE_OK;
+
  char *zWrite = 0;
+

+
  /* Create the table and its triggers in the temp schema */
+
  rc = idxPrintfPrepareStmt(p->db, &pSelect, pzErr, zSql, zTab, zTab);
+
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSelect) ){
+
    const char *zCreate = (const char*)sqlite3_column_text(pSelect, 0);
+
    if( zCreate==0 ) continue;
+
    rc = sqlite3_exec(p->dbv, zCreate, 0, 0, pzErr);
+
  }
+
  idxFinalize(&rc, pSelect);

-
  /* Now inspect the data. If this is NULL, then the new entry must be a
-
  ** directory.  Otherwise, figure out whether or not the data should
-
  ** be deflated or simply stored in the zip archive. */
-
  if( sqlite3_value_type(pData)==SQLITE_NULL ){
-
    bIsDir = 1;
-
    iMethod = 0;
-
  }else{
-
    aData = sqlite3_value_blob(pData);
-
    szUncompressed = nData = sqlite3_value_bytes(pData);
-
    iCrc32 = crc32(0, aData, nData);
-
    if( iMethod<0 || iMethod==8 ){
-
      int nOut = 0;
-
      rc = zipfileDeflate(aData, nData, &aFree, &nOut, &zErr);
-
      if( rc!=SQLITE_OK ){
-
        goto zipfile_step_out;
-
      }
-
      if( iMethod==8 || nOut<nData ){
-
        aData = aFree;
-
        nData = nOut;
-
        iMethod = 8;
-
      }else{
-
        iMethod = 0;
-
      }
+
  /* Rename the table in the temp schema to zInt */
+
  if( rc==SQLITE_OK ){
+
    char *z = sqlite3_mprintf("ALTER TABLE temp.%Q RENAME TO %Q", zTab, zInt);
+
    if( z==0 ){
+
      rc = SQLITE_NOMEM;
+
    }else{
+
      rc = sqlite3_exec(p->dbv, z, 0, 0, pzErr);
+
      sqlite3_free(z);
    }
  }

-
  /* Decode the "mode" argument. */
-
  rc = zipfileGetMode(pMode, bIsDir, &mode, &zErr);
-
  if( rc ) goto zipfile_step_out;
-

-
  /* Decode the "mtime" argument. */
-
  e.mUnixTime = zipfileGetTime(pMtime);
-

-
  /* If this is a directory entry, ensure that there is exactly one '/'
-
  ** at the end of the path. Or, if this is not a directory and the path
-
  ** ends in '/' it is an error. */
-
  if( bIsDir==0 ){
-
    if( nName>0 && zName[nName-1]=='/' ){
-
      zErr = sqlite3_mprintf("non-directory name must not end with /");
-
      rc = SQLITE_ERROR;
-
      goto zipfile_step_out;
+
  switch( pWrite->eOp ){
+
    case SQLITE_INSERT: {
+
      int i;
+
      zWrite = idxAppendText(&rc, zWrite, "INSERT INTO %Q VALUES(", zInt);
+
      for(i=0; i<pTab->nCol; i++){
+
        zWrite = idxAppendText(&rc, zWrite, "%s?", i==0 ? "" : ", ");
+
      }
+
      zWrite = idxAppendText(&rc, zWrite, ")");
+
      break;
    }
-
  }else{
-
    if( nName==0 || zName[nName-1]!='/' ){
-
      zName = zFree = sqlite3_mprintf("%s/", zName);
-
      if( zName==0 ){
-
        rc = SQLITE_NOMEM;
-
        goto zipfile_step_out;
+
    case SQLITE_UPDATE: {
+
      int i;
+
      zWrite = idxAppendText(&rc, zWrite, "UPDATE %Q SET ", zInt);
+
      for(i=0; i<pTab->nCol; i++){
+
        zWrite = idxAppendText(&rc, zWrite, "%s%Q=?", i==0 ? "" : ", ", 
+
            pTab->aCol[i].zName
+
        );
+
      }
+
      break;
+
    }
+
    default: {
+
      assert( pWrite->eOp==SQLITE_DELETE );
+
      if( rc==SQLITE_OK ){
+
        zWrite = sqlite3_mprintf("DELETE FROM %Q", zInt);
+
        if( zWrite==0 ) rc = SQLITE_NOMEM;
      }
-
      nName = (int)strlen(zName);
-
    }else{
-
      while( nName>1 && zName[nName-2]=='/' ) nName--;
    }
  }

-
  /* Assemble the ZipfileEntry object for the new zip archive entry */
-
  e.cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
-
  e.cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
-
  e.cds.flags = ZIPFILE_NEWENTRY_FLAGS;
-
  e.cds.iCompression = (u16)iMethod;
-
  zipfileMtimeToDos(&e.cds, (u32)e.mUnixTime);
-
  e.cds.crc32 = iCrc32;
-
  e.cds.szCompressed = nData;
-
  e.cds.szUncompressed = szUncompressed;
-
  e.cds.iExternalAttr = (mode<<16);
-
  e.cds.iOffset = p->body.n;
-
  e.cds.nFile = (u16)nName;
-
  e.cds.zFile = zName;
-

-
  /* Append the LFH to the body of the new archive */
-
  nByte = ZIPFILE_LFH_FIXED_SZ + e.cds.nFile + 9;
-
  if( (rc = zipfileBufferGrow(&p->body, nByte)) ) goto zipfile_step_out;
-
  p->body.n += zipfileSerializeLFH(&e, &p->body.a[p->body.n]);
+
  if( rc==SQLITE_OK ){
+
    sqlite3_stmt *pX = 0;
+
    rc = sqlite3_prepare_v2(p->dbv, zWrite, -1, &pX, 0);
+
    idxFinalize(&rc, pX);
+
    if( rc!=SQLITE_OK ){
+
      idxDatabaseError(p->dbv, pzErr);
+
    }
+
  }
+
  sqlite3_free(zWrite);

-
  /* Append the data to the body of the new archive */
-
  if( nData>0 ){
-
    if( (rc = zipfileBufferGrow(&p->body, nData)) ) goto zipfile_step_out;
-
    memcpy(&p->body.a[p->body.n], aData, nData);
-
    p->body.n += nData;
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_exec(p->dbv, zDrop, 0, 0, pzErr);
  }

-
  /* Append the CDS record to the directory of the new archive */
-
  nByte = ZIPFILE_CDS_FIXED_SZ + e.cds.nFile + 9;
-
  if( (rc = zipfileBufferGrow(&p->cds, nByte)) ) goto zipfile_step_out;
-
  p->cds.n += zipfileSerializeCDS(&e, &p->cds.a[p->cds.n]);
+
  return rc;
+
}

-
  /* Increment the count of entries in the archive */
-
  p->nEntry++;
+
static int idxProcessTriggers(sqlite3expert *p, char **pzErr){
+
  int rc = SQLITE_OK;
+
  IdxWrite *pEnd = 0;
+
  IdxWrite *pFirst = p->pWrite;

-
 zipfile_step_out:
-
  sqlite3_free(aFree);
-
  sqlite3_free(zFree);
-
  if( rc ){
-
    if( zErr ){
-
      sqlite3_result_error(pCtx, zErr, -1);
-
    }else{
-
      sqlite3_result_error_code(pCtx, rc);
+
  while( rc==SQLITE_OK && pFirst!=pEnd ){
+
    IdxWrite *pIter;
+
    for(pIter=pFirst; rc==SQLITE_OK && pIter!=pEnd; pIter=pIter->pNext){
+
      rc = idxProcessOneTrigger(p, pIter, pzErr);
    }
+
    pEnd = pFirst;
+
    pFirst = p->pWrite;
  }
-
  sqlite3_free(zErr);
-
}

-
/*
-
** xFinalize() callback for zipfile aggregate function.
-
*/
-
static void zipfileFinal(sqlite3_context *pCtx){
-
  ZipfileCtx *p;
-
  ZipfileEOCD eocd;
-
  sqlite3_int64 nZip;
-
  u8 *aZip;
+
  return rc;
+
}

-
  p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx));
-
  if( p==0 ) return;
-
  if( p->nEntry>0 ){
-
    memset(&eocd, 0, sizeof(eocd));
-
    eocd.nEntry = (u16)p->nEntry;
-
    eocd.nEntryTotal = (u16)p->nEntry;
-
    eocd.nSize = p->cds.n;
-
    eocd.iOffset = p->body.n;

-
    nZip = p->body.n + p->cds.n + ZIPFILE_EOCD_FIXED_SZ;
-
    aZip = (u8*)sqlite3_malloc64(nZip);
-
    if( aZip==0 ){
-
      sqlite3_result_error_nomem(pCtx);
-
    }else{
-
      memcpy(aZip, p->body.a, p->body.n);
-
      memcpy(&aZip[p->body.n], p->cds.a, p->cds.n);
-
      zipfileSerializeEOCD(&eocd, &aZip[p->body.n + p->cds.n]);
-
      sqlite3_result_blob(pCtx, aZip, (int)nZip, zipfileFree);
-
    }
-
  }
+
static int idxCreateVtabSchema(sqlite3expert *p, char **pzErrmsg){
+
  int rc = idxRegisterVtab(p);
+
  sqlite3_stmt *pSchema = 0;

-
  sqlite3_free(p->body.a);
-
  sqlite3_free(p->cds.a);
-
}
+
  /* For each table in the main db schema:
+
  **
+
  **   1) Add an entry to the p->pTable list, and
+
  **   2) Create the equivalent virtual table in dbv.
+
  */
+
  rc = idxPrepareStmt(p->db, &pSchema, pzErrmsg,
+
      "SELECT type, name, sql, 1 FROM sqlite_schema "
+
      "WHERE type IN ('table','view') AND name NOT LIKE 'sqlite_%%' "
+
      " UNION ALL "
+
      "SELECT type, name, sql, 2 FROM sqlite_schema "
+
      "WHERE type = 'trigger'"
+
      "  AND tbl_name IN(SELECT name FROM sqlite_schema WHERE type = 'view') "
+
      "ORDER BY 4, 1"
+
  );
+
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSchema) ){
+
    const char *zType = (const char*)sqlite3_column_text(pSchema, 0);
+
    const char *zName = (const char*)sqlite3_column_text(pSchema, 1);
+
    const char *zSql = (const char*)sqlite3_column_text(pSchema, 2);

+
    if( zType==0 || zName==0 ) continue;
+
    if( zType[0]=='v' || zType[1]=='r' ){
+
      if( zSql ) rc = sqlite3_exec(p->dbv, zSql, 0, 0, pzErrmsg);
+
    }else{
+
      IdxTable *pTab;
+
      rc = idxGetTableInfo(p->db, zName, &pTab, pzErrmsg);
+
      if( rc==SQLITE_OK ){
+
        int i;
+
        char *zInner = 0;
+
        char *zOuter = 0;
+
        pTab->pNext = p->pTable;
+
        p->pTable = pTab;

-
/*
-
** Register the "zipfile" virtual table.
-
*/
-
static int zipfileRegister(sqlite3 *db){
-
  static sqlite3_module zipfileModule = {
-
    1,                         /* iVersion */
-
    zipfileConnect,            /* xCreate */
-
    zipfileConnect,            /* xConnect */
-
    zipfileBestIndex,          /* xBestIndex */
-
    zipfileDisconnect,         /* xDisconnect */
-
    zipfileDisconnect,         /* xDestroy */
-
    zipfileOpen,               /* xOpen - open a cursor */
-
    zipfileClose,              /* xClose - close a cursor */
-
    zipfileFilter,             /* xFilter - configure scan constraints */
-
    zipfileNext,               /* xNext - advance a cursor */
-
    zipfileEof,                /* xEof - check for end of scan */
-
    zipfileColumn,             /* xColumn - read data */
-
    0,                         /* xRowid - read data */
-
    zipfileUpdate,             /* xUpdate */
-
    zipfileBegin,              /* xBegin */
-
    0,                         /* xSync */
-
    zipfileCommit,             /* xCommit */
-
    zipfileRollback,           /* xRollback */
-
    zipfileFindFunction,       /* xFindMethod */
-
    0,                         /* xRename */
-
  };
+
        /* The statement the vtab will pass to sqlite3_declare_vtab() */
+
        zInner = idxAppendText(&rc, 0, "CREATE TABLE x(");
+
        for(i=0; i<pTab->nCol; i++){
+
          zInner = idxAppendText(&rc, zInner, "%s%Q COLLATE %s", 
+
              (i==0 ? "" : ", "), pTab->aCol[i].zName, pTab->aCol[i].zColl
+
          );
+
        }
+
        zInner = idxAppendText(&rc, zInner, ")");

-
  int rc = sqlite3_create_module(db, "zipfile"  , &zipfileModule, 0);
-
  if( rc==SQLITE_OK ) rc = sqlite3_overload_function(db, "zipfile_cds", -1);
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_function(db, "zipfile", -1, SQLITE_UTF8, 0, 0, 
-
        zipfileStep, zipfileFinal
-
    );
+
        /* The CVT statement to create the vtab */
+
        zOuter = idxAppendText(&rc, 0, 
+
            "CREATE VIRTUAL TABLE %Q USING expert(%Q)", zName, zInner
+
        );
+
        if( rc==SQLITE_OK ){
+
          rc = sqlite3_exec(p->dbv, zOuter, 0, 0, pzErrmsg);
+
        }
+
        sqlite3_free(zInner);
+
        sqlite3_free(zOuter);
+
      }
+
    }
  }
-
  assert( sizeof(i64)==8 );
-
  assert( sizeof(u32)==4 );
-
  assert( sizeof(u16)==2 );
-
  assert( sizeof(u8)==1 );
+
  idxFinalize(&rc, pSchema);
  return rc;
}
-
#else         /* SQLITE_OMIT_VIRTUALTABLE */
-
# define zipfileRegister(x) SQLITE_OK
-
#endif
-

-
#ifdef _WIN32
-

-
#endif
-
int sqlite3_zipfile_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  SQLITE_EXTENSION_INIT2(pApi);
-
  (void)pzErrMsg;  /* Unused parameter */
-
  return zipfileRegister(db);
-
}

-
/************************* End ../ext/misc/zipfile.c ********************/
-
/************************* Begin ../ext/misc/sqlar.c ******************/
-
/*
-
** 2017-12-17
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** Utility functions sqlar_compress() and sqlar_uncompress(). Useful
-
** for working with sqlar archives and used by the shell tool's built-in
-
** sqlar support.
-
*/
-
/* #include "sqlite3ext.h" */
-
SQLITE_EXTENSION_INIT1
-
#include <zlib.h>
-
#include <assert.h>
+
struct IdxSampleCtx {
+
  int iTarget;
+
  double target;                  /* Target nRet/nRow value */
+
  double nRow;                    /* Number of rows seen */
+
  double nRet;                    /* Number of rows returned */
+
};

-
/*
-
** Implementation of the "sqlar_compress(X)" SQL function.
-
**
-
** If the type of X is SQLITE_BLOB, and compressing that blob using
-
** zlib utility function compress() yields a smaller blob, return the
-
** compressed blob. Otherwise, return a copy of X.
-
**
-
** SQLar uses the "zlib format" for compressed content.  The zlib format
-
** contains a two-byte identification header and a four-byte checksum at
-
** the end.  This is different from ZIP which uses the raw deflate format.
-
**
-
** Future enhancements to SQLar might add support for new compression formats.
-
** If so, those new formats will be identified by alternative headers in the
-
** compressed data.
-
*/
-
static void sqlarCompressFunc(
-
  sqlite3_context *context,
+
static void idxSampleFunc(
+
  sqlite3_context *pCtx,
  int argc,
  sqlite3_value **argv
){
-
  assert( argc==1 );
-
  if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){
-
    const Bytef *pData = sqlite3_value_blob(argv[0]);
-
    uLong nData = sqlite3_value_bytes(argv[0]);
-
    uLongf nOut = compressBound(nData);
-
    Bytef *pOut;
+
  struct IdxSampleCtx *p = (struct IdxSampleCtx*)sqlite3_user_data(pCtx);
+
  int bRet;

-
    pOut = (Bytef*)sqlite3_malloc(nOut);
-
    if( pOut==0 ){
-
      sqlite3_result_error_nomem(context);
-
      return;
-
    }else{
-
      if( Z_OK!=compress(pOut, &nOut, pData, nData) ){
-
        sqlite3_result_error(context, "error in compress()", -1);
-
      }else if( nOut<nData ){
-
        sqlite3_result_blob(context, pOut, nOut, SQLITE_TRANSIENT);
-
      }else{
-
        sqlite3_result_value(context, argv[0]);
-
      }
-
      sqlite3_free(pOut);
-
    }
+
  (void)argv;
+
  assert( argc==0 );
+
  if( p->nRow==0.0 ){
+
    bRet = 1;
  }else{
-
    sqlite3_result_value(context, argv[0]);
+
    bRet = (p->nRet / p->nRow) <= p->target;
+
    if( bRet==0 ){
+
      unsigned short rnd;
+
      sqlite3_randomness(2, (void*)&rnd);
+
      bRet = ((int)rnd % 100) <= p->iTarget;
+
    }
  }
+

+
  sqlite3_result_int(pCtx, bRet);
+
  p->nRow += 1.0;
+
  p->nRet += (double)bRet;
}

+
struct IdxRemCtx {
+
  int nSlot;
+
  struct IdxRemSlot {
+
    int eType;                    /* SQLITE_NULL, INTEGER, REAL, TEXT, BLOB */
+
    i64 iVal;                     /* SQLITE_INTEGER value */
+
    double rVal;                  /* SQLITE_FLOAT value */
+
    int nByte;                    /* Bytes of space allocated at z */
+
    int n;                        /* Size of buffer z */
+
    char *z;                      /* SQLITE_TEXT/BLOB value */
+
  } aSlot[1];
+
};
+

/*
-
** Implementation of the "sqlar_uncompress(X,SZ)" SQL function
-
**
-
** Parameter SZ is interpreted as an integer. If it is less than or
-
** equal to zero, then this function returns a copy of X. Or, if
-
** SZ is equal to the size of X when interpreted as a blob, also
-
** return a copy of X. Otherwise, decompress blob X using zlib
-
** utility function uncompress() and return the results (another
-
** blob).
+
** Implementation of scalar function rem().
*/
-
static void sqlarUncompressFunc(
-
  sqlite3_context *context,
+
static void idxRemFunc(
+
  sqlite3_context *pCtx,
  int argc,
  sqlite3_value **argv
){
-
  uLong nData;
-
  uLongf sz;
-

+
  struct IdxRemCtx *p = (struct IdxRemCtx*)sqlite3_user_data(pCtx);
+
  struct IdxRemSlot *pSlot;
+
  int iSlot;
  assert( argc==2 );
-
  sz = sqlite3_value_int(argv[1]);
-

-
  if( sz<=0 || sz==(nData = sqlite3_value_bytes(argv[0])) ){
-
    sqlite3_result_value(context, argv[0]);
-
  }else{
-
    const Bytef *pData= sqlite3_value_blob(argv[0]);
-
    Bytef *pOut = sqlite3_malloc(sz);
-
    if( Z_OK!=uncompress(pOut, &sz, pData, nData) ){
-
      sqlite3_result_error(context, "error in uncompress()", -1);
-
    }else{
-
      sqlite3_result_blob(context, pOut, sz, SQLITE_TRANSIENT);
-
    }
-
    sqlite3_free(pOut);
-
  }
-
}

+
  iSlot = sqlite3_value_int(argv[0]);
+
  assert( iSlot<=p->nSlot );
+
  pSlot = &p->aSlot[iSlot];

-
#ifdef _WIN32
+
  switch( pSlot->eType ){
+
    case SQLITE_NULL:
+
      /* no-op */
+
      break;

-
#endif
-
int sqlite3_sqlar_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  int rc = SQLITE_OK;
-
  SQLITE_EXTENSION_INIT2(pApi);
-
  (void)pzErrMsg;  /* Unused parameter */
-
  rc = sqlite3_create_function(db, "sqlar_compress", 1, 
-
                               SQLITE_UTF8|SQLITE_INNOCUOUS, 0,
-
                               sqlarCompressFunc, 0, 0);
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_function(db, "sqlar_uncompress", 2,
-
                                 SQLITE_UTF8|SQLITE_INNOCUOUS, 0,
-
                                 sqlarUncompressFunc, 0, 0);
-
  }
-
  return rc;
-
}
+
    case SQLITE_INTEGER:
+
      sqlite3_result_int64(pCtx, pSlot->iVal);
+
      break;

-
/************************* End ../ext/misc/sqlar.c ********************/
-
#endif
-
/************************* Begin ../ext/expert/sqlite3expert.h ******************/
-
/*
-
** 2017 April 07
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
*************************************************************************
-
*/
-
#if !defined(SQLITEEXPERT_H)
-
#define SQLITEEXPERT_H 1
-
/* #include "sqlite3.h" */
+
    case SQLITE_FLOAT:
+
      sqlite3_result_double(pCtx, pSlot->rVal);
+
      break;

-
typedef struct sqlite3expert sqlite3expert;
+
    case SQLITE_BLOB:
+
      sqlite3_result_blob(pCtx, pSlot->z, pSlot->n, SQLITE_TRANSIENT);
+
      break;

-
/*
-
** Create a new sqlite3expert object.
-
**
-
** If successful, a pointer to the new object is returned and (*pzErr) set
-
** to NULL. Or, if an error occurs, NULL is returned and (*pzErr) set to
-
** an English-language error message. In this case it is the responsibility
-
** of the caller to eventually free the error message buffer using
-
** sqlite3_free().
-
*/
-
sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErr);
+
    case SQLITE_TEXT:
+
      sqlite3_result_text(pCtx, pSlot->z, pSlot->n, SQLITE_TRANSIENT);
+
      break;
+
  }

-
/*
-
** Configure an sqlite3expert object.
-
**
-
** EXPERT_CONFIG_SAMPLE:
-
**   By default, sqlite3_expert_analyze() generates sqlite_stat1 data for
-
**   each candidate index. This involves scanning and sorting the entire
-
**   contents of each user database table once for each candidate index
-
**   associated with the table. For large databases, this can be 
-
**   prohibitively slow. This option allows the sqlite3expert object to
-
**   be configured so that sqlite_stat1 data is instead generated based on a
-
**   subset of each table, or so that no sqlite_stat1 data is used at all.
-
**
-
**   A single integer argument is passed to this option. If the value is less
-
**   than or equal to zero, then no sqlite_stat1 data is generated or used by
-
**   the analysis - indexes are recommended based on the database schema only.
-
**   Or, if the value is 100 or greater, complete sqlite_stat1 data is
-
**   generated for each candidate index (this is the default). Finally, if the
-
**   value falls between 0 and 100, then it represents the percentage of user
-
**   table rows that should be considered when generating sqlite_stat1 data.
-
**
-
**   Examples:
-
**
-
**     // Do not generate any sqlite_stat1 data
-
**     sqlite3_expert_config(pExpert, EXPERT_CONFIG_SAMPLE, 0);
-
**
-
**     // Generate sqlite_stat1 data based on 10% of the rows in each table.
-
**     sqlite3_expert_config(pExpert, EXPERT_CONFIG_SAMPLE, 10);
-
*/
-
int sqlite3_expert_config(sqlite3expert *p, int op, ...);
+
  pSlot->eType = sqlite3_value_type(argv[1]);
+
  switch( pSlot->eType ){
+
    case SQLITE_NULL:
+
      /* no-op */
+
      break;

-
#define EXPERT_CONFIG_SAMPLE 1    /* int */
+
    case SQLITE_INTEGER:
+
      pSlot->iVal = sqlite3_value_int64(argv[1]);
+
      break;

-
/*
-
** Specify zero or more SQL statements to be included in the analysis.
-
**
-
** Buffer zSql must contain zero or more complete SQL statements. This
-
** function parses all statements contained in the buffer and adds them
-
** to the internal list of statements to analyze. If successful, SQLITE_OK
-
** is returned and (*pzErr) set to NULL. Or, if an error occurs - for example
-
** due to a error in the SQL - an SQLite error code is returned and (*pzErr)
-
** may be set to point to an English language error message. In this case
-
** the caller is responsible for eventually freeing the error message buffer
-
** using sqlite3_free().
-
**
-
** If an error does occur while processing one of the statements in the
-
** buffer passed as the second argument, none of the statements in the
-
** buffer are added to the analysis.
-
**
-
** This function must be called before sqlite3_expert_analyze(). If a call
-
** to this function is made on an sqlite3expert object that has already
-
** been passed to sqlite3_expert_analyze() SQLITE_MISUSE is returned
-
** immediately and no statements are added to the analysis.
-
*/
-
int sqlite3_expert_sql(
-
  sqlite3expert *p,               /* From a successful sqlite3_expert_new() */
-
  const char *zSql,               /* SQL statement(s) to add */
-
  char **pzErr                    /* OUT: Error message (if any) */
-
);
+
    case SQLITE_FLOAT:
+
      pSlot->rVal = sqlite3_value_double(argv[1]);
+
      break;

+
    case SQLITE_BLOB:
+
    case SQLITE_TEXT: {
+
      int nByte = sqlite3_value_bytes(argv[1]);
+
      const void *pData = 0;
+
      if( nByte>pSlot->nByte ){
+
        char *zNew = (char*)sqlite3_realloc(pSlot->z, nByte*2);
+
        if( zNew==0 ){
+
          sqlite3_result_error_nomem(pCtx);
+
          return;
+
        }
+
        pSlot->nByte = nByte*2;
+
        pSlot->z = zNew;
+
      }
+
      pSlot->n = nByte;
+
      if( pSlot->eType==SQLITE_BLOB ){
+
        pData = sqlite3_value_blob(argv[1]);
+
        if( pData ) memcpy(pSlot->z, pData, nByte);
+
      }else{
+
        pData = sqlite3_value_text(argv[1]);
+
        memcpy(pSlot->z, pData, nByte);
+
      }
+
      break;
+
    }
+
  }
+
}

-
/*
-
** This function is called after the sqlite3expert object has been configured
-
** with all SQL statements using sqlite3_expert_sql() to actually perform
-
** the analysis. Once this function has been called, it is not possible to
-
** add further SQL statements to the analysis.
-
**
-
** If successful, SQLITE_OK is returned and (*pzErr) is set to NULL. Or, if
-
** an error occurs, an SQLite error code is returned and (*pzErr) set to 
-
** point to a buffer containing an English language error message. In this
-
** case it is the responsibility of the caller to eventually free the buffer
-
** using sqlite3_free().
-
**
-
** If an error does occur within this function, the sqlite3expert object
-
** is no longer useful for any purpose. At that point it is no longer
-
** possible to add further SQL statements to the object or to re-attempt
-
** the analysis. The sqlite3expert object must still be freed using a call
-
** sqlite3_expert_destroy().
-
*/
-
int sqlite3_expert_analyze(sqlite3expert *p, char **pzErr);
+
static int idxLargestIndex(sqlite3 *db, int *pnMax, char **pzErr){
+
  int rc = SQLITE_OK;
+
  const char *zMax = 
+
    "SELECT max(i.seqno) FROM "
+
    "  sqlite_schema AS s, "
+
    "  pragma_index_list(s.name) AS l, "
+
    "  pragma_index_info(l.name) AS i "
+
    "WHERE s.type = 'table'";
+
  sqlite3_stmt *pMax = 0;

-
/*
-
** Return the total number of statements loaded using sqlite3_expert_sql().
-
** The total number of SQL statements may be different from the total number
-
** to calls to sqlite3_expert_sql().
-
*/
-
int sqlite3_expert_count(sqlite3expert*);
+
  *pnMax = 0;
+
  rc = idxPrepareStmt(db, &pMax, pzErr, zMax);
+
  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pMax) ){
+
    *pnMax = sqlite3_column_int(pMax, 0) + 1;
+
  }
+
  idxFinalize(&rc, pMax);

-
/*
-
** Return a component of the report.
-
**
-
** This function is called after sqlite3_expert_analyze() to extract the
-
** results of the analysis. Each call to this function returns either a
-
** NULL pointer or a pointer to a buffer containing a nul-terminated string.
-
** The value passed as the third argument must be one of the EXPERT_REPORT_*
-
** #define constants defined below.
-
**
-
** For some EXPERT_REPORT_* parameters, the buffer returned contains 
-
** information relating to a specific SQL statement. In these cases that
-
** SQL statement is identified by the value passed as the second argument.
-
** SQL statements are numbered from 0 in the order in which they are parsed.
-
** If an out-of-range value (less than zero or equal to or greater than the
-
** value returned by sqlite3_expert_count()) is passed as the second argument
-
** along with such an EXPERT_REPORT_* parameter, NULL is always returned.
-
**
-
** EXPERT_REPORT_SQL:
-
**   Return the text of SQL statement iStmt.
-
**
-
** EXPERT_REPORT_INDEXES:
-
**   Return a buffer containing the CREATE INDEX statements for all recommended
-
**   indexes for statement iStmt. If there are no new recommeded indexes, NULL 
-
**   is returned.
-
**
-
** EXPERT_REPORT_PLAN:
-
**   Return a buffer containing the EXPLAIN QUERY PLAN output for SQL query
-
**   iStmt after the proposed indexes have been added to the database schema.
-
**
-
** EXPERT_REPORT_CANDIDATES:
-
**   Return a pointer to a buffer containing the CREATE INDEX statements 
-
**   for all indexes that were tested (for all SQL statements). The iStmt
-
**   parameter is ignored for EXPERT_REPORT_CANDIDATES calls.
-
*/
-
const char *sqlite3_expert_report(sqlite3expert*, int iStmt, int eReport);
+
  return rc;
+
}

-
/*
-
** Values for the third argument passed to sqlite3_expert_report().
-
*/
-
#define EXPERT_REPORT_SQL        1
-
#define EXPERT_REPORT_INDEXES    2
-
#define EXPERT_REPORT_PLAN       3
-
#define EXPERT_REPORT_CANDIDATES 4
+
static int idxPopulateOneStat1(
+
  sqlite3expert *p,
+
  sqlite3_stmt *pIndexXInfo,
+
  sqlite3_stmt *pWriteStat,
+
  const char *zTab,
+
  const char *zIdx,
+
  char **pzErr
+
){
+
  char *zCols = 0;
+
  char *zOrder = 0;
+
  char *zQuery = 0;
+
  int nCol = 0;
+
  int i;
+
  sqlite3_stmt *pQuery = 0;
+
  int *aStat = 0;
+
  int rc = SQLITE_OK;

-
/*
-
** Free an (sqlite3expert*) handle and all associated resources. There 
-
** should be one call to this function for each successful call to 
-
** sqlite3-expert_new().
-
*/
-
void sqlite3_expert_destroy(sqlite3expert*);
+
  assert( p->iSample>0 );

-
#endif  /* !defined(SQLITEEXPERT_H) */
+
  /* Formulate the query text */
+
  sqlite3_bind_text(pIndexXInfo, 1, zIdx, -1, SQLITE_STATIC);
+
  while( SQLITE_OK==rc && SQLITE_ROW==sqlite3_step(pIndexXInfo) ){
+
    const char *zComma = zCols==0 ? "" : ", ";
+
    const char *zName = (const char*)sqlite3_column_text(pIndexXInfo, 0);
+
    const char *zColl = (const char*)sqlite3_column_text(pIndexXInfo, 1);
+
    zCols = idxAppendText(&rc, zCols, 
+
        "%sx.%Q IS rem(%d, x.%Q) COLLATE %s", zComma, zName, nCol, zName, zColl
+
    );
+
    zOrder = idxAppendText(&rc, zOrder, "%s%d", zComma, ++nCol);
+
  }
+
  sqlite3_reset(pIndexXInfo);
+
  if( rc==SQLITE_OK ){
+
    if( p->iSample==100 ){
+
      zQuery = sqlite3_mprintf(
+
          "SELECT %s FROM %Q x ORDER BY %s", zCols, zTab, zOrder
+
      );
+
    }else{
+
      zQuery = sqlite3_mprintf(
+
          "SELECT %s FROM temp."UNIQUE_TABLE_NAME" x ORDER BY %s", zCols, zOrder
+
      );
+
    }
+
  }
+
  sqlite3_free(zCols);
+
  sqlite3_free(zOrder);

-
/************************* End ../ext/expert/sqlite3expert.h ********************/
-
/************************* Begin ../ext/expert/sqlite3expert.c ******************/
-
/*
-
** 2017 April 09
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
*************************************************************************
-
*/
-
/* #include "sqlite3expert.h" */
-
#include <assert.h>
-
#include <string.h>
-
#include <stdio.h>
+
  /* Formulate the query text */
+
  if( rc==SQLITE_OK ){
+
    sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv);
+
    rc = idxPrepareStmt(dbrem, &pQuery, pzErr, zQuery);
+
  }
+
  sqlite3_free(zQuery);

-
#if !defined(SQLITE_AMALGAMATION)
-
#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
-
# define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1
-
#endif
-
#if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS)
-
# define ALWAYS(X)      (1)
-
# define NEVER(X)       (0)
-
#elif !defined(NDEBUG)
-
# define ALWAYS(X)      ((X)?1:(assert(0),0))
-
# define NEVER(X)       ((X)?(assert(0),1):0)
-
#else
-
# define ALWAYS(X)      (X)
-
# define NEVER(X)       (X)
-
#endif
-
#endif /* !defined(SQLITE_AMALGAMATION) */
+
  if( rc==SQLITE_OK ){
+
    aStat = (int*)idxMalloc(&rc, sizeof(int)*(nCol+1));
+
  }
+
  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pQuery) ){
+
    IdxHashEntry *pEntry;
+
    char *zStat = 0;
+
    for(i=0; i<=nCol; i++) aStat[i] = 1;
+
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pQuery) ){
+
      aStat[0]++;
+
      for(i=0; i<nCol; i++){
+
        if( sqlite3_column_int(pQuery, i)==0 ) break;
+
      }
+
      for(/*no-op*/; i<nCol; i++){
+
        aStat[i+1]++;
+
      }
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      int s0 = aStat[0];
+
      zStat = sqlite3_mprintf("%d", s0);
+
      if( zStat==0 ) rc = SQLITE_NOMEM;
+
      for(i=1; rc==SQLITE_OK && i<=nCol; i++){
+
        zStat = idxAppendText(&rc, zStat, " %d", (s0+aStat[i]/2) / aStat[i]);
+
      }
+
    }
+

+
    if( rc==SQLITE_OK ){
+
      sqlite3_bind_text(pWriteStat, 1, zTab, -1, SQLITE_STATIC);
+
      sqlite3_bind_text(pWriteStat, 2, zIdx, -1, SQLITE_STATIC);
+
      sqlite3_bind_text(pWriteStat, 3, zStat, -1, SQLITE_STATIC);
+
      sqlite3_step(pWriteStat);
+
      rc = sqlite3_reset(pWriteStat);
+
    }

+
    pEntry = idxHashFind(&p->hIdx, zIdx, STRLEN(zIdx));
+
    if( pEntry ){
+
      assert( pEntry->zVal2==0 );
+
      pEntry->zVal2 = zStat;
+
    }else{
+
      sqlite3_free(zStat);
+
    }
+
  }
+
  sqlite3_free(aStat);
+
  idxFinalize(&rc, pQuery);

-
#ifndef SQLITE_OMIT_VIRTUALTABLE 
+
  return rc;
+
}

-
/* typedef sqlite3_int64 i64; */
-
/* typedef sqlite3_uint64 u64; */
+
static int idxBuildSampleTable(sqlite3expert *p, const char *zTab){
+
  int rc;
+
  char *zSql;

-
typedef struct IdxColumn IdxColumn;
-
typedef struct IdxConstraint IdxConstraint;
-
typedef struct IdxScan IdxScan;
-
typedef struct IdxStatement IdxStatement;
-
typedef struct IdxTable IdxTable;
-
typedef struct IdxWrite IdxWrite;
+
  rc = sqlite3_exec(p->dbv,"DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0);
+
  if( rc!=SQLITE_OK ) return rc;

-
#define STRLEN  (int)strlen
+
  zSql = sqlite3_mprintf(
+
      "CREATE TABLE temp." UNIQUE_TABLE_NAME " AS SELECT * FROM %Q", zTab
+
  );
+
  if( zSql==0 ) return SQLITE_NOMEM;
+
  rc = sqlite3_exec(p->dbv, zSql, 0, 0, 0);
+
  sqlite3_free(zSql);

-
/*
-
** A temp table name that we assume no user database will actually use.
-
** If this assumption proves incorrect triggers on the table with the
-
** conflicting name will be ignored.
-
*/
-
#define UNIQUE_TABLE_NAME "t592690916721053953805701627921227776"
+
  return rc;
+
}

/*
-
** A single constraint. Equivalent to either "col = ?" or "col < ?" (or
-
** any other type of single-ended range constraint on a column).
+
** This function is called as part of sqlite3_expert_analyze(). Candidate
+
** indexes have already been created in database sqlite3expert.dbm, this
+
** function populates sqlite_stat1 table in the same database.
**
-
** pLink:
-
**   Used to temporarily link IdxConstraint objects into lists while
-
**   creating candidate indexes.
+
** The stat1 data is generated by querying the 
*/
-
struct IdxConstraint {
-
  char *zColl;                    /* Collation sequence */
-
  int bRange;                     /* True for range, false for eq */
-
  int iCol;                       /* Constrained table column */
-
  int bFlag;                      /* Used by idxFindCompatible() */
-
  int bDesc;                      /* True if ORDER BY <expr> DESC */
-
  IdxConstraint *pNext;           /* Next constraint in pEq or pRange list */
-
  IdxConstraint *pLink;           /* See above */
-
};
+
static int idxPopulateStat1(sqlite3expert *p, char **pzErr){
+
  int rc = SQLITE_OK;
+
  int nMax =0;
+
  struct IdxRemCtx *pCtx = 0;
+
  struct IdxSampleCtx samplectx; 
+
  int i;
+
  i64 iPrev = -100000;
+
  sqlite3_stmt *pAllIndex = 0;
+
  sqlite3_stmt *pIndexXInfo = 0;
+
  sqlite3_stmt *pWrite = 0;

-
/*
-
** A single scan of a single table.
-
*/
-
struct IdxScan {
-
  IdxTable *pTab;                 /* Associated table object */
-
  int iDb;                        /* Database containing table zTable */
-
  i64 covering;                   /* Mask of columns required for cov. index */
-
  IdxConstraint *pOrder;          /* ORDER BY columns */
-
  IdxConstraint *pEq;             /* List of == constraints */
-
  IdxConstraint *pRange;          /* List of < constraints */
-
  IdxScan *pNextScan;             /* Next IdxScan object for same analysis */
-
};
+
  const char *zAllIndex =
+
    "SELECT s.rowid, s.name, l.name FROM "
+
    "  sqlite_schema AS s, "
+
    "  pragma_index_list(s.name) AS l "
+
    "WHERE s.type = 'table'";
+
  const char *zIndexXInfo = 
+
    "SELECT name, coll FROM pragma_index_xinfo(?) WHERE key";
+
  const char *zWrite = "INSERT INTO sqlite_stat1 VALUES(?, ?, ?)";

-
/*
-
** Information regarding a single database table. Extracted from 
-
** "PRAGMA table_info" by function idxGetTableInfo().
-
*/
-
struct IdxColumn {
-
  char *zName;
-
  char *zColl;
-
  int iPk;
-
};
-
struct IdxTable {
-
  int nCol;
-
  char *zName;                    /* Table name */
-
  IdxColumn *aCol;
-
  IdxTable *pNext;                /* Next table in linked list of all tables */
-
};
+
  /* If iSample==0, no sqlite_stat1 data is required. */
+
  if( p->iSample==0 ) return SQLITE_OK;

-
/*
-
** An object of the following type is created for each unique table/write-op
-
** seen. The objects are stored in a singly-linked list beginning at
-
** sqlite3expert.pWrite.
-
*/
-
struct IdxWrite {
-
  IdxTable *pTab;
-
  int eOp;                        /* SQLITE_UPDATE, DELETE or INSERT */
-
  IdxWrite *pNext;
-
};
+
  rc = idxLargestIndex(p->dbm, &nMax, pzErr);
+
  if( nMax<=0 || rc!=SQLITE_OK ) return rc;

-
/*
-
** Each statement being analyzed is represented by an instance of this
-
** structure.
-
*/
-
struct IdxStatement {
-
  int iId;                        /* Statement number */
-
  char *zSql;                     /* SQL statement */
-
  char *zIdx;                     /* Indexes */
-
  char *zEQP;                     /* Plan */
-
  IdxStatement *pNext;
-
};
+
  rc = sqlite3_exec(p->dbm, "ANALYZE; PRAGMA writable_schema=1", 0, 0, 0);

+
  if( rc==SQLITE_OK ){
+
    int nByte = sizeof(struct IdxRemCtx) + (sizeof(struct IdxRemSlot) * nMax);
+
    pCtx = (struct IdxRemCtx*)idxMalloc(&rc, nByte);
+
  }

-
/*
-
** A hash table for storing strings. With space for a payload string
-
** with each entry. Methods are:
-
**
-
**   idxHashInit()
-
**   idxHashClear()
-
**   idxHashAdd()
-
**   idxHashSearch()
-
*/
-
#define IDX_HASH_SIZE 1023
-
typedef struct IdxHashEntry IdxHashEntry;
-
typedef struct IdxHash IdxHash;
-
struct IdxHashEntry {
-
  char *zKey;                     /* nul-terminated key */
-
  char *zVal;                     /* nul-terminated value string */
-
  char *zVal2;                    /* nul-terminated value string 2 */
-
  IdxHashEntry *pHashNext;        /* Next entry in same hash bucket */
-
  IdxHashEntry *pNext;            /* Next entry in hash */
-
};
-
struct IdxHash {
-
  IdxHashEntry *pFirst;
-
  IdxHashEntry *aHash[IDX_HASH_SIZE];
-
};
+
  if( rc==SQLITE_OK ){
+
    sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv);
+
    rc = sqlite3_create_function(
+
        dbrem, "rem", 2, SQLITE_UTF8, (void*)pCtx, idxRemFunc, 0, 0
+
    );
+
  }
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_create_function(
+
        p->db, "sample", 0, SQLITE_UTF8, (void*)&samplectx, idxSampleFunc, 0, 0
+
    );
+
  }

-
/*
-
** sqlite3expert object.
-
*/
-
struct sqlite3expert {
-
  int iSample;                    /* Percentage of tables to sample for stat1 */
-
  sqlite3 *db;                    /* User database */
-
  sqlite3 *dbm;                   /* In-memory db for this analysis */
-
  sqlite3 *dbv;                   /* Vtab schema for this analysis */
-
  IdxTable *pTable;               /* List of all IdxTable objects */
-
  IdxScan *pScan;                 /* List of scan objects */
-
  IdxWrite *pWrite;               /* List of write objects */
-
  IdxStatement *pStatement;       /* List of IdxStatement objects */
-
  int bRun;                       /* True once analysis has run */
-
  char **pzErrmsg;
-
  int rc;                         /* Error code from whereinfo hook */
-
  IdxHash hIdx;                   /* Hash containing all candidate indexes */
-
  char *zCandidates;              /* For EXPERT_REPORT_CANDIDATES */
-
};
+
  if( rc==SQLITE_OK ){
+
    pCtx->nSlot = nMax+1;
+
    rc = idxPrepareStmt(p->dbm, &pAllIndex, pzErr, zAllIndex);
+
  }
+
  if( rc==SQLITE_OK ){
+
    rc = idxPrepareStmt(p->dbm, &pIndexXInfo, pzErr, zIndexXInfo);
+
  }
+
  if( rc==SQLITE_OK ){
+
    rc = idxPrepareStmt(p->dbm, &pWrite, pzErr, zWrite);
+
  }
+

+
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pAllIndex) ){
+
    i64 iRowid = sqlite3_column_int64(pAllIndex, 0);
+
    const char *zTab = (const char*)sqlite3_column_text(pAllIndex, 1);
+
    const char *zIdx = (const char*)sqlite3_column_text(pAllIndex, 2);
+
    if( zTab==0 || zIdx==0 ) continue;
+
    if( p->iSample<100 && iPrev!=iRowid ){
+
      samplectx.target = (double)p->iSample / 100.0;
+
      samplectx.iTarget = p->iSample;
+
      samplectx.nRow = 0.0;
+
      samplectx.nRet = 0.0;
+
      rc = idxBuildSampleTable(p, zTab);
+
      if( rc!=SQLITE_OK ) break;
+
    }
+
    rc = idxPopulateOneStat1(p, pIndexXInfo, pWrite, zTab, zIdx, pzErr);
+
    iPrev = iRowid;
+
  }
+
  if( rc==SQLITE_OK && p->iSample<100 ){
+
    rc = sqlite3_exec(p->dbv, 
+
        "DROP TABLE IF EXISTS temp." UNIQUE_TABLE_NAME, 0,0,0
+
    );
+
  }

+
  idxFinalize(&rc, pAllIndex);
+
  idxFinalize(&rc, pIndexXInfo);
+
  idxFinalize(&rc, pWrite);

-
/*
-
** Allocate and return nByte bytes of zeroed memory using sqlite3_malloc(). 
-
** If the allocation fails, set *pRc to SQLITE_NOMEM and return NULL.
-
*/
-
static void *idxMalloc(int *pRc, int nByte){
-
  void *pRet;
-
  assert( *pRc==SQLITE_OK );
-
  assert( nByte>0 );
-
  pRet = sqlite3_malloc(nByte);
-
  if( pRet ){
-
    memset(pRet, 0, nByte);
-
  }else{
-
    *pRc = SQLITE_NOMEM;
+
  if( pCtx ){
+
    for(i=0; i<pCtx->nSlot; i++){
+
      sqlite3_free(pCtx->aSlot[i].z);
+
    }
+
    sqlite3_free(pCtx);
  }
-
  return pRet;
-
}

-
/*
-
** Initialize an IdxHash hash table.
-
*/
-
static void idxHashInit(IdxHash *pHash){
-
  memset(pHash, 0, sizeof(IdxHash));
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_exec(p->dbm, "ANALYZE sqlite_schema", 0, 0, 0);
+
  }
+

+
  sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0);
+
  return rc;
}

/*
-
** Reset an IdxHash hash table.
+
** Allocate a new sqlite3expert object.
*/
-
static void idxHashClear(IdxHash *pHash){
-
  int i;
-
  for(i=0; i<IDX_HASH_SIZE; i++){
-
    IdxHashEntry *pEntry;
-
    IdxHashEntry *pNext;
-
    for(pEntry=pHash->aHash[i]; pEntry; pEntry=pNext){
-
      pNext = pEntry->pHashNext;
-
      sqlite3_free(pEntry->zVal2);
-
      sqlite3_free(pEntry);
+
sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErrmsg){
+
  int rc = SQLITE_OK;
+
  sqlite3expert *pNew;
+

+
  pNew = (sqlite3expert*)idxMalloc(&rc, sizeof(sqlite3expert));
+

+
  /* Open two in-memory databases to work with. The "vtab database" (dbv)
+
  ** will contain a virtual table corresponding to each real table in
+
  ** the user database schema, and a copy of each view. It is used to
+
  ** collect information regarding the WHERE, ORDER BY and other clauses
+
  ** of the user's query.
+
  */
+
  if( rc==SQLITE_OK ){
+
    pNew->db = db;
+
    pNew->iSample = 100;
+
    rc = sqlite3_open(":memory:", &pNew->dbv);
+
  }
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_open(":memory:", &pNew->dbm);
+
    if( rc==SQLITE_OK ){
+
      sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0);
    }
  }
-
  memset(pHash, 0, sizeof(IdxHash));
+
  
+

+
  /* Copy the entire schema of database [db] into [dbm]. */
+
  if( rc==SQLITE_OK ){
+
    sqlite3_stmt *pSql = 0;
+
    rc = idxPrintfPrepareStmt(pNew->db, &pSql, pzErrmsg, 
+
        "SELECT sql FROM sqlite_schema WHERE name NOT LIKE 'sqlite_%%'"
+
        " AND sql NOT LIKE 'CREATE VIRTUAL %%'"
+
    );
+
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSql) ){
+
      const char *zSql = (const char*)sqlite3_column_text(pSql, 0);
+
      if( zSql ) rc = sqlite3_exec(pNew->dbm, zSql, 0, 0, pzErrmsg);
+
    }
+
    idxFinalize(&rc, pSql);
+
  }
+

+
  /* Create the vtab schema */
+
  if( rc==SQLITE_OK ){
+
    rc = idxCreateVtabSchema(pNew, pzErrmsg);
+
  }
+

+
  /* Register the auth callback with dbv */
+
  if( rc==SQLITE_OK ){
+
    sqlite3_set_authorizer(pNew->dbv, idxAuthCallback, (void*)pNew);
+
  }
+

+
  /* If an error has occurred, free the new object and reutrn NULL. Otherwise,
+
  ** return the new sqlite3expert handle.  */
+
  if( rc!=SQLITE_OK ){
+
    sqlite3_expert_destroy(pNew);
+
    pNew = 0;
+
  }
+
  return pNew;
}

/*
-
** Return the index of the hash bucket that the string specified by the
-
** arguments to this function belongs.
+
** Configure an sqlite3expert object.
*/
-
static int idxHashString(const char *z, int n){
-
  unsigned int ret = 0;
-
  int i;
-
  for(i=0; i<n; i++){
-
    ret += (ret<<3) + (unsigned char)(z[i]);
+
int sqlite3_expert_config(sqlite3expert *p, int op, ...){
+
  int rc = SQLITE_OK;
+
  va_list ap;
+
  va_start(ap, op);
+
  switch( op ){
+
    case EXPERT_CONFIG_SAMPLE: {
+
      int iVal = va_arg(ap, int);
+
      if( iVal<0 ) iVal = 0;
+
      if( iVal>100 ) iVal = 100;
+
      p->iSample = iVal;
+
      break;
+
    }
+
    default:
+
      rc = SQLITE_NOTFOUND;
+
      break;
  }
-
  return (int)(ret % IDX_HASH_SIZE);
+

+
  va_end(ap);
+
  return rc;
}

/*
-
** If zKey is already present in the hash table, return non-zero and do
-
** nothing. Otherwise, add an entry with key zKey and payload string zVal to
-
** the hash table passed as the second argument. 
+
** Add an SQL statement to the analysis.
*/
-
static int idxHashAdd(
-
  int *pRc, 
-
  IdxHash *pHash, 
-
  const char *zKey,
-
  const char *zVal
+
int sqlite3_expert_sql(
+
  sqlite3expert *p,               /* From sqlite3_expert_new() */
+
  const char *zSql,               /* SQL statement to add */
+
  char **pzErr                    /* OUT: Error message (if any) */
){
-
  int nKey = STRLEN(zKey);
-
  int iHash = idxHashString(zKey, nKey);
-
  int nVal = (zVal ? STRLEN(zVal) : 0);
-
  IdxHashEntry *pEntry;
-
  assert( iHash>=0 );
-
  for(pEntry=pHash->aHash[iHash]; pEntry; pEntry=pEntry->pHashNext){
-
    if( STRLEN(pEntry->zKey)==nKey && 0==memcmp(pEntry->zKey, zKey, nKey) ){
-
      return 1;
+
  IdxScan *pScanOrig = p->pScan;
+
  IdxStatement *pStmtOrig = p->pStatement;
+
  int rc = SQLITE_OK;
+
  const char *zStmt = zSql;
+

+
  if( p->bRun ) return SQLITE_MISUSE;
+

+
  while( rc==SQLITE_OK && zStmt && zStmt[0] ){
+
    sqlite3_stmt *pStmt = 0;
+
    rc = sqlite3_prepare_v2(p->dbv, zStmt, -1, &pStmt, &zStmt);
+
    if( rc==SQLITE_OK ){
+
      if( pStmt ){
+
        IdxStatement *pNew;
+
        const char *z = sqlite3_sql(pStmt);
+
        int n = STRLEN(z);
+
        pNew = (IdxStatement*)idxMalloc(&rc, sizeof(IdxStatement) + n+1);
+
        if( rc==SQLITE_OK ){
+
          pNew->zSql = (char*)&pNew[1];
+
          memcpy(pNew->zSql, z, n+1);
+
          pNew->pNext = p->pStatement;
+
          if( p->pStatement ) pNew->iId = p->pStatement->iId+1;
+
          p->pStatement = pNew;
+
        }
+
        sqlite3_finalize(pStmt);
+
      }
+
    }else{
+
      idxDatabaseError(p->dbv, pzErr);
    }
  }
-
  pEntry = idxMalloc(pRc, sizeof(IdxHashEntry) + nKey+1 + nVal+1);
-
  if( pEntry ){
-
    pEntry->zKey = (char*)&pEntry[1];
-
    memcpy(pEntry->zKey, zKey, nKey);
-
    if( zVal ){
-
      pEntry->zVal = &pEntry->zKey[nKey+1];
-
      memcpy(pEntry->zVal, zVal, nVal);
-
    }
-
    pEntry->pHashNext = pHash->aHash[iHash];
-
    pHash->aHash[iHash] = pEntry;

-
    pEntry->pNext = pHash->pFirst;
-
    pHash->pFirst = pEntry;
+
  if( rc!=SQLITE_OK ){
+
    idxScanFree(p->pScan, pScanOrig);
+
    idxStatementFree(p->pStatement, pStmtOrig);
+
    p->pScan = pScanOrig;
+
    p->pStatement = pStmtOrig;
  }
-
  return 0;
+

+
  return rc;
}

-
/*
-
** If zKey/nKey is present in the hash table, return a pointer to the 
-
** hash-entry object.
-
*/
-
static IdxHashEntry *idxHashFind(IdxHash *pHash, const char *zKey, int nKey){
-
  int iHash;
+
int sqlite3_expert_analyze(sqlite3expert *p, char **pzErr){
+
  int rc;
  IdxHashEntry *pEntry;
-
  if( nKey<0 ) nKey = STRLEN(zKey);
-
  iHash = idxHashString(zKey, nKey);
-
  assert( iHash>=0 );
-
  for(pEntry=pHash->aHash[iHash]; pEntry; pEntry=pEntry->pHashNext){
-
    if( STRLEN(pEntry->zKey)==nKey && 0==memcmp(pEntry->zKey, zKey, nKey) ){
-
      return pEntry;
-
    }
+

+
  /* Do trigger processing to collect any extra IdxScan structures */
+
  rc = idxProcessTriggers(p, pzErr);
+

+
  /* Create candidate indexes within the in-memory database file */
+
  if( rc==SQLITE_OK ){
+
    rc = idxCreateCandidates(p);
+
  }else if ( rc==SQLITE_BUSY_TIMEOUT ){
+
    if( pzErr )
+
      *pzErr = sqlite3_mprintf("Cannot find a unique index name to propose.");
+
    return rc;
  }
-
  return 0;
+

+
  /* Generate the stat1 data */
+
  if( rc==SQLITE_OK ){
+
    rc = idxPopulateStat1(p, pzErr);
+
  }
+

+
  /* Formulate the EXPERT_REPORT_CANDIDATES text */
+
  for(pEntry=p->hIdx.pFirst; pEntry; pEntry=pEntry->pNext){
+
    p->zCandidates = idxAppendText(&rc, p->zCandidates, 
+
        "%s;%s%s\n", pEntry->zVal, 
+
        pEntry->zVal2 ? " -- stat1: " : "", pEntry->zVal2
+
    );
+
  }
+

+
  /* Figure out which of the candidate indexes are preferred by the query
+
  ** planner and report the results to the user.  */
+
  if( rc==SQLITE_OK ){
+
    rc = idxFindIndexes(p, pzErr);
+
  }
+

+
  if( rc==SQLITE_OK ){
+
    p->bRun = 1;
+
  }
+
  return rc;
}

/*
-
** If the hash table contains an entry with a key equal to the string
-
** passed as the final two arguments to this function, return a pointer
-
** to the payload string. Otherwise, if zKey/nKey is not present in the
-
** hash table, return NULL.
+
** Return the total number of statements that have been added to this
+
** sqlite3expert using sqlite3_expert_sql().
*/
-
static const char *idxHashSearch(IdxHash *pHash, const char *zKey, int nKey){
-
  IdxHashEntry *pEntry = idxHashFind(pHash, zKey, nKey);
-
  if( pEntry ) return pEntry->zVal;
-
  return 0;
+
int sqlite3_expert_count(sqlite3expert *p){
+
  int nRet = 0;
+
  if( p->pStatement ) nRet = p->pStatement->iId+1;
+
  return nRet;
}

/*
-
** Allocate and return a new IdxConstraint object. Set the IdxConstraint.zColl
-
** variable to point to a copy of nul-terminated string zColl.
+
** Return a component of the report.
*/
-
static IdxConstraint *idxNewConstraint(int *pRc, const char *zColl){
-
  IdxConstraint *pNew;
-
  int nColl = STRLEN(zColl);
+
const char *sqlite3_expert_report(sqlite3expert *p, int iStmt, int eReport){
+
  const char *zRet = 0;
+
  IdxStatement *pStmt;

-
  assert( *pRc==SQLITE_OK );
-
  pNew = (IdxConstraint*)idxMalloc(pRc, sizeof(IdxConstraint) * nColl + 1);
-
  if( pNew ){
-
    pNew->zColl = (char*)&pNew[1];
-
    memcpy(pNew->zColl, zColl, nColl+1);
+
  if( p->bRun==0 ) return 0;
+
  for(pStmt=p->pStatement; pStmt && pStmt->iId!=iStmt; pStmt=pStmt->pNext);
+
  switch( eReport ){
+
    case EXPERT_REPORT_SQL:
+
      if( pStmt ) zRet = pStmt->zSql;
+
      break;
+
    case EXPERT_REPORT_INDEXES:
+
      if( pStmt ) zRet = pStmt->zIdx;
+
      break;
+
    case EXPERT_REPORT_PLAN:
+
      if( pStmt ) zRet = pStmt->zEQP;
+
      break;
+
    case EXPERT_REPORT_CANDIDATES:
+
      zRet = p->zCandidates;
+
      break;
  }
-
  return pNew;
-
}
-

-
/*
-
** An error associated with database handle db has just occurred. Pass
-
** the error message to callback function xOut.
-
*/
-
static void idxDatabaseError(
-
  sqlite3 *db,                    /* Database handle */
-
  char **pzErrmsg                 /* Write error here */
-
){
-
  *pzErrmsg = sqlite3_mprintf("%s", sqlite3_errmsg(db));
+
  return zRet;
}

/*
-
** Prepare an SQL statement.
+
** Free an sqlite3expert object.
*/
-
static int idxPrepareStmt(
-
  sqlite3 *db,                    /* Database handle to compile against */
-
  sqlite3_stmt **ppStmt,          /* OUT: Compiled SQL statement */
-
  char **pzErrmsg,                /* OUT: sqlite3_malloc()ed error message */
-
  const char *zSql                /* SQL statement to compile */
-
){
-
  int rc = sqlite3_prepare_v2(db, zSql, -1, ppStmt, 0);
-
  if( rc!=SQLITE_OK ){
-
    *ppStmt = 0;
-
    idxDatabaseError(db, pzErrmsg);
+
void sqlite3_expert_destroy(sqlite3expert *p){
+
  if( p ){
+
    sqlite3_close(p->dbm);
+
    sqlite3_close(p->dbv);
+
    idxScanFree(p->pScan, 0);
+
    idxStatementFree(p->pStatement, 0);
+
    idxTableFree(p->pTable);
+
    idxWriteFree(p->pWrite);
+
    idxHashClear(&p->hIdx);
+
    sqlite3_free(p->zCandidates);
+
    sqlite3_free(p);
  }
-
  return rc;
}

+
#endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */
+

+
/************************* End ../ext/expert/sqlite3expert.c ********************/
+

+
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
+
#define SQLITE_SHELL_HAVE_RECOVER 1
+
#else
+
#define SQLITE_SHELL_HAVE_RECOVER 0
+
#endif
+
#if SQLITE_SHELL_HAVE_RECOVER
+
/************************* Begin ../ext/recover/dbdata.c ******************/
/*
-
** Prepare an SQL statement using the results of a printf() formatting.
+
** 2019-04-17
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** This file contains an implementation of two eponymous virtual tables,
+
** "sqlite_dbdata" and "sqlite_dbptr". Both modules require that the
+
** "sqlite_dbpage" eponymous virtual table be available.
+
**
+
** SQLITE_DBDATA:
+
**   sqlite_dbdata is used to extract data directly from a database b-tree
+
**   page and its associated overflow pages, bypassing the b-tree layer.
+
**   The table schema is equivalent to:
+
**
+
**     CREATE TABLE sqlite_dbdata(
+
**       pgno INTEGER,
+
**       cell INTEGER,
+
**       field INTEGER,
+
**       value ANY,
+
**       schema TEXT HIDDEN
+
**     );
+
**
+
**   IMPORTANT: THE VIRTUAL TABLE SCHEMA ABOVE IS SUBJECT TO CHANGE. IN THE
+
**   FUTURE NEW NON-HIDDEN COLUMNS MAY BE ADDED BETWEEN "value" AND
+
**   "schema".
+
**
+
**   Each page of the database is inspected. If it cannot be interpreted as
+
**   a b-tree page, or if it is a b-tree page containing 0 entries, the
+
**   sqlite_dbdata table contains no rows for that page.  Otherwise, the
+
**   table contains one row for each field in the record associated with
+
**   each cell on the page. For intkey b-trees, the key value is stored in
+
**   field -1.
+
**
+
**   For example, for the database:
+
**
+
**     CREATE TABLE t1(a, b);     -- root page is page 2
+
**     INSERT INTO t1(rowid, a, b) VALUES(5, 'v', 'five');
+
**     INSERT INTO t1(rowid, a, b) VALUES(10, 'x', 'ten');
+
**
+
**   the sqlite_dbdata table contains, as well as from entries related to 
+
**   page 1, content equivalent to:
+
**
+
**     INSERT INTO sqlite_dbdata(pgno, cell, field, value) VALUES
+
**         (2, 0, -1, 5     ),
+
**         (2, 0,  0, 'v'   ),
+
**         (2, 0,  1, 'five'),
+
**         (2, 1, -1, 10    ),
+
**         (2, 1,  0, 'x'   ),
+
**         (2, 1,  1, 'ten' );
+
**
+
**   If database corruption is encountered, this module does not report an
+
**   error. Instead, it attempts to extract as much data as possible and
+
**   ignores the corruption.
+
**
+
** SQLITE_DBPTR:
+
**   The sqlite_dbptr table has the following schema:
+
**
+
**     CREATE TABLE sqlite_dbptr(
+
**       pgno INTEGER,
+
**       child INTEGER,
+
**       schema TEXT HIDDEN
+
**     );
+
**
+
**   It contains one entry for each b-tree pointer between a parent and
+
**   child page in the database.
*/
-
static int idxPrintfPrepareStmt(
-
  sqlite3 *db,                    /* Database handle to compile against */
-
  sqlite3_stmt **ppStmt,          /* OUT: Compiled SQL statement */
-
  char **pzErrmsg,                /* OUT: sqlite3_malloc()ed error message */
-
  const char *zFmt,               /* printf() format of SQL statement */
-
  ...                             /* Trailing printf() arguments */
-
){
-
  va_list ap;
-
  int rc;
-
  char *zSql;
-
  va_start(ap, zFmt);
-
  zSql = sqlite3_vmprintf(zFmt, ap);
-
  if( zSql==0 ){
-
    rc = SQLITE_NOMEM;
-
  }else{
-
    rc = idxPrepareStmt(db, ppStmt, pzErrmsg, zSql);
-
    sqlite3_free(zSql);
-
  }
-
  va_end(ap);
-
  return rc;
-
}

+
#if !defined(SQLITEINT_H) 
+
/* #include "sqlite3ext.h" */

-
/*************************************************************************
-
** Beginning of virtual table implementation.
-
*/
-
typedef struct ExpertVtab ExpertVtab;
-
struct ExpertVtab {
-
  sqlite3_vtab base;
-
  IdxTable *pTab;
-
  sqlite3expert *pExpert;
-
};
+
/* typedef unsigned char u8; */
+
/* typedef unsigned int u32; */

-
typedef struct ExpertCsr ExpertCsr;
-
struct ExpertCsr {
-
  sqlite3_vtab_cursor base;
-
  sqlite3_stmt *pData;
-
};
+
#endif
+
SQLITE_EXTENSION_INIT1
+
#include <string.h>
+
#include <assert.h>

-
static char *expertDequote(const char *zIn){
-
  int n = STRLEN(zIn);
-
  char *zRet = sqlite3_malloc(n);
+
#ifndef SQLITE_OMIT_VIRTUALTABLE

-
  assert( zIn[0]=='\'' );
-
  assert( zIn[n-1]=='\'' );
+
#define DBDATA_PADDING_BYTES 100 

-
  if( zRet ){
-
    int iOut = 0;
-
    int iIn = 0;
-
    for(iIn=1; iIn<(n-1); iIn++){
-
      if( zIn[iIn]=='\'' ){
-
        assert( zIn[iIn+1]=='\'' );
-
        iIn++;
-
      }
-
      zRet[iOut++] = zIn[iIn];
-
    }
-
    zRet[iOut] = '\0';
-
  }
+
typedef struct DbdataTable DbdataTable;
+
typedef struct DbdataCursor DbdataCursor;

-
  return zRet;
-
}
+
/* Cursor object */
+
struct DbdataCursor {
+
  sqlite3_vtab_cursor base;       /* Base class.  Must be first */
+
  sqlite3_stmt *pStmt;            /* For fetching database pages */

-
/* 
-
** This function is the implementation of both the xConnect and xCreate
-
** methods of the r-tree virtual table.
-
**
-
**   argv[0]   -> module name
-
**   argv[1]   -> database name
-
**   argv[2]   -> table name
-
**   argv[...] -> column names...
+
  int iPgno;                      /* Current page number */
+
  u8 *aPage;                      /* Buffer containing page */
+
  int nPage;                      /* Size of aPage[] in bytes */
+
  int nCell;                      /* Number of cells on aPage[] */
+
  int iCell;                      /* Current cell number */
+
  int bOnePage;                   /* True to stop after one page */
+
  int szDb;
+
  sqlite3_int64 iRowid;
+

+
  /* Only for the sqlite_dbdata table */
+
  u8 *pRec;                       /* Buffer containing current record */
+
  sqlite3_int64 nRec;             /* Size of pRec[] in bytes */
+
  sqlite3_int64 nHdr;             /* Size of header in bytes */
+
  int iField;                     /* Current field number */
+
  u8 *pHdrPtr;
+
  u8 *pPtr;
+
  u32 enc;                        /* Text encoding */
+
  
+
  sqlite3_int64 iIntkey;          /* Integer key value */
+
};
+

+
/* Table object */
+
struct DbdataTable {
+
  sqlite3_vtab base;              /* Base class.  Must be first */
+
  sqlite3 *db;                    /* The database connection */
+
  sqlite3_stmt *pStmt;            /* For fetching database pages */
+
  int bPtr;                       /* True for sqlite3_dbptr table */
+
};
+

+
/* Column and schema definitions for sqlite_dbdata */
+
#define DBDATA_COLUMN_PGNO        0
+
#define DBDATA_COLUMN_CELL        1
+
#define DBDATA_COLUMN_FIELD       2
+
#define DBDATA_COLUMN_VALUE       3
+
#define DBDATA_COLUMN_SCHEMA      4
+
#define DBDATA_SCHEMA             \
+
      "CREATE TABLE x("           \
+
      "  pgno INTEGER,"           \
+
      "  cell INTEGER,"           \
+
      "  field INTEGER,"          \
+
      "  value ANY,"              \
+
      "  schema TEXT HIDDEN"      \
+
      ")"
+

+
/* Column and schema definitions for sqlite_dbptr */
+
#define DBPTR_COLUMN_PGNO         0
+
#define DBPTR_COLUMN_CHILD        1
+
#define DBPTR_COLUMN_SCHEMA       2
+
#define DBPTR_SCHEMA              \
+
      "CREATE TABLE x("           \
+
      "  pgno INTEGER,"           \
+
      "  child INTEGER,"          \
+
      "  schema TEXT HIDDEN"      \
+
      ")"
+

+
/*
+
** Connect to an sqlite_dbdata (pAux==0) or sqlite_dbptr (pAux!=0) virtual 
+
** table.
*/
-
static int expertConnect(
+
static int dbdataConnect(
  sqlite3 *db,
  void *pAux,
  int argc, const char *const*argv,
  sqlite3_vtab **ppVtab,
  char **pzErr
){
-
  sqlite3expert *pExpert = (sqlite3expert*)pAux;
-
  ExpertVtab *p = 0;
-
  int rc;
+
  DbdataTable *pTab = 0;
+
  int rc = sqlite3_declare_vtab(db, pAux ? DBPTR_SCHEMA : DBDATA_SCHEMA);

-
  if( argc!=4 ){
-
    *pzErr = sqlite3_mprintf("internal error!");
-
    rc = SQLITE_ERROR;
-
  }else{
-
    char *zCreateTable = expertDequote(argv[3]);
-
    if( zCreateTable ){
-
      rc = sqlite3_declare_vtab(db, zCreateTable);
-
      if( rc==SQLITE_OK ){
-
        p = idxMalloc(&rc, sizeof(ExpertVtab));
-
      }
-
      if( rc==SQLITE_OK ){
-
        p->pExpert = pExpert;
-
        p->pTab = pExpert->pTable;
-
        assert( sqlite3_stricmp(p->pTab->zName, argv[2])==0 );
-
      }
-
      sqlite3_free(zCreateTable);
-
    }else{
+
  if( rc==SQLITE_OK ){
+
    pTab = (DbdataTable*)sqlite3_malloc64(sizeof(DbdataTable));
+
    if( pTab==0 ){
      rc = SQLITE_NOMEM;
+
    }else{
+
      memset(pTab, 0, sizeof(DbdataTable));
+
      pTab->db = db;
+
      pTab->bPtr = (pAux!=0);
    }
  }

-
  *ppVtab = (sqlite3_vtab*)p;
+
  *ppVtab = (sqlite3_vtab*)pTab;
  return rc;
}

-
static int expertDisconnect(sqlite3_vtab *pVtab){
-
  ExpertVtab *p = (ExpertVtab*)pVtab;
-
  sqlite3_free(p);
+
/*
+
** Disconnect from or destroy a sqlite_dbdata or sqlite_dbptr virtual table.
+
*/
+
static int dbdataDisconnect(sqlite3_vtab *pVtab){
+
  DbdataTable *pTab = (DbdataTable*)pVtab;
+
  if( pTab ){
+
    sqlite3_finalize(pTab->pStmt);
+
    sqlite3_free(pVtab);
+
  }
  return SQLITE_OK;
}

-
static int expertBestIndex(sqlite3_vtab *pVtab, sqlite3_index_info *pIdxInfo){
-
  ExpertVtab *p = (ExpertVtab*)pVtab;
-
  int rc = SQLITE_OK;
-
  int n = 0;
-
  IdxScan *pScan;
-
  const int opmask = 
-
    SQLITE_INDEX_CONSTRAINT_EQ | SQLITE_INDEX_CONSTRAINT_GT |
-
    SQLITE_INDEX_CONSTRAINT_LT | SQLITE_INDEX_CONSTRAINT_GE |
-
    SQLITE_INDEX_CONSTRAINT_LE;
-

-
  pScan = idxMalloc(&rc, sizeof(IdxScan));
-
  if( pScan ){
-
    int i;
-

-
    /* Link the new scan object into the list */
-
    pScan->pTab = p->pTab;
-
    pScan->pNextScan = p->pExpert->pScan;
-
    p->pExpert->pScan = pScan;
+
/*
+
** This function interprets two types of constraints:
+
**
+
**       schema=?
+
**       pgno=?
+
**
+
** If neither are present, idxNum is set to 0. If schema=? is present,
+
** the 0x01 bit in idxNum is set. If pgno=? is present, the 0x02 bit
+
** in idxNum is set.
+
**
+
** If both parameters are present, schema is in position 0 and pgno in
+
** position 1.
+
*/
+
static int dbdataBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdx){
+
  DbdataTable *pTab = (DbdataTable*)tab;
+
  int i;
+
  int iSchema = -1;
+
  int iPgno = -1;
+
  int colSchema = (pTab->bPtr ? DBPTR_COLUMN_SCHEMA : DBDATA_COLUMN_SCHEMA);

-
    /* Add the constraints to the IdxScan object */
-
    for(i=0; i<pIdxInfo->nConstraint; i++){
-
      struct sqlite3_index_constraint *pCons = &pIdxInfo->aConstraint[i];
-
      if( pCons->usable 
-
       && pCons->iColumn>=0 
-
       && p->pTab->aCol[pCons->iColumn].iPk==0
-
       && (pCons->op & opmask) 
-
      ){
-
        IdxConstraint *pNew;
-
        const char *zColl = sqlite3_vtab_collation(pIdxInfo, i);
-
        pNew = idxNewConstraint(&rc, zColl);
-
        if( pNew ){
-
          pNew->iCol = pCons->iColumn;
-
          if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ ){
-
            pNew->pNext = pScan->pEq;
-
            pScan->pEq = pNew;
-
          }else{
-
            pNew->bRange = 1;
-
            pNew->pNext = pScan->pRange;
-
            pScan->pRange = pNew;
-
          }
-
        }
-
        n++;
-
        pIdxInfo->aConstraintUsage[i].argvIndex = n;
+
  for(i=0; i<pIdx->nConstraint; i++){
+
    struct sqlite3_index_constraint *p = &pIdx->aConstraint[i];
+
    if( p->op==SQLITE_INDEX_CONSTRAINT_EQ ){
+
      if( p->iColumn==colSchema ){
+
        if( p->usable==0 ) return SQLITE_CONSTRAINT;
+
        iSchema = i;
+
      }
+
      if( p->iColumn==DBDATA_COLUMN_PGNO && p->usable ){
+
        iPgno = i;
      }
    }
+
  }

-
    /* Add the ORDER BY to the IdxScan object */
-
    for(i=pIdxInfo->nOrderBy-1; i>=0; i--){
-
      int iCol = pIdxInfo->aOrderBy[i].iColumn;
-
      if( iCol>=0 ){
-
        IdxConstraint *pNew = idxNewConstraint(&rc, p->pTab->aCol[iCol].zColl);
-
        if( pNew ){
-
          pNew->iCol = iCol;
-
          pNew->bDesc = pIdxInfo->aOrderBy[i].desc;
-
          pNew->pNext = pScan->pOrder;
-
          pNew->pLink = pScan->pOrder;
-
          pScan->pOrder = pNew;
-
          n++;
-
        }
+
  if( iSchema>=0 ){
+
    pIdx->aConstraintUsage[iSchema].argvIndex = 1;
+
    pIdx->aConstraintUsage[iSchema].omit = 1;
+
  }
+
  if( iPgno>=0 ){
+
    pIdx->aConstraintUsage[iPgno].argvIndex = 1 + (iSchema>=0);
+
    pIdx->aConstraintUsage[iPgno].omit = 1;
+
    pIdx->estimatedCost = 100;
+
    pIdx->estimatedRows =  50;
+

+
    if( pTab->bPtr==0 && pIdx->nOrderBy && pIdx->aOrderBy[0].desc==0 ){
+
      int iCol = pIdx->aOrderBy[0].iColumn;
+
      if( pIdx->nOrderBy==1 ){
+
        pIdx->orderByConsumed = (iCol==0 || iCol==1);
+
      }else if( pIdx->nOrderBy==2 && pIdx->aOrderBy[1].desc==0 && iCol==0 ){
+
        pIdx->orderByConsumed = (pIdx->aOrderBy[1].iColumn==1);
      }
    }
-
  }

-
  pIdxInfo->estimatedCost = 1000000.0 / (n+1);
-
  return rc;
+
  }else{
+
    pIdx->estimatedCost = 100000000;
+
    pIdx->estimatedRows = 1000000000;
+
  }
+
  pIdx->idxNum = (iSchema>=0 ? 0x01 : 0x00) | (iPgno>=0 ? 0x02 : 0x00);
+
  return SQLITE_OK;
}

-
static int expertUpdate(
-
  sqlite3_vtab *pVtab, 
-
  int nData, 
-
  sqlite3_value **azData, 
-
  sqlite_int64 *pRowid
-
){
-
  (void)pVtab;
-
  (void)nData;
-
  (void)azData;
-
  (void)pRowid;
+
/*
+
** Open a new sqlite_dbdata or sqlite_dbptr cursor.
+
*/
+
static int dbdataOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
+
  DbdataCursor *pCsr;
+

+
  pCsr = (DbdataCursor*)sqlite3_malloc64(sizeof(DbdataCursor));
+
  if( pCsr==0 ){
+
    return SQLITE_NOMEM;
+
  }else{
+
    memset(pCsr, 0, sizeof(DbdataCursor));
+
    pCsr->base.pVtab = pVTab;
+
  }
+

+
  *ppCursor = (sqlite3_vtab_cursor *)pCsr;
  return SQLITE_OK;
}

-
/* 
-
** Virtual table module xOpen method.
+
/*
+
** Restore a cursor object to the state it was in when first allocated 
+
** by dbdataOpen().
*/
-
static int expertOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
-
  int rc = SQLITE_OK;
-
  ExpertCsr *pCsr;
-
  (void)pVTab;
-
  pCsr = idxMalloc(&rc, sizeof(ExpertCsr));
-
  *ppCursor = (sqlite3_vtab_cursor*)pCsr;
-
  return rc;
+
static void dbdataResetCursor(DbdataCursor *pCsr){
+
  DbdataTable *pTab = (DbdataTable*)(pCsr->base.pVtab);
+
  if( pTab->pStmt==0 ){
+
    pTab->pStmt = pCsr->pStmt;
+
  }else{
+
    sqlite3_finalize(pCsr->pStmt);
+
  }
+
  pCsr->pStmt = 0;
+
  pCsr->iPgno = 1;
+
  pCsr->iCell = 0;
+
  pCsr->iField = 0;
+
  pCsr->bOnePage = 0;
+
  sqlite3_free(pCsr->aPage);
+
  sqlite3_free(pCsr->pRec);
+
  pCsr->pRec = 0;
+
  pCsr->aPage = 0;
}

-
/* 
-
** Virtual table module xClose method.
+
/*
+
** Close an sqlite_dbdata or sqlite_dbptr cursor.
*/
-
static int expertClose(sqlite3_vtab_cursor *cur){
-
  ExpertCsr *pCsr = (ExpertCsr*)cur;
-
  sqlite3_finalize(pCsr->pData);
+
static int dbdataClose(sqlite3_vtab_cursor *pCursor){
+
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
+
  dbdataResetCursor(pCsr);
  sqlite3_free(pCsr);
  return SQLITE_OK;
}

-
/*
-
** Virtual table module xEof method.
-
**
-
** Return non-zero if the cursor does not currently point to a valid 
-
** record (i.e if the scan has finished), or zero otherwise.
+
/* 
+
** Utility methods to decode 16 and 32-bit big-endian unsigned integers. 
*/
-
static int expertEof(sqlite3_vtab_cursor *cur){
-
  ExpertCsr *pCsr = (ExpertCsr*)cur;
-
  return pCsr->pData==0;
+
static u32 get_uint16(unsigned char *a){
+
  return (a[0]<<8)|a[1];
+
}
+
static u32 get_uint32(unsigned char *a){
+
  return ((u32)a[0]<<24)
+
       | ((u32)a[1]<<16)
+
       | ((u32)a[2]<<8)
+
       | ((u32)a[3]);
}

-
/* 
-
** Virtual table module xNext method.
+
/*
+
** Load page pgno from the database via the sqlite_dbpage virtual table.
+
** If successful, set (*ppPage) to point to a buffer containing the page
+
** data, (*pnPage) to the size of that buffer in bytes and return
+
** SQLITE_OK. In this case it is the responsibility of the caller to
+
** eventually free the buffer using sqlite3_free().
+
**
+
** Or, if an error occurs, set both (*ppPage) and (*pnPage) to 0 and
+
** return an SQLite error code.
*/
-
static int expertNext(sqlite3_vtab_cursor *cur){
-
  ExpertCsr *pCsr = (ExpertCsr*)cur;
+
static int dbdataLoadPage(
+
  DbdataCursor *pCsr,             /* Cursor object */
+
  u32 pgno,                       /* Page number of page to load */
+
  u8 **ppPage,                    /* OUT: pointer to page buffer */
+
  int *pnPage                     /* OUT: Size of (*ppPage) in bytes */
+
){
+
  int rc2;
  int rc = SQLITE_OK;
+
  sqlite3_stmt *pStmt = pCsr->pStmt;

-
  assert( pCsr->pData );
-
  rc = sqlite3_step(pCsr->pData);
-
  if( rc!=SQLITE_ROW ){
-
    rc = sqlite3_finalize(pCsr->pData);
-
    pCsr->pData = 0;
-
  }else{
-
    rc = SQLITE_OK;
+
  *ppPage = 0;
+
  *pnPage = 0;
+
  if( pgno>0 ){
+
    sqlite3_bind_int64(pStmt, 2, pgno);
+
    if( SQLITE_ROW==sqlite3_step(pStmt) ){
+
      int nCopy = sqlite3_column_bytes(pStmt, 0);
+
      if( nCopy>0 ){
+
        u8 *pPage;
+
        pPage = (u8*)sqlite3_malloc64(nCopy + DBDATA_PADDING_BYTES);
+
        if( pPage==0 ){
+
          rc = SQLITE_NOMEM;
+
        }else{
+
          const u8 *pCopy = sqlite3_column_blob(pStmt, 0);
+
          memcpy(pPage, pCopy, nCopy);
+
          memset(&pPage[nCopy], 0, DBDATA_PADDING_BYTES);
+
        }
+
        *ppPage = pPage;
+
        *pnPage = nCopy;
+
      }
+
    }
+
    rc2 = sqlite3_reset(pStmt);
+
    if( rc==SQLITE_OK ) rc = rc2;
  }

  return rc;
}

-
/* 
-
** Virtual table module xRowid method.
+
/*
+
** Read a varint.  Put the value in *pVal and return the number of bytes.
*/
-
static int expertRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
-
  (void)cur;
-
  *pRowid = 0;
-
  return SQLITE_OK;
+
static int dbdataGetVarint(const u8 *z, sqlite3_int64 *pVal){
+
  sqlite3_uint64 u = 0;
+
  int i;
+
  for(i=0; i<8; i++){
+
    u = (u<<7) + (z[i]&0x7f);
+
    if( (z[i]&0x80)==0 ){ *pVal = (sqlite3_int64)u; return i+1; }
+
  }
+
  u = (u<<8) + (z[i]&0xff);
+
  *pVal = (sqlite3_int64)u;
+
  return 9;
}

-
/* 
-
** Virtual table module xColumn method.
+
/*
+
** Like dbdataGetVarint(), but set the output to 0 if it is less than 0
+
** or greater than 0xFFFFFFFF. This can be used for all varints in an
+
** SQLite database except for key values in intkey tables.
*/
-
static int expertColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i){
-
  ExpertCsr *pCsr = (ExpertCsr*)cur;
-
  sqlite3_value *pVal;
-
  pVal = sqlite3_column_value(pCsr->pData, i);
-
  if( pVal ){
-
    sqlite3_result_value(ctx, pVal);
-
  }
-
  return SQLITE_OK;
+
static int dbdataGetVarintU32(const u8 *z, sqlite3_int64 *pVal){
+
  sqlite3_int64 val;
+
  int nRet = dbdataGetVarint(z, &val);
+
  if( val<0 || val>0xFFFFFFFF ) val = 0;
+
  *pVal = val;
+
  return nRet;
}

-
/* 
-
** Virtual table module xFilter method.
+
/*
+
** Return the number of bytes of space used by an SQLite value of type
+
** eType.
*/
-
static int expertFilter(
-
  sqlite3_vtab_cursor *cur, 
-
  int idxNum, const char *idxStr,
-
  int argc, sqlite3_value **argv
-
){
-
  ExpertCsr *pCsr = (ExpertCsr*)cur;
-
  ExpertVtab *pVtab = (ExpertVtab*)(cur->pVtab);
-
  sqlite3expert *pExpert = pVtab->pExpert;
-
  int rc;
-

-
  (void)idxNum;
-
  (void)idxStr;
-
  (void)argc;
-
  (void)argv;
-
  rc = sqlite3_finalize(pCsr->pData);
-
  pCsr->pData = 0;
-
  if( rc==SQLITE_OK ){
-
    rc = idxPrintfPrepareStmt(pExpert->db, &pCsr->pData, &pVtab->base.zErrMsg,
-
        "SELECT * FROM main.%Q WHERE sample()", pVtab->pTab->zName
-
    );
-
  }
-

-
  if( rc==SQLITE_OK ){
-
    rc = expertNext(cur);
+
static int dbdataValueBytes(int eType){
+
  switch( eType ){
+
    case 0: case 8: case 9:
+
    case 10: case 11:
+
      return 0;
+
    case 1:
+
      return 1;
+
    case 2:
+
      return 2;
+
    case 3:
+
      return 3;
+
    case 4:
+
      return 4;
+
    case 5:
+
      return 6;
+
    case 6:
+
    case 7:
+
      return 8;
+
    default:
+
      if( eType>0 ){
+
        return ((eType-12) / 2);
+
      }
+
      return 0;
  }
-
  return rc;
}

-
static int idxRegisterVtab(sqlite3expert *p){
-
  static sqlite3_module expertModule = {
-
    2,                            /* iVersion */
-
    expertConnect,                /* xCreate - create a table */
-
    expertConnect,                /* xConnect - connect to an existing table */
-
    expertBestIndex,              /* xBestIndex - Determine search strategy */
-
    expertDisconnect,             /* xDisconnect - Disconnect from a table */
-
    expertDisconnect,             /* xDestroy - Drop a table */
-
    expertOpen,                   /* xOpen - open a cursor */
-
    expertClose,                  /* xClose - close a cursor */
-
    expertFilter,                 /* xFilter - configure scan constraints */
-
    expertNext,                   /* xNext - advance a cursor */
-
    expertEof,                    /* xEof */
-
    expertColumn,                 /* xColumn - read data */
-
    expertRowid,                  /* xRowid - read data */
-
    expertUpdate,                 /* xUpdate - write data */
-
    0,                            /* xBegin - begin transaction */
-
    0,                            /* xSync - sync transaction */
-
    0,                            /* xCommit - commit transaction */
-
    0,                            /* xRollback - rollback transaction */
-
    0,                            /* xFindFunction - function overloading */
-
    0,                            /* xRename - rename the table */
-
    0,                            /* xSavepoint */
-
    0,                            /* xRelease */
-
    0,                            /* xRollbackTo */
-
    0,                            /* xShadowName */
-
  };
-

-
  return sqlite3_create_module(p->dbv, "expert", &expertModule, (void*)p);
-
}
-
/*
-
** End of virtual table implementation.
-
*************************************************************************/
/*
-
** Finalize SQL statement pStmt. If (*pRc) is SQLITE_OK when this function
-
** is called, set it to the return value of sqlite3_finalize() before
-
** returning. Otherwise, discard the sqlite3_finalize() return value.
+
** Load a value of type eType from buffer pData and use it to set the
+
** result of context object pCtx.
*/
-
static void idxFinalize(int *pRc, sqlite3_stmt *pStmt){
-
  int rc = sqlite3_finalize(pStmt);
-
  if( *pRc==SQLITE_OK ) *pRc = rc;
+
static void dbdataValue(
+
  sqlite3_context *pCtx, 
+
  u32 enc,
+
  int eType, 
+
  u8 *pData,
+
  sqlite3_int64 nData
+
){
+
  if( eType>=0 && dbdataValueBytes(eType)<=nData ){
+
    switch( eType ){
+
      case 0: 
+
      case 10: 
+
      case 11: 
+
        sqlite3_result_null(pCtx);
+
        break;
+
      
+
      case 8: 
+
        sqlite3_result_int(pCtx, 0);
+
        break;
+
      case 9:
+
        sqlite3_result_int(pCtx, 1);
+
        break;
+
  
+
      case 1: case 2: case 3: case 4: case 5: case 6: case 7: {
+
        sqlite3_uint64 v = (signed char)pData[0];
+
        pData++;
+
        switch( eType ){
+
          case 7:
+
          case 6:  v = (v<<16) + (pData[0]<<8) + pData[1];  pData += 2;
+
          case 5:  v = (v<<16) + (pData[0]<<8) + pData[1];  pData += 2;
+
          case 4:  v = (v<<8) + pData[0];  pData++;
+
          case 3:  v = (v<<8) + pData[0];  pData++;
+
          case 2:  v = (v<<8) + pData[0];  pData++;
+
        }
+
  
+
        if( eType==7 ){
+
          double r;
+
          memcpy(&r, &v, sizeof(r));
+
          sqlite3_result_double(pCtx, r);
+
        }else{
+
          sqlite3_result_int64(pCtx, (sqlite3_int64)v);
+
        }
+
        break;
+
      }
+
  
+
      default: {
+
        int n = ((eType-12) / 2);
+
        if( eType % 2 ){
+
          switch( enc ){
+
#ifndef SQLITE_OMIT_UTF16
+
            case SQLITE_UTF16BE:
+
              sqlite3_result_text16be(pCtx, (void*)pData, n, SQLITE_TRANSIENT);
+
              break;
+
            case SQLITE_UTF16LE:
+
              sqlite3_result_text16le(pCtx, (void*)pData, n, SQLITE_TRANSIENT);
+
              break;
+
#endif
+
            default:
+
              sqlite3_result_text(pCtx, (char*)pData, n, SQLITE_TRANSIENT);
+
              break;
+
          }
+
        }else{
+
          sqlite3_result_blob(pCtx, pData, n, SQLITE_TRANSIENT);
+
        }
+
      }
+
    }
+
  }
}

/*
-
** Attempt to allocate an IdxTable structure corresponding to table zTab
-
** in the main database of connection db. If successful, set (*ppOut) to
-
** point to the new object and return SQLITE_OK. Otherwise, return an
-
** SQLite error code and set (*ppOut) to NULL. In this case *pzErrmsg may be
-
** set to point to an error string.
-
**
-
** It is the responsibility of the caller to eventually free either the
-
** IdxTable object or error message using sqlite3_free().
+
** Move an sqlite_dbdata or sqlite_dbptr cursor to the next entry.
*/
-
static int idxGetTableInfo(
-
  sqlite3 *db,                    /* Database connection to read details from */
-
  const char *zTab,               /* Table name */
-
  IdxTable **ppOut,               /* OUT: New object (if successful) */
-
  char **pzErrmsg                 /* OUT: Error message (if not) */
-
){
-
  sqlite3_stmt *p1 = 0;
-
  int nCol = 0;
-
  int nTab;
-
  int nByte;
-
  IdxTable *pNew = 0;
-
  int rc, rc2;
-
  char *pCsr = 0;
-
  int nPk = 0;
+
static int dbdataNext(sqlite3_vtab_cursor *pCursor){
+
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
+
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
+

+
  pCsr->iRowid++;
+
  while( 1 ){
+
    int rc;
+
    int iOff = (pCsr->iPgno==1 ? 100 : 0);
+
    int bNextPage = 0;
+

+
    if( pCsr->aPage==0 ){
+
      while( 1 ){
+
        if( pCsr->bOnePage==0 && pCsr->iPgno>pCsr->szDb ) return SQLITE_OK;
+
        rc = dbdataLoadPage(pCsr, pCsr->iPgno, &pCsr->aPage, &pCsr->nPage);
+
        if( rc!=SQLITE_OK ) return rc;
+
        if( pCsr->aPage ) break;
+
        if( pCsr->bOnePage ) return SQLITE_OK;
+
        pCsr->iPgno++;
+
      }
+
      pCsr->iCell = pTab->bPtr ? -2 : 0;
+
      pCsr->nCell = get_uint16(&pCsr->aPage[iOff+3]);
+
    }
+

+
    if( pTab->bPtr ){
+
      if( pCsr->aPage[iOff]!=0x02 && pCsr->aPage[iOff]!=0x05 ){
+
        pCsr->iCell = pCsr->nCell;
+
      }
+
      pCsr->iCell++;
+
      if( pCsr->iCell>=pCsr->nCell ){
+
        sqlite3_free(pCsr->aPage);
+
        pCsr->aPage = 0;
+
        if( pCsr->bOnePage ) return SQLITE_OK;
+
        pCsr->iPgno++;
+
      }else{
+
        return SQLITE_OK;
+
      }
+
    }else{
+
      /* If there is no record loaded, load it now. */
+
      if( pCsr->pRec==0 ){
+
        int bHasRowid = 0;
+
        int nPointer = 0;
+
        sqlite3_int64 nPayload = 0;
+
        sqlite3_int64 nHdr = 0;
+
        int iHdr;
+
        int U, X;
+
        int nLocal;
+
  
+
        switch( pCsr->aPage[iOff] ){
+
          case 0x02:
+
            nPointer = 4;
+
            break;
+
          case 0x0a:
+
            break;
+
          case 0x0d:
+
            bHasRowid = 1;
+
            break;
+
          default:
+
            /* This is not a b-tree page with records on it. Continue. */
+
            pCsr->iCell = pCsr->nCell;
+
            break;
+
        }

-
  *ppOut = 0;
-
  if( zTab==0 ) return SQLITE_ERROR;
-
  nTab = STRLEN(zTab);
-
  nByte = sizeof(IdxTable) + nTab + 1;
-
  rc = idxPrintfPrepareStmt(db, &p1, pzErrmsg, "PRAGMA table_xinfo=%Q", zTab);
-
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){
-
    const char *zCol = (const char*)sqlite3_column_text(p1, 1);
-
    const char *zColSeq = 0;
-
    if( zCol==0 ){
-
      rc = SQLITE_ERROR;
-
      break;
-
    }
-
    nByte += 1 + STRLEN(zCol);
-
    rc = sqlite3_table_column_metadata(
-
        db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0
-
    );
-
    if( zColSeq==0 ) zColSeq = "binary";
-
    nByte += 1 + STRLEN(zColSeq);
-
    nCol++;
-
    nPk += (sqlite3_column_int(p1, 5)>0);
-
  }
-
  rc2 = sqlite3_reset(p1);
-
  if( rc==SQLITE_OK ) rc = rc2;
+
        if( pCsr->iCell>=pCsr->nCell ){
+
          bNextPage = 1;
+
        }else{
+
  
+
          iOff += 8 + nPointer + pCsr->iCell*2;
+
          if( iOff>pCsr->nPage ){
+
            bNextPage = 1;
+
          }else{
+
            iOff = get_uint16(&pCsr->aPage[iOff]);
+
          }
+
    
+
          /* For an interior node cell, skip past the child-page number */
+
          iOff += nPointer;
+
    
+
          /* Load the "byte of payload including overflow" field */
+
          if( bNextPage || iOff>pCsr->nPage ){
+
            bNextPage = 1;
+
          }else{
+
            iOff += dbdataGetVarintU32(&pCsr->aPage[iOff], &nPayload);
+
          }
+
    
+
          /* If this is a leaf intkey cell, load the rowid */
+
          if( bHasRowid && !bNextPage && iOff<pCsr->nPage ){
+
            iOff += dbdataGetVarint(&pCsr->aPage[iOff], &pCsr->iIntkey);
+
          }
+
    
+
          /* Figure out how much data to read from the local page */
+
          U = pCsr->nPage;
+
          if( bHasRowid ){
+
            X = U-35;
+
          }else{
+
            X = ((U-12)*64/255)-23;
+
          }
+
          if( nPayload<=X ){
+
            nLocal = nPayload;
+
          }else{
+
            int M, K;
+
            M = ((U-12)*32/255)-23;
+
            K = M+((nPayload-M)%(U-4));
+
            if( K<=X ){
+
              nLocal = K;
+
            }else{
+
              nLocal = M;
+
            }
+
          }

-
  nByte += sizeof(IdxColumn) * nCol;
-
  if( rc==SQLITE_OK ){
-
    pNew = idxMalloc(&rc, nByte);
-
  }
-
  if( rc==SQLITE_OK ){
-
    pNew->aCol = (IdxColumn*)&pNew[1];
-
    pNew->nCol = nCol;
-
    pCsr = (char*)&pNew->aCol[nCol];
-
  }
+
          if( bNextPage || nLocal+iOff>pCsr->nPage ){
+
            bNextPage = 1;
+
          }else{

-
  nCol = 0;
-
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){
-
    const char *zCol = (const char*)sqlite3_column_text(p1, 1);
-
    const char *zColSeq = 0;
-
    int nCopy;
-
    if( zCol==0 ) continue;
-
    nCopy = STRLEN(zCol) + 1;
-
    pNew->aCol[nCol].zName = pCsr;
-
    pNew->aCol[nCol].iPk = (sqlite3_column_int(p1, 5)==1 && nPk==1);
-
    memcpy(pCsr, zCol, nCopy);
-
    pCsr += nCopy;
+
            /* Allocate space for payload. And a bit more to catch small buffer
+
            ** overruns caused by attempting to read a varint or similar from 
+
            ** near the end of a corrupt record.  */
+
            pCsr->pRec = (u8*)sqlite3_malloc64(nPayload+DBDATA_PADDING_BYTES);
+
            if( pCsr->pRec==0 ) return SQLITE_NOMEM;
+
            memset(pCsr->pRec, 0, nPayload+DBDATA_PADDING_BYTES);
+
            pCsr->nRec = nPayload;

-
    rc = sqlite3_table_column_metadata(
-
        db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0
-
    );
-
    if( rc==SQLITE_OK ){
-
      if( zColSeq==0 ) zColSeq = "binary";
-
      nCopy = STRLEN(zColSeq) + 1;
-
      pNew->aCol[nCol].zColl = pCsr;
-
      memcpy(pCsr, zColSeq, nCopy);
-
      pCsr += nCopy;
-
    }
+
            /* Load the nLocal bytes of payload */
+
            memcpy(pCsr->pRec, &pCsr->aPage[iOff], nLocal);
+
            iOff += nLocal;

-
    nCol++;
-
  }
-
  idxFinalize(&rc, p1);
+
            /* Load content from overflow pages */
+
            if( nPayload>nLocal ){
+
              sqlite3_int64 nRem = nPayload - nLocal;
+
              u32 pgnoOvfl = get_uint32(&pCsr->aPage[iOff]);
+
              while( nRem>0 ){
+
                u8 *aOvfl = 0;
+
                int nOvfl = 0;
+
                int nCopy;
+
                rc = dbdataLoadPage(pCsr, pgnoOvfl, &aOvfl, &nOvfl);
+
                assert( rc!=SQLITE_OK || aOvfl==0 || nOvfl==pCsr->nPage );
+
                if( rc!=SQLITE_OK ) return rc;
+
                if( aOvfl==0 ) break;

-
  if( rc!=SQLITE_OK ){
-
    sqlite3_free(pNew);
-
    pNew = 0;
-
  }else if( ALWAYS(pNew!=0) ){
-
    pNew->zName = pCsr;
-
    if( ALWAYS(pNew->zName!=0) ) memcpy(pNew->zName, zTab, nTab+1);
-
  }
+
                nCopy = U-4;
+
                if( nCopy>nRem ) nCopy = nRem;
+
                memcpy(&pCsr->pRec[nPayload-nRem], &aOvfl[4], nCopy);
+
                nRem -= nCopy;

-
  *ppOut = pNew;
-
  return rc;
-
}
+
                pgnoOvfl = get_uint32(aOvfl);
+
                sqlite3_free(aOvfl);
+
              }
+
            }
+
    
+
            iHdr = dbdataGetVarintU32(pCsr->pRec, &nHdr);
+
            if( nHdr>nPayload ) nHdr = 0;
+
            pCsr->nHdr = nHdr;
+
            pCsr->pHdrPtr = &pCsr->pRec[iHdr];
+
            pCsr->pPtr = &pCsr->pRec[pCsr->nHdr];
+
            pCsr->iField = (bHasRowid ? -1 : 0);
+
          }
+
        }
+
      }else{
+
        pCsr->iField++;
+
        if( pCsr->iField>0 ){
+
          sqlite3_int64 iType;
+
          if( pCsr->pHdrPtr>&pCsr->pRec[pCsr->nRec] ){
+
            bNextPage = 1;
+
          }else{
+
            pCsr->pHdrPtr += dbdataGetVarintU32(pCsr->pHdrPtr, &iType);
+
            pCsr->pPtr += dbdataValueBytes(iType);
+
          }
+
        }
+
      }

-
/*
-
** This function is a no-op if *pRc is set to anything other than 
-
** SQLITE_OK when it is called.
-
**
-
** If *pRc is initially set to SQLITE_OK, then the text specified by
-
** the printf() style arguments is appended to zIn and the result returned
-
** in a buffer allocated by sqlite3_malloc(). sqlite3_free() is called on
-
** zIn before returning.
-
*/
-
static char *idxAppendText(int *pRc, char *zIn, const char *zFmt, ...){
-
  va_list ap;
-
  char *zAppend = 0;
-
  char *zRet = 0;
-
  int nIn = zIn ? STRLEN(zIn) : 0;
-
  int nAppend = 0;
-
  va_start(ap, zFmt);
-
  if( *pRc==SQLITE_OK ){
-
    zAppend = sqlite3_vmprintf(zFmt, ap);
-
    if( zAppend ){
-
      nAppend = STRLEN(zAppend);
-
      zRet = (char*)sqlite3_malloc(nIn + nAppend + 1);
-
    }
-
    if( zAppend && zRet ){
-
      if( nIn ) memcpy(zRet, zIn, nIn);
-
      memcpy(&zRet[nIn], zAppend, nAppend+1);
-
    }else{
-
      sqlite3_free(zRet);
-
      zRet = 0;
-
      *pRc = SQLITE_NOMEM;
-
    }
-
    sqlite3_free(zAppend);
-
    sqlite3_free(zIn);
-
  }
-
  va_end(ap);
-
  return zRet;
-
}
+
      if( bNextPage ){
+
        sqlite3_free(pCsr->aPage);
+
        sqlite3_free(pCsr->pRec);
+
        pCsr->aPage = 0;
+
        pCsr->pRec = 0;
+
        if( pCsr->bOnePage ) return SQLITE_OK;
+
        pCsr->iPgno++;
+
      }else{
+
        if( pCsr->iField<0 || pCsr->pHdrPtr<&pCsr->pRec[pCsr->nHdr] ){
+
          return SQLITE_OK;
+
        }

-
/*
-
** Return true if zId must be quoted in order to use it as an SQL
-
** identifier, or false otherwise.
-
*/
-
static int idxIdentifierRequiresQuotes(const char *zId){
-
  int i;
-
  for(i=0; zId[i]; i++){
-
    if( !(zId[i]=='_')
-
     && !(zId[i]>='0' && zId[i]<='9')
-
     && !(zId[i]>='a' && zId[i]<='z')
-
     && !(zId[i]>='A' && zId[i]<='Z')
-
    ){
-
      return 1;
+
        /* Advance to the next cell. The next iteration of the loop will load
+
        ** the record and so on. */
+
        sqlite3_free(pCsr->pRec);
+
        pCsr->pRec = 0;
+
        pCsr->iCell++;
+
      }
    }
  }
-
  return 0;
+

+
  assert( !"can't get here" );
+
  return SQLITE_OK;
}

-
/*
-
** This function appends an index column definition suitable for constraint
-
** pCons to the string passed as zIn and returns the result.
+
/* 
+
** Return true if the cursor is at EOF.
*/
-
static char *idxAppendColDefn(
-
  int *pRc,                       /* IN/OUT: Error code */
-
  char *zIn,                      /* Column defn accumulated so far */
-
  IdxTable *pTab,                 /* Table index will be created on */
-
  IdxConstraint *pCons
-
){
-
  char *zRet = zIn;
-
  IdxColumn *p = &pTab->aCol[pCons->iCol];
-
  if( zRet ) zRet = idxAppendText(pRc, zRet, ", ");
+
static int dbdataEof(sqlite3_vtab_cursor *pCursor){
+
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
+
  return pCsr->aPage==0;
+
}

-
  if( idxIdentifierRequiresQuotes(p->zName) ){
-
    zRet = idxAppendText(pRc, zRet, "%Q", p->zName);
-
  }else{
-
    zRet = idxAppendText(pRc, zRet, "%s", p->zName);
+
/*
+
** Return true if nul-terminated string zSchema ends in "()". Or false
+
** otherwise.
+
*/
+
static int dbdataIsFunction(const char *zSchema){
+
  size_t n = strlen(zSchema);
+
  if( n>2 && zSchema[n-2]=='(' && zSchema[n-1]==')' ){
+
    return (int)n-2;
  }
+
  return 0;
+
}

-
  if( sqlite3_stricmp(p->zColl, pCons->zColl) ){
-
    if( idxIdentifierRequiresQuotes(pCons->zColl) ){
-
      zRet = idxAppendText(pRc, zRet, " COLLATE %Q", pCons->zColl);
-
    }else{
-
      zRet = idxAppendText(pRc, zRet, " COLLATE %s", pCons->zColl);
-
    }
+
/* 
+
** Determine the size in pages of database zSchema (where zSchema is
+
** "main", "temp" or the name of an attached database) and set 
+
** pCsr->szDb accordingly. If successful, return SQLITE_OK. Otherwise,
+
** an SQLite error code.
+
*/
+
static int dbdataDbsize(DbdataCursor *pCsr, const char *zSchema){
+
  DbdataTable *pTab = (DbdataTable*)pCsr->base.pVtab;
+
  char *zSql = 0;
+
  int rc, rc2;
+
  int nFunc = 0;
+
  sqlite3_stmt *pStmt = 0;
+

+
  if( (nFunc = dbdataIsFunction(zSchema))>0 ){
+
    zSql = sqlite3_mprintf("SELECT %.*s(0)", nFunc, zSchema);
+
  }else{
+
    zSql = sqlite3_mprintf("PRAGMA %Q.page_count", zSchema);
  }
+
  if( zSql==0 ) return SQLITE_NOMEM;

-
  if( pCons->bDesc ){
-
    zRet = idxAppendText(pRc, zRet, " DESC");
+
  rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pStmt, 0);
+
  sqlite3_free(zSql);
+
  if( rc==SQLITE_OK && sqlite3_step(pStmt)==SQLITE_ROW ){
+
    pCsr->szDb = sqlite3_column_int(pStmt, 0);
  }
-
  return zRet;
+
  rc2 = sqlite3_finalize(pStmt);
+
  if( rc==SQLITE_OK ) rc = rc2;
+
  return rc;
}

/*
-
** Search database dbm for an index compatible with the one idxCreateFromCons()
-
** would create from arguments pScan, pEq and pTail. If no error occurs and 
-
** such an index is found, return non-zero. Or, if no such index is found,
-
** return zero.
-
**
-
** If an error occurs, set *pRc to an SQLite error code and return zero.
+
** Attempt to figure out the encoding of the database by retrieving page 1
+
** and inspecting the header field. If successful, set the pCsr->enc variable
+
** and return SQLITE_OK. Otherwise, return an SQLite error code.
*/
-
static int idxFindCompatible(
-
  int *pRc,                       /* OUT: Error code */
-
  sqlite3* dbm,                   /* Database to search */
-
  IdxScan *pScan,                 /* Scan for table to search for index on */
-
  IdxConstraint *pEq,             /* List of == constraints */
-
  IdxConstraint *pTail            /* List of range constraints */
-
){
-
  const char *zTbl = pScan->pTab->zName;
-
  sqlite3_stmt *pIdxList = 0;
-
  IdxConstraint *pIter;
-
  int nEq = 0;                    /* Number of elements in pEq */
-
  int rc;
-

-
  /* Count the elements in list pEq */
-
  for(pIter=pEq; pIter; pIter=pIter->pLink) nEq++;
+
static int dbdataGetEncoding(DbdataCursor *pCsr){
+
  int rc = SQLITE_OK;
+
  int nPg1 = 0;
+
  u8 *aPg1 = 0;
+
  rc = dbdataLoadPage(pCsr, 1, &aPg1, &nPg1);
+
  assert( rc!=SQLITE_OK || nPg1==0 || nPg1>=512 );
+
  if( rc==SQLITE_OK && nPg1>0 ){
+
    pCsr->enc = get_uint32(&aPg1[56]);
+
  }
+
  sqlite3_free(aPg1);
+
  return rc;
+
}

-
  rc = idxPrintfPrepareStmt(dbm, &pIdxList, 0, "PRAGMA index_list=%Q", zTbl);
-
  while( rc==SQLITE_OK && sqlite3_step(pIdxList)==SQLITE_ROW ){
-
    int bMatch = 1;
-
    IdxConstraint *pT = pTail;
-
    sqlite3_stmt *pInfo = 0;
-
    const char *zIdx = (const char*)sqlite3_column_text(pIdxList, 1);
-
    if( zIdx==0 ) continue;

-
    /* Zero the IdxConstraint.bFlag values in the pEq list */
-
    for(pIter=pEq; pIter; pIter=pIter->pLink) pIter->bFlag = 0;
+
/* 
+
** xFilter method for sqlite_dbdata and sqlite_dbptr.
+
*/
+
static int dbdataFilter(
+
  sqlite3_vtab_cursor *pCursor, 
+
  int idxNum, const char *idxStr,
+
  int argc, sqlite3_value **argv
+
){
+
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
+
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
+
  int rc = SQLITE_OK;
+
  const char *zSchema = "main";

-
    rc = idxPrintfPrepareStmt(dbm, &pInfo, 0, "PRAGMA index_xInfo=%Q", zIdx);
-
    while( rc==SQLITE_OK && sqlite3_step(pInfo)==SQLITE_ROW ){
-
      int iIdx = sqlite3_column_int(pInfo, 0);
-
      int iCol = sqlite3_column_int(pInfo, 1);
-
      const char *zColl = (const char*)sqlite3_column_text(pInfo, 4);
+
  dbdataResetCursor(pCsr);
+
  assert( pCsr->iPgno==1 );
+
  if( idxNum & 0x01 ){
+
    zSchema = (const char*)sqlite3_value_text(argv[0]);
+
    if( zSchema==0 ) zSchema = "";
+
  }
+
  if( idxNum & 0x02 ){
+
    pCsr->iPgno = sqlite3_value_int(argv[(idxNum & 0x01)]);
+
    pCsr->bOnePage = 1;
+
  }else{
+
    rc = dbdataDbsize(pCsr, zSchema);
+
  }

-
      if( iIdx<nEq ){
-
        for(pIter=pEq; pIter; pIter=pIter->pLink){
-
          if( pIter->bFlag ) continue;
-
          if( pIter->iCol!=iCol ) continue;
-
          if( sqlite3_stricmp(pIter->zColl, zColl) ) continue;
-
          pIter->bFlag = 1;
-
          break;
-
        }
-
        if( pIter==0 ){
-
          bMatch = 0;
-
          break;
-
        }
+
  if( rc==SQLITE_OK ){
+
    int nFunc = 0;
+
    if( pTab->pStmt ){
+
      pCsr->pStmt = pTab->pStmt;
+
      pTab->pStmt = 0;
+
    }else if( (nFunc = dbdataIsFunction(zSchema))>0 ){
+
      char *zSql = sqlite3_mprintf("SELECT %.*s(?2)", nFunc, zSchema);
+
      if( zSql==0 ){
+
        rc = SQLITE_NOMEM;
      }else{
-
        if( pT ){
-
          if( pT->iCol!=iCol || sqlite3_stricmp(pT->zColl, zColl) ){
-
            bMatch = 0;
-
            break;
-
          }
-
          pT = pT->pLink;
-
        }
+
        rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pCsr->pStmt, 0);
+
        sqlite3_free(zSql);
      }
-
    }
-
    idxFinalize(&rc, pInfo);
-

-
    if( rc==SQLITE_OK && bMatch ){
-
      sqlite3_finalize(pIdxList);
-
      return 1;
+
    }else{
+
      rc = sqlite3_prepare_v2(pTab->db, 
+
          "SELECT data FROM sqlite_dbpage(?) WHERE pgno=?", -1,
+
          &pCsr->pStmt, 0
+
      );
    }
  }
-
  idxFinalize(&rc, pIdxList);
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_bind_text(pCsr->pStmt, 1, zSchema, -1, SQLITE_TRANSIENT);
+
  }else{
+
    pTab->base.zErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pTab->db));
+
  }

-
  *pRc = rc;
-
  return 0;
-
}
+
  /* Try to determine the encoding of the db by inspecting the header
+
  ** field on page 1. */
+
  if( rc==SQLITE_OK ){
+
    rc = dbdataGetEncoding(pCsr);
+
  }

-
/* Callback for sqlite3_exec() with query with leading count(*) column.
-
 * The first argument is expected to be an int*, referent to be incremented
-
 * if that leading column is not exactly '0'.
-
 */
-
static int countNonzeros(void* pCount, int nc,
-
                         char* azResults[], char* azColumns[]){
-
  (void)azColumns;  /* Suppress unused parameter warning */
-
  if( nc>0 && (azResults[0][0]!='0' || azResults[0][1]!=0) ){
-
    *((int *)pCount) += 1;
+
  if( rc==SQLITE_OK ){
+
    rc = dbdataNext(pCursor);
  }
-
  return 0;
+
  return rc;
}

-
static int idxCreateFromCons(
-
  sqlite3expert *p,
-
  IdxScan *pScan,
-
  IdxConstraint *pEq, 
-
  IdxConstraint *pTail
+
/*
+
** Return a column for the sqlite_dbdata or sqlite_dbptr table.
+
*/
+
static int dbdataColumn(
+
  sqlite3_vtab_cursor *pCursor, 
+
  sqlite3_context *ctx, 
+
  int i
){
-
  sqlite3 *dbm = p->dbm;
-
  int rc = SQLITE_OK;
-
  if( (pEq || pTail) && 0==idxFindCompatible(&rc, dbm, pScan, pEq, pTail) ){
-
    IdxTable *pTab = pScan->pTab;
-
    char *zCols = 0;
-
    char *zIdx = 0;
-
    IdxConstraint *pCons;
-
    unsigned int h = 0;
-
    const char *zFmt;
-

-
    for(pCons=pEq; pCons; pCons=pCons->pLink){
-
      zCols = idxAppendColDefn(&rc, zCols, pTab, pCons);
-
    }
-
    for(pCons=pTail; pCons; pCons=pCons->pLink){
-
      zCols = idxAppendColDefn(&rc, zCols, pTab, pCons);
-
    }
-

-
    if( rc==SQLITE_OK ){
-
      /* Hash the list of columns to come up with a name for the index */
-
      const char *zTable = pScan->pTab->zName;
-
      int quoteTable = idxIdentifierRequiresQuotes(zTable);
-
      char *zName = 0;          /* Index name */
-
      int collisions = 0;
-
      do{
-
        int i;
-
        char *zFind;
-
        for(i=0; zCols[i]; i++){
-
          h += ((h<<3) + zCols[i]);
-
        }
-
        sqlite3_free(zName);
-
        zName = sqlite3_mprintf("%s_idx_%08x", zTable, h);
-
        if( zName==0 ) break;
-
        /* Is is unique among table, view and index names? */
-
        zFmt = "SELECT count(*) FROM sqlite_schema WHERE name=%Q"
-
          " AND type in ('index','table','view')";
-
        zFind = sqlite3_mprintf(zFmt, zName);
-
        i = 0;
-
        rc = sqlite3_exec(dbm, zFind, countNonzeros, &i, 0);
-
        assert(rc==SQLITE_OK);
-
        sqlite3_free(zFind);
-
        if( i==0 ){
-
          collisions = 0;
-
          break;
-
        }
-
        ++collisions;
-
      }while( collisions<50 && zName!=0 );
-
      if( collisions ){
-
        /* This return means "Gave up trying to find a unique index name." */
-
        rc = SQLITE_BUSY_TIMEOUT;
-
      }else if( zName==0 ){
-
        rc = SQLITE_NOMEM;
-
      }else{
-
        if( quoteTable ){
-
          zFmt = "CREATE INDEX \"%w\" ON \"%w\"(%s)";
+
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
+
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
+
  if( pTab->bPtr ){
+
    switch( i ){
+
      case DBPTR_COLUMN_PGNO:
+
        sqlite3_result_int64(ctx, pCsr->iPgno);
+
        break;
+
      case DBPTR_COLUMN_CHILD: {
+
        int iOff = pCsr->iPgno==1 ? 100 : 0;
+
        if( pCsr->iCell<0 ){
+
          iOff += 8;
        }else{
-
          zFmt = "CREATE INDEX %s ON %s(%s)";
+
          iOff += 12 + pCsr->iCell*2;
+
          if( iOff>pCsr->nPage ) return SQLITE_OK;
+
          iOff = get_uint16(&pCsr->aPage[iOff]);
        }
-
        zIdx = sqlite3_mprintf(zFmt, zName, zTable, zCols);
-
        if( !zIdx ){
-
          rc = SQLITE_NOMEM;
-
        }else{
-
          rc = sqlite3_exec(dbm, zIdx, 0, 0, p->pzErrmsg);
-
          if( rc!=SQLITE_OK ){
-
            rc = SQLITE_BUSY_TIMEOUT;
-
          }else{
-
            idxHashAdd(&rc, &p->hIdx, zName, zIdx);
-
          }
+
        if( iOff<=pCsr->nPage ){
+
          sqlite3_result_int64(ctx, get_uint32(&pCsr->aPage[iOff]));
+
        }
+
        break;
+
      }
+
    }
+
  }else{
+
    switch( i ){
+
      case DBDATA_COLUMN_PGNO:
+
        sqlite3_result_int64(ctx, pCsr->iPgno);
+
        break;
+
      case DBDATA_COLUMN_CELL:
+
        sqlite3_result_int(ctx, pCsr->iCell);
+
        break;
+
      case DBDATA_COLUMN_FIELD:
+
        sqlite3_result_int(ctx, pCsr->iField);
+
        break;
+
      case DBDATA_COLUMN_VALUE: {
+
        if( pCsr->iField<0 ){
+
          sqlite3_result_int64(ctx, pCsr->iIntkey);
+
        }else if( &pCsr->pRec[pCsr->nRec] >= pCsr->pPtr ){
+
          sqlite3_int64 iType;
+
          dbdataGetVarintU32(pCsr->pHdrPtr, &iType);
+
          dbdataValue(
+
              ctx, pCsr->enc, iType, pCsr->pPtr, 
+
              &pCsr->pRec[pCsr->nRec] - pCsr->pPtr
+
          );
        }
-
        sqlite3_free(zName);
-
        sqlite3_free(zIdx);
+
        break;
      }
    }
-

-
    sqlite3_free(zCols);
  }
-
  return rc;
+
  return SQLITE_OK;
+
}
+

+
/* 
+
** Return the rowid for an sqlite_dbdata or sqlite_dptr table.
+
*/
+
static int dbdataRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
+
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
+
  *pRowid = pCsr->iRowid;
+
  return SQLITE_OK;
}

+

/*
-
** Return true if list pList (linked by IdxConstraint.pLink) contains
-
** a constraint compatible with *p. Otherwise return false.
+
** Invoke this routine to register the "sqlite_dbdata" virtual table module
*/
-
static int idxFindConstraint(IdxConstraint *pList, IdxConstraint *p){
-
  IdxConstraint *pCmp;
-
  for(pCmp=pList; pCmp; pCmp=pCmp->pLink){
-
    if( p->iCol==pCmp->iCol ) return 1;
+
static int sqlite3DbdataRegister(sqlite3 *db){
+
  static sqlite3_module dbdata_module = {
+
    0,                            /* iVersion */
+
    0,                            /* xCreate */
+
    dbdataConnect,                /* xConnect */
+
    dbdataBestIndex,              /* xBestIndex */
+
    dbdataDisconnect,             /* xDisconnect */
+
    0,                            /* xDestroy */
+
    dbdataOpen,                   /* xOpen - open a cursor */
+
    dbdataClose,                  /* xClose - close a cursor */
+
    dbdataFilter,                 /* xFilter - configure scan constraints */
+
    dbdataNext,                   /* xNext - advance a cursor */
+
    dbdataEof,                    /* xEof - check for end of scan */
+
    dbdataColumn,                 /* xColumn - read data */
+
    dbdataRowid,                  /* xRowid - read data */
+
    0,                            /* xUpdate */
+
    0,                            /* xBegin */
+
    0,                            /* xSync */
+
    0,                            /* xCommit */
+
    0,                            /* xRollback */
+
    0,                            /* xFindMethod */
+
    0,                            /* xRename */
+
    0,                            /* xSavepoint */
+
    0,                            /* xRelease */
+
    0,                            /* xRollbackTo */
+
    0                             /* xShadowName */
+
  };
+

+
  int rc = sqlite3_create_module(db, "sqlite_dbdata", &dbdata_module, 0);
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_create_module(db, "sqlite_dbptr", &dbdata_module, (void*)1);
  }
-
  return 0;
+
  return rc;
}

-
static int idxCreateFromWhere(
-
  sqlite3expert *p, 
-
  IdxScan *pScan,                 /* Create indexes for this scan */
-
  IdxConstraint *pTail            /* range/ORDER BY constraints for inclusion */
+
#ifdef _WIN32
+

+
#endif
+
int sqlite3_dbdata_init(
+
  sqlite3 *db, 
+
  char **pzErrMsg, 
+
  const sqlite3_api_routines *pApi
){
-
  IdxConstraint *p1 = 0;
-
  IdxConstraint *pCon;
-
  int rc;
+
  SQLITE_EXTENSION_INIT2(pApi);
+
  return sqlite3DbdataRegister(db);
+
}

-
  /* Gather up all the == constraints. */
-
  for(pCon=pScan->pEq; pCon; pCon=pCon->pNext){
-
    if( !idxFindConstraint(p1, pCon) && !idxFindConstraint(pTail, pCon) ){
-
      pCon->pLink = p1;
-
      p1 = pCon;
-
    }
-
  }
+
#endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */

-
  /* Create an index using the == constraints collected above. And the
-
  ** range constraint/ORDER BY terms passed in by the caller, if any. */
-
  rc = idxCreateFromCons(p, pScan, p1, pTail);
+
/************************* End ../ext/recover/dbdata.c ********************/
+
/************************* Begin ../ext/recover/sqlite3recover.h ******************/
+
/*
+
** 2022-08-27
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
*************************************************************************
+
**
+
** This file contains the public interface to the "recover" extension -
+
** an SQLite extension designed to recover data from corrupted database
+
** files.
+
*/

-
  /* If no range/ORDER BY passed by the caller, create a version of the
-
  ** index for each range constraint.  */
-
  if( pTail==0 ){
-
    for(pCon=pScan->pRange; rc==SQLITE_OK && pCon; pCon=pCon->pNext){
-
      assert( pCon->pLink==0 );
-
      if( !idxFindConstraint(p1, pCon) && !idxFindConstraint(pTail, pCon) ){
-
        rc = idxCreateFromCons(p, pScan, p1, pCon);
-
      }
-
    }
-
  }
+
/*
+
** OVERVIEW:
+
**
+
** To use the API to recover data from a corrupted database, an
+
** application:
+
**
+
**   1) Creates an sqlite3_recover handle by calling either
+
**      sqlite3_recover_init() or sqlite3_recover_init_sql().
+
**
+
**   2) Configures the new handle using one or more calls to
+
**      sqlite3_recover_config().
+
**
+
**   3) Executes the recovery by repeatedly calling sqlite3_recover_step() on
+
**      the handle until it returns something other than SQLITE_OK. If it
+
**      returns SQLITE_DONE, then the recovery operation completed without 
+
**      error. If it returns some other non-SQLITE_OK value, then an error 
+
**      has occurred.
+
**
+
**   4) Retrieves any error code and English language error message using the
+
**      sqlite3_recover_errcode() and sqlite3_recover_errmsg() APIs,
+
**      respectively.
+
**
+
**   5) Destroys the sqlite3_recover handle and frees all resources
+
**      using sqlite3_recover_finish().
+
**
+
** The application may abandon the recovery operation at any point 
+
** before it is finished by passing the sqlite3_recover handle to
+
** sqlite3_recover_finish(). This is not an error, but the final state
+
** of the output database, or the results of running the partial script
+
** delivered to the SQL callback, are undefined.
+
*/

-
  return rc;
-
}
+
#ifndef _SQLITE_RECOVER_H
+
#define _SQLITE_RECOVER_H
+

+
/* #include "sqlite3.h" */
+

+
#ifdef __cplusplus
+
extern "C" {
+
#endif

/*
-
** Create candidate indexes in database [dbm] based on the data in 
-
** linked-list pScan.
+
** An instance of the sqlite3_recover object represents a recovery
+
** operation in progress.
+
**
+
** Constructors:
+
**
+
**    sqlite3_recover_init()
+
**    sqlite3_recover_init_sql()
+
**
+
** Destructor:
+
**
+
**    sqlite3_recover_finish()
+
**
+
** Methods:
+
**
+
**    sqlite3_recover_config()
+
**    sqlite3_recover_errcode()
+
**    sqlite3_recover_errmsg()
+
**    sqlite3_recover_run()
+
**    sqlite3_recover_step()
*/
-
static int idxCreateCandidates(sqlite3expert *p){
-
  int rc = SQLITE_OK;
-
  IdxScan *pIter;
+
typedef struct sqlite3_recover sqlite3_recover;

-
  for(pIter=p->pScan; pIter && rc==SQLITE_OK; pIter=pIter->pNextScan){
-
    rc = idxCreateFromWhere(p, pIter, 0);
-
    if( rc==SQLITE_OK && pIter->pOrder ){
-
      rc = idxCreateFromWhere(p, pIter, pIter->pOrder);
-
    }
-
  }
+
/* 
+
** These two APIs attempt to create and return a new sqlite3_recover object.
+
** In both cases the first two arguments identify the (possibly
+
** corrupt) database to recover data from. The first argument is an open
+
** database handle and the second the name of a database attached to that
+
** handle (i.e. "main", "temp" or the name of an attached database).
+
**
+
** If sqlite3_recover_init() is used to create the new sqlite3_recover
+
** handle, then data is recovered into a new database, identified by
+
** string parameter zUri. zUri may be an absolute or relative file path,
+
** or may be an SQLite URI. If the identified database file already exists,
+
** it is overwritten.
+
**
+
** If sqlite3_recover_init_sql() is invoked, then any recovered data will
+
** be returned to the user as a series of SQL statements. Executing these
+
** SQL statements results in the same database as would have been created
+
** had sqlite3_recover_init() been used. For each SQL statement in the
+
** output, the callback function passed as the third argument (xSql) is 
+
** invoked once. The first parameter is a passed a copy of the fourth argument
+
** to this function (pCtx) as its first parameter, and a pointer to a
+
** nul-terminated buffer containing the SQL statement formated as UTF-8 as 
+
** the second. If the xSql callback returns any value other than SQLITE_OK,
+
** then processing is immediately abandoned and the value returned used as
+
** the recover handle error code (see below).
+
**
+
** If an out-of-memory error occurs, NULL may be returned instead of
+
** a valid handle. In all other cases, it is the responsibility of the
+
** application to avoid resource leaks by ensuring that
+
** sqlite3_recover_finish() is called on all allocated handles.
+
*/
+
sqlite3_recover *sqlite3_recover_init(
+
  sqlite3* db, 
+
  const char *zDb, 
+
  const char *zUri
+
);
+
sqlite3_recover *sqlite3_recover_init_sql(
+
  sqlite3* db, 
+
  const char *zDb, 
+
  int (*xSql)(void*, const char*),
+
  void *pCtx
+
);

-
  return rc;
-
}
+
/*
+
** Configure an sqlite3_recover object that has just been created using
+
** sqlite3_recover_init() or sqlite3_recover_init_sql(). This function
+
** may only be called before the first call to sqlite3_recover_step()
+
** or sqlite3_recover_run() on the object.
+
**
+
** The second argument passed to this function must be one of the
+
** SQLITE_RECOVER_* symbols defined below. Valid values for the third argument
+
** depend on the specific SQLITE_RECOVER_* symbol in use.
+
**
+
** SQLITE_OK is returned if the configuration operation was successful,
+
** or an SQLite error code otherwise.
+
*/
+
int sqlite3_recover_config(sqlite3_recover*, int op, void *pArg);
+

+
/*
+
** SQLITE_RECOVER_LOST_AND_FOUND:
+
**   The pArg argument points to a string buffer containing the name
+
**   of a "lost-and-found" table in the output database, or NULL. If
+
**   the argument is non-NULL and the database contains seemingly
+
**   valid pages that cannot be associated with any table in the
+
**   recovered part of the schema, data is extracted from these
+
**   pages to add to the lost-and-found table.
+
**
+
** SQLITE_RECOVER_FREELIST_CORRUPT:
+
**   The pArg value must actually be a pointer to a value of type
+
**   int containing value 0 or 1 cast as a (void*). If this option is set
+
**   (argument is 1) and a lost-and-found table has been configured using
+
**   SQLITE_RECOVER_LOST_AND_FOUND, then is assumed that the freelist is 
+
**   corrupt and an attempt is made to recover records from pages that
+
**   appear to be linked into the freelist. Otherwise, pages on the freelist
+
**   are ignored. Setting this option can recover more data from the
+
**   database, but often ends up "recovering" deleted records. The default 
+
**   value is 0 (clear).
+
**
+
** SQLITE_RECOVER_ROWIDS:
+
**   The pArg value must actually be a pointer to a value of type
+
**   int containing value 0 or 1 cast as a (void*). If this option is set
+
**   (argument is 1), then an attempt is made to recover rowid values
+
**   that are not also INTEGER PRIMARY KEY values. If this option is
+
**   clear, then new rowids are assigned to all recovered rows. The
+
**   default value is 1 (set).
+
**
+
** SQLITE_RECOVER_SLOWINDEXES:
+
**   The pArg value must actually be a pointer to a value of type
+
**   int containing value 0 or 1 cast as a (void*). If this option is clear
+
**   (argument is 0), then when creating an output database, the recover 
+
**   module creates and populates non-UNIQUE indexes right at the end of the
+
**   recovery operation - after all recoverable data has been inserted
+
**   into the new database. This is faster overall, but means that the
+
**   final call to sqlite3_recover_step() for a recovery operation may
+
**   be need to create a large number of indexes, which may be very slow.
+
**
+
**   Or, if this option is set (argument is 1), then non-UNIQUE indexes
+
**   are created in the output database before it is populated with 
+
**   recovered data. This is slower overall, but avoids the slow call
+
**   to sqlite3_recover_step() at the end of the recovery operation.
+
**
+
**   The default option value is 0.
+
*/
+
#define SQLITE_RECOVER_LOST_AND_FOUND   1
+
#define SQLITE_RECOVER_FREELIST_CORRUPT 2
+
#define SQLITE_RECOVER_ROWIDS           3
+
#define SQLITE_RECOVER_SLOWINDEXES      4
+

+
/*
+
** Perform a unit of work towards the recovery operation. This function 
+
** must normally be called multiple times to complete database recovery.
+
**
+
** If no error occurs but the recovery operation is not completed, this
+
** function returns SQLITE_OK. If recovery has been completed successfully
+
** then SQLITE_DONE is returned. If an error has occurred, then an SQLite
+
** error code (e.g. SQLITE_IOERR or SQLITE_NOMEM) is returned. It is not
+
** considered an error if some or all of the data cannot be recovered
+
** due to database corruption.
+
**
+
** Once sqlite3_recover_step() has returned a value other than SQLITE_OK,
+
** all further such calls on the same recover handle are no-ops that return
+
** the same non-SQLITE_OK value.
+
*/
+
int sqlite3_recover_step(sqlite3_recover*);
+

+
/* 
+
** Run the recovery operation to completion. Return SQLITE_OK if successful,
+
** or an SQLite error code otherwise. Calling this function is the same
+
** as executing:
+
**
+
**     while( SQLITE_OK==sqlite3_recover_step(p) );
+
**     return sqlite3_recover_errcode(p);
+
*/
+
int sqlite3_recover_run(sqlite3_recover*);

/*
-
** Free all elements of the linked list starting at pConstraint.
+
** If an error has been encountered during a prior call to
+
** sqlite3_recover_step(), then this function attempts to return a 
+
** pointer to a buffer containing an English language explanation of 
+
** the error. If no error message is available, or if an out-of memory 
+
** error occurs while attempting to allocate a buffer in which to format
+
** the error message, NULL is returned.
+
**
+
** The returned buffer remains valid until the sqlite3_recover handle is
+
** destroyed using sqlite3_recover_finish().
*/
-
static void idxConstraintFree(IdxConstraint *pConstraint){
-
  IdxConstraint *pNext;
-
  IdxConstraint *p;
+
const char *sqlite3_recover_errmsg(sqlite3_recover*);
+

+
/*
+
** If this function is called on an sqlite3_recover handle after
+
** an error occurs, an SQLite error code is returned. Otherwise, SQLITE_OK.
+
*/
+
int sqlite3_recover_errcode(sqlite3_recover*);
+

+
/* 
+
** Clean up a recovery object created by a call to sqlite3_recover_init().
+
** The results of using a recovery object with any API after it has been
+
** passed to this function are undefined.
+
**
+
** This function returns the same value as sqlite3_recover_errcode().
+
*/
+
int sqlite3_recover_finish(sqlite3_recover*);

-
  for(p=pConstraint; p; p=pNext){
-
    pNext = p->pNext;
-
    sqlite3_free(p);
-
  }
-
}

+
#ifdef __cplusplus
+
}  /* end of the 'extern "C"' block */
+
#endif
+

+
#endif /* ifndef _SQLITE_RECOVER_H */
+

+
/************************* End ../ext/recover/sqlite3recover.h ********************/
+
/************************* Begin ../ext/recover/sqlite3recover.c ******************/
/*
-
** Free all elements of the linked list starting from pScan up until pLast
-
** (pLast is not freed).
+
** 2022-08-27
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
*************************************************************************
+
**
*/
-
static void idxScanFree(IdxScan *pScan, IdxScan *pLast){
-
  IdxScan *p;
-
  IdxScan *pNext;
-
  for(p=pScan; p!=pLast; p=pNext){
-
    pNext = p->pNextScan;
-
    idxConstraintFree(p->pOrder);
-
    idxConstraintFree(p->pEq);
-
    idxConstraintFree(p->pRange);
-
    sqlite3_free(p);
-
  }
-
}
+

+

+
/* #include "sqlite3recover.h" */
+
#include <assert.h>
+
#include <string.h>
+

+
#ifndef SQLITE_OMIT_VIRTUALTABLE

/*
-
** Free all elements of the linked list starting from pStatement up 
-
** until pLast (pLast is not freed).
+
** Declaration for public API function in file dbdata.c. This may be called
+
** with NULL as the final two arguments to register the sqlite_dbptr and
+
** sqlite_dbdata virtual tables with a database handle.
*/
-
static void idxStatementFree(IdxStatement *pStatement, IdxStatement *pLast){
-
  IdxStatement *p;
-
  IdxStatement *pNext;
-
  for(p=pStatement; p!=pLast; p=pNext){
-
    pNext = p->pNext;
-
    sqlite3_free(p->zEQP);
-
    sqlite3_free(p->zIdx);
-
    sqlite3_free(p);
-
  }
-
}
+
#ifdef _WIN32
+

+
#endif
+
int sqlite3_dbdata_init(sqlite3*, char**, const sqlite3_api_routines*);
+

+
/* typedef unsigned int u32; */
+
/* typedef unsigned char u8; */
+
/* typedef sqlite3_int64 i64; */
+

+
typedef struct RecoverTable RecoverTable;
+
typedef struct RecoverColumn RecoverColumn;

/*
-
** Free the linked list of IdxTable objects starting at pTab.
+
** When recovering rows of data that can be associated with table
+
** definitions recovered from the sqlite_schema table, each table is
+
** represented by an instance of the following object.
+
**
+
** iRoot:
+
**   The root page in the original database. Not necessarily (and usually
+
**   not) the same in the recovered database.
+
**
+
** zTab:
+
**   Name of the table.
+
**
+
** nCol/aCol[]:
+
**   aCol[] is an array of nCol columns. In the order in which they appear 
+
**   in the table.
+
**
+
** bIntkey:
+
**   Set to true for intkey tables, false for WITHOUT ROWID.
+
**
+
** iRowidBind:
+
**   Each column in the aCol[] array has associated with it the index of
+
**   the bind parameter its values will be bound to in the INSERT statement
+
**   used to construct the output database. If the table does has a rowid
+
**   but not an INTEGER PRIMARY KEY column, then iRowidBind contains the
+
**   index of the bind paramater to which the rowid value should be bound.
+
**   Otherwise, it contains -1. If the table does contain an INTEGER PRIMARY 
+
**   KEY column, then the rowid value should be bound to the index associated
+
**   with the column.
+
**
+
** pNext:
+
**   All RecoverTable objects used by the recovery operation are allocated
+
**   and populated as part of creating the recovered database schema in
+
**   the output database, before any non-schema data are recovered. They
+
**   are then stored in a singly-linked list linked by this variable beginning
+
**   at sqlite3_recover.pTblList.
*/
-
static void idxTableFree(IdxTable *pTab){
-
  IdxTable *pIter;
-
  IdxTable *pNext;
-
  for(pIter=pTab; pIter; pIter=pNext){
-
    pNext = pIter->pNext;
-
    sqlite3_free(pIter);
-
  }
-
}
+
struct RecoverTable {
+
  u32 iRoot;                      /* Root page in original database */
+
  char *zTab;                     /* Name of table */
+
  int nCol;                       /* Number of columns in table */
+
  RecoverColumn *aCol;            /* Array of columns */
+
  int bIntkey;                    /* True for intkey, false for without rowid */
+
  int iRowidBind;                 /* If >0, bind rowid to INSERT here */
+
  RecoverTable *pNext;
+
};
+

+
/*
+
** Each database column is represented by an instance of the following object
+
** stored in the RecoverTable.aCol[] array of the associated table.
+
**
+
** iField:
+
**   The index of the associated field within database records. Or -1 if
+
**   there is no associated field (e.g. for virtual generated columns).
+
**
+
** iBind:
+
**   The bind index of the INSERT statement to bind this columns values
+
**   to. Or 0 if there is no such index (iff (iField<0)).
+
**
+
** bIPK:
+
**   True if this is the INTEGER PRIMARY KEY column.
+
**
+
** zCol:
+
**   Name of column.
+
**
+
** eHidden:
+
**   A RECOVER_EHIDDEN_* constant value (see below for interpretation of each).
+
*/
+
struct RecoverColumn {
+
  int iField;                     /* Field in record on disk */
+
  int iBind;                      /* Binding to use in INSERT */
+
  int bIPK;                       /* True for IPK column */
+
  char *zCol;
+
  int eHidden;
+
};
+

+
#define RECOVER_EHIDDEN_NONE    0      /* Normal database column */
+
#define RECOVER_EHIDDEN_HIDDEN  1      /* Column is __HIDDEN__ */
+
#define RECOVER_EHIDDEN_VIRTUAL 2      /* Virtual generated column */
+
#define RECOVER_EHIDDEN_STORED  3      /* Stored generated column */

/*
-
** Free the linked list of IdxWrite objects starting at pTab.
+
** Bitmap object used to track pages in the input database. Allocated
+
** and manipulated only by the following functions:
+
**
+
**     recoverBitmapAlloc()
+
**     recoverBitmapFree()
+
**     recoverBitmapSet()
+
**     recoverBitmapQuery()
+
**
+
** nPg:
+
**   Largest page number that may be stored in the bitmap. The range
+
**   of valid keys is 1 to nPg, inclusive.
+
**
+
** aElem[]:
+
**   Array large enough to contain a bit for each key. For key value
+
**   iKey, the associated bit is the bit (iKey%32) of aElem[iKey/32].
+
**   In other words, the following is true if bit iKey is set, or 
+
**   false if it is clear:
+
**
+
**       (aElem[iKey/32] & (1 << (iKey%32))) ? 1 : 0
*/
-
static void idxWriteFree(IdxWrite *pTab){
-
  IdxWrite *pIter;
-
  IdxWrite *pNext;
-
  for(pIter=pTab; pIter; pIter=pNext){
-
    pNext = pIter->pNext;
-
    sqlite3_free(pIter);
-
  }
-
}
+
typedef struct RecoverBitmap RecoverBitmap;
+
struct RecoverBitmap {
+
  i64 nPg;                        /* Size of bitmap */
+
  u32 aElem[1];                   /* Array of 32-bit bitmasks */
+
};

+
/*
+
** State variables (part of the sqlite3_recover structure) used while
+
** recovering data for tables identified in the recovered schema (state
+
** RECOVER_STATE_WRITING).
+
*/
+
typedef struct RecoverStateW1 RecoverStateW1;
+
struct RecoverStateW1 {
+
  sqlite3_stmt *pTbls;
+
  sqlite3_stmt *pSel;
+
  sqlite3_stmt *pInsert;
+
  int nInsert;
+

+
  RecoverTable *pTab;             /* Table currently being written */
+
  int nMax;                       /* Max column count in any schema table */
+
  sqlite3_value **apVal;          /* Array of nMax values */
+
  int nVal;                       /* Number of valid entries in apVal[] */
+
  int bHaveRowid;
+
  i64 iRowid;
+
  i64 iPrevPage;
+
  int iPrevCell;
+
};

+
/*
+
** State variables (part of the sqlite3_recover structure) used while
+
** recovering data destined for the lost and found table (states
+
** RECOVER_STATE_LOSTANDFOUND[123]).
+
*/
+
typedef struct RecoverStateLAF RecoverStateLAF;
+
struct RecoverStateLAF {
+
  RecoverBitmap *pUsed;
+
  i64 nPg;                        /* Size of db in pages */
+
  sqlite3_stmt *pAllAndParent;
+
  sqlite3_stmt *pMapInsert;
+
  sqlite3_stmt *pMaxField;
+
  sqlite3_stmt *pUsedPages;
+
  sqlite3_stmt *pFindRoot;
+
  sqlite3_stmt *pInsert;          /* INSERT INTO lost_and_found ... */
+
  sqlite3_stmt *pAllPage;
+
  sqlite3_stmt *pPageData;
+
  sqlite3_value **apVal;
+
  int nMaxField;
+
};

/*
-
** This function is called after candidate indexes have been created. It
-
** runs all the queries to see which indexes they prefer, and populates
-
** IdxStatement.zIdx and IdxStatement.zEQP with the results.
+
** Main recover handle structure.
*/
-
static int idxFindIndexes(
-
  sqlite3expert *p,
-
  char **pzErr                         /* OUT: Error message (sqlite3_malloc) */
-
){
-
  IdxStatement *pStmt;
-
  sqlite3 *dbm = p->dbm;
-
  int rc = SQLITE_OK;
+
struct sqlite3_recover {
+
  /* Copies of sqlite3_recover_init[_sql]() parameters */
+
  sqlite3 *dbIn;                  /* Input database */
+
  char *zDb;                      /* Name of input db ("main" etc.) */
+
  char *zUri;                     /* URI for output database */
+
  void *pSqlCtx;                  /* SQL callback context */
+
  int (*xSql)(void*,const char*); /* Pointer to SQL callback function */

-
  IdxHash hIdx;
-
  idxHashInit(&hIdx);
+
  /* Values configured by sqlite3_recover_config() */
+
  char *zStateDb;                 /* State database to use (or NULL) */
+
  char *zLostAndFound;            /* Name of lost-and-found table (or NULL) */
+
  int bFreelistCorrupt;           /* SQLITE_RECOVER_FREELIST_CORRUPT setting */
+
  int bRecoverRowid;              /* SQLITE_RECOVER_ROWIDS setting */
+
  int bSlowIndexes;               /* SQLITE_RECOVER_SLOWINDEXES setting */

-
  for(pStmt=p->pStatement; rc==SQLITE_OK && pStmt; pStmt=pStmt->pNext){
-
    IdxHashEntry *pEntry;
-
    sqlite3_stmt *pExplain = 0;
-
    idxHashClear(&hIdx);
-
    rc = idxPrintfPrepareStmt(dbm, &pExplain, pzErr,
-
        "EXPLAIN QUERY PLAN %s", pStmt->zSql
-
    );
-
    while( rc==SQLITE_OK && sqlite3_step(pExplain)==SQLITE_ROW ){
-
      /* int iId = sqlite3_column_int(pExplain, 0); */
-
      /* int iParent = sqlite3_column_int(pExplain, 1); */
-
      /* int iNotUsed = sqlite3_column_int(pExplain, 2); */
-
      const char *zDetail = (const char*)sqlite3_column_text(pExplain, 3);
-
      int nDetail;
-
      int i;
+
  int pgsz;
+
  int detected_pgsz;
+
  int nReserve;
+
  u8 *pPage1Disk;
+
  u8 *pPage1Cache;

-
      if( !zDetail ) continue;
-
      nDetail = STRLEN(zDetail);
+
  /* Error code and error message */
+
  int errCode;                    /* For sqlite3_recover_errcode() */
+
  char *zErrMsg;                  /* For sqlite3_recover_errmsg() */

-
      for(i=0; i<nDetail; i++){
-
        const char *zIdx = 0;
-
        if( i+13<nDetail && memcmp(&zDetail[i], " USING INDEX ", 13)==0 ){
-
          zIdx = &zDetail[i+13];
-
        }else if( i+22<nDetail 
-
            && memcmp(&zDetail[i], " USING COVERING INDEX ", 22)==0 
-
        ){
-
          zIdx = &zDetail[i+22];
-
        }
-
        if( zIdx ){
-
          const char *zSql;
-
          int nIdx = 0;
-
          while( zIdx[nIdx]!='\0' && (zIdx[nIdx]!=' ' || zIdx[nIdx+1]!='(') ){
-
            nIdx++;
-
          }
-
          zSql = idxHashSearch(&p->hIdx, zIdx, nIdx);
-
          if( zSql ){
-
            idxHashAdd(&rc, &hIdx, zSql, 0);
-
            if( rc ) goto find_indexes_out;
-
          }
-
          break;
-
        }
-
      }
+
  int eState;
+
  int bCloseTransaction;

-
      if( zDetail[0]!='-' ){
-
        pStmt->zEQP = idxAppendText(&rc, pStmt->zEQP, "%s\n", zDetail);
-
      }
-
    }
+
  /* Variables used with eState==RECOVER_STATE_WRITING */
+
  RecoverStateW1 w1;

-
    for(pEntry=hIdx.pFirst; pEntry; pEntry=pEntry->pNext){
-
      pStmt->zIdx = idxAppendText(&rc, pStmt->zIdx, "%s;\n", pEntry->zKey);
-
    }
+
  /* Variables used with states RECOVER_STATE_LOSTANDFOUND[123] */
+
  RecoverStateLAF laf;

-
    idxFinalize(&rc, pExplain);
-
  }
+
  /* Fields used within sqlite3_recover_run() */
+
  sqlite3 *dbOut;                 /* Output database */
+
  sqlite3_stmt *pGetPage;         /* SELECT against input db sqlite_dbdata */
+
  RecoverTable *pTblList;         /* List of tables recovered from schema */
+
};

-
 find_indexes_out:
-
  idxHashClear(&hIdx);
-
  return rc;
-
}
+
/*
+
** The various states in which an sqlite3_recover object may exist:
+
**
+
**   RECOVER_STATE_INIT:
+
**    The object is initially created in this state. sqlite3_recover_step()
+
**    has yet to be called. This is the only state in which it is permitted
+
**    to call sqlite3_recover_config().
+
**
+
**   RECOVER_STATE_WRITING:
+
**
+
**   RECOVER_STATE_LOSTANDFOUND1:
+
**    State to populate the bitmap of pages used by other tables or the
+
**    database freelist.
+
**
+
**   RECOVER_STATE_LOSTANDFOUND2:
+
**    Populate the recovery.map table - used to figure out a "root" page
+
**    for each lost page from in the database from which records are
+
**    extracted.
+
**
+
**   RECOVER_STATE_LOSTANDFOUND3:
+
**    Populate the lost-and-found table itself.
+
*/
+
#define RECOVER_STATE_INIT           0
+
#define RECOVER_STATE_WRITING        1
+
#define RECOVER_STATE_LOSTANDFOUND1  2
+
#define RECOVER_STATE_LOSTANDFOUND2  3
+
#define RECOVER_STATE_LOSTANDFOUND3  4
+
#define RECOVER_STATE_SCHEMA2        5
+
#define RECOVER_STATE_DONE           6

-
static int idxAuthCallback(
-
  void *pCtx,
-
  int eOp,
-
  const char *z3,
-
  const char *z4,
-
  const char *zDb,
-
  const char *zTrigger
-
){
-
  int rc = SQLITE_OK;
-
  (void)z4;
-
  (void)zTrigger;
-
  if( eOp==SQLITE_INSERT || eOp==SQLITE_UPDATE || eOp==SQLITE_DELETE ){
-
    if( sqlite3_stricmp(zDb, "main")==0 ){
-
      sqlite3expert *p = (sqlite3expert*)pCtx;
-
      IdxTable *pTab;
-
      for(pTab=p->pTable; pTab; pTab=pTab->pNext){
-
        if( 0==sqlite3_stricmp(z3, pTab->zName) ) break;
-
      }
-
      if( pTab ){
-
        IdxWrite *pWrite;
-
        for(pWrite=p->pWrite; pWrite; pWrite=pWrite->pNext){
-
          if( pWrite->pTab==pTab && pWrite->eOp==eOp ) break;
-
        }
-
        if( pWrite==0 ){
-
          pWrite = idxMalloc(&rc, sizeof(IdxWrite));
-
          if( rc==SQLITE_OK ){
-
            pWrite->pTab = pTab;
-
            pWrite->eOp = eOp;
-
            pWrite->pNext = p->pWrite;
-
            p->pWrite = pWrite;
-
          }
-
        }
-
      }
-
    }
-
  }
-
  return rc;
-
}

-
static int idxProcessOneTrigger(
-
  sqlite3expert *p, 
-
  IdxWrite *pWrite, 
-
  char **pzErr
-
){
-
  static const char *zInt = UNIQUE_TABLE_NAME;
-
  static const char *zDrop = "DROP TABLE " UNIQUE_TABLE_NAME;
-
  IdxTable *pTab = pWrite->pTab;
-
  const char *zTab = pTab->zName;
-
  const char *zSql = 
-
    "SELECT 'CREATE TEMP' || substr(sql, 7) FROM sqlite_schema "
-
    "WHERE tbl_name = %Q AND type IN ('table', 'trigger') "
-
    "ORDER BY type;";
-
  sqlite3_stmt *pSelect = 0;
-
  int rc = SQLITE_OK;
-
  char *zWrite = 0;
+
/*
+
** Global variables used by this extension.
+
*/
+
typedef struct RecoverGlobal RecoverGlobal;
+
struct RecoverGlobal {
+
  const sqlite3_io_methods *pMethods;
+
  sqlite3_recover *p;
+
};
+
static RecoverGlobal recover_g;

-
  /* Create the table and its triggers in the temp schema */
-
  rc = idxPrintfPrepareStmt(p->db, &pSelect, pzErr, zSql, zTab, zTab);
-
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSelect) ){
-
    const char *zCreate = (const char*)sqlite3_column_text(pSelect, 0);
-
    if( zCreate==0 ) continue;
-
    rc = sqlite3_exec(p->dbv, zCreate, 0, 0, pzErr);
-
  }
-
  idxFinalize(&rc, pSelect);
+
/*
+
** Use this static SQLite mutex to protect the globals during the
+
** first call to sqlite3_recover_step().
+
*/ 
+
#define RECOVER_MUTEX_ID SQLITE_MUTEX_STATIC_APP2

-
  /* Rename the table in the temp schema to zInt */
-
  if( rc==SQLITE_OK ){
-
    char *z = sqlite3_mprintf("ALTER TABLE temp.%Q RENAME TO %Q", zTab, zInt);
-
    if( z==0 ){
-
      rc = SQLITE_NOMEM;
-
    }else{
-
      rc = sqlite3_exec(p->dbv, z, 0, 0, pzErr);
-
      sqlite3_free(z);
-
    }
-
  }

-
  switch( pWrite->eOp ){
-
    case SQLITE_INSERT: {
-
      int i;
-
      zWrite = idxAppendText(&rc, zWrite, "INSERT INTO %Q VALUES(", zInt);
-
      for(i=0; i<pTab->nCol; i++){
-
        zWrite = idxAppendText(&rc, zWrite, "%s?", i==0 ? "" : ", ");
-
      }
-
      zWrite = idxAppendText(&rc, zWrite, ")");
-
      break;
-
    }
-
    case SQLITE_UPDATE: {
-
      int i;
-
      zWrite = idxAppendText(&rc, zWrite, "UPDATE %Q SET ", zInt);
-
      for(i=0; i<pTab->nCol; i++){
-
        zWrite = idxAppendText(&rc, zWrite, "%s%Q=?", i==0 ? "" : ", ", 
-
            pTab->aCol[i].zName
-
        );
-
      }
-
      break;
-
    }
-
    default: {
-
      assert( pWrite->eOp==SQLITE_DELETE );
-
      if( rc==SQLITE_OK ){
-
        zWrite = sqlite3_mprintf("DELETE FROM %Q", zInt);
-
        if( zWrite==0 ) rc = SQLITE_NOMEM;
-
      }
-
    }
-
  }
+
/* 
+
** Default value for SQLITE_RECOVER_ROWIDS (sqlite3_recover.bRecoverRowid).
+
*/
+
#define RECOVER_ROWID_DEFAULT 1

-
  if( rc==SQLITE_OK ){
-
    sqlite3_stmt *pX = 0;
-
    rc = sqlite3_prepare_v2(p->dbv, zWrite, -1, &pX, 0);
-
    idxFinalize(&rc, pX);
-
    if( rc!=SQLITE_OK ){
-
      idxDatabaseError(p->dbv, pzErr);
-
    }
-
  }
-
  sqlite3_free(zWrite);
+
/*
+
** Mutex handling:
+
**
+
**    recoverEnterMutex()       -   Enter the recovery mutex
+
**    recoverLeaveMutex()       -   Leave the recovery mutex
+
**    recoverAssertMutexHeld()  -   Assert that the recovery mutex is held
+
*/
+
#if defined(SQLITE_THREADSAFE) && SQLITE_THREADSAFE==0
+
# define recoverEnterMutex()
+
# define recoverLeaveMutex()
+
#else
+
static void recoverEnterMutex(void){
+
  sqlite3_mutex_enter(sqlite3_mutex_alloc(RECOVER_MUTEX_ID));
+
}
+
static void recoverLeaveMutex(void){
+
  sqlite3_mutex_leave(sqlite3_mutex_alloc(RECOVER_MUTEX_ID));
+
}
+
#endif
+
#if SQLITE_THREADSAFE+0>=1 && defined(SQLITE_DEBUG)
+
static void recoverAssertMutexHeld(void){
+
  assert( sqlite3_mutex_held(sqlite3_mutex_alloc(RECOVER_MUTEX_ID)) );
+
}
+
#else
+
# define recoverAssertMutexHeld()
+
#endif

-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_exec(p->dbv, zDrop, 0, 0, pzErr);
-
  }

-
  return rc;
+
/*
+
** Like strlen(). But handles NULL pointer arguments.
+
*/
+
static int recoverStrlen(const char *zStr){
+
  if( zStr==0 ) return 0;
+
  return (int)(strlen(zStr)&0x7fffffff);
}

-
static int idxProcessTriggers(sqlite3expert *p, char **pzErr){
-
  int rc = SQLITE_OK;
-
  IdxWrite *pEnd = 0;
-
  IdxWrite *pFirst = p->pWrite;
-

-
  while( rc==SQLITE_OK && pFirst!=pEnd ){
-
    IdxWrite *pIter;
-
    for(pIter=pFirst; rc==SQLITE_OK && pIter!=pEnd; pIter=pIter->pNext){
-
      rc = idxProcessOneTrigger(p, pIter, pzErr);
+
/*
+
** This function is a no-op if the recover handle passed as the first 
+
** argument already contains an error (if p->errCode!=SQLITE_OK). 
+
**
+
** Otherwise, an attempt is made to allocate, zero and return a buffer nByte
+
** bytes in size. If successful, a pointer to the new buffer is returned. Or,
+
** if an OOM error occurs, NULL is returned and the handle error code
+
** (p->errCode) set to SQLITE_NOMEM.
+
*/
+
static void *recoverMalloc(sqlite3_recover *p, i64 nByte){
+
  void *pRet = 0;
+
  assert( nByte>0 );
+
  if( p->errCode==SQLITE_OK ){
+
    pRet = sqlite3_malloc64(nByte);
+
    if( pRet ){
+
      memset(pRet, 0, nByte);
+
    }else{
+
      p->errCode = SQLITE_NOMEM;
    }
-
    pEnd = pFirst;
-
    pFirst = p->pWrite;
  }
-

-
  return rc;
+
  return pRet;
+
}
+

+
/*
+
** Set the error code and error message for the recover handle passed as
+
** the first argument. The error code is set to the value of parameter
+
** errCode.
+
**
+
** Parameter zFmt must be a printf() style formatting string. The handle 
+
** error message is set to the result of using any trailing arguments for 
+
** parameter substitutions in the formatting string.
+
**
+
** For example:
+
**
+
**   recoverError(p, SQLITE_ERROR, "no such table: %s", zTablename);
+
*/
+
static int recoverError(
+
  sqlite3_recover *p, 
+
  int errCode, 
+
  const char *zFmt, ...
+
){
+
  char *z = 0;
+
  va_list ap;
+
  va_start(ap, zFmt);
+
  if( zFmt ){
+
    z = sqlite3_vmprintf(zFmt, ap);
+
    va_end(ap);
+
  }
+
  sqlite3_free(p->zErrMsg);
+
  p->zErrMsg = z;
+
  p->errCode = errCode;
+
  return errCode;
}


-
static int idxCreateVtabSchema(sqlite3expert *p, char **pzErrmsg){
-
  int rc = idxRegisterVtab(p);
-
  sqlite3_stmt *pSchema = 0;
+
/*
+
** This function is a no-op if p->errCode is initially other than SQLITE_OK.
+
** In this case it returns NULL.
+
**
+
** Otherwise, an attempt is made to allocate and return a bitmap object
+
** large enough to store a bit for all page numbers between 1 and nPg,
+
** inclusive. The bitmap is initially zeroed.
+
*/
+
static RecoverBitmap *recoverBitmapAlloc(sqlite3_recover *p, i64 nPg){
+
  int nElem = (nPg+1+31) / 32;
+
  int nByte = sizeof(RecoverBitmap) + nElem*sizeof(u32);
+
  RecoverBitmap *pRet = (RecoverBitmap*)recoverMalloc(p, nByte);

-
  /* For each table in the main db schema:
-
  **
-
  **   1) Add an entry to the p->pTable list, and
-
  **   2) Create the equivalent virtual table in dbv.
-
  */
-
  rc = idxPrepareStmt(p->db, &pSchema, pzErrmsg,
-
      "SELECT type, name, sql, 1 FROM sqlite_schema "
-
      "WHERE type IN ('table','view') AND name NOT LIKE 'sqlite_%%' "
-
      " UNION ALL "
-
      "SELECT type, name, sql, 2 FROM sqlite_schema "
-
      "WHERE type = 'trigger'"
-
      "  AND tbl_name IN(SELECT name FROM sqlite_schema WHERE type = 'view') "
-
      "ORDER BY 4, 1"
-
  );
-
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSchema) ){
-
    const char *zType = (const char*)sqlite3_column_text(pSchema, 0);
-
    const char *zName = (const char*)sqlite3_column_text(pSchema, 1);
-
    const char *zSql = (const char*)sqlite3_column_text(pSchema, 2);
+
  if( pRet ){
+
    pRet->nPg = nPg;
+
  }
+
  return pRet;
+
}

-
    if( zType==0 || zName==0 ) continue;
-
    if( zType[0]=='v' || zType[1]=='r' ){
-
      if( zSql ) rc = sqlite3_exec(p->dbv, zSql, 0, 0, pzErrmsg);
-
    }else{
-
      IdxTable *pTab;
-
      rc = idxGetTableInfo(p->db, zName, &pTab, pzErrmsg);
-
      if( rc==SQLITE_OK ){
-
        int i;
-
        char *zInner = 0;
-
        char *zOuter = 0;
-
        pTab->pNext = p->pTable;
-
        p->pTable = pTab;
+
/*
+
** Free a bitmap object allocated by recoverBitmapAlloc().
+
*/
+
static void recoverBitmapFree(RecoverBitmap *pMap){
+
  sqlite3_free(pMap);
+
}

-
        /* The statement the vtab will pass to sqlite3_declare_vtab() */
-
        zInner = idxAppendText(&rc, 0, "CREATE TABLE x(");
-
        for(i=0; i<pTab->nCol; i++){
-
          zInner = idxAppendText(&rc, zInner, "%s%Q COLLATE %s", 
-
              (i==0 ? "" : ", "), pTab->aCol[i].zName, pTab->aCol[i].zColl
-
          );
-
        }
-
        zInner = idxAppendText(&rc, zInner, ")");
+
/*
+
** Set the bit associated with page iPg in bitvec pMap.
+
*/
+
static void recoverBitmapSet(RecoverBitmap *pMap, i64 iPg){
+
  if( iPg<=pMap->nPg ){
+
    int iElem = (iPg / 32);
+
    int iBit = (iPg % 32);
+
    pMap->aElem[iElem] |= (((u32)1) << iBit);
+
  }
+
}

-
        /* The CVT statement to create the vtab */
-
        zOuter = idxAppendText(&rc, 0, 
-
            "CREATE VIRTUAL TABLE %Q USING expert(%Q)", zName, zInner
-
        );
-
        if( rc==SQLITE_OK ){
-
          rc = sqlite3_exec(p->dbv, zOuter, 0, 0, pzErrmsg);
-
        }
-
        sqlite3_free(zInner);
-
        sqlite3_free(zOuter);
-
      }
-
    }
+
/*
+
** Query bitmap object pMap for the state of the bit associated with page
+
** iPg. Return 1 if it is set, or 0 otherwise.
+
*/
+
static int recoverBitmapQuery(RecoverBitmap *pMap, i64 iPg){
+
  int ret = 1;
+
  if( iPg<=pMap->nPg && iPg>0 ){
+
    int iElem = (iPg / 32);
+
    int iBit = (iPg % 32);
+
    ret = (pMap->aElem[iElem] & (((u32)1) << iBit)) ? 1 : 0;
  }
-
  idxFinalize(&rc, pSchema);
-
  return rc;
+
  return ret;
}

-
struct IdxSampleCtx {
-
  int iTarget;
-
  double target;                  /* Target nRet/nRow value */
-
  double nRow;                    /* Number of rows seen */
-
  double nRet;                    /* Number of rows returned */
-
};
+
/*
+
** Set the recover handle error to the error code and message returned by
+
** calling sqlite3_errcode() and sqlite3_errmsg(), respectively, on database
+
** handle db.
+
*/
+
static int recoverDbError(sqlite3_recover *p, sqlite3 *db){
+
  return recoverError(p, sqlite3_errcode(db), "%s", sqlite3_errmsg(db));
+
}

-
static void idxSampleFunc(
-
  sqlite3_context *pCtx,
-
  int argc,
-
  sqlite3_value **argv
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). 
+
**
+
** Otherwise, it attempts to prepare the SQL statement in zSql against
+
** database handle db. If successful, the statement handle is returned.
+
** Or, if an error occurs, NULL is returned and an error left in the
+
** recover handle.
+
*/
+
static sqlite3_stmt *recoverPrepare(
+
  sqlite3_recover *p,
+
  sqlite3 *db, 
+
  const char *zSql
){
-
  struct IdxSampleCtx *p = (struct IdxSampleCtx*)sqlite3_user_data(pCtx);
-
  int bRet;
-

-
  (void)argv;
-
  assert( argc==0 );
-
  if( p->nRow==0.0 ){
-
    bRet = 1;
-
  }else{
-
    bRet = (p->nRet / p->nRow) <= p->target;
-
    if( bRet==0 ){
-
      unsigned short rnd;
-
      sqlite3_randomness(2, (void*)&rnd);
-
      bRet = ((int)rnd % 100) <= p->iTarget;
+
  sqlite3_stmt *pStmt = 0;
+
  if( p->errCode==SQLITE_OK ){
+
    if( sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0) ){
+
      recoverDbError(p, db);
    }
  }
-

-
  sqlite3_result_int(pCtx, bRet);
-
  p->nRow += 1.0;
-
  p->nRet += (double)bRet;
+
  return pStmt;
}

-
struct IdxRemCtx {
-
  int nSlot;
-
  struct IdxRemSlot {
-
    int eType;                    /* SQLITE_NULL, INTEGER, REAL, TEXT, BLOB */
-
    i64 iVal;                     /* SQLITE_INTEGER value */
-
    double rVal;                  /* SQLITE_FLOAT value */
-
    int nByte;                    /* Bytes of space allocated at z */
-
    int n;                        /* Size of buffer z */
-
    char *z;                      /* SQLITE_TEXT/BLOB value */
-
  } aSlot[1];
-
};
-

/*
-
** Implementation of scalar function rem().
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). 
+
**
+
** Otherwise, argument zFmt is used as a printf() style format string,
+
** along with any trailing arguments, to create an SQL statement. This
+
** SQL statement is prepared against database handle db and, if successful,
+
** the statment handle returned. Or, if an error occurs - either during
+
** the printf() formatting or when preparing the resulting SQL - an
+
** error code and message are left in the recover handle.
*/
-
static void idxRemFunc(
-
  sqlite3_context *pCtx,
-
  int argc,
-
  sqlite3_value **argv
+
static sqlite3_stmt *recoverPreparePrintf(
+
  sqlite3_recover *p,
+
  sqlite3 *db, 
+
  const char *zFmt, ...
){
-
  struct IdxRemCtx *p = (struct IdxRemCtx*)sqlite3_user_data(pCtx);
-
  struct IdxRemSlot *pSlot;
-
  int iSlot;
-
  assert( argc==2 );
-

-
  iSlot = sqlite3_value_int(argv[0]);
-
  assert( iSlot<=p->nSlot );
-
  pSlot = &p->aSlot[iSlot];
-

-
  switch( pSlot->eType ){
-
    case SQLITE_NULL:
-
      /* no-op */
-
      break;
-

-
    case SQLITE_INTEGER:
-
      sqlite3_result_int64(pCtx, pSlot->iVal);
-
      break;
-

-
    case SQLITE_FLOAT:
-
      sqlite3_result_double(pCtx, pSlot->rVal);
-
      break;
+
  sqlite3_stmt *pStmt = 0;
+
  if( p->errCode==SQLITE_OK ){
+
    va_list ap;
+
    char *z;
+
    va_start(ap, zFmt);
+
    z = sqlite3_vmprintf(zFmt, ap);
+
    va_end(ap);
+
    if( z==0 ){
+
      p->errCode = SQLITE_NOMEM;
+
    }else{
+
      pStmt = recoverPrepare(p, db, z);
+
      sqlite3_free(z);
+
    }
+
  }
+
  return pStmt;
+
}

-
    case SQLITE_BLOB:
-
      sqlite3_result_blob(pCtx, pSlot->z, pSlot->n, SQLITE_TRANSIENT);
-
      break;
+
/*
+
** Reset SQLite statement handle pStmt. If the call to sqlite3_reset() 
+
** indicates that an error occurred, and there is not already an error
+
** in the recover handle passed as the first argument, set the error
+
** code and error message appropriately.
+
**
+
** This function returns a copy of the statement handle pointer passed
+
** as the second argument.
+
*/
+
static sqlite3_stmt *recoverReset(sqlite3_recover *p, sqlite3_stmt *pStmt){
+
  int rc = sqlite3_reset(pStmt);
+
  if( rc!=SQLITE_OK && rc!=SQLITE_CONSTRAINT && p->errCode==SQLITE_OK ){
+
    recoverDbError(p, sqlite3_db_handle(pStmt));
+
  }
+
  return pStmt;
+
}

-
    case SQLITE_TEXT:
-
      sqlite3_result_text(pCtx, pSlot->z, pSlot->n, SQLITE_TRANSIENT);
-
      break;
+
/*
+
** Finalize SQLite statement handle pStmt. If the call to sqlite3_reset() 
+
** indicates that an error occurred, and there is not already an error
+
** in the recover handle passed as the first argument, set the error
+
** code and error message appropriately.
+
*/
+
static void recoverFinalize(sqlite3_recover *p, sqlite3_stmt *pStmt){
+
  sqlite3 *db = sqlite3_db_handle(pStmt);
+
  int rc = sqlite3_finalize(pStmt);
+
  if( rc!=SQLITE_OK && p->errCode==SQLITE_OK ){
+
    recoverDbError(p, db);
  }
+
}

-
  pSlot->eType = sqlite3_value_type(argv[1]);
-
  switch( pSlot->eType ){
-
    case SQLITE_NULL:
-
      /* no-op */
-
      break;
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). A copy of p->errCode is returned in this 
+
** case.
+
**
+
** Otherwise, execute SQL script zSql. If successful, return SQLITE_OK.
+
** Or, if an error occurs, leave an error code and message in the recover
+
** handle and return a copy of the error code.
+
*/
+
static int recoverExec(sqlite3_recover *p, sqlite3 *db, const char *zSql){
+
  if( p->errCode==SQLITE_OK ){
+
    int rc = sqlite3_exec(db, zSql, 0, 0, 0);
+
    if( rc ){
+
      recoverDbError(p, db);
+
    }
+
  }
+
  return p->errCode;
+
}

-
    case SQLITE_INTEGER:
-
      pSlot->iVal = sqlite3_value_int64(argv[1]);
-
      break;
+
/*
+
** Bind the value pVal to parameter iBind of statement pStmt. Leave an
+
** error in the recover handle passed as the first argument if an error
+
** (e.g. an OOM) occurs.
+
*/
+
static void recoverBindValue(
+
  sqlite3_recover *p, 
+
  sqlite3_stmt *pStmt, 
+
  int iBind, 
+
  sqlite3_value *pVal
+
){
+
  if( p->errCode==SQLITE_OK ){
+
    int rc = sqlite3_bind_value(pStmt, iBind, pVal);
+
    if( rc ) recoverError(p, rc, 0);
+
  }
+
}

-
    case SQLITE_FLOAT:
-
      pSlot->rVal = sqlite3_value_double(argv[1]);
-
      break;
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). NULL is returned in this case.
+
**
+
** Otherwise, an attempt is made to interpret zFmt as a printf() style
+
** formatting string and the result of using the trailing arguments for
+
** parameter substitution with it written into a buffer obtained from
+
** sqlite3_malloc(). If successful, a pointer to the buffer is returned.
+
** It is the responsibility of the caller to eventually free the buffer
+
** using sqlite3_free().
+
**
+
** Or, if an error occurs, an error code and message is left in the recover
+
** handle and NULL returned.
+
*/
+
static char *recoverMPrintf(sqlite3_recover *p, const char *zFmt, ...){
+
  va_list ap;
+
  char *z;
+
  va_start(ap, zFmt);
+
  z = sqlite3_vmprintf(zFmt, ap);
+
  va_end(ap);
+
  if( p->errCode==SQLITE_OK ){
+
    if( z==0 ) p->errCode = SQLITE_NOMEM;
+
  }else{
+
    sqlite3_free(z);
+
    z = 0;
+
  }
+
  return z;
+
}

-
    case SQLITE_BLOB:
-
    case SQLITE_TEXT: {
-
      int nByte = sqlite3_value_bytes(argv[1]);
-
      const void *pData = 0;
-
      if( nByte>pSlot->nByte ){
-
        char *zNew = (char*)sqlite3_realloc(pSlot->z, nByte*2);
-
        if( zNew==0 ){
-
          sqlite3_result_error_nomem(pCtx);
-
          return;
-
        }
-
        pSlot->nByte = nByte*2;
-
        pSlot->z = zNew;
-
      }
-
      pSlot->n = nByte;
-
      if( pSlot->eType==SQLITE_BLOB ){
-
        pData = sqlite3_value_blob(argv[1]);
-
        if( pData ) memcpy(pSlot->z, pData, nByte);
-
      }else{
-
        pData = sqlite3_value_text(argv[1]);
-
        memcpy(pSlot->z, pData, nByte);
-
      }
-
      break;
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). Zero is returned in this case.
+
**
+
** Otherwise, execute "PRAGMA page_count" against the input database. If
+
** successful, return the integer result. Or, if an error occurs, leave an
+
** error code and error message in the sqlite3_recover handle and return
+
** zero.
+
*/
+
static i64 recoverPageCount(sqlite3_recover *p){
+
  i64 nPg = 0;
+
  if( p->errCode==SQLITE_OK ){
+
    sqlite3_stmt *pStmt = 0;
+
    pStmt = recoverPreparePrintf(p, p->dbIn, "PRAGMA %Q.page_count", p->zDb);
+
    if( pStmt ){
+
      sqlite3_step(pStmt);
+
      nPg = sqlite3_column_int64(pStmt, 0);
    }
+
    recoverFinalize(p, pStmt);
  }
+
  return nPg;
}

-
static int idxLargestIndex(sqlite3 *db, int *pnMax, char **pzErr){
-
  int rc = SQLITE_OK;
-
  const char *zMax = 
-
    "SELECT max(i.seqno) FROM "
-
    "  sqlite_schema AS s, "
-
    "  pragma_index_list(s.name) AS l, "
-
    "  pragma_index_info(l.name) AS i "
-
    "WHERE s.type = 'table'";
-
  sqlite3_stmt *pMax = 0;
+
/*
+
** Implementation of SQL scalar function "read_i32". The first argument to 
+
** this function must be a blob. The second a non-negative integer. This 
+
** function reads and returns a 32-bit big-endian integer from byte
+
** offset (4*<arg2>) of the blob.
+
**
+
**     SELECT read_i32(<blob>, <idx>)
+
*/
+
static void recoverReadI32(
+
  sqlite3_context *context, 
+
  int argc, 
+
  sqlite3_value **argv
+
){
+
  const unsigned char *pBlob;
+
  int nBlob;
+
  int iInt;
+

+
  assert( argc==2 );
+
  nBlob = sqlite3_value_bytes(argv[0]);
+
  pBlob = (const unsigned char*)sqlite3_value_blob(argv[0]);
+
  iInt = sqlite3_value_int(argv[1]) & 0xFFFF;

-
  *pnMax = 0;
-
  rc = idxPrepareStmt(db, &pMax, pzErr, zMax);
-
  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pMax) ){
-
    *pnMax = sqlite3_column_int(pMax, 0) + 1;
+
  if( (iInt+1)*4<=nBlob ){
+
    const unsigned char *a = &pBlob[iInt*4];
+
    i64 iVal = ((i64)a[0]<<24)
+
             + ((i64)a[1]<<16)
+
             + ((i64)a[2]<< 8)
+
             + ((i64)a[3]<< 0);
+
    sqlite3_result_int64(context, iVal);
  }
-
  idxFinalize(&rc, pMax);
-

-
  return rc;
}

-
static int idxPopulateOneStat1(
-
  sqlite3expert *p,
-
  sqlite3_stmt *pIndexXInfo,
-
  sqlite3_stmt *pWriteStat,
-
  const char *zTab,
-
  const char *zIdx,
-
  char **pzErr
+
/*
+
** Implementation of SQL scalar function "page_is_used". This function
+
** is used as part of the procedure for locating orphan rows for the
+
** lost-and-found table, and it depends on those routines having populated
+
** the sqlite3_recover.laf.pUsed variable.
+
**
+
** The only argument to this function is a page-number. It returns true 
+
** if the page has already been used somehow during data recovery, or false
+
** otherwise.
+
**
+
**     SELECT page_is_used(<pgno>);
+
*/
+
static void recoverPageIsUsed(
+
  sqlite3_context *pCtx,
+
  int nArg,
+
  sqlite3_value **apArg
){
-
  char *zCols = 0;
-
  char *zOrder = 0;
-
  char *zQuery = 0;
-
  int nCol = 0;
-
  int i;
-
  sqlite3_stmt *pQuery = 0;
-
  int *aStat = 0;
-
  int rc = SQLITE_OK;
+
  sqlite3_recover *p = (sqlite3_recover*)sqlite3_user_data(pCtx);
+
  i64 pgno = sqlite3_value_int64(apArg[0]);
+
  assert( nArg==1 );
+
  sqlite3_result_int(pCtx, recoverBitmapQuery(p->laf.pUsed, pgno));
+
}

-
  assert( p->iSample>0 );
+
/*
+
** The implementation of a user-defined SQL function invoked by the 
+
** sqlite_dbdata and sqlite_dbptr virtual table modules to access pages
+
** of the database being recovered.
+
**
+
** This function always takes a single integer argument. If the argument
+
** is zero, then the value returned is the number of pages in the db being
+
** recovered. If the argument is greater than zero, it is a page number. 
+
** The value returned in this case is an SQL blob containing the data for 
+
** the identified page of the db being recovered. e.g.
+
**
+
**     SELECT getpage(0);       -- return number of pages in db
+
**     SELECT getpage(4);       -- return page 4 of db as a blob of data 
+
*/
+
static void recoverGetPage(
+
  sqlite3_context *pCtx,
+
  int nArg,
+
  sqlite3_value **apArg
+
){
+
  sqlite3_recover *p = (sqlite3_recover*)sqlite3_user_data(pCtx);
+
  i64 pgno = sqlite3_value_int64(apArg[0]);
+
  sqlite3_stmt *pStmt = 0;

-
  /* Formulate the query text */
-
  sqlite3_bind_text(pIndexXInfo, 1, zIdx, -1, SQLITE_STATIC);
-
  while( SQLITE_OK==rc && SQLITE_ROW==sqlite3_step(pIndexXInfo) ){
-
    const char *zComma = zCols==0 ? "" : ", ";
-
    const char *zName = (const char*)sqlite3_column_text(pIndexXInfo, 0);
-
    const char *zColl = (const char*)sqlite3_column_text(pIndexXInfo, 1);
-
    zCols = idxAppendText(&rc, zCols, 
-
        "%sx.%Q IS rem(%d, x.%Q) COLLATE %s", zComma, zName, nCol, zName, zColl
-
    );
-
    zOrder = idxAppendText(&rc, zOrder, "%s%d", zComma, ++nCol);
-
  }
-
  sqlite3_reset(pIndexXInfo);
-
  if( rc==SQLITE_OK ){
-
    if( p->iSample==100 ){
-
      zQuery = sqlite3_mprintf(
-
          "SELECT %s FROM %Q x ORDER BY %s", zCols, zTab, zOrder
-
      );
-
    }else{
-
      zQuery = sqlite3_mprintf(
-
          "SELECT %s FROM temp."UNIQUE_TABLE_NAME" x ORDER BY %s", zCols, zOrder
+
  assert( nArg==1 );
+
  if( pgno==0 ){
+
    i64 nPg = recoverPageCount(p);
+
    sqlite3_result_int64(pCtx, nPg);
+
    return;
+
  }else{
+
    if( p->pGetPage==0 ){
+
      pStmt = p->pGetPage = recoverPreparePrintf(
+
          p, p->dbIn, "SELECT data FROM sqlite_dbpage(%Q) WHERE pgno=?", p->zDb
      );
+
    }else if( p->errCode==SQLITE_OK ){
+
      pStmt = p->pGetPage;
+
    }
+

+
    if( pStmt ){
+
      sqlite3_bind_int64(pStmt, 1, pgno);
+
      if( SQLITE_ROW==sqlite3_step(pStmt) ){
+
        const u8 *aPg;
+
        int nPg;
+
        assert( p->errCode==SQLITE_OK );
+
        aPg = sqlite3_column_blob(pStmt, 0);
+
        nPg = sqlite3_column_bytes(pStmt, 0);
+
        if( pgno==1 && nPg==p->pgsz && 0==memcmp(p->pPage1Cache, aPg, nPg) ){
+
          aPg = p->pPage1Disk;
+
        }
+
        sqlite3_result_blob(pCtx, aPg, nPg-p->nReserve, SQLITE_TRANSIENT);
+
      }
+
      recoverReset(p, pStmt);
    }
  }
-
  sqlite3_free(zCols);
-
  sqlite3_free(zOrder);

-
  /* Formulate the query text */
-
  if( rc==SQLITE_OK ){
-
    sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv);
-
    rc = idxPrepareStmt(dbrem, &pQuery, pzErr, zQuery);
+
  if( p->errCode ){
+
    if( p->zErrMsg ) sqlite3_result_error(pCtx, p->zErrMsg, -1);
+
    sqlite3_result_error_code(pCtx, p->errCode);
  }
-
  sqlite3_free(zQuery);
+
}

-
  if( rc==SQLITE_OK ){
-
    aStat = (int*)idxMalloc(&rc, sizeof(int)*(nCol+1));
-
  }
-
  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pQuery) ){
-
    IdxHashEntry *pEntry;
-
    char *zStat = 0;
-
    for(i=0; i<=nCol; i++) aStat[i] = 1;
-
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pQuery) ){
-
      aStat[0]++;
-
      for(i=0; i<nCol; i++){
-
        if( sqlite3_column_int(pQuery, i)==0 ) break;
+
/*
+
** Find a string that is not found anywhere in z[].  Return a pointer
+
** to that string.
+
**
+
** Try to use zA and zB first.  If both of those are already found in z[]
+
** then make up some string and store it in the buffer zBuf.
+
*/
+
static const char *recoverUnusedString(
+
  const char *z,                    /* Result must not appear anywhere in z */
+
  const char *zA, const char *zB,   /* Try these first */
+
  char *zBuf                        /* Space to store a generated string */
+
){
+
  unsigned i = 0;
+
  if( strstr(z, zA)==0 ) return zA;
+
  if( strstr(z, zB)==0 ) return zB;
+
  do{
+
    sqlite3_snprintf(20,zBuf,"(%s%u)", zA, i++);
+
  }while( strstr(z,zBuf)!=0 );
+
  return zBuf;
+
}
+

+
/*
+
** Implementation of scalar SQL function "escape_crnl".  The argument passed to
+
** this function is the output of built-in function quote(). If the first
+
** character of the input is "'", indicating that the value passed to quote()
+
** was a text value, then this function searches the input for "\n" and "\r"
+
** characters and adds a wrapper similar to the following:
+
**
+
**   replace(replace(<input>, '\n', char(10), '\r', char(13));
+
**
+
** Or, if the first character of the input is not "'", then a copy of the input
+
** is returned.
+
*/
+
static void recoverEscapeCrnl(
+
  sqlite3_context *context, 
+
  int argc, 
+
  sqlite3_value **argv
+
){
+
  const char *zText = (const char*)sqlite3_value_text(argv[0]);
+
  if( zText && zText[0]=='\'' ){
+
    int nText = sqlite3_value_bytes(argv[0]);
+
    int i;
+
    char zBuf1[20];
+
    char zBuf2[20];
+
    const char *zNL = 0;
+
    const char *zCR = 0;
+
    int nCR = 0;
+
    int nNL = 0;
+

+
    for(i=0; zText[i]; i++){
+
      if( zNL==0 && zText[i]=='\n' ){
+
        zNL = recoverUnusedString(zText, "\\n", "\\012", zBuf1);
+
        nNL = (int)strlen(zNL);
      }
-
      for(/*no-op*/; i<nCol; i++){
-
        aStat[i+1]++;
+
      if( zCR==0 && zText[i]=='\r' ){
+
        zCR = recoverUnusedString(zText, "\\r", "\\015", zBuf2);
+
        nCR = (int)strlen(zCR);
      }
    }

-
    if( rc==SQLITE_OK ){
-
      int s0 = aStat[0];
-
      zStat = sqlite3_mprintf("%d", s0);
-
      if( zStat==0 ) rc = SQLITE_NOMEM;
-
      for(i=1; rc==SQLITE_OK && i<=nCol; i++){
-
        zStat = idxAppendText(&rc, zStat, " %d", (s0+aStat[i]/2) / aStat[i]);
+
    if( zNL || zCR ){
+
      int iOut = 0;
+
      i64 nMax = (nNL > nCR) ? nNL : nCR;
+
      i64 nAlloc = nMax * nText + (nMax+64)*2;
+
      char *zOut = (char*)sqlite3_malloc64(nAlloc);
+
      if( zOut==0 ){
+
        sqlite3_result_error_nomem(context);
+
        return;
      }
-
    }

-
    if( rc==SQLITE_OK ){
-
      sqlite3_bind_text(pWriteStat, 1, zTab, -1, SQLITE_STATIC);
-
      sqlite3_bind_text(pWriteStat, 2, zIdx, -1, SQLITE_STATIC);
-
      sqlite3_bind_text(pWriteStat, 3, zStat, -1, SQLITE_STATIC);
-
      sqlite3_step(pWriteStat);
-
      rc = sqlite3_reset(pWriteStat);
-
    }
+
      if( zNL && zCR ){
+
        memcpy(&zOut[iOut], "replace(replace(", 16);
+
        iOut += 16;
+
      }else{
+
        memcpy(&zOut[iOut], "replace(", 8);
+
        iOut += 8;
+
      }
+
      for(i=0; zText[i]; i++){
+
        if( zText[i]=='\n' ){
+
          memcpy(&zOut[iOut], zNL, nNL);
+
          iOut += nNL;
+
        }else if( zText[i]=='\r' ){
+
          memcpy(&zOut[iOut], zCR, nCR);
+
          iOut += nCR;
+
        }else{
+
          zOut[iOut] = zText[i];
+
          iOut++;
+
        }
+
      }

-
    pEntry = idxHashFind(&p->hIdx, zIdx, STRLEN(zIdx));
-
    if( pEntry ){
-
      assert( pEntry->zVal2==0 );
-
      pEntry->zVal2 = zStat;
-
    }else{
-
      sqlite3_free(zStat);
+
      if( zNL ){
+
        memcpy(&zOut[iOut], ",'", 2); iOut += 2;
+
        memcpy(&zOut[iOut], zNL, nNL); iOut += nNL;
+
        memcpy(&zOut[iOut], "', char(10))", 12); iOut += 12;
+
      }
+
      if( zCR ){
+
        memcpy(&zOut[iOut], ",'", 2); iOut += 2;
+
        memcpy(&zOut[iOut], zCR, nCR); iOut += nCR;
+
        memcpy(&zOut[iOut], "', char(13))", 12); iOut += 12;
+
      }
+

+
      sqlite3_result_text(context, zOut, iOut, SQLITE_TRANSIENT);
+
      sqlite3_free(zOut);
+
      return;
    }
  }
-
  sqlite3_free(aStat);
-
  idxFinalize(&rc, pQuery);

-
  return rc;
+
  sqlite3_result_value(context, argv[0]);
}

-
static int idxBuildSampleTable(sqlite3expert *p, const char *zTab){
-
  int rc;
-
  char *zSql;
-

-
  rc = sqlite3_exec(p->dbv,"DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0);
-
  if( rc!=SQLITE_OK ) return rc;
-

-
  zSql = sqlite3_mprintf(
-
      "CREATE TABLE temp." UNIQUE_TABLE_NAME " AS SELECT * FROM %Q", zTab
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). A copy of the error code is returned in
+
** this case. 
+
**
+
** Otherwise, attempt to populate temporary table "recovery.schema" with the
+
** parts of the database schema that can be extracted from the input database.
+
**
+
** If no error occurs, SQLITE_OK is returned. Otherwise, an error code
+
** and error message are left in the recover handle and a copy of the
+
** error code returned. It is not considered an error if part of all of
+
** the database schema cannot be recovered due to corruption.
+
*/
+
static int recoverCacheSchema(sqlite3_recover *p){
+
  return recoverExec(p, p->dbOut,
+
    "WITH RECURSIVE pages(p) AS ("
+
    "  SELECT 1"
+
    "    UNION"
+
    "  SELECT child FROM sqlite_dbptr('getpage()'), pages WHERE pgno=p"
+
    ")"
+
    "INSERT INTO recovery.schema SELECT"
+
    "  max(CASE WHEN field=0 THEN value ELSE NULL END),"
+
    "  max(CASE WHEN field=1 THEN value ELSE NULL END),"
+
    "  max(CASE WHEN field=2 THEN value ELSE NULL END),"
+
    "  max(CASE WHEN field=3 THEN value ELSE NULL END),"
+
    "  max(CASE WHEN field=4 THEN value ELSE NULL END)"
+
    "FROM sqlite_dbdata('getpage()') WHERE pgno IN ("
+
    "  SELECT p FROM pages"
+
    ") GROUP BY pgno, cell"
  );
-
  if( zSql==0 ) return SQLITE_NOMEM;
-
  rc = sqlite3_exec(p->dbv, zSql, 0, 0, 0);
-
  sqlite3_free(zSql);
-

-
  return rc;
}

/*
-
** This function is called as part of sqlite3_expert_analyze(). Candidate
-
** indexes have already been created in database sqlite3expert.dbm, this
-
** function populates sqlite_stat1 table in the same database.
+
** If this recover handle is not in SQL callback mode (i.e. was not created 
+
** using sqlite3_recover_init_sql()) of if an error has already occurred, 
+
** this function is a no-op. Otherwise, issue a callback with SQL statement
+
** zSql as the parameter. 
**
-
** The stat1 data is generated by querying the 
+
** If the callback returns non-zero, set the recover handle error code to
+
** the value returned (so that the caller will abandon processing).
*/
-
static int idxPopulateStat1(sqlite3expert *p, char **pzErr){
-
  int rc = SQLITE_OK;
-
  int nMax =0;
-
  struct IdxRemCtx *pCtx = 0;
-
  struct IdxSampleCtx samplectx; 
-
  int i;
-
  i64 iPrev = -100000;
-
  sqlite3_stmt *pAllIndex = 0;
-
  sqlite3_stmt *pIndexXInfo = 0;
-
  sqlite3_stmt *pWrite = 0;
-

-
  const char *zAllIndex =
-
    "SELECT s.rowid, s.name, l.name FROM "
-
    "  sqlite_schema AS s, "
-
    "  pragma_index_list(s.name) AS l "
-
    "WHERE s.type = 'table'";
-
  const char *zIndexXInfo = 
-
    "SELECT name, coll FROM pragma_index_xinfo(?) WHERE key";
-
  const char *zWrite = "INSERT INTO sqlite_stat1 VALUES(?, ?, ?)";
+
static void recoverSqlCallback(sqlite3_recover *p, const char *zSql){
+
  if( p->errCode==SQLITE_OK && p->xSql ){
+
    int res = p->xSql(p->pSqlCtx, zSql);
+
    if( res ){
+
      recoverError(p, SQLITE_ERROR, "callback returned an error - %d", res);
+
    }
+
  }
+
}

-
  /* If iSample==0, no sqlite_stat1 data is required. */
-
  if( p->iSample==0 ) return SQLITE_OK;
+
/*
+
** Transfer the following settings from the input database to the output
+
** database:
+
**
+
**   + page-size,
+
**   + auto-vacuum settings,
+
**   + database encoding,
+
**   + user-version (PRAGMA user_version), and
+
**   + application-id (PRAGMA application_id), and
+
*/
+
static void recoverTransferSettings(sqlite3_recover *p){
+
  const char *aPragma[] = {
+
    "encoding",
+
    "page_size",
+
    "auto_vacuum",
+
    "user_version",
+
    "application_id"
+
  };
+
  int ii;
+

+
  /* Truncate the output database to 0 pages in size. This is done by 
+
  ** opening a new, empty, temp db, then using the backup API to clobber 
+
  ** any existing output db with a copy of it. */
+
  if( p->errCode==SQLITE_OK ){
+
    sqlite3 *db2 = 0;
+
    int rc = sqlite3_open("", &db2);
+
    if( rc!=SQLITE_OK ){
+
      recoverDbError(p, db2);
+
      return;
+
    }

-
  rc = idxLargestIndex(p->dbm, &nMax, pzErr);
-
  if( nMax<=0 || rc!=SQLITE_OK ) return rc;
+
    for(ii=0; ii<sizeof(aPragma)/sizeof(aPragma[0]); ii++){
+
      const char *zPrag = aPragma[ii];
+
      sqlite3_stmt *p1 = 0;
+
      p1 = recoverPreparePrintf(p, p->dbIn, "PRAGMA %Q.%s", p->zDb, zPrag);
+
      if( p->errCode==SQLITE_OK && sqlite3_step(p1)==SQLITE_ROW ){
+
        const char *zArg = (const char*)sqlite3_column_text(p1, 0);
+
        char *z2 = recoverMPrintf(p, "PRAGMA %s = %Q", zPrag, zArg);
+
        recoverSqlCallback(p, z2);
+
        recoverExec(p, db2, z2);
+
        sqlite3_free(z2);
+
        if( zArg==0 ){
+
          recoverError(p, SQLITE_NOMEM, 0);
+
        }
+
      }
+
      recoverFinalize(p, p1);
+
    }
+
    recoverExec(p, db2, "CREATE TABLE t1(a); DROP TABLE t1;");

-
  rc = sqlite3_exec(p->dbm, "ANALYZE; PRAGMA writable_schema=1", 0, 0, 0);
+
    if( p->errCode==SQLITE_OK ){
+
      sqlite3 *db = p->dbOut;
+
      sqlite3_backup *pBackup = sqlite3_backup_init(db, "main", db2, "main");
+
      if( pBackup ){
+
        sqlite3_backup_step(pBackup, -1);
+
        p->errCode = sqlite3_backup_finish(pBackup);
+
      }else{
+
        recoverDbError(p, db);
+
      }
+
    }

-
  if( rc==SQLITE_OK ){
-
    int nByte = sizeof(struct IdxRemCtx) + (sizeof(struct IdxRemSlot) * nMax);
-
    pCtx = (struct IdxRemCtx*)idxMalloc(&rc, nByte);
+
    sqlite3_close(db2);
  }
+
}

-
  if( rc==SQLITE_OK ){
-
    sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv);
-
    rc = sqlite3_create_function(
-
        dbrem, "rem", 2, SQLITE_UTF8, (void*)pCtx, idxRemFunc, 0, 0
-
    );
-
  }
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_function(
-
        p->db, "sample", 0, SQLITE_UTF8, (void*)&samplectx, idxSampleFunc, 0, 0
-
    );
-
  }
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). A copy of the error code is returned in
+
** this case. 
+
**
+
** Otherwise, an attempt is made to open the output database, attach
+
** and create the schema of the temporary database used to store
+
** intermediate data, and to register all required user functions and
+
** virtual table modules with the output handle.
+
**
+
** If no error occurs, SQLITE_OK is returned. Otherwise, an error code
+
** and error message are left in the recover handle and a copy of the
+
** error code returned.
+
*/
+
static int recoverOpenOutput(sqlite3_recover *p){
+
  struct Func {
+
    const char *zName;
+
    int nArg;
+
    void (*xFunc)(sqlite3_context*,int,sqlite3_value **);
+
  } aFunc[] = {
+
    { "getpage", 1, recoverGetPage },
+
    { "page_is_used", 1, recoverPageIsUsed },
+
    { "read_i32", 2, recoverReadI32 },
+
    { "escape_crnl", 1, recoverEscapeCrnl },
+
  };

-
  if( rc==SQLITE_OK ){
-
    pCtx->nSlot = nMax+1;
-
    rc = idxPrepareStmt(p->dbm, &pAllIndex, pzErr, zAllIndex);
-
  }
-
  if( rc==SQLITE_OK ){
-
    rc = idxPrepareStmt(p->dbm, &pIndexXInfo, pzErr, zIndexXInfo);
-
  }
-
  if( rc==SQLITE_OK ){
-
    rc = idxPrepareStmt(p->dbm, &pWrite, pzErr, zWrite);
-
  }
+
  const int flags = SQLITE_OPEN_URI|SQLITE_OPEN_CREATE|SQLITE_OPEN_READWRITE;
+
  sqlite3 *db = 0;                /* New database handle */
+
  int ii;                         /* For iterating through aFunc[] */

-
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pAllIndex) ){
-
    i64 iRowid = sqlite3_column_int64(pAllIndex, 0);
-
    const char *zTab = (const char*)sqlite3_column_text(pAllIndex, 1);
-
    const char *zIdx = (const char*)sqlite3_column_text(pAllIndex, 2);
-
    if( zTab==0 || zIdx==0 ) continue;
-
    if( p->iSample<100 && iPrev!=iRowid ){
-
      samplectx.target = (double)p->iSample / 100.0;
-
      samplectx.iTarget = p->iSample;
-
      samplectx.nRow = 0.0;
-
      samplectx.nRet = 0.0;
-
      rc = idxBuildSampleTable(p, zTab);
-
      if( rc!=SQLITE_OK ) break;
-
    }
-
    rc = idxPopulateOneStat1(p, pIndexXInfo, pWrite, zTab, zIdx, pzErr);
-
    iPrev = iRowid;
-
  }
-
  if( rc==SQLITE_OK && p->iSample<100 ){
-
    rc = sqlite3_exec(p->dbv, 
-
        "DROP TABLE IF EXISTS temp." UNIQUE_TABLE_NAME, 0,0,0
-
    );
-
  }
+
  assert( p->dbOut==0 );

-
  idxFinalize(&rc, pAllIndex);
-
  idxFinalize(&rc, pIndexXInfo);
-
  idxFinalize(&rc, pWrite);
+
  if( sqlite3_open_v2(p->zUri, &db, flags, 0) ){
+
    recoverDbError(p, db);
+
  }

-
  if( pCtx ){
-
    for(i=0; i<pCtx->nSlot; i++){
-
      sqlite3_free(pCtx->aSlot[i].z);
-
    }
-
    sqlite3_free(pCtx);
+
  /* Register the sqlite_dbdata and sqlite_dbptr virtual table modules.
+
  ** These two are registered with the output database handle - this
+
  ** module depends on the input handle supporting the sqlite_dbpage
+
  ** virtual table only.  */
+
  if( p->errCode==SQLITE_OK ){
+
    p->errCode = sqlite3_dbdata_init(db, 0, 0);
  }

-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_exec(p->dbm, "ANALYZE sqlite_schema", 0, 0, 0);
+
  /* Register the custom user-functions with the output handle. */
+
  for(ii=0; p->errCode==SQLITE_OK && ii<sizeof(aFunc)/sizeof(aFunc[0]); ii++){
+
    p->errCode = sqlite3_create_function(db, aFunc[ii].zName, 
+
        aFunc[ii].nArg, SQLITE_UTF8, (void*)p, aFunc[ii].xFunc, 0, 0
+
    );
  }

-
  sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0);
-
  return rc;
+
  p->dbOut = db;
+
  return p->errCode;
}

/*
-
** Allocate a new sqlite3expert object.
+
** Attach the auxiliary database 'recovery' to the output database handle.
+
** This temporary database is used during the recovery process and then 
+
** discarded.
*/
-
sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErrmsg){
-
  int rc = SQLITE_OK;
-
  sqlite3expert *pNew;
+
static void recoverOpenRecovery(sqlite3_recover *p){
+
  char *zSql = recoverMPrintf(p, "ATTACH %Q AS recovery;", p->zStateDb);
+
  recoverExec(p, p->dbOut, zSql);
+
  recoverExec(p, p->dbOut,
+
      "PRAGMA writable_schema = 1;"
+
      "CREATE TABLE recovery.map(pgno INTEGER PRIMARY KEY, parent INT);" 
+
      "CREATE TABLE recovery.schema(type, name, tbl_name, rootpage, sql);"
+
  );
+
  sqlite3_free(zSql);
+
}

-
  pNew = (sqlite3expert*)idxMalloc(&rc, sizeof(sqlite3expert));

-
  /* Open two in-memory databases to work with. The "vtab database" (dbv)
-
  ** will contain a virtual table corresponding to each real table in
-
  ** the user database schema, and a copy of each view. It is used to
-
  ** collect information regarding the WHERE, ORDER BY and other clauses
-
  ** of the user's query.
-
  */
-
  if( rc==SQLITE_OK ){
-
    pNew->db = db;
-
    pNew->iSample = 100;
-
    rc = sqlite3_open(":memory:", &pNew->dbv);
-
  }
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_open(":memory:", &pNew->dbm);
-
    if( rc==SQLITE_OK ){
-
      sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0);
-
    }
-
  }
-
  
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK).
+
**
+
** Otherwise, argument zName must be the name of a table that has just been
+
** created in the output database. This function queries the output db
+
** for the schema of said table, and creates a RecoverTable object to
+
** store the schema in memory. The new RecoverTable object is linked into
+
** the list at sqlite3_recover.pTblList.
+
**
+
** Parameter iRoot must be the root page of table zName in the INPUT 
+
** database.
+
*/
+
static void recoverAddTable(
+
  sqlite3_recover *p, 
+
  const char *zName,              /* Name of table created in output db */
+
  i64 iRoot                       /* Root page of same table in INPUT db */
+
){
+
  sqlite3_stmt *pStmt = recoverPreparePrintf(p, p->dbOut, 
+
      "PRAGMA table_xinfo(%Q)", zName
+
  );

-
  /* Copy the entire schema of database [db] into [dbm]. */
-
  if( rc==SQLITE_OK ){
-
    sqlite3_stmt *pSql = 0;
-
    rc = idxPrintfPrepareStmt(pNew->db, &pSql, pzErrmsg, 
-
        "SELECT sql FROM sqlite_schema WHERE name NOT LIKE 'sqlite_%%'"
-
        " AND sql NOT LIKE 'CREATE VIRTUAL %%'"
-
    );
-
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSql) ){
-
      const char *zSql = (const char*)sqlite3_column_text(pSql, 0);
-
      if( zSql ) rc = sqlite3_exec(pNew->dbm, zSql, 0, 0, pzErrmsg);
+
  if( pStmt ){
+
    int iPk = -1;
+
    int iBind = 1;
+
    RecoverTable *pNew = 0;
+
    int nCol = 0;
+
    int nName = recoverStrlen(zName);
+
    int nByte = 0;
+
    while( sqlite3_step(pStmt)==SQLITE_ROW ){
+
      nCol++;
+
      nByte += (sqlite3_column_bytes(pStmt, 1)+1);
+
    }
+
    nByte += sizeof(RecoverTable) + nCol*sizeof(RecoverColumn) + nName+1;
+
    recoverReset(p, pStmt);
+

+
    pNew = recoverMalloc(p, nByte);
+
    if( pNew ){
+
      int i = 0;
+
      int iField = 0;
+
      char *csr = 0;
+
      pNew->aCol = (RecoverColumn*)&pNew[1];
+
      pNew->zTab = csr = (char*)&pNew->aCol[nCol];
+
      pNew->nCol = nCol;
+
      pNew->iRoot = iRoot;
+
      memcpy(csr, zName, nName);
+
      csr += nName+1;
+

+
      for(i=0; sqlite3_step(pStmt)==SQLITE_ROW; i++){
+
        int iPKF = sqlite3_column_int(pStmt, 5);
+
        int n = sqlite3_column_bytes(pStmt, 1);
+
        const char *z = (const char*)sqlite3_column_text(pStmt, 1);
+
        const char *zType = (const char*)sqlite3_column_text(pStmt, 2);
+
        int eHidden = sqlite3_column_int(pStmt, 6);
+

+
        if( iPk==-1 && iPKF==1 && !sqlite3_stricmp("integer", zType) ) iPk = i;
+
        if( iPKF>1 ) iPk = -2;
+
        pNew->aCol[i].zCol = csr;
+
        pNew->aCol[i].eHidden = eHidden;
+
        if( eHidden==RECOVER_EHIDDEN_VIRTUAL ){
+
          pNew->aCol[i].iField = -1;
+
        }else{
+
          pNew->aCol[i].iField = iField++;
+
        }
+
        if( eHidden!=RECOVER_EHIDDEN_VIRTUAL
+
         && eHidden!=RECOVER_EHIDDEN_STORED
+
        ){
+
          pNew->aCol[i].iBind = iBind++;
+
        }
+
        memcpy(csr, z, n);
+
        csr += (n+1);
+
      }
+

+
      pNew->pNext = p->pTblList;
+
      p->pTblList = pNew;
+
      pNew->bIntkey = 1;
    }
-
    idxFinalize(&rc, pSql);
-
  }

-
  /* Create the vtab schema */
-
  if( rc==SQLITE_OK ){
-
    rc = idxCreateVtabSchema(pNew, pzErrmsg);
-
  }
+
    recoverFinalize(p, pStmt);

-
  /* Register the auth callback with dbv */
-
  if( rc==SQLITE_OK ){
-
    sqlite3_set_authorizer(pNew->dbv, idxAuthCallback, (void*)pNew);
-
  }
+
    pStmt = recoverPreparePrintf(p, p->dbOut, "PRAGMA index_xinfo(%Q)", zName);
+
    while( pStmt && sqlite3_step(pStmt)==SQLITE_ROW ){
+
      int iField = sqlite3_column_int(pStmt, 0);
+
      int iCol = sqlite3_column_int(pStmt, 1);

-
  /* If an error has occurred, free the new object and reutrn NULL. Otherwise,
-
  ** return the new sqlite3expert handle.  */
-
  if( rc!=SQLITE_OK ){
-
    sqlite3_expert_destroy(pNew);
-
    pNew = 0;
+
      assert( iField<pNew->nCol && iCol<pNew->nCol );
+
      pNew->aCol[iCol].iField = iField;
+

+
      pNew->bIntkey = 0;
+
      iPk = -2;
+
    }
+
    recoverFinalize(p, pStmt);
+

+
    if( p->errCode==SQLITE_OK ){
+
      if( iPk>=0 ){
+
        pNew->aCol[iPk].bIPK = 1;
+
      }else if( pNew->bIntkey ){
+
        pNew->iRowidBind = iBind++;
+
      }
+
    }
  }
-
  return pNew;
}

/*
-
** Configure an sqlite3expert object.
+
** This function is called after recoverCacheSchema() has cached those parts
+
** of the input database schema that could be recovered in temporary table
+
** "recovery.schema". This function creates in the output database copies
+
** of all parts of that schema that must be created before the tables can
+
** be populated. Specifically, this means:
+
**
+
**     * all tables that are not VIRTUAL, and
+
**     * UNIQUE indexes.
+
**
+
** If the recovery handle uses SQL callbacks, then callbacks containing
+
** the associated "CREATE TABLE" and "CREATE INDEX" statements are made.
+
**
+
** Additionally, records are added to the sqlite_schema table of the
+
** output database for any VIRTUAL tables. The CREATE VIRTUAL TABLE
+
** records are written directly to sqlite_schema, not actually executed.
+
** If the handle is in SQL callback mode, then callbacks are invoked 
+
** with equivalent SQL statements.
*/
-
int sqlite3_expert_config(sqlite3expert *p, int op, ...){
-
  int rc = SQLITE_OK;
-
  va_list ap;
-
  va_start(ap, op);
-
  switch( op ){
-
    case EXPERT_CONFIG_SAMPLE: {
-
      int iVal = va_arg(ap, int);
-
      if( iVal<0 ) iVal = 0;
-
      if( iVal>100 ) iVal = 100;
-
      p->iSample = iVal;
-
      break;
+
static int recoverWriteSchema1(sqlite3_recover *p){
+
  sqlite3_stmt *pSelect = 0;
+
  sqlite3_stmt *pTblname = 0;
+

+
  pSelect = recoverPrepare(p, p->dbOut,
+
      "WITH dbschema(rootpage, name, sql, tbl, isVirtual, isIndex) AS ("
+
      "  SELECT rootpage, name, sql, "
+
      "    type='table', "
+
      "    sql LIKE 'create virtual%',"
+
      "    (type='index' AND (sql LIKE '%unique%' OR ?1))"
+
      "  FROM recovery.schema"
+
      ")"
+
      "SELECT rootpage, tbl, isVirtual, name, sql"
+
      " FROM dbschema "
+
      "  WHERE tbl OR isIndex"
+
      "  ORDER BY tbl DESC, name=='sqlite_sequence' DESC"
+
  );
+

+
  pTblname = recoverPrepare(p, p->dbOut,
+
      "SELECT name FROM sqlite_schema "
+
      "WHERE type='table' ORDER BY rowid DESC LIMIT 1"
+
  );
+

+
  if( pSelect ){
+
    sqlite3_bind_int(pSelect, 1, p->bSlowIndexes);
+
    while( sqlite3_step(pSelect)==SQLITE_ROW ){
+
      i64 iRoot = sqlite3_column_int64(pSelect, 0);
+
      int bTable = sqlite3_column_int(pSelect, 1);
+
      int bVirtual = sqlite3_column_int(pSelect, 2);
+
      const char *zName = (const char*)sqlite3_column_text(pSelect, 3);
+
      const char *zSql = (const char*)sqlite3_column_text(pSelect, 4);
+
      char *zFree = 0;
+
      int rc = SQLITE_OK;
+

+
      if( bVirtual ){
+
        zSql = (const char*)(zFree = recoverMPrintf(p,
+
            "INSERT INTO sqlite_schema VALUES('table', %Q, %Q, 0, %Q)",
+
            zName, zName, zSql
+
        ));
+
      }
+
      rc = sqlite3_exec(p->dbOut, zSql, 0, 0, 0);
+
      if( rc==SQLITE_OK ){
+
        recoverSqlCallback(p, zSql);
+
        if( bTable && !bVirtual ){
+
          if( SQLITE_ROW==sqlite3_step(pTblname) ){
+
            const char *zTbl = (const char*)sqlite3_column_text(pTblname, 0);
+
            recoverAddTable(p, zTbl, iRoot);
+
          }
+
          recoverReset(p, pTblname);
+
        }
+
      }else if( rc!=SQLITE_ERROR ){
+
        recoverDbError(p, p->dbOut);
+
      }
+
      sqlite3_free(zFree);
    }
-
    default:
-
      rc = SQLITE_NOTFOUND;
-
      break;
  }
+
  recoverFinalize(p, pSelect);
+
  recoverFinalize(p, pTblname);

-
  va_end(ap);
-
  return rc;
+
  return p->errCode;
}

/*
-
** Add an SQL statement to the analysis.
+
** This function is called after the output database has been populated. It
+
** adds all recovered schema elements that were not created in the output
+
** database by recoverWriteSchema1() - everything except for tables and
+
** UNIQUE indexes. Specifically:
+
**
+
**     * views,
+
**     * triggers,
+
**     * non-UNIQUE indexes.
+
**
+
** If the recover handle is in SQL callback mode, then equivalent callbacks
+
** are issued to create the schema elements.
*/
-
int sqlite3_expert_sql(
-
  sqlite3expert *p,               /* From sqlite3_expert_new() */
-
  const char *zSql,               /* SQL statement to add */
-
  char **pzErr                    /* OUT: Error message (if any) */
-
){
-
  IdxScan *pScanOrig = p->pScan;
-
  IdxStatement *pStmtOrig = p->pStatement;
-
  int rc = SQLITE_OK;
-
  const char *zStmt = zSql;
+
static int recoverWriteSchema2(sqlite3_recover *p){
+
  sqlite3_stmt *pSelect = 0;

-
  if( p->bRun ) return SQLITE_MISUSE;
+
  pSelect = recoverPrepare(p, p->dbOut,
+
      p->bSlowIndexes ?
+
      "SELECT rootpage, sql FROM recovery.schema "
+
      "  WHERE type!='table' AND type!='index'"
+
      :
+
      "SELECT rootpage, sql FROM recovery.schema "
+
      "  WHERE type!='table' AND (type!='index' OR sql NOT LIKE '%unique%')"
+
  );

-
  while( rc==SQLITE_OK && zStmt && zStmt[0] ){
-
    sqlite3_stmt *pStmt = 0;
-
    rc = sqlite3_prepare_v2(p->dbv, zStmt, -1, &pStmt, &zStmt);
-
    if( rc==SQLITE_OK ){
-
      if( pStmt ){
-
        IdxStatement *pNew;
-
        const char *z = sqlite3_sql(pStmt);
-
        int n = STRLEN(z);
-
        pNew = (IdxStatement*)idxMalloc(&rc, sizeof(IdxStatement) + n+1);
-
        if( rc==SQLITE_OK ){
-
          pNew->zSql = (char*)&pNew[1];
-
          memcpy(pNew->zSql, z, n+1);
-
          pNew->pNext = p->pStatement;
-
          if( p->pStatement ) pNew->iId = p->pStatement->iId+1;
-
          p->pStatement = pNew;
-
        }
-
        sqlite3_finalize(pStmt);
+
  if( pSelect ){
+
    while( sqlite3_step(pSelect)==SQLITE_ROW ){
+
      const char *zSql = (const char*)sqlite3_column_text(pSelect, 1);
+
      int rc = sqlite3_exec(p->dbOut, zSql, 0, 0, 0);
+
      if( rc==SQLITE_OK ){
+
        recoverSqlCallback(p, zSql);
+
      }else if( rc!=SQLITE_ERROR ){
+
        recoverDbError(p, p->dbOut);
      }
-
    }else{
-
      idxDatabaseError(p->dbv, pzErr);
    }
  }
+
  recoverFinalize(p, pSelect);

-
  if( rc!=SQLITE_OK ){
-
    idxScanFree(p->pScan, pScanOrig);
-
    idxStatementFree(p->pStatement, pStmtOrig);
-
    p->pScan = pScanOrig;
-
    p->pStatement = pStmtOrig;
-
  }
-

-
  return rc;
+
  return p->errCode;
}

-
int sqlite3_expert_analyze(sqlite3expert *p, char **pzErr){
-
  int rc;
-
  IdxHashEntry *pEntry;
+
/*
+
** This function is a no-op if recover handle p already contains an error
+
** (if p->errCode!=SQLITE_OK). In this case it returns NULL.
+
**
+
** Otherwise, if the recover handle is configured to create an output
+
** database (was created by sqlite3_recover_init()), then this function
+
** prepares and returns an SQL statement to INSERT a new record into table
+
** pTab, assuming the first nField fields of a record extracted from disk
+
** are valid.
+
**
+
** For example, if table pTab is:
+
**
+
**     CREATE TABLE name(a, b GENERATED ALWAYS AS (a+1) STORED, c, d, e);
+
**
+
** And nField is 4, then the SQL statement prepared and returned is:
+
**
+
**     INSERT INTO (a, c, d) VALUES (?1, ?2, ?3);
+
**
+
** In this case even though 4 values were extracted from the input db,
+
** only 3 are written to the output, as the generated STORED column 
+
** cannot be written.
+
**
+
** If the recover handle is in SQL callback mode, then the SQL statement
+
** prepared is such that evaluating it returns a single row containing
+
** a single text value - itself an SQL statement similar to the above,
+
** except with SQL literals in place of the variables. For example:
+
**
+
**     SELECT 'INSERT INTO (a, c, d) VALUES (' 
+
**          || quote(?1) || ', '
+
**          || quote(?2) || ', '
+
**          || quote(?3) || ')';
+
**
+
** In either case, it is the responsibility of the caller to eventually
+
** free the statement handle using sqlite3_finalize().
+
*/
+
static sqlite3_stmt *recoverInsertStmt(
+
  sqlite3_recover *p, 
+
  RecoverTable *pTab,
+
  int nField
+
){
+
  sqlite3_stmt *pRet = 0;
+
  const char *zSep = "";
+
  const char *zSqlSep = "";
+
  char *zSql = 0;
+
  char *zFinal = 0;
+
  char *zBind = 0;
+
  int ii;
+
  int bSql = p->xSql ? 1 : 0;

-
  /* Do trigger processing to collect any extra IdxScan structures */
-
  rc = idxProcessTriggers(p, pzErr);
+
  if( nField<=0 ) return 0;

-
  /* Create candidate indexes within the in-memory database file */
-
  if( rc==SQLITE_OK ){
-
    rc = idxCreateCandidates(p);
-
  }else if ( rc==SQLITE_BUSY_TIMEOUT ){
-
    if( pzErr )
-
      *pzErr = sqlite3_mprintf("Cannot find a unique index name to propose.");
-
    return rc;
-
  }
+
  assert( nField<=pTab->nCol );

-
  /* Generate the stat1 data */
-
  if( rc==SQLITE_OK ){
-
    rc = idxPopulateStat1(p, pzErr);
-
  }
+
  zSql = recoverMPrintf(p, "INSERT OR IGNORE INTO %Q(", pTab->zTab);

-
  /* Formulate the EXPERT_REPORT_CANDIDATES text */
-
  for(pEntry=p->hIdx.pFirst; pEntry; pEntry=pEntry->pNext){
-
    p->zCandidates = idxAppendText(&rc, p->zCandidates, 
-
        "%s;%s%s\n", pEntry->zVal, 
-
        pEntry->zVal2 ? " -- stat1: " : "", pEntry->zVal2
-
    );
+
  if( pTab->iRowidBind ){
+
    assert( pTab->bIntkey );
+
    zSql = recoverMPrintf(p, "%z_rowid_", zSql);
+
    if( bSql ){
+
      zBind = recoverMPrintf(p, "%zquote(?%d)", zBind, pTab->iRowidBind);
+
    }else{
+
      zBind = recoverMPrintf(p, "%z?%d", zBind, pTab->iRowidBind);
+
    }
+
    zSqlSep = "||', '||";
+
    zSep = ", ";
  }

-
  /* Figure out which of the candidate indexes are preferred by the query
-
  ** planner and report the results to the user.  */
-
  if( rc==SQLITE_OK ){
-
    rc = idxFindIndexes(p, pzErr);
+
  for(ii=0; ii<nField; ii++){
+
    int eHidden = pTab->aCol[ii].eHidden;
+
    if( eHidden!=RECOVER_EHIDDEN_VIRTUAL
+
     && eHidden!=RECOVER_EHIDDEN_STORED
+
    ){
+
      assert( pTab->aCol[ii].iField>=0 && pTab->aCol[ii].iBind>=1 );
+
      zSql = recoverMPrintf(p, "%z%s%Q", zSql, zSep, pTab->aCol[ii].zCol);
+

+
      if( bSql ){
+
        zBind = recoverMPrintf(p, 
+
            "%z%sescape_crnl(quote(?%d))", zBind, zSqlSep, pTab->aCol[ii].iBind
+
        );
+
        zSqlSep = "||', '||";
+
      }else{
+
        zBind = recoverMPrintf(p, "%z%s?%d", zBind, zSep, pTab->aCol[ii].iBind);
+
      }
+
      zSep = ", ";
+
    }
  }

-
  if( rc==SQLITE_OK ){
-
    p->bRun = 1;
+
  if( bSql ){
+
    zFinal = recoverMPrintf(p, "SELECT %Q || ') VALUES (' || %s || ')'", 
+
        zSql, zBind
+
    );
+
  }else{
+
    zFinal = recoverMPrintf(p, "%s) VALUES (%s)", zSql, zBind);
  }
-
  return rc;
+

+
  pRet = recoverPrepare(p, p->dbOut, zFinal);
+
  sqlite3_free(zSql);
+
  sqlite3_free(zBind);
+
  sqlite3_free(zFinal);
+
  
+
  return pRet;
}

+

/*
-
** Return the total number of statements that have been added to this
-
** sqlite3expert using sqlite3_expert_sql().
+
** Search the list of RecoverTable objects at p->pTblList for one that
+
** has root page iRoot in the input database. If such an object is found,
+
** return a pointer to it. Otherwise, return NULL.
*/
-
int sqlite3_expert_count(sqlite3expert *p){
-
  int nRet = 0;
-
  if( p->pStatement ) nRet = p->pStatement->iId+1;
-
  return nRet;
+
static RecoverTable *recoverFindTable(sqlite3_recover *p, u32 iRoot){
+
  RecoverTable *pRet = 0;
+
  for(pRet=p->pTblList; pRet && pRet->iRoot!=iRoot; pRet=pRet->pNext);
+
  return pRet;
}

/*
-
** Return a component of the report.
+
** This function attempts to create a lost and found table within the 
+
** output db. If successful, it returns a pointer to a buffer containing
+
** the name of the new table. It is the responsibility of the caller to
+
** eventually free this buffer using sqlite3_free().
+
**
+
** If an error occurs, NULL is returned and an error code and error 
+
** message left in the recover handle.
*/
-
const char *sqlite3_expert_report(sqlite3expert *p, int iStmt, int eReport){
-
  const char *zRet = 0;
-
  IdxStatement *pStmt;
+
static char *recoverLostAndFoundCreate(
+
  sqlite3_recover *p,             /* Recover object */
+
  int nField                      /* Number of column fields in new table */
+
){
+
  char *zTbl = 0;
+
  sqlite3_stmt *pProbe = 0;
+
  int ii = 0;

-
  if( p->bRun==0 ) return 0;
-
  for(pStmt=p->pStatement; pStmt && pStmt->iId!=iStmt; pStmt=pStmt->pNext);
-
  switch( eReport ){
-
    case EXPERT_REPORT_SQL:
-
      if( pStmt ) zRet = pStmt->zSql;
-
      break;
-
    case EXPERT_REPORT_INDEXES:
-
      if( pStmt ) zRet = pStmt->zIdx;
-
      break;
-
    case EXPERT_REPORT_PLAN:
-
      if( pStmt ) zRet = pStmt->zEQP;
-
      break;
-
    case EXPERT_REPORT_CANDIDATES:
-
      zRet = p->zCandidates;
-
      break;
+
  pProbe = recoverPrepare(p, p->dbOut,
+
    "SELECT 1 FROM sqlite_schema WHERE name=?"
+
  );
+
  for(ii=-1; zTbl==0 && p->errCode==SQLITE_OK && ii<1000; ii++){
+
    int bFail = 0;
+
    if( ii<0 ){
+
      zTbl = recoverMPrintf(p, "%s", p->zLostAndFound);
+
    }else{
+
      zTbl = recoverMPrintf(p, "%s_%d", p->zLostAndFound, ii);
+
    }
+

+
    if( p->errCode==SQLITE_OK ){
+
      sqlite3_bind_text(pProbe, 1, zTbl, -1, SQLITE_STATIC);
+
      if( SQLITE_ROW==sqlite3_step(pProbe) ){
+
        bFail = 1;
+
      }
+
      recoverReset(p, pProbe);
+
    }
+

+
    if( bFail ){
+
      sqlite3_clear_bindings(pProbe);
+
      sqlite3_free(zTbl);
+
      zTbl = 0;
+
    }
  }
-
  return zRet;
+
  recoverFinalize(p, pProbe);
+

+
  if( zTbl ){
+
    const char *zSep = 0;
+
    char *zField = 0;
+
    char *zSql = 0;
+

+
    zSep = "rootpgno INTEGER, pgno INTEGER, nfield INTEGER, id INTEGER, ";
+
    for(ii=0; p->errCode==SQLITE_OK && ii<nField; ii++){
+
      zField = recoverMPrintf(p, "%z%sc%d", zField, zSep, ii);
+
      zSep = ", ";
+
    }
+

+
    zSql = recoverMPrintf(p, "CREATE TABLE %s(%s)", zTbl, zField);
+
    sqlite3_free(zField);
+

+
    recoverExec(p, p->dbOut, zSql);
+
    recoverSqlCallback(p, zSql);
+
    sqlite3_free(zSql);
+
  }else if( p->errCode==SQLITE_OK ){
+
    recoverError(
+
        p, SQLITE_ERROR, "failed to create %s output table", p->zLostAndFound
+
    );
+
  }
+

+
  return zTbl;
}

/*
-
** Free an sqlite3expert object.
+
** Synthesize and prepare an INSERT statement to write to the lost_and_found
+
** table in the output database. The name of the table is zTab, and it has
+
** nField c* fields.
*/
-
void sqlite3_expert_destroy(sqlite3expert *p){
-
  if( p ){
-
    sqlite3_close(p->dbm);
-
    sqlite3_close(p->dbv);
-
    idxScanFree(p->pScan, 0);
-
    idxStatementFree(p->pStatement, 0);
-
    idxTableFree(p->pTable);
-
    idxWriteFree(p->pWrite);
-
    idxHashClear(&p->hIdx);
-
    sqlite3_free(p->zCandidates);
-
    sqlite3_free(p);
-
  }
-
}
+
static sqlite3_stmt *recoverLostAndFoundInsert(
+
  sqlite3_recover *p,
+
  const char *zTab,
+
  int nField
+
){
+
  int nTotal = nField + 4;
+
  int ii;
+
  char *zBind = 0;
+
  sqlite3_stmt *pRet = 0;

-
#endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */
+
  if( p->xSql==0 ){
+
    for(ii=0; ii<nTotal; ii++){
+
      zBind = recoverMPrintf(p, "%z%s?", zBind, zBind?", ":"", ii);
+
    }
+
    pRet = recoverPreparePrintf(
+
        p, p->dbOut, "INSERT INTO %s VALUES(%s)", zTab, zBind
+
    );
+
  }else{
+
    const char *zSep = "";
+
    for(ii=0; ii<nTotal; ii++){
+
      zBind = recoverMPrintf(p, "%z%squote(?)", zBind, zSep);
+
      zSep = "|| ', ' ||";
+
    }
+
    pRet = recoverPreparePrintf(
+
        p, p->dbOut, "SELECT 'INSERT INTO %s VALUES(' || %s || ')'", zTab, zBind
+
    );
+
  }

-
/************************* End ../ext/expert/sqlite3expert.c ********************/
+
  sqlite3_free(zBind);
+
  return pRet;
+
}

-
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
-
/************************* Begin ../ext/misc/dbdata.c ******************/
/*
-
** 2019-04-17
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** This file contains an implementation of two eponymous virtual tables,
-
** "sqlite_dbdata" and "sqlite_dbptr". Both modules require that the
-
** "sqlite_dbpage" eponymous virtual table be available.
-
**
-
** SQLITE_DBDATA:
-
**   sqlite_dbdata is used to extract data directly from a database b-tree
-
**   page and its associated overflow pages, bypassing the b-tree layer.
-
**   The table schema is equivalent to:
-
**
-
**     CREATE TABLE sqlite_dbdata(
-
**       pgno INTEGER,
-
**       cell INTEGER,
-
**       field INTEGER,
-
**       value ANY,
-
**       schema TEXT HIDDEN
-
**     );
-
**
-
**   IMPORTANT: THE VIRTUAL TABLE SCHEMA ABOVE IS SUBJECT TO CHANGE. IN THE
-
**   FUTURE NEW NON-HIDDEN COLUMNS MAY BE ADDED BETWEEN "value" AND
-
**   "schema".
-
**
-
**   Each page of the database is inspected. If it cannot be interpreted as
-
**   a b-tree page, or if it is a b-tree page containing 0 entries, the
-
**   sqlite_dbdata table contains no rows for that page.  Otherwise, the
-
**   table contains one row for each field in the record associated with
-
**   each cell on the page. For intkey b-trees, the key value is stored in
-
**   field -1.
-
**
-
**   For example, for the database:
-
**
-
**     CREATE TABLE t1(a, b);     -- root page is page 2
-
**     INSERT INTO t1(rowid, a, b) VALUES(5, 'v', 'five');
-
**     INSERT INTO t1(rowid, a, b) VALUES(10, 'x', 'ten');
-
**
-
**   the sqlite_dbdata table contains, as well as from entries related to 
-
**   page 1, content equivalent to:
-
**
-
**     INSERT INTO sqlite_dbdata(pgno, cell, field, value) VALUES
-
**         (2, 0, -1, 5     ),
-
**         (2, 0,  0, 'v'   ),
-
**         (2, 0,  1, 'five'),
-
**         (2, 1, -1, 10    ),
-
**         (2, 1,  0, 'x'   ),
-
**         (2, 1,  1, 'ten' );
-
**
-
**   If database corruption is encountered, this module does not report an
-
**   error. Instead, it attempts to extract as much data as possible and
-
**   ignores the corruption.
-
**
-
** SQLITE_DBPTR:
-
**   The sqlite_dbptr table has the following schema:
-
**
-
**     CREATE TABLE sqlite_dbptr(
-
**       pgno INTEGER,
-
**       child INTEGER,
-
**       schema TEXT HIDDEN
-
**     );
-
**
-
**   It contains one entry for each b-tree pointer between a parent and
-
**   child page in the database.
+
** Input database page iPg contains data that will be written to the
+
** lost-and-found table of the output database. This function attempts
+
** to identify the root page of the tree that page iPg belonged to.
+
** If successful, it sets output variable (*piRoot) to the page number
+
** of the root page and returns SQLITE_OK. Otherwise, if an error occurs,
+
** an SQLite error code is returned and the final value of *piRoot 
+
** undefined.
*/
-
#if !defined(SQLITEINT_H) 
-
/* #include "sqlite3ext.h" */
-

-
/* typedef unsigned char u8; */
-

-
#endif
-
SQLITE_EXTENSION_INIT1
-
#include <string.h>
-
#include <assert.h>
+
static int recoverLostAndFoundFindRoot(
+
  sqlite3_recover *p, 
+
  i64 iPg,
+
  i64 *piRoot
+
){
+
  RecoverStateLAF *pLaf = &p->laf;
+

+
  if( pLaf->pFindRoot==0 ){
+
    pLaf->pFindRoot = recoverPrepare(p, p->dbOut,
+
        "WITH RECURSIVE p(pgno) AS ("
+
        "  SELECT ?"
+
        "    UNION"
+
        "  SELECT parent FROM recovery.map AS m, p WHERE m.pgno=p.pgno"
+
        ") "
+
        "SELECT p.pgno FROM p, recovery.map m WHERE m.pgno=p.pgno "
+
        "    AND m.parent IS NULL"
+
    );
+
  }
+
  if( p->errCode==SQLITE_OK ){
+
    sqlite3_bind_int64(pLaf->pFindRoot, 1, iPg);
+
    if( sqlite3_step(pLaf->pFindRoot)==SQLITE_ROW ){
+
      *piRoot = sqlite3_column_int64(pLaf->pFindRoot, 0);
+
    }else{
+
      *piRoot = iPg;
+
    }
+
    recoverReset(p, pLaf->pFindRoot);
+
  }
+
  return p->errCode;
+
}

-
#define DBDATA_PADDING_BYTES 100 
+
/*
+
** Recover data from page iPage of the input database and write it to
+
** the lost-and-found table in the output database.
+
*/
+
static void recoverLostAndFoundOnePage(sqlite3_recover *p, i64 iPage){
+
  RecoverStateLAF *pLaf = &p->laf;
+
  sqlite3_value **apVal = pLaf->apVal;
+
  sqlite3_stmt *pPageData = pLaf->pPageData;
+
  sqlite3_stmt *pInsert = pLaf->pInsert;

-
typedef struct DbdataTable DbdataTable;
-
typedef struct DbdataCursor DbdataCursor;
+
  int nVal = -1;
+
  int iPrevCell = 0;
+
  i64 iRoot = 0;
+
  int bHaveRowid = 0;
+
  i64 iRowid = 0;
+
  int ii = 0;

-
/* Cursor object */
-
struct DbdataCursor {
-
  sqlite3_vtab_cursor base;       /* Base class.  Must be first */
-
  sqlite3_stmt *pStmt;            /* For fetching database pages */
+
  if( recoverLostAndFoundFindRoot(p, iPage, &iRoot) ) return;
+
  sqlite3_bind_int64(pPageData, 1, iPage);
+
  while( p->errCode==SQLITE_OK && SQLITE_ROW==sqlite3_step(pPageData) ){
+
    int iCell = sqlite3_column_int64(pPageData, 0);
+
    int iField = sqlite3_column_int64(pPageData, 1);

-
  int iPgno;                      /* Current page number */
-
  u8 *aPage;                      /* Buffer containing page */
-
  int nPage;                      /* Size of aPage[] in bytes */
-
  int nCell;                      /* Number of cells on aPage[] */
-
  int iCell;                      /* Current cell number */
-
  int bOnePage;                   /* True to stop after one page */
-
  int szDb;
-
  sqlite3_int64 iRowid;
+
    if( iPrevCell!=iCell && nVal>=0 ){
+
      /* Insert the new row */
+
      sqlite3_bind_int64(pInsert, 1, iRoot);      /* rootpgno */
+
      sqlite3_bind_int64(pInsert, 2, iPage);      /* pgno */
+
      sqlite3_bind_int(pInsert, 3, nVal);         /* nfield */
+
      if( bHaveRowid ){
+
        sqlite3_bind_int64(pInsert, 4, iRowid);   /* id */
+
      }
+
      for(ii=0; ii<nVal; ii++){
+
        recoverBindValue(p, pInsert, 5+ii, apVal[ii]);
+
      }
+
      if( sqlite3_step(pInsert)==SQLITE_ROW ){
+
        recoverSqlCallback(p, (const char*)sqlite3_column_text(pInsert, 0));
+
      }
+
      recoverReset(p, pInsert);

-
  /* Only for the sqlite_dbdata table */
-
  u8 *pRec;                       /* Buffer containing current record */
-
  int nRec;                       /* Size of pRec[] in bytes */
-
  int nHdr;                       /* Size of header in bytes */
-
  int iField;                     /* Current field number */
-
  u8 *pHdrPtr;
-
  u8 *pPtr;
-
  
-
  sqlite3_int64 iIntkey;          /* Integer key value */
-
};
+
      /* Discard the accumulated row data */
+
      for(ii=0; ii<nVal; ii++){
+
        sqlite3_value_free(apVal[ii]);
+
        apVal[ii] = 0;
+
      }
+
      sqlite3_clear_bindings(pInsert);
+
      bHaveRowid = 0;
+
      nVal = -1;
+
    }

-
/* Table object */
-
struct DbdataTable {
-
  sqlite3_vtab base;              /* Base class.  Must be first */
-
  sqlite3 *db;                    /* The database connection */
-
  sqlite3_stmt *pStmt;            /* For fetching database pages */
-
  int bPtr;                       /* True for sqlite3_dbptr table */
-
};
+
    if( iCell<0 ) break;

-
/* Column and schema definitions for sqlite_dbdata */
-
#define DBDATA_COLUMN_PGNO        0
-
#define DBDATA_COLUMN_CELL        1
-
#define DBDATA_COLUMN_FIELD       2
-
#define DBDATA_COLUMN_VALUE       3
-
#define DBDATA_COLUMN_SCHEMA      4
-
#define DBDATA_SCHEMA             \
-
      "CREATE TABLE x("           \
-
      "  pgno INTEGER,"           \
-
      "  cell INTEGER,"           \
-
      "  field INTEGER,"          \
-
      "  value ANY,"              \
-
      "  schema TEXT HIDDEN"      \
-
      ")"
+
    if( iField<0 ){
+
      assert( nVal==-1 );
+
      iRowid = sqlite3_column_int64(pPageData, 2);
+
      bHaveRowid = 1;
+
      nVal = 0;
+
    }else if( iField<pLaf->nMaxField ){
+
      sqlite3_value *pVal = sqlite3_column_value(pPageData, 2);
+
      apVal[iField] = sqlite3_value_dup(pVal);
+
      assert( iField==nVal || (nVal==-1 && iField==0) );
+
      nVal = iField+1;
+
      if( apVal[iField]==0 ){
+
        recoverError(p, SQLITE_NOMEM, 0);
+
      }
+
    }

-
/* Column and schema definitions for sqlite_dbptr */
-
#define DBPTR_COLUMN_PGNO         0
-
#define DBPTR_COLUMN_CHILD        1
-
#define DBPTR_COLUMN_SCHEMA       2
-
#define DBPTR_SCHEMA              \
-
      "CREATE TABLE x("           \
-
      "  pgno INTEGER,"           \
-
      "  child INTEGER,"          \
-
      "  schema TEXT HIDDEN"      \
-
      ")"
+
    iPrevCell = iCell;
+
  }
+
  recoverReset(p, pPageData);

-
/*
-
** Connect to an sqlite_dbdata (pAux==0) or sqlite_dbptr (pAux!=0) virtual 
-
** table.
-
*/
-
static int dbdataConnect(
-
  sqlite3 *db,
-
  void *pAux,
-
  int argc, const char *const*argv,
-
  sqlite3_vtab **ppVtab,
-
  char **pzErr
-
){
-
  DbdataTable *pTab = 0;
-
  int rc = sqlite3_declare_vtab(db, pAux ? DBPTR_SCHEMA : DBDATA_SCHEMA);
+
  for(ii=0; ii<nVal; ii++){
+
    sqlite3_value_free(apVal[ii]);
+
    apVal[ii] = 0;
+
  }
+
}

-
  if( rc==SQLITE_OK ){
-
    pTab = (DbdataTable*)sqlite3_malloc64(sizeof(DbdataTable));
-
    if( pTab==0 ){
-
      rc = SQLITE_NOMEM;
+
/*
+
** Perform one step (sqlite3_recover_step()) of work for the connection 
+
** passed as the only argument, which is guaranteed to be in
+
** RECOVER_STATE_LOSTANDFOUND3 state - during which the lost-and-found 
+
** table of the output database is populated with recovered data that can 
+
** not be assigned to any recovered schema object.
+
*/ 
+
static int recoverLostAndFound3Step(sqlite3_recover *p){
+
  RecoverStateLAF *pLaf = &p->laf;
+
  if( p->errCode==SQLITE_OK ){
+
    if( pLaf->pInsert==0 ){
+
      return SQLITE_DONE;
    }else{
-
      memset(pTab, 0, sizeof(DbdataTable));
-
      pTab->db = db;
-
      pTab->bPtr = (pAux!=0);
+
      if( p->errCode==SQLITE_OK ){
+
        int res = sqlite3_step(pLaf->pAllPage);
+
        if( res==SQLITE_ROW ){
+
          i64 iPage = sqlite3_column_int64(pLaf->pAllPage, 0);
+
          if( recoverBitmapQuery(pLaf->pUsed, iPage)==0 ){
+
            recoverLostAndFoundOnePage(p, iPage);
+
          }
+
        }else{
+
          recoverReset(p, pLaf->pAllPage);
+
          return SQLITE_DONE;
+
        }
+
      }
    }
  }
+
  return SQLITE_OK;
+
}

-
  *ppVtab = (sqlite3_vtab*)pTab;
-
  return rc;
+
/*
+
** Initialize resources required in RECOVER_STATE_LOSTANDFOUND3 
+
** state - during which the lost-and-found table of the output database 
+
** is populated with recovered data that can not be assigned to any 
+
** recovered schema object.
+
*/ 
+
static void recoverLostAndFound3Init(sqlite3_recover *p){
+
  RecoverStateLAF *pLaf = &p->laf;
+

+
  if( pLaf->nMaxField>0 ){
+
    char *zTab = 0;               /* Name of lost_and_found table */
+

+
    zTab = recoverLostAndFoundCreate(p, pLaf->nMaxField);
+
    pLaf->pInsert = recoverLostAndFoundInsert(p, zTab, pLaf->nMaxField);
+
    sqlite3_free(zTab);
+

+
    pLaf->pAllPage = recoverPreparePrintf(p, p->dbOut,
+
        "WITH RECURSIVE seq(ii) AS ("
+
        "  SELECT 1 UNION ALL SELECT ii+1 FROM seq WHERE ii<%lld"
+
        ")"
+
        "SELECT ii FROM seq" , p->laf.nPg
+
    );
+
    pLaf->pPageData = recoverPrepare(p, p->dbOut,
+
        "SELECT cell, field, value "
+
        "FROM sqlite_dbdata('getpage()') d WHERE d.pgno=? "
+
        "UNION ALL "
+
        "SELECT -1, -1, -1"
+
    );
+

+
    pLaf->apVal = (sqlite3_value**)recoverMalloc(p, 
+
        pLaf->nMaxField*sizeof(sqlite3_value*)
+
    );
+
  }
}

/*
-
** Disconnect from or destroy a sqlite_dbdata or sqlite_dbptr virtual table.
-
*/
-
static int dbdataDisconnect(sqlite3_vtab *pVtab){
-
  DbdataTable *pTab = (DbdataTable*)pVtab;
-
  if( pTab ){
-
    sqlite3_finalize(pTab->pStmt);
-
    sqlite3_free(pVtab);
+
** Initialize resources required in RECOVER_STATE_WRITING state - during which
+
** tables recovered from the schema of the input database are populated with
+
** recovered data.
+
*/ 
+
static int recoverWriteDataInit(sqlite3_recover *p){
+
  RecoverStateW1 *p1 = &p->w1;
+
  RecoverTable *pTbl = 0;
+
  int nByte = 0;
+

+
  /* Figure out the maximum number of columns for any table in the schema */
+
  assert( p1->nMax==0 );
+
  for(pTbl=p->pTblList; pTbl; pTbl=pTbl->pNext){
+
    if( pTbl->nCol>p1->nMax ) p1->nMax = pTbl->nCol;
  }
-
  return SQLITE_OK;
+

+
  /* Allocate an array of (sqlite3_value*) in which to accumulate the values
+
  ** that will be written to the output database in a single row. */
+
  nByte = sizeof(sqlite3_value*) * (p1->nMax+1);
+
  p1->apVal = (sqlite3_value**)recoverMalloc(p, nByte);
+
  if( p1->apVal==0 ) return p->errCode;
+

+
  /* Prepare the SELECT to loop through schema tables (pTbls) and the SELECT
+
  ** to loop through cells that appear to belong to a single table (pSel). */
+
  p1->pTbls = recoverPrepare(p, p->dbOut,
+
      "SELECT rootpage FROM recovery.schema "
+
      "  WHERE type='table' AND (sql NOT LIKE 'create virtual%')"
+
      "  ORDER BY (tbl_name='sqlite_sequence') ASC"
+
  );
+
  p1->pSel = recoverPrepare(p, p->dbOut, 
+
      "WITH RECURSIVE pages(page) AS ("
+
      "  SELECT ?1"
+
      "    UNION"
+
      "  SELECT child FROM sqlite_dbptr('getpage()'), pages "
+
      "    WHERE pgno=page"
+
      ") "
+
      "SELECT page, cell, field, value "
+
      "FROM sqlite_dbdata('getpage()') d, pages p WHERE p.page=d.pgno "
+
      "UNION ALL "
+
      "SELECT 0, 0, 0, 0"
+
  );
+

+
  return p->errCode;
}

/*
-
** This function interprets two types of constraints:
-
**
-
**       schema=?
-
**       pgno=?
-
**
-
** If neither are present, idxNum is set to 0. If schema=? is present,
-
** the 0x01 bit in idxNum is set. If pgno=? is present, the 0x02 bit
-
** in idxNum is set.
-
**
-
** If both parameters are present, schema is in position 0 and pgno in
-
** position 1.
+
** Clean up resources allocated by recoverWriteDataInit() (stuff in 
+
** sqlite3_recover.w1).
*/
-
static int dbdataBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdx){
-
  DbdataTable *pTab = (DbdataTable*)tab;
-
  int i;
-
  int iSchema = -1;
-
  int iPgno = -1;
-
  int colSchema = (pTab->bPtr ? DBPTR_COLUMN_SCHEMA : DBDATA_COLUMN_SCHEMA);
+
static void recoverWriteDataCleanup(sqlite3_recover *p){
+
  RecoverStateW1 *p1 = &p->w1;
+
  int ii;
+
  for(ii=0; ii<p1->nVal; ii++){
+
    sqlite3_value_free(p1->apVal[ii]);
+
  }
+
  sqlite3_free(p1->apVal);
+
  recoverFinalize(p, p1->pInsert);
+
  recoverFinalize(p, p1->pTbls);
+
  recoverFinalize(p, p1->pSel);
+
  memset(p1, 0, sizeof(*p1));
+
}

-
  for(i=0; i<pIdx->nConstraint; i++){
-
    struct sqlite3_index_constraint *p = &pIdx->aConstraint[i];
-
    if( p->op==SQLITE_INDEX_CONSTRAINT_EQ ){
-
      if( p->iColumn==colSchema ){
-
        if( p->usable==0 ) return SQLITE_CONSTRAINT;
-
        iSchema = i;
-
      }
-
      if( p->iColumn==DBDATA_COLUMN_PGNO && p->usable ){
-
        iPgno = i;
+
/*
+
** Perform one step (sqlite3_recover_step()) of work for the connection 
+
** passed as the only argument, which is guaranteed to be in
+
** RECOVER_STATE_WRITING state - during which tables recovered from the
+
** schema of the input database are populated with recovered data.
+
*/ 
+
static int recoverWriteDataStep(sqlite3_recover *p){
+
  RecoverStateW1 *p1 = &p->w1;
+
  sqlite3_stmt *pSel = p1->pSel;
+
  sqlite3_value **apVal = p1->apVal;
+

+
  if( p->errCode==SQLITE_OK && p1->pTab==0 ){
+
    if( sqlite3_step(p1->pTbls)==SQLITE_ROW ){
+
      i64 iRoot = sqlite3_column_int64(p1->pTbls, 0);
+
      p1->pTab = recoverFindTable(p, iRoot);
+

+
      recoverFinalize(p, p1->pInsert);
+
      p1->pInsert = 0;
+

+
      /* If this table is unknown, return early. The caller will invoke this
+
      ** function again and it will move on to the next table.  */
+
      if( p1->pTab==0 ) return p->errCode;
+

+
      /* If this is the sqlite_sequence table, delete any rows added by
+
      ** earlier INSERT statements on tables with AUTOINCREMENT primary
+
      ** keys before recovering its contents. The p1->pTbls SELECT statement
+
      ** is rigged to deliver "sqlite_sequence" last of all, so we don't
+
      ** worry about it being modified after it is recovered. */
+
      if( sqlite3_stricmp("sqlite_sequence", p1->pTab->zTab)==0 ){
+
        recoverExec(p, p->dbOut, "DELETE FROM sqlite_sequence");
+
        recoverSqlCallback(p, "DELETE FROM sqlite_sequence");
      }
+

+
      /* Bind the root page of this table within the original database to 
+
      ** SELECT statement p1->pSel. The SELECT statement will then iterate
+
      ** through cells that look like they belong to table pTab.  */
+
      sqlite3_bind_int64(pSel, 1, iRoot);
+

+
      p1->nVal = 0;
+
      p1->bHaveRowid = 0;
+
      p1->iPrevPage = -1;
+
      p1->iPrevCell = -1;
+
    }else{
+
      return SQLITE_DONE;
    }
  }
+
  assert( p->errCode!=SQLITE_OK || p1->pTab );

-
  if( iSchema>=0 ){
-
    pIdx->aConstraintUsage[iSchema].argvIndex = 1;
-
    pIdx->aConstraintUsage[iSchema].omit = 1;
-
  }
-
  if( iPgno>=0 ){
-
    pIdx->aConstraintUsage[iPgno].argvIndex = 1 + (iSchema>=0);
-
    pIdx->aConstraintUsage[iPgno].omit = 1;
-
    pIdx->estimatedCost = 100;
-
    pIdx->estimatedRows =  50;
+
  if( p->errCode==SQLITE_OK && sqlite3_step(pSel)==SQLITE_ROW ){
+
    RecoverTable *pTab = p1->pTab;

-
    if( pTab->bPtr==0 && pIdx->nOrderBy && pIdx->aOrderBy[0].desc==0 ){
-
      int iCol = pIdx->aOrderBy[0].iColumn;
-
      if( pIdx->nOrderBy==1 ){
-
        pIdx->orderByConsumed = (iCol==0 || iCol==1);
-
      }else if( pIdx->nOrderBy==2 && pIdx->aOrderBy[1].desc==0 && iCol==0 ){
-
        pIdx->orderByConsumed = (pIdx->aOrderBy[1].iColumn==1);
+
    i64 iPage = sqlite3_column_int64(pSel, 0);
+
    int iCell = sqlite3_column_int(pSel, 1);
+
    int iField = sqlite3_column_int(pSel, 2);
+
    sqlite3_value *pVal = sqlite3_column_value(pSel, 3);
+
    int bNewCell = (p1->iPrevPage!=iPage || p1->iPrevCell!=iCell);
+

+
    assert( bNewCell==0 || (iField==-1 || iField==0) );
+
    assert( bNewCell || iField==p1->nVal || p1->nVal==pTab->nCol );
+

+
    if( bNewCell ){
+
      int ii = 0;
+
      if( p1->nVal>=0 ){
+
        if( p1->pInsert==0 || p1->nVal!=p1->nInsert ){
+
          recoverFinalize(p, p1->pInsert);
+
          p1->pInsert = recoverInsertStmt(p, pTab, p1->nVal);
+
          p1->nInsert = p1->nVal;
+
        }
+
        if( p1->nVal>0 ){
+
          sqlite3_stmt *pInsert = p1->pInsert;
+
          for(ii=0; ii<pTab->nCol; ii++){
+
            RecoverColumn *pCol = &pTab->aCol[ii];
+
            int iBind = pCol->iBind;
+
            if( iBind>0 ){
+
              if( pCol->bIPK ){
+
                sqlite3_bind_int64(pInsert, iBind, p1->iRowid);
+
              }else if( pCol->iField<p1->nVal ){
+
                recoverBindValue(p, pInsert, iBind, apVal[pCol->iField]);
+
              }
+
            }
+
          }
+
          if( p->bRecoverRowid && pTab->iRowidBind>0 && p1->bHaveRowid ){
+
            sqlite3_bind_int64(pInsert, pTab->iRowidBind, p1->iRowid);
+
          }
+
          if( SQLITE_ROW==sqlite3_step(pInsert) ){
+
            const char *z = (const char*)sqlite3_column_text(pInsert, 0);
+
            recoverSqlCallback(p, z);
+
          }
+
          recoverReset(p, pInsert);
+
          assert( p->errCode || pInsert );
+
          if( pInsert ) sqlite3_clear_bindings(pInsert);
+
        }
+
      }
+

+
      for(ii=0; ii<p1->nVal; ii++){
+
        sqlite3_value_free(apVal[ii]);
+
        apVal[ii] = 0;
      }
+
      p1->nVal = -1;
+
      p1->bHaveRowid = 0;
    }

+
    if( iPage!=0 ){
+
      if( iField<0 ){
+
        p1->iRowid = sqlite3_column_int64(pSel, 3);
+
        assert( p1->nVal==-1 );
+
        p1->nVal = 0;
+
        p1->bHaveRowid = 1;
+
      }else if( iField<pTab->nCol ){
+
        assert( apVal[iField]==0 );
+
        apVal[iField] = sqlite3_value_dup( pVal );
+
        if( apVal[iField]==0 ){
+
          recoverError(p, SQLITE_NOMEM, 0);
+
        }
+
        p1->nVal = iField+1;
+
      }
+
      p1->iPrevCell = iCell;
+
      p1->iPrevPage = iPage;
+
    }
  }else{
-
    pIdx->estimatedCost = 100000000;
-
    pIdx->estimatedRows = 1000000000;
+
    recoverReset(p, pSel);
+
    p1->pTab = 0;
  }
-
  pIdx->idxNum = (iSchema>=0 ? 0x01 : 0x00) | (iPgno>=0 ? 0x02 : 0x00);
-
  return SQLITE_OK;
+

+
  return p->errCode;
}

/*
-
** Open a new sqlite_dbdata or sqlite_dbptr cursor.
-
*/
-
static int dbdataOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
-
  DbdataCursor *pCsr;
+
** Initialize resources required by sqlite3_recover_step() in
+
** RECOVER_STATE_LOSTANDFOUND1 state - during which the set of pages not
+
** already allocated to a recovered schema element is determined.
+
*/ 
+
static void recoverLostAndFound1Init(sqlite3_recover *p){
+
  RecoverStateLAF *pLaf = &p->laf;
+
  sqlite3_stmt *pStmt = 0;

-
  pCsr = (DbdataCursor*)sqlite3_malloc64(sizeof(DbdataCursor));
-
  if( pCsr==0 ){
-
    return SQLITE_NOMEM;
-
  }else{
-
    memset(pCsr, 0, sizeof(DbdataCursor));
-
    pCsr->base.pVtab = pVTab;
+
  assert( p->laf.pUsed==0 );
+
  pLaf->nPg = recoverPageCount(p);
+
  pLaf->pUsed = recoverBitmapAlloc(p, pLaf->nPg);
+

+
  /* Prepare a statement to iterate through all pages that are part of any tree
+
  ** in the recoverable part of the input database schema to the bitmap. And,
+
  ** if !p->bFreelistCorrupt, add all pages that appear to be part of the
+
  ** freelist.  */
+
  pStmt = recoverPrepare(
+
      p, p->dbOut,
+
      "WITH trunk(pgno) AS ("
+
      "  SELECT read_i32(getpage(1), 8) AS x WHERE x>0"
+
      "    UNION"
+
      "  SELECT read_i32(getpage(trunk.pgno), 0) AS x FROM trunk WHERE x>0"
+
      "),"
+
      "trunkdata(pgno, data) AS ("
+
      "  SELECT pgno, getpage(pgno) FROM trunk"
+
      "),"
+
      "freelist(data, n, freepgno) AS ("
+
      "  SELECT data, min(16384, read_i32(data, 1)-1), pgno FROM trunkdata"
+
      "    UNION ALL"
+
      "  SELECT data, n-1, read_i32(data, 2+n) FROM freelist WHERE n>=0"
+
      "),"
+
      ""
+
      "roots(r) AS ("
+
      "  SELECT 1 UNION ALL"
+
      "  SELECT rootpage FROM recovery.schema WHERE rootpage>0"
+
      "),"
+
      "used(page) AS ("
+
      "  SELECT r FROM roots"
+
      "    UNION"
+
      "  SELECT child FROM sqlite_dbptr('getpage()'), used "
+
      "    WHERE pgno=page"
+
      ") "
+
      "SELECT page FROM used"
+
      " UNION ALL "
+
      "SELECT freepgno FROM freelist WHERE NOT ?"
+
  );
+
  if( pStmt ) sqlite3_bind_int(pStmt, 1, p->bFreelistCorrupt);
+
  pLaf->pUsedPages = pStmt;
+
}
+

+
/*
+
** Perform one step (sqlite3_recover_step()) of work for the connection 
+
** passed as the only argument, which is guaranteed to be in
+
** RECOVER_STATE_LOSTANDFOUND1 state - during which the set of pages not
+
** already allocated to a recovered schema element is determined.
+
*/ 
+
static int recoverLostAndFound1Step(sqlite3_recover *p){
+
  RecoverStateLAF *pLaf = &p->laf;
+
  int rc = p->errCode;
+
  if( rc==SQLITE_OK ){
+
    rc = sqlite3_step(pLaf->pUsedPages);
+
    if( rc==SQLITE_ROW ){
+
      i64 iPg = sqlite3_column_int64(pLaf->pUsedPages, 0);
+
      recoverBitmapSet(pLaf->pUsed, iPg);
+
      rc = SQLITE_OK;
+
    }else{
+
      recoverFinalize(p, pLaf->pUsedPages);
+
      pLaf->pUsedPages = 0;
+
    }
  }
+
  return rc;
+
}

-
  *ppCursor = (sqlite3_vtab_cursor *)pCsr;
-
  return SQLITE_OK;
+
/*
+
** Initialize resources required by RECOVER_STATE_LOSTANDFOUND2 
+
** state - during which the pages identified in RECOVER_STATE_LOSTANDFOUND1
+
** are sorted into sets that likely belonged to the same database tree.
+
*/ 
+
static void recoverLostAndFound2Init(sqlite3_recover *p){
+
  RecoverStateLAF *pLaf = &p->laf;
+

+
  assert( p->laf.pAllAndParent==0 );
+
  assert( p->laf.pMapInsert==0 );
+
  assert( p->laf.pMaxField==0 );
+
  assert( p->laf.nMaxField==0 );
+

+
  pLaf->pMapInsert = recoverPrepare(p, p->dbOut,
+
      "INSERT OR IGNORE INTO recovery.map(pgno, parent) VALUES(?, ?)"
+
  );
+
  pLaf->pAllAndParent = recoverPreparePrintf(p, p->dbOut,
+
      "WITH RECURSIVE seq(ii) AS ("
+
      "  SELECT 1 UNION ALL SELECT ii+1 FROM seq WHERE ii<%lld"
+
      ")"
+
      "SELECT pgno, child FROM sqlite_dbptr('getpage()') "
+
      " UNION ALL "
+
      "SELECT NULL, ii FROM seq", p->laf.nPg
+
  );
+
  pLaf->pMaxField = recoverPreparePrintf(p, p->dbOut,
+
      "SELECT max(field)+1 FROM sqlite_dbdata('getpage') WHERE pgno = ?"
+
  );
}

/*
-
** Restore a cursor object to the state it was in when first allocated 
-
** by dbdataOpen().
-
*/
-
static void dbdataResetCursor(DbdataCursor *pCsr){
-
  DbdataTable *pTab = (DbdataTable*)(pCsr->base.pVtab);
-
  if( pTab->pStmt==0 ){
-
    pTab->pStmt = pCsr->pStmt;
-
  }else{
-
    sqlite3_finalize(pCsr->pStmt);
+
** Perform one step (sqlite3_recover_step()) of work for the connection 
+
** passed as the only argument, which is guaranteed to be in
+
** RECOVER_STATE_LOSTANDFOUND2 state - during which the pages identified 
+
** in RECOVER_STATE_LOSTANDFOUND1 are sorted into sets that likely belonged 
+
** to the same database tree.
+
*/ 
+
static int recoverLostAndFound2Step(sqlite3_recover *p){
+
  RecoverStateLAF *pLaf = &p->laf;
+
  if( p->errCode==SQLITE_OK ){
+
    int res = sqlite3_step(pLaf->pAllAndParent);
+
    if( res==SQLITE_ROW ){
+
      i64 iChild = sqlite3_column_int(pLaf->pAllAndParent, 1);
+
      if( recoverBitmapQuery(pLaf->pUsed, iChild)==0 ){
+
        sqlite3_bind_int64(pLaf->pMapInsert, 1, iChild);
+
        sqlite3_bind_value(pLaf->pMapInsert, 2, 
+
            sqlite3_column_value(pLaf->pAllAndParent, 0)
+
        );
+
        sqlite3_step(pLaf->pMapInsert);
+
        recoverReset(p, pLaf->pMapInsert);
+
        sqlite3_bind_int64(pLaf->pMaxField, 1, iChild);
+
        if( SQLITE_ROW==sqlite3_step(pLaf->pMaxField) ){
+
          int nMax = sqlite3_column_int(pLaf->pMaxField, 0);
+
          if( nMax>pLaf->nMaxField ) pLaf->nMaxField = nMax;
+
        }
+
        recoverReset(p, pLaf->pMaxField);
+
      }
+
    }else{
+
      recoverFinalize(p, pLaf->pAllAndParent);
+
      pLaf->pAllAndParent =0;
+
      return SQLITE_DONE;
+
    }
  }
-
  pCsr->pStmt = 0;
-
  pCsr->iPgno = 1;
-
  pCsr->iCell = 0;
-
  pCsr->iField = 0;
-
  pCsr->bOnePage = 0;
-
  sqlite3_free(pCsr->aPage);
-
  sqlite3_free(pCsr->pRec);
-
  pCsr->pRec = 0;
-
  pCsr->aPage = 0;
+
  return p->errCode;
}

/*
-
** Close an sqlite_dbdata or sqlite_dbptr cursor.
+
** Free all resources allocated as part of sqlite3_recover_step() calls
+
** in one of the RECOVER_STATE_LOSTANDFOUND[123] states.
*/
-
static int dbdataClose(sqlite3_vtab_cursor *pCursor){
-
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
-
  dbdataResetCursor(pCsr);
-
  sqlite3_free(pCsr);
-
  return SQLITE_OK;
+
static void recoverLostAndFoundCleanup(sqlite3_recover *p){
+
  recoverBitmapFree(p->laf.pUsed);
+
  p->laf.pUsed = 0;
+
  sqlite3_finalize(p->laf.pUsedPages);
+
  sqlite3_finalize(p->laf.pAllAndParent);
+
  sqlite3_finalize(p->laf.pMapInsert);
+
  sqlite3_finalize(p->laf.pMaxField);
+
  sqlite3_finalize(p->laf.pFindRoot);
+
  sqlite3_finalize(p->laf.pInsert);
+
  sqlite3_finalize(p->laf.pAllPage);
+
  sqlite3_finalize(p->laf.pPageData);
+
  p->laf.pUsedPages = 0;
+
  p->laf.pAllAndParent = 0;
+
  p->laf.pMapInsert = 0;
+
  p->laf.pMaxField = 0;
+
  p->laf.pFindRoot = 0;
+
  p->laf.pInsert = 0;
+
  p->laf.pAllPage = 0;
+
  p->laf.pPageData = 0;
+
  sqlite3_free(p->laf.apVal);
+
  p->laf.apVal = 0;
}

-
/* 
-
** Utility methods to decode 16 and 32-bit big-endian unsigned integers. 
+
/*
+
** Free all resources allocated as part of sqlite3_recover_step() calls.
*/
-
static unsigned int get_uint16(unsigned char *a){
-
  return (a[0]<<8)|a[1];
+
static void recoverFinalCleanup(sqlite3_recover *p){
+
  RecoverTable *pTab = 0;
+
  RecoverTable *pNext = 0;
+

+
  recoverWriteDataCleanup(p);
+
  recoverLostAndFoundCleanup(p);
+

+
  for(pTab=p->pTblList; pTab; pTab=pNext){
+
    pNext = pTab->pNext;
+
    sqlite3_free(pTab);
+
  }
+
  p->pTblList = 0;
+
  sqlite3_finalize(p->pGetPage);
+
  p->pGetPage = 0;
+

+
  {
+
#ifndef NDEBUG
+
    int res = 
+
#endif
+
       sqlite3_close(p->dbOut);
+
    assert( res==SQLITE_OK );
+
  }
+
  p->dbOut = 0;
}
-
static unsigned int get_uint32(unsigned char *a){
-
  return ((unsigned int)a[0]<<24)
-
       | ((unsigned int)a[1]<<16)
-
       | ((unsigned int)a[2]<<8)
-
       | ((unsigned int)a[3]);
+

+
/*
+
** Decode and return an unsigned 16-bit big-endian integer value from 
+
** buffer a[].
+
*/
+
static u32 recoverGetU16(const u8 *a){
+
  return (((u32)a[0])<<8) + ((u32)a[1]);
}

/*
-
** Load page pgno from the database via the sqlite_dbpage virtual table.
-
** If successful, set (*ppPage) to point to a buffer containing the page
-
** data, (*pnPage) to the size of that buffer in bytes and return
-
** SQLITE_OK. In this case it is the responsibility of the caller to
-
** eventually free the buffer using sqlite3_free().
-
**
-
** Or, if an error occurs, set both (*ppPage) and (*pnPage) to 0 and
-
** return an SQLite error code.
+
** Decode and return an unsigned 32-bit big-endian integer value from 
+
** buffer a[].
*/
-
static int dbdataLoadPage(
-
  DbdataCursor *pCsr,             /* Cursor object */
-
  unsigned int pgno,              /* Page number of page to load */
-
  u8 **ppPage,                    /* OUT: pointer to page buffer */
-
  int *pnPage                     /* OUT: Size of (*ppPage) in bytes */
-
){
-
  int rc2;
-
  int rc = SQLITE_OK;
-
  sqlite3_stmt *pStmt = pCsr->pStmt;
+
static u32 recoverGetU32(const u8 *a){
+
  return (((u32)a[0])<<24) + (((u32)a[1])<<16) + (((u32)a[2])<<8) + ((u32)a[3]);
+
}

-
  *ppPage = 0;
-
  *pnPage = 0;
-
  sqlite3_bind_int64(pStmt, 2, pgno);
-
  if( SQLITE_ROW==sqlite3_step(pStmt) ){
-
    int nCopy = sqlite3_column_bytes(pStmt, 0);
-
    if( nCopy>0 ){
-
      u8 *pPage;
-
      pPage = (u8*)sqlite3_malloc64(nCopy + DBDATA_PADDING_BYTES);
-
      if( pPage==0 ){
-
        rc = SQLITE_NOMEM;
+
/*
+
** Decode an SQLite varint from buffer a[]. Write the decoded value to (*pVal)
+
** and return the number of bytes consumed.
+
*/
+
static int recoverGetVarint(const u8 *a, i64 *pVal){
+
  sqlite3_uint64 u = 0;
+
  int i;
+
  for(i=0; i<8; i++){
+
    u = (u<<7) + (a[i]&0x7f);
+
    if( (a[i]&0x80)==0 ){ *pVal = (sqlite3_int64)u; return i+1; }
+
  }
+
  u = (u<<8) + (a[i]&0xff);
+
  *pVal = (sqlite3_int64)u;
+
  return 9;
+
}
+

+
/*
+
** The second argument points to a buffer n bytes in size. If this buffer
+
** or a prefix thereof appears to contain a well-formed SQLite b-tree page, 
+
** return the page-size in bytes. Otherwise, if the buffer does not 
+
** appear to contain a well-formed b-tree page, return 0.
+
*/
+
static int recoverIsValidPage(u8 *aTmp, const u8 *a, int n){
+
  u8 *aUsed = aTmp;
+
  int nFrag = 0;
+
  int nActual = 0;
+
  int iFree = 0;
+
  int nCell = 0;                  /* Number of cells on page */
+
  int iCellOff = 0;               /* Offset of cell array in page */
+
  int iContent = 0;
+
  int eType = 0;
+
  int ii = 0;
+

+
  eType = (int)a[0];
+
  if( eType!=0x02 && eType!=0x05 && eType!=0x0A && eType!=0x0D ) return 0;
+

+
  iFree = (int)recoverGetU16(&a[1]);
+
  nCell = (int)recoverGetU16(&a[3]);
+
  iContent = (int)recoverGetU16(&a[5]);
+
  if( iContent==0 ) iContent = 65536;
+
  nFrag = (int)a[7];
+

+
  if( iContent>n ) return 0;
+

+
  memset(aUsed, 0, n);
+
  memset(aUsed, 0xFF, iContent);
+

+
  /* Follow the free-list. This is the same format for all b-tree pages. */
+
  if( iFree && iFree<=iContent ) return 0;
+
  while( iFree ){
+
    int iNext = 0;
+
    int nByte = 0;
+
    if( iFree>(n-4) ) return 0;
+
    iNext = recoverGetU16(&a[iFree]);
+
    nByte = recoverGetU16(&a[iFree+2]);
+
    if( iFree+nByte>n ) return 0;
+
    if( iNext && iNext<iFree+nByte ) return 0;
+
    memset(&aUsed[iFree], 0xFF, nByte);
+
    iFree = iNext;
+
  }
+

+
  /* Run through the cells */
+
  if( eType==0x02 || eType==0x05 ){
+
    iCellOff = 12;
+
  }else{
+
    iCellOff = 8;
+
  }
+
  if( (iCellOff + 2*nCell)>iContent ) return 0;
+
  for(ii=0; ii<nCell; ii++){
+
    int iByte;
+
    i64 nPayload = 0;
+
    int nByte = 0;
+
    int iOff = recoverGetU16(&a[iCellOff + 2*ii]);
+
    if( iOff<iContent || iOff>n ){
+
      return 0;
+
    }
+
    if( eType==0x05 || eType==0x02 ) nByte += 4;
+
    nByte += recoverGetVarint(&a[iOff+nByte], &nPayload);
+
    if( eType==0x0D ){
+
      i64 dummy = 0;
+
      nByte += recoverGetVarint(&a[iOff+nByte], &dummy);
+
    }
+
    if( eType!=0x05 ){
+
      int X = (eType==0x0D) ? n-35 : (((n-12)*64/255)-23);
+
      int M = ((n-12)*32/255)-23;
+
      int K = M+((nPayload-M)%(n-4));
+

+
      if( nPayload<X ){
+
        nByte += nPayload;
+
      }else if( K<=X ){
+
        nByte += K+4;
      }else{
-
        const u8 *pCopy = sqlite3_column_blob(pStmt, 0);
-
        memcpy(pPage, pCopy, nCopy);
-
        memset(&pPage[nCopy], 0, DBDATA_PADDING_BYTES);
+
        nByte += M+4;
      }
-
      *ppPage = pPage;
-
      *pnPage = nCopy;
    }
-
  }
-
  rc2 = sqlite3_reset(pStmt);
-
  if( rc==SQLITE_OK ) rc = rc2;

-
  return rc;
+
    if( iOff+nByte>n ){
+
      return 0;
+
    }
+
    for(iByte=iOff; iByte<(iOff+nByte); iByte++){
+
      if( aUsed[iByte]!=0 ){
+
        return 0;
+
      }
+
      aUsed[iByte] = 0xFF;
+
    }
+
  }
+

+
  nActual = 0;
+
  for(ii=0; ii<n; ii++){
+
    if( aUsed[ii]==0 ) nActual++;
+
  }
+
  return (nActual==nFrag);
+
}
+

+

+
static int recoverVfsClose(sqlite3_file*);
+
static int recoverVfsRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
+
static int recoverVfsWrite(sqlite3_file*, const void*, int, sqlite3_int64);
+
static int recoverVfsTruncate(sqlite3_file*, sqlite3_int64 size);
+
static int recoverVfsSync(sqlite3_file*, int flags);
+
static int recoverVfsFileSize(sqlite3_file*, sqlite3_int64 *pSize);
+
static int recoverVfsLock(sqlite3_file*, int);
+
static int recoverVfsUnlock(sqlite3_file*, int);
+
static int recoverVfsCheckReservedLock(sqlite3_file*, int *pResOut);
+
static int recoverVfsFileControl(sqlite3_file*, int op, void *pArg);
+
static int recoverVfsSectorSize(sqlite3_file*);
+
static int recoverVfsDeviceCharacteristics(sqlite3_file*);
+
static int recoverVfsShmMap(sqlite3_file*, int, int, int, void volatile**);
+
static int recoverVfsShmLock(sqlite3_file*, int offset, int n, int flags);
+
static void recoverVfsShmBarrier(sqlite3_file*);
+
static int recoverVfsShmUnmap(sqlite3_file*, int deleteFlag);
+
static int recoverVfsFetch(sqlite3_file*, sqlite3_int64, int, void**);
+
static int recoverVfsUnfetch(sqlite3_file *pFd, sqlite3_int64 iOff, void *p);
+

+
static sqlite3_io_methods recover_methods = {
+
  2, /* iVersion */
+
  recoverVfsClose,
+
  recoverVfsRead,
+
  recoverVfsWrite,
+
  recoverVfsTruncate,
+
  recoverVfsSync,
+
  recoverVfsFileSize,
+
  recoverVfsLock,
+
  recoverVfsUnlock,
+
  recoverVfsCheckReservedLock,
+
  recoverVfsFileControl,
+
  recoverVfsSectorSize,
+
  recoverVfsDeviceCharacteristics,
+
  recoverVfsShmMap,
+
  recoverVfsShmLock,
+
  recoverVfsShmBarrier,
+
  recoverVfsShmUnmap,
+
  recoverVfsFetch,
+
  recoverVfsUnfetch
+
};
+

+
static int recoverVfsClose(sqlite3_file *pFd){
+
  assert( pFd->pMethods!=&recover_methods );
+
  return pFd->pMethods->xClose(pFd);
}

/*
-
** Read a varint.  Put the value in *pVal and return the number of bytes.
+
** Write value v to buffer a[] as a 16-bit big-endian unsigned integer.
*/
-
static int dbdataGetVarint(const u8 *z, sqlite3_int64 *pVal){
-
  sqlite3_int64 v = 0;
-
  int i;
-
  for(i=0; i<8; i++){
-
    v = (v<<7) + (z[i]&0x7f);
-
    if( (z[i]&0x80)==0 ){ *pVal = v; return i+1; }
-
  }
-
  v = (v<<8) + (z[i]&0xff);
-
  *pVal = v;
-
  return 9;
+
static void recoverPutU16(u8 *a, u32 v){
+
  a[0] = (v>>8) & 0x00FF;
+
  a[1] = (v>>0) & 0x00FF;
}

/*
-
** Return the number of bytes of space used by an SQLite value of type
-
** eType.
+
** Write value v to buffer a[] as a 32-bit big-endian unsigned integer.
*/
-
static int dbdataValueBytes(int eType){
-
  switch( eType ){
-
    case 0: case 8: case 9:
-
    case 10: case 11:
-
      return 0;
-
    case 1:
-
      return 1;
-
    case 2:
-
      return 2;
-
    case 3:
-
      return 3;
-
    case 4:
-
      return 4;
-
    case 5:
-
      return 6;
-
    case 6:
-
    case 7:
-
      return 8;
-
    default:
-
      if( eType>0 ){
-
        return ((eType-12) / 2);
-
      }
-
      return 0;
-
  }
+
static void recoverPutU32(u8 *a, u32 v){
+
  a[0] = (v>>24) & 0x00FF;
+
  a[1] = (v>>16) & 0x00FF;
+
  a[2] = (v>>8) & 0x00FF;
+
  a[3] = (v>>0) & 0x00FF;
}

/*
-
** Load a value of type eType from buffer pData and use it to set the
-
** result of context object pCtx.
+
** Detect the page-size of the database opened by file-handle pFd by 
+
** searching the first part of the file for a well-formed SQLite b-tree 
+
** page. If parameter nReserve is non-zero, then as well as searching for
+
** a b-tree page with zero reserved bytes, this function searches for one
+
** with nReserve reserved bytes at the end of it.
+
**
+
** If successful, set variable p->detected_pgsz to the detected page-size
+
** in bytes and return SQLITE_OK. Or, if no error occurs but no valid page
+
** can be found, return SQLITE_OK but leave p->detected_pgsz set to 0. Or,
+
** if an error occurs (e.g. an IO or OOM error), then an SQLite error code
+
** is returned. The final value of p->detected_pgsz is undefined in this
+
** case.
*/
-
static void dbdataValue(
-
  sqlite3_context *pCtx, 
-
  int eType, 
-
  u8 *pData,
-
  int nData
+
static int recoverVfsDetectPagesize(
+
  sqlite3_recover *p,             /* Recover handle */
+
  sqlite3_file *pFd,              /* File-handle open on input database */
+
  u32 nReserve,                   /* Possible nReserve value */
+
  i64 nSz                         /* Size of database file in bytes */
){
-
  if( eType>=0 && dbdataValueBytes(eType)<=nData ){
-
    switch( eType ){
-
      case 0: 
-
      case 10: 
-
      case 11: 
-
        sqlite3_result_null(pCtx);
-
        break;
-
      
-
      case 8: 
-
        sqlite3_result_int(pCtx, 0);
-
        break;
-
      case 9:
-
        sqlite3_result_int(pCtx, 1);
-
        break;
-
  
-
      case 1: case 2: case 3: case 4: case 5: case 6: case 7: {
-
        sqlite3_uint64 v = (signed char)pData[0];
-
        pData++;
-
        switch( eType ){
-
          case 7:
-
          case 6:  v = (v<<16) + (pData[0]<<8) + pData[1];  pData += 2;
-
          case 5:  v = (v<<16) + (pData[0]<<8) + pData[1];  pData += 2;
-
          case 4:  v = (v<<8) + pData[0];  pData++;
-
          case 3:  v = (v<<8) + pData[0];  pData++;
-
          case 2:  v = (v<<8) + pData[0];  pData++;
-
        }
-
  
-
        if( eType==7 ){
-
          double r;
-
          memcpy(&r, &v, sizeof(r));
-
          sqlite3_result_double(pCtx, r);
-
        }else{
-
          sqlite3_result_int64(pCtx, (sqlite3_int64)v);
-
        }
-
        break;
-
      }
-
  
-
      default: {
-
        int n = ((eType-12) / 2);
-
        if( eType % 2 ){
-
          sqlite3_result_text(pCtx, (const char*)pData, n, SQLITE_TRANSIENT);
-
        }else{
-
          sqlite3_result_blob(pCtx, pData, n, SQLITE_TRANSIENT);
+
  int rc = SQLITE_OK;
+
  const int nMin = 512;
+
  const int nMax = 65536;
+
  const int nMaxBlk = 4;
+
  u32 pgsz = 0;
+
  int iBlk = 0;
+
  u8 *aPg = 0;
+
  u8 *aTmp = 0;
+
  int nBlk = 0;
+

+
  aPg = (u8*)sqlite3_malloc(2*nMax);
+
  if( aPg==0 ) return SQLITE_NOMEM;
+
  aTmp = &aPg[nMax];
+

+
  nBlk = (nSz+nMax-1)/nMax;
+
  if( nBlk>nMaxBlk ) nBlk = nMaxBlk;
+

+
  do {
+
    for(iBlk=0; rc==SQLITE_OK && iBlk<nBlk; iBlk++){
+
      int nByte = (nSz>=((iBlk+1)*nMax)) ? nMax : (nSz % nMax);
+
      memset(aPg, 0, nMax);
+
      rc = pFd->pMethods->xRead(pFd, aPg, nByte, iBlk*nMax);
+
      if( rc==SQLITE_OK ){
+
        int pgsz2;
+
        for(pgsz2=(pgsz ? pgsz*2 : nMin); pgsz2<=nMax; pgsz2=pgsz2*2){
+
          int iOff;
+
          for(iOff=0; iOff<nMax; iOff+=pgsz2){
+
            if( recoverIsValidPage(aTmp, &aPg[iOff], pgsz2-nReserve) ){
+
              pgsz = pgsz2;
+
              break;
+
            }
+
          }
        }
      }
    }
-
  }
+
    if( pgsz>(u32)p->detected_pgsz ){
+
      p->detected_pgsz = pgsz;
+
      p->nReserve = nReserve;
+
    }
+
    if( nReserve==0 ) break;
+
    nReserve = 0;
+
  }while( 1 );
+

+
  p->detected_pgsz = pgsz;
+
  sqlite3_free(aPg);
+
  return rc;
}

/*
-
** Move an sqlite_dbdata or sqlite_dbptr cursor to the next entry.
+
** The xRead() method of the wrapper VFS. This is used to intercept calls
+
** to read page 1 of the input database.
*/
-
static int dbdataNext(sqlite3_vtab_cursor *pCursor){
-
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
-
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
+
static int recoverVfsRead(sqlite3_file *pFd, void *aBuf, int nByte, i64 iOff){
+
  int rc = SQLITE_OK;
+
  if( pFd->pMethods==&recover_methods ){
+
    pFd->pMethods = recover_g.pMethods;
+
    rc = pFd->pMethods->xRead(pFd, aBuf, nByte, iOff);
+
    if( nByte==16 ){
+
      sqlite3_randomness(16, aBuf);
+
    }else
+
    if( rc==SQLITE_OK && iOff==0 && nByte>=108 ){
+
      /* Ensure that the database has a valid header file. The only fields
+
      ** that really matter to recovery are:
+
      **
+
      **   + Database page size (16-bits at offset 16)
+
      **   + Size of db in pages (32-bits at offset 28)
+
      **   + Database encoding (32-bits at offset 56)
+
      **
+
      ** Also preserved are:
+
      **
+
      **   + first freelist page (32-bits at offset 32)
+
      **   + size of freelist (32-bits at offset 36)
+
      **
+
      ** We also try to preserve the auto-vacuum, incr-value, user-version
+
      ** and application-id fields - all 32 bit quantities at offsets 
+
      ** 52, 60, 64 and 68. All other fields are set to known good values.
+
      **
+
      ** Byte offset 105 should also contain the page-size as a 16-bit 
+
      ** integer.
+
      */
+
      const int aPreserve[] = {32, 36, 52, 60, 64, 68};
+
      u8 aHdr[108] = {
+
        0x53, 0x51, 0x4c, 0x69, 0x74, 0x65, 0x20, 0x66, 
+
        0x6f, 0x72, 0x6d, 0x61, 0x74, 0x20, 0x33, 0x00,
+
        0xFF, 0xFF, 0x01, 0x01, 0x00, 0x40, 0x20, 0x20,
+
        0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0xFF, 0xFF,
+
        0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04,
+
        0x00, 0x00, 0x10, 0x00, 0xFF, 0xFF, 0xFF, 0xFF,
+
        0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+
        0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+
        0x00, 0x2e, 0x5b, 0x30,
+

+
        0x0D, 0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0x00
+
      };
+
      u8 *a = (u8*)aBuf;
+

+
      u32 pgsz = recoverGetU16(&a[16]);
+
      u32 nReserve = a[20];
+
      u32 enc = recoverGetU32(&a[56]);
+
      u32 dbsz = 0;
+
      i64 dbFileSize = 0;
+
      int ii;
+
      sqlite3_recover *p = recover_g.p;

-
  pCsr->iRowid++;
-
  while( 1 ){
-
    int rc;
-
    int iOff = (pCsr->iPgno==1 ? 100 : 0);
-
    int bNextPage = 0;
+
      if( pgsz==0x01 ) pgsz = 65536;
+
      rc = pFd->pMethods->xFileSize(pFd, &dbFileSize);

-
    if( pCsr->aPage==0 ){
-
      while( 1 ){
-
        if( pCsr->bOnePage==0 && pCsr->iPgno>pCsr->szDb ) return SQLITE_OK;
-
        rc = dbdataLoadPage(pCsr, pCsr->iPgno, &pCsr->aPage, &pCsr->nPage);
-
        if( rc!=SQLITE_OK ) return rc;
-
        if( pCsr->aPage ) break;
-
        pCsr->iPgno++;
+
      if( rc==SQLITE_OK && p->detected_pgsz==0 ){
+
        rc = recoverVfsDetectPagesize(p, pFd, nReserve, dbFileSize);
+
      }
+
      if( p->detected_pgsz ){
+
        pgsz = p->detected_pgsz;
+
        nReserve = p->nReserve;
      }
-
      pCsr->iCell = pTab->bPtr ? -2 : 0;
-
      pCsr->nCell = get_uint16(&pCsr->aPage[iOff+3]);
-
    }

-
    if( pTab->bPtr ){
-
      if( pCsr->aPage[iOff]!=0x02 && pCsr->aPage[iOff]!=0x05 ){
-
        pCsr->iCell = pCsr->nCell;
+
      if( pgsz ){
+
        dbsz = dbFileSize / pgsz;
      }
-
      pCsr->iCell++;
-
      if( pCsr->iCell>=pCsr->nCell ){
-
        sqlite3_free(pCsr->aPage);
-
        pCsr->aPage = 0;
-
        if( pCsr->bOnePage ) return SQLITE_OK;
-
        pCsr->iPgno++;
-
      }else{
-
        return SQLITE_OK;
+
      if( enc!=SQLITE_UTF8 && enc!=SQLITE_UTF16BE && enc!=SQLITE_UTF16LE ){
+
        enc = SQLITE_UTF8;
      }
-
    }else{
-
      /* If there is no record loaded, load it now. */
-
      if( pCsr->pRec==0 ){
-
        int bHasRowid = 0;
-
        int nPointer = 0;
-
        sqlite3_int64 nPayload = 0;
-
        sqlite3_int64 nHdr = 0;
-
        int iHdr;
-
        int U, X;
-
        int nLocal;
-
  
-
        switch( pCsr->aPage[iOff] ){
-
          case 0x02:
-
            nPointer = 4;
-
            break;
-
          case 0x0a:
-
            break;
-
          case 0x0d:
-
            bHasRowid = 1;
-
            break;
-
          default:
-
            /* This is not a b-tree page with records on it. Continue. */
-
            pCsr->iCell = pCsr->nCell;
-
            break;
+

+
      sqlite3_free(p->pPage1Cache);
+
      p->pPage1Cache = 0;
+
      p->pPage1Disk = 0;
+

+
      p->pgsz = nByte;
+
      p->pPage1Cache = (u8*)recoverMalloc(p, nByte*2);
+
      if( p->pPage1Cache ){
+
        p->pPage1Disk = &p->pPage1Cache[nByte];
+
        memcpy(p->pPage1Disk, aBuf, nByte);
+

+
        recoverPutU32(&aHdr[28], dbsz);
+
        recoverPutU32(&aHdr[56], enc);
+
        recoverPutU16(&aHdr[105], pgsz-nReserve);
+
        if( pgsz==65536 ) pgsz = 1;
+
        recoverPutU16(&aHdr[16], pgsz);
+
        aHdr[20] = nReserve;
+
        for(ii=0; ii<sizeof(aPreserve)/sizeof(aPreserve[0]); ii++){
+
          memcpy(&aHdr[aPreserve[ii]], &a[aPreserve[ii]], 4);
        }
+
        memcpy(aBuf, aHdr, sizeof(aHdr));
+
        memset(&((u8*)aBuf)[sizeof(aHdr)], 0, nByte-sizeof(aHdr));

-
        if( pCsr->iCell>=pCsr->nCell ){
-
          bNextPage = 1;
-
        }else{
-
  
-
          iOff += 8 + nPointer + pCsr->iCell*2;
-
          if( iOff>pCsr->nPage ){
-
            bNextPage = 1;
-
          }else{
-
            iOff = get_uint16(&pCsr->aPage[iOff]);
-
          }
-
    
-
          /* For an interior node cell, skip past the child-page number */
-
          iOff += nPointer;
-
    
-
          /* Load the "byte of payload including overflow" field */
-
          if( bNextPage || iOff>pCsr->nPage ){
-
            bNextPage = 1;
-
          }else{
-
            iOff += dbdataGetVarint(&pCsr->aPage[iOff], &nPayload);
-
          }
-
    
-
          /* If this is a leaf intkey cell, load the rowid */
-
          if( bHasRowid && !bNextPage && iOff<pCsr->nPage ){
-
            iOff += dbdataGetVarint(&pCsr->aPage[iOff], &pCsr->iIntkey);
-
          }
-
    
-
          /* Figure out how much data to read from the local page */
-
          U = pCsr->nPage;
-
          if( bHasRowid ){
-
            X = U-35;
-
          }else{
-
            X = ((U-12)*64/255)-23;
-
          }
-
          if( nPayload<=X ){
-
            nLocal = nPayload;
-
          }else{
-
            int M, K;
-
            M = ((U-12)*32/255)-23;
-
            K = M+((nPayload-M)%(U-4));
-
            if( K<=X ){
-
              nLocal = K;
-
            }else{
-
              nLocal = M;
-
            }
-
          }
+
        memcpy(p->pPage1Cache, aBuf, nByte);
+
      }else{
+
        rc = p->errCode;
+
      }
+

+
    }
+
    pFd->pMethods = &recover_methods;
+
  }else{
+
    rc = pFd->pMethods->xRead(pFd, aBuf, nByte, iOff);
+
  }
+
  return rc;
+
}
+

+
/*
+
** Used to make sqlite3_io_methods wrapper methods less verbose.
+
*/
+
#define RECOVER_VFS_WRAPPER(code)                         \
+
  int rc = SQLITE_OK;                                     \
+
  if( pFd->pMethods==&recover_methods ){                  \
+
    pFd->pMethods = recover_g.pMethods;                   \
+
    rc = code;                                            \
+
    pFd->pMethods = &recover_methods;                     \
+
  }else{                                                  \
+
    rc = code;                                            \
+
  }                                                       \
+
  return rc;                                              
+

+
/*
+
** Methods of the wrapper VFS. All methods except for xRead() and xClose()
+
** simply uninstall the sqlite3_io_methods wrapper, invoke the equivalent
+
** method on the lower level VFS, then reinstall the wrapper before returning.
+
** Those that return an integer value use the RECOVER_VFS_WRAPPER macro.
+
*/
+
static int recoverVfsWrite(
+
  sqlite3_file *pFd, const void *aBuf, int nByte, i64 iOff
+
){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xWrite(pFd, aBuf, nByte, iOff)
+
  );
+
}
+
static int recoverVfsTruncate(sqlite3_file *pFd, sqlite3_int64 size){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xTruncate(pFd, size)
+
  );
+
}
+
static int recoverVfsSync(sqlite3_file *pFd, int flags){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xSync(pFd, flags)
+
  );
+
}
+
static int recoverVfsFileSize(sqlite3_file *pFd, sqlite3_int64 *pSize){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xFileSize(pFd, pSize)
+
  );
+
}
+
static int recoverVfsLock(sqlite3_file *pFd, int eLock){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xLock(pFd, eLock)
+
  );
+
}
+
static int recoverVfsUnlock(sqlite3_file *pFd, int eLock){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xUnlock(pFd, eLock)
+
  );
+
}
+
static int recoverVfsCheckReservedLock(sqlite3_file *pFd, int *pResOut){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xCheckReservedLock(pFd, pResOut)
+
  );
+
}
+
static int recoverVfsFileControl(sqlite3_file *pFd, int op, void *pArg){
+
  RECOVER_VFS_WRAPPER (
+
    (pFd->pMethods ?  pFd->pMethods->xFileControl(pFd, op, pArg) : SQLITE_NOTFOUND)
+
  );
+
}
+
static int recoverVfsSectorSize(sqlite3_file *pFd){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xSectorSize(pFd)
+
  );
+
}
+
static int recoverVfsDeviceCharacteristics(sqlite3_file *pFd){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xDeviceCharacteristics(pFd)
+
  );
+
}
+
static int recoverVfsShmMap(
+
  sqlite3_file *pFd, int iPg, int pgsz, int bExtend, void volatile **pp
+
){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xShmMap(pFd, iPg, pgsz, bExtend, pp)
+
  );
+
}
+
static int recoverVfsShmLock(sqlite3_file *pFd, int offset, int n, int flags){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xShmLock(pFd, offset, n, flags)
+
  );
+
}
+
static void recoverVfsShmBarrier(sqlite3_file *pFd){
+
  if( pFd->pMethods==&recover_methods ){
+
    pFd->pMethods = recover_g.pMethods;
+
    pFd->pMethods->xShmBarrier(pFd);
+
    pFd->pMethods = &recover_methods;
+
  }else{
+
    pFd->pMethods->xShmBarrier(pFd);
+
  }
+
}
+
static int recoverVfsShmUnmap(sqlite3_file *pFd, int deleteFlag){
+
  RECOVER_VFS_WRAPPER (
+
      pFd->pMethods->xShmUnmap(pFd, deleteFlag)
+
  );
+
}
+

+
static int recoverVfsFetch(
+
  sqlite3_file *pFd, 
+
  sqlite3_int64 iOff, 
+
  int iAmt, 
+
  void **pp
+
){
+
  *pp = 0;
+
  return SQLITE_OK;
+
}
+
static int recoverVfsUnfetch(sqlite3_file *pFd, sqlite3_int64 iOff, void *p){
+
  return SQLITE_OK;
+
}

-
          if( bNextPage || nLocal+iOff>pCsr->nPage ){
-
            bNextPage = 1;
-
          }else{
+
/*
+
** Install the VFS wrapper around the file-descriptor open on the input
+
** database for recover handle p. Mutex RECOVER_MUTEX_ID must be held
+
** when this function is called.
+
*/
+
static void recoverInstallWrapper(sqlite3_recover *p){
+
  sqlite3_file *pFd = 0;
+
  assert( recover_g.pMethods==0 );
+
  recoverAssertMutexHeld();
+
  sqlite3_file_control(p->dbIn, p->zDb, SQLITE_FCNTL_FILE_POINTER, (void*)&pFd);
+
  assert( pFd==0 || pFd->pMethods!=&recover_methods );
+
  if( pFd && pFd->pMethods ){
+
    int iVersion = 1 + (pFd->pMethods->iVersion>1 && pFd->pMethods->xShmMap!=0);
+
    recover_g.pMethods = pFd->pMethods;
+
    recover_g.p = p;
+
    recover_methods.iVersion = iVersion;
+
    pFd->pMethods = &recover_methods;
+
  }
+
}

-
            /* Allocate space for payload. And a bit more to catch small buffer
-
            ** overruns caused by attempting to read a varint or similar from 
-
            ** near the end of a corrupt record.  */
-
            pCsr->pRec = (u8*)sqlite3_malloc64(nPayload+DBDATA_PADDING_BYTES);
-
            if( pCsr->pRec==0 ) return SQLITE_NOMEM;
-
            memset(pCsr->pRec, 0, nPayload+DBDATA_PADDING_BYTES);
-
            pCsr->nRec = nPayload;
+
/*
+
** Uninstall the VFS wrapper that was installed around the file-descriptor open
+
** on the input database for recover handle p. Mutex RECOVER_MUTEX_ID must be
+
** held when this function is called.
+
*/
+
static void recoverUninstallWrapper(sqlite3_recover *p){
+
  sqlite3_file *pFd = 0;
+
  recoverAssertMutexHeld();
+
  sqlite3_file_control(p->dbIn, p->zDb,SQLITE_FCNTL_FILE_POINTER,(void*)&pFd);
+
  if( pFd && pFd->pMethods ){
+
    pFd->pMethods = recover_g.pMethods;
+
    recover_g.pMethods = 0;
+
    recover_g.p = 0;
+
  }
+
}

-
            /* Load the nLocal bytes of payload */
-
            memcpy(pCsr->pRec, &pCsr->aPage[iOff], nLocal);
-
            iOff += nLocal;
+
/*
+
** This function does the work of a single sqlite3_recover_step() call. It
+
** is guaranteed that the handle is not in an error state when this
+
** function is called.
+
*/
+
static void recoverStep(sqlite3_recover *p){
+
  assert( p && p->errCode==SQLITE_OK );
+
  switch( p->eState ){
+
    case RECOVER_STATE_INIT:
+
      /* This is the very first call to sqlite3_recover_step() on this object.
+
      */
+
      recoverSqlCallback(p, "BEGIN");
+
      recoverSqlCallback(p, "PRAGMA writable_schema = on");

-
            /* Load content from overflow pages */
-
            if( nPayload>nLocal ){
-
              sqlite3_int64 nRem = nPayload - nLocal;
-
              unsigned int pgnoOvfl = get_uint32(&pCsr->aPage[iOff]);
-
              while( nRem>0 ){
-
                u8 *aOvfl = 0;
-
                int nOvfl = 0;
-
                int nCopy;
-
                rc = dbdataLoadPage(pCsr, pgnoOvfl, &aOvfl, &nOvfl);
-
                assert( rc!=SQLITE_OK || aOvfl==0 || nOvfl==pCsr->nPage );
-
                if( rc!=SQLITE_OK ) return rc;
-
                if( aOvfl==0 ) break;
+
      recoverEnterMutex();
+
      recoverInstallWrapper(p);

-
                nCopy = U-4;
-
                if( nCopy>nRem ) nCopy = nRem;
-
                memcpy(&pCsr->pRec[nPayload-nRem], &aOvfl[4], nCopy);
-
                nRem -= nCopy;
+
      /* Open the output database. And register required virtual tables and 
+
      ** user functions with the new handle. */
+
      recoverOpenOutput(p);

-
                pgnoOvfl = get_uint32(aOvfl);
-
                sqlite3_free(aOvfl);
-
              }
-
            }
-
    
-
            iHdr = dbdataGetVarint(pCsr->pRec, &nHdr);
-
            pCsr->nHdr = nHdr;
-
            pCsr->pHdrPtr = &pCsr->pRec[iHdr];
-
            pCsr->pPtr = &pCsr->pRec[pCsr->nHdr];
-
            pCsr->iField = (bHasRowid ? -1 : 0);
-
          }
-
        }
-
      }else{
-
        pCsr->iField++;
-
        if( pCsr->iField>0 ){
-
          sqlite3_int64 iType;
-
          if( pCsr->pHdrPtr>&pCsr->pRec[pCsr->nRec] ){
-
            bNextPage = 1;
-
          }else{
-
            pCsr->pHdrPtr += dbdataGetVarint(pCsr->pHdrPtr, &iType);
-
            pCsr->pPtr += dbdataValueBytes(iType);
-
          }
+
      /* Open transactions on both the input and output databases. */
+
      recoverExec(p, p->dbIn, "PRAGMA writable_schema = on");
+
      recoverExec(p, p->dbIn, "BEGIN");
+
      if( p->errCode==SQLITE_OK ) p->bCloseTransaction = 1;
+
      recoverExec(p, p->dbIn, "SELECT 1 FROM sqlite_schema");
+
      recoverTransferSettings(p);
+
      recoverOpenRecovery(p);
+
      recoverCacheSchema(p);
+

+
      recoverUninstallWrapper(p);
+
      recoverLeaveMutex();
+

+
      recoverExec(p, p->dbOut, "BEGIN");
+

+
      recoverWriteSchema1(p);
+
      p->eState = RECOVER_STATE_WRITING;
+
      break;
+
      
+
    case RECOVER_STATE_WRITING: {
+
      if( p->w1.pTbls==0 ){
+
        recoverWriteDataInit(p);
+
      }
+
      if( SQLITE_DONE==recoverWriteDataStep(p) ){
+
        recoverWriteDataCleanup(p);
+
        if( p->zLostAndFound ){
+
          p->eState = RECOVER_STATE_LOSTANDFOUND1;
+
        }else{
+
          p->eState = RECOVER_STATE_SCHEMA2;
        }
      }
+
      break;
+
    }

-
      if( bNextPage ){
-
        sqlite3_free(pCsr->aPage);
-
        sqlite3_free(pCsr->pRec);
-
        pCsr->aPage = 0;
-
        pCsr->pRec = 0;
-
        if( pCsr->bOnePage ) return SQLITE_OK;
-
        pCsr->iPgno++;
-
      }else{
-
        if( pCsr->iField<0 || pCsr->pHdrPtr<&pCsr->pRec[pCsr->nHdr] ){
-
          return SQLITE_OK;
-
        }
+
    case RECOVER_STATE_LOSTANDFOUND1: {
+
      if( p->laf.pUsed==0 ){
+
        recoverLostAndFound1Init(p);
+
      }
+
      if( SQLITE_DONE==recoverLostAndFound1Step(p) ){
+
        p->eState = RECOVER_STATE_LOSTANDFOUND2;
+
      }
+
      break;
+
    }
+
    case RECOVER_STATE_LOSTANDFOUND2: {
+
      if( p->laf.pAllAndParent==0 ){
+
        recoverLostAndFound2Init(p);
+
      }
+
      if( SQLITE_DONE==recoverLostAndFound2Step(p) ){
+
        p->eState = RECOVER_STATE_LOSTANDFOUND3;
+
      }
+
      break;
+
    }

-
        /* Advance to the next cell. The next iteration of the loop will load
-
        ** the record and so on. */
-
        sqlite3_free(pCsr->pRec);
-
        pCsr->pRec = 0;
-
        pCsr->iCell++;
+
    case RECOVER_STATE_LOSTANDFOUND3: {
+
      if( p->laf.pInsert==0 ){
+
        recoverLostAndFound3Init(p);
      }
+
      if( SQLITE_DONE==recoverLostAndFound3Step(p) ){
+
        p->eState = RECOVER_STATE_SCHEMA2;
+
      }
+
      break;
    }
-
  }

-
  assert( !"can't get here" );
-
  return SQLITE_OK;
-
}
+
    case RECOVER_STATE_SCHEMA2: {
+
      int rc = SQLITE_OK;

-
/* 
-
** Return true if the cursor is at EOF.
-
*/
-
static int dbdataEof(sqlite3_vtab_cursor *pCursor){
-
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
-
  return pCsr->aPage==0;
-
}
+
      recoverWriteSchema2(p);
+
      p->eState = RECOVER_STATE_DONE;

-
/* 
-
** Determine the size in pages of database zSchema (where zSchema is
-
** "main", "temp" or the name of an attached database) and set 
-
** pCsr->szDb accordingly. If successful, return SQLITE_OK. Otherwise,
-
** an SQLite error code.
-
*/
-
static int dbdataDbsize(DbdataCursor *pCsr, const char *zSchema){
-
  DbdataTable *pTab = (DbdataTable*)pCsr->base.pVtab;
-
  char *zSql = 0;
-
  int rc, rc2;
-
  sqlite3_stmt *pStmt = 0;
+
      /* If no error has occurred, commit the write transaction on the output
+
      ** database. Regardless of whether or not an error has occurred, make
+
      ** an attempt to end the read transaction on the input database.  */
+
      recoverExec(p, p->dbOut, "COMMIT");
+
      rc = sqlite3_exec(p->dbIn, "END", 0, 0, 0);
+
      if( p->errCode==SQLITE_OK ) p->errCode = rc;

-
  zSql = sqlite3_mprintf("PRAGMA %Q.page_count", zSchema);
-
  if( zSql==0 ) return SQLITE_NOMEM;
-
  rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pStmt, 0);
-
  sqlite3_free(zSql);
-
  if( rc==SQLITE_OK && sqlite3_step(pStmt)==SQLITE_ROW ){
-
    pCsr->szDb = sqlite3_column_int(pStmt, 0);
+
      recoverSqlCallback(p, "PRAGMA writable_schema = off");
+
      recoverSqlCallback(p, "COMMIT");
+
      p->eState = RECOVER_STATE_DONE;
+
      recoverFinalCleanup(p);
+
      break;
+
    };
+

+
    case RECOVER_STATE_DONE: {
+
      /* no-op */
+
      break;
+
    };
  }
-
  rc2 = sqlite3_finalize(pStmt);
-
  if( rc==SQLITE_OK ) rc = rc2;
-
  return rc;
}

-
/* 
-
** xFilter method for sqlite_dbdata and sqlite_dbptr.
+

+
/*
+
** This is a worker function that does the heavy lifting for both init
+
** functions:
+
**
+
**     sqlite3_recover_init()
+
**     sqlite3_recover_init_sql()
+
**
+
** All this function does is allocate space for the recover handle and
+
** take copies of the input parameters. All the real work is done within
+
** sqlite3_recover_run().
*/
-
static int dbdataFilter(
-
  sqlite3_vtab_cursor *pCursor, 
-
  int idxNum, const char *idxStr,
-
  int argc, sqlite3_value **argv
+
sqlite3_recover *recoverInit(
+
  sqlite3* db, 
+
  const char *zDb, 
+
  const char *zUri,               /* Output URI for _recover_init() */
+
  int (*xSql)(void*, const char*),/* SQL callback for _recover_init_sql() */
+
  void *pSqlCtx                   /* Context arg for _recover_init_sql() */
){
-
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
-
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
-
  int rc = SQLITE_OK;
-
  const char *zSchema = "main";
+
  sqlite3_recover *pRet = 0;
+
  int nDb = 0;
+
  int nUri = 0;
+
  int nByte = 0;

-
  dbdataResetCursor(pCsr);
-
  assert( pCsr->iPgno==1 );
-
  if( idxNum & 0x01 ){
-
    zSchema = (const char*)sqlite3_value_text(argv[0]);
-
  }
-
  if( idxNum & 0x02 ){
-
    pCsr->iPgno = sqlite3_value_int(argv[(idxNum & 0x01)]);
-
    pCsr->bOnePage = 1;
-
  }else{
-
    pCsr->nPage = dbdataDbsize(pCsr, zSchema);
-
    rc = dbdataDbsize(pCsr, zSchema);
-
  }
+
  if( zDb==0 ){ zDb = "main"; }

-
  if( rc==SQLITE_OK ){
-
    if( pTab->pStmt ){
-
      pCsr->pStmt = pTab->pStmt;
-
      pTab->pStmt = 0;
-
    }else{
-
      rc = sqlite3_prepare_v2(pTab->db, 
-
          "SELECT data FROM sqlite_dbpage(?) WHERE pgno=?", -1,
-
          &pCsr->pStmt, 0
-
      );
-
    }
-
  }
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_bind_text(pCsr->pStmt, 1, zSchema, -1, SQLITE_TRANSIENT);
-
  }else{
-
    pTab->base.zErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pTab->db));
-
  }
-
  if( rc==SQLITE_OK ){
-
    rc = dbdataNext(pCursor);
+
  nDb = recoverStrlen(zDb);
+
  nUri = recoverStrlen(zUri);
+

+
  nByte = sizeof(sqlite3_recover) + nDb+1 + nUri+1;
+
  pRet = (sqlite3_recover*)sqlite3_malloc(nByte);
+
  if( pRet ){
+
    memset(pRet, 0, nByte);
+
    pRet->dbIn = db;
+
    pRet->zDb = (char*)&pRet[1];
+
    pRet->zUri = &pRet->zDb[nDb+1];
+
    memcpy(pRet->zDb, zDb, nDb);
+
    if( nUri>0 && zUri ) memcpy(pRet->zUri, zUri, nUri);
+
    pRet->xSql = xSql;
+
    pRet->pSqlCtx = pSqlCtx;
+
    pRet->bRecoverRowid = RECOVER_ROWID_DEFAULT;
  }
-
  return rc;
+

+
  return pRet;
}

-
/* 
-
** Return a column for the sqlite_dbdata or sqlite_dbptr table.
+
/*
+
** Initialize a recovery handle that creates a new database containing
+
** the recovered data.
*/
-
static int dbdataColumn(
-
  sqlite3_vtab_cursor *pCursor, 
-
  sqlite3_context *ctx, 
-
  int i
+
sqlite3_recover *sqlite3_recover_init(
+
  sqlite3* db, 
+
  const char *zDb, 
+
  const char *zUri
){
-
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
-
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
-
  if( pTab->bPtr ){
-
    switch( i ){
-
      case DBPTR_COLUMN_PGNO:
-
        sqlite3_result_int64(ctx, pCsr->iPgno);
+
  return recoverInit(db, zDb, zUri, 0, 0);
+
}
+

+
/*
+
** Initialize a recovery handle that returns recovered data in the
+
** form of SQL statements via a callback.
+
*/
+
sqlite3_recover *sqlite3_recover_init_sql(
+
  sqlite3* db, 
+
  const char *zDb, 
+
  int (*xSql)(void*, const char*),
+
  void *pSqlCtx
+
){
+
  return recoverInit(db, zDb, 0, xSql, pSqlCtx);
+
}
+

+
/*
+
** Return the handle error message, if any.
+
*/
+
const char *sqlite3_recover_errmsg(sqlite3_recover *p){
+
  return (p && p->errCode!=SQLITE_NOMEM) ? p->zErrMsg : "out of memory";
+
}
+

+
/*
+
** Return the handle error code.
+
*/
+
int sqlite3_recover_errcode(sqlite3_recover *p){
+
  return p ? p->errCode : SQLITE_NOMEM;
+
}
+

+
/*
+
** Configure the handle.
+
*/
+
int sqlite3_recover_config(sqlite3_recover *p, int op, void *pArg){
+
  int rc = SQLITE_OK;
+
  if( p==0 ){
+
    rc = SQLITE_NOMEM;
+
  }else if( p->eState!=RECOVER_STATE_INIT ){
+
    rc = SQLITE_MISUSE;
+
  }else{
+
    switch( op ){
+
      case 789:
+
        /* This undocumented magic configuration option is used to set the
+
        ** name of the auxiliary database that is ATTACH-ed to the database
+
        ** connection and used to hold state information during the
+
        ** recovery process.  This option is for debugging use only and
+
        ** is subject to change or removal at any time. */
+
        sqlite3_free(p->zStateDb);
+
        p->zStateDb = recoverMPrintf(p, "%s", (char*)pArg);
        break;
-
      case DBPTR_COLUMN_CHILD: {
-
        int iOff = pCsr->iPgno==1 ? 100 : 0;
-
        if( pCsr->iCell<0 ){
-
          iOff += 8;
+

+
      case SQLITE_RECOVER_LOST_AND_FOUND: {
+
        const char *zArg = (const char*)pArg;
+
        sqlite3_free(p->zLostAndFound);
+
        if( zArg ){
+
          p->zLostAndFound = recoverMPrintf(p, "%s", zArg);
        }else{
-
          iOff += 12 + pCsr->iCell*2;
-
          if( iOff>pCsr->nPage ) return SQLITE_OK;
-
          iOff = get_uint16(&pCsr->aPage[iOff]);
-
        }
-
        if( iOff<=pCsr->nPage ){
-
          sqlite3_result_int64(ctx, get_uint32(&pCsr->aPage[iOff]));
+
          p->zLostAndFound = 0;
        }
        break;
      }
-
    }
-
  }else{
-
    switch( i ){
-
      case DBDATA_COLUMN_PGNO:
-
        sqlite3_result_int64(ctx, pCsr->iPgno);
+

+
      case SQLITE_RECOVER_FREELIST_CORRUPT:
+
        p->bFreelistCorrupt = *(int*)pArg;
        break;
-
      case DBDATA_COLUMN_CELL:
-
        sqlite3_result_int(ctx, pCsr->iCell);
+

+
      case SQLITE_RECOVER_ROWIDS:
+
        p->bRecoverRowid = *(int*)pArg;
        break;
-
      case DBDATA_COLUMN_FIELD:
-
        sqlite3_result_int(ctx, pCsr->iField);
+

+
      case SQLITE_RECOVER_SLOWINDEXES:
+
        p->bSlowIndexes = *(int*)pArg;
        break;
-
      case DBDATA_COLUMN_VALUE: {
-
        if( pCsr->iField<0 ){
-
          sqlite3_result_int64(ctx, pCsr->iIntkey);
-
        }else{
-
          sqlite3_int64 iType;
-
          dbdataGetVarint(pCsr->pHdrPtr, &iType);
-
          dbdataValue(
-
              ctx, iType, pCsr->pPtr, &pCsr->pRec[pCsr->nRec] - pCsr->pPtr
-
          );
-
        }
+

+
      default:
+
        rc = SQLITE_NOTFOUND;
        break;
-
      }
    }
  }
-
  return SQLITE_OK;
+

+
  return rc;
}

-
/* 
-
** Return the rowid for an sqlite_dbdata or sqlite_dptr table.
+
/*
+
** Do a unit of work towards the recovery job. Return SQLITE_OK if
+
** no error has occurred but database recovery is not finished, SQLITE_DONE
+
** if database recovery has been successfully completed, or an SQLite
+
** error code if an error has occurred.
+
*/
+
int sqlite3_recover_step(sqlite3_recover *p){
+
  if( p==0 ) return SQLITE_NOMEM;
+
  if( p->errCode==SQLITE_OK ) recoverStep(p);
+
  if( p->eState==RECOVER_STATE_DONE && p->errCode==SQLITE_OK ){
+
    return SQLITE_DONE;
+
  }
+
  return p->errCode;
+
}
+

+
/*
+
** Do the configured recovery operation. Return SQLITE_OK if successful, or
+
** else an SQLite error code.
*/
-
static int dbdataRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
-
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
-
  *pRowid = pCsr->iRowid;
-
  return SQLITE_OK;
+
int sqlite3_recover_run(sqlite3_recover *p){
+
  while( SQLITE_OK==sqlite3_recover_step(p) );
+
  return sqlite3_recover_errcode(p);
}


/*
-
** Invoke this routine to register the "sqlite_dbdata" virtual table module
+
** Free all resources associated with the recover handle passed as the only
+
** argument. The results of using a handle with any sqlite3_recover_**
+
** API function after it has been passed to this function are undefined.
+
**
+
** A copy of the value returned by the first call made to sqlite3_recover_run()
+
** on this handle is returned, or SQLITE_OK if sqlite3_recover_run() has
+
** not been called on this handle.
*/
-
static int sqlite3DbdataRegister(sqlite3 *db){
-
  static sqlite3_module dbdata_module = {
-
    0,                            /* iVersion */
-
    0,                            /* xCreate */
-
    dbdataConnect,                /* xConnect */
-
    dbdataBestIndex,              /* xBestIndex */
-
    dbdataDisconnect,             /* xDisconnect */
-
    0,                            /* xDestroy */
-
    dbdataOpen,                   /* xOpen - open a cursor */
-
    dbdataClose,                  /* xClose - close a cursor */
-
    dbdataFilter,                 /* xFilter - configure scan constraints */
-
    dbdataNext,                   /* xNext - advance a cursor */
-
    dbdataEof,                    /* xEof - check for end of scan */
-
    dbdataColumn,                 /* xColumn - read data */
-
    dbdataRowid,                  /* xRowid - read data */
-
    0,                            /* xUpdate */
-
    0,                            /* xBegin */
-
    0,                            /* xSync */
-
    0,                            /* xCommit */
-
    0,                            /* xRollback */
-
    0,                            /* xFindMethod */
-
    0,                            /* xRename */
-
    0,                            /* xSavepoint */
-
    0,                            /* xRelease */
-
    0,                            /* xRollbackTo */
-
    0                             /* xShadowName */
-
  };
-

-
  int rc = sqlite3_create_module(db, "sqlite_dbdata", &dbdata_module, 0);
-
  if( rc==SQLITE_OK ){
-
    rc = sqlite3_create_module(db, "sqlite_dbptr", &dbdata_module, (void*)1);
+
int sqlite3_recover_finish(sqlite3_recover *p){
+
  int rc;
+
  if( p==0 ){
+
    rc = SQLITE_NOMEM;
+
  }else{
+
    recoverFinalCleanup(p);
+
    if( p->bCloseTransaction && sqlite3_get_autocommit(p->dbIn)==0 ){
+
      rc = sqlite3_exec(p->dbIn, "END", 0, 0, 0);
+
      if( p->errCode==SQLITE_OK ) p->errCode = rc;
+
    }
+
    rc = p->errCode;
+
    sqlite3_free(p->zErrMsg);
+
    sqlite3_free(p->zStateDb);
+
    sqlite3_free(p->zLostAndFound);
+
    sqlite3_free(p->pPage1Cache);
+
    sqlite3_free(p);
  }
  return rc;
}

-
#ifdef _WIN32
-

-
#endif
-
int sqlite3_dbdata_init(
-
  sqlite3 *db, 
-
  char **pzErrMsg, 
-
  const sqlite3_api_routines *pApi
-
){
-
  SQLITE_EXTENSION_INIT2(pApi);
-
  return sqlite3DbdataRegister(db);
-
}
+
#endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */

-
/************************* End ../ext/misc/dbdata.c ********************/
+
/************************* End ../ext/recover/sqlite3recover.c ********************/
#endif

#if defined(SQLITE_ENABLE_SESSION)
@@ -12233,8 +15590,19 @@ struct ShellState {
  char *zNonce;          /* Nonce for temporary safe-mode excapes */
  EQPGraph sGraph;       /* Information for the graphical EXPLAIN QUERY PLAN */
  ExpertInfo expert;     /* Valid if previous command was ".expert OPT..." */
+
#ifdef SQLITE_SHELL_FIDDLE
+
  struct {
+
    const char * zInput; /* Input string from wasm/JS proxy */
+
    const char * zPos;   /* Cursor pos into zInput */
+
    const char * zDefaultDbName; /* Default name for db file */
+
  } wasm;
+
#endif
};

+
#ifdef SQLITE_SHELL_FIDDLE
+
static ShellState shellState;
+
#endif
+


/* Allowed values for ShellState.autoEQP
*/
@@ -12275,7 +15643,7 @@ struct ShellState {
#define SHFLG_PreserveRowid  0x00000008 /* .dump preserves rowid values */
#define SHFLG_Newlines       0x00000010 /* .dump --newline flag */
#define SHFLG_CountChanges   0x00000020 /* .changes setting */
-
#define SHFLG_Echo           0x00000040 /* .echo or --echo setting */
+
#define SHFLG_Echo           0x00000040 /* .echo on/off, or --echo setting */
#define SHFLG_HeaderSet      0x00000080 /* showHeader has been specified */
#define SHFLG_DumpDataOnly   0x00000100 /* .dump show data only */
#define SHFLG_DumpNoSys      0x00000200 /* .dump omits system tables */
@@ -12563,10 +15931,23 @@ static void outputModePop(ShellState *p){
*/
static void output_hex_blob(FILE *out, const void *pBlob, int nBlob){
  int i;
-
  char *zBlob = (char *)pBlob;
-
  raw_printf(out,"X'");
-
  for(i=0; i<nBlob; i++){ raw_printf(out,"%02x",zBlob[i]&0xff); }
-
  raw_printf(out,"'");
+
  unsigned char *aBlob = (unsigned char*)pBlob;
+

+
  char *zStr = sqlite3_malloc(nBlob*2 + 1);
+
  shell_check_oom(zStr);
+

+
  for(i=0; i<nBlob; i++){
+
    static const char aHex[] = {
+
        '0', '1', '2', '3', '4', '5', '6', '7',
+
        '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'
+
    };
+
    zStr[i*2] = aHex[ (aBlob[i] >> 4) ];
+
    zStr[i*2+1] = aHex[ (aBlob[i] & 0x0F) ];
+
  }
+
  zStr[i*2] = '\0';
+

+
  raw_printf(out,"X'%s'", zStr);
+
  sqlite3_free(zStr);
}

/*
@@ -12726,9 +16107,9 @@ static void output_c_string(FILE *out, const char *z){
/*
** Output the given string as a quoted according to JSON quoting rules.
*/
-
static void output_json_string(FILE *out, const char *z, int n){
+
static void output_json_string(FILE *out, const char *z, i64 n){
  unsigned int c;
-
  if( n<0 ) n = (int)strlen(z);
+
  if( n<0 ) n = strlen(z);
  fputc('"', out);
  while( n-- ){
    c = *(z++);
@@ -12899,7 +16280,11 @@ static int safeModeAuth(
  UNUSED_PARAMETER(zA4);
  switch( op ){
    case SQLITE_ATTACH: {
+
#ifndef SQLITE_SHELL_FIDDLE
+
      /* In WASM builds the filesystem is a virtual sandbox, so
+
      ** there's no harm in using ATTACH. */
      failIfSafeMode(p, "cannot run ATTACH in safe mode");
+
#endif
      break;
    }
    case SQLITE_FUNCTION: {
@@ -12968,15 +16353,37 @@ static int shellAuth(
**
** This routine converts some CREATE TABLE statements for shadow tables
** in FTS3/4/5 into CREATE TABLE IF NOT EXISTS statements.
+
**
+
** If the schema statement in z[] contains a start-of-comment and if
+
** sqlite3_complete() returns false, try to terminate the comment before
+
** printing the result.  https://sqlite.org/forum/forumpost/d7be961c5c
*/
static void printSchemaLine(FILE *out, const char *z, const char *zTail){
+
  char *zToFree = 0;
  if( z==0 ) return;
  if( zTail==0 ) return;
+
  if( zTail[0]==';' && (strstr(z, "/*")!=0 || strstr(z,"--")!=0) ){
+
    const char *zOrig = z;
+
    static const char *azTerm[] = { "", "*/", "\n" };
+
    int i;
+
    for(i=0; i<ArraySize(azTerm); i++){
+
      char *zNew = sqlite3_mprintf("%s%s;", zOrig, azTerm[i]);
+
      if( sqlite3_complete(zNew) ){
+
        size_t n = strlen(zNew);
+
        zNew[n-1] = 0;
+
        zToFree = zNew;
+
        z = zNew;
+
        break;
+
      }
+
      sqlite3_free(zNew);
+
    }
+
  }
  if( sqlite3_strglob("CREATE TABLE ['\"]*", z)==0 ){
    utf8_printf(out, "CREATE TABLE IF NOT EXISTS %s%s", z+13, zTail);
  }else{
    utf8_printf(out, "%s%s", z, zTail);
  }
+
  sqlite3_free(zToFree);
}
static void printSchemaLineN(FILE *out, char *z, int n, const char *zTail){
  char c = z[n];
@@ -13005,7 +16412,9 @@ static int wsToEol(const char *z){
*/
static void eqp_append(ShellState *p, int iEqpId, int p2, const char *zText){
  EQPGraphRow *pNew;
-
  int nText = strlen30(zText);
+
  i64 nText;
+
  if( zText==0 ) return;
+
  nText = strlen(zText);
  if( p->autoEQPtest ){
    utf8_printf(p->out, "%d,%d,%s\n", iEqpId, p2, zText);
  }
@@ -13050,14 +16459,14 @@ static EQPGraphRow *eqp_next_row(ShellState *p, int iEqpId, EQPGraphRow *pOld){
*/
static void eqp_render_level(ShellState *p, int iEqpId){
  EQPGraphRow *pRow, *pNext;
-
  int n = strlen30(p->sGraph.zPrefix);
+
  i64 n = strlen(p->sGraph.zPrefix);
  char *z;
  for(pRow = eqp_next_row(p, iEqpId, 0); pRow; pRow = pNext){
    pNext = eqp_next_row(p, iEqpId, pRow);
    z = pRow->zText;
    utf8_printf(p->out, "%s%s%s\n", p->sGraph.zPrefix,
                pNext ? "|--" : "`--", z);
-
    if( n<(int)sizeof(p->sGraph.zPrefix)-7 ){
+
    if( n<(i64)sizeof(p->sGraph.zPrefix)-7 ){
      memcpy(&p->sGraph.zPrefix[n], pNext ? "|  " : "   ", 4);
      eqp_render_level(p, pRow->iEqpId);
      p->sGraph.zPrefix[n] = 0;
@@ -13657,6 +17066,7 @@ static char *shell_error_context(const char *zSql, sqlite3 *db){
    while( (zSql[len]&0xc0)==0x80 ) len--;
  }
  zCode = sqlite3_mprintf("%.*s", len, zSql);
+
  shell_check_oom(zCode);
  for(i=0; zCode[i]; i++){ if( IsSpace(zSql[i]) ) zCode[i] = ' '; }
  if( iOffset<25 ){
    zMsg = sqlite3_mprintf("\n  %z\n  %*s^--- error here", zCode, iOffset, "");
@@ -13773,7 +17183,7 @@ static void displayLinuxIoStats(FILE *out){
    int i;
    for(i=0; i<ArraySize(aTrans); i++){
      int n = strlen30(aTrans[i].zPattern);
-
      if( strncmp(aTrans[i].zPattern, z, n)==0 ){
+
      if( cli_strncmp(aTrans[i].zPattern, z, n)==0 ){
        utf8_printf(out, "%-36s %s", aTrans[i].zDesc, &z[n]);
        break;
      }
@@ -14012,7 +17422,7 @@ static void display_scanstats(
static int str_in_array(const char *zStr, const char **azArray){
  int i;
  for(i=0; azArray[i]; i++){
-
    if( 0==strcmp(zStr, azArray[i]) ) return 1;
+
    if( 0==cli_strcmp(zStr, azArray[i]) ) return 1;
  }
  return 0;
}
@@ -14028,6 +17438,9 @@ static int str_in_array(const char *zStr, const char **azArray){
**       all opcodes that occur between the p2 jump destination and the opcode
**       itself by 2 spaces.
**
+
**     * Do the previous for "Return" instructions for when P2 is positive.
+
**       See tag-20220407a in wherecode.c and vdbe.c.
+
**
**     * For each "Goto", if the jump destination is earlier in the program
**       and ends on one of:
**          Yield  SeekGt  SeekLt  RowSetRead  Rewind
@@ -14042,7 +17455,8 @@ static void explain_data_prepare(ShellState *p, sqlite3_stmt *pSql){
  int nAlloc = 0;                 /* Allocated size of p->aiIndent[], abYield */
  int iOp;                        /* Index of operation in p->aiIndent[] */

-
  const char *azNext[] = { "Next", "Prev", "VPrev", "VNext", "SorterNext", 0 };
+
  const char *azNext[] = { "Next", "Prev", "VPrev", "VNext", "SorterNext",
+
                           "Return", 0 };
  const char *azYield[] = { "Yield", "SeekLT", "SeekGT", "RowSetRead",
                            "Rewind", 0 };
  const char *azGoto[] = { "Goto", 0 };
@@ -14083,7 +17497,7 @@ static void explain_data_prepare(ShellState *p, sqlite3_stmt *pSql){
           "addr", "opcode", "p1", "p2", "p3", "p4", "p5", "comment" };
        int jj;
        for(jj=0; jj<ArraySize(explainCols); jj++){
-
          if( strcmp(sqlite3_column_name(pSql,jj),explainCols[jj])!=0 ){
+
          if( cli_strcmp(sqlite3_column_name(pSql,jj),explainCols[jj])!=0 ){
            p->cMode = p->mode;
            sqlite3_reset(pSql);
            return;
@@ -14100,7 +17514,7 @@ static void explain_data_prepare(ShellState *p, sqlite3_stmt *pSql){
    p->aiIndent[iOp] = 0;
    p->nIndent = iOp+1;

-
    if( str_in_array(zOp, azNext) ){
+
    if( str_in_array(zOp, azNext) && p2op>0 ){
      for(i=p2op; i<iOp; i++) p->aiIndent[i] += 2;
    }
    if( str_in_array(zOp, azGoto) && p2op<p->nIndent
@@ -14126,7 +17540,7 @@ static void explain_data_delete(ShellState *p){
}

/*
-
** Disable and restore .wheretrace and .selecttrace settings.
+
** Disable and restore .wheretrace and .treetrace/.selecttrace settings.
*/
static unsigned int savedSelectTrace;
static unsigned int savedWhereTrace;
@@ -14807,10 +18221,10 @@ static int expertDotCommand(
    int n;
    if( z[0]=='-' && z[1]=='-' ) z++;
    n = strlen30(z);
-
    if( n>=2 && 0==strncmp(z, "-verbose", n) ){
+
    if( n>=2 && 0==cli_strncmp(z, "-verbose", n) ){
      pState->expert.bVerbose = 1;
    }
-
    else if( n>=2 && 0==strncmp(z, "-sample", n) ){
+
    else if( n>=2 && 0==cli_strncmp(z, "-sample", n) ){
      if( i==(nArg-1) ){
        raw_printf(stderr, "option requires an argument: %s\n", z);
        rc = SQLITE_ERROR;
@@ -14900,11 +18314,6 @@ static int shell_exec(
        pArg->cnt = 0;
      }

-
      /* echo the sql statement if echo on */
-
      if( pArg && ShellHasFlag(pArg, SHFLG_Echo) ){
-
        utf8_printf(pArg->out, "%s\n", zStmtSql ? zStmtSql : zSql);
-
      }
-

      /* Show the EXPLAIN QUERY PLAN if .eqp is on */
      if( pArg && pArg->autoEQP && sqlite3_stmt_isexplain(pStmt)==0 ){
        sqlite3_stmt *pExplain;
@@ -15163,18 +18572,20 @@ static int dump_callback(void *pArg, int nArg, char **azArg, char **azNotUsed){
  zTable = azArg[0];
  zType = azArg[1];
  zSql = azArg[2];
+
  if( zTable==0 ) return 0;
+
  if( zType==0 ) return 0;
  dataOnly = (p->shellFlgs & SHFLG_DumpDataOnly)!=0;
  noSys    = (p->shellFlgs & SHFLG_DumpNoSys)!=0;

-
  if( strcmp(zTable, "sqlite_sequence")==0 && !noSys ){
+
  if( cli_strcmp(zTable, "sqlite_sequence")==0 && !noSys ){
    if( !dataOnly ) raw_printf(p->out, "DELETE FROM sqlite_sequence;\n");
  }else if( sqlite3_strglob("sqlite_stat?", zTable)==0 && !noSys ){
    if( !dataOnly ) raw_printf(p->out, "ANALYZE sqlite_schema;\n");
-
  }else if( strncmp(zTable, "sqlite_", 7)==0 ){
+
  }else if( cli_strncmp(zTable, "sqlite_", 7)==0 ){
    return 0;
  }else if( dataOnly ){
    /* no-op */
-
  }else if( strncmp(zSql, "CREATE VIRTUAL TABLE", 20)==0 ){
+
  }else if( cli_strncmp(zSql, "CREATE VIRTUAL TABLE", 20)==0 ){
    char *zIns;
    if( !p->writableSchema ){
      raw_printf(p->out, "PRAGMA writable_schema=ON;\n");
@@ -15192,7 +18603,7 @@ static int dump_callback(void *pArg, int nArg, char **azArg, char **azNotUsed){
    printSchemaLine(p->out, zSql, ";\n");
  }

-
  if( strcmp(zType, "table")==0 ){
+
  if( cli_strcmp(zType, "table")==0 ){
    ShellText sSelect;
    ShellText sTable;
    char **azCol;
@@ -15303,13 +18714,14 @@ static int run_schema_dump_query(
** Text of help messages.
**
** The help text for each individual command begins with a line that starts
-
** with ".".  Subsequent lines are supplimental information.
+
** with ".".  Subsequent lines are supplemental information.
**
** There must be two or more spaces between the end of the command and the
** start of the description of what that command does.
*/
static const char *(azHelp[]) = {
-
#if defined(SQLITE_HAVE_ZLIB) && !defined(SQLITE_OMIT_VIRTUALTABLE)
+
#if defined(SQLITE_HAVE_ZLIB) && !defined(SQLITE_OMIT_VIRTUALTABLE) \
+
  && !defined(SQLITE_SHELL_FIDDLE)
  ".archive ...             Manage SQL archives",
  "   Each command must have exactly one of the following options:",
  "     -c, --create               Create a new archive",
@@ -15335,20 +18747,28 @@ static const char *(azHelp[]) = {
#ifndef SQLITE_OMIT_AUTHORIZATION
  ".auth ON|OFF             Show authorizer callbacks",
#endif
+
#ifndef SQLITE_SHELL_FIDDLE
  ".backup ?DB? FILE        Backup DB (default \"main\") to FILE",
  "   Options:",
  "       --append            Use the appendvfs",
  "       --async             Write to FILE without journal and fsync()",
+
#endif
  ".bail on|off             Stop after hitting an error.  Default OFF",
  ".binary on|off           Turn binary output on or off.  Default OFF",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".cd DIRECTORY            Change the working directory to DIRECTORY",
+
#endif
  ".changes on|off          Show number of rows changed by SQL",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".check GLOB              Fail if output since .testcase does not match",
  ".clone NEWDB             Clone data into NEWDB from the existing database",
+
#endif
  ".connection [close] [#]  Open or close an auxiliary database connection",
  ".databases               List names and files of attached databases",
  ".dbconfig ?op? ?val?     List or change sqlite3_db_config() options",
+
#if SQLITE_SHELL_HAVE_RECOVER
  ".dbinfo ?DB?             Show status information about the database",
+
#endif
  ".dump ?OBJECTS?          Render database content as SQL",
  "   Options:",
  "     --data-only            Output only INSERT statements",
@@ -15365,9 +18785,13 @@ static const char *(azHelp[]) = {
  "      trace                 Like \"full\" but enable \"PRAGMA vdbe_trace\"",
#endif
  "      trigger               Like \"full\" but also show trigger bytecode",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".excel                   Display the output of next command in spreadsheet",
  "   --bom                   Put a UTF8 byte-order mark on intermediate file",
+
#endif
+
#ifndef SQLITE_SHELL_FIDDLE
  ".exit ?CODE?             Exit this program with return-code CODE",
+
#endif
  ".expert                  EXPERIMENTAL. Suggest indexes for queries",
  ".explain ?on|off|auto?   Change the EXPLAIN formatting mode.  Default: auto",
  ".filectrl CMD ...        Run various sqlite3_file_control() operations",
@@ -15376,6 +18800,7 @@ static const char *(azHelp[]) = {
  ".fullschema ?--indent?   Show schema and the content of sqlite_stat tables",
  ".headers on|off          Turn display of headers on or off",
  ".help ?-all? ?PATTERN?   Show help text for PATTERN",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".import FILE TABLE       Import data from FILE into TABLE",
  "   Options:",
  "     --ascii               Use \\037 and \\036 as column and row separators",
@@ -15390,6 +18815,7 @@ static const char *(azHelp[]) = {
  "        from the \".mode\" output mode",
  "     *  If FILE begins with \"|\" then it is a command that generates the",
  "        input text.",
+
#endif
#ifndef SQLITE_OMIT_TEST_CONTROL
  ".imposter INDEX TABLE    Create imposter table TABLE on index INDEX",
#endif
@@ -15403,10 +18829,12 @@ static const char *(azHelp[]) = {
  ".lint OPTIONS            Report potential schema issues.",
  "     Options:",
  "        fkey-indexes     Find missing foreign key indexes",
-
#ifndef SQLITE_OMIT_LOAD_EXTENSION
+
#if !defined(SQLITE_OMIT_LOAD_EXTENSION) && !defined(SQLITE_SHELL_FIDDLE)
  ".load FILE ?ENTRY?       Load an extension library",
#endif
+
#ifndef SQLITE_SHELL_FIDDLE
  ".log FILE|off            Turn logging on or off.  FILE can be stderr/stdout",
+
#endif
  ".mode MODE ?OPTIONS?     Set output mode",
  "   MODE is one of:",
  "     ascii       Columns/rows delimited by 0x1F and 0x1E",
@@ -15419,7 +18847,7 @@ static const char *(azHelp[]) = {
  "     line        One value per line",
  "     list        Values delimited by \"|\"",
  "     markdown    Markdown table format",
-
  "     qbox        Shorthand for \"box --width 60 --quote\"",
+
  "     qbox        Shorthand for \"box --wrap 60 --quote\"",
  "     quote       Escape answers as for SQL",
  "     table       ASCII-art table",
  "     tabs        Tab-separated values",
@@ -15431,16 +18859,23 @@ static const char *(azHelp[]) = {
  "     --quote        Quote output text as SQL literals",
  "     --noquote      Do not quote output text",
  "     TABLE          The name of SQL table used for \"insert\" mode",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".nonce STRING            Suspend safe mode for one command if nonce matches",
+
#endif
  ".nullvalue STRING        Use STRING in place of NULL values",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".once ?OPTIONS? ?FILE?   Output for the next SQL command only to FILE",
  "     If FILE begins with '|' then open as a pipe",
  "       --bom  Put a UTF8 byte-order mark at the beginning",
  "       -e     Send output to the system text editor",
  "       -x     Send output as CSV to a spreadsheet (same as \".excel\")",
+
  /* Note that .open is (partially) available in WASM builds but is
+
  ** currently only intended to be used by the fiddle tool, not
+
  ** end users, so is "undocumented." */
  ".open ?OPTIONS? ?FILE?   Close existing database and reopen FILE",
  "     Options:",
  "        --append        Use appendvfs to append database to the end of FILE",
+
#endif
#ifndef SQLITE_OMIT_DESERIALIZE
  "        --deserialize   Load into memory using sqlite3_deserialize()",
  "        --hexdb         Load the output of \"dbtotxt\" as an in-memory db",
@@ -15450,12 +18885,14 @@ static const char *(azHelp[]) = {
  "        --nofollow      Do not follow symbolic links",
  "        --readonly      Open FILE readonly",
  "        --zip           FILE is a ZIP archive",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".output ?FILE?           Send output to FILE or stdout if FILE is omitted",
  "   If FILE begins with '|' then open it as a pipe.",
  "   Options:",
  "     --bom                 Prefix output with a UTF8 byte-order mark",
  "     -e                    Send output to the system text editor",
  "     -x                    Send output as CSV to a spreadsheet",
+
#endif
  ".parameter CMD ...       Manage SQL parameter bindings",
  "   clear                   Erase all bindings",
  "   init                    Initialize the TEMP table that holds bindings",
@@ -15472,19 +18909,22 @@ static const char *(azHelp[]) = {
  "   --reset                   Reset the count for each input and interrupt",
#endif
  ".prompt MAIN CONTINUE    Replace the standard prompts",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".quit                    Exit this program",
  ".read FILE               Read input from FILE or command output",
  "    If FILE begins with \"|\", it is a command that generates the input.",
-
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
+
#endif
+
#if SQLITE_SHELL_HAVE_RECOVER
  ".recover                 Recover as much data as possible from corrupt db.",
-
  "   --freelist-corrupt       Assume the freelist is corrupt",
-
  "   --recovery-db NAME       Store recovery metadata in database file NAME",
+
  "   --ignore-freelist        Ignore pages that appear to be on db freelist",
  "   --lost-and-found TABLE   Alternative name for the lost-and-found table",
  "   --no-rowids              Do not attempt to recover rowid values",
  "                            that are not also INTEGER PRIMARY KEYs",
#endif
+
#ifndef SQLITE_SHELL_FIDDLE
  ".restore ?DB? FILE       Restore content of DB (default \"main\") from FILE",
  ".save ?OPTIONS? FILE     Write database to FILE (an alias for .backup ...)",
+
#endif
  ".scanstats on|off        Turn sqlite3_stmt_scanstatus() metrics on or off",
  ".schema ?PATTERN?        Show the CREATE statements matching PATTERN",
  "   Options:",
@@ -15518,7 +18958,7 @@ static const char *(azHelp[]) = {
  "      --sha3-384            Use the sha3-384 algorithm",
  "      --sha3-512            Use the sha3-512 algorithm",
  "    Any other argument is a LIKE pattern for tables to hash",
-
#ifndef SQLITE_NOHAVE_SYSTEM
+
#if !defined(SQLITE_NOHAVE_SYSTEM) && !defined(SQLITE_SHELL_FIDDLE)
  ".shell CMD ARGS...       Run CMD ARGS... in a system shell",
#endif
  ".show                    Show the current values for various settings",
@@ -15527,11 +18967,13 @@ static const char *(azHelp[]) = {
  "   on                       Turn on automatic stat display",
  "   stmt                     Show statement stats",
  "   vmstep                   Show the virtual machine step count only",
-
#ifndef SQLITE_NOHAVE_SYSTEM
+
#if !defined(SQLITE_NOHAVE_SYSTEM) && !defined(SQLITE_SHELL_FIDDLE)
  ".system CMD ARGS...      Run CMD ARGS... in a system shell",
#endif
  ".tables ?TABLE?          List names of tables matching LIKE pattern TABLE",
+
#ifndef SQLITE_SHELL_FIDDLE
  ".testcase NAME           Begin redirecting output to 'testcase-out.txt'",
+
#endif
  ".testctrl CMD ...        Run various sqlite3_test_control() operations",
  "                           Run \".testctrl\" with no arguments for details",
  ".timeout MS              Try opening locked tables for MS milliseconds",
@@ -15579,9 +19021,9 @@ static int showHelp(FILE *out, const char *zPattern){
  char *zPat;
  if( zPattern==0
   || zPattern[0]=='0'
-
   || strcmp(zPattern,"-a")==0
-
   || strcmp(zPattern,"-all")==0
-
   || strcmp(zPattern,"--all")==0
+
   || cli_strcmp(zPattern,"-a")==0
+
   || cli_strcmp(zPattern,"-all")==0
+
   || cli_strcmp(zPattern,"--all")==0
  ){
    /* Show all commands, but only one line per command */
    if( zPattern==0 ) zPattern = "";
@@ -15818,7 +19260,7 @@ static unsigned char *readHexDb(ShellState *p, int *pnData){
      iOffset = k;
      continue;
    }
-
    if( strncmp(zLine, "| end ", 6)==0 ){
+
    if( cli_strncmp(zLine, "| end ", 6)==0 ){
      break;
    }
    rc = sscanf(zLine,"| %d: %x %x %x %x %x %x %x %x %x %x %x %x %x %x %x %x",
@@ -15846,7 +19288,7 @@ readHexDb_error:
  }else{
    while( fgets(zLine, sizeof(zLine), p->in)!=0 ){
      nLine++;
-
      if(strncmp(zLine, "| end ", 6)==0 ) break;
+
      if(cli_strncmp(zLine, "| end ", 6)==0 ) break;
    }
    p->lineno = nLine;
  }
@@ -15938,28 +19380,28 @@ static void shellEscapeCrnl(
  const char *zText = (const char*)sqlite3_value_text(argv[0]);
  UNUSED_PARAMETER(argc);
  if( zText && zText[0]=='\'' ){
-
    int nText = sqlite3_value_bytes(argv[0]);
-
    int i;
+
    i64 nText = sqlite3_value_bytes(argv[0]);
+
    i64 i;
    char zBuf1[20];
    char zBuf2[20];
    const char *zNL = 0;
    const char *zCR = 0;
-
    int nCR = 0;
-
    int nNL = 0;
+
    i64 nCR = 0;
+
    i64 nNL = 0;

    for(i=0; zText[i]; i++){
      if( zNL==0 && zText[i]=='\n' ){
        zNL = unused_string(zText, "\\n", "\\012", zBuf1);
-
        nNL = (int)strlen(zNL);
+
        nNL = strlen(zNL);
      }
      if( zCR==0 && zText[i]=='\r' ){
        zCR = unused_string(zText, "\\r", "\\015", zBuf2);
-
        nCR = (int)strlen(zCR);
+
        nCR = strlen(zCR);
      }
    }

    if( zNL || zCR ){
-
      int iOut = 0;
+
      i64 iOut = 0;
      i64 nMax = (nNL > nCR) ? nNL : nCR;
      i64 nAlloc = nMax * nText + (nMax+64)*2;
      char *zOut = (char*)sqlite3_malloc64(nAlloc);
@@ -16076,20 +19518,24 @@ static void open_db(ShellState *p, int openFlags){
#ifndef SQLITE_OMIT_LOAD_EXTENSION
    sqlite3_enable_load_extension(p->db, 1);
#endif
-
    sqlite3_fileio_init(p->db, 0, 0);
    sqlite3_shathree_init(p->db, 0, 0);
-
    sqlite3_completion_init(p->db, 0, 0);
    sqlite3_uint_init(p->db, 0, 0);
    sqlite3_decimal_init(p->db, 0, 0);
    sqlite3_regexp_init(p->db, 0, 0);
    sqlite3_ieee_init(p->db, 0, 0);
    sqlite3_series_init(p->db, 0, 0);
-
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
+
#ifndef SQLITE_SHELL_FIDDLE
+
    sqlite3_fileio_init(p->db, 0, 0);
+
    sqlite3_completion_init(p->db, 0, 0);
+
#endif
+
#if SQLITE_SHELL_HAVE_RECOVER
    sqlite3_dbdata_init(p->db, 0, 0);
#endif
#ifdef SQLITE_HAVE_ZLIB
-
    sqlite3_zipfile_init(p->db, 0, 0);
-
    sqlite3_sqlar_init(p->db, 0, 0);
+
    if( !p->bSafeModePersist ){
+
      sqlite3_zipfile_init(p->db, 0, 0);
+
      sqlite3_sqlar_init(p->db, 0, 0);
+
    }
#endif
    sqlite3_create_function(p->db, "shell_add_schema", 3, SQLITE_UTF8, 0,
                            shellAddSchemaName, 0, 0);
@@ -16196,8 +19642,8 @@ static char **readline_completion(const char *zText, int iStart, int iEnd){
** Linenoise completion callback
*/
static void linenoise_completion(const char *zLine, linenoiseCompletions *lc){
-
  int nLine = strlen30(zLine);
-
  int i, iStart;
+
  i64 nLine = strlen(zLine);
+
  i64 i, iStart;
  sqlite3_stmt *pStmt = 0;
  char *zSql;
  char zBuf[1000];
@@ -16335,11 +19781,11 @@ static void output_file_close(FILE *f){
*/
static FILE *output_file_open(const char *zFile, int bTextMode){
  FILE *f;
-
  if( strcmp(zFile,"stdout")==0 ){
+
  if( cli_strcmp(zFile,"stdout")==0 ){
    f = stdout;
-
  }else if( strcmp(zFile, "stderr")==0 ){
+
  }else if( cli_strcmp(zFile, "stderr")==0 ){
    f = stderr;
-
  }else if( strcmp(zFile, "off")==0 ){
+
  }else if( cli_strcmp(zFile, "off")==0 ){
    f = 0;
  }else{
    f = fopen(zFile, bTextMode ? "w" : "wb");
@@ -16363,7 +19809,7 @@ static int sql_trace_callback(
  ShellState *p = (ShellState*)pArg;
  sqlite3_stmt *pStmt;
  const char *zSql;
-
  int nSql;
+
  i64 nSql;
  if( p->traceOut==0 ) return 0;
  if( mType==SQLITE_TRACE_CLOSE ){
    utf8_printf(p->traceOut, "-- closing database connection\n");
@@ -16391,17 +19837,18 @@ static int sql_trace_callback(
    }
  }
  if( zSql==0 ) return 0;
-
  nSql = strlen30(zSql);
+
  nSql = strlen(zSql);
+
  if( nSql>1000000000 ) nSql = 1000000000;
  while( nSql>0 && zSql[nSql-1]==';' ){ nSql--; }
  switch( mType ){
    case SQLITE_TRACE_ROW:
    case SQLITE_TRACE_STMT: {
-
      utf8_printf(p->traceOut, "%.*s;\n", nSql, zSql);
+
      utf8_printf(p->traceOut, "%.*s;\n", (int)nSql, zSql);
      break;
    }
    case SQLITE_TRACE_PROFILE: {
      sqlite3_int64 nNanosec = *(sqlite3_int64*)pX;
-
      utf8_printf(p->traceOut, "%.*s; -- %lld ns\n", nSql, zSql, nNanosec);
+
      utf8_printf(p->traceOut, "%.*s; -- %lld ns\n", (int)nSql, zSql, nNanosec);
      break;
    }
  }
@@ -16859,6 +20306,7 @@ static int db_int(sqlite3 *db, const char *zSql){
  return res;
}

+
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
/*
** Convert a 2-byte or 4-byte big-endian integer into a native integer
*/
@@ -16949,7 +20397,7 @@ static int shell_dbinfo_command(ShellState *p, int nArg, char **azArg){
  }
  if( zDb==0 ){
    zSchemaTab = sqlite3_mprintf("main.sqlite_schema");
-
  }else if( strcmp(zDb,"temp")==0 ){
+
  }else if( cli_strcmp(zDb,"temp")==0 ){
    zSchemaTab = sqlite3_mprintf("%s", "sqlite_temp_schema");
  }else{
    zSchemaTab = sqlite3_mprintf("\"%w\".sqlite_schema", zDb);
@@ -16965,6 +20413,7 @@ static int shell_dbinfo_command(ShellState *p, int nArg, char **azArg){
  utf8_printf(p->out, "%-20s %u\n", "data version", iDataVersion);
  return 0;
}
+
#endif /* SQLITE_SHELL_HAVE_RECOVER */

/*
** Print the current sqlite3_errmsg() value to stderr and return 1.
@@ -17081,7 +20530,7 @@ static int optionMatch(const char *zStr, const char *zOpt){
  if( zStr[0]!='-' ) return 0;
  zStr++;
  if( zStr[0]=='-' ) zStr++;
-
  return strcmp(zStr, zOpt)==0;
+
  return cli_strcmp(zStr, zOpt)==0;
}

/*
@@ -18192,409 +21641,61 @@ static int arDotCommand(
        goto end_ar_command;
      }
      cmd.zSrcTable = sqlite3_mprintf("sqlar");
-
    }
-

-
    switch( cmd.eCmd ){
-
      case AR_CMD_CREATE:
-
        rc = arCreateOrUpdateCommand(&cmd, 0, 0);
-
        break;
-

-
      case AR_CMD_EXTRACT:
-
        rc = arExtractCommand(&cmd);
-
        break;
-

-
      case AR_CMD_LIST:
-
        rc = arListCommand(&cmd);
-
        break;
-

-
      case AR_CMD_HELP:
-
        arUsage(pState->out);
-
        break;
-

-
      case AR_CMD_INSERT:
-
        rc = arCreateOrUpdateCommand(&cmd, 1, 0);
-
        break;
-

-
      case AR_CMD_REMOVE:
-
        rc = arRemoveCommand(&cmd);
-
        break;
-

-
      default:
-
        assert( cmd.eCmd==AR_CMD_UPDATE );
-
        rc = arCreateOrUpdateCommand(&cmd, 1, 1);
-
        break;
-
    }
-
  }
-
end_ar_command:
-
  if( cmd.db!=pState->db ){
-
    close_db(cmd.db);
-
  }
-
  sqlite3_free(cmd.zSrcTable);
-

-
  return rc;
-
}
-
/* End of the ".archive" or ".ar" command logic
-
*******************************************************************************/
-
#endif /* !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB) */
-

-
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
-
/*
-
** If (*pRc) is not SQLITE_OK when this function is called, it is a no-op.
-
** Otherwise, the SQL statement or statements in zSql are executed using
-
** database connection db and the error code written to *pRc before
-
** this function returns.
-
*/
-
static void shellExec(sqlite3 *db, int *pRc, const char *zSql){
-
  int rc = *pRc;
-
  if( rc==SQLITE_OK ){
-
    char *zErr = 0;
-
    rc = sqlite3_exec(db, zSql, 0, 0, &zErr);
-
    if( rc!=SQLITE_OK ){
-
      raw_printf(stderr, "SQL error: %s\n", zErr);
-
    }
-
    sqlite3_free(zErr);
-
    *pRc = rc;
-
  }
-
}
-

-
/*
-
** Like shellExec(), except that zFmt is a printf() style format string.
-
*/
-
static void shellExecPrintf(sqlite3 *db, int *pRc, const char *zFmt, ...){
-
  char *z = 0;
-
  if( *pRc==SQLITE_OK ){
-
    va_list ap;
-
    va_start(ap, zFmt);
-
    z = sqlite3_vmprintf(zFmt, ap);
-
    va_end(ap);
-
    if( z==0 ){
-
      *pRc = SQLITE_NOMEM;
-
    }else{
-
      shellExec(db, pRc, z);
-
    }
-
    sqlite3_free(z);
-
  }
-
}
-

-
/*
-
** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
-
** Otherwise, an attempt is made to allocate, zero and return a pointer
-
** to a buffer nByte bytes in size. If an OOM error occurs, *pRc is set
-
** to SQLITE_NOMEM and NULL returned.
-
*/
-
static void *shellMalloc(int *pRc, sqlite3_int64 nByte){
-
  void *pRet = 0;
-
  if( *pRc==SQLITE_OK ){
-
    pRet = sqlite3_malloc64(nByte);
-
    if( pRet==0 ){
-
      *pRc = SQLITE_NOMEM;
-
    }else{
-
      memset(pRet, 0, nByte);
-
    }
-
  }
-
  return pRet;
-
}
-

-
/*
-
** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
-
** Otherwise, zFmt is treated as a printf() style string. The result of
-
** formatting it along with any trailing arguments is written into a 
-
** buffer obtained from sqlite3_malloc(), and pointer to which is returned.
-
** It is the responsibility of the caller to eventually free this buffer
-
** using a call to sqlite3_free().
-
** 
-
** If an OOM error occurs, (*pRc) is set to SQLITE_NOMEM and a NULL 
-
** pointer returned.
-
*/
-
static char *shellMPrintf(int *pRc, const char *zFmt, ...){
-
  char *z = 0;
-
  if( *pRc==SQLITE_OK ){
-
    va_list ap;
-
    va_start(ap, zFmt);
-
    z = sqlite3_vmprintf(zFmt, ap);
-
    va_end(ap);
-
    if( z==0 ){
-
      *pRc = SQLITE_NOMEM;
-
    }
-
  }
-
  return z;
-
}
-

-

-
/*
-
** When running the ".recover" command, each output table, and the special
-
** orphaned row table if it is required, is represented by an instance
-
** of the following struct.
-
*/
-
typedef struct RecoverTable RecoverTable;
-
struct RecoverTable {
-
  char *zQuoted;                  /* Quoted version of table name */
-
  int nCol;                       /* Number of columns in table */
-
  char **azlCol;                  /* Array of column lists */
-
  int iPk;                        /* Index of IPK column */
-
};
-

-
/*
-
** Free a RecoverTable object allocated by recoverFindTable() or
-
** recoverOrphanTable().
-
*/
-
static void recoverFreeTable(RecoverTable *pTab){
-
  if( pTab ){
-
    sqlite3_free(pTab->zQuoted);
-
    if( pTab->azlCol ){
-
      int i;
-
      for(i=0; i<=pTab->nCol; i++){
-
        sqlite3_free(pTab->azlCol[i]);
-
      }
-
      sqlite3_free(pTab->azlCol);
-
    }
-
    sqlite3_free(pTab);
-
  }
-
}
-

-
/*
-
** This function is a no-op if (*pRc) is not SQLITE_OK when it is called.
-
** Otherwise, it allocates and returns a RecoverTable object based on the
-
** final four arguments passed to this function. It is the responsibility
-
** of the caller to eventually free the returned object using
-
** recoverFreeTable().
-
*/
-
static RecoverTable *recoverNewTable(
-
  int *pRc,                       /* IN/OUT: Error code */
-
  const char *zName,              /* Name of table */
-
  const char *zSql,               /* CREATE TABLE statement */
-
  int bIntkey, 
-
  int nCol
-
){
-
  sqlite3 *dbtmp = 0;             /* sqlite3 handle for testing CREATE TABLE */
-
  int rc = *pRc;
-
  RecoverTable *pTab = 0;
-

-
  pTab = (RecoverTable*)shellMalloc(&rc, sizeof(RecoverTable));
-
  if( rc==SQLITE_OK ){
-
    int nSqlCol = 0;
-
    int bSqlIntkey = 0;
-
    sqlite3_stmt *pStmt = 0;
-
    
-
    rc = sqlite3_open("", &dbtmp);
-
    if( rc==SQLITE_OK ){
-
      sqlite3_create_function(dbtmp, "shell_idquote", 1, SQLITE_UTF8, 0,
-
                              shellIdQuote, 0, 0);
-
    }
-
    if( rc==SQLITE_OK ){
-
      rc = sqlite3_exec(dbtmp, "PRAGMA writable_schema = on", 0, 0, 0);
-
    }
-
    if( rc==SQLITE_OK ){
-
      rc = sqlite3_exec(dbtmp, zSql, 0, 0, 0);
-
      if( rc==SQLITE_ERROR ){
-
        rc = SQLITE_OK;
-
        goto finished;
-
      }
-
    }
-
    shellPreparePrintf(dbtmp, &rc, &pStmt, 
-
        "SELECT count(*) FROM pragma_table_info(%Q)", zName
-
    );
-
    if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
-
      nSqlCol = sqlite3_column_int(pStmt, 0);
-
    }
-
    shellFinalize(&rc, pStmt);
-

-
    if( rc!=SQLITE_OK || nSqlCol<nCol ){
-
      goto finished;
-
    }
-

-
    shellPreparePrintf(dbtmp, &rc, &pStmt, 
-
      "SELECT ("
-
      "  SELECT substr(data,1,1)==X'0D' FROM sqlite_dbpage WHERE pgno=rootpage"
-
      ") FROM sqlite_schema WHERE name = %Q", zName
-
    );
-
    if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
-
      bSqlIntkey = sqlite3_column_int(pStmt, 0);
-
    }
-
    shellFinalize(&rc, pStmt);
-

-
    if( bIntkey==bSqlIntkey ){
-
      int i;
-
      const char *zPk = "_rowid_";
-
      sqlite3_stmt *pPkFinder = 0;
-

-
      /* If this is an intkey table and there is an INTEGER PRIMARY KEY,
-
      ** set zPk to the name of the PK column, and pTab->iPk to the index
-
      ** of the column, where columns are 0-numbered from left to right.
-
      ** Or, if this is a WITHOUT ROWID table or if there is no IPK column,
-
      ** leave zPk as "_rowid_" and pTab->iPk at -2.  */
-
      pTab->iPk = -2;
-
      if( bIntkey ){
-
        shellPreparePrintf(dbtmp, &rc, &pPkFinder, 
-
          "SELECT cid, name FROM pragma_table_info(%Q) "
-
          "  WHERE pk=1 AND type='integer' COLLATE nocase"
-
          "  AND NOT EXISTS (SELECT cid FROM pragma_table_info(%Q) WHERE pk=2)"
-
          , zName, zName
-
        );
-
        if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pPkFinder) ){
-
          pTab->iPk = sqlite3_column_int(pPkFinder, 0);
-
          zPk = (const char*)sqlite3_column_text(pPkFinder, 1);
-
          if( zPk==0 ){ zPk = "_";  /* Defensive.  Should never happen */ }
-
        }
-
      }
+
    }

-
      pTab->zQuoted = shellMPrintf(&rc, "\"%w\"", zName);
-
      pTab->azlCol = (char**)shellMalloc(&rc, sizeof(char*) * (nSqlCol+1));
-
      pTab->nCol = nSqlCol;
+
    switch( cmd.eCmd ){
+
      case AR_CMD_CREATE:
+
        rc = arCreateOrUpdateCommand(&cmd, 0, 0);
+
        break;

-
      if( bIntkey ){
-
        pTab->azlCol[0] = shellMPrintf(&rc, "\"%w\"", zPk);
-
      }else{
-
        pTab->azlCol[0] = shellMPrintf(&rc, "");
-
      }
-
      i = 1;
-
      shellPreparePrintf(dbtmp, &rc, &pStmt, 
-
          "SELECT %Q || group_concat(shell_idquote(name), ', ') "
-
          "  FILTER (WHERE cid!=%d) OVER (ORDER BY %s cid) "
-
          "FROM pragma_table_info(%Q)", 
-
          bIntkey ? ", " : "", pTab->iPk, 
-
          bIntkey ? "" : "(CASE WHEN pk=0 THEN 1000000 ELSE pk END), ",
-
          zName
-
      );
-
      while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
-
        const char *zText = (const char*)sqlite3_column_text(pStmt, 0);
-
        pTab->azlCol[i] = shellMPrintf(&rc, "%s%s", pTab->azlCol[0], zText);
-
        i++;
-
      }
-
      shellFinalize(&rc, pStmt);
+
      case AR_CMD_EXTRACT:
+
        rc = arExtractCommand(&cmd);
+
        break;

-
      shellFinalize(&rc, pPkFinder);
-
    }
-
  }
+
      case AR_CMD_LIST:
+
        rc = arListCommand(&cmd);
+
        break;

-
 finished:
-
  sqlite3_close(dbtmp);
-
  *pRc = rc;
-
  if( rc!=SQLITE_OK || (pTab && pTab->zQuoted==0) ){
-
    recoverFreeTable(pTab);
-
    pTab = 0;
-
  }
-
  return pTab;
-
}
+
      case AR_CMD_HELP:
+
        arUsage(pState->out);
+
        break;

-
/*
-
** This function is called to search the schema recovered from the
-
** sqlite_schema table of the (possibly) corrupt database as part
-
** of a ".recover" command. Specifically, for a table with root page
-
** iRoot and at least nCol columns. Additionally, if bIntkey is 0, the
-
** table must be a WITHOUT ROWID table, or if non-zero, not one of
-
** those.
-
**
-
** If a table is found, a (RecoverTable*) object is returned. Or, if
-
** no such table is found, but bIntkey is false and iRoot is the 
-
** root page of an index in the recovered schema, then (*pbNoop) is
-
** set to true and NULL returned. Or, if there is no such table or
-
** index, NULL is returned and (*pbNoop) set to 0, indicating that
-
** the caller should write data to the orphans table.
-
*/
-
static RecoverTable *recoverFindTable(
-
  ShellState *pState,             /* Shell state object */
-
  int *pRc,                       /* IN/OUT: Error code */
-
  int iRoot,                      /* Root page of table */
-
  int bIntkey,                    /* True for an intkey table */
-
  int nCol,                       /* Number of columns in table */
-
  int *pbNoop                     /* OUT: True if iRoot is root of index */
-
){
-
  sqlite3_stmt *pStmt = 0;
-
  RecoverTable *pRet = 0;
-
  int bNoop = 0;
-
  const char *zSql = 0;
-
  const char *zName = 0;
+
      case AR_CMD_INSERT:
+
        rc = arCreateOrUpdateCommand(&cmd, 1, 0);
+
        break;

-
  /* Search the recovered schema for an object with root page iRoot. */
-
  shellPreparePrintf(pState->db, pRc, &pStmt,
-
      "SELECT type, name, sql FROM recovery.schema WHERE rootpage=%d", iRoot
-
  );
-
  while( *pRc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
-
    const char *zType = (const char*)sqlite3_column_text(pStmt, 0);
-
    if( bIntkey==0 && sqlite3_stricmp(zType, "index")==0 ){
-
      bNoop = 1;
-
      break;
-
    }
-
    if( sqlite3_stricmp(zType, "table")==0 ){
-
      zName = (const char*)sqlite3_column_text(pStmt, 1);
-
      zSql = (const char*)sqlite3_column_text(pStmt, 2);
-
      if( zName!=0 && zSql!=0 ){
-
        pRet = recoverNewTable(pRc, zName, zSql, bIntkey, nCol);
+
      case AR_CMD_REMOVE:
+
        rc = arRemoveCommand(&cmd);
+
        break;
+

+
      default:
+
        assert( cmd.eCmd==AR_CMD_UPDATE );
+
        rc = arCreateOrUpdateCommand(&cmd, 1, 1);
        break;
-
      }
    }
  }
+
end_ar_command:
+
  if( cmd.db!=pState->db ){
+
    close_db(cmd.db);
+
  }
+
  sqlite3_free(cmd.zSrcTable);

-
  shellFinalize(pRc, pStmt);
-
  *pbNoop = bNoop;
-
  return pRet;
+
  return rc;
}
+
/* End of the ".archive" or ".ar" command logic
+
*******************************************************************************/
+
#endif /* !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB) */
+

+
#if SQLITE_SHELL_HAVE_RECOVER

/*
-
** Return a RecoverTable object representing the orphans table.
+
** This function is used as a callback by the recover extension. Simply
+
** print the supplied SQL statement to stdout.
*/
-
static RecoverTable *recoverOrphanTable(
-
  ShellState *pState,             /* Shell state object */
-
  int *pRc,                       /* IN/OUT: Error code */
-
  const char *zLostAndFound,      /* Base name for orphans table */
-
  int nCol                        /* Number of user data columns */
-
){
-
  RecoverTable *pTab = 0;
-
  if( nCol>=0 && *pRc==SQLITE_OK ){
-
    int i;
-

-
    /* This block determines the name of the orphan table. The prefered
-
    ** name is zLostAndFound. But if that clashes with another name
-
    ** in the recovered schema, try zLostAndFound_0, zLostAndFound_1
-
    ** and so on until a non-clashing name is found.  */
-
    int iTab = 0;
-
    char *zTab = shellMPrintf(pRc, "%s", zLostAndFound);
-
    sqlite3_stmt *pTest = 0;
-
    shellPrepare(pState->db, pRc,
-
        "SELECT 1 FROM recovery.schema WHERE name=?", &pTest
-
    );
-
    if( pTest ) sqlite3_bind_text(pTest, 1, zTab, -1, SQLITE_TRANSIENT);
-
    while( *pRc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pTest) ){
-
      shellReset(pRc, pTest);
-
      sqlite3_free(zTab);
-
      zTab = shellMPrintf(pRc, "%s_%d", zLostAndFound, iTab++);
-
      sqlite3_bind_text(pTest, 1, zTab, -1, SQLITE_TRANSIENT);
-
    }
-
    shellFinalize(pRc, pTest);
-

-
    pTab = (RecoverTable*)shellMalloc(pRc, sizeof(RecoverTable));
-
    if( pTab ){
-
      pTab->zQuoted = shellMPrintf(pRc, "\"%w\"", zTab);
-
      pTab->nCol = nCol;
-
      pTab->iPk = -2;
-
      if( nCol>0 ){
-
        pTab->azlCol = (char**)shellMalloc(pRc, sizeof(char*) * (nCol+1));
-
        if( pTab->azlCol ){
-
          pTab->azlCol[nCol] = shellMPrintf(pRc, "");
-
          for(i=nCol-1; i>=0; i--){
-
            pTab->azlCol[i] = shellMPrintf(pRc, "%s, NULL", pTab->azlCol[i+1]);
-
          }
-
        }
-
      }
-

-
      if( *pRc!=SQLITE_OK ){
-
        recoverFreeTable(pTab);
-
        pTab = 0;
-
      }else{
-
        raw_printf(pState->out, 
-
            "CREATE TABLE %s(rootpgno INTEGER, "
-
            "pgno INTEGER, nfield INTEGER, id INTEGER", pTab->zQuoted
-
        );
-
        for(i=0; i<nCol; i++){
-
          raw_printf(pState->out, ", c%d", i);
-
        }
-
        raw_printf(pState->out, ");\n");
-
      }
-
    }
-
    sqlite3_free(zTab);
-
  }
-
  return pTab;
+
static int recoverSqlCb(void *pCtx, const char *zSql){
+
  ShellState *pState = (ShellState*)pCtx;
+
  utf8_printf(pState->out, "%s;\n", zSql);
+
  return SQLITE_OK;
}

/*
@@ -18604,32 +21705,33 @@ static RecoverTable *recoverOrphanTable(
*/
static int recoverDatabaseCmd(ShellState *pState, int nArg, char **azArg){
  int rc = SQLITE_OK;
-
  sqlite3_stmt *pLoop = 0;        /* Loop through all root pages */
-
  sqlite3_stmt *pPages = 0;       /* Loop through all pages in a group */
-
  sqlite3_stmt *pCells = 0;       /* Loop through all cells in a page */
-
  const char *zRecoveryDb = "";   /* Name of "recovery" database */
-
  const char *zLostAndFound = "lost_and_found";
-
  int i;
-
  int nOrphan = -1;
-
  RecoverTable *pOrphan = 0;
-

-
  int bFreelist = 1;              /* 0 if --freelist-corrupt is specified */
+
  const char *zRecoveryDb = "";   /* Name of "recovery" database.  Debug only */
+
  const char *zLAF = "lost_and_found";
+
  int bFreelist = 1;              /* 0 if --ignore-freelist is specified */
  int bRowids = 1;                /* 0 if --no-rowids */
+
  sqlite3_recover *p = 0;
+
  int i = 0;
+

  for(i=1; i<nArg; i++){
    char *z = azArg[i];
    int n;
    if( z[0]=='-' && z[1]=='-' ) z++;
    n = strlen30(z);
-
    if( n<=17 && memcmp("-freelist-corrupt", z, n)==0 ){
+
    if( n<=17 && memcmp("-ignore-freelist", z, n)==0 ){
      bFreelist = 0;
    }else
    if( n<=12 && memcmp("-recovery-db", z, n)==0 && i<(nArg-1) ){
+
      /* This option determines the name of the ATTACH-ed database used
+
      ** internally by the recovery extension.  The default is "" which
+
      ** means to use a temporary database that is automatically deleted
+
      ** when closed.  This option is undocumented and might disappear at
+
      ** any moment. */
      i++;
      zRecoveryDb = azArg[i];
    }else
    if( n<=15 && memcmp("-lost-and-found", z, n)==0 && i<(nArg-1) ){
      i++;
-
      zLostAndFound = azArg[i];
+
      zLAF = azArg[i];
    }else
    if( n<=10 && memcmp("-no-rowids", z, n)==0 ){
      bRowids = 0;
@@ -18641,284 +21743,28 @@ static int recoverDatabaseCmd(ShellState *pState, int nArg, char **azArg){
    }
  }

-
  shellExecPrintf(pState->db, &rc,
-
    /* Attach an in-memory database named 'recovery'. Create an indexed 
-
    ** cache of the sqlite_dbptr virtual table. */
-
    "PRAGMA writable_schema = on;"
-
    "ATTACH %Q AS recovery;"
-
    "DROP TABLE IF EXISTS recovery.dbptr;"
-
    "DROP TABLE IF EXISTS recovery.freelist;"
-
    "DROP TABLE IF EXISTS recovery.map;"
-
    "DROP TABLE IF EXISTS recovery.schema;"
-
    "CREATE TABLE recovery.freelist(pgno INTEGER PRIMARY KEY);", zRecoveryDb
-
  );
-

-
  if( bFreelist ){
-
    shellExec(pState->db, &rc,
-
      "WITH trunk(pgno) AS ("
-
      "  SELECT shell_int32("
-
      "      (SELECT data FROM sqlite_dbpage WHERE pgno=1), 8) AS x "
-
      "      WHERE x>0"
-
      "    UNION"
-
      "  SELECT shell_int32("
-
      "      (SELECT data FROM sqlite_dbpage WHERE pgno=trunk.pgno), 0) AS x "
-
      "      FROM trunk WHERE x>0"
-
      "),"
-
      "freelist(data, n, freepgno) AS ("
-
      "  SELECT data, min(16384, shell_int32(data, 1)-1), t.pgno "
-
      "      FROM trunk t, sqlite_dbpage s WHERE s.pgno=t.pgno"
-
      "    UNION ALL"
-
      "  SELECT data, n-1, shell_int32(data, 2+n) "
-
      "      FROM freelist WHERE n>=0"
-
      ")"
-
      "REPLACE INTO recovery.freelist SELECT freepgno FROM freelist;"
-
    );
-
  }
-

-
  /* If this is an auto-vacuum database, add all pointer-map pages to
-
  ** the freelist table. Do this regardless of whether or not 
-
  ** --freelist-corrupt was specified.  */
-
  shellExec(pState->db, &rc, 
-
    "WITH ptrmap(pgno) AS ("
-
    "  SELECT 2 WHERE shell_int32("
-
    "    (SELECT data FROM sqlite_dbpage WHERE pgno=1), 13"
-
    "  )"
-
    "    UNION ALL "
-
    "  SELECT pgno+1+(SELECT page_size FROM pragma_page_size)/5 AS pp "
-
    "  FROM ptrmap WHERE pp<=(SELECT page_count FROM pragma_page_count)"
-
    ")"
-
    "REPLACE INTO recovery.freelist SELECT pgno FROM ptrmap"
-
  );
-

-
  shellExec(pState->db, &rc, 
-
    "CREATE TABLE recovery.dbptr("
-
    "      pgno, child, PRIMARY KEY(child, pgno)"
-
    ") WITHOUT ROWID;"
-
    "INSERT OR IGNORE INTO recovery.dbptr(pgno, child) "
-
    "    SELECT * FROM sqlite_dbptr"
-
    "      WHERE pgno NOT IN freelist AND child NOT IN freelist;"
-

-
    /* Delete any pointer to page 1. This ensures that page 1 is considered
-
    ** a root page, regardless of how corrupt the db is. */
-
    "DELETE FROM recovery.dbptr WHERE child = 1;"
-

-
    /* Delete all pointers to any pages that have more than one pointer
-
    ** to them. Such pages will be treated as root pages when recovering
-
    ** data.  */
-
    "DELETE FROM recovery.dbptr WHERE child IN ("
-
    "  SELECT child FROM recovery.dbptr GROUP BY child HAVING count(*)>1"
-
    ");"
-

-
    /* Create the "map" table that will (eventually) contain instructions
-
    ** for dealing with each page in the db that contains one or more 
-
    ** records. */
-
    "CREATE TABLE recovery.map("
-
      "pgno INTEGER PRIMARY KEY, maxlen INT, intkey, root INT"
-
    ");"
-

-
    /* Populate table [map]. If there are circular loops of pages in the
-
    ** database, the following adds all pages in such a loop to the map
-
    ** as individual root pages. This could be handled better.  */
-
    "WITH pages(i, maxlen) AS ("
-
    "  SELECT page_count, ("
-
    "    SELECT max(field+1) FROM sqlite_dbdata WHERE pgno=page_count"
-
    "  ) FROM pragma_page_count WHERE page_count>0"
-
    "    UNION ALL"
-
    "  SELECT i-1, ("
-
    "    SELECT max(field+1) FROM sqlite_dbdata WHERE pgno=i-1"
-
    "  ) FROM pages WHERE i>=2"
-
    ")"
-
    "INSERT INTO recovery.map(pgno, maxlen, intkey, root) "
-
    "  SELECT i, maxlen, NULL, ("
-
    "    WITH p(orig, pgno, parent) AS ("
-
    "      SELECT 0, i, (SELECT pgno FROM recovery.dbptr WHERE child=i)"
-
    "        UNION "
-
    "      SELECT i, p.parent, "
-
    "        (SELECT pgno FROM recovery.dbptr WHERE child=p.parent) FROM p"
-
    "    )"
-
    "    SELECT pgno FROM p WHERE (parent IS NULL OR pgno = orig)"
-
    ") "
-
    "FROM pages WHERE maxlen IS NOT NULL AND i NOT IN freelist;"
-
    "UPDATE recovery.map AS o SET intkey = ("
-
    "  SELECT substr(data, 1, 1)==X'0D' FROM sqlite_dbpage WHERE pgno=o.pgno"
-
    ");"
-

-
    /* Extract data from page 1 and any linked pages into table
-
    ** recovery.schema. With the same schema as an sqlite_schema table.  */
-
    "CREATE TABLE recovery.schema(type, name, tbl_name, rootpage, sql);"
-
    "INSERT INTO recovery.schema SELECT "
-
    "  max(CASE WHEN field=0 THEN value ELSE NULL END),"
-
    "  max(CASE WHEN field=1 THEN value ELSE NULL END),"
-
    "  max(CASE WHEN field=2 THEN value ELSE NULL END),"
-
    "  max(CASE WHEN field=3 THEN value ELSE NULL END),"
-
    "  max(CASE WHEN field=4 THEN value ELSE NULL END)"
-
    "FROM sqlite_dbdata WHERE pgno IN ("
-
    "  SELECT pgno FROM recovery.map WHERE root=1"
-
    ")"
-
    "GROUP BY pgno, cell;"
-
    "CREATE INDEX recovery.schema_rootpage ON schema(rootpage);"
-
  );
-

-
  /* Open a transaction, then print out all non-virtual, non-"sqlite_%" 
-
  ** CREATE TABLE statements that extracted from the existing schema.  */
-
  if( rc==SQLITE_OK ){
-
    sqlite3_stmt *pStmt = 0;
-
    /* ".recover" might output content in an order which causes immediate
-
    ** foreign key constraints to be violated. So disable foreign-key
-
    ** constraint enforcement to prevent problems when running the output
-
    ** script. */
-
    raw_printf(pState->out, "PRAGMA foreign_keys=OFF;\n");
-
    raw_printf(pState->out, "BEGIN;\n");
-
    raw_printf(pState->out, "PRAGMA writable_schema = on;\n");
-
    shellPrepare(pState->db, &rc,
-
        "SELECT sql FROM recovery.schema "
-
        "WHERE type='table' AND sql LIKE 'create table%'", &pStmt
-
    );
-
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
-
      const char *zCreateTable = (const char*)sqlite3_column_text(pStmt, 0);
-
      raw_printf(pState->out, "CREATE TABLE IF NOT EXISTS %s;\n", 
-
          &zCreateTable[12]
-
      );
-
    }
-
    shellFinalize(&rc, pStmt);
-
  }
-

-
  /* Figure out if an orphan table will be required. And if so, how many
-
  ** user columns it should contain */
-
  shellPrepare(pState->db, &rc, 
-
      "SELECT coalesce(max(maxlen), -2) FROM recovery.map WHERE root>1"
-
      , &pLoop
-
  );
-
  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pLoop) ){
-
    nOrphan = sqlite3_column_int(pLoop, 0);
-
  }
-
  shellFinalize(&rc, pLoop);
-
  pLoop = 0;
-

-
  shellPrepare(pState->db, &rc,
-
      "SELECT pgno FROM recovery.map WHERE root=?", &pPages
-
  );
-

-
  shellPrepare(pState->db, &rc,
-
      "SELECT max(field), group_concat(shell_escape_crnl(quote"
-
      "(case when (? AND field<0) then NULL else value end)"
-
      "), ', ')"
-
      ", min(field) "
-
      "FROM sqlite_dbdata WHERE pgno = ? AND field != ?"
-
      "GROUP BY cell", &pCells
-
  );
-

-
  /* Loop through each root page. */
-
  shellPrepare(pState->db, &rc, 
-
      "SELECT root, intkey, max(maxlen) FROM recovery.map" 
-
      " WHERE root>1 GROUP BY root, intkey ORDER BY root=("
-
      "  SELECT rootpage FROM recovery.schema WHERE name='sqlite_sequence'"
-
      ")", &pLoop
+
  p = sqlite3_recover_init_sql(
+
      pState->db, "main", recoverSqlCb, (void*)pState
  );
-
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pLoop) ){
-
    int iRoot = sqlite3_column_int(pLoop, 0);
-
    int bIntkey = sqlite3_column_int(pLoop, 1);
-
    int nCol = sqlite3_column_int(pLoop, 2);
-
    int bNoop = 0;
-
    RecoverTable *pTab;
-

-
    assert( bIntkey==0 || bIntkey==1 );
-
    pTab = recoverFindTable(pState, &rc, iRoot, bIntkey, nCol, &bNoop);
-
    if( bNoop || rc ) continue;
-
    if( pTab==0 ){
-
      if( pOrphan==0 ){
-
        pOrphan = recoverOrphanTable(pState, &rc, zLostAndFound, nOrphan);
-
      }
-
      pTab = pOrphan;
-
      if( pTab==0 ) break;
-
    }
-

-
    if( 0==sqlite3_stricmp(pTab->zQuoted, "\"sqlite_sequence\"") ){
-
      raw_printf(pState->out, "DELETE FROM sqlite_sequence;\n");
-
    }
-
    sqlite3_bind_int(pPages, 1, iRoot);
-
    if( bRowids==0 && pTab->iPk<0 ){
-
      sqlite3_bind_int(pCells, 1, 1);
-
    }else{
-
      sqlite3_bind_int(pCells, 1, 0);
-
    }
-
    sqlite3_bind_int(pCells, 3, pTab->iPk);
-

-
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pPages) ){
-
      int iPgno = sqlite3_column_int(pPages, 0);
-
      sqlite3_bind_int(pCells, 2, iPgno);
-
      while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pCells) ){
-
        int nField = sqlite3_column_int(pCells, 0);
-
        int iMin = sqlite3_column_int(pCells, 2);
-
        const char *zVal = (const char*)sqlite3_column_text(pCells, 1);
-

-
        RecoverTable *pTab2 = pTab;
-
        if( pTab!=pOrphan && (iMin<0)!=bIntkey ){
-
          if( pOrphan==0 ){
-
            pOrphan = recoverOrphanTable(pState, &rc, zLostAndFound, nOrphan);
-
          }
-
          pTab2 = pOrphan;
-
          if( pTab2==0 ) break;
-
        }
-

-
        nField = nField+1;
-
        if( pTab2==pOrphan ){
-
          raw_printf(pState->out, 
-
              "INSERT INTO %s VALUES(%d, %d, %d, %s%s%s);\n",
-
              pTab2->zQuoted, iRoot, iPgno, nField,
-
              iMin<0 ? "" : "NULL, ", zVal, pTab2->azlCol[nField]
-
          );
-
        }else{
-
          raw_printf(pState->out, "INSERT INTO %s(%s) VALUES( %s );\n", 
-
              pTab2->zQuoted, pTab2->azlCol[nField], zVal
-
          );
-
        }
-
      }
-
      shellReset(&rc, pCells);
-
    }
-
    shellReset(&rc, pPages);
-
    if( pTab!=pOrphan ) recoverFreeTable(pTab);
-
  }
-
  shellFinalize(&rc, pLoop);
-
  shellFinalize(&rc, pPages);
-
  shellFinalize(&rc, pCells);
-
  recoverFreeTable(pOrphan);

-
  /* The rest of the schema */
-
  if( rc==SQLITE_OK ){
-
    sqlite3_stmt *pStmt = 0;
-
    shellPrepare(pState->db, &rc, 
-
        "SELECT sql, name FROM recovery.schema "
-
        "WHERE sql NOT LIKE 'create table%'", &pStmt
-
    );
-
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
-
      const char *zSql = (const char*)sqlite3_column_text(pStmt, 0);
-
      if( sqlite3_strnicmp(zSql, "create virt", 11)==0 ){
-
        const char *zName = (const char*)sqlite3_column_text(pStmt, 1);
-
        char *zPrint = shellMPrintf(&rc, 
-
          "INSERT INTO sqlite_schema VALUES('table', %Q, %Q, 0, %Q)",
-
          zName, zName, zSql
-
        );
-
        raw_printf(pState->out, "%s;\n", zPrint);
-
        sqlite3_free(zPrint);
-
      }else{
-
        raw_printf(pState->out, "%s;\n", zSql);
-
      }
-
    }
-
    shellFinalize(&rc, pStmt);
-
  }
+
  sqlite3_recover_config(p, 789, (void*)zRecoveryDb);  /* Debug use only */
+
  sqlite3_recover_config(p, SQLITE_RECOVER_LOST_AND_FOUND, (void*)zLAF);
+
  sqlite3_recover_config(p, SQLITE_RECOVER_ROWIDS, (void*)&bRowids);
+
  sqlite3_recover_config(p, SQLITE_RECOVER_FREELIST_CORRUPT,(void*)&bFreelist);

-
  if( rc==SQLITE_OK ){
-
    raw_printf(pState->out, "PRAGMA writable_schema = off;\n");
-
    raw_printf(pState->out, "COMMIT;\n");
+
  sqlite3_recover_run(p);
+
  if( sqlite3_recover_errcode(p)!=SQLITE_OK ){
+
    const char *zErr = sqlite3_recover_errmsg(p);
+
    int errCode = sqlite3_recover_errcode(p);
+
    raw_printf(stderr, "sql error: %s (%d)\n", zErr, errCode);
  }
-
  sqlite3_exec(pState->db, "DETACH recovery", 0, 0, 0);
+
  rc = sqlite3_recover_finish(p);
  return rc;
}
-
#endif /* !(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB) */
+
#endif /* SQLITE_SHELL_HAVE_RECOVER */


-
/* 
+
/*
 * zAutoColumn(zCol, &db, ?) => Maybe init db, add column zCol to it.
 * zAutoColumn(0, &db, ?) => (db!=0) Form columns spec for CREATE TABLE,
 *   close db and set it to 0, and return the columns spec, to later
@@ -19001,6 +21847,13 @@ UPDATE ColNames AS t SET reps=\
  static const char * const zColDigits = "\
SELECT CAST(ceil(log(count(*)+0.5)) AS INT) FROM ColNames \
";
+
#else
+
  /* Counting on SQLITE_MAX_COLUMN < 100,000 here. (32767 is the hard limit.) */
+
  static const char * const zColDigits = "\
+
SELECT CASE WHEN (nc < 10) THEN 1 WHEN (nc < 100) THEN 2 \
+
 WHEN (nc < 1000) THEN 3 WHEN (nc < 10000) THEN 4 \
+
 ELSE 5 FROM (SELECT count(*) AS nc FROM ColNames) \
+
";
#endif
  static const char * const zRenameRank =
#ifdef SHELL_COLUMN_RENAME_CLEAN
@@ -19086,11 +21939,7 @@ FROM (\
    /* Formulate the columns spec, close the DB, zero *pDb. */
    char *zColsSpec = 0;
    int hasDupes = db_int(*pDb, zHasDupes);
-
#ifdef SQLITE_ENABLE_MATH_FUNCTIONS
    int nDigits = (hasDupes)? db_int(*pDb, zColDigits) : 0;
-
#else
-
# define nDigits 2
-
#endif
    if( hasDupes ){
#ifdef SHELL_COLUMN_RENAME_CLEAN
      rc = sqlite3_exec(*pDb, zDedoctor, 0, 0, 0);
@@ -19184,7 +22033,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  clearTempFile(p);

#ifndef SQLITE_OMIT_AUTHORIZATION
-
  if( c=='a' && strncmp(azArg[0], "auth", n)==0 ){
+
  if( c=='a' && cli_strncmp(azArg[0], "auth", n)==0 ){
    if( nArg!=2 ){
      raw_printf(stderr, "Usage: .auth ON|OFF\n");
      rc = 1;
@@ -19201,16 +22050,18 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif

-
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
-
  if( c=='a' && strncmp(azArg[0], "archive", n)==0 ){
+
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB) \
+
  && !defined(SQLITE_SHELL_FIDDLE)
+
  if( c=='a' && cli_strncmp(azArg[0], "archive", n)==0 ){
    open_db(p, 0);
    failIfSafeMode(p, "cannot run .archive in safe mode");
    rc = arDotCommand(p, 0, azArg, nArg);
  }else
#endif

-
  if( (c=='b' && n>=3 && strncmp(azArg[0], "backup", n)==0)
-
   || (c=='s' && n>=3 && strncmp(azArg[0], "save", n)==0)
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( (c=='b' && n>=3 && cli_strncmp(azArg[0], "backup", n)==0)
+
   || (c=='s' && n>=3 && cli_strncmp(azArg[0], "save", n)==0)
  ){
    const char *zDestFile = 0;
    const char *zDb = 0;
@@ -19224,10 +22075,10 @@ static int do_meta_command(char *zLine, ShellState *p){
      const char *z = azArg[j];
      if( z[0]=='-' ){
        if( z[1]=='-' ) z++;
-
        if( strcmp(z, "-append")==0 ){
+
        if( cli_strcmp(z, "-append")==0 ){
          zVfs = "apndvfs";
        }else
-
        if( strcmp(z, "-async")==0 ){
+
        if( cli_strcmp(z, "-async")==0 ){
          bAsync = 1;
        }else
        {
@@ -19277,8 +22128,9 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
    close_db(pDest);
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='b' && n>=3 && strncmp(azArg[0], "bail", n)==0 ){
+
  if( c=='b' && n>=3 && cli_strncmp(azArg[0], "bail", n)==0 ){
    if( nArg==2 ){
      bail_on_error = booleanValue(azArg[1]);
    }else{
@@ -19287,7 +22139,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='b' && n>=3 && strncmp(azArg[0], "binary", n)==0 ){
+
  if( c=='b' && n>=3 && cli_strncmp(azArg[0], "binary", n)==0 ){
    if( nArg==2 ){
      if( booleanValue(azArg[1]) ){
        setBinaryMode(p->out, 1);
@@ -19303,11 +22155,12 @@ static int do_meta_command(char *zLine, ShellState *p){
  /* The undocumented ".breakpoint" command causes a call to the no-op
  ** routine named test_breakpoint().
  */
-
  if( c=='b' && n>=3 && strncmp(azArg[0], "breakpoint", n)==0 ){
+
  if( c=='b' && n>=3 && cli_strncmp(azArg[0], "breakpoint", n)==0 ){
    test_breakpoint();
  }else

-
  if( c=='c' && strcmp(azArg[0],"cd")==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='c' && cli_strcmp(azArg[0],"cd")==0 ){
    failIfSafeMode(p, "cannot run .cd in safe mode");
    if( nArg==2 ){
#if defined(_WIN32) || defined(WIN32)
@@ -19326,8 +22179,9 @@ static int do_meta_command(char *zLine, ShellState *p){
      rc = 1;
    }
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='c' && n>=3 && strncmp(azArg[0], "changes", n)==0 ){
+
  if( c=='c' && n>=3 && cli_strncmp(azArg[0], "changes", n)==0 ){
    if( nArg==2 ){
      setOrClearFlag(p, SHFLG_CountChanges, azArg[1]);
    }else{
@@ -19336,11 +22190,12 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

+
#ifndef SQLITE_SHELL_FIDDLE
  /* Cancel output redirection, if it is currently set (by .testcase)
  ** Then read the content of the testcase-out.txt file and compare against
  ** azArg[1].  If there are differences, report an error and exit.
  */
-
  if( c=='c' && n>=3 && strncmp(azArg[0], "check", n)==0 ){
+
  if( c=='c' && n>=3 && cli_strncmp(azArg[0], "check", n)==0 ){
    char *zRes = 0;
    output_reset(p);
    if( nArg!=2 ){
@@ -19360,8 +22215,10 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
    sqlite3_free(zRes);
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='c' && strncmp(azArg[0], "clone", n)==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='c' && cli_strncmp(azArg[0], "clone", n)==0 ){
    failIfSafeMode(p, "cannot run .clone in safe mode");
    if( nArg==2 ){
      tryToClone(p, azArg[1]);
@@ -19370,8 +22227,9 @@ static int do_meta_command(char *zLine, ShellState *p){
      rc = 1;
    }
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='c' && strncmp(azArg[0], "connection", n)==0 ){
+
  if( c=='c' && cli_strncmp(azArg[0], "connection", n)==0 ){
    if( nArg==1 ){
      /* List available connections */
      int i;
@@ -19398,7 +22256,7 @@ static int do_meta_command(char *zLine, ShellState *p){
        globalDb = p->db = p->pAuxDb->db;
        p->pAuxDb->db = 0;
      }
-
    }else if( nArg==3 && strcmp(azArg[1], "close")==0
+
    }else if( nArg==3 && cli_strcmp(azArg[1], "close")==0
           && IsDigit(azArg[2][0]) && azArg[2][1]==0 ){
      int i = azArg[2][0] - '0';
      if( i<0 || i>=ArraySize(p->aAuxDb) ){
@@ -19417,7 +22275,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='d' && n>1 && strncmp(azArg[0], "databases", n)==0 ){
+
  if( c=='d' && n>1 && cli_strncmp(azArg[0], "databases", n)==0 ){
    char **azName = 0;
    int nName = 0;
    sqlite3_stmt *pStmt;
@@ -19456,7 +22314,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    sqlite3_free(azName);
  }else

-
  if( c=='d' && n>=3 && strncmp(azArg[0], "dbconfig", n)==0 ){
+
  if( c=='d' && n>=3 && cli_strncmp(azArg[0], "dbconfig", n)==0 ){
    static const struct DbConfigChoices {
      const char *zName;
      int op;
@@ -19481,7 +22339,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    int ii, v;
    open_db(p, 0);
    for(ii=0; ii<ArraySize(aDbConfig); ii++){
-
      if( nArg>1 && strcmp(azArg[1], aDbConfig[ii].zName)!=0 ) continue;
+
      if( nArg>1 && cli_strcmp(azArg[1], aDbConfig[ii].zName)!=0 ) continue;
      if( nArg>=3 ){
        sqlite3_db_config(p->db, aDbConfig[ii].op, booleanValue(azArg[2]), 0);
      }
@@ -19495,31 +22353,31 @@ static int do_meta_command(char *zLine, ShellState *p){
    }   
  }else

-
  if( c=='d' && n>=3 && strncmp(azArg[0], "dbinfo", n)==0 ){
+
#if SQLITE_SHELL_HAVE_RECOVER
+
  if( c=='d' && n>=3 && cli_strncmp(azArg[0], "dbinfo", n)==0 ){
    rc = shell_dbinfo_command(p, nArg, azArg);
  }else

-
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
-
  if( c=='r' && strncmp(azArg[0], "recover", n)==0 ){
+
  if( c=='r' && cli_strncmp(azArg[0], "recover", n)==0 ){
    open_db(p, 0);
    rc = recoverDatabaseCmd(p, nArg, azArg);
  }else
-
#endif /* !(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB) */
+
#endif /* SQLITE_SHELL_HAVE_RECOVER */

-
  if( c=='d' && strncmp(azArg[0], "dump", n)==0 ){
+
  if( c=='d' && cli_strncmp(azArg[0], "dump", n)==0 ){
    char *zLike = 0;
    char *zSql;
    int i;
    int savedShowHeader = p->showHeader;
    int savedShellFlags = p->shellFlgs;
-
    ShellClearFlag(p, 
+
    ShellClearFlag(p,
       SHFLG_PreserveRowid|SHFLG_Newlines|SHFLG_Echo
       |SHFLG_DumpDataOnly|SHFLG_DumpNoSys);
    for(i=1; i<nArg; i++){
      if( azArg[i][0]=='-' ){
        const char *z = azArg[i]+1;
        if( z[0]=='-' ) z++;
-
        if( strcmp(z,"preserve-rowids")==0 ){
+
        if( cli_strcmp(z,"preserve-rowids")==0 ){
#ifdef SQLITE_OMIT_VIRTUALTABLE
          raw_printf(stderr, "The --preserve-rowids option is not compatible"
                             " with SQLITE_OMIT_VIRTUALTABLE\n");
@@ -19530,13 +22388,13 @@ static int do_meta_command(char *zLine, ShellState *p){
          ShellSetFlag(p, SHFLG_PreserveRowid);
#endif
        }else
-
        if( strcmp(z,"newlines")==0 ){
+
        if( cli_strcmp(z,"newlines")==0 ){
          ShellSetFlag(p, SHFLG_Newlines);
        }else
-
        if( strcmp(z,"data-only")==0 ){
+
        if( cli_strcmp(z,"data-only")==0 ){
          ShellSetFlag(p, SHFLG_DumpDataOnly);
        }else
-
        if( strcmp(z,"nosys")==0 ){
+
        if( cli_strcmp(z,"nosys")==0 ){
          ShellSetFlag(p, SHFLG_DumpNoSys);
        }else
        {
@@ -19618,7 +22476,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    p->shellFlgs = savedShellFlags;
  }else

-
  if( c=='e' && strncmp(azArg[0], "echo", n)==0 ){
+
  if( c=='e' && cli_strncmp(azArg[0], "echo", n)==0 ){
    if( nArg==2 ){
      setOrClearFlag(p, SHFLG_Echo, azArg[1]);
    }else{
@@ -19627,22 +22485,22 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='e' && strncmp(azArg[0], "eqp", n)==0 ){
+
  if( c=='e' && cli_strncmp(azArg[0], "eqp", n)==0 ){
    if( nArg==2 ){
      p->autoEQPtest = 0;
      if( p->autoEQPtrace ){
        if( p->db ) sqlite3_exec(p->db, "PRAGMA vdbe_trace=OFF;", 0, 0, 0);
        p->autoEQPtrace = 0;
      }
-
      if( strcmp(azArg[1],"full")==0 ){
+
      if( cli_strcmp(azArg[1],"full")==0 ){
        p->autoEQP = AUTOEQP_full;
-
      }else if( strcmp(azArg[1],"trigger")==0 ){
+
      }else if( cli_strcmp(azArg[1],"trigger")==0 ){
        p->autoEQP = AUTOEQP_trigger;
#ifdef SQLITE_DEBUG
-
      }else if( strcmp(azArg[1],"test")==0 ){
+
      }else if( cli_strcmp(azArg[1],"test")==0 ){
        p->autoEQP = AUTOEQP_on;
        p->autoEQPtest = 1;
-
      }else if( strcmp(azArg[1],"trace")==0 ){
+
      }else if( cli_strcmp(azArg[1],"trace")==0 ){
        p->autoEQP = AUTOEQP_full;
        p->autoEQPtrace = 1;
        open_db(p, 0);
@@ -19658,17 +22516,19 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='e' && strncmp(azArg[0], "exit", n)==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='e' && cli_strncmp(azArg[0], "exit", n)==0 ){
    if( nArg>1 && (rc = (int)integerValue(azArg[1]))!=0 ) exit(rc);
    rc = 2;
  }else
+
#endif

  /* The ".explain" command is automatic now.  It is largely pointless.  It
  ** retained purely for backwards compatibility */
-
  if( c=='e' && strncmp(azArg[0], "explain", n)==0 ){
+
  if( c=='e' && cli_strncmp(azArg[0], "explain", n)==0 ){
    int val = 1;
    if( nArg>=2 ){
-
      if( strcmp(azArg[1],"auto")==0 ){
+
      if( cli_strcmp(azArg[1],"auto")==0 ){
        val = 99;
      }else{
        val =  booleanValue(azArg[1]);
@@ -19688,7 +22548,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else

#ifndef SQLITE_OMIT_VIRTUALTABLE
-
  if( c=='e' && strncmp(azArg[0], "expert", n)==0 ){
+
  if( c=='e' && cli_strncmp(azArg[0], "expert", n)==0 ){
    if( p->bSafeMode ){
      raw_printf(stderr, 
        "Cannot run experimental commands such as \"%s\" in safe mode\n",
@@ -19701,7 +22561,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif

-
  if( c=='f' && strncmp(azArg[0], "filectrl", n)==0 ){
+
  if( c=='f' && cli_strncmp(azArg[0], "filectrl", n)==0 ){
    static const struct {
       const char *zCtrlName;   /* Name of a test-control option */
       int ctrlCode;            /* Integer code for that option */
@@ -19731,7 +22591,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    zCmd = nArg>=2 ? azArg[1] : "help";

    if( zCmd[0]=='-' 
-
     && (strcmp(zCmd,"--schema")==0 || strcmp(zCmd,"-schema")==0)
+
     && (cli_strcmp(zCmd,"--schema")==0 || cli_strcmp(zCmd,"-schema")==0)
     && nArg>=4
    ){
      zSchema = azArg[2];
@@ -19747,7 +22607,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }

    /* --help lists all file-controls */
-
    if( strcmp(zCmd,"help")==0 ){
+
    if( cli_strcmp(zCmd,"help")==0 ){
      utf8_printf(p->out, "Available file-controls:\n");
      for(i=0; i<ArraySize(aCtrl); i++){
        utf8_printf(p->out, "  .filectrl %s %s\n",
@@ -19761,7 +22621,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** of the option name, or a numerical value. */
    n2 = strlen30(zCmd);
    for(i=0; i<ArraySize(aCtrl); i++){
-
      if( strncmp(zCmd, aCtrl[i].zCtrlName, n2)==0 ){
+
      if( cli_strncmp(zCmd, aCtrl[i].zCtrlName, n2)==0 ){
        if( filectrl<0 ){
          filectrl = aCtrl[i].ctrlCode;
          iCtrl = i;
@@ -19848,7 +22708,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='f' && strncmp(azArg[0], "fullschema", n)==0 ){
+
  if( c=='f' && cli_strncmp(azArg[0], "fullschema", n)==0 ){
    ShellState data;
    int doStats = 0;
    memcpy(&data, p, sizeof(data));
@@ -19895,7 +22755,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='h' && strncmp(azArg[0], "headers", n)==0 ){
+
  if( c=='h' && cli_strncmp(azArg[0], "headers", n)==0 ){
    if( nArg==2 ){
      p->showHeader = booleanValue(azArg[1]);
      p->shellFlgs |= SHFLG_HeaderSet;
@@ -19905,7 +22765,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='h' && strncmp(azArg[0], "help", n)==0 ){
+
  if( c=='h' && cli_strncmp(azArg[0], "help", n)==0 ){
    if( nArg>=2 ){
      n = showHelp(p->out, azArg[1]);
      if( n==0 ){
@@ -19916,7 +22776,8 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='i' && strncmp(azArg[0], "import", n)==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='i' && cli_strncmp(azArg[0], "import", n)==0 ){
    char *zTable = 0;           /* Insert data into this table */
    char *zSchema = 0;          /* within this schema (may default to "main") */
    char *zFile = 0;            /* Name of file to extra content from */
@@ -19937,16 +22798,12 @@ static int do_meta_command(char *zLine, ShellState *p){

    failIfSafeMode(p, "cannot run .import in safe mode");
    memset(&sCtx, 0, sizeof(sCtx));
-
    sCtx.z = sqlite3_malloc64(120);
-
    if( sCtx.z==0 ){
-
      import_cleanup(&sCtx);
-
      shell_out_of_memory();
-
    }
    if( p->mode==MODE_Ascii ){
      xRead = ascii_read_one_field;
    }else{
      xRead = csv_read_one_field;
    }
+
    rc = 1;
    for(i=1; i<nArg; i++){
      char *z = azArg[i];
      if( z[0]=='-' && z[1]=='-' ) z++;
@@ -19958,21 +22815,20 @@ static int do_meta_command(char *zLine, ShellState *p){
        }else{
          utf8_printf(p->out, "ERROR: extra argument: \"%s\".  Usage:\n", z);
          showHelp(p->out, "import");
-
          rc = 1;
          goto meta_command_exit;
        }
-
      }else if( strcmp(z,"-v")==0 ){
+
      }else if( cli_strcmp(z,"-v")==0 ){
        eVerbose++;
-
      }else if( strcmp(z,"-schema")==0 && i<nArg-1 ){
+
      }else if( cli_strcmp(z,"-schema")==0 && i<nArg-1 ){
        zSchema = azArg[++i];
-
      }else if( strcmp(z,"-skip")==0 && i<nArg-1 ){
+
      }else if( cli_strcmp(z,"-skip")==0 && i<nArg-1 ){
        nSkip = integerValue(azArg[++i]);
-
      }else if( strcmp(z,"-ascii")==0 ){
+
      }else if( cli_strcmp(z,"-ascii")==0 ){
        sCtx.cColSep = SEP_Unit[0];
        sCtx.cRowSep = SEP_Record[0];
        xRead = ascii_read_one_field;
        useOutputMode = 0;
-
      }else if( strcmp(z,"-csv")==0 ){
+
      }else if( cli_strcmp(z,"-csv")==0 ){
        sCtx.cColSep = ',';
        sCtx.cRowSep = '\n';
        xRead = csv_read_one_field;
@@ -19980,7 +22836,6 @@ static int do_meta_command(char *zLine, ShellState *p){
      }else{
        utf8_printf(p->out, "ERROR: unknown option: \"%s\".  Usage:\n", z);
        showHelp(p->out, "import");
-
        rc = 1;
        goto meta_command_exit;
      }
    }
@@ -19988,7 +22843,6 @@ static int do_meta_command(char *zLine, ShellState *p){
      utf8_printf(p->out, "ERROR: missing %s argument. Usage:\n",
                  zFile==0 ? "FILE" : "TABLE");
      showHelp(p->out, "import");
-
      rc = 1;
      goto meta_command_exit;
    }
    seenInterrupt = 0;
@@ -20000,24 +22854,23 @@ static int do_meta_command(char *zLine, ShellState *p){
      if( nSep==0 ){
        raw_printf(stderr,
                   "Error: non-null column separator required for import\n");
-
        rc = 1;
        goto meta_command_exit;
      }
      if( nSep>1 ){
-
        raw_printf(stderr, 
+
        raw_printf(stderr,
              "Error: multi-character column separators not allowed"
              " for import\n");
-
        rc = 1;
        goto meta_command_exit;
      }
      nSep = strlen30(p->rowSeparator);
      if( nSep==0 ){
        raw_printf(stderr,
            "Error: non-null row separator required for import\n");
-
        rc = 1;
        goto meta_command_exit;
      }
-
      if( nSep==2 && p->mode==MODE_Csv && strcmp(p->rowSeparator,SEP_CrLf)==0 ){
+
      if( nSep==2 && p->mode==MODE_Csv
+
       && cli_strcmp(p->rowSeparator,SEP_CrLf)==0
+
      ){
        /* When importing CSV (only), if the row separator is set to the
        ** default output row separator, change it to the default input
        ** row separator.  This avoids having to maintain different input
@@ -20028,7 +22881,6 @@ static int do_meta_command(char *zLine, ShellState *p){
      if( nSep>1 ){
        raw_printf(stderr, "Error: multi-character row separators not allowed"
                           " for import\n");
-
        rc = 1;
        goto meta_command_exit;
      }
      sCtx.cColSep = p->colSeparator[0];
@@ -20039,7 +22891,6 @@ static int do_meta_command(char *zLine, ShellState *p){
    if( sCtx.zFile[0]=='|' ){
#ifdef SQLITE_OMIT_POPEN
      raw_printf(stderr, "Error: pipes are not supported in this OS\n");
-
      rc = 1;
      goto meta_command_exit;
#else
      sCtx.in = popen(sCtx.zFile+1, "r");
@@ -20052,8 +22903,6 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
    if( sCtx.in==0 ){
      utf8_printf(stderr, "Error: cannot open \"%s\"\n", zFile);
-
      rc = 1;
-
      import_cleanup(&sCtx);
      goto meta_command_exit;
    }
    if( eVerbose>=2 || (eVerbose>=1 && useOutputMode) ){
@@ -20067,6 +22916,11 @@ static int do_meta_command(char *zLine, ShellState *p){
      output_c_string(p->out, zSep);
      utf8_printf(p->out, "\n");
    }
+
    sCtx.z = sqlite3_malloc64(120);
+
    if( sCtx.z==0 ){
+
      import_cleanup(&sCtx);
+
      shell_out_of_memory();
+
    }
    /* Below, resources must be freed before exit. */
    while( (nSkip--)>0 ){
      while( xRead(&sCtx) && sCtx.cTerm==sCtx.cColSep ){}
@@ -20215,9 +23069,10 @@ static int do_meta_command(char *zLine, ShellState *p){
          sCtx.nRow, sCtx.nErr, sCtx.nLine-1);
    }
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

#ifndef SQLITE_UNTESTABLE
-
  if( c=='i' && strncmp(azArg[0], "imposter", n)==0 ){
+
  if( c=='i' && cli_strncmp(azArg[0], "imposter", n)==0 ){
    char *zSql;
    char *zCollist = 0;
    sqlite3_stmt *pStmt;
@@ -20318,13 +23173,13 @@ static int do_meta_command(char *zLine, ShellState *p){
#endif /* !defined(SQLITE_OMIT_TEST_CONTROL) */

#ifdef SQLITE_ENABLE_IOTRACE
-
  if( c=='i' && strncmp(azArg[0], "iotrace", n)==0 ){
+
  if( c=='i' && cli_strncmp(azArg[0], "iotrace", n)==0 ){
    SQLITE_API extern void (SQLITE_CDECL *sqlite3IoTrace)(const char*, ...);
    if( iotrace && iotrace!=stdout ) fclose(iotrace);
    iotrace = 0;
    if( nArg<2 ){
      sqlite3IoTrace = 0;
-
    }else if( strcmp(azArg[1], "-")==0 ){
+
    }else if( cli_strcmp(azArg[1], "-")==0 ){
      sqlite3IoTrace = iotracePrintf;
      iotrace = stdout;
    }else{
@@ -20340,7 +23195,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif

-
  if( c=='l' && n>=5 && strncmp(azArg[0], "limits", n)==0 ){
+
  if( c=='l' && n>=5 && cli_strncmp(azArg[0], "limits", n)==0 ){
    static const struct {
       const char *zLimitName;   /* Name of a limit */
       int limitCode;            /* Integer code for that limit */
@@ -20399,13 +23254,13 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='l' && n>2 && strncmp(azArg[0], "lint", n)==0 ){
+
  if( c=='l' && n>2 && cli_strncmp(azArg[0], "lint", n)==0 ){
    open_db(p, 0);
    lintDotCommand(p, azArg, nArg);
  }else

-
#ifndef SQLITE_OMIT_LOAD_EXTENSION
-
  if( c=='l' && strncmp(azArg[0], "load", n)==0 ){
+
#if !defined(SQLITE_OMIT_LOAD_EXTENSION) && !defined(SQLITE_SHELL_FIDDLE)
+
  if( c=='l' && cli_strncmp(azArg[0], "load", n)==0 ){
    const char *zFile, *zProc;
    char *zErrMsg = 0;
    failIfSafeMode(p, "cannot run .load in safe mode");
@@ -20426,7 +23281,8 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif

-
  if( c=='l' && strncmp(azArg[0], "log", n)==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='l' && cli_strncmp(azArg[0], "log", n)==0 ){
    failIfSafeMode(p, "cannot run .log in safe mode");
    if( nArg!=2 ){
      raw_printf(stderr, "Usage: .log FILENAME\n");
@@ -20437,8 +23293,9 @@ static int do_meta_command(char *zLine, ShellState *p){
      p->pLog = output_file_open(zFile, 0);
    }
  }else
+
#endif

-
  if( c=='m' && strncmp(azArg[0], "mode", n)==0 ){
+
  if( c=='m' && cli_strncmp(azArg[0], "mode", n)==0 ){
    const char *zMode = 0;
    const char *zTabname = 0;
    int i, n2;
@@ -20457,10 +23314,10 @@ static int do_meta_command(char *zLine, ShellState *p){
        cmOpts.bQuote = 0;
      }else if( zMode==0 ){
        zMode = z;
-
        /* Apply defaults for qbox pseudo-mods. If that
+
        /* Apply defaults for qbox pseudo-mode.  If that
         * overwrites already-set values, user was informed of this.
         */
-
        if( strcmp(z, "qbox")==0 ){
+
        if( cli_strcmp(z, "qbox")==0 ){
          ColModeOpts cmo = ColModeOpts_default_qbox;
          zMode = "box";
          cmOpts = cmo;
@@ -20499,58 +23356,58 @@ static int do_meta_command(char *zLine, ShellState *p){
      zMode = modeDescr[p->mode];
    }
    n2 = strlen30(zMode);
-
    if( strncmp(zMode,"lines",n2)==0 ){
+
    if( cli_strncmp(zMode,"lines",n2)==0 ){
      p->mode = MODE_Line;
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
-
    }else if( strncmp(zMode,"columns",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"columns",n2)==0 ){
      p->mode = MODE_Column;
      if( (p->shellFlgs & SHFLG_HeaderSet)==0 ){
        p->showHeader = 1;
      }
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
      p->cmOpts = cmOpts;
-
    }else if( strncmp(zMode,"list",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"list",n2)==0 ){
      p->mode = MODE_List;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Column);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
-
    }else if( strncmp(zMode,"html",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"html",n2)==0 ){
      p->mode = MODE_Html;
-
    }else if( strncmp(zMode,"tcl",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"tcl",n2)==0 ){
      p->mode = MODE_Tcl;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Space);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
-
    }else if( strncmp(zMode,"csv",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"csv",n2)==0 ){
      p->mode = MODE_Csv;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Comma);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_CrLf);
-
    }else if( strncmp(zMode,"tabs",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"tabs",n2)==0 ){
      p->mode = MODE_List;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Tab);
-
    }else if( strncmp(zMode,"insert",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"insert",n2)==0 ){
      p->mode = MODE_Insert;
      set_table_name(p, zTabname ? zTabname : "table");
-
    }else if( strncmp(zMode,"quote",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"quote",n2)==0 ){
      p->mode = MODE_Quote;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Comma);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Row);
-
    }else if( strncmp(zMode,"ascii",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"ascii",n2)==0 ){
      p->mode = MODE_Ascii;
      sqlite3_snprintf(sizeof(p->colSeparator), p->colSeparator, SEP_Unit);
      sqlite3_snprintf(sizeof(p->rowSeparator), p->rowSeparator, SEP_Record);
-
    }else if( strncmp(zMode,"markdown",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"markdown",n2)==0 ){
      p->mode = MODE_Markdown;
      p->cmOpts = cmOpts;
-
    }else if( strncmp(zMode,"table",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"table",n2)==0 ){
      p->mode = MODE_Table;
      p->cmOpts = cmOpts;
-
    }else if( strncmp(zMode,"box",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"box",n2)==0 ){
      p->mode = MODE_Box;
      p->cmOpts = cmOpts;
-
    }else if( strncmp(zMode,"count",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"count",n2)==0 ){
      p->mode = MODE_Count;
-
    }else if( strncmp(zMode,"off",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"off",n2)==0 ){
      p->mode = MODE_Off;
-
    }else if( strncmp(zMode,"json",n2)==0 ){
+
    }else if( cli_strncmp(zMode,"json",n2)==0 ){
      p->mode = MODE_Json;
    }else{
      raw_printf(stderr, "Error: mode should be one of: "
@@ -20561,11 +23418,12 @@ static int do_meta_command(char *zLine, ShellState *p){
    p->cMode = p->mode;
  }else

-
  if( c=='n' && strcmp(azArg[0], "nonce")==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='n' && cli_strcmp(azArg[0], "nonce")==0 ){
    if( nArg!=2 ){
      raw_printf(stderr, "Usage: .nonce NONCE\n");
      rc = 1;
-
    }else if( p->zNonce==0 || strcmp(azArg[1],p->zNonce)!=0 ){
+
    }else if( p->zNonce==0 || cli_strcmp(azArg[1],p->zNonce)!=0 ){
      raw_printf(stderr, "line %d: incorrect nonce: \"%s\"\n",
                 p->lineno, azArg[1]);
      exit(1);
@@ -20575,8 +23433,9 @@ static int do_meta_command(char *zLine, ShellState *p){
                 ** at the end of this procedure */
    }
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='n' && strncmp(azArg[0], "nullvalue", n)==0 ){
+
  if( c=='n' && cli_strncmp(azArg[0], "nullvalue", n)==0 ){
    if( nArg==2 ){
      sqlite3_snprintf(sizeof(p->nullValue), p->nullValue,
                       "%.*s", (int)ArraySize(p->nullValue)-1, azArg[1]);
@@ -20586,7 +23445,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='o' && strncmp(azArg[0], "open", n)==0 && n>=2 ){
+
  if( c=='o' && cli_strncmp(azArg[0], "open", n)==0 && n>=2 ){
    const char *zFN = 0;     /* Pointer to constant filename */
    char *zNewFilename = 0;  /* Name of the database file to open */
    int iName = 1;           /* Index in azArg[] of the filename */
@@ -20596,6 +23455,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* Check for command-line arguments */
    for(iName=1; iName<nArg; iName++){
      const char *z = azArg[iName];
+
#ifndef SQLITE_SHELL_FIDDLE
      if( optionMatch(z,"new") ){
        newFlag = 1;
#ifdef SQLITE_HAVE_ZLIB
@@ -20616,7 +23476,9 @@ static int do_meta_command(char *zLine, ShellState *p){
      }else if( optionMatch(z, "maxsize") && iName+1<nArg ){
        p->szMax = integerValue(azArg[++iName]);
#endif /* SQLITE_OMIT_DESERIALIZE */
-
      }else if( z[0]=='-' ){
+
      }else
+
#endif /* !SQLITE_SHELL_FIDDLE */
+
      if( z[0]=='-' ){
        utf8_printf(stderr, "unknown option: %s\n", z);
        rc = 1;
        goto meta_command_exit;
@@ -20643,13 +23505,17 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* If a filename is specified, try to open it first */
    if( zFN || p->openMode==SHELL_OPEN_HEXDB ){
      if( newFlag && zFN && !p->bSafeMode ) shellDeleteFile(zFN);
+
#ifndef SQLITE_SHELL_FIDDLE
      if( p->bSafeMode
       && p->openMode!=SHELL_OPEN_HEXDB
       && zFN
-
       && strcmp(zFN,":memory:")!=0
+
       && cli_strcmp(zFN,":memory:")!=0
      ){
        failIfSafeMode(p, "cannot open disk-based database files in safe mode");
      }
+
#else
+
      /* WASM mode has its own sandboxed pseudo-filesystem. */
+
#endif
      if( zFN ){
        zNewFilename = sqlite3_mprintf("%s", zFN);
        shell_check_oom(zNewFilename);
@@ -20672,33 +23538,39 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

+
#ifndef SQLITE_SHELL_FIDDLE
  if( (c=='o'
-
        && (strncmp(azArg[0], "output", n)==0||strncmp(azArg[0], "once", n)==0))
-
   || (c=='e' && n==5 && strcmp(azArg[0],"excel")==0)
+
        && (cli_strncmp(azArg[0], "output", n)==0
+
            || cli_strncmp(azArg[0], "once", n)==0))
+
   || (c=='e' && n==5 && cli_strcmp(azArg[0],"excel")==0)
  ){
    char *zFile = 0;
    int bTxtMode = 0;
    int i;
    int eMode = 0;
-
    int bBOM = 0;
-
    int bOnce = 0;  /* 0: .output, 1: .once, 2: .excel */
+
    int bOnce = 0;            /* 0: .output, 1: .once, 2: .excel */
+
    unsigned char zBOM[4];    /* Byte-order mark to using if --bom is present */

+
    zBOM[0] = 0;
    failIfSafeMode(p, "cannot run .%s in safe mode", azArg[0]);
    if( c=='e' ){
      eMode = 'x';
      bOnce = 2;
-
    }else if( strncmp(azArg[0],"once",n)==0 ){
+
    }else if( cli_strncmp(azArg[0],"once",n)==0 ){
      bOnce = 1;
    }
    for(i=1; i<nArg; i++){
      char *z = azArg[i];
      if( z[0]=='-' ){
        if( z[1]=='-' ) z++;
-
        if( strcmp(z,"-bom")==0 ){
-
          bBOM = 1;
-
        }else if( c!='e' && strcmp(z,"-x")==0 ){
+
        if( cli_strcmp(z,"-bom")==0 ){
+
          zBOM[0] = 0xef;
+
          zBOM[1] = 0xbb;
+
          zBOM[2] = 0xbf;
+
          zBOM[3] = 0;
+
        }else if( c!='e' && cli_strcmp(z,"-x")==0 ){
          eMode = 'x';  /* spreadsheet */
-
        }else if( c!='e' && strcmp(z,"-e")==0 ){
+
        }else if( c!='e' && cli_strcmp(z,"-e")==0 ){
          eMode = 'e';  /* text editor */
        }else{
          utf8_printf(p->out, "ERROR: unknown option: \"%s\".  Usage:\n",
@@ -20764,34 +23636,35 @@ static int do_meta_command(char *zLine, ShellState *p){
        p->out = stdout;
        rc = 1;
      }else{
-
        if( bBOM ) fprintf(p->out,"\357\273\277");
+
        if( zBOM[0] ) fwrite(zBOM, 1, 3, p->out);
        sqlite3_snprintf(sizeof(p->outfile), p->outfile, "%s", zFile);
      }
#endif
    }else{
      p->out = output_file_open(zFile, bTxtMode);
      if( p->out==0 ){
-
        if( strcmp(zFile,"off")!=0 ){
+
        if( cli_strcmp(zFile,"off")!=0 ){
          utf8_printf(stderr,"Error: cannot write to \"%s\"\n", zFile);
        }
        p->out = stdout;
        rc = 1;
      } else {
-
        if( bBOM ) fprintf(p->out,"\357\273\277");
+
        if( zBOM[0] ) fwrite(zBOM, 1, 3, p->out);
        sqlite3_snprintf(sizeof(p->outfile), p->outfile, "%s", zFile);
      }
    }
    sqlite3_free(zFile);
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='p' && n>=3 && strncmp(azArg[0], "parameter", n)==0 ){
+
  if( c=='p' && n>=3 && cli_strncmp(azArg[0], "parameter", n)==0 ){
    open_db(p,0);
    if( nArg<=1 ) goto parameter_syntax_error;

    /* .parameter clear
    ** Clear all bind parameters by dropping the TEMP table that holds them.
    */
-
    if( nArg==2 && strcmp(azArg[1],"clear")==0 ){
+
    if( nArg==2 && cli_strcmp(azArg[1],"clear")==0 ){
      sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp.sqlite_parameters;",
                   0, 0, 0);
    }else
@@ -20799,7 +23672,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* .parameter list
    ** List all bind parameters.
    */
-
    if( nArg==2 && strcmp(azArg[1],"list")==0 ){
+
    if( nArg==2 && cli_strcmp(azArg[1],"list")==0 ){
      sqlite3_stmt *pStmt = 0;
      int rx;
      int len = 0;
@@ -20828,7 +23701,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** Make sure the TEMP table used to hold bind parameters exists.
    ** Create it if necessary.
    */
-
    if( nArg==2 && strcmp(azArg[1],"init")==0 ){
+
    if( nArg==2 && cli_strcmp(azArg[1],"init")==0 ){
      bind_table_init(p);
    }else

@@ -20838,7 +23711,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** VALUE can be in either SQL literal notation, or if not it will be
    ** understood to be a text string.
    */
-
    if( nArg==4 && strcmp(azArg[1],"set")==0 ){
+
    if( nArg==4 && cli_strcmp(azArg[1],"set")==0 ){
      int rx;
      char *zSql;
      sqlite3_stmt *pStmt;
@@ -20876,7 +23749,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** Remove the NAME binding from the parameter binding table, if it
    ** exists.
    */
-
    if( nArg==3 && strcmp(azArg[1],"unset")==0 ){
+
    if( nArg==3 && cli_strcmp(azArg[1],"unset")==0 ){
      char *zSql = sqlite3_mprintf(
          "DELETE FROM temp.sqlite_parameters WHERE key=%Q", azArg[2]);
      shell_check_oom(zSql);
@@ -20888,7 +23761,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    showHelp(p->out, "parameter");
  }else

-
  if( c=='p' && n>=3 && strncmp(azArg[0], "print", n)==0 ){
+
  if( c=='p' && n>=3 && cli_strncmp(azArg[0], "print", n)==0 ){
    int i;
    for(i=1; i<nArg; i++){
      if( i>1 ) raw_printf(p->out, " ");
@@ -20898,7 +23771,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else

#ifndef SQLITE_OMIT_PROGRESS_CALLBACK
-
  if( c=='p' && n>=3 && strncmp(azArg[0], "progress", n)==0 ){
+
  if( c=='p' && n>=3 && cli_strncmp(azArg[0], "progress", n)==0 ){
    int i;
    int nn = 0;
    p->flgProgress = 0;
@@ -20909,19 +23782,19 @@ static int do_meta_command(char *zLine, ShellState *p){
      if( z[0]=='-' ){
        z++;
        if( z[0]=='-' ) z++;
-
        if( strcmp(z,"quiet")==0 || strcmp(z,"q")==0 ){
+
        if( cli_strcmp(z,"quiet")==0 || cli_strcmp(z,"q")==0 ){
          p->flgProgress |= SHELL_PROGRESS_QUIET;
          continue;
        }
-
        if( strcmp(z,"reset")==0 ){
+
        if( cli_strcmp(z,"reset")==0 ){
          p->flgProgress |= SHELL_PROGRESS_RESET;
          continue;
        }
-
        if( strcmp(z,"once")==0 ){
+
        if( cli_strcmp(z,"once")==0 ){
          p->flgProgress |= SHELL_PROGRESS_ONCE;
          continue;
        }
-
        if( strcmp(z,"limit")==0 ){
+
        if( cli_strcmp(z,"limit")==0 ){
          if( i+1>=nArg ){
            utf8_printf(stderr, "Error: missing argument on --limit\n");
            rc = 1;
@@ -20943,7 +23816,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif /* SQLITE_OMIT_PROGRESS_CALLBACK */

-
  if( c=='p' && strncmp(azArg[0], "prompt", n)==0 ){
+
  if( c=='p' && cli_strncmp(azArg[0], "prompt", n)==0 ){
    if( nArg >= 2) {
      strncpy(mainPrompt,azArg[1],(int)ArraySize(mainPrompt)-1);
    }
@@ -20952,11 +23825,14 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='q' && strncmp(azArg[0], "quit", n)==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='q' && cli_strncmp(azArg[0], "quit", n)==0 ){
    rc = 2;
  }else
+
#endif

-
  if( c=='r' && n>=3 && strncmp(azArg[0], "read", n)==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='r' && n>=3 && cli_strncmp(azArg[0], "read", n)==0 ){
    FILE *inSaved = p->in;
    int savedLineno = p->lineno;
    failIfSafeMode(p, "cannot run .read in safe mode");
@@ -20990,8 +23866,10 @@ static int do_meta_command(char *zLine, ShellState *p){
    p->in = inSaved;
    p->lineno = savedLineno;
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='r' && n>=3 && strncmp(azArg[0], "restore", n)==0 ){
+
#ifndef SQLITE_SHELL_FIDDLE
+
  if( c=='r' && n>=3 && cli_strncmp(azArg[0], "restore", n)==0 ){
    const char *zSrcFile;
    const char *zDb;
    sqlite3 *pSrc;
@@ -21042,8 +23920,9 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
    close_db(pSrc);
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='s' && strncmp(azArg[0], "scanstats", n)==0 ){
+
  if( c=='s' && cli_strncmp(azArg[0], "scanstats", n)==0 ){
    if( nArg==2 ){
      p->scanstatsOn = (u8)booleanValue(azArg[1]);
#ifndef SQLITE_ENABLE_STMT_SCANSTATUS
@@ -21055,7 +23934,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='s' && strncmp(azArg[0], "schema", n)==0 ){
+
  if( c=='s' && cli_strncmp(azArg[0], "schema", n)==0 ){
    ShellText sSelect;
    ShellState data;
    char *zErrMsg = 0;
@@ -21198,13 +24077,15 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='s' && n==11 && strncmp(azArg[0], "selecttrace", n)==0 ){
+
  if( (c=='s' && n==11 && cli_strncmp(azArg[0], "selecttrace", n)==0)
+
   || (c=='t' && n==9  && cli_strncmp(azArg[0], "treetrace", n)==0)
+
  ){
    unsigned int x = nArg>=2 ? (unsigned int)integerValue(azArg[1]) : 0xffffffff;
    sqlite3_test_control(SQLITE_TESTCTRL_TRACEFLAGS, 1, &x);
  }else

#if defined(SQLITE_ENABLE_SESSION)
-
  if( c=='s' && strncmp(azArg[0],"session",n)==0 && n>=3 ){
+
  if( c=='s' && cli_strncmp(azArg[0],"session",n)==0 && n>=3 ){
    struct AuxDb *pAuxDb = p->pAuxDb;
    OpenSession *pSession = &pAuxDb->aSession[0];
    char **azCmd = &azArg[1];
@@ -21215,7 +24096,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    open_db(p, 0);
    if( nArg>=3 ){
      for(iSes=0; iSes<pAuxDb->nSession; iSes++){
-
        if( strcmp(pAuxDb->aSession[iSes].zName, azArg[1])==0 ) break;
+
        if( cli_strcmp(pAuxDb->aSession[iSes].zName, azArg[1])==0 ) break;
      }
      if( iSes<pAuxDb->nSession ){
        pSession = &pAuxDb->aSession[iSes];
@@ -21231,7 +24112,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** Invoke the sqlite3session_attach() interface to attach a particular
    ** table so that it is never filtered.
    */
-
    if( strcmp(azCmd[0],"attach")==0 ){
+
    if( cli_strcmp(azCmd[0],"attach")==0 ){
      if( nCmd!=2 ) goto session_syntax_error;
      if( pSession->p==0 ){
        session_not_open:
@@ -21249,7 +24130,9 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** .session patchset FILE
    ** Write a changeset or patchset into a file.  The file is overwritten.
    */
-
    if( strcmp(azCmd[0],"changeset")==0 || strcmp(azCmd[0],"patchset")==0 ){
+
    if( cli_strcmp(azCmd[0],"changeset")==0
+
     || cli_strcmp(azCmd[0],"patchset")==0
+
    ){
      FILE *out = 0;
      failIfSafeMode(p, "cannot run \".session %s\" in safe mode", azCmd[0]);
      if( nCmd!=2 ) goto session_syntax_error;
@@ -21283,7 +24166,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* .session close
    ** Close the identified session
    */
-
    if( strcmp(azCmd[0], "close")==0 ){
+
    if( cli_strcmp(azCmd[0], "close")==0 ){
      if( nCmd!=1 ) goto session_syntax_error;
      if( pAuxDb->nSession ){
        session_close(pSession);
@@ -21294,7 +24177,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* .session enable ?BOOLEAN?
    ** Query or set the enable flag
    */
-
    if( strcmp(azCmd[0], "enable")==0 ){
+
    if( cli_strcmp(azCmd[0], "enable")==0 ){
      int ii;
      if( nCmd>2 ) goto session_syntax_error;
      ii = nCmd==1 ? -1 : booleanValue(azCmd[1]);
@@ -21308,7 +24191,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* .session filter GLOB ....
    ** Set a list of GLOB patterns of table names to be excluded.
    */
-
    if( strcmp(azCmd[0], "filter")==0 ){
+
    if( cli_strcmp(azCmd[0], "filter")==0 ){
      int ii, nByte;
      if( nCmd<2 ) goto session_syntax_error;
      if( pAuxDb->nSession ){
@@ -21333,7 +24216,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* .session indirect ?BOOLEAN?
    ** Query or set the indirect flag
    */
-
    if( strcmp(azCmd[0], "indirect")==0 ){
+
    if( cli_strcmp(azCmd[0], "indirect")==0 ){
      int ii;
      if( nCmd>2 ) goto session_syntax_error;
      ii = nCmd==1 ? -1 : booleanValue(azCmd[1]);
@@ -21347,7 +24230,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* .session isempty
    ** Determine if the session is empty
    */
-
    if( strcmp(azCmd[0], "isempty")==0 ){
+
    if( cli_strcmp(azCmd[0], "isempty")==0 ){
      int ii;
      if( nCmd!=1 ) goto session_syntax_error;
      if( pAuxDb->nSession ){
@@ -21360,7 +24243,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    /* .session list
    ** List all currently open sessions
    */
-
    if( strcmp(azCmd[0],"list")==0 ){
+
    if( cli_strcmp(azCmd[0],"list")==0 ){
      for(i=0; i<pAuxDb->nSession; i++){
        utf8_printf(p->out, "%d %s\n", i, pAuxDb->aSession[i].zName);
      }
@@ -21370,13 +24253,13 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** Open a new session called NAME on the attached database DB.
    ** DB is normally "main".
    */
-
    if( strcmp(azCmd[0],"open")==0 ){
+
    if( cli_strcmp(azCmd[0],"open")==0 ){
      char *zName;
      if( nCmd!=3 ) goto session_syntax_error;
      zName = azCmd[2];
      if( zName[0]==0 ) goto session_syntax_error;
      for(i=0; i<pAuxDb->nSession; i++){
-
        if( strcmp(pAuxDb->aSession[i].zName,zName)==0 ){
+
        if( cli_strcmp(pAuxDb->aSession[i].zName,zName)==0 ){
          utf8_printf(stderr, "Session \"%s\" already exists\n", zName);
          goto meta_command_exit;
        }
@@ -21407,15 +24290,15 @@ static int do_meta_command(char *zLine, ShellState *p){
#ifdef SQLITE_DEBUG
  /* Undocumented commands for internal testing.  Subject to change
  ** without notice. */
-
  if( c=='s' && n>=10 && strncmp(azArg[0], "selftest-", 9)==0 ){
-
    if( strncmp(azArg[0]+9, "boolean", n-9)==0 ){
+
  if( c=='s' && n>=10 && cli_strncmp(azArg[0], "selftest-", 9)==0 ){
+
    if( cli_strncmp(azArg[0]+9, "boolean", n-9)==0 ){
      int i, v;
      for(i=1; i<nArg; i++){
        v = booleanValue(azArg[i]);
        utf8_printf(p->out, "%s: %d 0x%x\n", azArg[i], v, v);
      }
    }
-
    if( strncmp(azArg[0]+9, "integer", n-9)==0 ){
+
    if( cli_strncmp(azArg[0]+9, "integer", n-9)==0 ){
      int i; sqlite3_int64 v;
      for(i=1; i<nArg; i++){
        char zBuf[200];
@@ -21427,7 +24310,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif

-
  if( c=='s' && n>=4 && strncmp(azArg[0],"selftest",n)==0 ){
+
  if( c=='s' && n>=4 && cli_strncmp(azArg[0],"selftest",n)==0 ){
    int bIsInit = 0;         /* True to initialize the SELFTEST table */
    int bVerbose = 0;        /* Verbose output */
    int bSelftestExists;     /* True if SELFTEST already exists */
@@ -21441,10 +24324,10 @@ static int do_meta_command(char *zLine, ShellState *p){
    for(i=1; i<nArg; i++){
      const char *z = azArg[i];
      if( z[0]=='-' && z[1]=='-' ) z++;
-
      if( strcmp(z,"-init")==0 ){
+
      if( cli_strcmp(z,"-init")==0 ){
        bIsInit = 1;
      }else
-
      if( strcmp(z,"-v")==0 ){
+
      if( cli_strcmp(z,"-v")==0 ){
        bVerbose++;
      }else
      {
@@ -21497,10 +24380,10 @@ static int do_meta_command(char *zLine, ShellState *p){
        if( bVerbose>0 ){
          printf("%d: %s %s\n", tno, zOp, zSql);
        }
-
        if( strcmp(zOp,"memo")==0 ){
+
        if( cli_strcmp(zOp,"memo")==0 ){
          utf8_printf(p->out, "%s\n", zSql);
        }else
-
        if( strcmp(zOp,"run")==0 ){
+
        if( cli_strcmp(zOp,"run")==0 ){
          char *zErrMsg = 0;
          str.n = 0;
          str.z[0] = 0;
@@ -21514,7 +24397,7 @@ static int do_meta_command(char *zLine, ShellState *p){
            rc = 1;
            utf8_printf(p->out, "%d: error-code-%d: %s\n", tno, rc, zErrMsg);
            sqlite3_free(zErrMsg);
-
          }else if( strcmp(zAns,str.z)!=0 ){
+
          }else if( cli_strcmp(zAns,str.z)!=0 ){
            nErr++;
            rc = 1;
            utf8_printf(p->out, "%d: Expected: [%s]\n", tno, zAns);
@@ -21534,7 +24417,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    utf8_printf(p->out, "%d errors out of %d tests\n", nErr, nTest);
  }else

-
  if( c=='s' && strncmp(azArg[0], "separator", n)==0 ){
+
  if( c=='s' && cli_strncmp(azArg[0], "separator", n)==0 ){
    if( nArg<2 || nArg>3 ){
      raw_printf(stderr, "Usage: .separator COL ?ROW?\n");
      rc = 1;
@@ -21549,7 +24432,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='s' && n>=4 && strncmp(azArg[0],"sha3sum",n)==0 ){
+
  if( c=='s' && n>=4 && cli_strncmp(azArg[0],"sha3sum",n)==0 ){
    const char *zLike = 0;   /* Which table to checksum. 0 means everything */
    int i;                   /* Loop counter */
    int bSchema = 0;         /* Also hash the schema */
@@ -21567,15 +24450,15 @@ static int do_meta_command(char *zLine, ShellState *p){
      if( z[0]=='-' ){
        z++;
        if( z[0]=='-' ) z++;
-
        if( strcmp(z,"schema")==0 ){
+
        if( cli_strcmp(z,"schema")==0 ){
          bSchema = 1;
        }else
-
        if( strcmp(z,"sha3-224")==0 || strcmp(z,"sha3-256")==0
-
         || strcmp(z,"sha3-384")==0 || strcmp(z,"sha3-512")==0
+
        if( cli_strcmp(z,"sha3-224")==0 || cli_strcmp(z,"sha3-256")==0
+
         || cli_strcmp(z,"sha3-384")==0 || cli_strcmp(z,"sha3-512")==0
        ){
          iSize = atoi(&z[5]);
        }else
-
        if( strcmp(z,"debug")==0 ){
+
        if( cli_strcmp(z,"debug")==0 ){
          bDebug = 1;
        }else
        {
@@ -21615,20 +24498,20 @@ static int do_meta_command(char *zLine, ShellState *p){
      const char *zTab = (const char*)sqlite3_column_text(pStmt,0);
      if( zTab==0 ) continue;
      if( zLike && sqlite3_strlike(zLike, zTab, 0)!=0 ) continue;
-
      if( strncmp(zTab, "sqlite_",7)!=0 ){
+
      if( cli_strncmp(zTab, "sqlite_",7)!=0 ){
        appendText(&sQuery,"SELECT * FROM ", 0);
        appendText(&sQuery,zTab,'"');
        appendText(&sQuery," NOT INDEXED;", 0);
-
      }else if( strcmp(zTab, "sqlite_schema")==0 ){
+
      }else if( cli_strcmp(zTab, "sqlite_schema")==0 ){
        appendText(&sQuery,"SELECT type,name,tbl_name,sql FROM sqlite_schema"
                           " ORDER BY name;", 0);
-
      }else if( strcmp(zTab, "sqlite_sequence")==0 ){
+
      }else if( cli_strcmp(zTab, "sqlite_sequence")==0 ){
        appendText(&sQuery,"SELECT name,seq FROM sqlite_sequence"
                           " ORDER BY name;", 0);
-
      }else if( strcmp(zTab, "sqlite_stat1")==0 ){
+
      }else if( cli_strcmp(zTab, "sqlite_stat1")==0 ){
        appendText(&sQuery,"SELECT tbl,idx,stat FROM sqlite_stat1"
                           " ORDER BY tbl,idx;", 0);
-
      }else if( strcmp(zTab, "sqlite_stat4")==0 ){
+
      }else if( cli_strcmp(zTab, "sqlite_stat4")==0 ){
        appendText(&sQuery, "SELECT * FROM ", 0);
        appendText(&sQuery, zTab, 0);
        appendText(&sQuery, " ORDER BY tbl, idx, rowid;\n", 0);
@@ -21665,9 +24548,10 @@ static int do_meta_command(char *zLine, ShellState *p){
    sqlite3_free(zSql);
  }else

-
#ifndef SQLITE_NOHAVE_SYSTEM
+
#if !defined(SQLITE_NOHAVE_SYSTEM) && !defined(SQLITE_SHELL_FIDDLE)
  if( c=='s'
-
   && (strncmp(azArg[0], "shell", n)==0 || strncmp(azArg[0],"system",n)==0)
+
   && (cli_strncmp(azArg[0], "shell", n)==0
+
       || cli_strncmp(azArg[0],"system",n)==0)
  ){
    char *zCmd;
    int i, x;
@@ -21686,9 +24570,9 @@ static int do_meta_command(char *zLine, ShellState *p){
    sqlite3_free(zCmd);
    if( x ) raw_printf(stderr, "System command returns %d\n", x);
  }else
-
#endif /* !defined(SQLITE_NOHAVE_SYSTEM) */
+
#endif /* !defined(SQLITE_NOHAVE_SYSTEM) && !defined(SQLITE_SHELL_FIDDLE) */

-
  if( c=='s' && strncmp(azArg[0], "show", n)==0 ){
+
  if( c=='s' && cli_strncmp(azArg[0], "show", n)==0 ){
    static const char *azBool[] = { "off", "on", "trigger", "full"};
    const char *zOut;
    int i;
@@ -21698,7 +24582,7 @@ static int do_meta_command(char *zLine, ShellState *p){
      goto meta_command_exit;
    }
    utf8_printf(p->out, "%12.12s: %s\n","echo",
-
                                  azBool[ShellHasFlag(p, SHFLG_Echo)]);
+
                azBool[ShellHasFlag(p, SHFLG_Echo)]);
    utf8_printf(p->out, "%12.12s: %s\n","eqp", azBool[p->autoEQP&3]);
    utf8_printf(p->out, "%12.12s: %s\n","explain",
         p->mode==MODE_Explain ? "on" : p->autoExplain ? "auto" : "off");
@@ -21741,11 +24625,11 @@ static int do_meta_command(char *zLine, ShellState *p){
                p->pAuxDb->zDbFilename ? p->pAuxDb->zDbFilename : "");
  }else

-
  if( c=='s' && strncmp(azArg[0], "stats", n)==0 ){
+
  if( c=='s' && cli_strncmp(azArg[0], "stats", n)==0 ){
    if( nArg==2 ){
-
      if( strcmp(azArg[1],"stmt")==0 ){
+
      if( cli_strcmp(azArg[1],"stmt")==0 ){
        p->statsOn = 2;
-
      }else if( strcmp(azArg[1],"vmstep")==0 ){
+
      }else if( cli_strcmp(azArg[1],"vmstep")==0 ){
        p->statsOn = 3;
      }else{
        p->statsOn = (u8)booleanValue(azArg[1]);
@@ -21758,9 +24642,9 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( (c=='t' && n>1 && strncmp(azArg[0], "tables", n)==0)
-
   || (c=='i' && (strncmp(azArg[0], "indices", n)==0
-
                 || strncmp(azArg[0], "indexes", n)==0) )
+
  if( (c=='t' && n>1 && cli_strncmp(azArg[0], "tables", n)==0)
+
   || (c=='i' && (cli_strncmp(azArg[0], "indices", n)==0
+
                 || cli_strncmp(azArg[0], "indexes", n)==0) )
  ){
    sqlite3_stmt *pStmt;
    char **azResult;
@@ -21866,8 +24750,9 @@ static int do_meta_command(char *zLine, ShellState *p){
    sqlite3_free(azResult);
  }else

+
#ifndef SQLITE_SHELL_FIDDLE
  /* Begin redirecting output to the file "testcase-out.txt" */
-
  if( c=='t' && strcmp(azArg[0],"testcase")==0 ){
+
  if( c=='t' && cli_strcmp(azArg[0],"testcase")==0 ){
    output_reset(p);
    p->out = output_file_open("testcase-out.txt", 0);
    if( p->out==0 ){
@@ -21879,9 +24764,10 @@ static int do_meta_command(char *zLine, ShellState *p){
      sqlite3_snprintf(sizeof(p->zTestcase), p->zTestcase, "?");
    }
  }else
+
#endif /* !defined(SQLITE_SHELL_FIDDLE) */

#ifndef SQLITE_UNTESTABLE
-
  if( c=='t' && n>=8 && strncmp(azArg[0], "testctrl", n)==0 ){
+
  if( c=='t' && n>=8 && cli_strncmp(azArg[0], "testctrl", n)==0 ){
    static const struct {
       const char *zCtrlName;   /* Name of a test-control option */
       int ctrlCode;            /* Integer code for that option */
@@ -21928,7 +24814,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }

    /* --help lists all test-controls */
-
    if( strcmp(zCmd,"help")==0 ){
+
    if( cli_strcmp(zCmd,"help")==0 ){
      utf8_printf(p->out, "Available test-controls:\n");
      for(i=0; i<ArraySize(aCtrl); i++){
        utf8_printf(p->out, "  .testctrl %s %s\n",
@@ -21942,7 +24828,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    ** of the option name, or a numerical value. */
    n2 = strlen30(zCmd);
    for(i=0; i<ArraySize(aCtrl); i++){
-
      if( strncmp(zCmd, aCtrl[i].zCtrlName, n2)==0 ){
+
      if( cli_strncmp(zCmd, aCtrl[i].zCtrlName, n2)==0 ){
        if( testctrl<0 ){
          testctrl = aCtrl[i].ctrlCode;
          iCtrl = i;
@@ -21998,7 +24884,7 @@ static int do_meta_command(char *zLine, ShellState *p){
          if( nArg==3 || nArg==4 ){
            int ii = (int)integerValue(azArg[2]);
            sqlite3 *db;
-
            if( ii==0 && strcmp(azArg[2],"random")==0 ){
+
            if( ii==0 && cli_strcmp(azArg[2],"random")==0 ){
              sqlite3_randomness(sizeof(ii),&ii);
              printf("-- random seed: %d\n", ii);
            }
@@ -22114,12 +25000,12 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif /* !defined(SQLITE_UNTESTABLE) */

-
  if( c=='t' && n>4 && strncmp(azArg[0], "timeout", n)==0 ){
+
  if( c=='t' && n>4 && cli_strncmp(azArg[0], "timeout", n)==0 ){
    open_db(p, 0);
    sqlite3_busy_timeout(p->db, nArg>=2 ? (int)integerValue(azArg[1]) : 0);
  }else

-
  if( c=='t' && n>=5 && strncmp(azArg[0], "timer", n)==0 ){
+
  if( c=='t' && n>=5 && cli_strncmp(azArg[0], "timer", n)==0 ){
    if( nArg==2 ){
      enableTimer = booleanValue(azArg[1]);
      if( enableTimer && !HAS_TIMER ){
@@ -22133,7 +25019,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else

#ifndef SQLITE_OMIT_TRACE
-
  if( c=='t' && strncmp(azArg[0], "trace", n)==0 ){
+
  if( c=='t' && cli_strncmp(azArg[0], "trace", n)==0 ){
    int mType = 0;
    int jj;
    open_db(p, 0);
@@ -22183,7 +25069,7 @@ static int do_meta_command(char *zLine, ShellState *p){
#endif /* !defined(SQLITE_OMIT_TRACE) */

#if defined(SQLITE_DEBUG) && !defined(SQLITE_OMIT_VIRTUALTABLE)
-
  if( c=='u' && strncmp(azArg[0], "unmodule", n)==0 ){
+
  if( c=='u' && cli_strncmp(azArg[0], "unmodule", n)==0 ){
    int ii;
    int lenOpt;
    char *zOpt;
@@ -22196,7 +25082,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    zOpt = azArg[1];
    if( zOpt[0]=='-' && zOpt[1]=='-' && zOpt[2]!=0 ) zOpt++;
    lenOpt = (int)strlen(zOpt);
-
    if( lenOpt>=3 && strncmp(zOpt, "-allexcept",lenOpt)==0 ){
+
    if( lenOpt>=3 && cli_strncmp(zOpt, "-allexcept",lenOpt)==0 ){
      assert( azArg[nArg]==0 );
      sqlite3_drop_modules(p->db, nArg>2 ? (const char**)(azArg+2) : 0);
    }else{
@@ -22208,14 +25094,14 @@ static int do_meta_command(char *zLine, ShellState *p){
#endif

#if SQLITE_USER_AUTHENTICATION
-
  if( c=='u' && strncmp(azArg[0], "user", n)==0 ){
+
  if( c=='u' && cli_strncmp(azArg[0], "user", n)==0 ){
    if( nArg<2 ){
      raw_printf(stderr, "Usage: .user SUBCOMMAND ...\n");
      rc = 1;
      goto meta_command_exit;
    }
    open_db(p, 0);
-
    if( strcmp(azArg[1],"login")==0 ){
+
    if( cli_strcmp(azArg[1],"login")==0 ){
      if( nArg!=4 ){
        raw_printf(stderr, "Usage: .user login USER PASSWORD\n");
        rc = 1;
@@ -22227,7 +25113,7 @@ static int do_meta_command(char *zLine, ShellState *p){
        utf8_printf(stderr, "Authentication failed for user %s\n", azArg[2]);
        rc = 1;
      }
-
    }else if( strcmp(azArg[1],"add")==0 ){
+
    }else if( cli_strcmp(azArg[1],"add")==0 ){
      if( nArg!=5 ){
        raw_printf(stderr, "Usage: .user add USER PASSWORD ISADMIN\n");
        rc = 1;
@@ -22239,7 +25125,7 @@ static int do_meta_command(char *zLine, ShellState *p){
        raw_printf(stderr, "User-Add failed: %d\n", rc);
        rc = 1;
      }
-
    }else if( strcmp(azArg[1],"edit")==0 ){
+
    }else if( cli_strcmp(azArg[1],"edit")==0 ){
      if( nArg!=5 ){
        raw_printf(stderr, "Usage: .user edit USER PASSWORD ISADMIN\n");
        rc = 1;
@@ -22251,7 +25137,7 @@ static int do_meta_command(char *zLine, ShellState *p){
        raw_printf(stderr, "User-Edit failed: %d\n", rc);
        rc = 1;
      }
-
    }else if( strcmp(azArg[1],"delete")==0 ){
+
    }else if( cli_strcmp(azArg[1],"delete")==0 ){
      if( nArg!=3 ){
        raw_printf(stderr, "Usage: .user delete USER\n");
        rc = 1;
@@ -22270,7 +25156,7 @@ static int do_meta_command(char *zLine, ShellState *p){
  }else
#endif /* SQLITE_USER_AUTHENTICATION */

-
  if( c=='v' && strncmp(azArg[0], "version", n)==0 ){
+
  if( c=='v' && cli_strncmp(azArg[0], "version", n)==0 ){
    utf8_printf(p->out, "SQLite %s %s\n" /*extra-version-info*/,
        sqlite3_libversion(), sqlite3_sourceid());
#if SQLITE_HAVE_ZLIB
@@ -22289,7 +25175,7 @@ static int do_meta_command(char *zLine, ShellState *p){
#endif
  }else

-
  if( c=='v' && strncmp(azArg[0], "vfsinfo", n)==0 ){
+
  if( c=='v' && cli_strncmp(azArg[0], "vfsinfo", n)==0 ){
    const char *zDbName = nArg==2 ? azArg[1] : "main";
    sqlite3_vfs *pVfs = 0;
    if( p->db ){
@@ -22303,7 +25189,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='v' && strncmp(azArg[0], "vfslist", n)==0 ){
+
  if( c=='v' && cli_strncmp(azArg[0], "vfslist", n)==0 ){
    sqlite3_vfs *pVfs;
    sqlite3_vfs *pCurrent = 0;
    if( p->db ){
@@ -22321,7 +25207,7 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='v' && strncmp(azArg[0], "vfsname", n)==0 ){
+
  if( c=='v' && cli_strncmp(azArg[0], "vfsname", n)==0 ){
    const char *zDbName = nArg==2 ? azArg[1] : "main";
    char *zVfsName = 0;
    if( p->db ){
@@ -22333,12 +25219,12 @@ static int do_meta_command(char *zLine, ShellState *p){
    }
  }else

-
  if( c=='w' && strncmp(azArg[0], "wheretrace", n)==0 ){
+
  if( c=='w' && cli_strncmp(azArg[0], "wheretrace", n)==0 ){
    unsigned int x = nArg>=2 ? (unsigned int)integerValue(azArg[1]) : 0xffffffff;
    sqlite3_test_control(SQLITE_TESTCTRL_TRACEFLAGS, 3, &x);
  }else

-
  if( c=='w' && strncmp(azArg[0], "width", n)==0 ){
+
  if( c=='w' && cli_strncmp(azArg[0], "width", n)==0 ){
    int j;
    assert( nArg<=ArraySize(azArg) );
    p->nWidth = nArg-1;
@@ -22448,7 +25334,7 @@ static QuickScanState quickscan(char *zLine, QuickScanState qss){
          cWait = 0;
          qss = QSS_SETV(qss, 0);
          goto PlainScan;
-
        default: assert(0); 
+
        default: assert(0);
        }
      }
    }
@@ -22469,7 +25355,7 @@ static int line_is_command_terminator(char *zLine){
    zLine += 2; /* SQL Server */
  else
    return 0;
-
  return quickscan(zLine,QSS_Start)==QSS_Start;
+
  return quickscan(zLine, QSS_Start)==QSS_Start;
}

/*
@@ -22516,10 +25402,10 @@ static int runOneSqlLine(ShellState *p, char *zSql, FILE *in, int startline){
    if( zErrMsg==0 ){
      zErrorType = "Error";
      zErrorTail = sqlite3_errmsg(p->db);
-
    }else if( strncmp(zErrMsg, "in prepare, ",12)==0 ){
+
    }else if( cli_strncmp(zErrMsg, "in prepare, ",12)==0 ){
      zErrorType = "Parse error";
      zErrorTail = &zErrMsg[12];
-
    }else if( strncmp(zErrMsg, "stepping, ", 10)==0 ){
+
    }else if( cli_strncmp(zErrMsg, "stepping, ", 10)==0 ){
      zErrorType = "Runtime error";
      zErrorTail = &zErrMsg[10];
    }else{
@@ -22546,6 +25432,42 @@ static int runOneSqlLine(ShellState *p, char *zSql, FILE *in, int startline){
  return 0;
}

+
static void echo_group_input(ShellState *p, const char *zDo){
+
  if( ShellHasFlag(p, SHFLG_Echo) ) utf8_printf(p->out, "%s\n", zDo);
+
}
+

+
#ifdef SQLITE_SHELL_FIDDLE
+
/*
+
** Alternate one_input_line() impl for wasm mode. This is not in the primary impl
+
** because we need the global shellState and cannot access it from that function
+
** without moving lots of code around (creating a larger/messier diff).
+
*/
+
static char *one_input_line(FILE *in, char *zPrior, int isContinuation){
+
  /* Parse the next line from shellState.wasm.zInput. */
+
  const char *zBegin = shellState.wasm.zPos;
+
  const char *z = zBegin;
+
  char *zLine = 0;
+
  i64 nZ = 0;
+

+
  UNUSED_PARAMETER(in);
+
  UNUSED_PARAMETER(isContinuation);
+
  if(!z || !*z){
+
    return 0;
+
  }
+
  while(*z && isspace(*z)) ++z;
+
  zBegin = z;
+
  for(; *z && '\n'!=*z; ++nZ, ++z){}
+
  if(nZ>0 && '\r'==zBegin[nZ-1]){
+
    --nZ;
+
  }
+
  shellState.wasm.zPos = z;
+
  zLine = realloc(zPrior, nZ+1);
+
  shell_check_oom(zLine);
+
  memcpy(zLine, zBegin, nZ);
+
  zLine[nZ] = 0;
+
  return zLine;
+
}
+
#endif /* SQLITE_SHELL_FIDDLE */

/*
** Read input from *in and process it.  If *in==0 then input
@@ -22559,12 +25481,12 @@ static int runOneSqlLine(ShellState *p, char *zSql, FILE *in, int startline){
static int process_input(ShellState *p){
  char *zLine = 0;          /* A single input line */
  char *zSql = 0;           /* Accumulated SQL text */
-
  int nLine;                /* Length of current line */
-
  int nSql = 0;             /* Bytes of zSql[] used */
-
  int nAlloc = 0;           /* Allocated zSql[] space */
+
  i64 nLine;                /* Length of current line */
+
  i64 nSql = 0;             /* Bytes of zSql[] used */
+
  i64 nAlloc = 0;           /* Allocated zSql[] space */
  int rc;                   /* Error code */
  int errCnt = 0;           /* Number of errors seen */
-
  int startline = 0;        /* Line number for start of current input */
+
  i64 startline = 0;        /* Line number for start of current input */
  QuickScanState qss = QSS_Start; /* Accumulated line status (so far) */

  if( p->inputNesting==MAX_INPUT_NESTING ){
@@ -22595,14 +25517,13 @@ static int process_input(ShellState *p){
    }
    qss = quickscan(zLine, qss);
    if( QSS_PLAINWHITE(qss) && nSql==0 ){
-
      if( ShellHasFlag(p, SHFLG_Echo) )
-
        printf("%s\n", zLine);
      /* Just swallow single-line whitespace */
+
      echo_group_input(p, zLine);
      qss = QSS_Start;
      continue;
    }
    if( zLine && (zLine[0]=='.' || zLine[0]=='#') && nSql==0 ){
-
      if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zLine);
+
      echo_group_input(p, zLine);
      if( zLine[0]=='.' ){
        rc = do_meta_command(zLine, p);
        if( rc==2 ){ /* exit requested */
@@ -22615,7 +25536,7 @@ static int process_input(ShellState *p){
      continue;
    }
    /* No single-line dispositions remain; accumulate line(s). */
-
    nLine = strlen30(zLine);
+
    nLine = strlen(zLine);
    if( nSql+nLine+2>=nAlloc ){
      /* Grow buffer by half-again increments when big. */
      nAlloc = nSql+(nSql>>1)+nLine+100;
@@ -22623,7 +25544,7 @@ static int process_input(ShellState *p){
      shell_check_oom(zSql);
    }
    if( nSql==0 ){
-
      int i;
+
      i64 i;
      for(i=0; zLine[i] && IsSpace(zLine[i]); i++){}
      assert( nAlloc>0 && zSql!=0 );
      memcpy(zSql, zLine+i, nLine+1-i);
@@ -22635,6 +25556,7 @@ static int process_input(ShellState *p){
      nSql += nLine;
    }
    if( nSql && QSS_SEMITERM(qss) && sqlite3_complete(zSql) ){
+
      echo_group_input(p, zSql);
      errCnt += runOneSqlLine(p, zSql, p->in, startline);
      nSql = 0;
      if( p->outCount ){
@@ -22646,13 +25568,14 @@ static int process_input(ShellState *p){
      p->bSafeMode = p->bSafeModePersist;
      qss = QSS_Start;
    }else if( nSql && QSS_PLAINWHITE(qss) ){
-
      if( ShellHasFlag(p, SHFLG_Echo) ) printf("%s\n", zSql);
+
      echo_group_input(p, zSql);
      nSql = 0;
      qss = QSS_Start;
    }
  }
  if( nSql ){
    /* This may be incomplete. Let the SQL parser deal with that. */
+
    echo_group_input(p, zSql);
    errCnt += runOneSqlLine(p, zSql, p->in, startline);
  }
  free(zSql);
@@ -22721,7 +25644,7 @@ static char *find_home_dir(int clearFlag){
#endif /* !_WIN32_WCE */

  if( home_dir ){
-
    int n = strlen30(home_dir) + 1;
+
    i64 n = strlen(home_dir) + 1;
    char *z = malloc( n );
    if( z ) memcpy(z, home_dir, n);
    home_dir = z;
@@ -22791,7 +25714,7 @@ static const char zOptions[] =
#if !defined(SQLITE_OMIT_DESERIALIZE)
  "   -deserialize         open the database using sqlite3_deserialize()\n"
#endif
-
  "   -echo                print commands before execution\n"
+
  "   -echo                print inputs before execution\n"
  "   -init FILENAME       read/process named file\n"
  "   -[no]header          turn headers on or off\n"
#if defined(SQLITE_ENABLE_MEMSYS3) || defined(SQLITE_ENABLE_MEMSYS5)
@@ -22927,14 +25850,25 @@ static char *cmdline_option_value(int argc, char **argv, int i){
#  endif
#endif

+
#ifdef SQLITE_SHELL_FIDDLE
+
#  define main fiddle_main
+
#endif
+

#if SQLITE_SHELL_IS_UTF8
int SQLITE_CDECL main(int argc, char **argv){
#else
int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
  char **argv;
#endif
+
#ifdef SQLITE_DEBUG
+
  sqlite3_int64 mem_main_enter = sqlite3_memory_used();
+
#endif
  char *zErrMsg = 0;
+
#ifdef SQLITE_SHELL_FIDDLE
+
#  define data shellState
+
#else
  ShellState data;
+
#endif
  const char *zInitFile = 0;
  int i;
  int rc = 0;
@@ -22950,8 +25884,14 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){

  setBinaryMode(stdin, 0);
  setvbuf(stderr, 0, _IONBF, 0); /* Make sure stderr is unbuffered */
+
#ifdef SQLITE_SHELL_FIDDLE
+
  stdin_is_interactive = 0;
+
  stdout_is_console = 1;
+
  data.wasm.zDefaultDbName = "/fiddle.sqlite3";
+
#else
  stdin_is_interactive = isatty(0);
  stdout_is_console = isatty(1);
+
#endif

#if !defined(_WIN32_WCE)
  if( getenv("SQLITE_DEBUG_BREAK") ){
@@ -22975,7 +25915,7 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
#endif

#if USE_SYSTEM_SQLITE+0!=1
-
  if( strncmp(sqlite3_sourceid(),SQLITE_SOURCE_ID,60)!=0 ){
+
  if( cli_strncmp(sqlite3_sourceid(),SQLITE_SOURCE_ID,60)!=0 ){
    utf8_printf(stderr, "SQLite header and source version mismatch\n%s\n%s\n",
            sqlite3_sourceid(), SQLITE_SOURCE_ID);
    exit(1);
@@ -22997,9 +25937,9 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
  argv = argvToFree + argc;
  for(i=0; i<argc; i++){
    char *z = sqlite3_win32_unicode_to_utf8(wargv[i]);
-
    int n;
+
    i64 n;
    shell_check_oom(z);
-
    n = (int)strlen(z);
+
    n = strlen(z);
    argv[i] = malloc( n+1 );
    shell_check_oom(argv[i]);
    memcpy(argv[i], z, n+1);
@@ -23056,21 +25996,21 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
      }
    }
    if( z[1]=='-' ) z++;
-
    if( strcmp(z,"-separator")==0
-
     || strcmp(z,"-nullvalue")==0
-
     || strcmp(z,"-newline")==0
-
     || strcmp(z,"-cmd")==0
+
    if( cli_strcmp(z,"-separator")==0
+
     || cli_strcmp(z,"-nullvalue")==0
+
     || cli_strcmp(z,"-newline")==0
+
     || cli_strcmp(z,"-cmd")==0
    ){
      (void)cmdline_option_value(argc, argv, ++i);
-
    }else if( strcmp(z,"-init")==0 ){
+
    }else if( cli_strcmp(z,"-init")==0 ){
      zInitFile = cmdline_option_value(argc, argv, ++i);
-
    }else if( strcmp(z,"-batch")==0 ){
+
    }else if( cli_strcmp(z,"-batch")==0 ){
      /* Need to check for batch mode here to so we can avoid printing
      ** informational messages (like from process_sqliterc) before
      ** we do the actual processing of arguments later in a second pass.
      */
      stdin_is_interactive = 0;
-
    }else if( strcmp(z,"-heap")==0 ){
+
    }else if( cli_strcmp(z,"-heap")==0 ){
#if defined(SQLITE_ENABLE_MEMSYS3) || defined(SQLITE_ENABLE_MEMSYS5)
      const char *zSize;
      sqlite3_int64 szHeap;
@@ -23082,7 +26022,7 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
#else
      (void)cmdline_option_value(argc, argv, ++i);
#endif
-
    }else if( strcmp(z,"-pagecache")==0 ){
+
    }else if( cli_strcmp(z,"-pagecache")==0 ){
      sqlite3_int64 n, sz;
      sz = integerValue(cmdline_option_value(argc,argv,++i));
      if( sz>70000 ) sz = 70000;
@@ -23094,7 +26034,7 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
      sqlite3_config(SQLITE_CONFIG_PAGECACHE,
                    (n>0 && sz>0) ? malloc(n*sz) : 0, sz, n);
      data.shellFlgs |= SHFLG_Pagecache;
-
    }else if( strcmp(z,"-lookaside")==0 ){
+
    }else if( cli_strcmp(z,"-lookaside")==0 ){
      int n, sz;
      sz = (int)integerValue(cmdline_option_value(argc,argv,++i));
      if( sz<0 ) sz = 0;
@@ -23102,7 +26042,7 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
      if( n<0 ) n = 0;
      sqlite3_config(SQLITE_CONFIG_LOOKASIDE, sz, n);
      if( sz*n==0 ) data.shellFlgs &= ~SHFLG_Lookaside;
-
    }else if( strcmp(z,"-threadsafe")==0 ){
+
    }else if( cli_strcmp(z,"-threadsafe")==0 ){
      int n;
      n = (int)integerValue(cmdline_option_value(argc,argv,++i));
      switch( n ){
@@ -23111,7 +26051,7 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
         default: sqlite3_config(SQLITE_CONFIG_SERIALIZED);    break;
      }
#ifdef SQLITE_ENABLE_VFSTRACE
-
    }else if( strcmp(z,"-vfstrace")==0 ){
+
    }else if( cli_strcmp(z,"-vfstrace")==0 ){
      extern int vfstrace_register(
         const char *zTraceName,
         const char *zOldVfsName,
@@ -23122,50 +26062,50 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
      vfstrace_register("trace",0,(int(*)(const char*,void*))fputs,stderr,1);
#endif
#ifdef SQLITE_ENABLE_MULTIPLEX
-
    }else if( strcmp(z,"-multiplex")==0 ){
+
    }else if( cli_strcmp(z,"-multiplex")==0 ){
      extern int sqlite3_multiple_initialize(const char*,int);
      sqlite3_multiplex_initialize(0, 1);
#endif
-
    }else if( strcmp(z,"-mmap")==0 ){
+
    }else if( cli_strcmp(z,"-mmap")==0 ){
      sqlite3_int64 sz = integerValue(cmdline_option_value(argc,argv,++i));
      sqlite3_config(SQLITE_CONFIG_MMAP_SIZE, sz, sz);
#ifdef SQLITE_ENABLE_SORTER_REFERENCES
-
    }else if( strcmp(z,"-sorterref")==0 ){
+
    }else if( cli_strcmp(z,"-sorterref")==0 ){
      sqlite3_int64 sz = integerValue(cmdline_option_value(argc,argv,++i));
      sqlite3_config(SQLITE_CONFIG_SORTERREF_SIZE, (int)sz);
#endif
-
    }else if( strcmp(z,"-vfs")==0 ){
+
    }else if( cli_strcmp(z,"-vfs")==0 ){
      zVfs = cmdline_option_value(argc, argv, ++i);
#ifdef SQLITE_HAVE_ZLIB
-
    }else if( strcmp(z,"-zip")==0 ){
+
    }else if( cli_strcmp(z,"-zip")==0 ){
      data.openMode = SHELL_OPEN_ZIPFILE;
#endif
-
    }else if( strcmp(z,"-append")==0 ){
+
    }else if( cli_strcmp(z,"-append")==0 ){
      data.openMode = SHELL_OPEN_APPENDVFS;
#ifndef SQLITE_OMIT_DESERIALIZE
-
    }else if( strcmp(z,"-deserialize")==0 ){
+
    }else if( cli_strcmp(z,"-deserialize")==0 ){
      data.openMode = SHELL_OPEN_DESERIALIZE;
-
    }else if( strcmp(z,"-maxsize")==0 && i+1<argc ){
+
    }else if( cli_strcmp(z,"-maxsize")==0 && i+1<argc ){
      data.szMax = integerValue(argv[++i]);
#endif
-
    }else if( strcmp(z,"-readonly")==0 ){
+
    }else if( cli_strcmp(z,"-readonly")==0 ){
      data.openMode = SHELL_OPEN_READONLY;
-
    }else if( strcmp(z,"-nofollow")==0 ){
+
    }else if( cli_strcmp(z,"-nofollow")==0 ){
      data.openFlags = SQLITE_OPEN_NOFOLLOW;
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
-
    }else if( strncmp(z, "-A",2)==0 ){
+
    }else if( cli_strncmp(z, "-A",2)==0 ){
      /* All remaining command-line arguments are passed to the ".archive"
      ** command, so ignore them */
      break;
#endif
-
    }else if( strcmp(z, "-memtrace")==0 ){
+
    }else if( cli_strcmp(z, "-memtrace")==0 ){
      sqlite3MemTraceActivate(stderr);
-
    }else if( strcmp(z,"-bail")==0 ){
+
    }else if( cli_strcmp(z,"-bail")==0 ){
      bail_on_error = 1;
-
    }else if( strcmp(z,"-nonce")==0 ){
+
    }else if( cli_strcmp(z,"-nonce")==0 ){
      free(data.zNonce);
      data.zNonce = strdup(argv[++i]);
-
    }else if( strcmp(z,"-safe")==0 ){
+
    }else if( cli_strcmp(z,"-safe")==0 ){
      /* no-op - catch this on the second pass */
    }
  }
@@ -23207,7 +26147,9 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
#endif
  }
  data.out = stdout;
+
#ifndef SQLITE_SHELL_FIDDLE
  sqlite3_appendvfs_init(0,0,0);
+
#endif

  /* Go ahead and open the database file if it already exists.  If the
  ** file does not exist, delay opening it.  This prevents empty database
@@ -23233,127 +26175,127 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
    char *z = argv[i];
    if( z[0]!='-' ) continue;
    if( z[1]=='-' ){ z++; }
-
    if( strcmp(z,"-init")==0 ){
+
    if( cli_strcmp(z,"-init")==0 ){
      i++;
-
    }else if( strcmp(z,"-html")==0 ){
+
    }else if( cli_strcmp(z,"-html")==0 ){
      data.mode = MODE_Html;
-
    }else if( strcmp(z,"-list")==0 ){
+
    }else if( cli_strcmp(z,"-list")==0 ){
      data.mode = MODE_List;
-
    }else if( strcmp(z,"-quote")==0 ){
+
    }else if( cli_strcmp(z,"-quote")==0 ){
      data.mode = MODE_Quote;
      sqlite3_snprintf(sizeof(data.colSeparator), data.colSeparator, SEP_Comma);
      sqlite3_snprintf(sizeof(data.rowSeparator), data.rowSeparator, SEP_Row);
-
    }else if( strcmp(z,"-line")==0 ){
+
    }else if( cli_strcmp(z,"-line")==0 ){
      data.mode = MODE_Line;
-
    }else if( strcmp(z,"-column")==0 ){
+
    }else if( cli_strcmp(z,"-column")==0 ){
      data.mode = MODE_Column;
-
    }else if( strcmp(z,"-json")==0 ){
+
    }else if( cli_strcmp(z,"-json")==0 ){
      data.mode = MODE_Json;
-
    }else if( strcmp(z,"-markdown")==0 ){
+
    }else if( cli_strcmp(z,"-markdown")==0 ){
      data.mode = MODE_Markdown;
-
    }else if( strcmp(z,"-table")==0 ){
+
    }else if( cli_strcmp(z,"-table")==0 ){
      data.mode = MODE_Table;
-
    }else if( strcmp(z,"-box")==0 ){
+
    }else if( cli_strcmp(z,"-box")==0 ){
      data.mode = MODE_Box;
-
    }else if( strcmp(z,"-csv")==0 ){
+
    }else if( cli_strcmp(z,"-csv")==0 ){
      data.mode = MODE_Csv;
      memcpy(data.colSeparator,",",2);
#ifdef SQLITE_HAVE_ZLIB
-
    }else if( strcmp(z,"-zip")==0 ){
+
    }else if( cli_strcmp(z,"-zip")==0 ){
      data.openMode = SHELL_OPEN_ZIPFILE;
#endif
-
    }else if( strcmp(z,"-append")==0 ){
+
    }else if( cli_strcmp(z,"-append")==0 ){
      data.openMode = SHELL_OPEN_APPENDVFS;
#ifndef SQLITE_OMIT_DESERIALIZE
-
    }else if( strcmp(z,"-deserialize")==0 ){
+
    }else if( cli_strcmp(z,"-deserialize")==0 ){
      data.openMode = SHELL_OPEN_DESERIALIZE;
-
    }else if( strcmp(z,"-maxsize")==0 && i+1<argc ){
+
    }else if( cli_strcmp(z,"-maxsize")==0 && i+1<argc ){
      data.szMax = integerValue(argv[++i]);
#endif
-
    }else if( strcmp(z,"-readonly")==0 ){
+
    }else if( cli_strcmp(z,"-readonly")==0 ){
      data.openMode = SHELL_OPEN_READONLY;
-
    }else if( strcmp(z,"-nofollow")==0 ){
+
    }else if( cli_strcmp(z,"-nofollow")==0 ){
      data.openFlags |= SQLITE_OPEN_NOFOLLOW;
-
    }else if( strcmp(z,"-ascii")==0 ){
+
    }else if( cli_strcmp(z,"-ascii")==0 ){
      data.mode = MODE_Ascii;
      sqlite3_snprintf(sizeof(data.colSeparator), data.colSeparator, SEP_Unit);
      sqlite3_snprintf(sizeof(data.rowSeparator), data.rowSeparator, SEP_Record);
-
    }else if( strcmp(z,"-tabs")==0 ){
+
    }else if( cli_strcmp(z,"-tabs")==0 ){
      data.mode = MODE_List;
      sqlite3_snprintf(sizeof(data.colSeparator), data.colSeparator, SEP_Tab);
      sqlite3_snprintf(sizeof(data.rowSeparator), data.rowSeparator, SEP_Row);
-
    }else if( strcmp(z,"-separator")==0 ){
+
    }else if( cli_strcmp(z,"-separator")==0 ){
      sqlite3_snprintf(sizeof(data.colSeparator), data.colSeparator,
                       "%s",cmdline_option_value(argc,argv,++i));
-
    }else if( strcmp(z,"-newline")==0 ){
+
    }else if( cli_strcmp(z,"-newline")==0 ){
      sqlite3_snprintf(sizeof(data.rowSeparator), data.rowSeparator,
                       "%s",cmdline_option_value(argc,argv,++i));
-
    }else if( strcmp(z,"-nullvalue")==0 ){
+
    }else if( cli_strcmp(z,"-nullvalue")==0 ){
      sqlite3_snprintf(sizeof(data.nullValue), data.nullValue,
                       "%s",cmdline_option_value(argc,argv,++i));
-
    }else if( strcmp(z,"-header")==0 ){
+
    }else if( cli_strcmp(z,"-header")==0 ){
      data.showHeader = 1;
      ShellSetFlag(&data, SHFLG_HeaderSet);
-
     }else if( strcmp(z,"-noheader")==0 ){
+
     }else if( cli_strcmp(z,"-noheader")==0 ){
      data.showHeader = 0;
      ShellSetFlag(&data, SHFLG_HeaderSet);
-
    }else if( strcmp(z,"-echo")==0 ){
+
    }else if( cli_strcmp(z,"-echo")==0 ){
      ShellSetFlag(&data, SHFLG_Echo);
-
    }else if( strcmp(z,"-eqp")==0 ){
+
    }else if( cli_strcmp(z,"-eqp")==0 ){
      data.autoEQP = AUTOEQP_on;
-
    }else if( strcmp(z,"-eqpfull")==0 ){
+
    }else if( cli_strcmp(z,"-eqpfull")==0 ){
      data.autoEQP = AUTOEQP_full;
-
    }else if( strcmp(z,"-stats")==0 ){
+
    }else if( cli_strcmp(z,"-stats")==0 ){
      data.statsOn = 1;
-
    }else if( strcmp(z,"-scanstats")==0 ){
+
    }else if( cli_strcmp(z,"-scanstats")==0 ){
      data.scanstatsOn = 1;
-
    }else if( strcmp(z,"-backslash")==0 ){
+
    }else if( cli_strcmp(z,"-backslash")==0 ){
      /* Undocumented command-line option: -backslash
      ** Causes C-style backslash escapes to be evaluated in SQL statements
      ** prior to sending the SQL into SQLite.  Useful for injecting
      ** crazy bytes in the middle of SQL statements for testing and debugging.
      */
      ShellSetFlag(&data, SHFLG_Backslash);
-
    }else if( strcmp(z,"-bail")==0 ){
+
    }else if( cli_strcmp(z,"-bail")==0 ){
      /* No-op.  The bail_on_error flag should already be set. */
-
    }else if( strcmp(z,"-version")==0 ){
+
    }else if( cli_strcmp(z,"-version")==0 ){
      printf("%s %s\n", sqlite3_libversion(), sqlite3_sourceid());
      return 0;
-
    }else if( strcmp(z,"-interactive")==0 ){
+
    }else if( cli_strcmp(z,"-interactive")==0 ){
      stdin_is_interactive = 1;
-
    }else if( strcmp(z,"-batch")==0 ){
+
    }else if( cli_strcmp(z,"-batch")==0 ){
      stdin_is_interactive = 0;
-
    }else if( strcmp(z,"-heap")==0 ){
+
    }else if( cli_strcmp(z,"-heap")==0 ){
      i++;
-
    }else if( strcmp(z,"-pagecache")==0 ){
+
    }else if( cli_strcmp(z,"-pagecache")==0 ){
      i+=2;
-
    }else if( strcmp(z,"-lookaside")==0 ){
+
    }else if( cli_strcmp(z,"-lookaside")==0 ){
      i+=2;
-
    }else if( strcmp(z,"-threadsafe")==0 ){
+
    }else if( cli_strcmp(z,"-threadsafe")==0 ){
      i+=2;
-
    }else if( strcmp(z,"-nonce")==0 ){
+
    }else if( cli_strcmp(z,"-nonce")==0 ){
      i += 2;
-
    }else if( strcmp(z,"-mmap")==0 ){
+
    }else if( cli_strcmp(z,"-mmap")==0 ){
      i++;
-
    }else if( strcmp(z,"-memtrace")==0 ){
+
    }else if( cli_strcmp(z,"-memtrace")==0 ){
      i++;
#ifdef SQLITE_ENABLE_SORTER_REFERENCES
-
    }else if( strcmp(z,"-sorterref")==0 ){
+
    }else if( cli_strcmp(z,"-sorterref")==0 ){
      i++;
#endif
-
    }else if( strcmp(z,"-vfs")==0 ){
+
    }else if( cli_strcmp(z,"-vfs")==0 ){
      i++;
#ifdef SQLITE_ENABLE_VFSTRACE
-
    }else if( strcmp(z,"-vfstrace")==0 ){
+
    }else if( cli_strcmp(z,"-vfstrace")==0 ){
      i++;
#endif
#ifdef SQLITE_ENABLE_MULTIPLEX
-
    }else if( strcmp(z,"-multiplex")==0 ){
+
    }else if( cli_strcmp(z,"-multiplex")==0 ){
      i++;
#endif
-
    }else if( strcmp(z,"-help")==0 ){
+
    }else if( cli_strcmp(z,"-help")==0 ){
      usage(1);
-
    }else if( strcmp(z,"-cmd")==0 ){
+
    }else if( cli_strcmp(z,"-cmd")==0 ){
      /* Run commands that follow -cmd first and separately from commands
      ** that simply appear on the command-line.  This seems goofy.  It would
      ** be better if all commands ran in the order that they appear.  But
@@ -23375,7 +26317,7 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
        }
      }
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
-
    }else if( strncmp(z, "-A", 2)==0 ){
+
    }else if( cli_strncmp(z, "-A", 2)==0 ){
      if( nCmd>0 ){
        utf8_printf(stderr, "Error: cannot mix regular SQL or dot-commands"
                            " with \"%s\"\n", z);
@@ -23391,7 +26333,7 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
      readStdin = 0;
      break;
#endif
-
    }else if( strcmp(z,"-safe")==0 ){
+
    }else if( cli_strcmp(z,"-safe")==0 ){
      data.bSafeMode = data.bSafeModePersist = 1;
    }else{
      utf8_printf(stderr,"%s: Error: unknown option: %s\n", Argv0, z);
@@ -23473,6 +26415,9 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
      rc = process_input(&data);
    }
  }
+
#ifndef SQLITE_SHELL_FIDDLE
+
  /* In WASM mode we have to leave the db state in place so that
+
  ** client code can "push" SQL into it after this call returns. */
  free(azCmd);
  set_table_name(&data, 0);
  if( data.db ){
@@ -23499,5 +26444,137 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
  /* Clear the global data structure so that valgrind will detect memory
  ** leaks */
  memset(&data, 0, sizeof(data));
+
#ifdef SQLITE_DEBUG
+
  if( sqlite3_memory_used()>mem_main_enter ){
+
    utf8_printf(stderr, "Memory leaked: %u bytes\n",
+
                (unsigned int)(sqlite3_memory_used()-mem_main_enter));
+
  }
+
#endif
+
#endif /* !SQLITE_SHELL_FIDDLE */
+
  return rc;
+
}
+

+

+
#ifdef SQLITE_SHELL_FIDDLE
+
/* Only for emcc experimentation purposes. */
+
int fiddle_experiment(int a,int b){
+
  return a + b;
+
}
+

+
/*
+
** Returns a pointer to the current DB handle.
+
*/
+
sqlite3 * fiddle_db_handle(){
+
  return globalDb;
+
}
+

+
/*
+
** Returns a pointer to the given DB name's VFS. If zDbName is 0 then
+
** "main" is assumed. Returns 0 if no db with the given name is
+
** open.
+
*/
+
sqlite3_vfs * fiddle_db_vfs(const char *zDbName){
+
  sqlite3_vfs * pVfs = 0;
+
  if(globalDb){
+
    sqlite3_file_control(globalDb, zDbName ? zDbName : "main",
+
                         SQLITE_FCNTL_VFS_POINTER, &pVfs);
+
  }
+
  return pVfs;
+
}
+

+
/* Only for emcc experimentation purposes. */
+
sqlite3 * fiddle_db_arg(sqlite3 *arg){
+
    printf("fiddle_db_arg(%p)\n", (const void*)arg);
+
    return arg;
+
}
+

+
/*
+
** Intended to be called via a SharedWorker() while a separate
+
** SharedWorker() (which manages the wasm module) is performing work
+
** which should be interrupted. Unfortunately, SharedWorker is not
+
** portable enough to make real use of.
+
*/
+
void fiddle_interrupt(void){
+
  if( globalDb ) sqlite3_interrupt(globalDb);
+
}
+

+
/*
+
** Returns the filename of the given db name, assuming "main" if
+
** zDbName is NULL. Returns NULL if globalDb is not opened.
+
*/
+
const char * fiddle_db_filename(const char * zDbName){
+
    return globalDb
+
      ? sqlite3_db_filename(globalDb, zDbName ? zDbName : "main")
+
      : NULL;
+
}
+

+
/*
+
** Completely wipes out the contents of the currently-opened database
+
** but leaves its storage intact for reuse.
+
*/
+
void fiddle_reset_db(void){
+
  if( globalDb ){
+
    int rc = sqlite3_db_config(globalDb, SQLITE_DBCONFIG_RESET_DATABASE, 1, 0);
+
    if( 0==rc ) rc = sqlite3_exec(globalDb, "VACUUM", 0, 0, 0);
+
    sqlite3_db_config(globalDb, SQLITE_DBCONFIG_RESET_DATABASE, 0, 0);
+
  }
+
}
+

+
/*
+
** Uses the current database's VFS xRead to stream the db file's
+
** contents out to the given callback. The callback gets a single
+
** chunk of size n (its 2nd argument) on each call and must return 0
+
** on success, non-0 on error. This function returns 0 on success,
+
** SQLITE_NOTFOUND if no db is open, or propagates any other non-0
+
** code from the callback. Note that this is not thread-friendly: it
+
** expects that it will be the only thread reading the db file and
+
** takes no measures to ensure that is the case.
+
*/
+
int fiddle_export_db( int (*xCallback)(unsigned const char *zOut, int n) ){
+
  sqlite3_int64 nSize = 0;
+
  sqlite3_int64 nPos = 0;
+
  sqlite3_file * pFile = 0;
+
  unsigned char buf[1024 * 8];
+
  int nBuf = (int)sizeof(buf);
+
  int rc = shellState.db
+
    ? sqlite3_file_control(shellState.db, "main",
+
                           SQLITE_FCNTL_FILE_POINTER, &pFile)
+
    : SQLITE_NOTFOUND;
+
  if( rc ) return rc;
+
  rc = pFile->pMethods->xFileSize(pFile, &nSize);
+
  if( rc ) return rc;
+
  if(nSize % nBuf){
+
    /* DB size is not an even multiple of the buffer size. Reduce
+
    ** buffer size so that we do not unduly inflate the db size when
+
    ** exporting. */
+
    if(0 == nSize % 4096) nBuf = 4096;
+
    else if(0 == nSize % 2048) nBuf = 2048;
+
    else if(0 == nSize % 1024) nBuf = 1024;
+
    else nBuf = 512;
+
  }
+
  for( ; 0==rc && nPos<nSize; nPos += nBuf ){
+
    rc = pFile->pMethods->xRead(pFile, buf, nBuf, nPos);
+
    if(SQLITE_IOERR_SHORT_READ == rc){
+
      rc = (nPos + nBuf) < nSize ? rc : 0/*assume EOF*/;
+
    }
+
    if( 0==rc ) rc = xCallback(buf, nBuf);
+
  }
  return rc;
}
+

+
/*
+
** Trivial exportable function for emscripten. It processes zSql as if
+
** it were input to the sqlite3 shell and redirects all output to the
+
** wasm binding. fiddle_main() must have been called before this
+
** is called, or results are undefined.
+
*/
+
void fiddle_exec(const char * zSql){
+
  if(zSql && *zSql){
+
    if('.'==*zSql) puts(zSql);
+
    shellState.wasm.zInput = zSql;
+
    shellState.wasm.zPos = zSql;
+
    process_input(&shellState);
+
    shellState.wasm.zInput = shellState.wasm.zPos = 0;
+
  }
+
}
+
#endif /* SQLITE_SHELL_FIDDLE */
modified external/sqlite/sqlite3.c
@@ -1,6 +1,6 @@
/******************************************************************************
** This file is an amalgamation of many separate C source files from SQLite
-
** version 3.38.5.  By combining all the individual C code files into this
+
** version 3.40.0.  By combining all the individual C code files into this
** single large file, the entire code can be compiled as a single translation
** unit.  This allows many compilers to do optimizations that would not be
** possible if the files were compiled separately.  Performance improvements
@@ -452,9 +452,9 @@ extern "C" {
** [sqlite3_libversion_number()], [sqlite3_sourceid()],
** [sqlite_version()] and [sqlite_source_id()].
*/
-
#define SQLITE_VERSION        "3.38.5"
-
#define SQLITE_VERSION_NUMBER 3038005
-
#define SQLITE_SOURCE_ID      "2022-05-06 15:25:27 78d9c993d404cdfaa7fdd2973fa1052e3da9f66215cff9c5540ebe55c407d9fe"
+
#define SQLITE_VERSION        "3.40.0"
+
#define SQLITE_VERSION_NUMBER 3040000
+
#define SQLITE_SOURCE_ID      "2022-11-16 12:10:08 89c459e766ea7e9165d0beeb124708b955a4950d0f4792f457465d71b158d318"

/*
** CAPI3REF: Run-Time Library Version Numbers
@@ -976,13 +976,17 @@ SQLITE_API int sqlite3_exec(
**
** SQLite uses one of these integer values as the second
** argument to calls it makes to the xLock() and xUnlock() methods
-
** of an [sqlite3_io_methods] object.
+
** of an [sqlite3_io_methods] object.  These values are ordered from
+
** lest restrictive to most restrictive.
+
**
+
** The argument to xLock() is always SHARED or higher.  The argument to
+
** xUnlock is either SHARED or NONE.
*/
-
#define SQLITE_LOCK_NONE          0
-
#define SQLITE_LOCK_SHARED        1
-
#define SQLITE_LOCK_RESERVED      2
-
#define SQLITE_LOCK_PENDING       3
-
#define SQLITE_LOCK_EXCLUSIVE     4
+
#define SQLITE_LOCK_NONE          0       /* xUnlock() only */
+
#define SQLITE_LOCK_SHARED        1       /* xLock() or xUnlock() */
+
#define SQLITE_LOCK_RESERVED      2       /* xLock() only */
+
#define SQLITE_LOCK_PENDING       3       /* xLock() only */
+
#define SQLITE_LOCK_EXCLUSIVE     4       /* xLock() only */

/*
** CAPI3REF: Synchronization Type Flags
@@ -1060,7 +1064,14 @@ struct sqlite3_file {
** <li> [SQLITE_LOCK_PENDING], or
** <li> [SQLITE_LOCK_EXCLUSIVE].
** </ul>
-
** xLock() increases the lock. xUnlock() decreases the lock.
+
** xLock() upgrades the database file lock.  In other words, xLock() moves the
+
** database file lock in the direction NONE toward EXCLUSIVE. The argument to
+
** xLock() is always on of SHARED, RESERVED, PENDING, or EXCLUSIVE, never
+
** SQLITE_LOCK_NONE.  If the database file lock is already at or above the
+
** requested lock, then the call to xLock() is a no-op.
+
** xUnlock() downgrades the database file lock to either SHARED or NONE.
+
*  If the lock is already at or below the requested lock state, then the call
+
** to xUnlock() is a no-op.
** The xCheckReservedLock() method checks whether any database connection,
** either in this process or in some other process, is holding a RESERVED,
** PENDING, or EXCLUSIVE lock on the file.  It returns true
@@ -1165,9 +1176,8 @@ struct sqlite3_io_methods {
** opcode causes the xFileControl method to write the current state of
** the lock (one of [SQLITE_LOCK_NONE], [SQLITE_LOCK_SHARED],
** [SQLITE_LOCK_RESERVED], [SQLITE_LOCK_PENDING], or [SQLITE_LOCK_EXCLUSIVE])
-
** into an integer that the pArg argument points to. This capability
-
** is used during testing and is only available when the SQLITE_TEST
-
** compile-time option is used.
+
** into an integer that the pArg argument points to.
+
** This capability is only available if SQLite is compiled with [SQLITE_DEBUG].
**
** <li>[[SQLITE_FCNTL_SIZE_HINT]]
** The [SQLITE_FCNTL_SIZE_HINT] opcode is used by SQLite to give the VFS
@@ -1560,6 +1570,26 @@ typedef struct sqlite3_mutex sqlite3_mutex;
typedef struct sqlite3_api_routines sqlite3_api_routines;

/*
+
** CAPI3REF: File Name
+
**
+
** Type [sqlite3_filename] is used by SQLite to pass filenames to the
+
** xOpen method of a [VFS]. It may be cast to (const char*) and treated
+
** as a normal, nul-terminated, UTF-8 buffer containing the filename, but
+
** may also be passed to special APIs such as:
+
**
+
** <ul>
+
** <li>  sqlite3_filename_database()
+
** <li>  sqlite3_filename_journal()
+
** <li>  sqlite3_filename_wal()
+
** <li>  sqlite3_uri_parameter()
+
** <li>  sqlite3_uri_boolean()
+
** <li>  sqlite3_uri_int64()
+
** <li>  sqlite3_uri_key()
+
** </ul>
+
*/
+
typedef const char *sqlite3_filename;
+

+
/*
** CAPI3REF: OS Interface Object
**
** An instance of the sqlite3_vfs object defines the interface between
@@ -1737,7 +1767,7 @@ struct sqlite3_vfs {
  sqlite3_vfs *pNext;      /* Next registered VFS */
  const char *zName;       /* Name of this virtual file system */
  void *pAppData;          /* Pointer to application-specific data */
-
  int (*xOpen)(sqlite3_vfs*, const char *zName, sqlite3_file*,
+
  int (*xOpen)(sqlite3_vfs*, sqlite3_filename zName, sqlite3_file*,
               int flags, int *pOutFlags);
  int (*xDelete)(sqlite3_vfs*, const char *zName, int syncDir);
  int (*xAccess)(sqlite3_vfs*, const char *zName, int flags, int *pResOut);
@@ -2615,6 +2645,7 @@ struct sqlite3_mem_methods {
** <ul>
** <li> The [PRAGMA writable_schema=ON] statement.
** <li> The [PRAGMA journal_mode=OFF] statement.
+
** <li> The [PRAGMA schema_version=N] statement.
** <li> Writes to the [sqlite_dbpage] virtual table.
** <li> Direct writes to [shadow tables].
** </ul>
@@ -3730,6 +3761,9 @@ SQLITE_API void sqlite3_progress_handler(sqlite3*, int, int(*)(void*), void*);
** <dd>The database is opened [shared cache] enabled, overriding
** the default shared cache setting provided by
** [sqlite3_enable_shared_cache()].)^
+
** The [use of shared cache mode is discouraged] and hence shared cache
+
** capabilities may be omitted from many builds of SQLite.  In such cases,
+
** this option is a no-op.
**
** ^(<dt>[SQLITE_OPEN_PRIVATECACHE]</dt>
** <dd>The database is opened [shared cache] disabled, overriding
@@ -3745,7 +3779,7 @@ SQLITE_API void sqlite3_progress_handler(sqlite3*, int, int(*)(void*), void*);
** to return an extended result code.</dd>
**
** [[OPEN_NOFOLLOW]] ^(<dt>[SQLITE_OPEN_NOFOLLOW]</dt>
-
** <dd>The database filename is not allowed to be a symbolic link</dd>
+
** <dd>The database filename is not allowed to contain a symbolic link</dd>
** </dl>)^
**
** If the 3rd parameter to sqlite3_open_v2() is not one of the
@@ -4004,10 +4038,10 @@ SQLITE_API int sqlite3_open_v2(
**
** See the [URI filename] documentation for additional information.
*/
-
SQLITE_API const char *sqlite3_uri_parameter(const char *zFilename, const char *zParam);
-
SQLITE_API int sqlite3_uri_boolean(const char *zFile, const char *zParam, int bDefault);
-
SQLITE_API sqlite3_int64 sqlite3_uri_int64(const char*, const char*, sqlite3_int64);
-
SQLITE_API const char *sqlite3_uri_key(const char *zFilename, int N);
+
SQLITE_API const char *sqlite3_uri_parameter(sqlite3_filename z, const char *zParam);
+
SQLITE_API int sqlite3_uri_boolean(sqlite3_filename z, const char *zParam, int bDefault);
+
SQLITE_API sqlite3_int64 sqlite3_uri_int64(sqlite3_filename, const char*, sqlite3_int64);
+
SQLITE_API const char *sqlite3_uri_key(sqlite3_filename z, int N);

/*
** CAPI3REF:  Translate filenames
@@ -4036,9 +4070,9 @@ SQLITE_API const char *sqlite3_uri_key(const char *zFilename, int N);
** return value from [sqlite3_db_filename()], then the result is
** undefined and is likely a memory access violation.
*/
-
SQLITE_API const char *sqlite3_filename_database(const char*);
-
SQLITE_API const char *sqlite3_filename_journal(const char*);
-
SQLITE_API const char *sqlite3_filename_wal(const char*);
+
SQLITE_API const char *sqlite3_filename_database(sqlite3_filename);
+
SQLITE_API const char *sqlite3_filename_journal(sqlite3_filename);
+
SQLITE_API const char *sqlite3_filename_wal(sqlite3_filename);

/*
** CAPI3REF:  Database File Corresponding To A Journal
@@ -4104,14 +4138,14 @@ SQLITE_API sqlite3_file *sqlite3_database_file_object(const char*);
** then the corresponding [sqlite3_module.xClose() method should also be
** invoked prior to calling sqlite3_free_filename(Y).
*/
-
SQLITE_API char *sqlite3_create_filename(
+
SQLITE_API sqlite3_filename sqlite3_create_filename(
  const char *zDatabase,
  const char *zJournal,
  const char *zWal,
  int nParam,
  const char **azParam
);
-
SQLITE_API void sqlite3_free_filename(char*);
+
SQLITE_API void sqlite3_free_filename(sqlite3_filename);

/*
** CAPI3REF: Error Codes And Messages
@@ -5814,6 +5848,16 @@ SQLITE_API SQLITE_DEPRECATED int sqlite3_memory_alarm(void(*)(void*,sqlite3_int6
** then the conversion is performed.  Otherwise no conversion occurs.
** The [SQLITE_INTEGER | datatype] after conversion is returned.)^
**
+
** ^(The sqlite3_value_encoding(X) interface returns one of [SQLITE_UTF8],
+
** [SQLITE_UTF16BE], or [SQLITE_UTF16LE] according to the current encoding
+
** of the value X, assuming that X has type TEXT.)^  If sqlite3_value_type(X)
+
** returns something other than SQLITE_TEXT, then the return value from
+
** sqlite3_value_encoding(X) is meaningless.  ^Calls to
+
** sqlite3_value_text(X), sqlite3_value_text16(X), sqlite3_value_text16be(X),
+
** sqlite3_value_text16le(X), sqlite3_value_bytes(X), or
+
** sqlite3_value_bytes16(X) might change the encoding of the value X and
+
** thus change the return from subsequent calls to sqlite3_value_encoding(X).
+
**
** ^Within the [xUpdate] method of a [virtual table], the
** sqlite3_value_nochange(X) interface returns true if and only if
** the column corresponding to X is unchanged by the UPDATE operation
@@ -5878,6 +5922,7 @@ SQLITE_API int sqlite3_value_type(sqlite3_value*);
SQLITE_API int sqlite3_value_numeric_type(sqlite3_value*);
SQLITE_API int sqlite3_value_nochange(sqlite3_value*);
SQLITE_API int sqlite3_value_frombind(sqlite3_value*);
+
SQLITE_API int sqlite3_value_encoding(sqlite3_value*);

/*
** CAPI3REF: Finding The Subtype Of SQL Values
@@ -5899,7 +5944,8 @@ SQLITE_API unsigned int sqlite3_value_subtype(sqlite3_value*);
** object D and returns a pointer to that copy.  ^The [sqlite3_value] returned
** is a [protected sqlite3_value] object even if the input is not.
** ^The sqlite3_value_dup(V) interface returns NULL if V is NULL or if a
-
** memory allocation fails.
+
** memory allocation fails. ^If V is a [pointer value], then the result
+
** of sqlite3_value_dup(V) is a NULL value.
**
** ^The sqlite3_value_free(V) interface frees an [sqlite3_value] object
** previously obtained from [sqlite3_value_dup()].  ^If V is a NULL pointer
@@ -5930,7 +5976,7 @@ SQLITE_API void sqlite3_value_free(sqlite3_value*);
**
** ^The sqlite3_aggregate_context(C,N) routine returns a NULL pointer
** when first called if N is less than or equal to zero or if a memory
-
** allocate error occurs.
+
** allocation error occurs.
**
** ^(The amount of space allocated by sqlite3_aggregate_context(C,N) is
** determined by the N parameter on first successful call.  Changing the
@@ -6135,9 +6181,10 @@ typedef void (*sqlite3_destructor_type)(void*);
** of [SQLITE_UTF8], [SQLITE_UTF16], [SQLITE_UTF16BE], or [SQLITE_UTF16LE].
** ^SQLite takes the text result from the application from
** the 2nd parameter of the sqlite3_result_text* interfaces.
-
** ^If the 3rd parameter to the sqlite3_result_text* interfaces
-
** is negative, then SQLite takes result text from the 2nd parameter
-
** through the first zero character.
+
** ^If the 3rd parameter to any of the sqlite3_result_text* interfaces
+
** other than sqlite3_result_text64() is negative, then SQLite computes
+
** the string length itself by searching the 2nd parameter for the first
+
** zero character.
** ^If the 3rd parameter to the sqlite3_result_text* interfaces
** is non-negative, then as many bytes (not characters) of the text
** pointed to by the 2nd parameter are taken as the application-defined
@@ -6582,6 +6629,28 @@ SQLITE_API int sqlite3_get_autocommit(sqlite3*);
SQLITE_API sqlite3 *sqlite3_db_handle(sqlite3_stmt*);

/*
+
** CAPI3REF: Return The Schema Name For A Database Connection
+
** METHOD: sqlite3
+
**
+
** ^The sqlite3_db_name(D,N) interface returns a pointer to the schema name
+
** for the N-th database on database connection D, or a NULL pointer of N is
+
** out of range.  An N value of 0 means the main database file.  An N of 1 is
+
** the "temp" schema.  Larger values of N correspond to various ATTACH-ed
+
** databases.
+
**
+
** Space to hold the string that is returned by sqlite3_db_name() is managed
+
** by SQLite itself.  The string might be deallocated by any operation that
+
** changes the schema, including [ATTACH] or [DETACH] or calls to
+
** [sqlite3_serialize()] or [sqlite3_deserialize()], even operations that
+
** occur on a different thread.  Applications that need to
+
** remember the string long-term should make their own copy.  Applications that
+
** are accessing the same database connection simultaneously on multiple
+
** threads should mutex-protect calls to this API and should make their own
+
** private copy of the result prior to releasing the mutex.
+
*/
+
SQLITE_API const char *sqlite3_db_name(sqlite3 *db, int N);
+

+
/*
** CAPI3REF: Return The Filename For A Database Connection
** METHOD: sqlite3
**
@@ -6611,7 +6680,7 @@ SQLITE_API sqlite3 *sqlite3_db_handle(sqlite3_stmt*);
** <li> [sqlite3_filename_wal()]
** </ul>
*/
-
SQLITE_API const char *sqlite3_db_filename(sqlite3 *db, const char *zDbName);
+
SQLITE_API sqlite3_filename sqlite3_db_filename(sqlite3 *db, const char *zDbName);

/*
** CAPI3REF: Determine if a database is read-only
@@ -6748,7 +6817,7 @@ SQLITE_API void *sqlite3_rollback_hook(sqlite3*, void(*)(void *), void*);
** function C that is invoked prior to each autovacuum of the database
** file.  ^The callback is passed a copy of the generic data pointer (P),
** the schema-name of the attached database that is being autovacuumed,
-
** the the size of the database file in pages, the number of free pages,
+
** the size of the database file in pages, the number of free pages,
** and the number of bytes per page, respectively.  The callback should
** return the number of free pages that should be removed by the
** autovacuum.  ^If the callback returns zero, then no autovacuum happens.
@@ -6869,6 +6938,11 @@ SQLITE_API void *sqlite3_update_hook(
** to the same database. Sharing is enabled if the argument is true
** and disabled if the argument is false.)^
**
+
** This interface is omitted if SQLite is compiled with
+
** [-DSQLITE_OMIT_SHARED_CACHE].  The [-DSQLITE_OMIT_SHARED_CACHE]
+
** compile-time option is recommended because the
+
** [use of shared cache mode is discouraged].
+
**
** ^Cache sharing is enabled and disabled for an entire process.
** This is a change as of SQLite [version 3.5.0] ([dateof:3.5.0]).
** In prior versions of SQLite,
@@ -6967,7 +7041,7 @@ SQLITE_API int sqlite3_db_release_memory(sqlite3*);
** ^The soft heap limit may not be greater than the hard heap limit.
** ^If the hard heap limit is enabled and if sqlite3_soft_heap_limit(N)
** is invoked with a value of N that is greater than the hard heap limit,
-
** the the soft heap limit is set to the value of the hard heap limit.
+
** the soft heap limit is set to the value of the hard heap limit.
** ^The soft heap limit is automatically enabled whenever the hard heap
** limit is enabled. ^When sqlite3_hard_heap_limit64(N) is invoked and
** the soft heap limit is outside the range of 1..N, then the soft heap
@@ -9262,7 +9336,7 @@ typedef struct sqlite3_backup sqlite3_backup;
** if the application incorrectly accesses the destination [database connection]
** and so no error code is reported, but the operations may malfunction
** nevertheless.  Use of the destination database connection while a
-
** backup is in progress might also also cause a mutex deadlock.
+
** backup is in progress might also cause a mutex deadlock.
**
** If running in [shared cache mode], the application must
** guarantee that the shared cache used by the destination database
@@ -9690,7 +9764,7 @@ SQLITE_API int sqlite3_wal_checkpoint_v2(
*/
#define SQLITE_CHECKPOINT_PASSIVE  0  /* Do as much as possible w/o blocking */
#define SQLITE_CHECKPOINT_FULL     1  /* Wait for writers, then checkpoint */
-
#define SQLITE_CHECKPOINT_RESTART  2  /* Like FULL but wait for for readers */
+
#define SQLITE_CHECKPOINT_RESTART  2  /* Like FULL but wait for readers */
#define SQLITE_CHECKPOINT_TRUNCATE 3  /* Like RESTART but also truncate WAL */

/*
@@ -9860,8 +9934,8 @@ SQLITE_API SQLITE_EXPERIMENTAL const char *sqlite3_vtab_collation(sqlite3_index_
** of a [virtual table] implementation. The result of calling this
** interface from outside of xBestIndex() is undefined and probably harmful.
**
-
** ^The sqlite3_vtab_distinct() interface returns an integer that is
-
** either 0, 1, or 2.  The integer returned by sqlite3_vtab_distinct()
+
** ^The sqlite3_vtab_distinct() interface returns an integer between 0 and
+
** 3.  The integer returned by sqlite3_vtab_distinct()
** gives the virtual table additional information about how the query
** planner wants the output to be ordered. As long as the virtual table
** can meet the ordering requirements of the query planner, it may set
@@ -9893,6 +9967,13 @@ SQLITE_API SQLITE_EXPERIMENTAL const char *sqlite3_vtab_collation(sqlite3_index_
** that have the same value for all columns identified by "aOrderBy".
** ^However omitting the extra rows is optional.
** This mode is used for a DISTINCT query.
+
** <li value="3"><p>
+
** ^(If the sqlite3_vtab_distinct() interface returns 3, that means
+
** that the query planner needs only distinct rows but it does need the
+
** rows to be sorted.)^ ^The virtual table implementation is free to omit
+
** rows that are identical in all aOrderBy columns, if it wants to, but
+
** it is not required to omit any rows.  This mode is used for queries
+
** that have both DISTINCT and ORDER BY clauses.
** </ol>
**
** ^For the purposes of comparing virtual table output values to see if the
@@ -13115,11 +13196,16 @@ struct fts5_api {
/************** Continuing where we left off in sqliteInt.h ******************/

/*
+
** Reuse the STATIC_LRU for mutex access to sqlite3_temp_directory.
+
*/
+
#define SQLITE_MUTEX_STATIC_TEMPDIR SQLITE_MUTEX_STATIC_VFS1
+

+
/*
** Include the configuration header output by 'configure' if we're using the
** autoconf-based build
*/
#if defined(_HAVE_SQLITE_CONFIG_H) && !defined(SQLITECONFIG_H)
-
#include "config.h"
+
#include "sqlite_cfg.h"
#define SQLITECONFIG_H 1
#endif

@@ -14357,8 +14443,19 @@ typedef INT16_TYPE LogEst;
/*
** Round up a number to the next larger multiple of 8.  This is used
** to force 8-byte alignment on 64-bit architectures.
+
**
+
** ROUND8() always does the rounding, for any argument.
+
**
+
** ROUND8P() assumes that the argument is already an integer number of
+
** pointers in size, and so it is a no-op on systems where the pointer
+
** size is 8.
*/
#define ROUND8(x)     (((x)+7)&~7)
+
#if SQLITE_PTRSIZE==8
+
# define ROUND8P(x)   (x)
+
#else
+
# define ROUND8P(x)   (((x)+7)&~7)
+
#endif

/*
** Round down to the nearest multiple of 8
@@ -14421,22 +14518,23 @@ typedef INT16_TYPE LogEst;
#endif

/*
-
** SELECTTRACE_ENABLED will be either 1 or 0 depending on whether or not
-
** the Select query generator tracing logic is turned on.
+
** TREETRACE_ENABLED will be either 1 or 0 depending on whether or not
+
** the Abstract Syntax Tree tracing logic is turned on.
*/
#if !defined(SQLITE_AMALGAMATION)
-
SQLITE_PRIVATE u32 sqlite3SelectTrace;
+
SQLITE_PRIVATE u32 sqlite3TreeTrace;
#endif
#if defined(SQLITE_DEBUG) \
-
    && (defined(SQLITE_TEST) || defined(SQLITE_ENABLE_SELECTTRACE))
-
# define SELECTTRACE_ENABLED 1
+
    && (defined(SQLITE_TEST) || defined(SQLITE_ENABLE_SELECTTRACE) \
+
                             || defined(SQLITE_ENABLE_TREETRACE))
+
# define TREETRACE_ENABLED 1
# define SELECTTRACE(K,P,S,X)  \
-
  if(sqlite3SelectTrace&(K))   \
+
  if(sqlite3TreeTrace&(K))   \
    sqlite3DebugPrintf("%u/%d/%p: ",(S)->selId,(P)->addrExplain,(S)),\
    sqlite3DebugPrintf X
#else
# define SELECTTRACE(K,P,S,X)
-
# define SELECTTRACE_ENABLED 0
+
# define TREETRACE_ENABLED 0
#endif

/*
@@ -14521,7 +14619,7 @@ struct BusyHandler {
** pointer will work here as long as it is distinct from SQLITE_STATIC
** and SQLITE_TRANSIENT.
*/
-
#define SQLITE_DYNAMIC   ((sqlite3_destructor_type)sqlite3OomFault)
+
#define SQLITE_DYNAMIC   ((sqlite3_destructor_type)sqlite3OomClear)

/*
** When SQLITE_OMIT_WSD is defined, it means that the target platform does
@@ -14590,6 +14688,7 @@ typedef struct FuncDef FuncDef;
typedef struct FuncDefHash FuncDefHash;
typedef struct IdList IdList;
typedef struct Index Index;
+
typedef struct IndexedExpr IndexedExpr;
typedef struct IndexSample IndexSample;
typedef struct KeyClass KeyClass;
typedef struct KeyInfo KeyInfo;
@@ -14597,6 +14696,7 @@ typedef struct Lookaside Lookaside;
typedef struct LookasideSlot LookasideSlot;
typedef struct Module Module;
typedef struct NameContext NameContext;
+
typedef struct OnOrUsing OnOrUsing;
typedef struct Parse Parse;
typedef struct ParseCleanup ParseCleanup;
typedef struct PreUpdate PreUpdate;
@@ -14654,6 +14754,7 @@ typedef struct With With;
#define MASKBIT32(n)  (((unsigned int)1)<<(n))
#define SMASKBIT32(n) ((n)<=31?((unsigned int)1)<<(n):0)
#define ALLBITS       ((Bitmask)-1)
+
#define TOPBIT        (((Bitmask)1)<<(BMS-1))

/* A VList object records a mapping between parameters/variables/wildcards
** in the SQL statement (such as $abc, @pqr, or :xyz) and the integer
@@ -14668,6 +14769,331 @@ typedef int VList;
** "BusyHandler" typedefs. vdbe.h also requires a few of the opaque
** pointer types (i.e. FuncDef) defined above.
*/
+
/************** Include os.h in the middle of sqliteInt.h ********************/
+
/************** Begin file os.h **********************************************/
+
/*
+
** 2001 September 16
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** This header file (together with is companion C source-code file
+
** "os.c") attempt to abstract the underlying operating system so that
+
** the SQLite library will work on both POSIX and windows systems.
+
**
+
** This header file is #include-ed by sqliteInt.h and thus ends up
+
** being included by every source file.
+
*/
+
#ifndef _SQLITE_OS_H_
+
#define _SQLITE_OS_H_
+

+
/*
+
** Attempt to automatically detect the operating system and setup the
+
** necessary pre-processor macros for it.
+
*/
+
/************** Include os_setup.h in the middle of os.h *********************/
+
/************** Begin file os_setup.h ****************************************/
+
/*
+
** 2013 November 25
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** This file contains pre-processor directives related to operating system
+
** detection and/or setup.
+
*/
+
#ifndef SQLITE_OS_SETUP_H
+
#define SQLITE_OS_SETUP_H
+

+
/*
+
** Figure out if we are dealing with Unix, Windows, or some other operating
+
** system.
+
**
+
** After the following block of preprocess macros, all of
+
**
+
**    SQLITE_OS_KV
+
**    SQLITE_OS_OTHER
+
**    SQLITE_OS_UNIX
+
**    SQLITE_OS_WIN
+
**
+
** will defined to either 1 or 0. One of them will be 1. The others will be 0.
+
** If none of the macros are initially defined, then select either
+
** SQLITE_OS_UNIX or SQLITE_OS_WIN depending on the target platform.
+
**
+
** If SQLITE_OS_OTHER=1 is specified at compile-time, then the application
+
** must provide its own VFS implementation together with sqlite3_os_init()
+
** and sqlite3_os_end() routines.
+
*/
+
#if !defined(SQLITE_OS_KV) && !defined(SQLITE_OS_OTHER) && \
+
       !defined(SQLITE_OS_UNIX) && !defined(SQLITE_OS_WIN)
+
#  if defined(_WIN32) || defined(WIN32) || defined(__CYGWIN__) || \
+
          defined(__MINGW32__) || defined(__BORLANDC__)
+
#    define SQLITE_OS_WIN 1
+
#    define SQLITE_OS_UNIX 0
+
#  else
+
#    define SQLITE_OS_WIN 0
+
#    define SQLITE_OS_UNIX 1
+
#  endif
+
#endif
+
#if SQLITE_OS_OTHER+1>1
+
#  undef SQLITE_OS_KV
+
#  define SQLITE_OS_KV 0
+
#  undef SQLITE_OS_UNIX
+
#  define SQLITE_OS_UNIX 0
+
#  undef SQLITE_OS_WIN
+
#  define SQLITE_OS_WIN 0
+
#endif
+
#if SQLITE_OS_KV+1>1
+
#  undef SQLITE_OS_OTHER
+
#  define SQLITE_OS_OTHER 0
+
#  undef SQLITE_OS_UNIX
+
#  define SQLITE_OS_UNIX 0
+
#  undef SQLITE_OS_WIN
+
#  define SQLITE_OS_WIN 0
+
#  define SQLITE_OMIT_LOAD_EXTENSION 1
+
#  define SQLITE_OMIT_WAL 1
+
#  define SQLITE_OMIT_DEPRECATED 1
+
#  undef SQLITE_TEMP_STORE
+
#  define SQLITE_TEMP_STORE 3  /* Always use memory for temporary storage */
+
#  define SQLITE_DQS 0
+
#  define SQLITE_OMIT_SHARED_CACHE 1
+
#  define SQLITE_OMIT_AUTOINIT 1
+
#endif
+
#if SQLITE_OS_UNIX+1>1
+
#  undef SQLITE_OS_KV
+
#  define SQLITE_OS_KV 0
+
#  undef SQLITE_OS_OTHER
+
#  define SQLITE_OS_OTHER 0
+
#  undef SQLITE_OS_WIN
+
#  define SQLITE_OS_WIN 0
+
#endif
+
#if SQLITE_OS_WIN+1>1
+
#  undef SQLITE_OS_KV
+
#  define SQLITE_OS_KV 0
+
#  undef SQLITE_OS_OTHER
+
#  define SQLITE_OS_OTHER 0
+
#  undef SQLITE_OS_UNIX
+
#  define SQLITE_OS_UNIX 0
+
#endif
+

+

+
#endif /* SQLITE_OS_SETUP_H */
+

+
/************** End of os_setup.h ********************************************/
+
/************** Continuing where we left off in os.h *************************/
+

+
/* If the SET_FULLSYNC macro is not defined above, then make it
+
** a no-op
+
*/
+
#ifndef SET_FULLSYNC
+
# define SET_FULLSYNC(x,y)
+
#endif
+

+
/* Maximum pathname length.  Note: FILENAME_MAX defined by stdio.h
+
*/
+
#ifndef SQLITE_MAX_PATHLEN
+
# define SQLITE_MAX_PATHLEN FILENAME_MAX
+
#endif
+

+
/* Maximum number of symlinks that will be resolved while trying to
+
** expand a filename in xFullPathname() in the VFS.
+
*/
+
#ifndef SQLITE_MAX_SYMLINK
+
# define SQLITE_MAX_SYMLINK 200
+
#endif
+

+
/*
+
** The default size of a disk sector
+
*/
+
#ifndef SQLITE_DEFAULT_SECTOR_SIZE
+
# define SQLITE_DEFAULT_SECTOR_SIZE 4096
+
#endif
+

+
/*
+
** Temporary files are named starting with this prefix followed by 16 random
+
** alphanumeric characters, and no file extension. They are stored in the
+
** OS's standard temporary file directory, and are deleted prior to exit.
+
** If sqlite is being embedded in another program, you may wish to change the
+
** prefix to reflect your program's name, so that if your program exits
+
** prematurely, old temporary files can be easily identified. This can be done
+
** using -DSQLITE_TEMP_FILE_PREFIX=myprefix_ on the compiler command line.
+
**
+
** 2006-10-31:  The default prefix used to be "sqlite_".  But then
+
** Mcafee started using SQLite in their anti-virus product and it
+
** started putting files with the "sqlite" name in the c:/temp folder.
+
** This annoyed many windows users.  Those users would then do a
+
** Google search for "sqlite", find the telephone numbers of the
+
** developers and call to wake them up at night and complain.
+
** For this reason, the default name prefix is changed to be "sqlite"
+
** spelled backwards.  So the temp files are still identified, but
+
** anybody smart enough to figure out the code is also likely smart
+
** enough to know that calling the developer will not help get rid
+
** of the file.
+
*/
+
#ifndef SQLITE_TEMP_FILE_PREFIX
+
# define SQLITE_TEMP_FILE_PREFIX "etilqs_"
+
#endif
+

+
/*
+
** The following values may be passed as the second argument to
+
** sqlite3OsLock(). The various locks exhibit the following semantics:
+
**
+
** SHARED:    Any number of processes may hold a SHARED lock simultaneously.
+
** RESERVED:  A single process may hold a RESERVED lock on a file at
+
**            any time. Other processes may hold and obtain new SHARED locks.
+
** PENDING:   A single process may hold a PENDING lock on a file at
+
**            any one time. Existing SHARED locks may persist, but no new
+
**            SHARED locks may be obtained by other processes.
+
** EXCLUSIVE: An EXCLUSIVE lock precludes all other locks.
+
**
+
** PENDING_LOCK may not be passed directly to sqlite3OsLock(). Instead, a
+
** process that requests an EXCLUSIVE lock may actually obtain a PENDING
+
** lock. This can be upgraded to an EXCLUSIVE lock by a subsequent call to
+
** sqlite3OsLock().
+
*/
+
#define NO_LOCK         0
+
#define SHARED_LOCK     1
+
#define RESERVED_LOCK   2
+
#define PENDING_LOCK    3
+
#define EXCLUSIVE_LOCK  4
+

+
/*
+
** File Locking Notes:  (Mostly about windows but also some info for Unix)
+
**
+
** We cannot use LockFileEx() or UnlockFileEx() on Win95/98/ME because
+
** those functions are not available.  So we use only LockFile() and
+
** UnlockFile().
+
**
+
** LockFile() prevents not just writing but also reading by other processes.
+
** A SHARED_LOCK is obtained by locking a single randomly-chosen
+
** byte out of a specific range of bytes. The lock byte is obtained at
+
** random so two separate readers can probably access the file at the
+
** same time, unless they are unlucky and choose the same lock byte.
+
** An EXCLUSIVE_LOCK is obtained by locking all bytes in the range.
+
** There can only be one writer.  A RESERVED_LOCK is obtained by locking
+
** a single byte of the file that is designated as the reserved lock byte.
+
** A PENDING_LOCK is obtained by locking a designated byte different from
+
** the RESERVED_LOCK byte.
+
**
+
** On WinNT/2K/XP systems, LockFileEx() and UnlockFileEx() are available,
+
** which means we can use reader/writer locks.  When reader/writer locks
+
** are used, the lock is placed on the same range of bytes that is used
+
** for probabilistic locking in Win95/98/ME.  Hence, the locking scheme
+
** will support two or more Win95 readers or two or more WinNT readers.
+
** But a single Win95 reader will lock out all WinNT readers and a single
+
** WinNT reader will lock out all other Win95 readers.
+
**
+
** The following #defines specify the range of bytes used for locking.
+
** SHARED_SIZE is the number of bytes available in the pool from which
+
** a random byte is selected for a shared lock.  The pool of bytes for
+
** shared locks begins at SHARED_FIRST.
+
**
+
** The same locking strategy and
+
** byte ranges are used for Unix.  This leaves open the possibility of having
+
** clients on win95, winNT, and unix all talking to the same shared file
+
** and all locking correctly.  To do so would require that samba (or whatever
+
** tool is being used for file sharing) implements locks correctly between
+
** windows and unix.  I'm guessing that isn't likely to happen, but by
+
** using the same locking range we are at least open to the possibility.
+
**
+
** Locking in windows is manditory.  For this reason, we cannot store
+
** actual data in the bytes used for locking.  The pager never allocates
+
** the pages involved in locking therefore.  SHARED_SIZE is selected so
+
** that all locks will fit on a single page even at the minimum page size.
+
** PENDING_BYTE defines the beginning of the locks.  By default PENDING_BYTE
+
** is set high so that we don't have to allocate an unused page except
+
** for very large databases.  But one should test the page skipping logic
+
** by setting PENDING_BYTE low and running the entire regression suite.
+
**
+
** Changing the value of PENDING_BYTE results in a subtly incompatible
+
** file format.  Depending on how it is changed, you might not notice
+
** the incompatibility right away, even running a full regression test.
+
** The default location of PENDING_BYTE is the first byte past the
+
** 1GB boundary.
+
**
+
*/
+
#ifdef SQLITE_OMIT_WSD
+
# define PENDING_BYTE     (0x40000000)
+
#else
+
# define PENDING_BYTE      sqlite3PendingByte
+
#endif
+
#define RESERVED_BYTE     (PENDING_BYTE+1)
+
#define SHARED_FIRST      (PENDING_BYTE+2)
+
#define SHARED_SIZE       510
+

+
/*
+
** Wrapper around OS specific sqlite3_os_init() function.
+
*/
+
SQLITE_PRIVATE int sqlite3OsInit(void);
+

+
/*
+
** Functions for accessing sqlite3_file methods
+
*/
+
SQLITE_PRIVATE void sqlite3OsClose(sqlite3_file*);
+
SQLITE_PRIVATE int sqlite3OsRead(sqlite3_file*, void*, int amt, i64 offset);
+
SQLITE_PRIVATE int sqlite3OsWrite(sqlite3_file*, const void*, int amt, i64 offset);
+
SQLITE_PRIVATE int sqlite3OsTruncate(sqlite3_file*, i64 size);
+
SQLITE_PRIVATE int sqlite3OsSync(sqlite3_file*, int);
+
SQLITE_PRIVATE int sqlite3OsFileSize(sqlite3_file*, i64 *pSize);
+
SQLITE_PRIVATE int sqlite3OsLock(sqlite3_file*, int);
+
SQLITE_PRIVATE int sqlite3OsUnlock(sqlite3_file*, int);
+
SQLITE_PRIVATE int sqlite3OsCheckReservedLock(sqlite3_file *id, int *pResOut);
+
SQLITE_PRIVATE int sqlite3OsFileControl(sqlite3_file*,int,void*);
+
SQLITE_PRIVATE void sqlite3OsFileControlHint(sqlite3_file*,int,void*);
+
#define SQLITE_FCNTL_DB_UNCHANGED 0xca093fa0
+
SQLITE_PRIVATE int sqlite3OsSectorSize(sqlite3_file *id);
+
SQLITE_PRIVATE int sqlite3OsDeviceCharacteristics(sqlite3_file *id);
+
#ifndef SQLITE_OMIT_WAL
+
SQLITE_PRIVATE int sqlite3OsShmMap(sqlite3_file *,int,int,int,void volatile **);
+
SQLITE_PRIVATE int sqlite3OsShmLock(sqlite3_file *id, int, int, int);
+
SQLITE_PRIVATE void sqlite3OsShmBarrier(sqlite3_file *id);
+
SQLITE_PRIVATE int sqlite3OsShmUnmap(sqlite3_file *id, int);
+
#endif /* SQLITE_OMIT_WAL */
+
SQLITE_PRIVATE int sqlite3OsFetch(sqlite3_file *id, i64, int, void **);
+
SQLITE_PRIVATE int sqlite3OsUnfetch(sqlite3_file *, i64, void *);
+

+

+
/*
+
** Functions for accessing sqlite3_vfs methods
+
*/
+
SQLITE_PRIVATE int sqlite3OsOpen(sqlite3_vfs *, const char *, sqlite3_file*, int, int *);
+
SQLITE_PRIVATE int sqlite3OsDelete(sqlite3_vfs *, const char *, int);
+
SQLITE_PRIVATE int sqlite3OsAccess(sqlite3_vfs *, const char *, int, int *pResOut);
+
SQLITE_PRIVATE int sqlite3OsFullPathname(sqlite3_vfs *, const char *, int, char *);
+
#ifndef SQLITE_OMIT_LOAD_EXTENSION
+
SQLITE_PRIVATE void *sqlite3OsDlOpen(sqlite3_vfs *, const char *);
+
SQLITE_PRIVATE void sqlite3OsDlError(sqlite3_vfs *, int, char *);
+
SQLITE_PRIVATE void (*sqlite3OsDlSym(sqlite3_vfs *, void *, const char *))(void);
+
SQLITE_PRIVATE void sqlite3OsDlClose(sqlite3_vfs *, void *);
+
#endif /* SQLITE_OMIT_LOAD_EXTENSION */
+
SQLITE_PRIVATE int sqlite3OsRandomness(sqlite3_vfs *, int, char *);
+
SQLITE_PRIVATE int sqlite3OsSleep(sqlite3_vfs *, int);
+
SQLITE_PRIVATE int sqlite3OsGetLastError(sqlite3_vfs*);
+
SQLITE_PRIVATE int sqlite3OsCurrentTimeInt64(sqlite3_vfs *, sqlite3_int64*);
+

+
/*
+
** Convenience functions for opening and closing files using
+
** sqlite3_malloc() to obtain space for the file-handle structure.
+
*/
+
SQLITE_PRIVATE int sqlite3OsOpenMalloc(sqlite3_vfs *, const char *, sqlite3_file **, int,int*);
+
SQLITE_PRIVATE void sqlite3OsCloseFree(sqlite3_file *);
+

+
#endif /* _SQLITE_OS_H_ */
+

+
/************** End of os.h **************************************************/
+
/************** Continuing where we left off in sqliteInt.h ******************/
/************** Include pager.h in the middle of sqliteInt.h *****************/
/************** Begin file pager.h *******************************************/
/*
@@ -14715,14 +15141,15 @@ typedef struct Pager Pager;
typedef struct PgHdr DbPage;

/*
-
** Page number PAGER_MJ_PGNO is never used in an SQLite database (it is
+
** Page number PAGER_SJ_PGNO is never used in an SQLite database (it is
** reserved for working around a windows/posix incompatibility). It is
** used in the journal to signify that the remainder of the journal file
** is devoted to storing a super-journal name - there are no more pages to
** roll back. See comments for function writeSuperJournal() in pager.c
** for details.
*/
-
#define PAGER_MJ_PGNO(x) ((Pgno)((PENDING_BYTE/((x)->pageSize))+1))
+
#define PAGER_SJ_PGNO_COMPUTED(x) ((Pgno)((PENDING_BYTE/((x)->pageSize))+1))
+
#define PAGER_SJ_PGNO(x)          ((x)->lckPgno)

/*
** Allowed values for the flags parameter to sqlite3PagerOpen().
@@ -15399,7 +15826,6 @@ struct VdbeOp {
#ifdef SQLITE_ENABLE_CURSOR_HINTS
    Expr *pExpr;           /* Used when p4type is P4_EXPR */
#endif
-
    int (*xAdvance)(BtCursor *, int);
  } p4;
#ifdef SQLITE_ENABLE_EXPLAIN_COMMENTS
  char *zComment;          /* Comment to improve readability */
@@ -15450,21 +15876,19 @@ typedef struct VdbeOpList VdbeOpList;
#define P4_COLLSEQ    (-2)  /* P4 is a pointer to a CollSeq structure */
#define P4_INT32      (-3)  /* P4 is a 32-bit signed integer */
#define P4_SUBPROGRAM (-4)  /* P4 is a pointer to a SubProgram structure */
-
#define P4_ADVANCE    (-5)  /* P4 is a pointer to BtreeNext() or BtreePrev() */
-
#define P4_TABLE      (-6)  /* P4 is a pointer to a Table structure */
+
#define P4_TABLE      (-5)  /* P4 is a pointer to a Table structure */
/* Above do not own any resources.  Must free those below */
-
#define P4_FREE_IF_LE (-7)
-
#define P4_DYNAMIC    (-7)  /* Pointer to memory from sqliteMalloc() */
-
#define P4_FUNCDEF    (-8)  /* P4 is a pointer to a FuncDef structure */
-
#define P4_KEYINFO    (-9)  /* P4 is a pointer to a KeyInfo structure */
-
#define P4_EXPR       (-10) /* P4 is a pointer to an Expr tree */
-
#define P4_MEM        (-11) /* P4 is a pointer to a Mem*    structure */
-
#define P4_VTAB       (-12) /* P4 is a pointer to an sqlite3_vtab structure */
-
#define P4_REAL       (-13) /* P4 is a 64-bit floating point value */
-
#define P4_INT64      (-14) /* P4 is a 64-bit signed integer */
-
#define P4_INTARRAY   (-15) /* P4 is a vector of 32-bit integers */
-
#define P4_FUNCCTX    (-16) /* P4 is a pointer to an sqlite3_context object */
-
#define P4_DYNBLOB    (-17) /* Pointer to memory from sqliteMalloc() */
+
#define P4_FREE_IF_LE (-6)
+
#define P4_DYNAMIC    (-6)  /* Pointer to memory from sqliteMalloc() */
+
#define P4_FUNCDEF    (-7)  /* P4 is a pointer to a FuncDef structure */
+
#define P4_KEYINFO    (-8)  /* P4 is a pointer to a KeyInfo structure */
+
#define P4_EXPR       (-9) /* P4 is a pointer to an Expr tree */
+
#define P4_MEM        (-10) /* P4 is a pointer to a Mem*    structure */
+
#define P4_VTAB       (-11) /* P4 is a pointer to an sqlite3_vtab structure */
+
#define P4_REAL       (-12) /* P4 is a 64-bit floating point value */
+
#define P4_INT64      (-13) /* P4 is a 64-bit signed integer */
+
#define P4_INTARRAY   (-14) /* P4 is a vector of 32-bit integers */
+
#define P4_FUNCCTX    (-15) /* P4 is a pointer to an sqlite3_context object */

/* Error message codes for OP_Halt */
#define P5_ConstraintNotNull 1
@@ -15509,53 +15933,53 @@ typedef struct VdbeOpList VdbeOpList;
#define OP_Savepoint       0
#define OP_AutoCommit      1
#define OP_Transaction     2
-
#define OP_SorterNext      3 /* jump                                       */
-
#define OP_Prev            4 /* jump                                       */
-
#define OP_Next            5 /* jump                                       */
-
#define OP_Checkpoint      6
-
#define OP_JournalMode     7
-
#define OP_Vacuum          8
-
#define OP_VFilter         9 /* jump, synopsis: iplan=r[P3] zplan='P4'     */
-
#define OP_VUpdate        10 /* synopsis: data=r[P3@P2]                    */
-
#define OP_Goto           11 /* jump                                       */
-
#define OP_Gosub          12 /* jump                                       */
-
#define OP_InitCoroutine  13 /* jump                                       */
-
#define OP_Yield          14 /* jump                                       */
-
#define OP_MustBeInt      15 /* jump                                       */
-
#define OP_Jump           16 /* jump                                       */
-
#define OP_Once           17 /* jump                                       */
-
#define OP_If             18 /* jump                                       */
+
#define OP_Checkpoint      3
+
#define OP_JournalMode     4
+
#define OP_Vacuum          5
+
#define OP_VFilter         6 /* jump, synopsis: iplan=r[P3] zplan='P4'     */
+
#define OP_VUpdate         7 /* synopsis: data=r[P3@P2]                    */
+
#define OP_Init            8 /* jump, synopsis: Start at P2                */
+
#define OP_Goto            9 /* jump                                       */
+
#define OP_Gosub          10 /* jump                                       */
+
#define OP_InitCoroutine  11 /* jump                                       */
+
#define OP_Yield          12 /* jump                                       */
+
#define OP_MustBeInt      13 /* jump                                       */
+
#define OP_Jump           14 /* jump                                       */
+
#define OP_Once           15 /* jump                                       */
+
#define OP_If             16 /* jump                                       */
+
#define OP_IfNot          17 /* jump                                       */
+
#define OP_IsType         18 /* jump, synopsis: if typeof(P1.P3) in P5 goto P2 */
#define OP_Not            19 /* same as TK_NOT, synopsis: r[P2]= !r[P1]    */
-
#define OP_IfNot          20 /* jump                                       */
-
#define OP_IsNullOrType   21 /* jump, synopsis: if typeof(r[P1]) IN (P3,5) goto P2 */
-
#define OP_IfNullRow      22 /* jump, synopsis: if P1.nullRow then r[P3]=NULL, goto P2 */
-
#define OP_SeekLT         23 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_SeekLE         24 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_SeekGE         25 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_SeekGT         26 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_IfNotOpen      27 /* jump, synopsis: if( !csr[P1] ) goto P2     */
-
#define OP_IfNoHope       28 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_NoConflict     29 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_NotFound       30 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_Found          31 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_SeekRowid      32 /* jump, synopsis: intkey=r[P3]               */
-
#define OP_NotExists      33 /* jump, synopsis: intkey=r[P3]               */
-
#define OP_Last           34 /* jump                                       */
-
#define OP_IfSmaller      35 /* jump                                       */
-
#define OP_SorterSort     36 /* jump                                       */
-
#define OP_Sort           37 /* jump                                       */
-
#define OP_Rewind         38 /* jump                                       */
-
#define OP_IdxLE          39 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_IdxGT          40 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_IdxLT          41 /* jump, synopsis: key=r[P3@P4]               */
-
#define OP_IdxGE          42 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_IfNullRow      20 /* jump, synopsis: if P1.nullRow then r[P3]=NULL, goto P2 */
+
#define OP_SeekLT         21 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_SeekLE         22 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_SeekGE         23 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_SeekGT         24 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_IfNotOpen      25 /* jump, synopsis: if( !csr[P1] ) goto P2     */
+
#define OP_IfNoHope       26 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_NoConflict     27 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_NotFound       28 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_Found          29 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_SeekRowid      30 /* jump, synopsis: intkey=r[P3]               */
+
#define OP_NotExists      31 /* jump, synopsis: intkey=r[P3]               */
+
#define OP_Last           32 /* jump                                       */
+
#define OP_IfSmaller      33 /* jump                                       */
+
#define OP_SorterSort     34 /* jump                                       */
+
#define OP_Sort           35 /* jump                                       */
+
#define OP_Rewind         36 /* jump                                       */
+
#define OP_SorterNext     37 /* jump                                       */
+
#define OP_Prev           38 /* jump                                       */
+
#define OP_Next           39 /* jump                                       */
+
#define OP_IdxLE          40 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_IdxGT          41 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_IdxLT          42 /* jump, synopsis: key=r[P3@P4]               */
#define OP_Or             43 /* same as TK_OR, synopsis: r[P3]=(r[P1] || r[P2]) */
#define OP_And            44 /* same as TK_AND, synopsis: r[P3]=(r[P1] && r[P2]) */
-
#define OP_RowSetRead     45 /* jump, synopsis: r[P3]=rowset(P1)           */
-
#define OP_RowSetTest     46 /* jump, synopsis: if r[P3] in rowset(P1) goto P2 */
-
#define OP_Program        47 /* jump                                       */
-
#define OP_FkIfZero       48 /* jump, synopsis: if fkctr[P1]==0 goto P2    */
-
#define OP_IfPos          49 /* jump, synopsis: if r[P1]>0 then r[P1]-=P3, goto P2 */
+
#define OP_IdxGE          45 /* jump, synopsis: key=r[P3@P4]               */
+
#define OP_RowSetRead     46 /* jump, synopsis: r[P3]=rowset(P1)           */
+
#define OP_RowSetTest     47 /* jump, synopsis: if r[P3] in rowset(P1) goto P2 */
+
#define OP_Program        48 /* jump                                       */
+
#define OP_FkIfZero       49 /* jump, synopsis: if fkctr[P1]==0 goto P2    */
#define OP_IsNull         50 /* jump, same as TK_ISNULL, synopsis: if r[P1]==NULL goto P2 */
#define OP_NotNull        51 /* jump, same as TK_NOTNULL, synopsis: if r[P1]!=NULL goto P2 */
#define OP_Ne             52 /* jump, same as TK_NE, synopsis: IF r[P3]!=r[P1] */
@@ -15565,12 +15989,12 @@ typedef struct VdbeOpList VdbeOpList;
#define OP_Lt             56 /* jump, same as TK_LT, synopsis: IF r[P3]<r[P1] */
#define OP_Ge             57 /* jump, same as TK_GE, synopsis: IF r[P3]>=r[P1] */
#define OP_ElseEq         58 /* jump, same as TK_ESCAPE                    */
-
#define OP_IfNotZero      59 /* jump, synopsis: if r[P1]!=0 then r[P1]--, goto P2 */
-
#define OP_DecrJumpZero   60 /* jump, synopsis: if (--r[P1])==0 goto P2    */
-
#define OP_IncrVacuum     61 /* jump                                       */
-
#define OP_VNext          62 /* jump                                       */
-
#define OP_Filter         63 /* jump, synopsis: if key(P3@P4) not in filter(P1) goto P2 */
-
#define OP_Init           64 /* jump, synopsis: Start at P2                */
+
#define OP_IfPos          59 /* jump, synopsis: if r[P1]>0 then r[P1]-=P3, goto P2 */
+
#define OP_IfNotZero      60 /* jump, synopsis: if r[P1]!=0 then r[P1]--, goto P2 */
+
#define OP_DecrJumpZero   61 /* jump, synopsis: if (--r[P1])==0 goto P2    */
+
#define OP_IncrVacuum     62 /* jump                                       */
+
#define OP_VNext          63 /* jump                                       */
+
#define OP_Filter         64 /* jump, synopsis: if key(P3@P4) not in filter(P1) goto P2 */
#define OP_PureFunc       65 /* synopsis: r[P3]=func(r[P2@NP])             */
#define OP_Function       66 /* synopsis: r[P3]=func(r[P2@NP])             */
#define OP_Return         67
@@ -15580,34 +16004,34 @@ typedef struct VdbeOpList VdbeOpList;
#define OP_Integer        71 /* synopsis: r[P2]=P1                         */
#define OP_Int64          72 /* synopsis: r[P2]=P4                         */
#define OP_String         73 /* synopsis: r[P2]='P4' (len=P1)              */
-
#define OP_Null           74 /* synopsis: r[P2..P3]=NULL                   */
-
#define OP_SoftNull       75 /* synopsis: r[P1]=NULL                       */
-
#define OP_Blob           76 /* synopsis: r[P2]=P4 (len=P1)                */
-
#define OP_Variable       77 /* synopsis: r[P2]=parameter(P1,P4)           */
-
#define OP_Move           78 /* synopsis: r[P2@P3]=r[P1@P3]                */
-
#define OP_Copy           79 /* synopsis: r[P2@P3+1]=r[P1@P3+1]            */
-
#define OP_SCopy          80 /* synopsis: r[P2]=r[P1]                      */
-
#define OP_IntCopy        81 /* synopsis: r[P2]=r[P1]                      */
-
#define OP_FkCheck        82
-
#define OP_ResultRow      83 /* synopsis: output=r[P1@P2]                  */
-
#define OP_CollSeq        84
-
#define OP_AddImm         85 /* synopsis: r[P1]=r[P1]+P2                   */
-
#define OP_RealAffinity   86
-
#define OP_Cast           87 /* synopsis: affinity(r[P1])                  */
-
#define OP_Permutation    88
-
#define OP_Compare        89 /* synopsis: r[P1@P3] <-> r[P2@P3]            */
-
#define OP_IsTrue         90 /* synopsis: r[P2] = coalesce(r[P1]==TRUE,P3) ^ P4 */
-
#define OP_ZeroOrNull     91 /* synopsis: r[P2] = 0 OR NULL                */
-
#define OP_Offset         92 /* synopsis: r[P3] = sqlite_offset(P1)        */
-
#define OP_Column         93 /* synopsis: r[P3]=PX                         */
-
#define OP_TypeCheck      94 /* synopsis: typecheck(r[P1@P2])              */
-
#define OP_Affinity       95 /* synopsis: affinity(r[P1@P2])               */
-
#define OP_MakeRecord     96 /* synopsis: r[P3]=mkrec(r[P1@P2])            */
-
#define OP_Count          97 /* synopsis: r[P2]=count()                    */
-
#define OP_ReadCookie     98
-
#define OP_SetCookie      99
-
#define OP_ReopenIdx     100 /* synopsis: root=P2 iDb=P3                   */
-
#define OP_OpenRead      101 /* synopsis: root=P2 iDb=P3                   */
+
#define OP_BeginSubrtn    74 /* synopsis: r[P2]=NULL                       */
+
#define OP_Null           75 /* synopsis: r[P2..P3]=NULL                   */
+
#define OP_SoftNull       76 /* synopsis: r[P1]=NULL                       */
+
#define OP_Blob           77 /* synopsis: r[P2]=P4 (len=P1)                */
+
#define OP_Variable       78 /* synopsis: r[P2]=parameter(P1,P4)           */
+
#define OP_Move           79 /* synopsis: r[P2@P3]=r[P1@P3]                */
+
#define OP_Copy           80 /* synopsis: r[P2@P3+1]=r[P1@P3+1]            */
+
#define OP_SCopy          81 /* synopsis: r[P2]=r[P1]                      */
+
#define OP_IntCopy        82 /* synopsis: r[P2]=r[P1]                      */
+
#define OP_FkCheck        83
+
#define OP_ResultRow      84 /* synopsis: output=r[P1@P2]                  */
+
#define OP_CollSeq        85
+
#define OP_AddImm         86 /* synopsis: r[P1]=r[P1]+P2                   */
+
#define OP_RealAffinity   87
+
#define OP_Cast           88 /* synopsis: affinity(r[P1])                  */
+
#define OP_Permutation    89
+
#define OP_Compare        90 /* synopsis: r[P1@P3] <-> r[P2@P3]            */
+
#define OP_IsTrue         91 /* synopsis: r[P2] = coalesce(r[P1]==TRUE,P3) ^ P4 */
+
#define OP_ZeroOrNull     92 /* synopsis: r[P2] = 0 OR NULL                */
+
#define OP_Offset         93 /* synopsis: r[P3] = sqlite_offset(P1)        */
+
#define OP_Column         94 /* synopsis: r[P3]=PX cursor P1 column P2     */
+
#define OP_TypeCheck      95 /* synopsis: typecheck(r[P1@P2])              */
+
#define OP_Affinity       96 /* synopsis: affinity(r[P1@P2])               */
+
#define OP_MakeRecord     97 /* synopsis: r[P3]=mkrec(r[P1@P2])            */
+
#define OP_Count          98 /* synopsis: r[P2]=count()                    */
+
#define OP_ReadCookie     99
+
#define OP_SetCookie     100
+
#define OP_ReopenIdx     101 /* synopsis: root=P2 iDb=P3                   */
#define OP_BitAnd        102 /* same as TK_BITAND, synopsis: r[P3]=r[P1]&r[P2] */
#define OP_BitOr         103 /* same as TK_BITOR, synopsis: r[P3]=r[P1]|r[P2] */
#define OP_ShiftLeft     104 /* same as TK_LSHIFT, synopsis: r[P3]=r[P2]<<r[P1] */
@@ -15618,79 +16042,81 @@ typedef struct VdbeOpList VdbeOpList;
#define OP_Divide        109 /* same as TK_SLASH, synopsis: r[P3]=r[P2]/r[P1] */
#define OP_Remainder     110 /* same as TK_REM, synopsis: r[P3]=r[P2]%r[P1] */
#define OP_Concat        111 /* same as TK_CONCAT, synopsis: r[P3]=r[P2]+r[P1] */
-
#define OP_OpenWrite     112 /* synopsis: root=P2 iDb=P3                   */
-
#define OP_OpenDup       113
+
#define OP_OpenRead      112 /* synopsis: root=P2 iDb=P3                   */
+
#define OP_OpenWrite     113 /* synopsis: root=P2 iDb=P3                   */
#define OP_BitNot        114 /* same as TK_BITNOT, synopsis: r[P2]= ~r[P1] */
-
#define OP_OpenAutoindex 115 /* synopsis: nColumn=P2                       */
-
#define OP_OpenEphemeral 116 /* synopsis: nColumn=P2                       */
+
#define OP_OpenDup       115
+
#define OP_OpenAutoindex 116 /* synopsis: nColumn=P2                       */
#define OP_String8       117 /* same as TK_STRING, synopsis: r[P2]='P4'    */
-
#define OP_SorterOpen    118
-
#define OP_SequenceTest  119 /* synopsis: if( cursor[P1].ctr++ ) pc = P2   */
-
#define OP_OpenPseudo    120 /* synopsis: P3 columns in r[P2]              */
-
#define OP_Close         121
-
#define OP_ColumnsUsed   122
-
#define OP_SeekScan      123 /* synopsis: Scan-ahead up to P1 rows         */
-
#define OP_SeekHit       124 /* synopsis: set P2<=seekHit<=P3              */
-
#define OP_Sequence      125 /* synopsis: r[P2]=cursor[P1].ctr++           */
-
#define OP_NewRowid      126 /* synopsis: r[P2]=rowid                      */
-
#define OP_Insert        127 /* synopsis: intkey=r[P3] data=r[P2]          */
-
#define OP_RowCell       128
-
#define OP_Delete        129
-
#define OP_ResetCount    130
-
#define OP_SorterCompare 131 /* synopsis: if key(P1)!=trim(r[P3],P4) goto P2 */
-
#define OP_SorterData    132 /* synopsis: r[P2]=data                       */
-
#define OP_RowData       133 /* synopsis: r[P2]=data                       */
-
#define OP_Rowid         134 /* synopsis: r[P2]=rowid                      */
-
#define OP_NullRow       135
-
#define OP_SeekEnd       136
-
#define OP_IdxInsert     137 /* synopsis: key=r[P2]                        */
-
#define OP_SorterInsert  138 /* synopsis: key=r[P2]                        */
-
#define OP_IdxDelete     139 /* synopsis: key=r[P2@P3]                     */
-
#define OP_DeferredSeek  140 /* synopsis: Move P3 to P1.rowid if needed    */
-
#define OP_IdxRowid      141 /* synopsis: r[P2]=rowid                      */
-
#define OP_FinishSeek    142
-
#define OP_Destroy       143
-
#define OP_Clear         144
-
#define OP_ResetSorter   145
-
#define OP_CreateBtree   146 /* synopsis: r[P2]=root iDb=P1 flags=P3       */
-
#define OP_SqlExec       147
-
#define OP_ParseSchema   148
-
#define OP_LoadAnalysis  149
-
#define OP_DropTable     150
-
#define OP_DropIndex     151
-
#define OP_DropTrigger   152
+
#define OP_OpenEphemeral 118 /* synopsis: nColumn=P2                       */
+
#define OP_SorterOpen    119
+
#define OP_SequenceTest  120 /* synopsis: if( cursor[P1].ctr++ ) pc = P2   */
+
#define OP_OpenPseudo    121 /* synopsis: P3 columns in r[P2]              */
+
#define OP_Close         122
+
#define OP_ColumnsUsed   123
+
#define OP_SeekScan      124 /* synopsis: Scan-ahead up to P1 rows         */
+
#define OP_SeekHit       125 /* synopsis: set P2<=seekHit<=P3              */
+
#define OP_Sequence      126 /* synopsis: r[P2]=cursor[P1].ctr++           */
+
#define OP_NewRowid      127 /* synopsis: r[P2]=rowid                      */
+
#define OP_Insert        128 /* synopsis: intkey=r[P3] data=r[P2]          */
+
#define OP_RowCell       129
+
#define OP_Delete        130
+
#define OP_ResetCount    131
+
#define OP_SorterCompare 132 /* synopsis: if key(P1)!=trim(r[P3],P4) goto P2 */
+
#define OP_SorterData    133 /* synopsis: r[P2]=data                       */
+
#define OP_RowData       134 /* synopsis: r[P2]=data                       */
+
#define OP_Rowid         135 /* synopsis: r[P2]=PX rowid of P1             */
+
#define OP_NullRow       136
+
#define OP_SeekEnd       137
+
#define OP_IdxInsert     138 /* synopsis: key=r[P2]                        */
+
#define OP_SorterInsert  139 /* synopsis: key=r[P2]                        */
+
#define OP_IdxDelete     140 /* synopsis: key=r[P2@P3]                     */
+
#define OP_DeferredSeek  141 /* synopsis: Move P3 to P1.rowid if needed    */
+
#define OP_IdxRowid      142 /* synopsis: r[P2]=rowid                      */
+
#define OP_FinishSeek    143
+
#define OP_Destroy       144
+
#define OP_Clear         145
+
#define OP_ResetSorter   146
+
#define OP_CreateBtree   147 /* synopsis: r[P2]=root iDb=P1 flags=P3       */
+
#define OP_SqlExec       148
+
#define OP_ParseSchema   149
+
#define OP_LoadAnalysis  150
+
#define OP_DropTable     151
+
#define OP_DropIndex     152
#define OP_Real          153 /* same as TK_FLOAT, synopsis: r[P2]=P4       */
-
#define OP_IntegrityCk   154
-
#define OP_RowSetAdd     155 /* synopsis: rowset(P1)=r[P2]                 */
-
#define OP_Param         156
-
#define OP_FkCounter     157 /* synopsis: fkctr[P1]+=P2                    */
-
#define OP_MemMax        158 /* synopsis: r[P1]=max(r[P1],r[P2])           */
-
#define OP_OffsetLimit   159 /* synopsis: if r[P1]>0 then r[P2]=r[P1]+max(0,r[P3]) else r[P2]=(-1) */
-
#define OP_AggInverse    160 /* synopsis: accum=r[P3] inverse(r[P2@P5])    */
-
#define OP_AggStep       161 /* synopsis: accum=r[P3] step(r[P2@P5])       */
-
#define OP_AggStep1      162 /* synopsis: accum=r[P3] step(r[P2@P5])       */
-
#define OP_AggValue      163 /* synopsis: r[P3]=value N=P2                 */
-
#define OP_AggFinal      164 /* synopsis: accum=r[P1] N=P2                 */
-
#define OP_Expire        165
-
#define OP_CursorLock    166
-
#define OP_CursorUnlock  167
-
#define OP_TableLock     168 /* synopsis: iDb=P1 root=P2 write=P3          */
-
#define OP_VBegin        169
-
#define OP_VCreate       170
-
#define OP_VDestroy      171
-
#define OP_VOpen         172
-
#define OP_VInitIn       173 /* synopsis: r[P2]=ValueList(P1,P3)           */
-
#define OP_VColumn       174 /* synopsis: r[P3]=vcolumn(P2)                */
-
#define OP_VRename       175
-
#define OP_Pagecount     176
-
#define OP_MaxPgcnt      177
-
#define OP_FilterAdd     178 /* synopsis: filter(P1) += key(P3@P4)         */
-
#define OP_Trace         179
-
#define OP_CursorHint    180
-
#define OP_ReleaseReg    181 /* synopsis: release r[P1@P2] mask P3         */
-
#define OP_Noop          182
-
#define OP_Explain       183
-
#define OP_Abortable     184
+
#define OP_DropTrigger   154
+
#define OP_IntegrityCk   155
+
#define OP_RowSetAdd     156 /* synopsis: rowset(P1)=r[P2]                 */
+
#define OP_Param         157
+
#define OP_FkCounter     158 /* synopsis: fkctr[P1]+=P2                    */
+
#define OP_MemMax        159 /* synopsis: r[P1]=max(r[P1],r[P2])           */
+
#define OP_OffsetLimit   160 /* synopsis: if r[P1]>0 then r[P2]=r[P1]+max(0,r[P3]) else r[P2]=(-1) */
+
#define OP_AggInverse    161 /* synopsis: accum=r[P3] inverse(r[P2@P5])    */
+
#define OP_AggStep       162 /* synopsis: accum=r[P3] step(r[P2@P5])       */
+
#define OP_AggStep1      163 /* synopsis: accum=r[P3] step(r[P2@P5])       */
+
#define OP_AggValue      164 /* synopsis: r[P3]=value N=P2                 */
+
#define OP_AggFinal      165 /* synopsis: accum=r[P1] N=P2                 */
+
#define OP_Expire        166
+
#define OP_CursorLock    167
+
#define OP_CursorUnlock  168
+
#define OP_TableLock     169 /* synopsis: iDb=P1 root=P2 write=P3          */
+
#define OP_VBegin        170
+
#define OP_VCreate       171
+
#define OP_VDestroy      172
+
#define OP_VOpen         173
+
#define OP_VInitIn       174 /* synopsis: r[P2]=ValueList(P1,P3)           */
+
#define OP_VColumn       175 /* synopsis: r[P3]=vcolumn(P2)                */
+
#define OP_VRename       176
+
#define OP_Pagecount     177
+
#define OP_MaxPgcnt      178
+
#define OP_ClrSubtype    179 /* synopsis: r[P1].subtype = 0                */
+
#define OP_FilterAdd     180 /* synopsis: filter(P1) += key(P3@P4)         */
+
#define OP_Trace         181
+
#define OP_CursorHint    182
+
#define OP_ReleaseReg    183 /* synopsis: release r[P1@P2] mask P3         */
+
#define OP_Noop          184
+
#define OP_Explain       185
+
#define OP_Abortable     186

/* Properties such as "out2" or "jump" that are specified in
** comments following the "case" for each opcode in the vdbe.c
@@ -15703,30 +16129,30 @@ typedef struct VdbeOpList VdbeOpList;
#define OPFLG_OUT2        0x10  /* out2:  P2 is an output */
#define OPFLG_OUT3        0x20  /* out3:  P3 is an output */
#define OPFLG_INITIALIZER {\
-
/*   0 */ 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x00, 0x10,\
-
/*   8 */ 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x03, 0x03,\
-
/*  16 */ 0x01, 0x01, 0x03, 0x12, 0x03, 0x03, 0x01, 0x09,\
-
/*  24 */ 0x09, 0x09, 0x09, 0x01, 0x09, 0x09, 0x09, 0x09,\
-
/*  32 */ 0x09, 0x09, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,\
-
/*  40 */ 0x01, 0x01, 0x01, 0x26, 0x26, 0x23, 0x0b, 0x01,\
-
/*  48 */ 0x01, 0x03, 0x03, 0x03, 0x0b, 0x0b, 0x0b, 0x0b,\
-
/*  56 */ 0x0b, 0x0b, 0x01, 0x03, 0x03, 0x01, 0x01, 0x01,\
+
/*   0 */ 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x01, 0x00,\
+
/*   8 */ 0x01, 0x01, 0x01, 0x01, 0x03, 0x03, 0x01, 0x01,\
+
/*  16 */ 0x03, 0x03, 0x01, 0x12, 0x01, 0x09, 0x09, 0x09,\
+
/*  24 */ 0x09, 0x01, 0x09, 0x09, 0x09, 0x09, 0x09, 0x09,\
+
/*  32 */ 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,\
+
/*  40 */ 0x01, 0x01, 0x01, 0x26, 0x26, 0x01, 0x23, 0x0b,\
+
/*  48 */ 0x01, 0x01, 0x03, 0x03, 0x0b, 0x0b, 0x0b, 0x0b,\
+
/*  56 */ 0x0b, 0x0b, 0x01, 0x03, 0x03, 0x03, 0x01, 0x01,\
/*  64 */ 0x01, 0x00, 0x00, 0x02, 0x02, 0x08, 0x00, 0x10,\
-
/*  72 */ 0x10, 0x10, 0x10, 0x00, 0x10, 0x10, 0x00, 0x00,\
-
/*  80 */ 0x10, 0x10, 0x00, 0x00, 0x00, 0x02, 0x02, 0x02,\
-
/*  88 */ 0x00, 0x00, 0x12, 0x1e, 0x20, 0x00, 0x00, 0x00,\
-
/*  96 */ 0x00, 0x10, 0x10, 0x00, 0x00, 0x00, 0x26, 0x26,\
+
/*  72 */ 0x10, 0x10, 0x00, 0x10, 0x00, 0x10, 0x10, 0x00,\
+
/*  80 */ 0x00, 0x10, 0x10, 0x00, 0x00, 0x00, 0x02, 0x02,\
+
/*  88 */ 0x02, 0x00, 0x00, 0x12, 0x1e, 0x20, 0x00, 0x00,\
+
/*  96 */ 0x00, 0x00, 0x10, 0x10, 0x00, 0x00, 0x26, 0x26,\
/* 104 */ 0x26, 0x26, 0x26, 0x26, 0x26, 0x26, 0x26, 0x26,\
/* 112 */ 0x00, 0x00, 0x12, 0x00, 0x00, 0x10, 0x00, 0x00,\
-
/* 120 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x10, 0x00,\
-
/* 128 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00,\
-
/* 136 */ 0x00, 0x04, 0x04, 0x00, 0x00, 0x10, 0x00, 0x10,\
-
/* 144 */ 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00,\
-
/* 152 */ 0x00, 0x10, 0x00, 0x06, 0x10, 0x00, 0x04, 0x1a,\
-
/* 160 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\
-
/* 168 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00,\
-
/* 176 */ 0x10, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\
-
/* 184 */ 0x00,}
+
/* 120 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x10,\
+
/* 128 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10,\
+
/* 136 */ 0x00, 0x00, 0x04, 0x04, 0x00, 0x00, 0x10, 0x00,\
+
/* 144 */ 0x10, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00,\
+
/* 152 */ 0x00, 0x10, 0x00, 0x00, 0x06, 0x10, 0x00, 0x04,\
+
/* 160 */ 0x1a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\
+
/* 168 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00,\
+
/* 176 */ 0x00, 0x10, 0x10, 0x02, 0x00, 0x00, 0x00, 0x00,\
+
/* 184 */ 0x00, 0x00, 0x00,}

/* The resolve3P2Values() routine is able to run faster if it knows
** the value of the largest JUMP opcode.  The smaller the maximum
@@ -15772,8 +16198,10 @@ SQLITE_PRIVATE void sqlite3VdbeVerifyNoResultRow(Vdbe *p);
#endif
#if defined(SQLITE_DEBUG)
SQLITE_PRIVATE   void sqlite3VdbeVerifyAbortable(Vdbe *p, int);
+
SQLITE_PRIVATE   void sqlite3VdbeNoJumpsOutsideSubrtn(Vdbe*,int,int,int);
#else
# define sqlite3VdbeVerifyAbortable(A,B)
+
# define sqlite3VdbeNoJumpsOutsideSubrtn(A,B,C,D)
#endif
SQLITE_PRIVATE VdbeOp *sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp,int iLineno);
#ifndef SQLITE_OMIT_EXPLAIN
@@ -15800,6 +16228,7 @@ SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1);
SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2);
SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe*, int addr, int P3);
SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe*, u16 P5);
+
SQLITE_PRIVATE void sqlite3VdbeTypeofColumn(Vdbe*, int);
SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe*, int addr);
SQLITE_PRIVATE void sqlite3VdbeJumpHereOrPopInst(Vdbe*, int addr);
SQLITE_PRIVATE int sqlite3VdbeChangeToNoop(Vdbe*, int addr);
@@ -15814,11 +16243,11 @@ SQLITE_PRIVATE void sqlite3VdbeAppendP4(Vdbe*, void *pP4, int p4type);
SQLITE_PRIVATE void sqlite3VdbeSetP4KeyInfo(Parse*, Index*);
SQLITE_PRIVATE void sqlite3VdbeUsesBtree(Vdbe*, int);
SQLITE_PRIVATE VdbeOp *sqlite3VdbeGetOp(Vdbe*, int);
+
SQLITE_PRIVATE VdbeOp *sqlite3VdbeGetLastOp(Vdbe*);
SQLITE_PRIVATE int sqlite3VdbeMakeLabel(Parse*);
SQLITE_PRIVATE void sqlite3VdbeRunOnlyOnce(Vdbe*);
SQLITE_PRIVATE void sqlite3VdbeReusable(Vdbe*);
SQLITE_PRIVATE void sqlite3VdbeDelete(Vdbe*);
-
SQLITE_PRIVATE void sqlite3VdbeClearObject(sqlite3*,Vdbe*);
SQLITE_PRIVATE void sqlite3VdbeMakeReady(Vdbe*,Parse*);
SQLITE_PRIVATE int sqlite3VdbeFinalize(Vdbe*);
SQLITE_PRIVATE void sqlite3VdbeResolveLabel(Vdbe*, int);
@@ -16163,290 +16592,6 @@ SQLITE_PRIVATE int sqlite3PCacheIsDirty(PCache *pCache);

/************** End of pcache.h **********************************************/
/************** Continuing where we left off in sqliteInt.h ******************/
-
/************** Include os.h in the middle of sqliteInt.h ********************/
-
/************** Begin file os.h **********************************************/
-
/*
-
** 2001 September 16
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** This header file (together with is companion C source-code file
-
** "os.c") attempt to abstract the underlying operating system so that
-
** the SQLite library will work on both POSIX and windows systems.
-
**
-
** This header file is #include-ed by sqliteInt.h and thus ends up
-
** being included by every source file.
-
*/
-
#ifndef _SQLITE_OS_H_
-
#define _SQLITE_OS_H_
-

-
/*
-
** Attempt to automatically detect the operating system and setup the
-
** necessary pre-processor macros for it.
-
*/
-
/************** Include os_setup.h in the middle of os.h *********************/
-
/************** Begin file os_setup.h ****************************************/
-
/*
-
** 2013 November 25
-
**
-
** The author disclaims copyright to this source code.  In place of
-
** a legal notice, here is a blessing:
-
**
-
**    May you do good and not evil.
-
**    May you find forgiveness for yourself and forgive others.
-
**    May you share freely, never taking more than you give.
-
**
-
******************************************************************************
-
**
-
** This file contains pre-processor directives related to operating system
-
** detection and/or setup.
-
*/
-
#ifndef SQLITE_OS_SETUP_H
-
#define SQLITE_OS_SETUP_H
-

-
/*
-
** Figure out if we are dealing with Unix, Windows, or some other operating
-
** system.
-
**
-
** After the following block of preprocess macros, all of SQLITE_OS_UNIX,
-
** SQLITE_OS_WIN, and SQLITE_OS_OTHER will defined to either 1 or 0.  One of
-
** the three will be 1.  The other two will be 0.
-
*/
-
#if defined(SQLITE_OS_OTHER)
-
#  if SQLITE_OS_OTHER==1
-
#    undef SQLITE_OS_UNIX
-
#    define SQLITE_OS_UNIX 0
-
#    undef SQLITE_OS_WIN
-
#    define SQLITE_OS_WIN 0
-
#  else
-
#    undef SQLITE_OS_OTHER
-
#  endif
-
#endif
-
#if !defined(SQLITE_OS_UNIX) && !defined(SQLITE_OS_OTHER)
-
#  define SQLITE_OS_OTHER 0
-
#  ifndef SQLITE_OS_WIN
-
#    if defined(_WIN32) || defined(WIN32) || defined(__CYGWIN__) || \
-
        defined(__MINGW32__) || defined(__BORLANDC__)
-
#      define SQLITE_OS_WIN 1
-
#      define SQLITE_OS_UNIX 0
-
#    else
-
#      define SQLITE_OS_WIN 0
-
#      define SQLITE_OS_UNIX 1
-
#    endif
-
#  else
-
#    define SQLITE_OS_UNIX 0
-
#  endif
-
#else
-
#  ifndef SQLITE_OS_WIN
-
#    define SQLITE_OS_WIN 0
-
#  endif
-
#endif
-

-
#endif /* SQLITE_OS_SETUP_H */
-

-
/************** End of os_setup.h ********************************************/
-
/************** Continuing where we left off in os.h *************************/
-

-
/* If the SET_FULLSYNC macro is not defined above, then make it
-
** a no-op
-
*/
-
#ifndef SET_FULLSYNC
-
# define SET_FULLSYNC(x,y)
-
#endif
-

-
/* Maximum pathname length.  Note: FILENAME_MAX defined by stdio.h
-
*/
-
#ifndef SQLITE_MAX_PATHLEN
-
# define SQLITE_MAX_PATHLEN FILENAME_MAX
-
#endif
-

-
/*
-
** The default size of a disk sector
-
*/
-
#ifndef SQLITE_DEFAULT_SECTOR_SIZE
-
# define SQLITE_DEFAULT_SECTOR_SIZE 4096
-
#endif
-

-
/*
-
** Temporary files are named starting with this prefix followed by 16 random
-
** alphanumeric characters, and no file extension. They are stored in the
-
** OS's standard temporary file directory, and are deleted prior to exit.
-
** If sqlite is being embedded in another program, you may wish to change the
-
** prefix to reflect your program's name, so that if your program exits
-
** prematurely, old temporary files can be easily identified. This can be done
-
** using -DSQLITE_TEMP_FILE_PREFIX=myprefix_ on the compiler command line.
-
**
-
** 2006-10-31:  The default prefix used to be "sqlite_".  But then
-
** Mcafee started using SQLite in their anti-virus product and it
-
** started putting files with the "sqlite" name in the c:/temp folder.
-
** This annoyed many windows users.  Those users would then do a
-
** Google search for "sqlite", find the telephone numbers of the
-
** developers and call to wake them up at night and complain.
-
** For this reason, the default name prefix is changed to be "sqlite"
-
** spelled backwards.  So the temp files are still identified, but
-
** anybody smart enough to figure out the code is also likely smart
-
** enough to know that calling the developer will not help get rid
-
** of the file.
-
*/
-
#ifndef SQLITE_TEMP_FILE_PREFIX
-
# define SQLITE_TEMP_FILE_PREFIX "etilqs_"
-
#endif
-

-
/*
-
** The following values may be passed as the second argument to
-
** sqlite3OsLock(). The various locks exhibit the following semantics:
-
**
-
** SHARED:    Any number of processes may hold a SHARED lock simultaneously.
-
** RESERVED:  A single process may hold a RESERVED lock on a file at
-
**            any time. Other processes may hold and obtain new SHARED locks.
-
** PENDING:   A single process may hold a PENDING lock on a file at
-
**            any one time. Existing SHARED locks may persist, but no new
-
**            SHARED locks may be obtained by other processes.
-
** EXCLUSIVE: An EXCLUSIVE lock precludes all other locks.
-
**
-
** PENDING_LOCK may not be passed directly to sqlite3OsLock(). Instead, a
-
** process that requests an EXCLUSIVE lock may actually obtain a PENDING
-
** lock. This can be upgraded to an EXCLUSIVE lock by a subsequent call to
-
** sqlite3OsLock().
-
*/
-
#define NO_LOCK         0
-
#define SHARED_LOCK     1
-
#define RESERVED_LOCK   2
-
#define PENDING_LOCK    3
-
#define EXCLUSIVE_LOCK  4
-

-
/*
-
** File Locking Notes:  (Mostly about windows but also some info for Unix)
-
**
-
** We cannot use LockFileEx() or UnlockFileEx() on Win95/98/ME because
-
** those functions are not available.  So we use only LockFile() and
-
** UnlockFile().
-
**
-
** LockFile() prevents not just writing but also reading by other processes.
-
** A SHARED_LOCK is obtained by locking a single randomly-chosen
-
** byte out of a specific range of bytes. The lock byte is obtained at
-
** random so two separate readers can probably access the file at the
-
** same time, unless they are unlucky and choose the same lock byte.
-
** An EXCLUSIVE_LOCK is obtained by locking all bytes in the range.
-
** There can only be one writer.  A RESERVED_LOCK is obtained by locking
-
** a single byte of the file that is designated as the reserved lock byte.
-
** A PENDING_LOCK is obtained by locking a designated byte different from
-
** the RESERVED_LOCK byte.
-
**
-
** On WinNT/2K/XP systems, LockFileEx() and UnlockFileEx() are available,
-
** which means we can use reader/writer locks.  When reader/writer locks
-
** are used, the lock is placed on the same range of bytes that is used
-
** for probabilistic locking in Win95/98/ME.  Hence, the locking scheme
-
** will support two or more Win95 readers or two or more WinNT readers.
-
** But a single Win95 reader will lock out all WinNT readers and a single
-
** WinNT reader will lock out all other Win95 readers.
-
**
-
** The following #defines specify the range of bytes used for locking.
-
** SHARED_SIZE is the number of bytes available in the pool from which
-
** a random byte is selected for a shared lock.  The pool of bytes for
-
** shared locks begins at SHARED_FIRST.
-
**
-
** The same locking strategy and
-
** byte ranges are used for Unix.  This leaves open the possibility of having
-
** clients on win95, winNT, and unix all talking to the same shared file
-
** and all locking correctly.  To do so would require that samba (or whatever
-
** tool is being used for file sharing) implements locks correctly between
-
** windows and unix.  I'm guessing that isn't likely to happen, but by
-
** using the same locking range we are at least open to the possibility.
-
**
-
** Locking in windows is manditory.  For this reason, we cannot store
-
** actual data in the bytes used for locking.  The pager never allocates
-
** the pages involved in locking therefore.  SHARED_SIZE is selected so
-
** that all locks will fit on a single page even at the minimum page size.
-
** PENDING_BYTE defines the beginning of the locks.  By default PENDING_BYTE
-
** is set high so that we don't have to allocate an unused page except
-
** for very large databases.  But one should test the page skipping logic
-
** by setting PENDING_BYTE low and running the entire regression suite.
-
**
-
** Changing the value of PENDING_BYTE results in a subtly incompatible
-
** file format.  Depending on how it is changed, you might not notice
-
** the incompatibility right away, even running a full regression test.
-
** The default location of PENDING_BYTE is the first byte past the
-
** 1GB boundary.
-
**
-
*/
-
#ifdef SQLITE_OMIT_WSD
-
# define PENDING_BYTE     (0x40000000)
-
#else
-
# define PENDING_BYTE      sqlite3PendingByte
-
#endif
-
#define RESERVED_BYTE     (PENDING_BYTE+1)
-
#define SHARED_FIRST      (PENDING_BYTE+2)
-
#define SHARED_SIZE       510
-

-
/*
-
** Wrapper around OS specific sqlite3_os_init() function.
-
*/
-
SQLITE_PRIVATE int sqlite3OsInit(void);
-

-
/*
-
** Functions for accessing sqlite3_file methods
-
*/
-
SQLITE_PRIVATE void sqlite3OsClose(sqlite3_file*);
-
SQLITE_PRIVATE int sqlite3OsRead(sqlite3_file*, void*, int amt, i64 offset);
-
SQLITE_PRIVATE int sqlite3OsWrite(sqlite3_file*, const void*, int amt, i64 offset);
-
SQLITE_PRIVATE int sqlite3OsTruncate(sqlite3_file*, i64 size);
-
SQLITE_PRIVATE int sqlite3OsSync(sqlite3_file*, int);
-
SQLITE_PRIVATE int sqlite3OsFileSize(sqlite3_file*, i64 *pSize);
-
SQLITE_PRIVATE int sqlite3OsLock(sqlite3_file*, int);
-
SQLITE_PRIVATE int sqlite3OsUnlock(sqlite3_file*, int);
-
SQLITE_PRIVATE int sqlite3OsCheckReservedLock(sqlite3_file *id, int *pResOut);
-
SQLITE_PRIVATE int sqlite3OsFileControl(sqlite3_file*,int,void*);
-
SQLITE_PRIVATE void sqlite3OsFileControlHint(sqlite3_file*,int,void*);
-
#define SQLITE_FCNTL_DB_UNCHANGED 0xca093fa0
-
SQLITE_PRIVATE int sqlite3OsSectorSize(sqlite3_file *id);
-
SQLITE_PRIVATE int sqlite3OsDeviceCharacteristics(sqlite3_file *id);
-
#ifndef SQLITE_OMIT_WAL
-
SQLITE_PRIVATE int sqlite3OsShmMap(sqlite3_file *,int,int,int,void volatile **);
-
SQLITE_PRIVATE int sqlite3OsShmLock(sqlite3_file *id, int, int, int);
-
SQLITE_PRIVATE void sqlite3OsShmBarrier(sqlite3_file *id);
-
SQLITE_PRIVATE int sqlite3OsShmUnmap(sqlite3_file *id, int);
-
#endif /* SQLITE_OMIT_WAL */
-
SQLITE_PRIVATE int sqlite3OsFetch(sqlite3_file *id, i64, int, void **);
-
SQLITE_PRIVATE int sqlite3OsUnfetch(sqlite3_file *, i64, void *);
-

-

-
/*
-
** Functions for accessing sqlite3_vfs methods
-
*/
-
SQLITE_PRIVATE int sqlite3OsOpen(sqlite3_vfs *, const char *, sqlite3_file*, int, int *);
-
SQLITE_PRIVATE int sqlite3OsDelete(sqlite3_vfs *, const char *, int);
-
SQLITE_PRIVATE int sqlite3OsAccess(sqlite3_vfs *, const char *, int, int *pResOut);
-
SQLITE_PRIVATE int sqlite3OsFullPathname(sqlite3_vfs *, const char *, int, char *);
-
#ifndef SQLITE_OMIT_LOAD_EXTENSION
-
SQLITE_PRIVATE void *sqlite3OsDlOpen(sqlite3_vfs *, const char *);
-
SQLITE_PRIVATE void sqlite3OsDlError(sqlite3_vfs *, int, char *);
-
SQLITE_PRIVATE void (*sqlite3OsDlSym(sqlite3_vfs *, void *, const char *))(void);
-
SQLITE_PRIVATE void sqlite3OsDlClose(sqlite3_vfs *, void *);
-
#endif /* SQLITE_OMIT_LOAD_EXTENSION */
-
SQLITE_PRIVATE int sqlite3OsRandomness(sqlite3_vfs *, int, char *);
-
SQLITE_PRIVATE int sqlite3OsSleep(sqlite3_vfs *, int);
-
SQLITE_PRIVATE int sqlite3OsGetLastError(sqlite3_vfs*);
-
SQLITE_PRIVATE int sqlite3OsCurrentTimeInt64(sqlite3_vfs *, sqlite3_int64*);
-

-
/*
-
** Convenience functions for opening and closing files using
-
** sqlite3_malloc() to obtain space for the file-handle structure.
-
*/
-
SQLITE_PRIVATE int sqlite3OsOpenMalloc(sqlite3_vfs *, const char *, sqlite3_file **, int,int*);
-
SQLITE_PRIVATE void sqlite3OsCloseFree(sqlite3_file *);
-

-
#endif /* _SQLITE_OS_H_ */
-

-
/************** End of os.h **************************************************/
-
/************** Continuing where we left off in sqliteInt.h ******************/
/************** Include mutex.h in the middle of sqliteInt.h *****************/
/************** Begin file mutex.h *******************************************/
/*
@@ -16692,6 +16837,7 @@ struct Lookaside {
#endif /* SQLITE_OMIT_TWOSIZE_LOOKASIDE */
  void *pStart;           /* First byte of available memory space */
  void *pEnd;             /* First byte past end of available space */
+
  void *pTrueEnd;         /* True value of pEnd, when db->pnBytesFreed!=0 */
};
struct LookasideSlot {
  LookasideSlot *pNext;    /* Next buffer in the list of free buffers */
@@ -17033,6 +17179,10 @@ struct sqlite3 {
#define SQLITE_BloomFilter    0x00080000 /* Use a Bloom filter on searches */
#define SQLITE_BloomPulldown  0x00100000 /* Run Bloom filters early */
#define SQLITE_BalancedMerge  0x00200000 /* Balance multi-way merges */
+
#define SQLITE_ReleaseReg     0x00400000 /* Use OP_ReleaseReg for testing */
+
#define SQLITE_FlttnUnionAll  0x00800000 /* Disable the UNION ALL flattener */
+
   /* TH3 expects this value  ^^^^^^^^^^ See flatten04.test */
+
#define SQLITE_IndexedExpr    0x01000000 /* Pull exprs from index when able */
#define SQLITE_AllOpts        0xffffffff /* All optimizations */

/*
@@ -17135,7 +17285,7 @@ struct FuncDestructor {
#define SQLITE_FUNC_SLOCHNG  0x2000 /* "Slow Change". Value constant during a
                                    ** single query - might change over time */
#define SQLITE_FUNC_TEST     0x4000 /* Built-in testing functions */
-
#define SQLITE_FUNC_OFFSET   0x8000 /* Built-in sqlite_offset() function */
+
/*                           0x8000 -- available for reuse */
#define SQLITE_FUNC_WINDOW   0x00010000 /* Built-in window-only function */
#define SQLITE_FUNC_INTERNAL 0x00040000 /* For use by NestedParse() only */
#define SQLITE_FUNC_DIRECT   0x00080000 /* Not for use in TRIGGERs or VIEWs */
@@ -17152,6 +17302,7 @@ struct FuncDestructor {
#define INLINEFUNC_expr_compare         3
#define INLINEFUNC_affinity             4
#define INLINEFUNC_iif                  5
+
#define INLINEFUNC_sqlite_offset        6
#define INLINEFUNC_unlikely            99  /* Default case */

/*
@@ -17378,6 +17529,7 @@ struct Column {
#define COLFLAG_NOTAVAIL  0x0080   /* STORED column not yet calculated */
#define COLFLAG_BUSY      0x0100   /* Blocks recursion on GENERATED columns */
#define COLFLAG_HASCOLL   0x0200   /* Has collating sequence name in zCnName */
+
#define COLFLAG_NOEXPAND  0x0400   /* Omit this column when expanding "*" */
#define COLFLAG_GENERATED 0x0060   /* Combo: _STORED, _VIRTUAL */
#define COLFLAG_NOINSERT  0x0062   /* Combo: _HIDDEN, _STORED, _VIRTUAL */

@@ -17603,7 +17755,7 @@ struct Table {
#ifndef SQLITE_OMIT_VIRTUALTABLE
#  define IsVirtual(X)      ((X)->eTabType==TABTYP_VTAB)
#  define ExprIsVtab(X)  \
-
    ((X)->op==TK_COLUMN && (X)->y.pTab!=0 && (X)->y.pTab->eTabType==TABTYP_VTAB)
+
   ((X)->op==TK_COLUMN && (X)->y.pTab->eTabType==TABTYP_VTAB)
#else
#  define IsVirtual(X)      0
#  define ExprIsVtab(X)     0
@@ -17784,6 +17936,11 @@ struct KeyInfo {
struct UnpackedRecord {
  KeyInfo *pKeyInfo;  /* Collation and sort-order information */
  Mem *aMem;          /* Values */
+
  union {
+
    char *z;            /* Cache of aMem[0].z for vdbeRecordCompareString() */
+
    i64 i;              /* Cache of aMem[0].u.i for vdbeRecordCompareInt() */
+
  } u;
+
  int n;              /* Cache of aMem[0].n used by vdbeRecordCompareString() */
  u16 nField;         /* Number of entries in apMem[] */
  i8 default_rc;      /* Comparison result if keys are equal */
  u8 errCode;         /* Error detected by xRecordCompare (CORRUPT or NOMEM) */
@@ -17815,10 +17972,22 @@ struct UnpackedRecord {
** The Index.onError field determines whether or not the indexed columns
** must be unique and what to do if they are not.  When Index.onError=OE_None,
** it means this is not a unique index.  Otherwise it is a unique index
-
** and the value of Index.onError indicate the which conflict resolution
-
** algorithm to employ whenever an attempt is made to insert a non-unique
+
** and the value of Index.onError indicates which conflict resolution
+
** algorithm to employ when an attempt is made to insert a non-unique
** element.
**
+
** The colNotIdxed bitmask is used in combination with SrcItem.colUsed
+
** for a fast test to see if an index can serve as a covering index.
+
** colNotIdxed has a 1 bit for every column of the original table that
+
** is *not* available in the index.  Thus the expression
+
** "colUsed & colNotIdxed" will be non-zero if the index is not a
+
** covering index.  The most significant bit of of colNotIdxed will always
+
** be true (note-20221022-a).  If a column beyond the 63rd column of the
+
** table is used, the "colUsed & colNotIdxed" test will always be non-zero
+
** and we have to assume either that the index is not covering, or use
+
** an alternative (slower) algorithm to determine whether or not
+
** the index is covering.
+
**
** While parsing a CREATE TABLE or CREATE INDEX statement in order to
** generate VDBE code (as opposed to parsing one read from an sqlite_schema
** table as part of parsing an existing database schema), transient instances
@@ -17854,6 +18023,8 @@ struct Index {
  unsigned bNoQuery:1;     /* Do not use this index to optimize queries */
  unsigned bAscKeyBug:1;   /* True if the bba7b69f9849b5bf bug applies */
  unsigned bHasVCol:1;     /* Index references one or more VIRTUAL columns */
+
  unsigned bHasExpr:1;     /* Index contains an expression, either a literal
+
                           ** expression, or a reference to a VIRTUAL column */
#ifdef SQLITE_ENABLE_STAT4
  int nSample;             /* Number of elements in aSample[] */
  int nSampleCol;          /* Size of IndexSample.anEq[] and so on */
@@ -17862,7 +18033,7 @@ struct Index {
  tRowcnt *aiRowEst;       /* Non-logarithmic stat1 data for this index */
  tRowcnt nRowEst0;        /* Non-logarithmic number of rows in the index */
#endif
-
  Bitmask colNotIdxed;     /* 0 for unindexed columns in pTab */
+
  Bitmask colNotIdxed;     /* Unindexed columns in pTab */
};

/*
@@ -18092,7 +18263,7 @@ struct Expr {
                         ** TK_SELECT_COLUMN: column of the result vector */
  i16 iAgg;              /* Which entry in pAggInfo->aCol[] or ->aFunc[] */
  union {
-
    int iRightJoinTable;   /* If EP_FromJoin, the right table of the join */
+
    int iJoin;             /* If EP_OuterON or EP_InnerON, the right table */
    int iOfst;             /* else: start of token from start of statement */
  } w;
  AggInfo *pAggInfo;     /* Used by TK_AGG_COLUMN and TK_AGG_FUNCTION */
@@ -18113,29 +18284,29 @@ struct Expr {
**          EP_Agg == NC_HasAgg == SF_HasAgg
**          EP_Win == NC_HasWin
*/
-
#define EP_FromJoin   0x000001 /* Originates in ON/USING clause of outer join */
-
#define EP_Distinct   0x000002 /* Aggregate function with DISTINCT keyword */
-
#define EP_HasFunc    0x000004 /* Contains one or more functions of any kind */
-
#define EP_FixedCol   0x000008 /* TK_Column with a known fixed value */
+
#define EP_OuterON    0x000001 /* Originates in ON/USING clause of outer join */
+
#define EP_InnerON    0x000002 /* Originates in ON/USING of an inner join */
+
#define EP_Distinct   0x000004 /* Aggregate function with DISTINCT keyword */
+
#define EP_HasFunc    0x000008 /* Contains one or more functions of any kind */
#define EP_Agg        0x000010 /* Contains one or more aggregate functions */
-
#define EP_VarSelect  0x000020 /* pSelect is correlated, not constant */
-
#define EP_DblQuoted  0x000040 /* token.z was originally in "..." */
-
#define EP_InfixFunc  0x000080 /* True for an infix function: LIKE, GLOB, etc */
-
#define EP_Collate    0x000100 /* Tree contains a TK_COLLATE operator */
-
#define EP_Commuted   0x000200 /* Comparison operator has been commuted */
-
#define EP_IntValue   0x000400 /* Integer value contained in u.iValue */
-
#define EP_xIsSelect  0x000800 /* x.pSelect is valid (otherwise x.pList is) */
-
#define EP_Skip       0x001000 /* Operator does not contribute to affinity */
-
#define EP_Reduced    0x002000 /* Expr struct EXPR_REDUCEDSIZE bytes only */
-
#define EP_TokenOnly  0x004000 /* Expr struct EXPR_TOKENONLYSIZE bytes only */
+
#define EP_FixedCol   0x000020 /* TK_Column with a known fixed value */
+
#define EP_VarSelect  0x000040 /* pSelect is correlated, not constant */
+
#define EP_DblQuoted  0x000080 /* token.z was originally in "..." */
+
#define EP_InfixFunc  0x000100 /* True for an infix function: LIKE, GLOB, etc */
+
#define EP_Collate    0x000200 /* Tree contains a TK_COLLATE operator */
+
#define EP_Commuted   0x000400 /* Comparison operator has been commuted */
+
#define EP_IntValue   0x000800 /* Integer value contained in u.iValue */
+
#define EP_xIsSelect  0x001000 /* x.pSelect is valid (otherwise x.pList is) */
+
#define EP_Skip       0x002000 /* Operator does not contribute to affinity */
+
#define EP_Reduced    0x004000 /* Expr struct EXPR_REDUCEDSIZE bytes only */
#define EP_Win        0x008000 /* Contains window functions */
-
#define EP_MemToken   0x010000 /* Need to sqlite3DbFree() Expr.zToken */
-
#define EP_IfNullRow  0x020000 /* The TK_IF_NULL_ROW opcode */
-
#define EP_Unlikely   0x040000 /* unlikely() or likelihood() function */
-
#define EP_ConstFunc  0x080000 /* A SQLITE_FUNC_CONSTANT or _SLOCHNG function */
-
#define EP_CanBeNull  0x100000 /* Can be null despite NOT NULL constraint */
-
#define EP_Subquery   0x200000 /* Tree contains a TK_SELECT operator */
-
                 /*   0x400000 // Available */
+
#define EP_TokenOnly  0x010000 /* Expr struct EXPR_TOKENONLYSIZE bytes only */
+
                   /* 0x020000 // Available for reuse */
+
#define EP_IfNullRow  0x040000 /* The TK_IF_NULL_ROW opcode */
+
#define EP_Unlikely   0x080000 /* unlikely() or likelihood() function */
+
#define EP_ConstFunc  0x100000 /* A SQLITE_FUNC_CONSTANT or _SLOCHNG function */
+
#define EP_CanBeNull  0x200000 /* Can be null despite NOT NULL constraint */
+
#define EP_Subquery   0x400000 /* Tree contains a TK_SELECT operator */
#define EP_Leaf       0x800000 /* Expr.pLeft, .pRight, .u.pSelect all NULL */
#define EP_WinFunc   0x1000000 /* TK_FUNCTION with Expr.y.pWin set */
#define EP_Subrtn    0x2000000 /* Uses Expr.y.sub. TK_IN, _SELECT, or _EXISTS */
@@ -18158,8 +18329,8 @@ struct Expr {
#define ExprHasAllProperty(E,P)  (((E)->flags&(P))==(P))
#define ExprSetProperty(E,P)     (E)->flags|=(P)
#define ExprClearProperty(E,P)   (E)->flags&=~(P)
-
#define ExprAlwaysTrue(E)   (((E)->flags&(EP_FromJoin|EP_IsTrue))==EP_IsTrue)
-
#define ExprAlwaysFalse(E)  (((E)->flags&(EP_FromJoin|EP_IsFalse))==EP_IsFalse)
+
#define ExprAlwaysTrue(E)   (((E)->flags&(EP_OuterON|EP_IsTrue))==EP_IsTrue)
+
#define ExprAlwaysFalse(E)  (((E)->flags&(EP_OuterON|EP_IsFalse))==EP_IsFalse)

/* Macros used to ensure that the correct members of unions are accessed
** in Expr.
@@ -18246,12 +18417,18 @@ struct ExprList {
  struct ExprList_item { /* For each expression in the list */
    Expr *pExpr;            /* The parse tree for this expression */
    char *zEName;           /* Token associated with this expression */
-
    u8 sortFlags;           /* Mask of KEYINFO_ORDER_* flags */
-
    unsigned eEName :2;     /* Meaning of zEName */
-
    unsigned done :1;       /* A flag to indicate when processing is finished */
-
    unsigned reusable :1;   /* Constant expression is reusable */
-
    unsigned bSorterRef :1; /* Defer evaluation until after sorting */
-
    unsigned bNulls: 1;     /* True if explicit "NULLS FIRST/LAST" */
+
    struct {
+
      u8 sortFlags;           /* Mask of KEYINFO_ORDER_* flags */
+
      unsigned eEName :2;     /* Meaning of zEName */
+
      unsigned done :1;       /* Indicates when processing is finished */
+
      unsigned reusable :1;   /* Constant expression is reusable */
+
      unsigned bSorterRef :1; /* Defer evaluation until after sorting */
+
      unsigned bNulls :1;     /* True if explicit "NULLS FIRST/LAST" */
+
      unsigned bUsed :1;      /* This column used in a SF_NestedFrom subquery */
+
      unsigned bUsingTerm:1;  /* Term from the USING clause of a NestedFrom */
+
      unsigned bNoExpand: 1;  /* Term is an auxiliary in NestedFrom and should
+
                              ** not be expanded by "*" in parent queries */
+
    } fg;
    union {
      struct {             /* Used by any ExprList other than Parse.pConsExpr */
        u16 iOrderByCol;      /* For ORDER BY, column number in result set */
@@ -18286,17 +18463,37 @@ struct ExprList {
** If "a" is the k-th column of table "t", then IdList.a[0].idx==k.
*/
struct IdList {
+
  int nId;         /* Number of identifiers on the list */
+
  u8 eU4;          /* Which element of a.u4 is valid */
  struct IdList_item {
    char *zName;      /* Name of the identifier */
-
    int idx;          /* Index in some Table.aCol[] of a column named zName */
-
  } *a;
-
  int nId;         /* Number of identifiers on the list */
+
    union {
+
      int idx;          /* Index in some Table.aCol[] of a column named zName */
+
      Expr *pExpr;      /* Expr to implement a USING variable -- NOT USED */
+
    } u4;
+
  } a[1];
};

/*
+
** Allowed values for IdList.eType, which determines which value of the a.u4
+
** is valid.
+
*/
+
#define EU4_NONE   0   /* Does not use IdList.a.u4 */
+
#define EU4_IDX    1   /* Uses IdList.a.u4.idx */
+
#define EU4_EXPR   2   /* Uses IdList.a.u4.pExpr -- NOT CURRENTLY USED */
+

+
/*
** The SrcItem object represents a single term in the FROM clause of a query.
** The SrcList object is mostly an array of SrcItems.
**
+
** The jointype starts out showing the join type between the current table
+
** and the next table on the list.  The parser builds the list this way.
+
** But sqlite3SrcListShiftJoinType() later shifts the jointypes so that each
+
** jointype expresses the join between the table and the previous table.
+
**
+
** In the colUsed field, the high-order bit (bit 63) is set if the table
+
** contains more than 63 columns and the 64-th or later column is used.
+
**
** Union member validity:
**
**    u1.zIndexedBy          fg.isIndexedBy && !fg.isTabFunc
@@ -18320,44 +18517,48 @@ struct SrcItem {
    unsigned isIndexedBy :1;   /* True if there is an INDEXED BY clause */
    unsigned isTabFunc :1;     /* True if table-valued-function syntax */
    unsigned isCorrelated :1;  /* True if sub-query is correlated */
+
    unsigned isMaterialized:1; /* This is a materialized view */
    unsigned viaCoroutine :1;  /* Implemented as a co-routine */
    unsigned isRecursive :1;   /* True for recursive reference in WITH */
    unsigned fromDDL :1;       /* Comes from sqlite_schema */
    unsigned isCte :1;         /* This is a CTE */
    unsigned notCte :1;        /* This item may not match a CTE */
+
    unsigned isUsing :1;       /* u3.pUsing is valid */
+
    unsigned isOn :1;          /* u3.pOn was once valid and non-NULL */
+
    unsigned isSynthUsing :1;  /* u3.pUsing is synthensized from NATURAL */
+
    unsigned isNestedFrom :1;  /* pSelect is a SF_NestedFrom subquery */
  } fg;
  int iCursor;      /* The VDBE cursor number used to access this table */
-
  Expr *pOn;        /* The ON clause of a join */
-
  IdList *pUsing;   /* The USING clause of a join */
-
  Bitmask colUsed;  /* Bit N (1<<N) set if column N of pTab is used */
+
  union {
+
    Expr *pOn;        /* fg.isUsing==0 =>  The ON clause of a join */
+
    IdList *pUsing;   /* fg.isUsing==1 =>  The USING clause of a join */
+
  } u3;
+
  Bitmask colUsed;  /* Bit N set if column N used. Details above for N>62 */
  union {
    char *zIndexedBy;    /* Identifier from "INDEXED BY <zIndex>" clause */
    ExprList *pFuncArg;  /* Arguments to table-valued-function */
  } u1;
  union {
    Index *pIBIndex;  /* Index structure corresponding to u1.zIndexedBy */
-
    CteUse *pCteUse;  /* CTE Usage info info fg.isCte is true */
+
    CteUse *pCteUse;  /* CTE Usage info when fg.isCte is true */
  } u2;
};

/*
-
** The following structure describes the FROM clause of a SELECT statement.
-
** Each table or subquery in the FROM clause is a separate element of
-
** the SrcList.a[] array.
-
**
-
** With the addition of multiple database support, the following structure
-
** can also be used to describe a particular table such as the table that
-
** is modified by an INSERT, DELETE, or UPDATE statement.  In standard SQL,
-
** such a table must be a simple name: ID.  But in SQLite, the table can
-
** now be identified by a database name, a dot, then the table name: ID.ID.
-
**
-
** The jointype starts out showing the join type between the current table
-
** and the next table on the list.  The parser builds the list this way.
-
** But sqlite3SrcListShiftJoinType() later shifts the jointypes so that each
-
** jointype expresses the join between the table and the previous table.
+
** The OnOrUsing object represents either an ON clause or a USING clause.
+
** It can never be both at the same time, but it can be neither.
+
*/
+
struct OnOrUsing {
+
  Expr *pOn;         /* The ON clause of a join */
+
  IdList *pUsing;    /* The USING clause of a join */
+
};
+

+
/*
+
** This object represents one or more tables that are the source of
+
** content for an SQL statement.  For example, a single SrcList object
+
** is used to hold the FROM clause of a SELECT statement.  SrcList also
+
** represents the target tables for DELETE, INSERT, and UPDATE statements.
**
-
** In the colUsed field, the high-order bit (bit 63) is set if the table
-
** contains more than 63 columns and the 64-th or later column is used.
*/
struct SrcList {
  int nSrc;        /* Number of tables or subqueries in the FROM clause */
@@ -18368,14 +18569,15 @@ struct SrcList {
/*
** Permitted values of the SrcList.a.jointype field
*/
-
#define JT_INNER     0x0001    /* Any kind of inner or cross join */
-
#define JT_CROSS     0x0002    /* Explicit use of the CROSS keyword */
-
#define JT_NATURAL   0x0004    /* True for a "natural" join */
-
#define JT_LEFT      0x0008    /* Left outer join */
-
#define JT_RIGHT     0x0010    /* Right outer join */
-
#define JT_OUTER     0x0020    /* The "OUTER" keyword is present */
-
#define JT_ERROR     0x0040    /* unknown or unsupported join type */
-

+
#define JT_INNER     0x01    /* Any kind of inner or cross join */
+
#define JT_CROSS     0x02    /* Explicit use of the CROSS keyword */
+
#define JT_NATURAL   0x04    /* True for a "natural" join */
+
#define JT_LEFT      0x08    /* Left outer join */
+
#define JT_RIGHT     0x10    /* Right outer join */
+
#define JT_OUTER     0x20    /* The "OUTER" keyword is present */
+
#define JT_LTORJ     0x40    /* One of the LEFT operands of a RIGHT JOIN
+
                             ** Mnemonic: Left Table Of Right Join */
+
#define JT_ERROR     0x80    /* unknown or unsupported join type */

/*
** Flags appropriate for the wctrlFlags parameter of sqlite3WhereBegin()
@@ -18398,7 +18600,7 @@ struct SrcList {
#define WHERE_SORTBYGROUP      0x0200 /* Support sqlite3WhereIsSorted() */
#define WHERE_AGG_DISTINCT     0x0400 /* Query is "SELECT agg(DISTINCT ...)" */
#define WHERE_ORDERBY_LIMIT    0x0800 /* ORDERBY+LIMIT on the inner loop */
-
                        /*     0x1000    not currently used */
+
#define WHERE_RIGHT_JOIN       0x1000 /* Processing a RIGHT JOIN */
                        /*     0x2000    not currently used */
#define WHERE_USE_LIMIT        0x4000 /* Use the LIMIT in cost estimates */
                        /*     0x8000    not currently used */
@@ -18594,6 +18796,9 @@ struct Select {
#define SF_CopyCte       0x4000000 /* SELECT statement is a copy of a CTE */
#define SF_OrderByReqd   0x8000000 /* The ORDER BY clause may not be omitted */

+
/* True if S exists and has SF_NestedFrom */
+
#define IsNestedFrom(S) ((S)!=0 && ((S)->selFlags&SF_NestedFrom)!=0)
+

/*
** The results of a SELECT can be distributed in several ways, as defined
** by one of the following macros.  The "SRT" prefix means "SELECT Result
@@ -18698,7 +18903,7 @@ struct SelectDest {
  int iSDParm2;        /* A second parameter for the eDest disposal method */
  int iSdst;           /* Base register where results are written */
  int nSdst;           /* Number of registers allocated */
-
  char *zAffSdst;      /* Affinity used when eDest==SRT_Set */
+
  char *zAffSdst;      /* Affinity used for SRT_Set, SRT_Table, and similar */
  ExprList *pOrderBy;  /* Key columns for SRT_Queue and SRT_DistQueue */
};

@@ -18764,6 +18969,28 @@ struct TriggerPrg {
#endif

/*
+
** For each index X that has as one of its arguments either an expression
+
** or the name of a virtual generated column, and if X is in scope such that
+
** the value of the expression can simply be read from the index, then
+
** there is an instance of this object on the Parse.pIdxExpr list.
+
**
+
** During code generation, while generating code to evaluate expressions,
+
** this list is consulted and if a matching expression is found, the value
+
** is read from the index rather than being recomputed.
+
*/
+
struct IndexedExpr {
+
  Expr *pExpr;            /* The expression contained in the index */
+
  int iDataCur;           /* The data cursor associated with the index */
+
  int iIdxCur;            /* The index cursor */
+
  int iIdxCol;            /* The index column that contains value of pExpr */
+
  u8 bMaybeNullRow;       /* True if we need an OP_IfNullRow check */
+
  IndexedExpr *pIENext;   /* Next in a list of all indexed expressions */
+
#ifdef SQLITE_ENABLE_EXPLAIN_COMMENTS
+
  const char *zIdxName;   /* Name of index, used only for bytecode comments */
+
#endif
+
};
+

+
/*
** An instance of the ParseCleanup object specifies an operation that
** should be performed after parsing to deallocation resources obtained
** during the parse and which are no longer needed.
@@ -18804,7 +19031,8 @@ struct Parse {
  u8 hasCompound;      /* Need to invoke convertCompoundSelectToSubquery() */
  u8 okConstFactor;    /* OK to factor out constants */
  u8 disableLookaside; /* Number of times lookaside has been disabled */
-
  u8 disableVtab;      /* Disable all virtual tables for this parse */
+
  u8 prepFlags;        /* SQLITE_PREPARE_* flags */
+
  u8 withinRJSubrtn;   /* Nesting level for RIGHT JOIN body subroutines */
#if defined(SQLITE_DEBUG) || defined(SQLITE_COVERAGE_TEST)
  u8 earlyCleanup;     /* OOM inside sqlite3ParserAddCleanup() */
#endif
@@ -18820,6 +19048,7 @@ struct Parse {
  int nLabelAlloc;     /* Number of slots in aLabel */
  int *aLabel;         /* Space to hold the labels */
  ExprList *pConstExpr;/* Constant expressions */
+
  IndexedExpr *pIdxExpr;/* List of expressions used by active indexes */
  Token constraintName;/* Name of the constraint currently being parsed */
  yDbMask writeMask;   /* Start a write transaction on these databases */
  yDbMask cookieMask;  /* Bitmask of schema verified databases */
@@ -18977,20 +19206,20 @@ struct AuthContext {
#define OPFLAG_PREFORMAT     0x80    /* OP_Insert uses preformatted cell */

/*
-
 * Each trigger present in the database schema is stored as an instance of
-
 * struct Trigger.
-
 *
-
 * Pointers to instances of struct Trigger are stored in two ways.
-
 * 1. In the "trigHash" hash table (part of the sqlite3* that represents the
-
 *    database). This allows Trigger structures to be retrieved by name.
-
 * 2. All triggers associated with a single table form a linked list, using the
-
 *    pNext member of struct Trigger. A pointer to the first element of the
-
 *    linked list is stored as the "pTrigger" member of the associated
-
 *    struct Table.
-
 *
-
 * The "step_list" member points to the first element of a linked list
-
 * containing the SQL statements specified as the trigger program.
-
 */
+
** Each trigger present in the database schema is stored as an instance of
+
** struct Trigger.
+
**
+
** Pointers to instances of struct Trigger are stored in two ways.
+
** 1. In the "trigHash" hash table (part of the sqlite3* that represents the
+
**    database). This allows Trigger structures to be retrieved by name.
+
** 2. All triggers associated with a single table form a linked list, using the
+
**    pNext member of struct Trigger. A pointer to the first element of the
+
**    linked list is stored as the "pTrigger" member of the associated
+
**    struct Table.
+
**
+
** The "step_list" member points to the first element of a linked list
+
** containing the SQL statements specified as the trigger program.
+
*/
struct Trigger {
  char *zName;            /* The name of the trigger                        */
  char *table;            /* The table or view to which the trigger applies */
@@ -19017,43 +19246,48 @@ struct Trigger {
#define TRIGGER_AFTER   2

/*
-
 * An instance of struct TriggerStep is used to store a single SQL statement
-
 * that is a part of a trigger-program.
-
 *
-
 * Instances of struct TriggerStep are stored in a singly linked list (linked
-
 * using the "pNext" member) referenced by the "step_list" member of the
-
 * associated struct Trigger instance. The first element of the linked list is
-
 * the first step of the trigger-program.
-
 *
-
 * The "op" member indicates whether this is a "DELETE", "INSERT", "UPDATE" or
-
 * "SELECT" statement. The meanings of the other members is determined by the
-
 * value of "op" as follows:
-
 *
-
 * (op == TK_INSERT)
-
 * orconf    -> stores the ON CONFLICT algorithm
-
 * pSelect   -> If this is an INSERT INTO ... SELECT ... statement, then
-
 *              this stores a pointer to the SELECT statement. Otherwise NULL.
-
 * zTarget   -> Dequoted name of the table to insert into.
-
 * pExprList -> If this is an INSERT INTO ... VALUES ... statement, then
-
 *              this stores values to be inserted. Otherwise NULL.
-
 * pIdList   -> If this is an INSERT INTO ... (<column-names>) VALUES ...
-
 *              statement, then this stores the column-names to be
-
 *              inserted into.
-
 *
-
 * (op == TK_DELETE)
-
 * zTarget   -> Dequoted name of the table to delete from.
-
 * pWhere    -> The WHERE clause of the DELETE statement if one is specified.
-
 *              Otherwise NULL.
-
 *
-
 * (op == TK_UPDATE)
-
 * zTarget   -> Dequoted name of the table to update.
-
 * pWhere    -> The WHERE clause of the UPDATE statement if one is specified.
-
 *              Otherwise NULL.
-
 * pExprList -> A list of the columns to update and the expressions to update
-
 *              them to. See sqlite3Update() documentation of "pChanges"
-
 *              argument.
-
 *
-
 */
+
** An instance of struct TriggerStep is used to store a single SQL statement
+
** that is a part of a trigger-program.
+
**
+
** Instances of struct TriggerStep are stored in a singly linked list (linked
+
** using the "pNext" member) referenced by the "step_list" member of the
+
** associated struct Trigger instance. The first element of the linked list is
+
** the first step of the trigger-program.
+
**
+
** The "op" member indicates whether this is a "DELETE", "INSERT", "UPDATE" or
+
** "SELECT" statement. The meanings of the other members is determined by the
+
** value of "op" as follows:
+
**
+
** (op == TK_INSERT)
+
** orconf    -> stores the ON CONFLICT algorithm
+
** pSelect   -> The content to be inserted - either a SELECT statement or
+
**              a VALUES clause.
+
** zTarget   -> Dequoted name of the table to insert into.
+
** pIdList   -> If this is an INSERT INTO ... (<column-names>) VALUES ...
+
**              statement, then this stores the column-names to be
+
**              inserted into.
+
** pUpsert   -> The ON CONFLICT clauses for an Upsert
+
**
+
** (op == TK_DELETE)
+
** zTarget   -> Dequoted name of the table to delete from.
+
** pWhere    -> The WHERE clause of the DELETE statement if one is specified.
+
**              Otherwise NULL.
+
**
+
** (op == TK_UPDATE)
+
** zTarget   -> Dequoted name of the table to update.
+
** pWhere    -> The WHERE clause of the UPDATE statement if one is specified.
+
**              Otherwise NULL.
+
** pExprList -> A list of the columns to update and the expressions to update
+
**              them to. See sqlite3Update() documentation of "pChanges"
+
**              argument.
+
**
+
** (op == TK_SELECT)
+
** pSelect   -> The SELECT statement
+
**
+
** (op == TK_RETURNING)
+
** pExprList -> The list of expressions that follow the RETURNING keyword.
+
**
+
*/
struct TriggerStep {
  u8 op;               /* One of TK_DELETE, TK_UPDATE, TK_INSERT, TK_SELECT,
                       ** or TK_RETURNING */
@@ -19250,15 +19484,15 @@ struct Walker {
    struct RefSrcList *pRefSrcList;           /* sqlite3ReferencesSrcList() */
    int *aiCol;                               /* array of column indexes */
    struct IdxCover *pIdxCover;               /* Check for index coverage */
-
    struct IdxExprTrans *pIdxTrans;           /* Convert idxed expr to column */
    ExprList *pGroupBy;                       /* GROUP BY clause */
    Select *pSelect;                          /* HAVING to WHERE clause ctx */
    struct WindowRewrite *pRewrite;           /* Window rewrite context */
    struct WhereConst *pConst;                /* WHERE clause constants */
    struct RenameCtx *pRename;                /* RENAME COLUMN context */
    struct Table *pTab;                       /* Table of generated column */
+
    struct CoveringIndexCheck *pCovIdxCk;     /* Check for covering index */
    SrcItem *pSrcItem;                        /* A single FROM clause item */
-
    DbFixer *pFix;
+
    DbFixer *pFix;                            /* See sqlite3FixSelect() */
  } u;
};

@@ -19408,7 +19642,7 @@ struct Window {
  Window **ppThis;        /* Pointer to this object in Select.pWin list */
  Window *pNextWin;       /* Next window function belonging to this SELECT */
  Expr *pFilter;          /* The FILTER expression */
-
  FuncDef *pFunc;         /* The function */
+
  FuncDef *pWFunc;        /* The function */
  int iEphCsr;            /* Partition buffer or Peer buffer */
  int regAccum;           /* Accumulator */
  int regResult;          /* Interim result */
@@ -19564,6 +19798,7 @@ SQLITE_PRIVATE void *sqlite3DbReallocOrFree(sqlite3 *, void *, u64);
SQLITE_PRIVATE void *sqlite3DbRealloc(sqlite3 *, void *, u64);
SQLITE_PRIVATE void sqlite3DbFree(sqlite3*, void*);
SQLITE_PRIVATE void sqlite3DbFreeNN(sqlite3*, void*);
+
SQLITE_PRIVATE void sqlite3DbNNFreeNN(sqlite3*, void*);
SQLITE_PRIVATE int sqlite3MallocSize(const void*);
SQLITE_PRIVATE int sqlite3DbMallocSize(sqlite3*, const void*);
SQLITE_PRIVATE void *sqlite3PageMalloc(int);
@@ -19584,12 +19819,16 @@ SQLITE_PRIVATE int sqlite3HeapNearlyFull(void);
*/
#ifdef SQLITE_USE_ALLOCA
# define sqlite3StackAllocRaw(D,N)   alloca(N)
+
# define sqlite3StackAllocRawNN(D,N) alloca(N)
# define sqlite3StackAllocZero(D,N)  memset(alloca(N), 0, N)
# define sqlite3StackFree(D,P)
+
# define sqlite3StackFreeNN(D,P)
#else
# define sqlite3StackAllocRaw(D,N)   sqlite3DbMallocRaw(D,N)
+
# define sqlite3StackAllocRawNN(D,N) sqlite3DbMallocRawNN(D,N)
# define sqlite3StackAllocZero(D,N)  sqlite3DbMallocZero(D,N)
# define sqlite3StackFree(D,P)       sqlite3DbFree(D,P)
+
# define sqlite3StackFreeNN(D,P)     sqlite3DbFreeNN(D,P)
#endif

/* Do not allow both MEMSYS5 and MEMSYS3 to be defined together.  If they
@@ -19663,18 +19902,53 @@ SQLITE_PRIVATE void *sqlite3TestTextToPtr(const char*);
#endif

#if defined(SQLITE_DEBUG)
+
SQLITE_PRIVATE   void sqlite3TreeViewLine(TreeView*, const char *zFormat, ...);
SQLITE_PRIVATE   void sqlite3TreeViewExpr(TreeView*, const Expr*, u8);
SQLITE_PRIVATE   void sqlite3TreeViewBareExprList(TreeView*, const ExprList*, const char*);
SQLITE_PRIVATE   void sqlite3TreeViewExprList(TreeView*, const ExprList*, u8, const char*);
+
SQLITE_PRIVATE   void sqlite3TreeViewBareIdList(TreeView*, const IdList*, const char*);
+
SQLITE_PRIVATE   void sqlite3TreeViewIdList(TreeView*, const IdList*, u8, const char*);
+
SQLITE_PRIVATE   void sqlite3TreeViewColumnList(TreeView*, const Column*, int, u8);
SQLITE_PRIVATE   void sqlite3TreeViewSrcList(TreeView*, const SrcList*);
SQLITE_PRIVATE   void sqlite3TreeViewSelect(TreeView*, const Select*, u8);
SQLITE_PRIVATE   void sqlite3TreeViewWith(TreeView*, const With*, u8);
+
SQLITE_PRIVATE   void sqlite3TreeViewUpsert(TreeView*, const Upsert*, u8);
+
#if TREETRACE_ENABLED
+
SQLITE_PRIVATE   void sqlite3TreeViewDelete(const With*, const SrcList*, const Expr*,
+
                             const ExprList*,const Expr*, const Trigger*);
+
SQLITE_PRIVATE   void sqlite3TreeViewInsert(const With*, const SrcList*,
+
                             const IdList*, const Select*, const ExprList*,
+
                             int, const Upsert*, const Trigger*);
+
SQLITE_PRIVATE   void sqlite3TreeViewUpdate(const With*, const SrcList*, const ExprList*,
+
                             const Expr*, int, const ExprList*, const Expr*,
+
                             const Upsert*, const Trigger*);
+
#endif
+
#ifndef SQLITE_OMIT_TRIGGER
+
SQLITE_PRIVATE   void sqlite3TreeViewTriggerStep(TreeView*, const TriggerStep*, u8, u8);
+
SQLITE_PRIVATE   void sqlite3TreeViewTrigger(TreeView*, const Trigger*, u8, u8);
+
#endif
#ifndef SQLITE_OMIT_WINDOWFUNC
SQLITE_PRIVATE   void sqlite3TreeViewWindow(TreeView*, const Window*, u8);
SQLITE_PRIVATE   void sqlite3TreeViewWinFunc(TreeView*, const Window*, u8);
#endif
+
SQLITE_PRIVATE   void sqlite3ShowExpr(const Expr*);
+
SQLITE_PRIVATE   void sqlite3ShowExprList(const ExprList*);
+
SQLITE_PRIVATE   void sqlite3ShowIdList(const IdList*);
+
SQLITE_PRIVATE   void sqlite3ShowSrcList(const SrcList*);
+
SQLITE_PRIVATE   void sqlite3ShowSelect(const Select*);
+
SQLITE_PRIVATE   void sqlite3ShowWith(const With*);
+
SQLITE_PRIVATE   void sqlite3ShowUpsert(const Upsert*);
+
#ifndef SQLITE_OMIT_TRIGGER
+
SQLITE_PRIVATE   void sqlite3ShowTriggerStep(const TriggerStep*);
+
SQLITE_PRIVATE   void sqlite3ShowTriggerStepList(const TriggerStep*);
+
SQLITE_PRIVATE   void sqlite3ShowTrigger(const Trigger*);
+
SQLITE_PRIVATE   void sqlite3ShowTriggerList(const Trigger*);
+
#endif
+
#ifndef SQLITE_OMIT_WINDOWFUNC
+
SQLITE_PRIVATE   void sqlite3ShowWindow(const Window*);
+
SQLITE_PRIVATE   void sqlite3ShowWinFunc(const Window*);
+
#endif
#endif
-


SQLITE_PRIVATE void sqlite3SetString(char **, sqlite3*, const char*);
SQLITE_PRIVATE void sqlite3ErrorMsg(Parse*, const char*, ...);
@@ -19823,13 +20097,14 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListEnlarge(Parse*, SrcList*, int, int);
SQLITE_PRIVATE SrcList *sqlite3SrcListAppendList(Parse *pParse, SrcList *p1, SrcList *p2);
SQLITE_PRIVATE SrcList *sqlite3SrcListAppend(Parse*, SrcList*, Token*, Token*);
SQLITE_PRIVATE SrcList *sqlite3SrcListAppendFromTerm(Parse*, SrcList*, Token*, Token*,
-
                                      Token*, Select*, Expr*, IdList*);
+
                                      Token*, Select*, OnOrUsing*);
SQLITE_PRIVATE void sqlite3SrcListIndexedBy(Parse *, SrcList *, Token *);
SQLITE_PRIVATE void sqlite3SrcListFuncArgs(Parse*, SrcList*, ExprList*);
SQLITE_PRIVATE int sqlite3IndexedByLookup(Parse *, SrcItem *);
-
SQLITE_PRIVATE void sqlite3SrcListShiftJoinType(SrcList*);
+
SQLITE_PRIVATE void sqlite3SrcListShiftJoinType(Parse*,SrcList*);
SQLITE_PRIVATE void sqlite3SrcListAssignCursors(Parse*, SrcList*);
SQLITE_PRIVATE void sqlite3IdListDelete(sqlite3*, IdList*);
+
SQLITE_PRIVATE void sqlite3ClearOnOrUsing(sqlite3*, OnOrUsing*);
SQLITE_PRIVATE void sqlite3SrcListDelete(sqlite3*, SrcList*);
SQLITE_PRIVATE Index *sqlite3AllocateIndexObject(sqlite3*,i16,int,char**);
SQLITE_PRIVATE void sqlite3CreateIndex(Parse*,Token*,Token*,SrcList*,ExprList*,int,Token*,
@@ -20027,7 +20302,8 @@ SQLITE_PRIVATE SrcList *sqlite3TriggerStepSrc(Parse*, TriggerStep*);

SQLITE_PRIVATE int sqlite3JoinType(Parse*, Token*, Token*, Token*);
SQLITE_PRIVATE int sqlite3ColumnIndex(Table *pTab, const char *zCol);
-
SQLITE_PRIVATE void sqlite3SetJoinExpr(Expr*,int);
+
SQLITE_PRIVATE void sqlite3SrcItemColumnUsed(SrcItem*,int);
+
SQLITE_PRIVATE void sqlite3SetJoinExpr(Expr*,int,u32);
SQLITE_PRIVATE void sqlite3CreateForeignKey(Parse*, ExprList*, Token*, ExprList*, int);
SQLITE_PRIVATE void sqlite3DeferForeignKey(Parse*, int);
#ifndef SQLITE_OMIT_AUTHORIZATION
@@ -20051,6 +20327,7 @@ SQLITE_PRIVATE int sqlite3FixSelect(DbFixer*, Select*);
SQLITE_PRIVATE int sqlite3FixExpr(DbFixer*, Expr*);
SQLITE_PRIVATE int sqlite3FixTriggerStep(DbFixer*, TriggerStep*);
SQLITE_PRIVATE int sqlite3RealSameAsInt(double,sqlite3_int64);
+
SQLITE_PRIVATE i64 sqlite3RealToI64(double);
SQLITE_PRIVATE void sqlite3Int64ToText(i64,char*);
SQLITE_PRIVATE int sqlite3AtoF(const char *z, double*, int, u8);
SQLITE_PRIVATE int sqlite3GetInt32(const char *, int*);
@@ -20096,6 +20373,7 @@ SQLITE_PRIVATE int sqlite3VarintLen(u64 v);


SQLITE_PRIVATE const char *sqlite3IndexAffinityStr(sqlite3*, Index*);
+
SQLITE_PRIVATE char *sqlite3TableAffinityStr(sqlite3*,const Table*);
SQLITE_PRIVATE void sqlite3TableAffinity(Vdbe*, Table*, int);
SQLITE_PRIVATE char sqlite3CompareAffinity(const Expr *pExpr, char aff2);
SQLITE_PRIVATE int sqlite3IndexAffinityOk(const Expr *pExpr, char idx_affinity);
@@ -20167,7 +20445,6 @@ SQLITE_PRIVATE const unsigned char sqlite3OpcodeProperty[];
SQLITE_PRIVATE const char sqlite3StrBINARY[];
SQLITE_PRIVATE const unsigned char sqlite3StdTypeLen[];
SQLITE_PRIVATE const char sqlite3StdTypeAffinity[];
-
SQLITE_PRIVATE const char sqlite3StdTypeMap[];
SQLITE_PRIVATE const char *sqlite3StdType[];
SQLITE_PRIVATE const unsigned char sqlite3UpperToLower[];
SQLITE_PRIVATE const unsigned char *sqlite3aLTb;
@@ -20373,7 +20650,7 @@ SQLITE_PRIVATE int sqlite3VtabBegin(sqlite3 *, VTable *);
SQLITE_PRIVATE FuncDef *sqlite3VtabOverloadFunction(sqlite3 *,FuncDef*, int nArg, Expr*);
#if (defined(SQLITE_ENABLE_DBPAGE_VTAB) || defined(SQLITE_TEST)) \
    && !defined(SQLITE_OMIT_VIRTUALTABLE)
-
SQLITE_PRIVATE   void sqlite3VtabWriteAll(sqlite3_index_info*);
+
SQLITE_PRIVATE   void sqlite3VtabUsesAllSchemas(sqlite3_index_info*);
#endif
SQLITE_PRIVATE sqlite3_int64 sqlite3StmtCurrentTime(sqlite3_context*);
SQLITE_PRIVATE int sqlite3VdbeParameterIndex(Vdbe*, const char*, int);
@@ -20611,6 +20888,10 @@ SQLITE_PRIVATE void sqlite3VectorErrorMsg(Parse*, Expr*);
SQLITE_PRIVATE const char **sqlite3CompileOptions(int *pnOpt);
#endif

+
#if SQLITE_OS_UNIX && defined(SQLITE_OS_KV_OPTIONAL)
+
SQLITE_PRIVATE int sqlite3KvvfsInit(void);
+
#endif
+

#endif /* SQLITEINT_H */

/************** End of sqliteInt.h *******************************************/
@@ -20842,7 +21123,7 @@ SQLITE_API extern int sqlite3_open_file_count;
** autoconf-based build
*/
#if defined(_HAVE_SQLITE_CONFIG_H) && !defined(SQLITECONFIG_H)
-
/* #include "config.h" */
+
/* #include "sqlite_cfg.h" */
#define SQLITECONFIG_H 1
#endif

@@ -21007,6 +21288,9 @@ static const char * const sqlite3azCompileOpt[] = {
#ifdef SQLITE_DISABLE_SKIPAHEAD_DISTINCT
  "DISABLE_SKIPAHEAD_DISTINCT",
#endif
+
#ifdef SQLITE_DQS
+
  "DQS=" CTIMEOPT_VAL(SQLITE_DQS),
+
#endif
#ifdef SQLITE_ENABLE_8_3_NAMES
  "ENABLE_8_3_NAMES=" CTIMEOPT_VAL(SQLITE_ENABLE_8_3_NAMES),
#endif
@@ -21121,9 +21405,6 @@ static const char * const sqlite3azCompileOpt[] = {
#ifdef SQLITE_ENABLE_RTREE
  "ENABLE_RTREE",
#endif
-
#ifdef SQLITE_ENABLE_SELECTTRACE
-
  "ENABLE_SELECTTRACE",
-
#endif
#ifdef SQLITE_ENABLE_SESSION
  "ENABLE_SESSION",
#endif
@@ -21145,6 +21426,9 @@ static const char * const sqlite3azCompileOpt[] = {
#ifdef SQLITE_ENABLE_STMT_SCANSTATUS
  "ENABLE_STMT_SCANSTATUS",
#endif
+
#ifdef SQLITE_ENABLE_TREETRACE
+
  "ENABLE_TREETRACE",
+
#endif
#ifdef SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
  "ENABLE_UNKNOWN_SQL_FUNCTION",
#endif
@@ -21497,9 +21781,6 @@ static const char * const sqlite3azCompileOpt[] = {
#ifdef SQLITE_OMIT_XFER_OPT
  "OMIT_XFER_OPT",
#endif
-
#ifdef SQLITE_PCACHE_SEPARATE_HEADER
-
  "PCACHE_SEPARATE_HEADER",
-
#endif
#ifdef SQLITE_PERFORMANCE_TRACE
  "PERFORMANCE_TRACE",
#endif
@@ -21894,6 +22175,9 @@ SQLITE_PRIVATE SQLITE_WSD struct Sqlite3Config sqlite3Config = {
   0x7ffffffe,                /* iOnceResetThreshold */
   SQLITE_DEFAULT_SORTERREF_SIZE,   /* szSorterRef */
   0,                         /* iPrngSeed */
+
#ifdef SQLITE_DEBUG
+
   {0,0,0,0,0,0}              /* aTune */
+
#endif
};

/*
@@ -21948,7 +22232,7 @@ SQLITE_PRIVATE int sqlite3PendingByte = 0x40000000;
/*
** Tracing flags set by SQLITE_TESTCTRL_TRACEFLAGS.
*/
-
SQLITE_PRIVATE u32 sqlite3SelectTrace = 0;
+
SQLITE_PRIVATE u32 sqlite3TreeTrace = 0;
SQLITE_PRIVATE u32 sqlite3WhereTrace = 0;

/* #include "opcodes.h" */
@@ -21976,10 +22260,6 @@ SQLITE_PRIVATE const char sqlite3StrBINARY[] = "BINARY";
**
**    sqlite3StdTypeAffinity[]    The affinity associated with each entry
**                                in sqlite3StdType[].
-
**
-
**    sqlite3StdTypeMap[]         The type value (as returned from
-
**                                sqlite3_column_type() or sqlite3_value_type())
-
**                                for each entry in sqlite3StdType[].
*/
SQLITE_PRIVATE const unsigned char sqlite3StdTypeLen[] = { 3, 4, 3, 7, 4, 4 };
SQLITE_PRIVATE const char sqlite3StdTypeAffinity[] = {
@@ -21990,14 +22270,6 @@ SQLITE_PRIVATE const char sqlite3StdTypeAffinity[] = {
  SQLITE_AFF_REAL,
  SQLITE_AFF_TEXT
};
-
SQLITE_PRIVATE const char sqlite3StdTypeMap[] = {
-
  0,
-
  SQLITE_BLOB,
-
  SQLITE_INTEGER,
-
  SQLITE_INTEGER,
-
  SQLITE_FLOAT,
-
  SQLITE_TEXT
-
};
SQLITE_PRIVATE const char *sqlite3StdType[] = {
  "ANY",
  "BLOB",
@@ -22115,7 +22387,7 @@ struct VdbeCursor {
  Bool isEphemeral:1;     /* True for an ephemeral table */
  Bool useRandomRowid:1;  /* Generate new record numbers semi-randomly */
  Bool isOrdered:1;       /* True if the table is not BTREE_UNORDERED */
-
  Bool hasBeenDuped:1;    /* This cursor was source or target of OP_OpenDup */
+
  Bool noReuse:1;         /* OpenEphemeral may not reuse this cursor */
  u16 seekHit;            /* See the OP_SeekHit and OP_IfNoHope opcodes */
  union {                 /* pBtx for isEphermeral.  pAltMap otherwise */
    Btree *pBtx;            /* Separate file holding temporary table */
@@ -22163,6 +22435,11 @@ struct VdbeCursor {
  u32 aType[1];           /* Type values record decode.  MUST BE LAST */
};

+
/* Return true if P is a null-only cursor
+
*/
+
#define IsNullCursor(P) \
+
  ((P)->eCurType==CURTYPE_PSEUDO && (P)->nullRow && (P)->seekResult==0)
+


/*
** A value for VdbeCursor.cacheStatus that means the cache is always invalid.
@@ -22237,16 +22514,16 @@ struct sqlite3_value {
    const char *zPType; /* Pointer type when MEM_Term|MEM_Subtype|MEM_Null */
    FuncDef *pDef;      /* Used only when flags==MEM_Agg */
  } u;
+
  char *z;            /* String or BLOB value */
+
  int n;              /* Number of characters in string value, excluding '\0' */
  u16 flags;          /* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */
  u8  enc;            /* SQLITE_UTF8, SQLITE_UTF16BE, SQLITE_UTF16LE */
  u8  eSubtype;       /* Subtype for this value */
-
  int n;              /* Number of characters in string value, excluding '\0' */
-
  char *z;            /* String or BLOB value */
  /* ShallowCopy only needs to copy the information above */
-
  char *zMalloc;      /* Space to hold MEM_Str or MEM_Blob if szMalloc>0 */
+
  sqlite3 *db;        /* The associated database connection */
  int szMalloc;       /* Size of the zMalloc allocation */
  u32 uTemp;          /* Transient storage for serial_type in OP_MakeRecord */
-
  sqlite3 *db;        /* The associated database connection */
+
  char *zMalloc;      /* Space to hold MEM_Str or MEM_Blob if szMalloc>0 */
  void (*xDel)(void*);/* Destructor for Mem.z - only valid if MEM_Dyn */
#ifdef SQLITE_DEBUG
  Mem *pScopyFrom;    /* This Mem is a shallow copy of pScopyFrom */
@@ -22258,11 +22535,43 @@ struct sqlite3_value {
** Size of struct Mem not including the Mem.zMalloc member or anything that
** follows.
*/
-
#define MEMCELLSIZE offsetof(Mem,zMalloc)
+
#define MEMCELLSIZE offsetof(Mem,db)

-
/* One or more of the following flags are set to indicate the validOK
+
/* One or more of the following flags are set to indicate the
** representations of the value stored in the Mem struct.
**
+
**  *  MEM_Null                An SQL NULL value
+
**
+
**  *  MEM_Null|MEM_Zero       An SQL NULL with the virtual table
+
**                             UPDATE no-change flag set
+
**
+
**  *  MEM_Null|MEM_Term|      An SQL NULL, but also contains a
+
**        MEM_Subtype          pointer accessible using
+
**                             sqlite3_value_pointer().
+
**
+
**  *  MEM_Null|MEM_Cleared    Special SQL NULL that compares non-equal
+
**                             to other NULLs even using the IS operator.
+
**
+
**  *  MEM_Str                 A string, stored in Mem.z with
+
**                             length Mem.n.  Zero-terminated if
+
**                             MEM_Term is set.  This flag is
+
**                             incompatible with MEM_Blob and
+
**                             MEM_Null, but can appear with MEM_Int,
+
**                             MEM_Real, and MEM_IntReal.
+
**
+
**  *  MEM_Blob                A blob, stored in Mem.z length Mem.n.
+
**                             Incompatible with MEM_Str, MEM_Null,
+
**                             MEM_Int, MEM_Real, and MEM_IntReal.
+
**
+
**  *  MEM_Blob|MEM_Zero       A blob in Mem.z of length Mem.n plus
+
**                             MEM.u.i extra 0x00 bytes at the end.
+
**
+
**  *  MEM_Int                 Integer stored in Mem.u.i.
+
**
+
**  *  MEM_Real                Real stored in Mem.u.r.
+
**
+
**  *  MEM_IntReal             Real stored as an integer in Mem.u.i.
+
**
** If the MEM_Null flag is set, then the value is an SQL NULL value.
** For a pointer type created using sqlite3_bind_pointer() or
** sqlite3_result_pointer() the MEM_Term and MEM_Subtype flags are also set.
@@ -22273,6 +22582,7 @@ struct sqlite3_value {
** set, then the string is nul terminated. The MEM_Int and MEM_Real
** flags may coexist with the MEM_Str flag.
*/
+
#define MEM_Undefined 0x0000   /* Value is undefined */
#define MEM_Null      0x0001   /* Value is NULL (or a pointer) */
#define MEM_Str       0x0002   /* Value is a string */
#define MEM_Int       0x0004   /* Value is an integer */
@@ -22280,28 +22590,24 @@ struct sqlite3_value {
#define MEM_Blob      0x0010   /* Value is a BLOB */
#define MEM_IntReal   0x0020   /* MEM_Int that stringifies like MEM_Real */
#define MEM_AffMask   0x003f   /* Mask of affinity bits */
+

+
/* Extra bits that modify the meanings of the core datatypes above
+
*/
#define MEM_FromBind  0x0040   /* Value originates from sqlite3_bind() */
-
#define MEM_Undefined 0x0080   /* Value is undefined */
+
 /*                   0x0080   // Available */
#define MEM_Cleared   0x0100   /* NULL set by OP_Null, not from data */
-
#define MEM_TypeMask  0xc1bf   /* Mask of type bits */
-

+
#define MEM_Term      0x0200   /* String in Mem.z is zero terminated */
+
#define MEM_Zero      0x0400   /* Mem.i contains count of 0s appended to blob */
+
#define MEM_Subtype   0x0800   /* Mem.eSubtype is valid */
+
#define MEM_TypeMask  0x0dbf   /* Mask of type bits */

-
/* Whenever Mem contains a valid string or blob representation, one of
-
** the following flags must be set to determine the memory management
-
** policy for Mem.z.  The MEM_Term flag tells us whether or not the
-
** string is \000 or \u0000 terminated
+
/* Bits that determine the storage for Mem.z for a string or blob or
+
** aggregate accumulator.
*/
-
#define MEM_Term      0x0200   /* String in Mem.z is zero terminated */
-
#define MEM_Dyn       0x0400   /* Need to call Mem.xDel() on Mem.z */
-
#define MEM_Static    0x0800   /* Mem.z points to a static string */
-
#define MEM_Ephem     0x1000   /* Mem.z points to an ephemeral string */
-
#define MEM_Agg       0x2000   /* Mem.z points to an agg function context */
-
#define MEM_Zero      0x4000   /* Mem.i contains count of 0s appended to blob */
-
#define MEM_Subtype   0x8000   /* Mem.eSubtype is valid */
-
#ifdef SQLITE_OMIT_INCRBLOB
-
  #undef MEM_Zero
-
  #define MEM_Zero 0x0000
-
#endif
+
#define MEM_Dyn       0x1000   /* Need to call Mem.xDel() on Mem.z */
+
#define MEM_Static    0x2000   /* Mem.z points to a static string */
+
#define MEM_Ephem     0x4000   /* Mem.z points to an ephemeral string */
+
#define MEM_Agg       0x8000   /* Mem.z points to an agg function context */

/* Return TRUE if Mem X contains dynamically allocated content - anything
** that needs to be deallocated to avoid a leak.
@@ -22323,11 +22629,15 @@ struct sqlite3_value {
    && (X)->n==0 && (X)->u.nZero==0)

/*
-
** Return true if a memory cell is not marked as invalid.  This macro
+
** Return true if a memory cell has been initialized and is valid.
** is for use inside assert() statements only.
+
**
+
** A Memory cell is initialized if at least one of the
+
** MEM_Null, MEM_Str, MEM_Int, MEM_Real, MEM_Blob, or MEM_IntReal bits
+
** is set.  It is "undefined" if all those bits are zero.
*/
#ifdef SQLITE_DEBUG
-
#define memIsValid(M)  ((M)->flags & MEM_Undefined)==0
+
#define memIsValid(M)  ((M)->flags & MEM_AffMask)!=0
#endif

/*
@@ -22365,6 +22675,7 @@ struct sqlite3_context {
  Vdbe *pVdbe;            /* The VM that owns this context */
  int iOp;                /* Instruction number of OP_Function */
  int isError;            /* Error code returned by the function. */
+
  u8 enc;                 /* Encoding to use for results */
  u8 skipFlag;            /* Skip accumulator loading if true */
  u8 argc;                /* Number of arguments */
  sqlite3_value *argv[1]; /* Argument set */
@@ -22410,10 +22721,9 @@ struct DblquoteStr {
*/
struct Vdbe {
  sqlite3 *db;            /* The database connection that owns this statement */
-
  Vdbe *pPrev,*pNext;     /* Linked list of VDBEs with the same Vdbe.db */
+
  Vdbe **ppVPrev,*pVNext; /* Linked list of VDBEs with the same Vdbe.db */
  Parse *pParse;          /* Parsing context used to create this Vdbe */
  ynVar nVar;             /* Number of entries in aVar[] */
-
  u32 iVdbeMagic;         /* Magic number defining state of the SQL statement */
  int nMem;               /* Number of memory locations currently allocated */
  int nCursor;            /* Number of slots in apCsr[] */
  u32 cacheCtr;           /* VdbeCursor row cache generation counter */
@@ -22451,11 +22761,10 @@ struct Vdbe {
  u8 errorAction;         /* Recovery action to do in case of an error */
  u8 minWriteFileFormat;  /* Minimum file format for writable database files */
  u8 prepFlags;           /* SQLITE_PREPARE_* flags */
-
  u8 doingRerun;          /* True if rerunning after an auto-reprepare */
+
  u8 eVdbeState;          /* On of the VDBE_*_STATE values */
  bft expired:2;          /* 1: recompile VM immediately  2: when convenient */
  bft explain:2;          /* True if EXPLAIN present on SQL command */
  bft changeCntOn:1;      /* True to update the change-counter */
-
  bft runOnlyOnce:1;      /* Automatically expire on reset */
  bft usesStmtJournal:1;  /* True if uses a statement journal */
  bft readOnly:1;         /* True for statements that do not write */
  bft bIsReader:1;        /* True for statements that read */
@@ -22482,13 +22791,12 @@ struct Vdbe {
};

/*
-
** The following are allowed values for Vdbe.magic
+
** The following are allowed values for Vdbe.eVdbeState
*/
-
#define VDBE_MAGIC_INIT     0x16bceaa5    /* Building a VDBE program */
-
#define VDBE_MAGIC_RUN      0x2df20da3    /* VDBE is ready to execute */
-
#define VDBE_MAGIC_HALT     0x319c2973    /* VDBE has completed execution */
-
#define VDBE_MAGIC_RESET    0x48fa9f76    /* Reset and ready to run again */
-
#define VDBE_MAGIC_DEAD     0x5606c3c8    /* The VDBE has been deallocated */
+
#define VDBE_INIT_STATE     0   /* Prepared statement under construction */
+
#define VDBE_READY_STATE    1   /* Ready to run but not yet started */
+
#define VDBE_RUN_STATE      2   /* Run in progress */
+
#define VDBE_HALT_STATE     3   /* Finished.  Need reset() or finalize() */

/*
** Structure used to store the context required by the
@@ -22529,18 +22837,31 @@ struct ValueList {
  sqlite3_value *pOut;     /* Register to hold each decoded output value */
};

+
/* Size of content associated with serial types that fit into a
+
** single-byte varint.
+
*/
+
#ifndef SQLITE_AMALGAMATION
+
SQLITE_PRIVATE const u8 sqlite3SmallTypeSizes[];
+
#endif
+

/*
** Function prototypes
*/
SQLITE_PRIVATE void sqlite3VdbeError(Vdbe*, const char *, ...);
SQLITE_PRIVATE void sqlite3VdbeFreeCursor(Vdbe *, VdbeCursor*);
+
SQLITE_PRIVATE void sqlite3VdbeFreeCursorNN(Vdbe*,VdbeCursor*);
void sqliteVdbePopStack(Vdbe*,int);
+
SQLITE_PRIVATE int SQLITE_NOINLINE sqlite3VdbeHandleMovedCursor(VdbeCursor *p);
SQLITE_PRIVATE int SQLITE_NOINLINE sqlite3VdbeFinishMoveto(VdbeCursor*);
-
SQLITE_PRIVATE int sqlite3VdbeCursorMoveto(VdbeCursor**, u32*);
SQLITE_PRIVATE int sqlite3VdbeCursorRestore(VdbeCursor*);
SQLITE_PRIVATE u32 sqlite3VdbeSerialTypeLen(u32);
SQLITE_PRIVATE u8 sqlite3VdbeOneByteSerialTypeLen(u8);
-
SQLITE_PRIVATE u32 sqlite3VdbeSerialPut(unsigned char*, Mem*, u32);
+
#ifdef SQLITE_MIXED_ENDIAN_64BIT_FLOAT
+
SQLITE_PRIVATE   u64 sqlite3FloatSwap(u64 in);
+
# define swapMixedEndianFloat(X)  X = sqlite3FloatSwap(X)
+
#else
+
# define swapMixedEndianFloat(X)
+
#endif
SQLITE_PRIVATE void sqlite3VdbeSerialGet(const unsigned char*, u32, Mem*);
SQLITE_PRIVATE void sqlite3VdbeDeleteAuxData(sqlite3*, AuxData**, int, int);

@@ -22598,6 +22919,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemCast(Mem*,u8,u8);
SQLITE_PRIVATE int sqlite3VdbeMemFromBtree(BtCursor*,u32,u32,Mem*);
SQLITE_PRIVATE int sqlite3VdbeMemFromBtreeZeroOffset(BtCursor*,u32,Mem*);
SQLITE_PRIVATE void sqlite3VdbeMemRelease(Mem *p);
+
SQLITE_PRIVATE void sqlite3VdbeMemReleaseMalloc(Mem*p);
SQLITE_PRIVATE int sqlite3VdbeMemFinalize(Mem*, FuncDef*);
#ifndef SQLITE_OMIT_WINDOWFUNC
SQLITE_PRIVATE int sqlite3VdbeMemAggValue(Mem*, Mem*, FuncDef*);
@@ -22957,6 +23279,8 @@ SQLITE_API int sqlite3_db_status(

      sqlite3BtreeEnterAll(db);
      db->pnBytesFreed = &nByte;
+
      assert( db->lookaside.pEnd==db->lookaside.pTrueEnd );
+
      db->lookaside.pEnd = db->lookaside.pStart;
      for(i=0; i<db->nDb; i++){
        Schema *pSchema = db->aDb[i].pSchema;
        if( ALWAYS(pSchema!=0) ){
@@ -22982,6 +23306,7 @@ SQLITE_API int sqlite3_db_status(
        }
      }
      db->pnBytesFreed = 0;
+
      db->lookaside.pEnd = db->lookaside.pTrueEnd;
      sqlite3BtreeLeaveAll(db);

      *pHighwater = 0;
@@ -22999,10 +23324,12 @@ SQLITE_API int sqlite3_db_status(
      int nByte = 0;              /* Used to accumulate return value */

      db->pnBytesFreed = &nByte;
-
      for(pVdbe=db->pVdbe; pVdbe; pVdbe=pVdbe->pNext){
-
        sqlite3VdbeClearObject(db, pVdbe);
-
        sqlite3DbFree(db, pVdbe);
+
      assert( db->lookaside.pEnd==db->lookaside.pTrueEnd );
+
      db->lookaside.pEnd = db->lookaside.pStart;
+
      for(pVdbe=db->pVdbe; pVdbe; pVdbe=pVdbe->pVNext){
+
        sqlite3VdbeDelete(pVdbe);
      }
+
      db->lookaside.pEnd = db->lookaside.pTrueEnd;
      db->pnBytesFreed = 0;

      *pHighwater = 0;  /* IMP: R-64479-57858 */
@@ -23338,7 +23665,7 @@ static void computeJD(DateTime *p){
  p->iJD = (sqlite3_int64)((X1 + X2 + D + B - 1524.5 ) * 86400000);
  p->validJD = 1;
  if( p->validHMS ){
-
    p->iJD += p->h*3600000 + p->m*60000 + (sqlite3_int64)(p->s*1000);
+
    p->iJD += p->h*3600000 + p->m*60000 + (sqlite3_int64)(p->s*1000 + 0.5);
    if( p->validTZ ){
      p->iJD -= p->tz*60000;
      p->validYMD = 0;
@@ -23847,7 +24174,7 @@ static int parseModifier(
      */
      if( sqlite3_strnicmp(z, "weekday ", 8)==0
               && sqlite3AtoF(&z[8], &r, sqlite3Strlen30(&z[8]), SQLITE_UTF8)>0
-
               && (n=(int)r)==r && n>=0 && r<7 ){
+
               && r>=0.0 && r<7.0 && (n=(int)r)==r ){
        sqlite3_int64 Z;
        computeYMD_HMS(p);
        p->validTZ = 0;
@@ -24528,9 +24855,11 @@ SQLITE_PRIVATE int sqlite3OsFileSize(sqlite3_file *id, i64 *pSize){
}
SQLITE_PRIVATE int sqlite3OsLock(sqlite3_file *id, int lockType){
  DO_OS_MALLOC_TEST(id);
+
  assert( lockType>=SQLITE_LOCK_SHARED && lockType<=SQLITE_LOCK_EXCLUSIVE );
  return id->pMethods->xLock(id, lockType);
}
SQLITE_PRIVATE int sqlite3OsUnlock(sqlite3_file *id, int lockType){
+
  assert( lockType==SQLITE_LOCK_NONE || lockType==SQLITE_LOCK_SHARED );
  return id->pMethods->xUnlock(id, lockType);
}
SQLITE_PRIVATE int sqlite3OsCheckReservedLock(sqlite3_file *id, int *pResOut){
@@ -24645,6 +24974,7 @@ SQLITE_PRIVATE int sqlite3OsOpen(
  ** down into the VFS layer.  Some SQLITE_OPEN_ flags (for example,
  ** SQLITE_OPEN_FULLMUTEX or SQLITE_OPEN_SHAREDCACHE) are blocked before
  ** reaching the VFS. */
+
  assert( zPath || (flags & SQLITE_OPEN_EXCLUSIVE) );
  rc = pVfs->xOpen(pVfs, zPath, pFile, flags & 0x1087f7f, pFlagsOut);
  assert( rc==SQLITE_OK || pFile->pMethods==0 );
  return rc;
@@ -26956,8 +27286,13 @@ static void *memsys5Realloc(void *pPrior, int nBytes){
*/
static int memsys5Roundup(int n){
  int iFullSz;
-
  if( n > 0x40000000 ) return 0;
-
  for(iFullSz=mem5.szAtom; iFullSz<n; iFullSz *= 2);
+
  if( n<=mem5.szAtom*2 ){
+
    if( n<=mem5.szAtom ) return mem5.szAtom;
+
    return mem5.szAtom*2;
+
  }
+
  if( n>0x40000000 ) return 0;
+
  for(iFullSz=mem5.szAtom*8; iFullSz<n; iFullSz *= 4);
+
  if( (iFullSz/2)>=n ) return iFullSz/2;
  return iFullSz;
}

@@ -28859,17 +29194,33 @@ static void mallocWithAlarm(int n, void **pp){
}

/*
+
** Maximum size of any single memory allocation.
+
**
+
** This is not a limit on the total amount of memory used.  This is
+
** a limit on the size parameter to sqlite3_malloc() and sqlite3_realloc().
+
**
+
** The upper bound is slightly less than 2GiB:  0x7ffffeff == 2,147,483,391
+
** This provides a 256-byte safety margin for defense against 32-bit
+
** signed integer overflow bugs when computing memory allocation sizes.
+
** Parnoid applications might want to reduce the maximum allocation size
+
** further for an even larger safety margin.  0x3fffffff or 0x0fffffff
+
** or even smaller would be reasonable upper bounds on the size of a memory
+
** allocations for most applications.
+
*/
+
#ifndef SQLITE_MAX_ALLOCATION_SIZE
+
# define SQLITE_MAX_ALLOCATION_SIZE  2147483391
+
#endif
+
#if SQLITE_MAX_ALLOCATION_SIZE>2147483391
+
# error Maximum size for SQLITE_MAX_ALLOCATION_SIZE is 2147483391
+
#endif
+

+
/*
** Allocate memory.  This routine is like sqlite3_malloc() except that it
** assumes the memory subsystem has already been initialized.
*/
SQLITE_PRIVATE void *sqlite3Malloc(u64 n){
  void *p;
-
  if( n==0 || n>=0x7fffff00 ){
-
    /* A memory allocation of a number of bytes which is near the maximum
-
    ** signed integer value might cause an integer overflow inside of the
-
    ** xMalloc().  Hence we limit the maximum size to 0x7fffff00, giving
-
    ** 255 bytes of overhead.  SQLite itself will never use anything near
-
    ** this amount.  The only way to reach the limit is with sqlite3_malloc() */
+
  if( n==0 || n>SQLITE_MAX_ALLOCATION_SIZE ){
    p = 0;
  }else if( sqlite3GlobalConfig.bMemstat ){
    sqlite3_mutex_enter(mem0.mutex);
@@ -28905,7 +29256,7 @@ SQLITE_API void *sqlite3_malloc64(sqlite3_uint64 n){
*/
#ifndef SQLITE_OMIT_LOOKASIDE
static int isLookaside(sqlite3 *db, const void *p){
-
  return SQLITE_WITHIN(p, db->lookaside.pStart, db->lookaside.pEnd);
+
  return SQLITE_WITHIN(p, db->lookaside.pStart, db->lookaside.pTrueEnd);
}
#else
#define isLookaside(A,B) 0
@@ -28929,18 +29280,16 @@ static int lookasideMallocSize(sqlite3 *db, const void *p){
SQLITE_PRIVATE int sqlite3DbMallocSize(sqlite3 *db, const void *p){
  assert( p!=0 );
#ifdef SQLITE_DEBUG
-
  if( db==0 || !isLookaside(db,p) ){
-
    if( db==0 ){
-
      assert( sqlite3MemdebugNoType(p, (u8)~MEMTYPE_HEAP) );
-
      assert( sqlite3MemdebugHasType(p, MEMTYPE_HEAP) );
-
    }else{
-
      assert( sqlite3MemdebugHasType(p, (MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
-
      assert( sqlite3MemdebugNoType(p, (u8)~(MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
-
    }
+
  if( db==0 ){
+
    assert( sqlite3MemdebugNoType(p, (u8)~MEMTYPE_HEAP) );
+
    assert( sqlite3MemdebugHasType(p, MEMTYPE_HEAP) );
+
  }else if( !isLookaside(db,p) ){
+
    assert( sqlite3MemdebugHasType(p, (MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
+
    assert( sqlite3MemdebugNoType(p, (u8)~(MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
  }
#endif
  if( db ){
-
    if( ((uptr)p)<(uptr)(db->lookaside.pEnd) ){
+
    if( ((uptr)p)<(uptr)(db->lookaside.pTrueEnd) ){
#ifndef SQLITE_OMIT_TWOSIZE_LOOKASIDE
      if( ((uptr)p)>=(uptr)(db->lookaside.pMiddle) ){
        assert( sqlite3_mutex_held(db->mutex) );
@@ -28996,14 +29345,11 @@ SQLITE_PRIVATE void sqlite3DbFreeNN(sqlite3 *db, void *p){
  assert( db==0 || sqlite3_mutex_held(db->mutex) );
  assert( p!=0 );
  if( db ){
-
    if( db->pnBytesFreed ){
-
      measureAllocationSize(db, p);
-
      return;
-
    }
    if( ((uptr)p)<(uptr)(db->lookaside.pEnd) ){
#ifndef SQLITE_OMIT_TWOSIZE_LOOKASIDE
      if( ((uptr)p)>=(uptr)(db->lookaside.pMiddle) ){
        LookasideSlot *pBuf = (LookasideSlot*)p;
+
        assert( db->pnBytesFreed==0 );
#ifdef SQLITE_DEBUG
        memset(p, 0xaa, LOOKASIDE_SMALL);  /* Trash freed content */
#endif
@@ -29014,6 +29360,7 @@ SQLITE_PRIVATE void sqlite3DbFreeNN(sqlite3 *db, void *p){
#endif /* SQLITE_OMIT_TWOSIZE_LOOKASIDE */
      if( ((uptr)p)>=(uptr)(db->lookaside.pStart) ){
        LookasideSlot *pBuf = (LookasideSlot*)p;
+
        assert( db->pnBytesFreed==0 );
#ifdef SQLITE_DEBUG
        memset(p, 0xaa, db->lookaside.szTrue);  /* Trash freed content */
#endif
@@ -29022,6 +29369,10 @@ SQLITE_PRIVATE void sqlite3DbFreeNN(sqlite3 *db, void *p){
        return;
      }
    }
+
    if( db->pnBytesFreed ){
+
      measureAllocationSize(db, p);
+
      return;
+
    }
  }
  assert( sqlite3MemdebugHasType(p, (MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
  assert( sqlite3MemdebugNoType(p, (u8)~(MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
@@ -29029,6 +29380,43 @@ SQLITE_PRIVATE void sqlite3DbFreeNN(sqlite3 *db, void *p){
  sqlite3MemdebugSetType(p, MEMTYPE_HEAP);
  sqlite3_free(p);
}
+
SQLITE_PRIVATE void sqlite3DbNNFreeNN(sqlite3 *db, void *p){
+
  assert( db!=0 );
+
  assert( sqlite3_mutex_held(db->mutex) );
+
  assert( p!=0 );
+
  if( ((uptr)p)<(uptr)(db->lookaside.pEnd) ){
+
#ifndef SQLITE_OMIT_TWOSIZE_LOOKASIDE
+
    if( ((uptr)p)>=(uptr)(db->lookaside.pMiddle) ){
+
      LookasideSlot *pBuf = (LookasideSlot*)p;
+
      assert( db->pnBytesFreed==0 );
+
#ifdef SQLITE_DEBUG
+
      memset(p, 0xaa, LOOKASIDE_SMALL);  /* Trash freed content */
+
#endif
+
      pBuf->pNext = db->lookaside.pSmallFree;
+
      db->lookaside.pSmallFree = pBuf;
+
      return;
+
    }
+
#endif /* SQLITE_OMIT_TWOSIZE_LOOKASIDE */
+
    if( ((uptr)p)>=(uptr)(db->lookaside.pStart) ){
+
      LookasideSlot *pBuf = (LookasideSlot*)p;
+
      assert( db->pnBytesFreed==0 );
+
#ifdef SQLITE_DEBUG
+
      memset(p, 0xaa, db->lookaside.szTrue);  /* Trash freed content */
+
#endif
+
      pBuf->pNext = db->lookaside.pFree;
+
      db->lookaside.pFree = pBuf;
+
      return;
+
    }
+
  }
+
  if( db->pnBytesFreed ){
+
    measureAllocationSize(db, p);
+
    return;
+
  }
+
  assert( sqlite3MemdebugHasType(p, (MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
+
  assert( sqlite3MemdebugNoType(p, (u8)~(MEMTYPE_LOOKASIDE|MEMTYPE_HEAP)) );
+
  sqlite3MemdebugSetType(p, MEMTYPE_HEAP);
+
  sqlite3_free(p);
+
}
SQLITE_PRIVATE void sqlite3DbFree(sqlite3 *db, void *p){
  assert( db==0 || sqlite3_mutex_held(db->mutex) );
  if( p ) sqlite3DbFreeNN(db, p);
@@ -29364,8 +29752,13 @@ SQLITE_PRIVATE void *sqlite3OomFault(sqlite3 *db){
    }
    DisableLookaside;
    if( db->pParse ){
+
      Parse *pParse;
      sqlite3ErrorMsg(db->pParse, "out of memory");
      db->pParse->rc = SQLITE_NOMEM_BKPT;
+
      for(pParse=db->pParse->pOuterParse; pParse; pParse = pParse->pOuterParse){
+
        pParse->nErr++;
+
        pParse->rc = SQLITE_NOMEM;
+
      }
    }
  }
  return 0;
@@ -30231,8 +30624,8 @@ SQLITE_API void sqlite3_str_vappendf(
      case etSQLESCAPE:           /* %q: Escape ' characters */
      case etSQLESCAPE2:          /* %Q: Escape ' and enclose in '...' */
      case etSQLESCAPE3: {        /* %w: Escape " characters */
-
        int i, j, k, n, isnull;
-
        int needQuote;
+
        i64 i, j, k, n;
+
        int needQuote, isnull;
        char ch;
        char q = ((xtype==etSQLESCAPE3)?'"':'\'');   /* Quote character */
        char *escarg;
@@ -30312,8 +30705,14 @@ SQLITE_API void sqlite3_str_vappendf(
          sqlite3_str_appendall(pAccum, pItem->zName);
        }else if( pItem->zAlias ){
          sqlite3_str_appendall(pAccum, pItem->zAlias);
-
        }else if( ALWAYS(pItem->pSelect) ){
-
          sqlite3_str_appendf(pAccum, "SUBQUERY %u", pItem->pSelect->selId);
+
        }else{
+
          Select *pSel = pItem->pSelect;
+
          assert( pSel!=0 );
+
          if( pSel->selFlags & SF_NestedFrom ){
+
            sqlite3_str_appendf(pAccum, "(join-%u)", pSel->selId);
+
          }else{
+
            sqlite3_str_appendf(pAccum, "(subquery-%u)", pSel->selId);
+
          }
        }
        length = width = 0;
        break;
@@ -30376,7 +30775,9 @@ SQLITE_PRIVATE void sqlite3RecordErrorByteOffset(sqlite3 *db, const char *z){
** as the error offset.
*/
SQLITE_PRIVATE void sqlite3RecordErrorOffsetOfExpr(sqlite3 *db, const Expr *pExpr){
-
  while( pExpr && (ExprHasProperty(pExpr,EP_FromJoin) || pExpr->w.iOfst<=0) ){
+
  while( pExpr
+
     && (ExprHasProperty(pExpr,EP_OuterON|EP_InnerON) || pExpr->w.iOfst<=0)
+
  ){
    pExpr = pExpr->pLeft;
  }
  if( pExpr==0 ) return;
@@ -30836,40 +31237,44 @@ SQLITE_API void sqlite3_str_appendf(StrAccum *p, const char *zFormat, ...){
** Add a new subitem to the tree.  The moreToFollow flag indicates that this
** is not the last item in the tree.
*/
-
static TreeView *sqlite3TreeViewPush(TreeView *p, u8 moreToFollow){
+
static void sqlite3TreeViewPush(TreeView **pp, u8 moreToFollow){
+
  TreeView *p = *pp;
  if( p==0 ){
-
    p = sqlite3_malloc64( sizeof(*p) );
-
    if( p==0 ) return 0;
+
    *pp = p = sqlite3_malloc64( sizeof(*p) );
+
    if( p==0 ) return;
    memset(p, 0, sizeof(*p));
  }else{
    p->iLevel++;
  }
  assert( moreToFollow==0 || moreToFollow==1 );
-
  if( p->iLevel<sizeof(p->bLine) ) p->bLine[p->iLevel] = moreToFollow;
-
  return p;
+
  if( p->iLevel<(int)sizeof(p->bLine) ) p->bLine[p->iLevel] = moreToFollow;
}

/*
** Finished with one layer of the tree
*/
-
static void sqlite3TreeViewPop(TreeView *p){
+
static void sqlite3TreeViewPop(TreeView **pp){
+
  TreeView *p = *pp;
  if( p==0 ) return;
  p->iLevel--;
-
  if( p->iLevel<0 ) sqlite3_free(p);
+
  if( p->iLevel<0 ){
+
    sqlite3_free(p);
+
    *pp = 0;
+
  }
}

/*
** Generate a single line of output for the tree, with a prefix that contains
** all the appropriate tree lines
*/
-
static void sqlite3TreeViewLine(TreeView *p, const char *zFormat, ...){
+
SQLITE_PRIVATE void sqlite3TreeViewLine(TreeView *p, const char *zFormat, ...){
  va_list ap;
  int i;
  StrAccum acc;
-
  char zBuf[500];
+
  char zBuf[1000];
  sqlite3StrAccumInit(&acc, 0, zBuf, sizeof(zBuf), 0);
  if( p ){
-
    for(i=0; i<p->iLevel && i<sizeof(p->bLine)-1; i++){
+
    for(i=0; i<p->iLevel && i<(int)sizeof(p->bLine)-1; i++){
      sqlite3_str_append(&acc, p->bLine[i] ? "|   " : "    ", 4);
    }
    sqlite3_str_append(&acc, p->bLine[i] ? "|-- " : "'-- ", 4);
@@ -30890,11 +31295,58 @@ static void sqlite3TreeViewLine(TreeView *p, const char *zFormat, ...){
** Shorthand for starting a new tree item that consists of a single label
*/
static void sqlite3TreeViewItem(TreeView *p, const char *zLabel,u8 moreFollows){
-
  p = sqlite3TreeViewPush(p, moreFollows);
+
  sqlite3TreeViewPush(&p, moreFollows);
  sqlite3TreeViewLine(p, "%s", zLabel);
}

/*
+
** Show a list of Column objects in tree format.
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewColumnList(
+
  TreeView *pView,
+
  const Column *aCol,
+
  int nCol,
+
  u8 moreToFollow
+
){
+
  int i;
+
  sqlite3TreeViewPush(&pView, moreToFollow);
+
  sqlite3TreeViewLine(pView, "COLUMNS");
+
  for(i=0; i<nCol; i++){
+
    u16 flg = aCol[i].colFlags;
+
    int colMoreToFollow = i<(nCol - 1);
+
    sqlite3TreeViewPush(&pView, colMoreToFollow);
+
    sqlite3TreeViewLine(pView, 0);
+
    printf(" %s", aCol[i].zCnName);
+
    switch( aCol[i].eCType ){
+
      case COLTYPE_ANY:      printf(" ANY");        break;
+
      case COLTYPE_BLOB:     printf(" BLOB");       break;
+
      case COLTYPE_INT:      printf(" INT");        break;
+
      case COLTYPE_INTEGER:  printf(" INTEGER");    break;
+
      case COLTYPE_REAL:     printf(" REAL");       break;
+
      case COLTYPE_TEXT:     printf(" TEXT");       break;
+
      case COLTYPE_CUSTOM: {
+
        if( flg & COLFLAG_HASTYPE ){
+
          const char *z = aCol[i].zCnName;
+
          z += strlen(z)+1;
+
          printf(" X-%s", z);
+
          break;
+
        }
+
      }
+
    }
+
    if( flg & COLFLAG_PRIMKEY ) printf(" PRIMARY KEY");
+
    if( flg & COLFLAG_HIDDEN ) printf(" HIDDEN");
+
#ifdef COLFLAG_NOEXPAND
+
    if( flg & COLFLAG_NOEXPAND ) printf(" NO-EXPAND");
+
#endif
+
    if( flg ) printf(" flags=%04x", flg);
+
    printf("\n");
+
    fflush(stdout);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  sqlite3TreeViewPop(&pView);
+
}
+

+
/*
** Generate a human-readable description of a WITH clause.
*/
SQLITE_PRIVATE void sqlite3TreeViewWith(TreeView *pView, const With *pWith, u8 moreToFollow){
@@ -30907,7 +31359,7 @@ SQLITE_PRIVATE void sqlite3TreeViewWith(TreeView *pView, const With *pWith, u8 m
    sqlite3TreeViewLine(pView, "WITH (0x%p)", pWith);
  }
  if( pWith->nCte>0 ){
-
    pView = sqlite3TreeViewPush(pView, 1);
+
    sqlite3TreeViewPush(&pView, moreToFollow);
    for(i=0; i<pWith->nCte; i++){
      StrAccum x;
      char zLine[1000];
@@ -30923,6 +31375,10 @@ SQLITE_PRIVATE void sqlite3TreeViewWith(TreeView *pView, const With *pWith, u8 m
        }
        sqlite3_str_appendf(&x, ")");
      }
+
      if( pCte->eM10d!=M10d_Any ){
+
        sqlite3_str_appendf(&x, " %sMATERIALIZED",
+
           pCte->eM10d==M10d_No ? "NOT " : "");
+
      }
      if( pCte->pUse ){
        sqlite3_str_appendf(&x, " (pUse=0x%p, nUse=%d)", pCte->pUse,
                 pCte->pUse->nUse);
@@ -30930,9 +31386,9 @@ SQLITE_PRIVATE void sqlite3TreeViewWith(TreeView *pView, const With *pWith, u8 m
      sqlite3StrAccumFinish(&x);
      sqlite3TreeViewItem(pView, zLine, i<pWith->nCte-1);
      sqlite3TreeViewSelect(pView, pCte->pSelect, 0);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
    }
-
    sqlite3TreeViewPop(pView);
+
    sqlite3TreeViewPop(&pView);
  }
}

@@ -30941,10 +31397,12 @@ SQLITE_PRIVATE void sqlite3TreeViewWith(TreeView *pView, const With *pWith, u8 m
*/
SQLITE_PRIVATE void sqlite3TreeViewSrcList(TreeView *pView, const SrcList *pSrc){
  int i;
+
  if( pSrc==0 ) return;
  for(i=0; i<pSrc->nSrc; i++){
    const SrcItem *pItem = &pSrc->a[i];
    StrAccum x;
-
    char zLine[100];
+
    int n = 0;
+
    char zLine[1000];
    sqlite3StrAccumInit(&x, 0, zLine, sizeof(zLine), 0);
    x.printfFlags |= SQLITE_PRINTF_INTERNAL;
    sqlite3_str_appendf(&x, "{%d:*} %!S", pItem->iCursor, pItem);
@@ -30952,26 +31410,48 @@ SQLITE_PRIVATE void sqlite3TreeViewSrcList(TreeView *pView, const SrcList *pSrc)
      sqlite3_str_appendf(&x, " tab=%Q nCol=%d ptr=%p used=%llx",
           pItem->pTab->zName, pItem->pTab->nCol, pItem->pTab, pItem->colUsed);
    }
-
    if( pItem->fg.jointype & JT_LEFT ){
+
    if( (pItem->fg.jointype & (JT_LEFT|JT_RIGHT))==(JT_LEFT|JT_RIGHT) ){
+
      sqlite3_str_appendf(&x, " FULL-OUTER-JOIN");
+
    }else if( pItem->fg.jointype & JT_LEFT ){
      sqlite3_str_appendf(&x, " LEFT-JOIN");
+
    }else if( pItem->fg.jointype & JT_RIGHT ){
+
      sqlite3_str_appendf(&x, " RIGHT-JOIN");
    }else if( pItem->fg.jointype & JT_CROSS ){
      sqlite3_str_appendf(&x, " CROSS-JOIN");
    }
+
    if( pItem->fg.jointype & JT_LTORJ ){
+
      sqlite3_str_appendf(&x, " LTORJ");
+
    }
    if( pItem->fg.fromDDL ){
      sqlite3_str_appendf(&x, " DDL");
    }
    if( pItem->fg.isCte ){
      sqlite3_str_appendf(&x, " CteUse=0x%p", pItem->u2.pCteUse);
    }
+
    if( pItem->fg.isOn || (pItem->fg.isUsing==0 && pItem->u3.pOn!=0) ){
+
      sqlite3_str_appendf(&x, " ON");
+
    }
    sqlite3StrAccumFinish(&x);
    sqlite3TreeViewItem(pView, zLine, i<pSrc->nSrc-1);
+
    n = 0;
+
    if( pItem->pSelect ) n++;
+
    if( pItem->fg.isTabFunc ) n++;
+
    if( pItem->fg.isUsing ) n++;
+
    if( pItem->fg.isUsing ){
+
      sqlite3TreeViewIdList(pView, pItem->u3.pUsing, (--n)>0, "USING");
+
    }
    if( pItem->pSelect ){
-
      sqlite3TreeViewSelect(pView, pItem->pSelect, 0);
+
      if( pItem->pTab ){
+
        Table *pTab = pItem->pTab;
+
        sqlite3TreeViewColumnList(pView, pTab->aCol, pTab->nCol, 1);
+
      }
+
      assert( (int)pItem->fg.isNestedFrom == IsNestedFrom(pItem->pSelect) );
+
      sqlite3TreeViewSelect(pView, pItem->pSelect, (--n)>0);
    }
    if( pItem->fg.isTabFunc ){
      sqlite3TreeViewExprList(pView, pItem->u1.pFuncArg, 0, "func-args:");
    }
-
    sqlite3TreeViewPop(pView);
+
    sqlite3TreeViewPop(&pView);
  }
}

@@ -30985,11 +31465,11 @@ SQLITE_PRIVATE void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 m
    sqlite3TreeViewLine(pView, "nil-SELECT");
    return;
  }
-
  pView = sqlite3TreeViewPush(pView, moreToFollow);
+
  sqlite3TreeViewPush(&pView, moreToFollow);
  if( p->pWith ){
    sqlite3TreeViewWith(pView, p->pWith, 1);
    cnt = 1;
-
    sqlite3TreeViewPush(pView, 1);
+
    sqlite3TreeViewPush(&pView, 1);
  }
  do{
    if( p->selFlags & SF_WhereBegin ){
@@ -31003,7 +31483,7 @@ SQLITE_PRIVATE void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 m
        (int)p->nSelectRow
      );
    }
-
    if( cnt++ ) sqlite3TreeViewPop(pView);
+
    if( cnt++ ) sqlite3TreeViewPop(&pView);
    if( p->pPrior ){
      n = 1000;
    }else{
@@ -31026,24 +31506,24 @@ SQLITE_PRIVATE void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 m
#ifndef SQLITE_OMIT_WINDOWFUNC
    if( p->pWin ){
      Window *pX;
-
      pView = sqlite3TreeViewPush(pView, (n--)>0);
+
      sqlite3TreeViewPush(&pView, (n--)>0);
      sqlite3TreeViewLine(pView, "window-functions");
      for(pX=p->pWin; pX; pX=pX->pNextWin){
        sqlite3TreeViewWinFunc(pView, pX, pX->pNextWin!=0);
      }
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
    }
#endif
    if( p->pSrc && p->pSrc->nSrc ){
-
      pView = sqlite3TreeViewPush(pView, (n--)>0);
+
      sqlite3TreeViewPush(&pView, (n--)>0);
      sqlite3TreeViewLine(pView, "FROM");
      sqlite3TreeViewSrcList(pView, p->pSrc);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
    }
    if( p->pWhere ){
      sqlite3TreeViewItem(pView, "WHERE", (n--)>0);
      sqlite3TreeViewExpr(pView, p->pWhere, 0);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
    }
    if( p->pGroupBy ){
      sqlite3TreeViewExprList(pView, p->pGroupBy, (n--)>0, "GROUPBY");
@@ -31051,7 +31531,7 @@ SQLITE_PRIVATE void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 m
    if( p->pHaving ){
      sqlite3TreeViewItem(pView, "HAVING", (n--)>0);
      sqlite3TreeViewExpr(pView, p->pHaving, 0);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
    }
#ifndef SQLITE_OMIT_WINDOWFUNC
    if( p->pWinDefn ){
@@ -31060,7 +31540,7 @@ SQLITE_PRIVATE void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 m
      for(pX=p->pWinDefn; pX; pX=pX->pNextWin){
        sqlite3TreeViewWindow(pView, pX, pX->pNextWin!=0);
      }
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
    }
#endif
    if( p->pOrderBy ){
@@ -31072,9 +31552,9 @@ SQLITE_PRIVATE void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 m
      if( p->pLimit->pRight ){
        sqlite3TreeViewItem(pView, "OFFSET", (n--)>0);
        sqlite3TreeViewExpr(pView, p->pLimit->pRight, 0);
-
        sqlite3TreeViewPop(pView);
+
        sqlite3TreeViewPop(&pView);
      }
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
    }
    if( p->pPrior ){
      const char *zOp = "UNION";
@@ -31087,7 +31567,7 @@ SQLITE_PRIVATE void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 m
    }
    p = p->pPrior;
  }while( p!=0 );
-
  sqlite3TreeViewPop(pView);
+
  sqlite3TreeViewPop(&pView);
}

#ifndef SQLITE_OMIT_WINDOWFUNC
@@ -31103,24 +31583,24 @@ SQLITE_PRIVATE void sqlite3TreeViewBound(
  switch( eBound ){
    case TK_UNBOUNDED: {
      sqlite3TreeViewItem(pView, "UNBOUNDED", moreToFollow);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
      break;
    }
    case TK_CURRENT: {
      sqlite3TreeViewItem(pView, "CURRENT", moreToFollow);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
      break;
    }
    case TK_PRECEDING: {
      sqlite3TreeViewItem(pView, "PRECEDING", moreToFollow);
      sqlite3TreeViewExpr(pView, pExpr, 0);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
      break;
    }
    case TK_FOLLOWING: {
      sqlite3TreeViewItem(pView, "FOLLOWING", moreToFollow);
      sqlite3TreeViewExpr(pView, pExpr, 0);
-
      sqlite3TreeViewPop(pView);
+
      sqlite3TreeViewPop(&pView);
      break;
    }
  }
@@ -31133,12 +31613,13 @@ SQLITE_PRIVATE void sqlite3TreeViewBound(
*/
SQLITE_PRIVATE void sqlite3TreeViewWindow(TreeView *pView, const Window *pWin, u8 more){
  int nElement = 0;
+
  if( pWin==0 ) return;
  if( pWin->pFilter ){
    sqlite3TreeViewItem(pView, "FILTER", 1);
    sqlite3TreeViewExpr(pView, pWin->pFilter, 0);
-
    sqlite3TreeViewPop(pView);
+
    sqlite3TreeViewPop(&pView);
  }
-
  pView = sqlite3TreeViewPush(pView, more);
+
  sqlite3TreeViewPush(&pView, more);
  if( pWin->zName ){
    sqlite3TreeViewLine(pView, "OVER %s (%p)", pWin->zName, pWin);
  }else{
@@ -31149,9 +31630,9 @@ SQLITE_PRIVATE void sqlite3TreeViewWindow(TreeView *pView, const Window *pWin, u
  if( pWin->eFrmType ) nElement++;
  if( pWin->eExclude ) nElement++;
  if( pWin->zBase ){
-
    sqlite3TreeViewPush(pView, (--nElement)>0);
+
    sqlite3TreeViewPush(&pView, (--nElement)>0);
    sqlite3TreeViewLine(pView, "window: %s", pWin->zBase);
-
    sqlite3TreeViewPop(pView);
+
    sqlite3TreeViewPop(&pView);
  }
  if( pWin->pPartition ){
    sqlite3TreeViewExprList(pView, pWin->pPartition, nElement>0,"PARTITION-BY");
@@ -31169,7 +31650,7 @@ SQLITE_PRIVATE void sqlite3TreeViewWindow(TreeView *pView, const Window *pWin, u
    sqlite3TreeViewItem(pView, zBuf, (--nElement)>0);
    sqlite3TreeViewBound(pView, pWin->eStart, pWin->pStart, 1);
    sqlite3TreeViewBound(pView, pWin->eEnd, pWin->pEnd, 0);
-
    sqlite3TreeViewPop(pView);
+
    sqlite3TreeViewPop(&pView);
  }
  if( pWin->eExclude ){
    char zBuf[30];
@@ -31184,11 +31665,11 @@ SQLITE_PRIVATE void sqlite3TreeViewWindow(TreeView *pView, const Window *pWin, u
        zExclude = zBuf;
        break;
    }
-
    sqlite3TreeViewPush(pView, 0);
+
    sqlite3TreeViewPush(&pView, 0);
    sqlite3TreeViewLine(pView, "EXCLUDE %s", zExclude);
-
    sqlite3TreeViewPop(pView);
+
    sqlite3TreeViewPop(&pView);
  }
-
  sqlite3TreeViewPop(pView);
+
  sqlite3TreeViewPop(&pView);
}
#endif /* SQLITE_OMIT_WINDOWFUNC */

@@ -31197,11 +31678,12 @@ SQLITE_PRIVATE void sqlite3TreeViewWindow(TreeView *pView, const Window *pWin, u
** Generate a human-readable explanation for a Window Function object
*/
SQLITE_PRIVATE void sqlite3TreeViewWinFunc(TreeView *pView, const Window *pWin, u8 more){
-
  pView = sqlite3TreeViewPush(pView, more);
+
  if( pWin==0 ) return;
+
  sqlite3TreeViewPush(&pView, more);
  sqlite3TreeViewLine(pView, "WINFUNC %s(%d)",
-
                       pWin->pFunc->zName, pWin->pFunc->nArg);
+
                       pWin->pWFunc->zName, pWin->pWFunc->nArg);
  sqlite3TreeViewWindow(pView, pWin, 0);
-
  sqlite3TreeViewPop(pView);
+
  sqlite3TreeViewPop(&pView);
}
#endif /* SQLITE_OMIT_WINDOWFUNC */

@@ -31212,10 +31694,10 @@ SQLITE_PRIVATE void sqlite3TreeViewExpr(TreeView *pView, const Expr *pExpr, u8 m
  const char *zBinOp = 0;   /* Binary operator */
  const char *zUniOp = 0;   /* Unary operator */
  char zFlgs[200];
-
  pView = sqlite3TreeViewPush(pView, moreToFollow);
+
  sqlite3TreeViewPush(&pView, moreToFollow);
  if( pExpr==0 ){
    sqlite3TreeViewLine(pView, "nil");
-
    sqlite3TreeViewPop(pView);
+
    sqlite3TreeViewPop(&pView);
    return;
  }
  if( pExpr->flags || pExpr->affExpr || pExpr->vvaFlags ){
@@ -31223,8 +31705,11 @@ SQLITE_PRIVATE void sqlite3TreeViewExpr(TreeView *pView, const Expr *pExpr, u8 m
    sqlite3StrAccumInit(&x, 0, zFlgs, sizeof(zFlgs), 0);
    sqlite3_str_appendf(&x, " fg.af=%x.%c",
      pExpr->flags, pExpr->affExpr ? pExpr->affExpr : 'n');
-
    if( ExprHasProperty(pExpr, EP_FromJoin) ){
-
      sqlite3_str_appendf(&x, " iRJT=%d", pExpr->w.iRightJoinTable);
+
    if( ExprHasProperty(pExpr, EP_OuterON) ){
+
      sqlite3_str_appendf(&x, " outer.iJoin=%d", pExpr->w.iJoin);
+
    }
+
    if( ExprHasProperty(pExpr, EP_InnerON) ){
+
      sqlite3_str_appendf(&x, " inner.iJoin=%d", pExpr->w.iJoin);
    }
    if( ExprHasProperty(pExpr, EP_FromDDL) ){
      sqlite3_str_appendf(&x, " DDL");
@@ -31448,7 +31933,17 @@ SQLITE_PRIVATE void sqlite3TreeViewExpr(TreeView *pView, const Expr *pExpr, u8 m
      break;
    }
    case TK_IN: {
-
      sqlite3TreeViewLine(pView, "IN flags=0x%x", pExpr->flags);
+
      sqlite3_str *pStr = sqlite3_str_new(0);
+
      char *z;
+
      sqlite3_str_appendf(pStr, "IN flags=0x%x", pExpr->flags);
+
      if( pExpr->iTable ) sqlite3_str_appendf(pStr, " iTable=%d",pExpr->iTable);
+
      if( ExprHasProperty(pExpr, EP_Subrtn) ){
+
        sqlite3_str_appendf(pStr, " subrtn(%d,%d)",
+
            pExpr->y.sub.regReturn, pExpr->y.sub.iAddr);
+
      }
+
      z = sqlite3_str_finish(pStr);
+
      sqlite3TreeViewLine(pView, z);
+
      sqlite3_free(z);
      sqlite3TreeViewExpr(pView, pExpr->pLeft, 1);
      if( ExprUseXSelect(pExpr) ){
        sqlite3TreeViewSelect(pView, pExpr->x.pSelect, 0);
@@ -31572,7 +32067,7 @@ SQLITE_PRIVATE void sqlite3TreeViewExpr(TreeView *pView, const Expr *pExpr, u8 m
    sqlite3TreeViewLine(pView, "%s%s", zUniOp, zFlgs);
   sqlite3TreeViewExpr(pView, pExpr->pLeft, 0);
  }
-
  sqlite3TreeViewPop(pView);
+
  sqlite3TreeViewPop(&pView);
}


@@ -31594,13 +32089,25 @@ SQLITE_PRIVATE void sqlite3TreeViewBareExprList(
      int j = pList->a[i].u.x.iOrderByCol;
      char *zName = pList->a[i].zEName;
      int moreToFollow = i<pList->nExpr - 1;
-
      if( pList->a[i].eEName!=ENAME_NAME ) zName = 0;
      if( j || zName ){
-
        sqlite3TreeViewPush(pView, moreToFollow);
+
        sqlite3TreeViewPush(&pView, moreToFollow);
        moreToFollow = 0;
        sqlite3TreeViewLine(pView, 0);
        if( zName ){
-
          fprintf(stdout, "AS %s ", zName);
+
          switch( pList->a[i].fg.eEName ){
+
            default:
+
              fprintf(stdout, "AS %s ", zName);
+
              break;
+
            case ENAME_TAB:
+
              fprintf(stdout, "TABLE-ALIAS-NAME(\"%s\") ", zName);
+
              if( pList->a[i].fg.bUsed ) fprintf(stdout, "(used) ");
+
              if( pList->a[i].fg.bUsingTerm ) fprintf(stdout, "(USING-term) ");
+
              if( pList->a[i].fg.bNoExpand ) fprintf(stdout, "(NoExpand) ");
+
              break;
+
            case ENAME_SPAN:
+
              fprintf(stdout, "SPAN(\"%s\") ", zName);
+
              break;
+
          }
        }
        if( j ){
          fprintf(stdout, "iOrderByCol=%d", j);
@@ -31610,7 +32117,7 @@ SQLITE_PRIVATE void sqlite3TreeViewBareExprList(
      }
      sqlite3TreeViewExpr(pView, pList->a[i].pExpr, moreToFollow);
      if( j || zName ){
-
        sqlite3TreeViewPop(pView);
+
        sqlite3TreeViewPop(&pView);
      }
    }
  }
@@ -31621,11 +32128,378 @@ SQLITE_PRIVATE void sqlite3TreeViewExprList(
  u8 moreToFollow,
  const char *zLabel
){
-
  pView = sqlite3TreeViewPush(pView, moreToFollow);
+
  sqlite3TreeViewPush(&pView, moreToFollow);
  sqlite3TreeViewBareExprList(pView, pList, zLabel);
-
  sqlite3TreeViewPop(pView);
+
  sqlite3TreeViewPop(&pView);
}

+
/*
+
** Generate a human-readable explanation of an id-list.
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewBareIdList(
+
  TreeView *pView,
+
  const IdList *pList,
+
  const char *zLabel
+
){
+
  if( zLabel==0 || zLabel[0]==0 ) zLabel = "LIST";
+
  if( pList==0 ){
+
    sqlite3TreeViewLine(pView, "%s (empty)", zLabel);
+
  }else{
+
    int i;
+
    sqlite3TreeViewLine(pView, "%s", zLabel);
+
    for(i=0; i<pList->nId; i++){
+
      char *zName = pList->a[i].zName;
+
      int moreToFollow = i<pList->nId - 1;
+
      if( zName==0 ) zName = "(null)";
+
      sqlite3TreeViewPush(&pView, moreToFollow);
+
      sqlite3TreeViewLine(pView, 0);
+
      if( pList->eU4==EU4_NONE ){
+
        fprintf(stdout, "%s\n", zName);
+
      }else if( pList->eU4==EU4_IDX ){
+
        fprintf(stdout, "%s (%d)\n", zName, pList->a[i].u4.idx);
+
      }else{
+
        assert( pList->eU4==EU4_EXPR );
+
        if( pList->a[i].u4.pExpr==0 ){
+
          fprintf(stdout, "%s (pExpr=NULL)\n", zName);
+
        }else{
+
          fprintf(stdout, "%s\n", zName);
+
          sqlite3TreeViewPush(&pView, i<pList->nId-1);
+
          sqlite3TreeViewExpr(pView, pList->a[i].u4.pExpr, 0);
+
          sqlite3TreeViewPop(&pView);
+
        }
+
      }
+
      sqlite3TreeViewPop(&pView);
+
    }
+
  }
+
}
+
SQLITE_PRIVATE void sqlite3TreeViewIdList(
+
  TreeView *pView,
+
  const IdList *pList,
+
  u8 moreToFollow,
+
  const char *zLabel
+
){
+
  sqlite3TreeViewPush(&pView, moreToFollow);
+
  sqlite3TreeViewBareIdList(pView, pList, zLabel);
+
  sqlite3TreeViewPop(&pView);
+
}
+

+
/*
+
** Generate a human-readable explanation of a list of Upsert objects
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewUpsert(
+
  TreeView *pView,
+
  const Upsert *pUpsert,
+
  u8 moreToFollow
+
){
+
  if( pUpsert==0 ) return;
+
  sqlite3TreeViewPush(&pView, moreToFollow);
+
  while( pUpsert ){
+
    int n;
+
    sqlite3TreeViewPush(&pView, pUpsert->pNextUpsert!=0 || moreToFollow);
+
    sqlite3TreeViewLine(pView, "ON CONFLICT DO %s",
+
         pUpsert->isDoUpdate ? "UPDATE" : "NOTHING");
+
    n = (pUpsert->pUpsertSet!=0) + (pUpsert->pUpsertWhere!=0);
+
    sqlite3TreeViewExprList(pView, pUpsert->pUpsertTarget, (n--)>0, "TARGET");
+
    sqlite3TreeViewExprList(pView, pUpsert->pUpsertSet, (n--)>0, "SET");
+
    if( pUpsert->pUpsertWhere ){
+
      sqlite3TreeViewItem(pView, "WHERE", (n--)>0);
+
      sqlite3TreeViewExpr(pView, pUpsert->pUpsertWhere, 0);
+
      sqlite3TreeViewPop(&pView);
+
    }
+
    sqlite3TreeViewPop(&pView);
+
    pUpsert = pUpsert->pNextUpsert;
+
  }
+
  sqlite3TreeViewPop(&pView);
+
}
+

+
#if TREETRACE_ENABLED
+
/*
+
** Generate a human-readable diagram of the data structure that go
+
** into generating an DELETE statement.
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewDelete(
+
  const With *pWith,
+
  const SrcList *pTabList,
+
  const Expr *pWhere,
+
  const ExprList *pOrderBy,
+
  const Expr *pLimit,
+
  const Trigger *pTrigger
+
){
+
  int n = 0;
+
  TreeView *pView = 0;
+
  sqlite3TreeViewPush(&pView, 0);
+
  sqlite3TreeViewLine(pView, "DELETE");
+
  if( pWith ) n++;
+
  if( pTabList ) n++;
+
  if( pWhere ) n++;
+
  if( pOrderBy ) n++;
+
  if( pLimit ) n++;
+
  if( pTrigger ) n++;
+
  if( pWith ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewWith(pView, pWith, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pTabList ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "FROM");
+
    sqlite3TreeViewSrcList(pView, pTabList);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pWhere ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "WHERE");
+
    sqlite3TreeViewExpr(pView, pWhere, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pOrderBy ){
+
    sqlite3TreeViewExprList(pView, pOrderBy, (--n)>0, "ORDER-BY");
+
  }
+
  if( pLimit ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "LIMIT");
+
    sqlite3TreeViewExpr(pView, pLimit, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pTrigger ){
+
    sqlite3TreeViewTrigger(pView, pTrigger, (--n)>0, 1);
+
  }
+
  sqlite3TreeViewPop(&pView);
+
}
+
#endif /* TREETRACE_ENABLED */
+

+
#if TREETRACE_ENABLED
+
/*
+
** Generate a human-readable diagram of the data structure that go
+
** into generating an INSERT statement.
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewInsert(
+
  const With *pWith,
+
  const SrcList *pTabList,
+
  const IdList *pColumnList,
+
  const Select *pSelect,
+
  const ExprList *pExprList,
+
  int onError,
+
  const Upsert *pUpsert,
+
  const Trigger *pTrigger
+
){
+
  TreeView *pView = 0;
+
  int n = 0;
+
  const char *zLabel = "INSERT";
+
  switch( onError ){
+
    case OE_Replace:  zLabel = "REPLACE";             break;
+
    case OE_Ignore:   zLabel = "INSERT OR IGNORE";    break;
+
    case OE_Rollback: zLabel = "INSERT OR ROLLBACK";  break;
+
    case OE_Abort:    zLabel = "INSERT OR ABORT";     break;
+
    case OE_Fail:     zLabel = "INSERT OR FAIL";      break;
+
  }
+
  sqlite3TreeViewPush(&pView, 0);
+
  sqlite3TreeViewLine(pView, zLabel);
+
  if( pWith ) n++;
+
  if( pTabList ) n++;
+
  if( pColumnList ) n++;
+
  if( pSelect ) n++;
+
  if( pExprList ) n++;
+
  if( pUpsert ) n++;
+
  if( pTrigger ) n++;
+
  if( pWith ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewWith(pView, pWith, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pTabList ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "INTO");
+
    sqlite3TreeViewSrcList(pView, pTabList);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pColumnList ){
+
    sqlite3TreeViewIdList(pView, pColumnList, (--n)>0, "COLUMNS");
+
  }
+
  if( pSelect ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "DATA-SOURCE");
+
    sqlite3TreeViewSelect(pView, pSelect, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pExprList ){
+
    sqlite3TreeViewExprList(pView, pExprList, (--n)>0, "VALUES");
+
  }
+
  if( pUpsert ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "UPSERT");
+
    sqlite3TreeViewUpsert(pView, pUpsert, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pTrigger ){
+
    sqlite3TreeViewTrigger(pView, pTrigger, (--n)>0, 1);
+
  }
+
  sqlite3TreeViewPop(&pView);
+
}
+
#endif /* TREETRACE_ENABLED */
+

+
#if TREETRACE_ENABLED
+
/*
+
** Generate a human-readable diagram of the data structure that go
+
** into generating an UPDATE statement.
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewUpdate(
+
  const With *pWith,
+
  const SrcList *pTabList,
+
  const ExprList *pChanges,
+
  const Expr *pWhere,
+
  int onError,
+
  const ExprList *pOrderBy,
+
  const Expr *pLimit,
+
  const Upsert *pUpsert,
+
  const Trigger *pTrigger
+
){
+
  int n = 0;
+
  TreeView *pView = 0;
+
  const char *zLabel = "UPDATE";
+
  switch( onError ){
+
    case OE_Replace:  zLabel = "UPDATE OR REPLACE";   break;
+
    case OE_Ignore:   zLabel = "UPDATE OR IGNORE";    break;
+
    case OE_Rollback: zLabel = "UPDATE OR ROLLBACK";  break;
+
    case OE_Abort:    zLabel = "UPDATE OR ABORT";     break;
+
    case OE_Fail:     zLabel = "UPDATE OR FAIL";      break;
+
  }
+
  sqlite3TreeViewPush(&pView, 0);
+
  sqlite3TreeViewLine(pView, zLabel);
+
  if( pWith ) n++;
+
  if( pTabList ) n++;
+
  if( pChanges ) n++;
+
  if( pWhere ) n++;
+
  if( pOrderBy ) n++;
+
  if( pLimit ) n++;
+
  if( pUpsert ) n++;
+
  if( pTrigger ) n++;
+
  if( pWith ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewWith(pView, pWith, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pTabList ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "FROM");
+
    sqlite3TreeViewSrcList(pView, pTabList);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pChanges ){
+
    sqlite3TreeViewExprList(pView, pChanges, (--n)>0, "SET");
+
  }
+
  if( pWhere ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "WHERE");
+
    sqlite3TreeViewExpr(pView, pWhere, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pOrderBy ){
+
    sqlite3TreeViewExprList(pView, pOrderBy, (--n)>0, "ORDER-BY");
+
  }
+
  if( pLimit ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "LIMIT");
+
    sqlite3TreeViewExpr(pView, pLimit, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pUpsert ){
+
    sqlite3TreeViewPush(&pView, (--n)>0);
+
    sqlite3TreeViewLine(pView, "UPSERT");
+
    sqlite3TreeViewUpsert(pView, pUpsert, 0);
+
    sqlite3TreeViewPop(&pView);
+
  }
+
  if( pTrigger ){
+
    sqlite3TreeViewTrigger(pView, pTrigger, (--n)>0, 1);
+
  }
+
  sqlite3TreeViewPop(&pView);
+
}
+
#endif /* TREETRACE_ENABLED */
+

+
#ifndef SQLITE_OMIT_TRIGGER
+
/*
+
** Show a human-readable graph of a TriggerStep
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewTriggerStep(
+
  TreeView *pView,
+
  const TriggerStep *pStep,
+
  u8 moreToFollow,
+
  u8 showFullList
+
){
+
  int cnt = 0;
+
  if( pStep==0 ) return;
+
  sqlite3TreeViewPush(&pView,
+
      moreToFollow || (showFullList && pStep->pNext!=0));
+
  do{
+
    if( cnt++ && pStep->pNext==0 ){
+
      sqlite3TreeViewPop(&pView);
+
      sqlite3TreeViewPush(&pView, 0);
+
    }
+
    sqlite3TreeViewLine(pView, "%s", pStep->zSpan ? pStep->zSpan : "RETURNING");
+
  }while( showFullList && (pStep = pStep->pNext)!=0 );
+
  sqlite3TreeViewPop(&pView);
+
}
+

+
/*
+
** Show a human-readable graph of a Trigger
+
*/
+
SQLITE_PRIVATE void sqlite3TreeViewTrigger(
+
  TreeView *pView,
+
  const Trigger *pTrigger,
+
  u8 moreToFollow,
+
  u8 showFullList
+
){
+
  int cnt = 0;
+
  if( pTrigger==0 ) return;
+
  sqlite3TreeViewPush(&pView,
+
     moreToFollow || (showFullList && pTrigger->pNext!=0));
+
  do{
+
    if( cnt++ && pTrigger->pNext==0 ){
+
      sqlite3TreeViewPop(&pView);
+
      sqlite3TreeViewPush(&pView, 0);
+
    }
+
    sqlite3TreeViewLine(pView, "TRIGGER %s", pTrigger->zName);
+
    sqlite3TreeViewPush(&pView, 0);
+
    sqlite3TreeViewTriggerStep(pView, pTrigger->step_list, 0, 1);
+
    sqlite3TreeViewPop(&pView);
+
  }while( showFullList && (pTrigger = pTrigger->pNext)!=0 );
+
  sqlite3TreeViewPop(&pView);
+
}
+
#endif /* SQLITE_OMIT_TRIGGER */
+

+

+
/*
+
** These simplified versions of the tree-view routines omit unnecessary
+
** parameters.  These variants are intended to be used from a symbolic
+
** debugger, such as "gdb", during interactive debugging sessions.
+
**
+
** This routines are given external linkage so that they will always be
+
** accessible to the debugging, and to avoid warnings about unused
+
** functions.  But these routines only exist in debugging builds, so they
+
** do not contaminate the interface.
+
*/
+
SQLITE_PRIVATE void sqlite3ShowExpr(const Expr *p){ sqlite3TreeViewExpr(0,p,0); }
+
SQLITE_PRIVATE void sqlite3ShowExprList(const ExprList *p){ sqlite3TreeViewExprList(0,p,0,0);}
+
SQLITE_PRIVATE void sqlite3ShowIdList(const IdList *p){ sqlite3TreeViewIdList(0,p,0,0); }
+
SQLITE_PRIVATE void sqlite3ShowSrcList(const SrcList *p){ sqlite3TreeViewSrcList(0,p); }
+
SQLITE_PRIVATE void sqlite3ShowSelect(const Select *p){ sqlite3TreeViewSelect(0,p,0); }
+
SQLITE_PRIVATE void sqlite3ShowWith(const With *p){ sqlite3TreeViewWith(0,p,0); }
+
SQLITE_PRIVATE void sqlite3ShowUpsert(const Upsert *p){ sqlite3TreeViewUpsert(0,p,0); }
+
#ifndef SQLITE_OMIT_TRIGGER
+
SQLITE_PRIVATE void sqlite3ShowTriggerStep(const TriggerStep *p){
+
  sqlite3TreeViewTriggerStep(0,p,0,0);
+
}
+
SQLITE_PRIVATE void sqlite3ShowTriggerStepList(const TriggerStep *p){
+
  sqlite3TreeViewTriggerStep(0,p,0,1);
+
}
+
SQLITE_PRIVATE void sqlite3ShowTrigger(const Trigger *p){ sqlite3TreeViewTrigger(0,p,0,0); }
+
SQLITE_PRIVATE void sqlite3ShowTriggerList(const Trigger *p){ sqlite3TreeViewTrigger(0,p,0,1);}
+
#endif
+
#ifndef SQLITE_OMIT_WINDOWFUNC
+
SQLITE_PRIVATE void sqlite3ShowWindow(const Window *p){ sqlite3TreeViewWindow(0,p,0); }
+
SQLITE_PRIVATE void sqlite3ShowWinFunc(const Window *p){ sqlite3TreeViewWinFunc(0,p,0); }
+
#endif
+

#endif /* SQLITE_DEBUG */

/************** End of treeview.c ********************************************/
@@ -31654,16 +32528,41 @@ SQLITE_PRIVATE void sqlite3TreeViewExprList(
** This structure is the current state of the generator.
*/
static SQLITE_WSD struct sqlite3PrngType {
-
  unsigned char isInit;          /* True if initialized */
-
  unsigned char i, j;            /* State variables */
-
  unsigned char s[256];          /* State variables */
+
  u32 s[16];                 /* 64 bytes of chacha20 state */
+
  u8 out[64];                /* Output bytes */
+
  u8 n;                      /* Output bytes remaining */
} sqlite3Prng;

+

+
/* The RFC-7539 ChaCha20 block function
+
*/
+
#define ROTL(a,b) (((a) << (b)) | ((a) >> (32 - (b))))
+
#define QR(a, b, c, d) ( \
+
    a += b, d ^= a, d = ROTL(d,16), \
+
    c += d, b ^= c, b = ROTL(b,12), \
+
    a += b, d ^= a, d = ROTL(d, 8), \
+
    c += d, b ^= c, b = ROTL(b, 7))
+
static void chacha_block(u32 *out, const u32 *in){
+
  int i;
+
  u32 x[16];
+
  memcpy(x, in, 64);
+
  for(i=0; i<10; i++){
+
    QR(x[0], x[4], x[ 8], x[12]);
+
    QR(x[1], x[5], x[ 9], x[13]);
+
    QR(x[2], x[6], x[10], x[14]);
+
    QR(x[3], x[7], x[11], x[15]);
+
    QR(x[0], x[5], x[10], x[15]);
+
    QR(x[1], x[6], x[11], x[12]);
+
    QR(x[2], x[7], x[ 8], x[13]);
+
    QR(x[3], x[4], x[ 9], x[14]);
+
  }
+
  for(i=0; i<16; i++) out[i] = x[i]+in[i];
+
}
+

/*
** Return N random bytes.
*/
SQLITE_API void sqlite3_randomness(int N, void *pBuf){
-
  unsigned char t;
  unsigned char *zBuf = pBuf;

  /* The "wsdPrng" macro will resolve to the pseudo-random number generator
@@ -31693,53 +32592,46 @@ SQLITE_API void sqlite3_randomness(int N, void *pBuf){

  sqlite3_mutex_enter(mutex);
  if( N<=0 || pBuf==0 ){
-
    wsdPrng.isInit = 0;
+
    wsdPrng.s[0] = 0;
    sqlite3_mutex_leave(mutex);
    return;
  }

  /* Initialize the state of the random number generator once,
-
  ** the first time this routine is called.  The seed value does
-
  ** not need to contain a lot of randomness since we are not
-
  ** trying to do secure encryption or anything like that...
-
  **
-
  ** Nothing in this file or anywhere else in SQLite does any kind of
-
  ** encryption.  The RC4 algorithm is being used as a PRNG (pseudo-random
-
  ** number generator) not as an encryption device.
+
  ** the first time this routine is called.
  */
-
  if( !wsdPrng.isInit ){
+
  if( wsdPrng.s[0]==0 ){
    sqlite3_vfs *pVfs = sqlite3_vfs_find(0);
-
    int i;
-
    char k[256];
-
    wsdPrng.j = 0;
-
    wsdPrng.i = 0;
+
    static const u32 chacha20_init[] = {
+
      0x61707865, 0x3320646e, 0x79622d32, 0x6b206574
+
    };
+
    memcpy(&wsdPrng.s[0], chacha20_init, 16);
    if( NEVER(pVfs==0) ){
-
      memset(k, 0, sizeof(k));
+
      memset(&wsdPrng.s[4], 0, 44);
    }else{
-
      sqlite3OsRandomness(pVfs, 256, k);
-
    }
-
    for(i=0; i<256; i++){
-
      wsdPrng.s[i] = (u8)i;
+
      sqlite3OsRandomness(pVfs, 44, (char*)&wsdPrng.s[4]);
    }
-
    for(i=0; i<256; i++){
-
      wsdPrng.j += wsdPrng.s[i] + k[i];
-
      t = wsdPrng.s[wsdPrng.j];
-
      wsdPrng.s[wsdPrng.j] = wsdPrng.s[i];
-
      wsdPrng.s[i] = t;
-
    }
-
    wsdPrng.isInit = 1;
+
    wsdPrng.s[15] = wsdPrng.s[12];
+
    wsdPrng.s[12] = 0;
+
    wsdPrng.n = 0;
  }

  assert( N>0 );
-
  do{
-
    wsdPrng.i++;
-
    t = wsdPrng.s[wsdPrng.i];
-
    wsdPrng.j += t;
-
    wsdPrng.s[wsdPrng.i] = wsdPrng.s[wsdPrng.j];
-
    wsdPrng.s[wsdPrng.j] = t;
-
    t += wsdPrng.s[wsdPrng.i];
-
    *(zBuf++) = wsdPrng.s[t];
-
  }while( --N );
+
  while( 1 /* exit by break */ ){
+
    if( N<=wsdPrng.n ){
+
      memcpy(zBuf, &wsdPrng.out[wsdPrng.n-N], N);
+
      wsdPrng.n -= N;
+
      break;
+
    }
+
    if( wsdPrng.n>0 ){
+
      memcpy(zBuf, wsdPrng.out, wsdPrng.n);
+
      N -= wsdPrng.n;
+
      zBuf += wsdPrng.n;
+
    }
+
    wsdPrng.s[12]++;
+
    chacha_block((u32*)wsdPrng.out, wsdPrng.s);
+
    wsdPrng.n = 64;
+
  }
  sqlite3_mutex_leave(mutex);
}

@@ -32779,7 +33671,7 @@ SQLITE_PRIVATE void sqlite3ErrorMsg(Parse *pParse, const char *zFormat, ...){
  va_list ap;
  sqlite3 *db = pParse->db;
  assert( db!=0 );
-
  assert( db->pParse==pParse );
+
  assert( db->pParse==pParse || db->pParse->pToplevel==pParse );
  db->errByteOffset = -2;
  va_start(ap, zFormat);
  zMsg = sqlite3VMPrintf(db, zFormat, ap);
@@ -34592,53 +35484,53 @@ SQLITE_PRIVATE const char *sqlite3OpcodeName(int i){
    /*   0 */ "Savepoint"        OpHelp(""),
    /*   1 */ "AutoCommit"       OpHelp(""),
    /*   2 */ "Transaction"      OpHelp(""),
-
    /*   3 */ "SorterNext"       OpHelp(""),
-
    /*   4 */ "Prev"             OpHelp(""),
-
    /*   5 */ "Next"             OpHelp(""),
-
    /*   6 */ "Checkpoint"       OpHelp(""),
-
    /*   7 */ "JournalMode"      OpHelp(""),
-
    /*   8 */ "Vacuum"           OpHelp(""),
-
    /*   9 */ "VFilter"          OpHelp("iplan=r[P3] zplan='P4'"),
-
    /*  10 */ "VUpdate"          OpHelp("data=r[P3@P2]"),
-
    /*  11 */ "Goto"             OpHelp(""),
-
    /*  12 */ "Gosub"            OpHelp(""),
-
    /*  13 */ "InitCoroutine"    OpHelp(""),
-
    /*  14 */ "Yield"            OpHelp(""),
-
    /*  15 */ "MustBeInt"        OpHelp(""),
-
    /*  16 */ "Jump"             OpHelp(""),
-
    /*  17 */ "Once"             OpHelp(""),
-
    /*  18 */ "If"               OpHelp(""),
+
    /*   3 */ "Checkpoint"       OpHelp(""),
+
    /*   4 */ "JournalMode"      OpHelp(""),
+
    /*   5 */ "Vacuum"           OpHelp(""),
+
    /*   6 */ "VFilter"          OpHelp("iplan=r[P3] zplan='P4'"),
+
    /*   7 */ "VUpdate"          OpHelp("data=r[P3@P2]"),
+
    /*   8 */ "Init"             OpHelp("Start at P2"),
+
    /*   9 */ "Goto"             OpHelp(""),
+
    /*  10 */ "Gosub"            OpHelp(""),
+
    /*  11 */ "InitCoroutine"    OpHelp(""),
+
    /*  12 */ "Yield"            OpHelp(""),
+
    /*  13 */ "MustBeInt"        OpHelp(""),
+
    /*  14 */ "Jump"             OpHelp(""),
+
    /*  15 */ "Once"             OpHelp(""),
+
    /*  16 */ "If"               OpHelp(""),
+
    /*  17 */ "IfNot"            OpHelp(""),
+
    /*  18 */ "IsType"           OpHelp("if typeof(P1.P3) in P5 goto P2"),
    /*  19 */ "Not"              OpHelp("r[P2]= !r[P1]"),
-
    /*  20 */ "IfNot"            OpHelp(""),
-
    /*  21 */ "IsNullOrType"     OpHelp("if typeof(r[P1]) IN (P3,5) goto P2"),
-
    /*  22 */ "IfNullRow"        OpHelp("if P1.nullRow then r[P3]=NULL, goto P2"),
-
    /*  23 */ "SeekLT"           OpHelp("key=r[P3@P4]"),
-
    /*  24 */ "SeekLE"           OpHelp("key=r[P3@P4]"),
-
    /*  25 */ "SeekGE"           OpHelp("key=r[P3@P4]"),
-
    /*  26 */ "SeekGT"           OpHelp("key=r[P3@P4]"),
-
    /*  27 */ "IfNotOpen"        OpHelp("if( !csr[P1] ) goto P2"),
-
    /*  28 */ "IfNoHope"         OpHelp("key=r[P3@P4]"),
-
    /*  29 */ "NoConflict"       OpHelp("key=r[P3@P4]"),
-
    /*  30 */ "NotFound"         OpHelp("key=r[P3@P4]"),
-
    /*  31 */ "Found"            OpHelp("key=r[P3@P4]"),
-
    /*  32 */ "SeekRowid"        OpHelp("intkey=r[P3]"),
-
    /*  33 */ "NotExists"        OpHelp("intkey=r[P3]"),
-
    /*  34 */ "Last"             OpHelp(""),
-
    /*  35 */ "IfSmaller"        OpHelp(""),
-
    /*  36 */ "SorterSort"       OpHelp(""),
-
    /*  37 */ "Sort"             OpHelp(""),
-
    /*  38 */ "Rewind"           OpHelp(""),
-
    /*  39 */ "IdxLE"            OpHelp("key=r[P3@P4]"),
-
    /*  40 */ "IdxGT"            OpHelp("key=r[P3@P4]"),
-
    /*  41 */ "IdxLT"            OpHelp("key=r[P3@P4]"),
-
    /*  42 */ "IdxGE"            OpHelp("key=r[P3@P4]"),
+
    /*  20 */ "IfNullRow"        OpHelp("if P1.nullRow then r[P3]=NULL, goto P2"),
+
    /*  21 */ "SeekLT"           OpHelp("key=r[P3@P4]"),
+
    /*  22 */ "SeekLE"           OpHelp("key=r[P3@P4]"),
+
    /*  23 */ "SeekGE"           OpHelp("key=r[P3@P4]"),
+
    /*  24 */ "SeekGT"           OpHelp("key=r[P3@P4]"),
+
    /*  25 */ "IfNotOpen"        OpHelp("if( !csr[P1] ) goto P2"),
+
    /*  26 */ "IfNoHope"         OpHelp("key=r[P3@P4]"),
+
    /*  27 */ "NoConflict"       OpHelp("key=r[P3@P4]"),
+
    /*  28 */ "NotFound"         OpHelp("key=r[P3@P4]"),
+
    /*  29 */ "Found"            OpHelp("key=r[P3@P4]"),
+
    /*  30 */ "SeekRowid"        OpHelp("intkey=r[P3]"),
+
    /*  31 */ "NotExists"        OpHelp("intkey=r[P3]"),
+
    /*  32 */ "Last"             OpHelp(""),
+
    /*  33 */ "IfSmaller"        OpHelp(""),
+
    /*  34 */ "SorterSort"       OpHelp(""),
+
    /*  35 */ "Sort"             OpHelp(""),
+
    /*  36 */ "Rewind"           OpHelp(""),
+
    /*  37 */ "SorterNext"       OpHelp(""),
+
    /*  38 */ "Prev"             OpHelp(""),
+
    /*  39 */ "Next"             OpHelp(""),
+
    /*  40 */ "IdxLE"            OpHelp("key=r[P3@P4]"),
+
    /*  41 */ "IdxGT"            OpHelp("key=r[P3@P4]"),
+
    /*  42 */ "IdxLT"            OpHelp("key=r[P3@P4]"),
    /*  43 */ "Or"               OpHelp("r[P3]=(r[P1] || r[P2])"),
    /*  44 */ "And"              OpHelp("r[P3]=(r[P1] && r[P2])"),
-
    /*  45 */ "RowSetRead"       OpHelp("r[P3]=rowset(P1)"),
-
    /*  46 */ "RowSetTest"       OpHelp("if r[P3] in rowset(P1) goto P2"),
-
    /*  47 */ "Program"          OpHelp(""),
-
    /*  48 */ "FkIfZero"         OpHelp("if fkctr[P1]==0 goto P2"),
-
    /*  49 */ "IfPos"            OpHelp("if r[P1]>0 then r[P1]-=P3, goto P2"),
+
    /*  45 */ "IdxGE"            OpHelp("key=r[P3@P4]"),
+
    /*  46 */ "RowSetRead"       OpHelp("r[P3]=rowset(P1)"),
+
    /*  47 */ "RowSetTest"       OpHelp("if r[P3] in rowset(P1) goto P2"),
+
    /*  48 */ "Program"          OpHelp(""),
+
    /*  49 */ "FkIfZero"         OpHelp("if fkctr[P1]==0 goto P2"),
    /*  50 */ "IsNull"           OpHelp("if r[P1]==NULL goto P2"),
    /*  51 */ "NotNull"          OpHelp("if r[P1]!=NULL goto P2"),
    /*  52 */ "Ne"               OpHelp("IF r[P3]!=r[P1]"),
@@ -34648,12 +35540,12 @@ SQLITE_PRIVATE const char *sqlite3OpcodeName(int i){
    /*  56 */ "Lt"               OpHelp("IF r[P3]<r[P1]"),
    /*  57 */ "Ge"               OpHelp("IF r[P3]>=r[P1]"),
    /*  58 */ "ElseEq"           OpHelp(""),
-
    /*  59 */ "IfNotZero"        OpHelp("if r[P1]!=0 then r[P1]--, goto P2"),
-
    /*  60 */ "DecrJumpZero"     OpHelp("if (--r[P1])==0 goto P2"),
-
    /*  61 */ "IncrVacuum"       OpHelp(""),
-
    /*  62 */ "VNext"            OpHelp(""),
-
    /*  63 */ "Filter"           OpHelp("if key(P3@P4) not in filter(P1) goto P2"),
-
    /*  64 */ "Init"             OpHelp("Start at P2"),
+
    /*  59 */ "IfPos"            OpHelp("if r[P1]>0 then r[P1]-=P3, goto P2"),
+
    /*  60 */ "IfNotZero"        OpHelp("if r[P1]!=0 then r[P1]--, goto P2"),
+
    /*  61 */ "DecrJumpZero"     OpHelp("if (--r[P1])==0 goto P2"),
+
    /*  62 */ "IncrVacuum"       OpHelp(""),
+
    /*  63 */ "VNext"            OpHelp(""),
+
    /*  64 */ "Filter"           OpHelp("if key(P3@P4) not in filter(P1) goto P2"),
    /*  65 */ "PureFunc"         OpHelp("r[P3]=func(r[P2@NP])"),
    /*  66 */ "Function"         OpHelp("r[P3]=func(r[P2@NP])"),
    /*  67 */ "Return"           OpHelp(""),
@@ -34663,34 +35555,34 @@ SQLITE_PRIVATE const char *sqlite3OpcodeName(int i){
    /*  71 */ "Integer"          OpHelp("r[P2]=P1"),
    /*  72 */ "Int64"            OpHelp("r[P2]=P4"),
    /*  73 */ "String"           OpHelp("r[P2]='P4' (len=P1)"),
-
    /*  74 */ "Null"             OpHelp("r[P2..P3]=NULL"),
-
    /*  75 */ "SoftNull"         OpHelp("r[P1]=NULL"),
-
    /*  76 */ "Blob"             OpHelp("r[P2]=P4 (len=P1)"),
-
    /*  77 */ "Variable"         OpHelp("r[P2]=parameter(P1,P4)"),
-
    /*  78 */ "Move"             OpHelp("r[P2@P3]=r[P1@P3]"),
-
    /*  79 */ "Copy"             OpHelp("r[P2@P3+1]=r[P1@P3+1]"),
-
    /*  80 */ "SCopy"            OpHelp("r[P2]=r[P1]"),
-
    /*  81 */ "IntCopy"          OpHelp("r[P2]=r[P1]"),
-
    /*  82 */ "FkCheck"          OpHelp(""),
-
    /*  83 */ "ResultRow"        OpHelp("output=r[P1@P2]"),
-
    /*  84 */ "CollSeq"          OpHelp(""),
-
    /*  85 */ "AddImm"           OpHelp("r[P1]=r[P1]+P2"),
-
    /*  86 */ "RealAffinity"     OpHelp(""),
-
    /*  87 */ "Cast"             OpHelp("affinity(r[P1])"),
-
    /*  88 */ "Permutation"      OpHelp(""),
-
    /*  89 */ "Compare"          OpHelp("r[P1@P3] <-> r[P2@P3]"),
-
    /*  90 */ "IsTrue"           OpHelp("r[P2] = coalesce(r[P1]==TRUE,P3) ^ P4"),
-
    /*  91 */ "ZeroOrNull"       OpHelp("r[P2] = 0 OR NULL"),
-
    /*  92 */ "Offset"           OpHelp("r[P3] = sqlite_offset(P1)"),
-
    /*  93 */ "Column"           OpHelp("r[P3]=PX"),
-
    /*  94 */ "TypeCheck"        OpHelp("typecheck(r[P1@P2])"),
-
    /*  95 */ "Affinity"         OpHelp("affinity(r[P1@P2])"),
-
    /*  96 */ "MakeRecord"       OpHelp("r[P3]=mkrec(r[P1@P2])"),
-
    /*  97 */ "Count"            OpHelp("r[P2]=count()"),
-
    /*  98 */ "ReadCookie"       OpHelp(""),
-
    /*  99 */ "SetCookie"        OpHelp(""),
-
    /* 100 */ "ReopenIdx"        OpHelp("root=P2 iDb=P3"),
-
    /* 101 */ "OpenRead"         OpHelp("root=P2 iDb=P3"),
+
    /*  74 */ "BeginSubrtn"      OpHelp("r[P2]=NULL"),
+
    /*  75 */ "Null"             OpHelp("r[P2..P3]=NULL"),
+
    /*  76 */ "SoftNull"         OpHelp("r[P1]=NULL"),
+
    /*  77 */ "Blob"             OpHelp("r[P2]=P4 (len=P1)"),
+
    /*  78 */ "Variable"         OpHelp("r[P2]=parameter(P1,P4)"),
+
    /*  79 */ "Move"             OpHelp("r[P2@P3]=r[P1@P3]"),
+
    /*  80 */ "Copy"             OpHelp("r[P2@P3+1]=r[P1@P3+1]"),
+
    /*  81 */ "SCopy"            OpHelp("r[P2]=r[P1]"),
+
    /*  82 */ "IntCopy"          OpHelp("r[P2]=r[P1]"),
+
    /*  83 */ "FkCheck"          OpHelp(""),
+
    /*  84 */ "ResultRow"        OpHelp("output=r[P1@P2]"),
+
    /*  85 */ "CollSeq"          OpHelp(""),
+
    /*  86 */ "AddImm"           OpHelp("r[P1]=r[P1]+P2"),
+
    /*  87 */ "RealAffinity"     OpHelp(""),
+
    /*  88 */ "Cast"             OpHelp("affinity(r[P1])"),
+
    /*  89 */ "Permutation"      OpHelp(""),
+
    /*  90 */ "Compare"          OpHelp("r[P1@P3] <-> r[P2@P3]"),
+
    /*  91 */ "IsTrue"           OpHelp("r[P2] = coalesce(r[P1]==TRUE,P3) ^ P4"),
+
    /*  92 */ "ZeroOrNull"       OpHelp("r[P2] = 0 OR NULL"),
+
    /*  93 */ "Offset"           OpHelp("r[P3] = sqlite_offset(P1)"),
+
    /*  94 */ "Column"           OpHelp("r[P3]=PX cursor P1 column P2"),
+
    /*  95 */ "TypeCheck"        OpHelp("typecheck(r[P1@P2])"),
+
    /*  96 */ "Affinity"         OpHelp("affinity(r[P1@P2])"),
+
    /*  97 */ "MakeRecord"       OpHelp("r[P3]=mkrec(r[P1@P2])"),
+
    /*  98 */ "Count"            OpHelp("r[P2]=count()"),
+
    /*  99 */ "ReadCookie"       OpHelp(""),
+
    /* 100 */ "SetCookie"        OpHelp(""),
+
    /* 101 */ "ReopenIdx"        OpHelp("root=P2 iDb=P3"),
    /* 102 */ "BitAnd"           OpHelp("r[P3]=r[P1]&r[P2]"),
    /* 103 */ "BitOr"            OpHelp("r[P3]=r[P1]|r[P2]"),
    /* 104 */ "ShiftLeft"        OpHelp("r[P3]=r[P2]<<r[P1]"),
@@ -34701,85 +35593,1062 @@ SQLITE_PRIVATE const char *sqlite3OpcodeName(int i){
    /* 109 */ "Divide"           OpHelp("r[P3]=r[P2]/r[P1]"),
    /* 110 */ "Remainder"        OpHelp("r[P3]=r[P2]%r[P1]"),
    /* 111 */ "Concat"           OpHelp("r[P3]=r[P2]+r[P1]"),
-
    /* 112 */ "OpenWrite"        OpHelp("root=P2 iDb=P3"),
-
    /* 113 */ "OpenDup"          OpHelp(""),
+
    /* 112 */ "OpenRead"         OpHelp("root=P2 iDb=P3"),
+
    /* 113 */ "OpenWrite"        OpHelp("root=P2 iDb=P3"),
    /* 114 */ "BitNot"           OpHelp("r[P2]= ~r[P1]"),
-
    /* 115 */ "OpenAutoindex"    OpHelp("nColumn=P2"),
-
    /* 116 */ "OpenEphemeral"    OpHelp("nColumn=P2"),
+
    /* 115 */ "OpenDup"          OpHelp(""),
+
    /* 116 */ "OpenAutoindex"    OpHelp("nColumn=P2"),
    /* 117 */ "String8"          OpHelp("r[P2]='P4'"),
-
    /* 118 */ "SorterOpen"       OpHelp(""),
-
    /* 119 */ "SequenceTest"     OpHelp("if( cursor[P1].ctr++ ) pc = P2"),
-
    /* 120 */ "OpenPseudo"       OpHelp("P3 columns in r[P2]"),
-
    /* 121 */ "Close"            OpHelp(""),
-
    /* 122 */ "ColumnsUsed"      OpHelp(""),
-
    /* 123 */ "SeekScan"         OpHelp("Scan-ahead up to P1 rows"),
-
    /* 124 */ "SeekHit"          OpHelp("set P2<=seekHit<=P3"),
-
    /* 125 */ "Sequence"         OpHelp("r[P2]=cursor[P1].ctr++"),
-
    /* 126 */ "NewRowid"         OpHelp("r[P2]=rowid"),
-
    /* 127 */ "Insert"           OpHelp("intkey=r[P3] data=r[P2]"),
-
    /* 128 */ "RowCell"          OpHelp(""),
-
    /* 129 */ "Delete"           OpHelp(""),
-
    /* 130 */ "ResetCount"       OpHelp(""),
-
    /* 131 */ "SorterCompare"    OpHelp("if key(P1)!=trim(r[P3],P4) goto P2"),
-
    /* 132 */ "SorterData"       OpHelp("r[P2]=data"),
-
    /* 133 */ "RowData"          OpHelp("r[P2]=data"),
-
    /* 134 */ "Rowid"            OpHelp("r[P2]=rowid"),
-
    /* 135 */ "NullRow"          OpHelp(""),
-
    /* 136 */ "SeekEnd"          OpHelp(""),
-
    /* 137 */ "IdxInsert"        OpHelp("key=r[P2]"),
-
    /* 138 */ "SorterInsert"     OpHelp("key=r[P2]"),
-
    /* 139 */ "IdxDelete"        OpHelp("key=r[P2@P3]"),
-
    /* 140 */ "DeferredSeek"     OpHelp("Move P3 to P1.rowid if needed"),
-
    /* 141 */ "IdxRowid"         OpHelp("r[P2]=rowid"),
-
    /* 142 */ "FinishSeek"       OpHelp(""),
-
    /* 143 */ "Destroy"          OpHelp(""),
-
    /* 144 */ "Clear"            OpHelp(""),
-
    /* 145 */ "ResetSorter"      OpHelp(""),
-
    /* 146 */ "CreateBtree"      OpHelp("r[P2]=root iDb=P1 flags=P3"),
-
    /* 147 */ "SqlExec"          OpHelp(""),
-
    /* 148 */ "ParseSchema"      OpHelp(""),
-
    /* 149 */ "LoadAnalysis"     OpHelp(""),
-
    /* 150 */ "DropTable"        OpHelp(""),
-
    /* 151 */ "DropIndex"        OpHelp(""),
-
    /* 152 */ "DropTrigger"      OpHelp(""),
+
    /* 118 */ "OpenEphemeral"    OpHelp("nColumn=P2"),
+
    /* 119 */ "SorterOpen"       OpHelp(""),
+
    /* 120 */ "SequenceTest"     OpHelp("if( cursor[P1].ctr++ ) pc = P2"),
+
    /* 121 */ "OpenPseudo"       OpHelp("P3 columns in r[P2]"),
+
    /* 122 */ "Close"            OpHelp(""),
+
    /* 123 */ "ColumnsUsed"      OpHelp(""),
+
    /* 124 */ "SeekScan"         OpHelp("Scan-ahead up to P1 rows"),
+
    /* 125 */ "SeekHit"          OpHelp("set P2<=seekHit<=P3"),
+
    /* 126 */ "Sequence"         OpHelp("r[P2]=cursor[P1].ctr++"),
+
    /* 127 */ "NewRowid"         OpHelp("r[P2]=rowid"),
+
    /* 128 */ "Insert"           OpHelp("intkey=r[P3] data=r[P2]"),
+
    /* 129 */ "RowCell"          OpHelp(""),
+
    /* 130 */ "Delete"           OpHelp(""),
+
    /* 131 */ "ResetCount"       OpHelp(""),
+
    /* 132 */ "SorterCompare"    OpHelp("if key(P1)!=trim(r[P3],P4) goto P2"),
+
    /* 133 */ "SorterData"       OpHelp("r[P2]=data"),
+
    /* 134 */ "RowData"          OpHelp("r[P2]=data"),
+
    /* 135 */ "Rowid"            OpHelp("r[P2]=PX rowid of P1"),
+
    /* 136 */ "NullRow"          OpHelp(""),
+
    /* 137 */ "SeekEnd"          OpHelp(""),
+
    /* 138 */ "IdxInsert"        OpHelp("key=r[P2]"),
+
    /* 139 */ "SorterInsert"     OpHelp("key=r[P2]"),
+
    /* 140 */ "IdxDelete"        OpHelp("key=r[P2@P3]"),
+
    /* 141 */ "DeferredSeek"     OpHelp("Move P3 to P1.rowid if needed"),
+
    /* 142 */ "IdxRowid"         OpHelp("r[P2]=rowid"),
+
    /* 143 */ "FinishSeek"       OpHelp(""),
+
    /* 144 */ "Destroy"          OpHelp(""),
+
    /* 145 */ "Clear"            OpHelp(""),
+
    /* 146 */ "ResetSorter"      OpHelp(""),
+
    /* 147 */ "CreateBtree"      OpHelp("r[P2]=root iDb=P1 flags=P3"),
+
    /* 148 */ "SqlExec"          OpHelp(""),
+
    /* 149 */ "ParseSchema"      OpHelp(""),
+
    /* 150 */ "LoadAnalysis"     OpHelp(""),
+
    /* 151 */ "DropTable"        OpHelp(""),
+
    /* 152 */ "DropIndex"        OpHelp(""),
    /* 153 */ "Real"             OpHelp("r[P2]=P4"),
-
    /* 154 */ "IntegrityCk"      OpHelp(""),
-
    /* 155 */ "RowSetAdd"        OpHelp("rowset(P1)=r[P2]"),
-
    /* 156 */ "Param"            OpHelp(""),
-
    /* 157 */ "FkCounter"        OpHelp("fkctr[P1]+=P2"),
-
    /* 158 */ "MemMax"           OpHelp("r[P1]=max(r[P1],r[P2])"),
-
    /* 159 */ "OffsetLimit"      OpHelp("if r[P1]>0 then r[P2]=r[P1]+max(0,r[P3]) else r[P2]=(-1)"),
-
    /* 160 */ "AggInverse"       OpHelp("accum=r[P3] inverse(r[P2@P5])"),
-
    /* 161 */ "AggStep"          OpHelp("accum=r[P3] step(r[P2@P5])"),
-
    /* 162 */ "AggStep1"         OpHelp("accum=r[P3] step(r[P2@P5])"),
-
    /* 163 */ "AggValue"         OpHelp("r[P3]=value N=P2"),
-
    /* 164 */ "AggFinal"         OpHelp("accum=r[P1] N=P2"),
-
    /* 165 */ "Expire"           OpHelp(""),
-
    /* 166 */ "CursorLock"       OpHelp(""),
-
    /* 167 */ "CursorUnlock"     OpHelp(""),
-
    /* 168 */ "TableLock"        OpHelp("iDb=P1 root=P2 write=P3"),
-
    /* 169 */ "VBegin"           OpHelp(""),
-
    /* 170 */ "VCreate"          OpHelp(""),
-
    /* 171 */ "VDestroy"         OpHelp(""),
-
    /* 172 */ "VOpen"            OpHelp(""),
-
    /* 173 */ "VInitIn"          OpHelp("r[P2]=ValueList(P1,P3)"),
-
    /* 174 */ "VColumn"          OpHelp("r[P3]=vcolumn(P2)"),
-
    /* 175 */ "VRename"          OpHelp(""),
-
    /* 176 */ "Pagecount"        OpHelp(""),
-
    /* 177 */ "MaxPgcnt"         OpHelp(""),
-
    /* 178 */ "FilterAdd"        OpHelp("filter(P1) += key(P3@P4)"),
-
    /* 179 */ "Trace"            OpHelp(""),
-
    /* 180 */ "CursorHint"       OpHelp(""),
-
    /* 181 */ "ReleaseReg"       OpHelp("release r[P1@P2] mask P3"),
-
    /* 182 */ "Noop"             OpHelp(""),
-
    /* 183 */ "Explain"          OpHelp(""),
-
    /* 184 */ "Abortable"        OpHelp(""),
+
    /* 154 */ "DropTrigger"      OpHelp(""),
+
    /* 155 */ "IntegrityCk"      OpHelp(""),
+
    /* 156 */ "RowSetAdd"        OpHelp("rowset(P1)=r[P2]"),
+
    /* 157 */ "Param"            OpHelp(""),
+
    /* 158 */ "FkCounter"        OpHelp("fkctr[P1]+=P2"),
+
    /* 159 */ "MemMax"           OpHelp("r[P1]=max(r[P1],r[P2])"),
+
    /* 160 */ "OffsetLimit"      OpHelp("if r[P1]>0 then r[P2]=r[P1]+max(0,r[P3]) else r[P2]=(-1)"),
+
    /* 161 */ "AggInverse"       OpHelp("accum=r[P3] inverse(r[P2@P5])"),
+
    /* 162 */ "AggStep"          OpHelp("accum=r[P3] step(r[P2@P5])"),
+
    /* 163 */ "AggStep1"         OpHelp("accum=r[P3] step(r[P2@P5])"),
+
    /* 164 */ "AggValue"         OpHelp("r[P3]=value N=P2"),
+
    /* 165 */ "AggFinal"         OpHelp("accum=r[P1] N=P2"),
+
    /* 166 */ "Expire"           OpHelp(""),
+
    /* 167 */ "CursorLock"       OpHelp(""),
+
    /* 168 */ "CursorUnlock"     OpHelp(""),
+
    /* 169 */ "TableLock"        OpHelp("iDb=P1 root=P2 write=P3"),
+
    /* 170 */ "VBegin"           OpHelp(""),
+
    /* 171 */ "VCreate"          OpHelp(""),
+
    /* 172 */ "VDestroy"         OpHelp(""),
+
    /* 173 */ "VOpen"            OpHelp(""),
+
    /* 174 */ "VInitIn"          OpHelp("r[P2]=ValueList(P1,P3)"),
+
    /* 175 */ "VColumn"          OpHelp("r[P3]=vcolumn(P2)"),
+
    /* 176 */ "VRename"          OpHelp(""),
+
    /* 177 */ "Pagecount"        OpHelp(""),
+
    /* 178 */ "MaxPgcnt"         OpHelp(""),
+
    /* 179 */ "ClrSubtype"       OpHelp("r[P1].subtype = 0"),
+
    /* 180 */ "FilterAdd"        OpHelp("filter(P1) += key(P3@P4)"),
+
    /* 181 */ "Trace"            OpHelp(""),
+
    /* 182 */ "CursorHint"       OpHelp(""),
+
    /* 183 */ "ReleaseReg"       OpHelp("release r[P1@P2] mask P3"),
+
    /* 184 */ "Noop"             OpHelp(""),
+
    /* 185 */ "Explain"          OpHelp(""),
+
    /* 186 */ "Abortable"        OpHelp(""),
  };
  return azName[i];
}
#endif

/************** End of opcodes.c *********************************************/
+
/************** Begin file os_kv.c *******************************************/
+
/*
+
** 2022-09-06
+
**
+
** The author disclaims copyright to this source code.  In place of
+
** a legal notice, here is a blessing:
+
**
+
**    May you do good and not evil.
+
**    May you find forgiveness for yourself and forgive others.
+
**    May you share freely, never taking more than you give.
+
**
+
******************************************************************************
+
**
+
** This file contains an experimental VFS layer that operates on a
+
** Key/Value storage engine where both keys and values must be pure
+
** text.
+
*/
+
/* #include <sqliteInt.h> */
+
#if SQLITE_OS_KV || (SQLITE_OS_UNIX && defined(SQLITE_OS_KV_OPTIONAL))
+

+
/*****************************************************************************
+
** Debugging logic
+
*/
+

+
/* SQLITE_KV_TRACE() is used for tracing calls to kvstorage routines. */
+
#if 0
+
#define SQLITE_KV_TRACE(X)  printf X
+
#else
+
#define SQLITE_KV_TRACE(X)
+
#endif
+

+
/* SQLITE_KV_LOG() is used for tracing calls to the VFS interface */
+
#if 0
+
#define SQLITE_KV_LOG(X)  printf X
+
#else
+
#define SQLITE_KV_LOG(X)
+
#endif
+

+

+
/*
+
** Forward declaration of objects used by this VFS implementation
+
*/
+
typedef struct KVVfsFile KVVfsFile;
+

+
/* A single open file.  There are only two files represented by this
+
** VFS - the database and the rollback journal.
+
*/
+
struct KVVfsFile {
+
  sqlite3_file base;              /* IO methods */
+
  const char *zClass;             /* Storage class */
+
  int isJournal;                  /* True if this is a journal file */
+
  unsigned int nJrnl;             /* Space allocated for aJrnl[] */
+
  char *aJrnl;                    /* Journal content */
+
  int szPage;                     /* Last known page size */
+
  sqlite3_int64 szDb;             /* Database file size.  -1 means unknown */
+
};
+

+
/*
+
** Methods for KVVfsFile
+
*/
+
static int kvvfsClose(sqlite3_file*);
+
static int kvvfsReadDb(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
+
static int kvvfsReadJrnl(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
+
static int kvvfsWriteDb(sqlite3_file*,const void*,int iAmt, sqlite3_int64);
+
static int kvvfsWriteJrnl(sqlite3_file*,const void*,int iAmt, sqlite3_int64);
+
static int kvvfsTruncateDb(sqlite3_file*, sqlite3_int64 size);
+
static int kvvfsTruncateJrnl(sqlite3_file*, sqlite3_int64 size);
+
static int kvvfsSyncDb(sqlite3_file*, int flags);
+
static int kvvfsSyncJrnl(sqlite3_file*, int flags);
+
static int kvvfsFileSizeDb(sqlite3_file*, sqlite3_int64 *pSize);
+
static int kvvfsFileSizeJrnl(sqlite3_file*, sqlite3_int64 *pSize);
+
static int kvvfsLock(sqlite3_file*, int);
+
static int kvvfsUnlock(sqlite3_file*, int);
+
static int kvvfsCheckReservedLock(sqlite3_file*, int *pResOut);
+
static int kvvfsFileControlDb(sqlite3_file*, int op, void *pArg);
+
static int kvvfsFileControlJrnl(sqlite3_file*, int op, void *pArg);
+
static int kvvfsSectorSize(sqlite3_file*);
+
static int kvvfsDeviceCharacteristics(sqlite3_file*);
+

+
/*
+
** Methods for sqlite3_vfs
+
*/
+
static int kvvfsOpen(sqlite3_vfs*, const char *, sqlite3_file*, int , int *);
+
static int kvvfsDelete(sqlite3_vfs*, const char *zName, int syncDir);
+
static int kvvfsAccess(sqlite3_vfs*, const char *zName, int flags, int *);
+
static int kvvfsFullPathname(sqlite3_vfs*, const char *zName, int, char *zOut);
+
static void *kvvfsDlOpen(sqlite3_vfs*, const char *zFilename);
+
static int kvvfsRandomness(sqlite3_vfs*, int nByte, char *zOut);
+
static int kvvfsSleep(sqlite3_vfs*, int microseconds);
+
static int kvvfsCurrentTime(sqlite3_vfs*, double*);
+
static int kvvfsCurrentTimeInt64(sqlite3_vfs*, sqlite3_int64*);
+

+
static sqlite3_vfs sqlite3OsKvvfsObject = {
+
  1,                              /* iVersion */
+
  sizeof(KVVfsFile),              /* szOsFile */
+
  1024,                           /* mxPathname */
+
  0,                              /* pNext */
+
  "kvvfs",                        /* zName */
+
  0,                              /* pAppData */
+
  kvvfsOpen,                      /* xOpen */
+
  kvvfsDelete,                    /* xDelete */
+
  kvvfsAccess,                    /* xAccess */
+
  kvvfsFullPathname,              /* xFullPathname */
+
  kvvfsDlOpen,                    /* xDlOpen */
+
  0,                              /* xDlError */
+
  0,                              /* xDlSym */
+
  0,                              /* xDlClose */
+
  kvvfsRandomness,                /* xRandomness */
+
  kvvfsSleep,                     /* xSleep */
+
  kvvfsCurrentTime,               /* xCurrentTime */
+
  0,                              /* xGetLastError */
+
  kvvfsCurrentTimeInt64           /* xCurrentTimeInt64 */
+
};
+

+
/* Methods for sqlite3_file objects referencing a database file
+
*/
+
static sqlite3_io_methods kvvfs_db_io_methods = {
+
  1,                              /* iVersion */
+
  kvvfsClose,                     /* xClose */
+
  kvvfsReadDb,                    /* xRead */
+
  kvvfsWriteDb,                   /* xWrite */
+
  kvvfsTruncateDb,                /* xTruncate */
+
  kvvfsSyncDb,                    /* xSync */
+
  kvvfsFileSizeDb,                /* xFileSize */
+
  kvvfsLock,                      /* xLock */
+
  kvvfsUnlock,                    /* xUnlock */
+
  kvvfsCheckReservedLock,         /* xCheckReservedLock */
+
  kvvfsFileControlDb,             /* xFileControl */
+
  kvvfsSectorSize,                /* xSectorSize */
+
  kvvfsDeviceCharacteristics,     /* xDeviceCharacteristics */
+
  0,                              /* xShmMap */
+
  0,                              /* xShmLock */
+
  0,                              /* xShmBarrier */
+
  0,                              /* xShmUnmap */
+
  0,                              /* xFetch */
+
  0                               /* xUnfetch */
+
};
+

+
/* Methods for sqlite3_file objects referencing a rollback journal
+
*/
+
static sqlite3_io_methods kvvfs_jrnl_io_methods = {
+
  1,                              /* iVersion */
+
  kvvfsClose,                     /* xClose */
+
  kvvfsReadJrnl,                  /* xRead */
+
  kvvfsWriteJrnl,                 /* xWrite */
+
  kvvfsTruncateJrnl,              /* xTruncate */
+
  kvvfsSyncJrnl,                  /* xSync */
+
  kvvfsFileSizeJrnl,              /* xFileSize */
+
  kvvfsLock,                      /* xLock */
+
  kvvfsUnlock,                    /* xUnlock */
+
  kvvfsCheckReservedLock,         /* xCheckReservedLock */
+
  kvvfsFileControlJrnl,           /* xFileControl */
+
  kvvfsSectorSize,                /* xSectorSize */
+
  kvvfsDeviceCharacteristics,     /* xDeviceCharacteristics */
+
  0,                              /* xShmMap */
+
  0,                              /* xShmLock */
+
  0,                              /* xShmBarrier */
+
  0,                              /* xShmUnmap */
+
  0,                              /* xFetch */
+
  0                               /* xUnfetch */
+
};
+

+
/****** Storage subsystem **************************************************/
+
#include <sys/types.h>
+
#include <sys/stat.h>
+
#include <unistd.h>
+

+
/* Forward declarations for the low-level storage engine
+
*/
+
static int kvstorageWrite(const char*, const char *zKey, const char *zData);
+
static int kvstorageDelete(const char*, const char *zKey);
+
static int kvstorageRead(const char*, const char *zKey, char *zBuf, int nBuf);
+
#define KVSTORAGE_KEY_SZ  32
+

+
/* Expand the key name with an appropriate prefix and put the result
+
** zKeyOut[].  The zKeyOut[] buffer is assumed to hold at least
+
** KVSTORAGE_KEY_SZ bytes.
+
*/
+
static void kvstorageMakeKey(
+
  const char *zClass,
+
  const char *zKeyIn,
+
  char *zKeyOut
+
){
+
  sqlite3_snprintf(KVSTORAGE_KEY_SZ, zKeyOut, "kvvfs-%s-%s", zClass, zKeyIn);
+
}
+

+
/* Write content into a key.  zClass is the particular namespace of the
+
** underlying key/value store to use - either "local" or "session".
+
**
+
** Both zKey and zData are zero-terminated pure text strings.
+
**
+
** Return the number of errors.
+
*/
+
static int kvstorageWrite(
+
  const char *zClass,
+
  const char *zKey,
+
  const char *zData
+
){
+
  FILE *fd;
+
  char zXKey[KVSTORAGE_KEY_SZ];
+
  kvstorageMakeKey(zClass, zKey, zXKey);
+
  fd = fopen(zXKey, "wb");
+
  if( fd ){
+
    SQLITE_KV_TRACE(("KVVFS-WRITE  %-15s (%d) %.50s%s\n", zXKey,
+
                 (int)strlen(zData), zData,
+
                 strlen(zData)>50 ? "..." : ""));
+
    fputs(zData, fd);
+
    fclose(fd);
+
    return 0;
+
  }else{
+
    return 1;
+
  }
+
}
+

+
/* Delete a key (with its corresponding data) from the key/value
+
** namespace given by zClass.  If the key does not previously exist,
+
** this routine is a no-op.
+
*/
+
static int kvstorageDelete(const char *zClass, const char *zKey){
+
  char zXKey[KVSTORAGE_KEY_SZ];
+
  kvstorageMakeKey(zClass, zKey, zXKey);
+
  unlink(zXKey);
+
  SQLITE_KV_TRACE(("KVVFS-DELETE %-15s\n", zXKey));
+
  return 0;
+
}
+

+
/* Read the value associated with a zKey from the key/value namespace given
+
** by zClass and put the text data associated with that key in the first
+
** nBuf bytes of zBuf[].  The value might be truncated if zBuf is not large
+
** enough to hold it all.  The value put into zBuf must always be zero
+
** terminated, even if it gets truncated because nBuf is not large enough.
+
**
+
** Return the total number of bytes in the data, without truncation, and
+
** not counting the final zero terminator.   Return -1 if the key does
+
** not exist.
+
**
+
** If nBuf<=0 then this routine simply returns the size of the data without
+
** actually reading it.
+
*/
+
static int kvstorageRead(
+
  const char *zClass,
+
  const char *zKey,
+
  char *zBuf,
+
  int nBuf
+
){
+
  FILE *fd;
+
  struct stat buf;
+
  char zXKey[KVSTORAGE_KEY_SZ];
+
  kvstorageMakeKey(zClass, zKey, zXKey);
+
  if( access(zXKey, R_OK)!=0
+
   || stat(zXKey, &buf)!=0
+
   || !S_ISREG(buf.st_mode)
+
  ){
+
    SQLITE_KV_TRACE(("KVVFS-READ   %-15s (-1)\n", zXKey));
+
    return -1;
+
  }
+
  if( nBuf<=0 ){
+
    return (int)buf.st_size;
+
  }else if( nBuf==1 ){
+
    zBuf[0] = 0;
+
    SQLITE_KV_TRACE(("KVVFS-READ   %-15s (%d)\n", zXKey,
+
                 (int)buf.st_size));
+
    return (int)buf.st_size;
+
  }
+
  if( nBuf > buf.st_size + 1 ){
+
    nBuf = buf.st_size + 1;
+
  }
+
  fd = fopen(zXKey, "rb");
+
  if( fd==0 ){
+
    SQLITE_KV_TRACE(("KVVFS-READ   %-15s (-1)\n", zXKey));
+
    return -1;
+
  }else{
+
    sqlite3_int64 n = fread(zBuf, 1, nBuf-1, fd);
+
    fclose(fd);
+
    zBuf[n] = 0;
+
    SQLITE_KV_TRACE(("KVVFS-READ   %-15s (%lld) %.50s%s\n", zXKey,
+
                 n, zBuf, n>50 ? "..." : ""));
+
    return (int)n;
+
  }
+
}
+

+
/*
+
** An internal level of indirection which enables us to replace the
+
** kvvfs i/o methods with JavaScript implementations in WASM builds.
+
** Maintenance reminder: if this struct changes in any way, the JSON
+
** rendering of its structure must be updated in
+
** sqlite3_wasm_enum_json(). There are no binary compatibility
+
** concerns, so it does not need an iVersion member. This file is
+
** necessarily always compiled together with sqlite3_wasm_enum_json(),
+
** and JS code dynamically creates the mapping of members based on
+
** that JSON description.
+
*/
+
typedef struct sqlite3_kvvfs_methods sqlite3_kvvfs_methods;
+
struct sqlite3_kvvfs_methods {
+
  int (*xRead)(const char *zClass, const char *zKey, char *zBuf, int nBuf);
+
  int (*xWrite)(const char *zClass, const char *zKey, const char *zData);
+
  int (*xDelete)(const char *zClass, const char *zKey);
+
  const int nKeySize;
+
};
+

+
/*
+
** This object holds the kvvfs I/O methods which may be swapped out
+
** for JavaScript-side implementations in WASM builds. In such builds
+
** it cannot be const, but in native builds it should be so that
+
** the compiler can hopefully optimize this level of indirection out.
+
** That said, kvvfs is intended primarily for use in WASM builds.
+
**
+
** Note that this is not explicitly flagged as static because the
+
** amalgamation build will tag it with SQLITE_PRIVATE.
+
*/
+
#ifndef SQLITE_WASM
+
const
+
#endif
+
SQLITE_PRIVATE sqlite3_kvvfs_methods sqlite3KvvfsMethods = {
+
kvstorageRead,
+
kvstorageWrite,
+
kvstorageDelete,
+
KVSTORAGE_KEY_SZ
+
};
+

+
/****** Utility subroutines ************************************************/
+

+
/*
+
** Encode binary into the text encoded used to persist on disk.
+
** The output text is stored in aOut[], which must be at least
+
** nData+1 bytes in length.
+
**
+
** Return the actual length of the encoded text, not counting the
+
** zero terminator at the end.
+
**
+
** Encoding format
+
** ---------------
+
**
+
**   *  Non-zero bytes are encoded as upper-case hexadecimal
+
**
+
**   *  A sequence of one or more zero-bytes that are not at the
+
**      beginning of the buffer are encoded as a little-endian
+
**      base-26 number using a..z.  "a" means 0.  "b" means 1,
+
**      "z" means 25.  "ab" means 26.  "ac" means 52.  And so forth.
+
**
+
**   *  Because there is no overlap between the encoding characters
+
**      of hexadecimal and base-26 numbers, it is always clear where
+
**      one stops and the next begins.
+
*/
+
static int kvvfsEncode(const char *aData, int nData, char *aOut){
+
  int i, j;
+
  const unsigned char *a = (const unsigned char*)aData;
+
  for(i=j=0; i<nData; i++){
+
    unsigned char c = a[i];
+
    if( c!=0 ){
+
      aOut[j++] = "0123456789ABCDEF"[c>>4];
+
      aOut[j++] = "0123456789ABCDEF"[c&0xf];
+
    }else{
+
      /* A sequence of 1 or more zeros is stored as a little-endian
+
      ** base-26 number using a..z as the digits. So one zero is "b".
+
      ** Two zeros is "c". 25 zeros is "z", 26 zeros is "ab", 27 is "bb",
+
      ** and so forth.
+
      */
+
      int k;
+
      for(k=1; i+k<nData && a[i+k]==0; k++){}
+
      i += k-1;
+
      while( k>0 ){
+
        aOut[j++] = 'a'+(k%26);
+
        k /= 26;
+
      }
+
    }
+
  }
+
  aOut[j] = 0;
+
  return j;
+
}
+

+
static const signed char kvvfsHexValue[256] = {
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
   0,  1,  2,  3,  4,  5,  6,  7,    8,  9, -1, -1, -1, -1, -1, -1,
+
  -1, 10, 11, 12, 13, 14, 15, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+

+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1,
+
  -1, -1, -1, -1, -1, -1, -1, -1,   -1, -1, -1, -1, -1, -1, -1, -1
+
};
+

+
/*
+
** Decode the text encoding back to binary.  The binary content is
+
** written into pOut, which must be at least nOut bytes in length.
+
**
+
** The return value is the number of bytes actually written into aOut[].
+
*/
+
static int kvvfsDecode(const char *a, char *aOut, int nOut){
+
  int i, j;
+
  int c;
+
  const unsigned char *aIn = (const unsigned char*)a;
+
  i = 0;
+
  j = 0;
+
  while( 1 ){
+
    c = kvvfsHexValue[aIn[i]];
+
    if( c<0 ){
+
      int n = 0;
+
      int mult = 1;
+
      c = aIn[i];
+
      if( c==0 ) break;
+
      while( c>='a' && c<='z' ){
+
        n += (c - 'a')*mult;
+
        mult *= 26;
+
        c = aIn[++i];
+
      }
+
      if( j+n>nOut ) return -1;
+
      memset(&aOut[j], 0, n);
+
      j += n;
+
      c = aIn[i];
+
      if( c==0 ) break;
+
    }else{
+
      aOut[j] = c<<4;
+
      c = kvvfsHexValue[aIn[++i]];
+
      if( c<0 ) break;
+
      aOut[j++] += c;
+
      i++;
+
    }
+
  }
+
  return j;
+
}
+

+
/*
+
** Decode a complete journal file.  Allocate space in pFile->aJrnl
+
** and store the decoding there.  Or leave pFile->aJrnl set to NULL
+
** if an error is encountered.
+
**
+
** The first few characters of the text encoding will be a little-endian
+
** base-26 number (digits a..z) that is the total number of bytes
+
** in the decoded journal file image.  This base-26 number is followed
+
** by a single space, then the encoding of the journal.  The space
+
** separator is required to act as a terminator for the base-26 number.
+
*/
+
static void kvvfsDecodeJournal(
+
  KVVfsFile *pFile,      /* Store decoding in pFile->aJrnl */
+
  const char *zTxt,      /* Text encoding.  Zero-terminated */
+
  int nTxt               /* Bytes in zTxt, excluding zero terminator */
+
){
+
  unsigned int n = 0;
+
  int c, i, mult;
+
  i = 0;
+
  mult = 1;
+
  while( (c = zTxt[i++])>='a' && c<='z' ){
+
    n += (zTxt[i] - 'a')*mult;
+
    mult *= 26;
+
  }
+
  sqlite3_free(pFile->aJrnl);
+
  pFile->aJrnl = sqlite3_malloc64( n );
+
  if( pFile->aJrnl==0 ){
+
    pFile->nJrnl = 0;
+
    return;
+
  }
+
  pFile->nJrnl = n;
+
  n = kvvfsDecode(zTxt+i, pFile->aJrnl, pFile->nJrnl);
+
  if( n<pFile->nJrnl ){
+
    sqlite3_free(pFile->aJrnl);
+
    pFile->aJrnl = 0;
+
    pFile->nJrnl = 0;
+
  }
+
}
+

+
/*
+
** Read or write the "sz" element, containing the database file size.
+
*/
+
static sqlite3_int64 kvvfsReadFileSize(KVVfsFile *pFile){
+
  char zData[50];
+
  zData[0] = 0;
+
  sqlite3KvvfsMethods.xRead(pFile->zClass, "sz", zData, sizeof(zData)-1);
+
  return strtoll(zData, 0, 0);
+
}
+
static int kvvfsWriteFileSize(KVVfsFile *pFile, sqlite3_int64 sz){
+
  char zData[50];
+
  sqlite3_snprintf(sizeof(zData), zData, "%lld", sz);
+
  return sqlite3KvvfsMethods.xWrite(pFile->zClass, "sz", zData);
+
}
+

+
/****** sqlite3_io_methods methods ******************************************/
+

+
/*
+
** Close an kvvfs-file.
+
*/
+
static int kvvfsClose(sqlite3_file *pProtoFile){
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+

+
  SQLITE_KV_LOG(("xClose %s %s\n", pFile->zClass,
+
             pFile->isJournal ? "journal" : "db"));
+
  sqlite3_free(pFile->aJrnl);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Read from the -journal file.
+
*/
+
static int kvvfsReadJrnl(
+
  sqlite3_file *pProtoFile,
+
  void *zBuf,
+
  int iAmt,
+
  sqlite_int64 iOfst
+
){
+
  KVVfsFile *pFile = (KVVfsFile*)pProtoFile;
+
  assert( pFile->isJournal );
+
  SQLITE_KV_LOG(("xRead('%s-journal',%d,%lld)\n", pFile->zClass, iAmt, iOfst));
+
  if( pFile->aJrnl==0 ){
+
    int szTxt = kvstorageRead(pFile->zClass, "jrnl", 0, 0);
+
    char *aTxt;
+
    if( szTxt<=4 ){
+
      return SQLITE_IOERR;
+
    }
+
    aTxt = sqlite3_malloc64( szTxt+1 );
+
    if( aTxt==0 ) return SQLITE_NOMEM;
+
    kvstorageRead(pFile->zClass, "jrnl", aTxt, szTxt+1);
+
    kvvfsDecodeJournal(pFile, aTxt, szTxt);
+
    sqlite3_free(aTxt);
+
    if( pFile->aJrnl==0 ) return SQLITE_IOERR;
+
  }
+
  if( iOfst+iAmt>pFile->nJrnl ){
+
    return SQLITE_IOERR_SHORT_READ;
+
  }
+
  memcpy(zBuf, pFile->aJrnl+iOfst, iAmt);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Read from the database file.
+
*/
+
static int kvvfsReadDb(
+
  sqlite3_file *pProtoFile,
+
  void *zBuf,
+
  int iAmt,
+
  sqlite_int64 iOfst
+
){
+
  KVVfsFile *pFile = (KVVfsFile*)pProtoFile;
+
  unsigned int pgno;
+
  int got, n;
+
  char zKey[30];
+
  char aData[133073];
+
  assert( iOfst>=0 );
+
  assert( iAmt>=0 );
+
  SQLITE_KV_LOG(("xRead('%s-db',%d,%lld)\n", pFile->zClass, iAmt, iOfst));
+
  if( iOfst+iAmt>=512 ){
+
    if( (iOfst % iAmt)!=0 ){
+
      return SQLITE_IOERR_READ;
+
    }
+
    if( (iAmt & (iAmt-1))!=0 || iAmt<512 || iAmt>65536 ){
+
      return SQLITE_IOERR_READ;
+
    }
+
    pFile->szPage = iAmt;
+
    pgno = 1 + iOfst/iAmt;
+
  }else{
+
    pgno = 1;
+
  }
+
  sqlite3_snprintf(sizeof(zKey), zKey, "%u", pgno);
+
  got = sqlite3KvvfsMethods.xRead(pFile->zClass, zKey, aData, sizeof(aData)-1);
+
  if( got<0 ){
+
    n = 0;
+
  }else{
+
    aData[got] = 0;
+
    if( iOfst+iAmt<512 ){
+
      int k = iOfst+iAmt;
+
      aData[k*2] = 0;
+
      n = kvvfsDecode(aData, &aData[2000], sizeof(aData)-2000);
+
      if( n>=iOfst+iAmt ){
+
        memcpy(zBuf, &aData[2000+iOfst], iAmt);
+
        n = iAmt;
+
      }else{
+
        n = 0;
+
      }
+
    }else{
+
      n = kvvfsDecode(aData, zBuf, iAmt);
+
    }
+
  }
+
  if( n<iAmt ){
+
    memset(zBuf+n, 0, iAmt-n);
+
    return SQLITE_IOERR_SHORT_READ;
+
  }
+
  return SQLITE_OK;
+
}
+

+

+
/*
+
** Write into the -journal file.
+
*/
+
static int kvvfsWriteJrnl(
+
  sqlite3_file *pProtoFile,
+
  const void *zBuf,
+
  int iAmt,
+
  sqlite_int64 iOfst
+
){
+
  KVVfsFile *pFile = (KVVfsFile*)pProtoFile;
+
  sqlite3_int64 iEnd = iOfst+iAmt;
+
  SQLITE_KV_LOG(("xWrite('%s-journal',%d,%lld)\n", pFile->zClass, iAmt, iOfst));
+
  if( iEnd>=0x10000000 ) return SQLITE_FULL;
+
  if( pFile->aJrnl==0 || pFile->nJrnl<iEnd ){
+
    char *aNew = sqlite3_realloc(pFile->aJrnl, iEnd);
+
    if( aNew==0 ){
+
      return SQLITE_IOERR_NOMEM;
+
    }
+
    pFile->aJrnl = aNew;
+
    if( pFile->nJrnl<iOfst ){
+
      memset(pFile->aJrnl+pFile->nJrnl, 0, iOfst-pFile->nJrnl);
+
    }
+
    pFile->nJrnl = iEnd;
+
  }
+
  memcpy(pFile->aJrnl+iOfst, zBuf, iAmt);
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Write into the database file.
+
*/
+
static int kvvfsWriteDb(
+
  sqlite3_file *pProtoFile,
+
  const void *zBuf,
+
  int iAmt,
+
  sqlite_int64 iOfst
+
){
+
  KVVfsFile *pFile = (KVVfsFile*)pProtoFile;
+
  unsigned int pgno;
+
  char zKey[30];
+
  char aData[131073];
+
  SQLITE_KV_LOG(("xWrite('%s-db',%d,%lld)\n", pFile->zClass, iAmt, iOfst));
+
  assert( iAmt>=512 && iAmt<=65536 );
+
  assert( (iAmt & (iAmt-1))==0 );
+
  assert( pFile->szPage<0 || pFile->szPage==iAmt );
+
  pFile->szPage = iAmt;
+
  pgno = 1 + iOfst/iAmt;
+
  sqlite3_snprintf(sizeof(zKey), zKey, "%u", pgno);
+
  kvvfsEncode(zBuf, iAmt, aData);
+
  if( sqlite3KvvfsMethods.xWrite(pFile->zClass, zKey, aData) ){
+
    return SQLITE_IOERR;
+
  }
+
  if( iOfst+iAmt > pFile->szDb ){
+
    pFile->szDb = iOfst + iAmt;
+
  }
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Truncate an kvvfs-file.
+
*/
+
static int kvvfsTruncateJrnl(sqlite3_file *pProtoFile, sqlite_int64 size){
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
  SQLITE_KV_LOG(("xTruncate('%s-journal',%lld)\n", pFile->zClass, size));
+
  assert( size==0 );
+
  sqlite3KvvfsMethods.xDelete(pFile->zClass, "jrnl");
+
  sqlite3_free(pFile->aJrnl);
+
  pFile->aJrnl = 0;
+
  pFile->nJrnl = 0;
+
  return SQLITE_OK;
+
}
+
static int kvvfsTruncateDb(sqlite3_file *pProtoFile, sqlite_int64 size){
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
  if( pFile->szDb>size
+
   && pFile->szPage>0
+
   && (size % pFile->szPage)==0
+
  ){
+
    char zKey[50];
+
    unsigned int pgno, pgnoMax;
+
    SQLITE_KV_LOG(("xTruncate('%s-db',%lld)\n", pFile->zClass, size));
+
    pgno = 1 + size/pFile->szPage;
+
    pgnoMax = 2 + pFile->szDb/pFile->szPage;
+
    while( pgno<=pgnoMax ){
+
      sqlite3_snprintf(sizeof(zKey), zKey, "%u", pgno);
+
      sqlite3KvvfsMethods.xDelete(pFile->zClass, zKey);
+
      pgno++;
+
    }
+
    pFile->szDb = size;
+
    return kvvfsWriteFileSize(pFile, size) ? SQLITE_IOERR : SQLITE_OK;
+
  }
+
  return SQLITE_IOERR;
+
}
+

+
/*
+
** Sync an kvvfs-file.
+
*/
+
static int kvvfsSyncJrnl(sqlite3_file *pProtoFile, int flags){
+
  int i, n;
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
  char *zOut;
+
  SQLITE_KV_LOG(("xSync('%s-journal')\n", pFile->zClass));
+
  if( pFile->nJrnl<=0 ){
+
    return kvvfsTruncateJrnl(pProtoFile, 0);
+
  }
+
  zOut = sqlite3_malloc64( pFile->nJrnl*2 + 50 );
+
  if( zOut==0 ){
+
    return SQLITE_IOERR_NOMEM;
+
  }
+
  n = pFile->nJrnl;
+
  i = 0;
+
  do{
+
    zOut[i++] = 'a' + (n%26);
+
    n /= 26;
+
  }while( n>0 );
+
  zOut[i++] = ' ';
+
  kvvfsEncode(pFile->aJrnl, pFile->nJrnl, &zOut[i]);
+
  i = sqlite3KvvfsMethods.xWrite(pFile->zClass, "jrnl", zOut);
+
  sqlite3_free(zOut);
+
  return i ? SQLITE_IOERR : SQLITE_OK;
+
}
+
static int kvvfsSyncDb(sqlite3_file *pProtoFile, int flags){
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return the current file-size of an kvvfs-file.
+
*/
+
static int kvvfsFileSizeJrnl(sqlite3_file *pProtoFile, sqlite_int64 *pSize){
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
  SQLITE_KV_LOG(("xFileSize('%s-journal')\n", pFile->zClass));
+
  *pSize = pFile->nJrnl;
+
  return SQLITE_OK;
+
}
+
static int kvvfsFileSizeDb(sqlite3_file *pProtoFile, sqlite_int64 *pSize){
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
  SQLITE_KV_LOG(("xFileSize('%s-db')\n", pFile->zClass));
+
  if( pFile->szDb>=0 ){
+
    *pSize = pFile->szDb;
+
  }else{
+
    *pSize = kvvfsReadFileSize(pFile);
+
  }
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Lock an kvvfs-file.
+
*/
+
static int kvvfsLock(sqlite3_file *pProtoFile, int eLock){
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
  assert( !pFile->isJournal );
+
  SQLITE_KV_LOG(("xLock(%s,%d)\n", pFile->zClass, eLock));
+

+
  if( eLock!=SQLITE_LOCK_NONE ){
+
    pFile->szDb = kvvfsReadFileSize(pFile);
+
  }
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Unlock an kvvfs-file.
+
*/
+
static int kvvfsUnlock(sqlite3_file *pProtoFile, int eLock){
+
  KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
  assert( !pFile->isJournal );
+
  SQLITE_KV_LOG(("xUnlock(%s,%d)\n", pFile->zClass, eLock));
+
  if( eLock==SQLITE_LOCK_NONE ){
+
    pFile->szDb = -1;
+
  }
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Check if another file-handle holds a RESERVED lock on an kvvfs-file.
+
*/
+
static int kvvfsCheckReservedLock(sqlite3_file *pProtoFile, int *pResOut){
+
  SQLITE_KV_LOG(("xCheckReservedLock\n"));
+
  *pResOut = 0;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** File control method. For custom operations on an kvvfs-file.
+
*/
+
static int kvvfsFileControlJrnl(sqlite3_file *pProtoFile, int op, void *pArg){
+
  SQLITE_KV_LOG(("xFileControl(%d) on journal\n", op));
+
  return SQLITE_NOTFOUND;
+
}
+
static int kvvfsFileControlDb(sqlite3_file *pProtoFile, int op, void *pArg){
+
  SQLITE_KV_LOG(("xFileControl(%d) on database\n", op));
+
  if( op==SQLITE_FCNTL_SYNC ){
+
    KVVfsFile *pFile = (KVVfsFile *)pProtoFile;
+
    int rc = SQLITE_OK;
+
    SQLITE_KV_LOG(("xSync('%s-db')\n", pFile->zClass));
+
    if( pFile->szDb>0 && 0!=kvvfsWriteFileSize(pFile, pFile->szDb) ){
+
      rc = SQLITE_IOERR;
+
    }
+
    return rc;
+
  }
+
  return SQLITE_NOTFOUND;
+
}
+

+
/*
+
** Return the sector-size in bytes for an kvvfs-file.
+
*/
+
static int kvvfsSectorSize(sqlite3_file *pFile){
+
  return 512;
+
}
+

+
/*
+
** Return the device characteristic flags supported by an kvvfs-file.
+
*/
+
static int kvvfsDeviceCharacteristics(sqlite3_file *pProtoFile){
+
  return 0;
+
}
+

+
/****** sqlite3_vfs methods *************************************************/
+

+
/*
+
** Open an kvvfs file handle.
+
*/
+
static int kvvfsOpen(
+
  sqlite3_vfs *pProtoVfs,
+
  const char *zName,
+
  sqlite3_file *pProtoFile,
+
  int flags,
+
  int *pOutFlags
+
){
+
  KVVfsFile *pFile = (KVVfsFile*)pProtoFile;
+
  if( zName==0 ) zName = "";
+
  SQLITE_KV_LOG(("xOpen(\"%s\")\n", zName));
+
  if( strcmp(zName, "local")==0
+
   || strcmp(zName, "session")==0
+
  ){
+
    pFile->isJournal = 0;
+
    pFile->base.pMethods = &kvvfs_db_io_methods;
+
  }else
+
  if( strcmp(zName, "local-journal")==0
+
   || strcmp(zName, "session-journal")==0
+
  ){
+
    pFile->isJournal = 1;
+
    pFile->base.pMethods = &kvvfs_jrnl_io_methods;
+
  }else{
+
    return SQLITE_CANTOPEN;
+
  }
+
  if( zName[0]=='s' ){
+
    pFile->zClass = "session";
+
  }else{
+
    pFile->zClass = "local";
+
  }
+
  pFile->aJrnl = 0;
+
  pFile->nJrnl = 0;
+
  pFile->szPage = -1;
+
  pFile->szDb = -1;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Delete the file located at zPath. If the dirSync argument is true,
+
** ensure the file-system modifications are synced to disk before
+
** returning.
+
*/
+
static int kvvfsDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){
+
  if( strcmp(zPath, "local-journal")==0 ){
+
    sqlite3KvvfsMethods.xDelete("local", "jrnl");
+
  }else
+
  if( strcmp(zPath, "session-journal")==0 ){
+
    sqlite3KvvfsMethods.xDelete("session", "jrnl");
+
  }
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Test for access permissions. Return true if the requested permission
+
** is available, or false otherwise.
+
*/
+
static int kvvfsAccess(
+
  sqlite3_vfs *pProtoVfs,
+
  const char *zPath,
+
  int flags,
+
  int *pResOut
+
){
+
  SQLITE_KV_LOG(("xAccess(\"%s\")\n", zPath));
+
  if( strcmp(zPath, "local-journal")==0 ){
+
    *pResOut = sqlite3KvvfsMethods.xRead("local", "jrnl", 0, 0)>0;
+
  }else
+
  if( strcmp(zPath, "session-journal")==0 ){
+
    *pResOut = sqlite3KvvfsMethods.xRead("session", "jrnl", 0, 0)>0;
+
  }else
+
  if( strcmp(zPath, "local")==0 ){
+
    *pResOut = sqlite3KvvfsMethods.xRead("local", "sz", 0, 0)>0;
+
  }else
+
  if( strcmp(zPath, "session")==0 ){
+
    *pResOut = sqlite3KvvfsMethods.xRead("session", "sz", 0, 0)>0;
+
  }else
+
  {
+
    *pResOut = 0;
+
  }
+
  SQLITE_KV_LOG(("xAccess returns %d\n",*pResOut));
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Populate buffer zOut with the full canonical pathname corresponding
+
** to the pathname in zPath. zOut is guaranteed to point to a buffer
+
** of at least (INST_MAX_PATHNAME+1) bytes.
+
*/
+
static int kvvfsFullPathname(
+
  sqlite3_vfs *pVfs,
+
  const char *zPath,
+
  int nOut,
+
  char *zOut
+
){
+
  size_t nPath;
+
#ifdef SQLITE_OS_KV_ALWAYS_LOCAL
+
  zPath = "local";
+
#endif
+
  nPath = strlen(zPath);
+
  SQLITE_KV_LOG(("xFullPathname(\"%s\")\n", zPath));
+
  if( nOut<nPath+1 ) nPath = nOut - 1;
+
  memcpy(zOut, zPath, nPath);
+
  zOut[nPath] = 0;
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Open the dynamic library located at zPath and return a handle.
+
*/
+
static void *kvvfsDlOpen(sqlite3_vfs *pVfs, const char *zPath){
+
  return 0;
+
}
+

+
/*
+
** Populate the buffer pointed to by zBufOut with nByte bytes of
+
** random data.
+
*/
+
static int kvvfsRandomness(sqlite3_vfs *pVfs, int nByte, char *zBufOut){
+
  memset(zBufOut, 0, nByte);
+
  return nByte;
+
}
+

+
/*
+
** Sleep for nMicro microseconds. Return the number of microseconds
+
** actually slept.
+
*/
+
static int kvvfsSleep(sqlite3_vfs *pVfs, int nMicro){
+
  return SQLITE_OK;
+
}
+

+
/*
+
** Return the current time as a Julian Day number in *pTimeOut.
+
*/
+
static int kvvfsCurrentTime(sqlite3_vfs *pVfs, double *pTimeOut){
+
  sqlite3_int64 i = 0;
+
  int rc;
+
  rc = kvvfsCurrentTimeInt64(0, &i);
+
  *pTimeOut = i/86400000.0;
+
  return rc;
+
}
+
#include <sys/time.h>
+
static int kvvfsCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *pTimeOut){
+
  static const sqlite3_int64 unixEpoch = 24405875*(sqlite3_int64)8640000;
+
  struct timeval sNow;
+
  (void)gettimeofday(&sNow, 0);  /* Cannot fail given valid arguments */
+
  *pTimeOut = unixEpoch + 1000*(sqlite3_int64)sNow.tv_sec + sNow.tv_usec/1000;
+
  return SQLITE_OK;
+
}
+
#endif /* SQLITE_OS_KV || SQLITE_OS_UNIX */
+

+
#if SQLITE_OS_KV
+
/*
+
** This routine is called initialize the KV-vfs as the default VFS.
+
*/
+
SQLITE_API int sqlite3_os_init(void){
+
  return sqlite3_vfs_register(&sqlite3OsKvvfsObject, 1);
+
}
+
SQLITE_API int sqlite3_os_end(void){
+
  return SQLITE_OK;
+
}
+
#endif /* SQLITE_OS_KV */
+

+
#if SQLITE_OS_UNIX && defined(SQLITE_OS_KV_OPTIONAL)
+
SQLITE_PRIVATE int sqlite3KvvfsInit(void){
+
  return sqlite3_vfs_register(&sqlite3OsKvvfsObject, 0);
+
}
+
#endif
+

+
/************** End of os_kv.c ***********************************************/
/************** Begin file os_unix.c *****************************************/
/*
** 2004 May 22
@@ -34870,13 +36739,13 @@ SQLITE_PRIVATE const char *sqlite3OpcodeName(int i){
/*
** standard include files.
*/
-
#include <sys/types.h>
-
#include <sys/stat.h>
+
#include <sys/types.h>   /* amalgamator: keep */
+
#include <sys/stat.h>    /* amalgamator: keep */
#include <fcntl.h>
#include <sys/ioctl.h>
-
#include <unistd.h>
+
#include <unistd.h>      /* amalgamator: keep */
/* #include <time.h> */
-
#include <sys/time.h>
+
#include <sys/time.h>    /* amalgamator: keep */
#include <errno.h>
#if !defined(SQLITE_OMIT_WAL) || SQLITE_MAX_MMAP_SIZE>0
# include <sys/mman.h>
@@ -40638,6 +42507,7 @@ static const char *unixTempFileDir(void){
static int unixGetTempname(int nBuf, char *zBuf){
  const char *zDir;
  int iLimit = 0;
+
  int rc = SQLITE_OK;

  /* It's odd to simulate an io-error here, but really this is just
  ** using the io-error infrastructure to test that SQLite handles this
@@ -40646,18 +42516,26 @@ static int unixGetTempname(int nBuf, char *zBuf){
  zBuf[0] = 0;
  SimulateIOError( return SQLITE_IOERR );

+
  sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
  zDir = unixTempFileDir();
-
  if( zDir==0 ) return SQLITE_IOERR_GETTEMPPATH;
-
  do{
-
    u64 r;
-
    sqlite3_randomness(sizeof(r), &r);
-
    assert( nBuf>2 );
-
    zBuf[nBuf-2] = 0;
-
    sqlite3_snprintf(nBuf, zBuf, "%s/"SQLITE_TEMP_FILE_PREFIX"%llx%c",
-
                     zDir, r, 0);
-
    if( zBuf[nBuf-2]!=0 || (iLimit++)>10 ) return SQLITE_ERROR;
-
  }while( osAccess(zBuf,0)==0 );
-
  return SQLITE_OK;
+
  if( zDir==0 ){
+
    rc = SQLITE_IOERR_GETTEMPPATH;
+
  }else{
+
    do{
+
      u64 r;
+
      sqlite3_randomness(sizeof(r), &r);
+
      assert( nBuf>2 );
+
      zBuf[nBuf-2] = 0;
+
      sqlite3_snprintf(nBuf, zBuf, "%s/"SQLITE_TEMP_FILE_PREFIX"%llx%c",
+
                       zDir, r, 0);
+
      if( zBuf[nBuf-2]!=0 || (iLimit++)>10 ){
+
        rc = SQLITE_ERROR;
+
        break;
+
      }
+
    }while( osAccess(zBuf,0)==0 );
+
  }
+
  sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
+
  return rc;
}

#if SQLITE_ENABLE_LOCKING_STYLE && defined(__APPLE__)
@@ -40800,20 +42678,23 @@ static int findCreateFileMode(
    **
    ** where NN is a decimal number. The NN naming schemes are
    ** used by the test_multiplex.c module.
+
    **
+
    ** In normal operation, the journal file name will always contain
+
    ** a '-' character.  However in 8+3 filename mode, or if a corrupt
+
    ** rollback journal specifies a super-journal with a goofy name, then
+
    ** the '-' might be missing or the '-' might be the first character in
+
    ** the filename.  In that case, just return SQLITE_OK with *pMode==0.
    */
    nDb = sqlite3Strlen30(zPath) - 1;
-
    while( zPath[nDb]!='-' ){
-
      /* In normal operation, the journal file name will always contain
-
      ** a '-' character.  However in 8+3 filename mode, or if a corrupt
-
      ** rollback journal specifies a super-journal with a goofy name, then
-
      ** the '-' might be missing. */
-
      if( nDb==0 || zPath[nDb]=='.' ) return SQLITE_OK;
+
    while( nDb>0 && zPath[nDb]!='.' ){
+
      if( zPath[nDb]=='-' ){
+
        memcpy(zDb, zPath, nDb);
+
        zDb[nDb] = '\0';
+
        rc = getFileMode(zDb, pMode, pUid, pGid);
+
        break;
+
      }
      nDb--;
    }
-
    memcpy(zDb, zPath, nDb);
-
    zDb[nDb] = '\0';
-

-
    rc = getFileMode(zDb, pMode, pUid, pGid);
  }else if( flags & SQLITE_OPEN_DELETEONCLOSE ){
    *pMode = 0600;
  }else if( flags & SQLITE_OPEN_URI ){
@@ -41203,86 +43084,99 @@ static int unixAccess(
}

/*
-
** If the last component of the pathname in z[0]..z[j-1] is something
-
** other than ".." then back it out and return true.  If the last
-
** component is empty or if it is ".." then return false.
+
** A pathname under construction
*/
-
static int unixBackupDir(const char *z, int *pJ){
-
  int j = *pJ;
-
  int i;
-
  if( j<=0 ) return 0;
-
  for(i=j-1; i>0 && z[i-1]!='/'; i--){}
-
  if( i==0 ) return 0;
-
  if( z[i]=='.' && i==j-2 && z[i+1]=='.' ) return 0;
-
  *pJ = i-1;
-
  return 1;
-
}
+
typedef struct DbPath DbPath;
+
struct DbPath {
+
  int rc;           /* Non-zero following any error */
+
  int nSymlink;     /* Number of symlinks resolved */
+
  char *zOut;       /* Write the pathname here */
+
  int nOut;         /* Bytes of space available to zOut[] */
+
  int nUsed;        /* Bytes of zOut[] currently being used */
+
};
+

+
/* Forward reference */
+
static void appendAllPathElements(DbPath*,const char*);

/*
-
** Convert a relative pathname into a full pathname.  Also
-
** simplify the pathname as follows:
-
**
-
**    Remove all instances of /./
-
**    Remove all isntances of /X/../ for any X
+
** Append a single path element to the DbPath under construction
*/
-
static int mkFullPathname(
-
  const char *zPath,              /* Input path */
-
  char *zOut,                     /* Output buffer */
-
  int nOut                        /* Allocated size of buffer zOut */
+
static void appendOnePathElement(
+
  DbPath *pPath,       /* Path under construction, to which to append zName */
+
  const char *zName,   /* Name to append to pPath.  Not zero-terminated */
+
  int nName            /* Number of significant bytes in zName */
){
-
  int nPath = sqlite3Strlen30(zPath);
-
  int iOff = 0;
-
  int i, j;
-
  if( zPath[0]!='/' ){
-
    if( osGetcwd(zOut, nOut-2)==0 ){
-
      return unixLogError(SQLITE_CANTOPEN_BKPT, "getcwd", zPath);
+
  assert( nName>0 );
+
  assert( zName!=0 );
+
  if( zName[0]=='.' ){
+
    if( nName==1 ) return;
+
    if( zName[1]=='.' && nName==2 ){
+
      if( pPath->nUsed<=1 ){
+
        pPath->rc = SQLITE_ERROR;
+
        return;
+
      }
+
      assert( pPath->zOut[0]=='/' );
+
      while( pPath->zOut[--pPath->nUsed]!='/' ){}
+
      return;
    }
-
    iOff = sqlite3Strlen30(zOut);
-
    zOut[iOff++] = '/';
-
  }
-
  if( (iOff+nPath+1)>nOut ){
-
    /* SQLite assumes that xFullPathname() nul-terminates the output buffer
-
    ** even if it returns an error.  */
-
    zOut[iOff] = '\0';
-
    return SQLITE_CANTOPEN_BKPT;
  }
-
  sqlite3_snprintf(nOut-iOff, &zOut[iOff], "%s", zPath);
-

-
  /* Remove duplicate '/' characters.  Except, two // at the beginning
-
  ** of a pathname is allowed since this is important on windows. */
-
  for(i=j=1; zOut[i]; i++){
-
    zOut[j++] = zOut[i];
-
    while( zOut[i]=='/' && zOut[i+1]=='/' ) i++;
+
  if( pPath->nUsed + nName + 2 >= pPath->nOut ){
+
    pPath->rc = SQLITE_ERROR;
+
    return;
  }
-
  zOut[j] = 0;
-

-
  assert( zOut[0]=='/' );
-
  for(i=j=0; zOut[i]; i++){
-
    if( zOut[i]=='/' ){
-
      /* Skip over internal "/." directory components */
-
      if( zOut[i+1]=='.' && zOut[i+2]=='/' ){
-
        i += 1;
-
        continue;
+
  pPath->zOut[pPath->nUsed++] = '/';
+
  memcpy(&pPath->zOut[pPath->nUsed], zName, nName);
+
  pPath->nUsed += nName;
+
#if defined(HAVE_READLINK) && defined(HAVE_LSTAT)
+
  if( pPath->rc==SQLITE_OK ){
+
    const char *zIn;
+
    struct stat buf;
+
    pPath->zOut[pPath->nUsed] = 0;
+
    zIn = pPath->zOut;
+
    if( osLstat(zIn, &buf)!=0 ){
+
      if( errno!=ENOENT ){
+
        pPath->rc = unixLogError(SQLITE_CANTOPEN_BKPT, "lstat", zIn);
      }
-

-
      /* If this is a "/.." directory component then back out the
-
      ** previous term of the directory if it is something other than "..".
-
      */
-
      if( zOut[i+1]=='.'
-
       && zOut[i+2]=='.'
-
       && zOut[i+3]=='/'
-
       && unixBackupDir(zOut, &j)
-
      ){
-
        i += 2;
-
        continue;
+
    }else if( S_ISLNK(buf.st_mode) ){
+
      ssize_t got;
+
      char zLnk[SQLITE_MAX_PATHLEN+2];
+
      if( pPath->nSymlink++ > SQLITE_MAX_SYMLINK ){
+
        pPath->rc = SQLITE_CANTOPEN_BKPT;
+
        return;
+
      }
+
      got = osReadlink(zIn, zLnk, sizeof(zLnk)-2);
+
      if( got<=0 || got>=(ssize_t)sizeof(zLnk)-2 ){
+
        pPath->rc = unixLogError(SQLITE_CANTOPEN_BKPT, "readlink", zIn);
+
        return;
+
      }
+
      zLnk[got] = 0;
+
      if( zLnk[0]=='/' ){
+
        pPath->nUsed = 0;
+
      }else{
+
        pPath->nUsed -= nName + 1;
      }
+
      appendAllPathElements(pPath, zLnk);
    }
-
    if( ALWAYS(j>=0) ) zOut[j] = zOut[i];
-
    j++;
  }
-
  if( NEVER(j==0) ) zOut[j++] = '/';
-
  zOut[j] = 0;
-
  return SQLITE_OK;
+
#endif
+
}
+

+
/*
+
** Append all path elements in zPath to the DbPath under construction.
+
*/
+
static void appendAllPathElements(
+
  DbPath *pPath,       /* Path under construction, to which to append zName */
+
  const char *zPath    /* Path to append to pPath.  Is zero-terminated */
+
){
+
  int i = 0;
+
  int j = 0;
+
  do{
+
    while( zPath[i] && zPath[i]!='/' ){ i++; }
+
    if( i>j ){
+
      appendOnePathElement(pPath, &zPath[j], i-j);
+
    }
+
    j = i+1;
+
  }while( zPath[i++] );
}

/*
@@ -41300,86 +43194,27 @@ static int unixFullPathname(
  int nOut,                     /* Size of output buffer in bytes */
  char *zOut                    /* Output buffer */
){
-
#if !defined(HAVE_READLINK) || !defined(HAVE_LSTAT)
-
  return mkFullPathname(zPath, zOut, nOut);
-
#else
-
  int rc = SQLITE_OK;
-
  int nByte;
-
  int nLink = 0;                /* Number of symbolic links followed so far */
-
  const char *zIn = zPath;      /* Input path for each iteration of loop */
-
  char *zDel = 0;
-

-
  assert( pVfs->mxPathname==MAX_PATHNAME );
+
  DbPath path;
  UNUSED_PARAMETER(pVfs);
-

-
  /* It's odd to simulate an io-error here, but really this is just
-
  ** using the io-error infrastructure to test that SQLite handles this
-
  ** function failing. This function could fail if, for example, the
-
  ** current working directory has been unlinked.
-
  */
-
  SimulateIOError( return SQLITE_ERROR );
-

-
  do {
-

-
    /* Call stat() on path zIn. Set bLink to true if the path is a symbolic
-
    ** link, or false otherwise.  */
-
    int bLink = 0;
-
    struct stat buf;
-
    if( osLstat(zIn, &buf)!=0 ){
-
      if( errno!=ENOENT ){
-
        rc = unixLogError(SQLITE_CANTOPEN_BKPT, "lstat", zIn);
-
      }
-
    }else{
-
      bLink = S_ISLNK(buf.st_mode);
-
    }
-

-
    if( bLink ){
-
      nLink++;
-
      if( zDel==0 ){
-
        zDel = sqlite3_malloc(nOut);
-
        if( zDel==0 ) rc = SQLITE_NOMEM_BKPT;
-
      }else if( nLink>=SQLITE_MAX_SYMLINKS ){
-
        rc = SQLITE_CANTOPEN_BKPT;
-
      }
-

-
      if( rc==SQLITE_OK ){
-
        nByte = osReadlink(zIn, zDel, nOut-1);
-
        if( nByte<0 ){
-
          rc = unixLogError(SQLITE_CANTOPEN_BKPT, "readlink", zIn);
-
        }else{
-
          if( zDel[0]!='/' ){
-
            int n;
-
            for(n = sqlite3Strlen30(zIn); n>0 && zIn[n-1]!='/'; n--);
-
            if( nByte+n+1>nOut ){
-
              rc = SQLITE_CANTOPEN_BKPT;
-
            }else{
-
              memmove(&zDel[n], zDel, nByte+1);
-
              memcpy(zDel, zIn, n);
-
              nByte += n;
-
            }
-
          }
-
          zDel[nByte] = '\0';
-
        }
-
      }
-

-
      zIn = zDel;
-
    }
-

-
    assert( rc!=SQLITE_OK || zIn!=zOut || zIn[0]=='/' );
-
    if( rc==SQLITE_OK && zIn!=zOut ){
-
      rc = mkFullPathname(zIn, zOut, nOut);
+
  path.rc = 0;
+
  path.nUsed = 0;
+
  path.nSymlink = 0;
+
  path.nOut = nOut;
+
  path.zOut = zOut;
+
  if( zPath[0]!='/' ){
+
    char zPwd[SQLITE_MAX_PATHLEN+2];
+
    if( osGetcwd(zPwd, sizeof(zPwd)-2)==0 ){
+
      return unixLogError(SQLITE_CANTOPEN_BKPT, "getcwd", zPath);
    }
-
    if( bLink==0 ) break;
-
    zIn = zOut;
-
  }while( rc==SQLITE_OK );
-

-
  sqlite3_free(zDel);
-
  if( rc==SQLITE_OK && nLink ) rc = SQLITE_OK_SYMLINK;
-
  return rc;
-
#endif   /* HAVE_READLINK && HAVE_LSTAT */
+
    appendAllPathElements(&path, zPwd);
+
  }
+
  appendAllPathElements(&path, zPath);
+
  zOut[path.nUsed] = 0;
+
  if( path.rc || path.nUsed<2 ) return SQLITE_CANTOPEN_BKPT;
+
  if( path.nSymlink ) return SQLITE_OK_SYMLINK;
+
  return SQLITE_OK;
}

-

#ifndef SQLITE_OMIT_LOAD_EXTENSION
/*
** Interfaces for opening a shared library, finding entry points
@@ -42875,8 +44710,16 @@ SQLITE_API int sqlite3_os_init(void){

  /* Register all VFSes defined in the aVfs[] array */
  for(i=0; i<(sizeof(aVfs)/sizeof(sqlite3_vfs)); i++){
+
#ifdef SQLITE_DEFAULT_UNIX_VFS
+
    sqlite3_vfs_register(&aVfs[i],
+
           0==strcmp(aVfs[i].zName,SQLITE_DEFAULT_UNIX_VFS));
+
#else
    sqlite3_vfs_register(&aVfs[i], i==0);
+
#endif
  }
+
#ifdef SQLITE_OS_KV_OPTIONAL
+
  sqlite3KvvfsInit();
+
#endif
  unixBigLock = sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_VFS1);

#ifndef SQLITE_OMIT_WAL
@@ -44839,10 +46682,12 @@ SQLITE_API int sqlite3_win32_set_directory8(
  const char *zValue  /* New value for directory being set or reset */
){
  char **ppDirectory = 0;
+
  int rc;
#ifndef SQLITE_OMIT_AUTOINIT
-
  int rc = sqlite3_initialize();
+
  rc = sqlite3_initialize();
  if( rc ) return rc;
#endif
+
  sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
  if( type==SQLITE_WIN32_DATA_DIRECTORY_TYPE ){
    ppDirectory = &sqlite3_data_directory;
  }else if( type==SQLITE_WIN32_TEMP_DIRECTORY_TYPE ){
@@ -44857,14 +46702,19 @@ SQLITE_API int sqlite3_win32_set_directory8(
    if( zValue && zValue[0] ){
      zCopy = sqlite3_mprintf("%s", zValue);
      if ( zCopy==0 ){
-
        return SQLITE_NOMEM_BKPT;
+
        rc = SQLITE_NOMEM_BKPT;
+
        goto set_directory8_done;
      }
    }
    sqlite3_free(*ppDirectory);
    *ppDirectory = zCopy;
-
    return SQLITE_OK;
+
    rc = SQLITE_OK;
+
  }else{
+
    rc = SQLITE_ERROR;
  }
-
  return SQLITE_ERROR;
+
set_directory8_done:
+
  sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
+
  return rc;
}

/*
@@ -47639,6 +49489,19 @@ static int winMakeEndInDirSep(int nBuf, char *zBuf){
}

/*
+
** If sqlite3_temp_directory is defined, take the mutex and return true.
+
**
+
** If sqlite3_temp_directory is NULL (undefined), omit the mutex and
+
** return false.
+
*/
+
static int winTempDirDefined(void){
+
  sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
+
  if( sqlite3_temp_directory!=0 ) return 1;
+
  sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
+
  return 0;
+
}
+

+
/*
** Create a temporary file name and store the resulting pointer into pzBuf.
** The pointer returned in pzBuf must be freed via sqlite3_free().
*/
@@ -47674,20 +49537,23 @@ static int winGetTempname(sqlite3_vfs *pVfs, char **pzBuf){
  */
  nDir = nMax - (nPre + 15);
  assert( nDir>0 );
-
  if( sqlite3_temp_directory ){
+
  if( winTempDirDefined() ){
    int nDirLen = sqlite3Strlen30(sqlite3_temp_directory);
    if( nDirLen>0 ){
      if( !winIsDirSep(sqlite3_temp_directory[nDirLen-1]) ){
        nDirLen++;
      }
      if( nDirLen>nDir ){
+
        sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
        sqlite3_free(zBuf);
        OSTRACE(("TEMP-FILENAME rc=SQLITE_ERROR\n"));
        return winLogError(SQLITE_ERROR, 0, "winGetTempname1", 0);
      }
      sqlite3_snprintf(nMax, zBuf, "%s", sqlite3_temp_directory);
    }
+
    sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
  }
+

#if defined(__CYGWIN__)
  else{
    static const char *azDirs[] = {
@@ -48476,7 +50342,7 @@ static BOOL winIsVerbatimPathname(
** pathname into zOut[].  zOut[] will be at least pVfs->mxPathname
** bytes in size.
*/
-
static int winFullPathname(
+
static int winFullPathnameNoMutex(
  sqlite3_vfs *pVfs,            /* Pointer to vfs object */
  const char *zRelative,        /* Possibly relative input path */
  int nFull,                    /* Size of output buffer in bytes */
@@ -48655,6 +50521,20 @@ static int winFullPathname(
  }
#endif
}
+
static int winFullPathname(
+
  sqlite3_vfs *pVfs,            /* Pointer to vfs object */
+
  const char *zRelative,        /* Possibly relative input path */
+
  int nFull,                    /* Size of output buffer in bytes */
+
  char *zFull                   /* Output buffer */
+
){
+
  int rc;
+
  MUTEX_LOGIC( sqlite3_mutex *pMutex; )
+
  MUTEX_LOGIC( pMutex = sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR); )
+
  sqlite3_mutex_enter(pMutex);
+
  rc = winFullPathnameNoMutex(pVfs, zRelative, nFull, zFull);
+
  sqlite3_mutex_leave(pMutex);
+
  return rc;
+
}

#ifndef SQLITE_OMIT_LOAD_EXTENSION
/*
@@ -50443,12 +52323,20 @@ struct PCache {
  int sqlite3PcacheTrace = 2;       /* 0: off  1: simple  2: cache dumps */
  int sqlite3PcacheMxDump = 9999;   /* Max cache entries for pcacheDump() */
# define pcacheTrace(X) if(sqlite3PcacheTrace){sqlite3DebugPrintf X;}
-
  void pcacheDump(PCache *pCache){
-
    int N;
-
    int i, j;
-
    sqlite3_pcache_page *pLower;
+
  static void pcachePageTrace(int i, sqlite3_pcache_page *pLower){
    PgHdr *pPg;
    unsigned char *a;
+
    int j;
+
    pPg = (PgHdr*)pLower->pExtra;
+
    printf("%3d: nRef %2d flgs %02x data ", i, pPg->nRef, pPg->flags);
+
    a = (unsigned char *)pLower->pBuf;
+
    for(j=0; j<12; j++) printf("%02x", a[j]);
+
    printf(" ptr %p\n", pPg);
+
  }
+
  static void pcacheDump(PCache *pCache){
+
    int N;
+
    int i;
+
    sqlite3_pcache_page *pLower;

    if( sqlite3PcacheTrace<2 ) return;
    if( pCache->pCache==0 ) return;
@@ -50457,22 +52345,33 @@ struct PCache {
    for(i=1; i<=N; i++){
       pLower = sqlite3GlobalConfig.pcache2.xFetch(pCache->pCache, i, 0);
       if( pLower==0 ) continue;
-
       pPg = (PgHdr*)pLower->pExtra;
-
       printf("%3d: nRef %2d flgs %02x data ", i, pPg->nRef, pPg->flags);
-
       a = (unsigned char *)pLower->pBuf;
-
       for(j=0; j<12; j++) printf("%02x", a[j]);
-
       printf("\n");
-
       if( pPg->pPage==0 ){
+
       pcachePageTrace(i, pLower);
+
       if( ((PgHdr*)pLower)->pPage==0 ){
         sqlite3GlobalConfig.pcache2.xUnpin(pCache->pCache, pLower, 0);
       }
    }
  }
-
  #else
+
#else
# define pcacheTrace(X)
+
# define pcachePageTrace(PGNO, X)
# define pcacheDump(X)
#endif

/*
+
** Return 1 if pPg is on the dirty list for pCache.  Return 0 if not.
+
** This routine runs inside of assert() statements only.
+
*/
+
#ifdef SQLITE_DEBUG
+
static int pageOnDirtyList(PCache *pCache, PgHdr *pPg){
+
  PgHdr *p;
+
  for(p=pCache->pDirty; p; p=p->pDirtyNext){
+
    if( p==pPg ) return 1;
+
  }
+
  return 0;
+
}
+
#endif
+

+
/*
** Check invariants on a PgHdr entry.  Return true if everything is OK.
** Return false if any invariant is violated.
**
@@ -50490,8 +52389,13 @@ SQLITE_PRIVATE int sqlite3PcachePageSanity(PgHdr *pPg){
  assert( pCache!=0 );      /* Every page has an associated PCache */
  if( pPg->flags & PGHDR_CLEAN ){
    assert( (pPg->flags & PGHDR_DIRTY)==0 );/* Cannot be both CLEAN and DIRTY */
-
    assert( pCache->pDirty!=pPg );          /* CLEAN pages not on dirty list */
-
    assert( pCache->pDirtyTail!=pPg );
+
    assert( !pageOnDirtyList(pCache, pPg) );/* CLEAN pages not on dirty list */
+
  }else{
+
    assert( (pPg->flags & PGHDR_DIRTY)!=0 );/* If not CLEAN must be DIRTY */
+
    assert( pPg->pDirtyNext==0 || pPg->pDirtyNext->pDirtyPrev==pPg );
+
    assert( pPg->pDirtyPrev==0 || pPg->pDirtyPrev->pDirtyNext==pPg );
+
    assert( pPg->pDirtyPrev!=0 || pCache->pDirty==pPg );
+
    assert( pageOnDirtyList(pCache, pPg) );
  }
  /* WRITEABLE pages must also be DIRTY */
  if( pPg->flags & PGHDR_WRITEABLE ){
@@ -50765,8 +52669,9 @@ SQLITE_PRIVATE sqlite3_pcache_page *sqlite3PcacheFetch(
  assert( createFlag==0 || pCache->eCreate==eCreate );
  assert( createFlag==0 || eCreate==1+(!pCache->bPurgeable||!pCache->pDirty) );
  pRes = sqlite3GlobalConfig.pcache2.xFetch(pCache->pCache, pgno, eCreate);
-
  pcacheTrace(("%p.FETCH %d%s (result: %p)\n",pCache,pgno,
+
  pcacheTrace(("%p.FETCH %d%s (result: %p) ",pCache,pgno,
               createFlag?" create":"",pRes));
+
  pcachePageTrace(pgno, pRes);
  return pRes;
}

@@ -50894,6 +52799,7 @@ SQLITE_PRIVATE void SQLITE_NOINLINE sqlite3PcacheRelease(PgHdr *p){
      pcacheUnpin(p);
    }else{
      pcacheManageDirtyList(p, PCACHE_DIRTYLIST_FRONT);
+
      assert( sqlite3PcachePageSanity(p) );
    }
  }
}
@@ -50928,8 +52834,7 @@ SQLITE_PRIVATE void sqlite3PcacheDrop(PgHdr *p){
** make it so.
*/
SQLITE_PRIVATE void sqlite3PcacheMakeDirty(PgHdr *p){
-
  assert( p->nRef>0 || p->pCache->bPurgeable==0 );
-
  testcase( p->nRef==0 );
+
  assert( p->nRef>0 );
  assert( sqlite3PcachePageSanity(p) );
  if( p->flags & (PGHDR_CLEAN|PGHDR_DONT_WRITE) ){    /*OPTIMIZATION-IF-FALSE*/
    p->flags &= ~PGHDR_DONT_WRITE;
@@ -50938,6 +52843,7 @@ SQLITE_PRIVATE void sqlite3PcacheMakeDirty(PgHdr *p){
      pcacheTrace(("%p.DIRTY %d\n",p->pCache,p->pgno));
      assert( (p->flags & (PGHDR_DIRTY|PGHDR_CLEAN))==PGHDR_DIRTY );
      pcacheManageDirtyList(p, PCACHE_DIRTYLIST_ADD);
+
      assert( sqlite3PcachePageSanity(p) );
    }
    assert( sqlite3PcachePageSanity(p) );
  }
@@ -51000,14 +52906,24 @@ SQLITE_PRIVATE void sqlite3PcacheClearSyncFlags(PCache *pCache){
*/
SQLITE_PRIVATE void sqlite3PcacheMove(PgHdr *p, Pgno newPgno){
  PCache *pCache = p->pCache;
+
  sqlite3_pcache_page *pOther;
  assert( p->nRef>0 );
  assert( newPgno>0 );
  assert( sqlite3PcachePageSanity(p) );
  pcacheTrace(("%p.MOVE %d -> %d\n",pCache,p->pgno,newPgno));
+
  pOther = sqlite3GlobalConfig.pcache2.xFetch(pCache->pCache, newPgno, 0);
+
  if( pOther ){
+
    PgHdr *pXPage = (PgHdr*)pOther->pExtra;
+
    assert( pXPage->nRef==0 );
+
    pXPage->nRef++;
+
    pCache->nRefSum++;
+
    sqlite3PcacheDrop(pXPage);
+
  }
  sqlite3GlobalConfig.pcache2.xRekey(pCache->pCache, p->pPage, p->pgno,newPgno);
  p->pgno = newPgno;
  if( (p->flags&PGHDR_DIRTY) && (p->flags&PGHDR_NEED_SYNC) ){
    pcacheManageDirtyList(p, PCACHE_DIRTYLIST_FRONT);
+
    assert( sqlite3PcachePageSanity(p) );
  }
}

@@ -51305,12 +53221,13 @@ SQLITE_PRIVATE void sqlite3PcacheIterateDirty(PCache *pCache, void (*xIter)(PgHd
** size can vary according to architecture, compile-time options, and
** SQLite library version number.
**
-
** If SQLITE_PCACHE_SEPARATE_HEADER is defined, then the extension is obtained
-
** using a separate memory allocation from the database page content.  This
-
** seeks to overcome the "clownshoe" problem (also called "internal
-
** fragmentation" in academic literature) of allocating a few bytes more
-
** than a power of two with the memory allocator rounding up to the next
-
** power of two, and leaving the rounded-up space unused.
+
** Historical note:  It used to be that if the SQLITE_PCACHE_SEPARATE_HEADER
+
** was defined, then the page content would be held in a separate memory
+
** allocation from the PgHdr1.  This was intended to avoid clownshoe memory
+
** allocations.  However, the btree layer needs a small (16-byte) overrun
+
** area after the page content buffer.  The header serves as that overrun
+
** area.  Therefore SQLITE_PCACHE_SEPARATE_HEADER was discontinued to avoid
+
** any possibility of a memory error.
**
** This module tracks pointers to PgHdr1 objects.  Only pcache.c communicates
** with this module.  Information is passed back and forth as PgHdr1 pointers.
@@ -51355,30 +53272,40 @@ typedef struct PGroup PGroup;

/*
** Each cache entry is represented by an instance of the following
-
** structure. Unless SQLITE_PCACHE_SEPARATE_HEADER is defined, a buffer of
-
** PgHdr1.pCache->szPage bytes is allocated directly before this structure
-
** in memory.
+
** structure. A buffer of PgHdr1.pCache->szPage bytes is allocated
+
** directly before this structure and is used to cache the page content.
**
-
** Note: Variables isBulkLocal and isAnchor were once type "u8". That works,
+
** When reading a corrupt database file, it is possible that SQLite might
+
** read a few bytes (no more than 16 bytes) past the end of the page buffer.
+
** It will only read past the end of the page buffer, never write.  This
+
** object is positioned immediately after the page buffer to serve as an
+
** overrun area, so that overreads are harmless.
+
**
+
** Variables isBulkLocal and isAnchor were once type "u8". That works,
** but causes a 2-byte gap in the structure for most architectures (since
** pointers must be either 4 or 8-byte aligned). As this structure is located
** in memory directly after the associated page data, if the database is
** corrupt, code at the b-tree layer may overread the page buffer and
** read part of this structure before the corruption is detected. This
** can cause a valgrind error if the unitialized gap is accessed. Using u16
-
** ensures there is no such gap, and therefore no bytes of unitialized memory
-
** in the structure.
+
** ensures there is no such gap, and therefore no bytes of uninitialized
+
** memory in the structure.
+
**
+
** The pLruNext and pLruPrev pointers form a double-linked circular list
+
** of all pages that are unpinned.  The PGroup.lru element (which should be
+
** the only element on the list with PgHdr1.isAnchor set to 1) forms the
+
** beginning and the end of the list.
*/
struct PgHdr1 {
-
  sqlite3_pcache_page page;      /* Base class. Must be first. pBuf & pExtra */
-
  unsigned int iKey;             /* Key value (page number) */
-
  u16 isBulkLocal;               /* This page from bulk local storage */
-
  u16 isAnchor;                  /* This is the PGroup.lru element */
-
  PgHdr1 *pNext;                 /* Next in hash table chain */
-
  PCache1 *pCache;               /* Cache that currently owns this page */
-
  PgHdr1 *pLruNext;              /* Next in LRU list of unpinned pages */
-
  PgHdr1 *pLruPrev;              /* Previous in LRU list of unpinned pages */
-
                                 /* NB: pLruPrev is only valid if pLruNext!=0 */
+
  sqlite3_pcache_page page; /* Base class. Must be first. pBuf & pExtra */
+
  unsigned int iKey;        /* Key value (page number) */
+
  u16 isBulkLocal;          /* This page from bulk local storage */
+
  u16 isAnchor;             /* This is the PGroup.lru element */
+
  PgHdr1 *pNext;            /* Next in hash table chain */
+
  PCache1 *pCache;          /* Cache that currently owns this page */
+
  PgHdr1 *pLruNext;         /* Next in circular LRU list of unpinned pages */
+
  PgHdr1 *pLruPrev;         /* Previous in LRU list of unpinned pages */
+
                            /* NB: pLruPrev is only valid if pLruNext!=0 */
};

/*
@@ -51704,25 +53631,13 @@ static PgHdr1 *pcache1AllocPage(PCache1 *pCache, int benignMalloc){
    pcache1LeaveMutex(pCache->pGroup);
#endif
    if( benignMalloc ){ sqlite3BeginBenignMalloc(); }
-
#ifdef SQLITE_PCACHE_SEPARATE_HEADER
-
    pPg = pcache1Alloc(pCache->szPage);
-
    p = sqlite3Malloc(sizeof(PgHdr1) + pCache->szExtra);
-
    if( !pPg || !p ){
-
      pcache1Free(pPg);
-
      sqlite3_free(p);
-
      pPg = 0;
-
    }
-
#else
    pPg = pcache1Alloc(pCache->szAlloc);
-
#endif
    if( benignMalloc ){ sqlite3EndBenignMalloc(); }
#ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT
    pcache1EnterMutex(pCache->pGroup);
#endif
    if( pPg==0 ) return 0;
-
#ifndef SQLITE_PCACHE_SEPARATE_HEADER
    p = (PgHdr1 *)&((u8 *)pPg)[pCache->szPage];
-
#endif
    p->page.pBuf = pPg;
    p->page.pExtra = &p[1];
    p->isBulkLocal = 0;
@@ -51746,9 +53661,6 @@ static void pcache1FreePage(PgHdr1 *p){
    pCache->pFree = p;
  }else{
    pcache1Free(p->page.pBuf);
-
#ifdef SQLITE_PCACHE_SEPARATE_HEADER
-
    sqlite3_free(p);
-
#endif
  }
  (*pCache->pnPurgeable)--;
}
@@ -52389,23 +54301,26 @@ static void pcache1Rekey(
  PCache1 *pCache = (PCache1 *)p;
  PgHdr1 *pPage = (PgHdr1 *)pPg;
  PgHdr1 **pp;
-
  unsigned int h;
+
  unsigned int hOld, hNew;
  assert( pPage->iKey==iOld );
  assert( pPage->pCache==pCache );
+
  assert( iOld!=iNew );               /* The page number really is changing */

  pcache1EnterMutex(pCache->pGroup);

-
  h = iOld%pCache->nHash;
-
  pp = &pCache->apHash[h];
+
  assert( pcache1FetchNoMutex(p, iOld, 0)==pPage ); /* pPg really is iOld */
+
  hOld = iOld%pCache->nHash;
+
  pp = &pCache->apHash[hOld];
  while( (*pp)!=pPage ){
    pp = &(*pp)->pNext;
  }
  *pp = pPage->pNext;

-
  h = iNew%pCache->nHash;
+
  assert( pcache1FetchNoMutex(p, iNew, 0)==0 ); /* iNew not in cache */
+
  hNew = iNew%pCache->nHash;
  pPage->iKey = iNew;
-
  pPage->pNext = pCache->apHash[h];
-
  pCache->apHash[h] = pPage;
+
  pPage->pNext = pCache->apHash[hNew];
+
  pCache->apHash[hNew] = pPage;
  if( iNew>pCache->iMaxKey ){
    pCache->iMaxKey = iNew;
  }
@@ -52512,9 +54427,6 @@ SQLITE_PRIVATE int sqlite3PcacheReleaseMemory(int nReq){
       &&  p->isAnchor==0
    ){
      nFree += pcache1MemSize(p->page.pBuf);
-
#ifdef SQLITE_PCACHE_SEPARATE_HEADER
-
      nFree += sqlite3MemSize(p);
-
#endif
      assert( PAGE_IS_UNPINNED(p) );
      pcache1PinPage(p);
      pcache1RemoveFromHash(p, 1);
@@ -53898,6 +55810,7 @@ struct Pager {
  u32 vfsFlags;               /* Flags for sqlite3_vfs.xOpen() */
  u32 sectorSize;             /* Assumed sector size during rollback */
  Pgno mxPgno;                /* Maximum allowed size of the database */
+
  Pgno lckPgno;               /* Page number for the locking page */
  i64 pageSize;               /* Number of bytes in a page */
  i64 journalSizeLimit;       /* Size limit for persistent journal files */
  char *zFilename;            /* Name of the database file */
@@ -54884,7 +56797,7 @@ static int readJournalHdr(
** journal file descriptor is advanced to the next sector boundary before
** anything is written. The format is:
**
-
**   + 4 bytes: PAGER_MJ_PGNO.
+
**   + 4 bytes: PAGER_SJ_PGNO.
**   + N bytes: super-journal filename in utf-8.
**   + 4 bytes: N (length of super-journal name in bytes, no nul-terminator).
**   + 4 bytes: super-journal name checksum.
@@ -54932,7 +56845,7 @@ static int writeSuperJournal(Pager *pPager, const char *zSuper){
  /* Write the super-journal data to the end of the journal file. If
  ** an error occurs, return the error code to the caller.
  */
-
  if( (0 != (rc = write32bits(pPager->jfd, iHdrOff, PAGER_MJ_PGNO(pPager))))
+
  if( (0 != (rc = write32bits(pPager->jfd, iHdrOff, PAGER_SJ_PGNO(pPager))))
   || (0 != (rc = sqlite3OsWrite(pPager->jfd, zSuper, nSuper, iHdrOff+4)))
   || (0 != (rc = write32bits(pPager->jfd, iHdrOff+4+nSuper, nSuper)))
   || (0 != (rc = write32bits(pPager->jfd, iHdrOff+4+nSuper+4, cksum)))
@@ -55442,7 +57355,7 @@ static u32 pager_cksum(Pager *pPager, const u8 *aData){
** corrupted, SQLITE_DONE is returned. Data is considered corrupted in
** two circumstances:
**
-
**   * If the record page-number is illegal (0 or PAGER_MJ_PGNO), or
+
**   * If the record page-number is illegal (0 or PAGER_SJ_PGNO), or
**   * If the record is being rolled back from the main journal file
**     and the checksum field does not match the record content.
**
@@ -55502,7 +57415,7 @@ static int pager_playback_one_page(
  ** it could cause invalid data to be written into the journal.  We need to
  ** detect this invalid data (with high probability) and ignore it.
  */
-
  if( pgno==0 || pgno==PAGER_MJ_PGNO(pPager) ){
+
  if( pgno==0 || pgno==PAGER_SJ_PGNO(pPager) ){
    assert( !isSavepnt );
    return SQLITE_DONE;
  }
@@ -55839,6 +57752,7 @@ static int pager_truncate(Pager *pPager, Pgno nPage){
        memset(pTmp, 0, szPage);
        testcase( (newSize-szPage) == currentSize );
        testcase( (newSize-szPage) >  currentSize );
+
        sqlite3OsFileControlHint(pPager->fd, SQLITE_FCNTL_SIZE_HINT, &newSize);
        rc = sqlite3OsWrite(pPager->fd, pTmp, szPage, newSize-szPage);
      }
      if( rc==SQLITE_OK ){
@@ -56960,6 +58874,7 @@ SQLITE_PRIVATE int sqlite3PagerSetPagesize(Pager *pPager, u32 *pPageSize, int nR
      pPager->pTmpSpace = pNew;
      pPager->dbSize = (Pgno)((nByte+pageSize-1)/pageSize);
      pPager->pageSize = pageSize;
+
      pPager->lckPgno = (Pgno)(PENDING_BYTE/pageSize) + 1;
    }else{
      sqlite3PageFree(pNew);
    }
@@ -58729,7 +60644,7 @@ static int getPageNormal(
  if( pPg->pPager && !noContent ){
    /* In this case the pcache already contains an initialized copy of
    ** the page. Return without further ado.  */
-
    assert( pgno!=PAGER_MJ_PGNO(pPager) );
+
    assert( pgno!=PAGER_SJ_PGNO(pPager) );
    pPager->aStat[PAGER_STAT_HIT]++;
    return SQLITE_OK;

@@ -58740,7 +60655,7 @@ static int getPageNormal(
    ** (*) obsolete.  Was: maximum page number is 2^31
    ** (2) Never try to fetch the locking page
    */
-
    if( pgno==PAGER_MJ_PGNO(pPager) ){
+
    if( pgno==PAGER_SJ_PGNO(pPager) ){
      rc = SQLITE_CORRUPT_BKPT;
      goto pager_acquire_err;
    }
@@ -59000,6 +60915,7 @@ static int pager_open_journal(Pager *pPager){

        if( pPager->tempFile ){
          flags |= (SQLITE_OPEN_DELETEONCLOSE|SQLITE_OPEN_TEMP_JOURNAL);
+
          flags |= SQLITE_OPEN_EXCLUSIVE;
          nSpill = sqlite3Config.nStmtSpill;
        }else{
          flags |= SQLITE_OPEN_MAIN_JOURNAL;
@@ -59035,6 +60951,7 @@ static int pager_open_journal(Pager *pPager){
  if( rc!=SQLITE_OK ){
    sqlite3BitvecDestroy(pPager->pInJournal);
    pPager->pInJournal = 0;
+
    pPager->journalOff = 0;
  }else{
    assert( pPager->eState==PAGER_WRITER_LOCKED );
    pPager->eState = PAGER_WRITER_CACHEMOD;
@@ -59139,7 +61056,7 @@ static SQLITE_NOINLINE int pagerAddPageToRollbackJournal(PgHdr *pPg){
  /* We should never write to the journal file the page that
  ** contains the database locks.  The following assert verifies
  ** that we do not. */
-
  assert( pPg->pgno!=PAGER_MJ_PGNO(pPager) );
+
  assert( pPg->pgno!=PAGER_SJ_PGNO(pPager) );

  assert( pPager->journalHdr<=pPager->journalOff );
  pData2 = pPg->pData;
@@ -59318,7 +61235,7 @@ static SQLITE_NOINLINE int pagerWriteLargeSector(PgHdr *pPg){
    Pgno pg = pg1+ii;
    PgHdr *pPage;
    if( pg==pPg->pgno || !sqlite3BitvecTest(pPager->pInJournal, pg) ){
-
      if( pg!=PAGER_MJ_PGNO(pPager) ){
+
      if( pg!=PAGER_SJ_PGNO(pPager) ){
        rc = sqlite3PagerGet(pPager, pg, &pPage, 0);
        if( rc==SQLITE_OK ){
          rc = pager_write(pPage);
@@ -59796,7 +61713,7 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseOne(
      ** last page is never written out to disk, leaving the database file
      ** undersized. Fix this now if it is the case.  */
      if( pPager->dbSize>pPager->dbFileSize ){
-
        Pgno nNew = pPager->dbSize - (pPager->dbSize==PAGER_MJ_PGNO(pPager));
+
        Pgno nNew = pPager->dbSize - (pPager->dbSize==PAGER_SJ_PGNO(pPager));
        assert( pPager->eState==PAGER_WRITER_DBMOD );
        rc = pager_truncate(pPager, nNew);
        if( rc!=SQLITE_OK ) goto commit_phase_one_exit;
@@ -65425,7 +67342,7 @@ struct MemPage {
  u8 *aData;           /* Pointer to disk image of the page data */
  u8 *aDataEnd;        /* One byte past the end of the entire page - not just
                       ** the usable space, the entire page.  Used to prevent
-
                       ** corruption-induced of buffer overflow. */
+
                       ** corruption-induced buffer overflow. */
  u8 *aCellIdx;        /* The cell index area */
  u8 *aDataOfst;       /* Same as aData for leaves.  aData+4 for interior */
  DbPage *pDbPage;     /* Pager page handle */
@@ -65730,7 +67647,7 @@ struct BtCursor {
/*
** The database page the PENDING_BYTE occupies. This page is never used.
*/
-
# define PENDING_BYTE_PAGE(pBt) PAGER_MJ_PGNO(pBt)
+
#define PENDING_BYTE_PAGE(pBt)  ((Pgno)((PENDING_BYTE/((pBt)->pageSize))+1))

/*
** These macros define the location of the pointer-map entry for a
@@ -66098,6 +68015,7 @@ SQLITE_PRIVATE int sqlite3BtreeHoldsAllMutexes(sqlite3 *db){
SQLITE_PRIVATE int sqlite3SchemaMutexHeld(sqlite3 *db, int iDb, Schema *pSchema){
  Btree *p;
  assert( db!=0 );
+
  if( db->pVfs==0 && db->nDb==0 ) return 1;
  if( pSchema ) iDb = sqlite3SchemaToIndex(db, pSchema);
  assert( iDb>=0 && iDb<db->nDb );
  if( !sqlite3_mutex_held(db->mutex) ) return 0;
@@ -66371,7 +68289,7 @@ static int hasSharedCacheTableLock(
    int bSeen = 0;
    for(p=sqliteHashFirst(&pSchema->idxHash); p; p=sqliteHashNext(p)){
      Index *pIdx = (Index *)sqliteHashData(p);
-
      if( pIdx->tnum==(int)iRoot ){
+
      if( pIdx->tnum==iRoot ){
        if( bSeen ){
          /* Two or more indexes share the same root page.  There must
          ** be imposter tables.  So just return true.  The assert is not
@@ -66964,7 +68882,7 @@ SQLITE_PRIVATE void sqlite3BtreeClearCursor(BtCursor *pCur){
/*
** In this version of BtreeMoveto, pKey is a packed index record
** such as is generated by the OP_MakeRecord opcode.  Unpack the
-
** record and then call BtreeMovetoUnpacked() to do the work.
+
** record and then call sqlite3BtreeIndexMoveto() to do the work.
*/
static int btreeMoveto(
  BtCursor *pCur,     /* Cursor open on the btree to be searched */
@@ -67484,6 +69402,7 @@ static void btreeParseCell(
** the space used by the cell pointer.
**
** cellSizePtrNoPayload()    =>   table internal nodes
+
** cellSizePtrTableLeaf()    =>   table leaf nodes
** cellSizePtr()             =>   all index nodes & table leaf nodes
*/
static u16 cellSizePtr(MemPage *pPage, u8 *pCell){
@@ -67509,13 +69428,6 @@ static u16 cellSizePtr(MemPage *pPage, u8 *pCell){
    }while( *(pIter)>=0x80 && pIter<pEnd );
  }
  pIter++;
-
  if( pPage->intKey ){
-
    /* pIter now points at the 64-bit integer key value, a variable length
-
    ** integer. The following block moves pIter to point at the first byte
-
    ** past the end of the key value. */
-
    pEnd = &pIter[9];
-
    while( (*pIter++)&0x80 && pIter<pEnd );
-
  }
  testcase( nSize==pPage->maxLocal );
  testcase( nSize==(u32)pPage->maxLocal+1 );
  if( nSize<=pPage->maxLocal ){
@@ -67555,6 +69467,58 @@ static u16 cellSizePtrNoPayload(MemPage *pPage, u8 *pCell){
  assert( debuginfo.nSize==(u16)(pIter - pCell) || CORRUPT_DB );
  return (u16)(pIter - pCell);
}
+
static u16 cellSizePtrTableLeaf(MemPage *pPage, u8 *pCell){
+
  u8 *pIter = pCell;   /* For looping over bytes of pCell */
+
  u8 *pEnd;            /* End mark for a varint */
+
  u32 nSize;           /* Size value to return */
+

+
#ifdef SQLITE_DEBUG
+
  /* The value returned by this function should always be the same as
+
  ** the (CellInfo.nSize) value found by doing a full parse of the
+
  ** cell. If SQLITE_DEBUG is defined, an assert() at the bottom of
+
  ** this function verifies that this invariant is not violated. */
+
  CellInfo debuginfo;
+
  pPage->xParseCell(pPage, pCell, &debuginfo);
+
#endif
+

+
  nSize = *pIter;
+
  if( nSize>=0x80 ){
+
    pEnd = &pIter[8];
+
    nSize &= 0x7f;
+
    do{
+
      nSize = (nSize<<7) | (*++pIter & 0x7f);
+
    }while( *(pIter)>=0x80 && pIter<pEnd );
+
  }
+
  pIter++;
+
  /* pIter now points at the 64-bit integer key value, a variable length
+
  ** integer. The following block moves pIter to point at the first byte
+
  ** past the end of the key value. */
+
  if( (*pIter++)&0x80
+
   && (*pIter++)&0x80
+
   && (*pIter++)&0x80
+
   && (*pIter++)&0x80
+
   && (*pIter++)&0x80
+
   && (*pIter++)&0x80
+
   && (*pIter++)&0x80
+
   && (*pIter++)&0x80 ){ pIter++; }
+
  testcase( nSize==pPage->maxLocal );
+
  testcase( nSize==(u32)pPage->maxLocal+1 );
+
  if( nSize<=pPage->maxLocal ){
+
    nSize += (u32)(pIter - pCell);
+
    if( nSize<4 ) nSize = 4;
+
  }else{
+
    int minLocal = pPage->minLocal;
+
    nSize = minLocal + (nSize - minLocal) % (pPage->pBt->usableSize - 4);
+
    testcase( nSize==pPage->maxLocal );
+
    testcase( nSize==(u32)pPage->maxLocal+1 );
+
    if( nSize>pPage->maxLocal ){
+
      nSize = minLocal;
+
    }
+
    nSize += 4 + (u16)(pIter - pCell);
+
  }
+
  assert( nSize==debuginfo.nSize || CORRUPT_DB );
+
  return (u16)nSize;
+
}


#ifdef SQLITE_DEBUG
@@ -67568,7 +69532,7 @@ static u16 cellSize(MemPage *pPage, int iCell){
#ifndef SQLITE_OMIT_AUTOVACUUM
/*
** The cell pCell is currently part of page pSrc but will ultimately be part
-
** of pPage.  (pSrc and pPager are often the same.)  If pCell contains a
+
** of pPage.  (pSrc and pPage are often the same.)  If pCell contains a
** pointer to an overflow page, insert an entry into the pointer-map for
** the overflow page that will be valid after pCell has been moved to pPage.
*/
@@ -67624,8 +69588,7 @@ static int defragmentPage(MemPage *pPage, int nMaxFrag){
  assert( pPage->pBt->usableSize <= SQLITE_MAX_PAGE_SIZE );
  assert( pPage->nOverflow==0 );
  assert( sqlite3_mutex_held(pPage->pBt->mutex) );
-
  temp = 0;
-
  src = data = pPage->aData;
+
  data = pPage->aData;
  hdr = pPage->hdrOffset;
  cellOffset = pPage->cellOffset;
  nCell = pPage->nCell;
@@ -67659,7 +69622,7 @@ static int defragmentPage(MemPage *pPage, int nMaxFrag){
          if( iFree2+sz2 > usableSize ) return SQLITE_CORRUPT_PAGE(pPage);
          memmove(&data[iFree+sz+sz2], &data[iFree+sz], iFree2-(iFree+sz));
          sz += sz2;
-
        }else if( NEVER(iFree+sz>usableSize) ){
+
        }else if( iFree+sz>usableSize ){
          return SQLITE_CORRUPT_PAGE(pPage);
        }

@@ -67679,39 +69642,38 @@ static int defragmentPage(MemPage *pPage, int nMaxFrag){
  cbrk = usableSize;
  iCellLast = usableSize - 4;
  iCellStart = get2byte(&data[hdr+5]);
-
  for(i=0; i<nCell; i++){
-
    u8 *pAddr;     /* The i-th cell pointer */
-
    pAddr = &data[cellOffset + i*2];
-
    pc = get2byte(pAddr);
-
    testcase( pc==iCellFirst );
-
    testcase( pc==iCellLast );
-
    /* These conditions have already been verified in btreeInitPage()
-
    ** if PRAGMA cell_size_check=ON.
-
    */
-
    if( pc<iCellStart || pc>iCellLast ){
-
      return SQLITE_CORRUPT_PAGE(pPage);
-
    }
-
    assert( pc>=iCellStart && pc<=iCellLast );
-
    size = pPage->xCellSize(pPage, &src[pc]);
-
    cbrk -= size;
-
    if( cbrk<iCellStart || pc+size>usableSize ){
-
      return SQLITE_CORRUPT_PAGE(pPage);
-
    }
-
    assert( cbrk+size<=usableSize && cbrk>=iCellStart );
-
    testcase( cbrk+size==usableSize );
-
    testcase( pc+size==usableSize );
-
    put2byte(pAddr, cbrk);
-
    if( temp==0 ){
-
      if( cbrk==pc ) continue;
-
      temp = sqlite3PagerTempSpace(pPage->pBt->pPager);
-
      memcpy(&temp[iCellStart], &data[iCellStart], usableSize - iCellStart);
-
      src = temp;
+
  if( nCell>0 ){
+
    temp = sqlite3PagerTempSpace(pPage->pBt->pPager);
+
    memcpy(&temp[iCellStart], &data[iCellStart], usableSize - iCellStart);
+
    src = temp;
+
    for(i=0; i<nCell; i++){
+
      u8 *pAddr;     /* The i-th cell pointer */
+
      pAddr = &data[cellOffset + i*2];
+
      pc = get2byte(pAddr);
+
      testcase( pc==iCellFirst );
+
      testcase( pc==iCellLast );
+
      /* These conditions have already been verified in btreeInitPage()
+
      ** if PRAGMA cell_size_check=ON.
+
      */
+
      if( pc<iCellStart || pc>iCellLast ){
+
        return SQLITE_CORRUPT_PAGE(pPage);
+
      }
+
      assert( pc>=iCellStart && pc<=iCellLast );
+
      size = pPage->xCellSize(pPage, &src[pc]);
+
      cbrk -= size;
+
      if( cbrk<iCellStart || pc+size>usableSize ){
+
        return SQLITE_CORRUPT_PAGE(pPage);
+
      }
+
      assert( cbrk+size<=usableSize && cbrk>=iCellStart );
+
      testcase( cbrk+size==usableSize );
+
      testcase( pc+size==usableSize );
+
      put2byte(pAddr, cbrk);
+
      memcpy(&data[cbrk], &src[pc], size);
    }
-
    memcpy(&data[cbrk], &src[pc], size);
  }
  data[hdr+7] = 0;

-
 defragment_out:
+
defragment_out:
  assert( pPage->nFree>=0 );
  if( data[hdr+7]+cbrk-iCellFirst!=pPage->nFree ){
    return SQLITE_CORRUPT_PAGE(pPage);
@@ -67743,7 +69705,8 @@ static u8 *pageFindSlot(MemPage *pPg, int nByte, int *pRc){
  const int hdr = pPg->hdrOffset;            /* Offset to page header */
  u8 * const aData = pPg->aData;             /* Page data */
  int iAddr = hdr + 1;                       /* Address of ptr to pc */
-
  int pc = get2byte(&aData[iAddr]);          /* Address of a free slot */
+
  u8 *pTmp = &aData[iAddr];                  /* Temporary ptr into aData[] */
+
  int pc = get2byte(pTmp);                   /* Address of a free slot */
  int x;                                     /* Excess size of the slot */
  int maxPC = pPg->pBt->usableSize - nByte;  /* Max address for a usable slot */
  int size;                                  /* Size of the free slot */
@@ -67753,7 +69716,8 @@ static u8 *pageFindSlot(MemPage *pPg, int nByte, int *pRc){
    /* EVIDENCE-OF: R-22710-53328 The third and fourth bytes of each
    ** freeblock form a big-endian integer which is the size of the freeblock
    ** in bytes, including the 4-byte header. */
-
    size = get2byte(&aData[pc+2]);
+
    pTmp = &aData[pc+2];
+
    size = get2byte(pTmp);
    if( (x = size - nByte)>=0 ){
      testcase( x==4 );
      testcase( x==3 );
@@ -67766,7 +69730,6 @@ static u8 *pageFindSlot(MemPage *pPg, int nByte, int *pRc){
        ** fragmented bytes within the page. */
        memcpy(&aData[iAddr], &aData[pc], 2);
        aData[hdr+7] += (u8)x;
-
        testcase( pc+x>maxPC );
        return &aData[pc];
      }else if( x+pc > maxPC ){
        /* This slot extends off the end of the usable part of the page */
@@ -67780,10 +69743,11 @@ static u8 *pageFindSlot(MemPage *pPg, int nByte, int *pRc){
      return &aData[pc + x];
    }
    iAddr = pc;
-
    pc = get2byte(&aData[pc]);
-
    if( pc<=iAddr+size ){
+
    pTmp = &aData[pc];
+
    pc = get2byte(pTmp);
+
    if( pc<=iAddr ){
      if( pc ){
-
        /* The next slot in the chain is not past the end of the current slot */
+
        /* The next slot in the chain comes before the current slot */
        *pRc = SQLITE_CORRUPT_PAGE(pPg);
      }
      return 0;
@@ -67814,6 +69778,7 @@ static int allocateSpace(MemPage *pPage, int nByte, int *pIdx){
  u8 * const data = pPage->aData;      /* Local cache of pPage->aData */
  int top;                             /* First byte of cell content area */
  int rc = SQLITE_OK;                  /* Integer return code */
+
  u8 *pTmp;                            /* Temp ptr into data[] */
  int gap;        /* First byte of gap between cell pointers and cell content */

  assert( sqlite3PagerIswriteable(pPage->pDbPage) );
@@ -67832,7 +69797,8 @@ static int allocateSpace(MemPage *pPage, int nByte, int *pIdx){
  ** then the cell content offset of an empty page wants to be 65536.
  ** However, that integer is too large to be stored in a 2-byte unsigned
  ** integer, so a value of 0 is used in its place. */
-
  top = get2byte(&data[hdr+5]);
+
  pTmp = &data[hdr+5];
+
  top = get2byte(pTmp);
  assert( top<=(int)pPage->pBt->usableSize ); /* by btreeComputeFreeSpace() */
  if( gap>top ){
    if( top==0 && pPage->pBt->usableSize==65536 ){
@@ -67914,6 +69880,7 @@ static int freeSpace(MemPage *pPage, u16 iStart, u16 iSize){
  u16 x;                                /* Offset to cell content area */
  u32 iEnd = iStart + iSize;            /* First byte past the iStart buffer */
  unsigned char *data = pPage->aData;   /* Page content */
+
  u8 *pTmp;                             /* Temporary ptr into data[] */

  assert( pPage->pBt!=0 );
  assert( sqlite3PagerIswriteable(pPage->pDbPage) );
@@ -67932,7 +69899,7 @@ static int freeSpace(MemPage *pPage, u16 iStart, u16 iSize){
    iFreeBlk = 0;  /* Shortcut for the case when the freelist is empty */
  }else{
    while( (iFreeBlk = get2byte(&data[iPtr]))<iStart ){
-
      if( iFreeBlk<iPtr+4 ){
+
      if( iFreeBlk<=iPtr ){
        if( iFreeBlk==0 ) break; /* TH3: corrupt082.100 */
        return SQLITE_CORRUPT_PAGE(pPage);
      }
@@ -67976,7 +69943,8 @@ static int freeSpace(MemPage *pPage, u16 iStart, u16 iSize){
    if( nFrag>data[hdr+7] ) return SQLITE_CORRUPT_PAGE(pPage);
    data[hdr+7] -= nFrag;
  }
-
  x = get2byte(&data[hdr+5]);
+
  pTmp = &data[hdr+5];
+
  x = get2byte(pTmp);
  if( iStart<=x ){
    /* The new freeblock is at the beginning of the cell content area,
    ** so just extend the cell content area rather than create another
@@ -68020,7 +69988,6 @@ static int decodeFlags(MemPage *pPage, int flagByte){
  pPage->leaf = (u8)(flagByte>>3);  assert( PTF_LEAF == 1<<3 );
  flagByte &= ~PTF_LEAF;
  pPage->childPtrSize = 4-4*pPage->leaf;
-
  pPage->xCellSize = cellSizePtr;
  pBt = pPage->pBt;
  if( flagByte==(PTF_LEAFDATA | PTF_INTKEY) ){
    /* EVIDENCE-OF: R-07291-35328 A value of 5 (0x05) means the page is an
@@ -68032,6 +69999,7 @@ static int decodeFlags(MemPage *pPage, int flagByte){
    pPage->intKey = 1;
    if( pPage->leaf ){
      pPage->intKeyLeaf = 1;
+
      pPage->xCellSize = cellSizePtrTableLeaf;
      pPage->xParseCell = btreeParseCellPtr;
    }else{
      pPage->intKeyLeaf = 0;
@@ -68049,12 +70017,17 @@ static int decodeFlags(MemPage *pPage, int flagByte){
    assert( (PTF_ZERODATA|PTF_LEAF)==10 );
    pPage->intKey = 0;
    pPage->intKeyLeaf = 0;
+
    pPage->xCellSize = cellSizePtr;
    pPage->xParseCell = btreeParseCellPtrIndex;
    pPage->maxLocal = pBt->maxLocal;
    pPage->minLocal = pBt->minLocal;
  }else{
    /* EVIDENCE-OF: R-47608-56469 Any other value for the b-tree page type is
    ** an error. */
+
    pPage->intKey = 0;
+
    pPage->intKeyLeaf = 0;
+
    pPage->xCellSize = cellSizePtr;
+
    pPage->xParseCell = btreeParseCellPtrIndex;
    return SQLITE_CORRUPT_PAGE(pPage);
  }
  pPage->max1bytePayload = pBt->max1bytePayload;
@@ -68408,7 +70381,7 @@ getAndInitPage_error1:
    pCur->pPage = pCur->apPage[pCur->iPage];
  }
  testcase( pgno==0 );
-
  assert( pgno!=0 || rc==SQLITE_CORRUPT );
+
  assert( pgno!=0 || rc!=SQLITE_OK );
  return rc;
}

@@ -69844,6 +71817,9 @@ static int modifyPagePointer(MemPage *pPage, Pgno iFrom, Pgno iTo, u8 eType){
          }
        }
      }else{
+
        if( pCell+4 > pPage->aData+pPage->pBt->usableSize ){
+
          return SQLITE_CORRUPT_PAGE(pPage);
+
        }
        if( get4byte(pCell)==iFrom ){
          put4byte(pCell, iTo);
          break;
@@ -70030,12 +72006,17 @@ static int incrVacuumStep(BtShared *pBt, Pgno nFin, Pgno iLastPg, int bCommit){
      }
      do {
        MemPage *pFreePg;
+
        Pgno dbSize = btreePagecount(pBt);
        rc = allocateBtreePage(pBt, &pFreePg, &iFreePg, iNear, eMode);
        if( rc!=SQLITE_OK ){
          releasePage(pLastPg);
          return rc;
        }
        releasePage(pFreePg);
+
        if( iFreePg>dbSize ){
+
          releasePage(pLastPg);
+
          return SQLITE_CORRUPT_BKPT;
+
        }
      }while( bCommit && iFreePg>nFin );
      assert( iFreePg<iLastPg );

@@ -71345,8 +73326,6 @@ SQLITE_PRIVATE const void *sqlite3BtreePayloadFetch(BtCursor *pCur, u32 *pAmt){
** vice-versa).
*/
static int moveToChild(BtCursor *pCur, u32 newPgno){
-
  BtShared *pBt = pCur->pBt;
-

  assert( cursorOwnsBtShared(pCur) );
  assert( pCur->eState==CURSOR_VALID );
  assert( pCur->iPage<BTCURSOR_MAX_DEPTH );
@@ -71360,7 +73339,8 @@ static int moveToChild(BtCursor *pCur, u32 newPgno){
  pCur->apPage[pCur->iPage] = pCur->pPage;
  pCur->ix = 0;
  pCur->iPage++;
-
  return getAndInitPage(pBt, newPgno, &pCur->pPage, pCur, pCur->curPagerFlags);
+
  return getAndInitPage(pCur->pBt, newPgno, &pCur->pPage, pCur,
+
                        pCur->curPagerFlags);
}

#ifdef SQLITE_DEBUG
@@ -71466,7 +73446,7 @@ static int moveToRoot(BtCursor *pCur){
      }
      sqlite3BtreeClearCursor(pCur);
    }
-
    rc = getAndInitPage(pCur->pBtree->pBt, pCur->pgnoRoot, &pCur->pPage,
+
    rc = getAndInitPage(pCur->pBt, pCur->pgnoRoot, &pCur->pPage,
                        0, pCur->curPagerFlags);
    if( rc!=SQLITE_OK ){
      pCur->eState = CURSOR_INVALID;
@@ -71797,6 +73777,69 @@ moveto_table_finish:
  return rc;
}

+
/*
+
** Compare the "idx"-th cell on the page the cursor pCur is currently
+
** pointing to to pIdxKey using xRecordCompare.  Return negative or
+
** zero if the cell is less than or equal pIdxKey.  Return positive
+
** if unknown.
+
**
+
**    Return value negative:     Cell at pCur[idx] less than pIdxKey
+
**
+
**    Return value is zero:      Cell at pCur[idx] equals pIdxKey
+
**
+
**    Return value positive:     Nothing is known about the relationship
+
**                               of the cell at pCur[idx] and pIdxKey.
+
**
+
** This routine is part of an optimization.  It is always safe to return
+
** a positive value as that will cause the optimization to be skipped.
+
*/
+
static int indexCellCompare(
+
  BtCursor *pCur,
+
  int idx,
+
  UnpackedRecord *pIdxKey,
+
  RecordCompare xRecordCompare
+
){
+
  MemPage *pPage = pCur->pPage;
+
  int c;
+
  int nCell;  /* Size of the pCell cell in bytes */
+
  u8 *pCell = findCellPastPtr(pPage, idx);
+

+
  nCell = pCell[0];
+
  if( nCell<=pPage->max1bytePayload ){
+
    /* This branch runs if the record-size field of the cell is a
+
    ** single byte varint and the record fits entirely on the main
+
    ** b-tree page.  */
+
    testcase( pCell+nCell+1==pPage->aDataEnd );
+
    c = xRecordCompare(nCell, (void*)&pCell[1], pIdxKey);
+
  }else if( !(pCell[1] & 0x80)
+
    && (nCell = ((nCell&0x7f)<<7) + pCell[1])<=pPage->maxLocal
+
  ){
+
    /* The record-size field is a 2 byte varint and the record
+
    ** fits entirely on the main b-tree page.  */
+
    testcase( pCell+nCell+2==pPage->aDataEnd );
+
    c = xRecordCompare(nCell, (void*)&pCell[2], pIdxKey);
+
  }else{
+
    /* If the record extends into overflow pages, do not attempt
+
    ** the optimization. */
+
    c = 99;
+
  }
+
  return c;
+
}
+

+
/*
+
** Return true (non-zero) if pCur is current pointing to the last
+
** page of a table.
+
*/
+
static int cursorOnLastPage(BtCursor *pCur){
+
  int i;
+
  assert( pCur->eState==CURSOR_VALID );
+
  for(i=0; i<pCur->iPage; i++){
+
    MemPage *pPage = pCur->apPage[i];
+
    if( pCur->aiIdx[i]<pPage->nCell ) return 0;
+
  }
+
  return 1;
+
}
+

/* Move the cursor so that it points to an entry in an index table
** near the key pIdxKey.   Return a success code.
**
@@ -71847,6 +73890,43 @@ SQLITE_PRIVATE int sqlite3BtreeIndexMoveto(
       || pIdxKey->default_rc==-1
  );

+

+
  /* Check to see if we can skip a lot of work.  Two cases:
+
  **
+
  **    (1) If the cursor is already pointing to the very last cell
+
  **        in the table and the pIdxKey search key is greater than or
+
  **        equal to that last cell, then no movement is required.
+
  **
+
  **    (2) If the cursor is on the last page of the table and the first
+
  **        cell on that last page is less than or equal to the pIdxKey
+
  **        search key, then we can start the search on the current page
+
  **        without needing to go back to root.
+
  */
+
  if( pCur->eState==CURSOR_VALID
+
   && pCur->pPage->leaf
+
   && cursorOnLastPage(pCur)
+
  ){
+
    int c;
+
    if( pCur->ix==pCur->pPage->nCell-1
+
     && (c = indexCellCompare(pCur, pCur->ix, pIdxKey, xRecordCompare))<=0
+
     && pIdxKey->errCode==SQLITE_OK
+
    ){
+
      *pRes = c;
+
      return SQLITE_OK;  /* Cursor already pointing at the correct spot */
+
    }
+
    if( pCur->iPage>0
+
     && indexCellCompare(pCur, 0, pIdxKey, xRecordCompare)<=0
+
     && pIdxKey->errCode==SQLITE_OK
+
    ){
+
      pCur->curFlags &= ~BTCF_ValidOvfl;
+
      if( !pCur->pPage->isInit ){
+
        return SQLITE_CORRUPT_BKPT;
+
      }
+
      goto bypass_moveto_root;  /* Start search on the current page */
+
    }
+
    pIdxKey->errCode = SQLITE_OK;
+
  }
+

  rc = moveToRoot(pCur);
  if( rc ){
    if( rc==SQLITE_EMPTY ){
@@ -71856,12 +73936,14 @@ SQLITE_PRIVATE int sqlite3BtreeIndexMoveto(
    }
    return rc;
  }
+

+
bypass_moveto_root:
  assert( pCur->pPage );
  assert( pCur->pPage->isInit );
  assert( pCur->eState==CURSOR_VALID );
  assert( pCur->pPage->nCell > 0 );
-
  assert( pCur->iPage==0 || pCur->apPage[0]->intKey==pCur->curIntKey );
-
  assert( pCur->curIntKey || pIdxKey );
+
  assert( pCur->curIntKey==0 );
+
  assert( pIdxKey!=0 );
  for(;;){
    int lwr, upr, idx, c;
    Pgno chldPg;
@@ -71875,7 +73957,7 @@ SQLITE_PRIVATE int sqlite3BtreeIndexMoveto(
    ** be the right kind (index or table) of b-tree page. Otherwise
    ** a moveToChild() or moveToRoot() call would have detected corruption.  */
    assert( pPage->nCell>0 );
-
    assert( pPage->intKey==(pIdxKey==0) );
+
    assert( pPage->intKey==0 );
    lwr = 0;
    upr = pPage->nCell-1;
    idx = upr>>1; /* idx = (lwr+upr)/2; */
@@ -72072,14 +74154,7 @@ static SQLITE_NOINLINE int btreeNext(BtCursor *pCur){

  pPage = pCur->pPage;
  idx = ++pCur->ix;
-
  if( !pPage->isInit || sqlite3FaultSim(412) ){
-
    /* The only known way for this to happen is for there to be a
-
    ** recursive SQL function that does a DELETE operation as part of a
-
    ** SELECT which deletes content out from under an active cursor
-
    ** in a corrupt database file where the table being DELETE-ed from
-
    ** has pages in common with the table being queried.  See TH3
-
    ** module cov1/btree78.test testcase 220 (2018-06-08) for an
-
    ** example. */
+
  if( NEVER(!pPage->isInit) || sqlite3FaultSim(412) ){
    return SQLITE_CORRUPT_BKPT;
  }

@@ -72255,8 +74330,8 @@ static int allocateBtreePage(
  assert( eMode==BTALLOC_ANY || (nearby>0 && IfNotOmitAV(pBt->autoVacuum)) );
  pPage1 = pBt->pPage1;
  mxPage = btreePagecount(pBt);
-
  /* EVIDENCE-OF: R-05119-02637 The 4-byte big-endian integer at offset 36
-
  ** stores stores the total number of pages on the freelist. */
+
  /* EVIDENCE-OF: R-21003-45125 The 4-byte big-endian integer at offset 36
+
  ** stores the total number of pages on the freelist. */
  n = get4byte(&pPage1->aData[36]);
  testcase( n==mxPage-1 );
  if( n>=mxPage ){
@@ -73005,12 +75080,6 @@ static void dropCell(MemPage *pPage, int idx, int sz, int *pRC){
  assert( pPage->pBt->usableSize > (u32)(ptr-data) );
  pc = get2byte(ptr);
  hdr = pPage->hdrOffset;
-
#if 0  /* Not required.  Omit for efficiency */
-
  if( pc<hdr+pPage->nCell*2 ){
-
    *pRC = SQLITE_CORRUPT_BKPT;
-
    return;
-
  }
-
#endif
  testcase( pc==(u32)get2byte(&data[hdr+5]) );
  testcase( pc+sz==pPage->pBt->usableSize );
  if( pc+sz > pPage->pBt->usableSize ){
@@ -73894,8 +75963,6 @@ static int balance_nonroot(
  Pgno pgno;                   /* Temp var to store a page number in */
  u8 abDone[NB+2];             /* True after i'th new page is populated */
  Pgno aPgno[NB+2];            /* Page numbers of new pages before shuffling */
-
  Pgno aPgOrder[NB+2];         /* Copy of aPgno[] used for sorting pages */
-
  u16 aPgFlags[NB+2];          /* flags field of new pages before shuffling */
  CellArray b;                 /* Parsed information on cells being balanced */

  memset(abDone, 0, sizeof(abDone));
@@ -74319,42 +76386,39 @@ static int balance_nonroot(
  ** of the table is closer to a linear scan through the file. That in turn
  ** helps the operating system to deliver pages from the disk more rapidly.
  **
-
  ** An O(n^2) insertion sort algorithm is used, but since n is never more
-
  ** than (NB+2) (a small constant), that should not be a problem.
+
  ** An O(N*N) sort algorithm is used, but since N is never more than NB+2
+
  ** (5), that is not a performance concern.
  **
  ** When NB==3, this one optimization makes the database about 25% faster
  ** for large insertions and deletions.
  */
  for(i=0; i<nNew; i++){
-
    aPgOrder[i] = aPgno[i] = apNew[i]->pgno;
-
    aPgFlags[i] = apNew[i]->pDbPage->flags;
-
    for(j=0; j<i; j++){
-
      if( NEVER(aPgno[j]==aPgno[i]) ){
-
        /* This branch is taken if the set of sibling pages somehow contains
-
        ** duplicate entries. This can happen if the database is corrupt.
-
        ** It would be simpler to detect this as part of the loop below, but
-
        ** we do the detection here in order to avoid populating the pager
-
        ** cache with two separate objects associated with the same
-
        ** page number.  */
-
        assert( CORRUPT_DB );
-
        rc = SQLITE_CORRUPT_BKPT;
-
        goto balance_cleanup;
-
      }
-
    }
+
    aPgno[i] = apNew[i]->pgno;
+
    assert( apNew[i]->pDbPage->flags & PGHDR_WRITEABLE );
+
    assert( apNew[i]->pDbPage->flags & PGHDR_DIRTY );
  }
-
  for(i=0; i<nNew; i++){
-
    int iBest = 0;                /* aPgno[] index of page number to use */
-
    for(j=1; j<nNew; j++){
-
      if( aPgOrder[j]<aPgOrder[iBest] ) iBest = j;
+
  for(i=0; i<nNew-1; i++){
+
    int iB = i;
+
    for(j=i+1; j<nNew; j++){
+
      if( apNew[j]->pgno < apNew[iB]->pgno ) iB = j;
    }
-
    pgno = aPgOrder[iBest];
-
    aPgOrder[iBest] = 0xffffffff;
-
    if( iBest!=i ){
-
      if( iBest>i ){
-
        sqlite3PagerRekey(apNew[iBest]->pDbPage, pBt->nPage+iBest+1, 0);
-
      }
-
      sqlite3PagerRekey(apNew[i]->pDbPage, pgno, aPgFlags[iBest]);
-
      apNew[i]->pgno = pgno;
+

+
    /* If apNew[i] has a page number that is bigger than any of the
+
    ** subsequence apNew[i] entries, then swap apNew[i] with the subsequent
+
    ** entry that has the smallest page number (which we know to be
+
    ** entry apNew[iB]).
+
    */
+
    if( iB!=i ){
+
      Pgno pgnoA = apNew[i]->pgno;
+
      Pgno pgnoB = apNew[iB]->pgno;
+
      Pgno pgnoTemp = (PENDING_BYTE/pBt->pageSize)+1;
+
      u16 fgA = apNew[i]->pDbPage->flags;
+
      u16 fgB = apNew[iB]->pDbPage->flags;
+
      sqlite3PagerRekey(apNew[i]->pDbPage, pgnoTemp, fgB);
+
      sqlite3PagerRekey(apNew[iB]->pDbPage, pgnoA, fgA);
+
      sqlite3PagerRekey(apNew[i]->pDbPage, pgnoB, fgB);
+
      apNew[i]->pgno = pgnoB;
+
      apNew[iB]->pgno = pgnoA;
    }
  }

@@ -74742,7 +76806,6 @@ static int anotherValidCursor(BtCursor *pCur){
*/
static int balance(BtCursor *pCur){
  int rc = SQLITE_OK;
-
  const int nMin = pCur->pBt->usableSize * 2 / 3;
  u8 aBalanceQuickSpace[13];
  u8 *pFree = 0;

@@ -74754,7 +76817,11 @@ static int balance(BtCursor *pCur){
    MemPage *pPage = pCur->pPage;

    if( NEVER(pPage->nFree<0) && btreeComputeFreeSpace(pPage) ) break;
-
    if( pPage->nOverflow==0 && pPage->nFree<=nMin ){
+
    if( pPage->nOverflow==0 && pPage->nFree*3<=(int)pCur->pBt->usableSize*2 ){
+
      /* No rebalance required as long as:
+
      **   (1) There are no overflow cells
+
      **   (2) The amount of free space on the page is less than 2/3rds of
+
      **       the total usable space on the page. */
      break;
    }else if( (iPage = pCur->iPage)==0 ){
      if( pPage->nOverflow && (rc = anotherValidCursor(pCur))==SQLITE_OK ){
@@ -74777,6 +76844,11 @@ static int balance(BtCursor *pCur){
      }else{
        break;
      }
+
    }else if( sqlite3PagerPageRefcount(pPage->pDbPage)>1 ){
+
      /* The page being written is not a root page, and there is currently
+
      ** more than one reference to it. This only happens if the page is one
+
      ** of its own ancestor pages. Corruption. */
+
      rc = SQLITE_CORRUPT_BKPT;
    }else{
      MemPage * const pParent = pCur->apPage[iPage-1];
      int const iIdx = pCur->aiIdx[iPage-1];
@@ -74974,7 +77046,7 @@ static int btreeOverwriteCell(BtCursor *pCur, const BtreePayload *pX){
** pX.pData,nData,nZero fields must be zero.
**
** If the seekResult parameter is non-zero, then a successful call to
-
** MovetoUnpacked() to seek cursor pCur to (pKey,nKey) has already
+
** sqlite3BtreeIndexMoveto() to seek cursor pCur to (pKey,nKey) has already
** been performed.  In other words, if seekResult!=0 then the cursor
** is currently pointing to a cell that will be adjacent to the cell
** to be inserted.  If seekResult<0 then pCur points to a cell that is
@@ -74992,7 +77064,7 @@ SQLITE_PRIVATE int sqlite3BtreeInsert(
  BtCursor *pCur,                /* Insert data into the table of this cursor */
  const BtreePayload *pX,        /* Content of the row to be inserted */
  int flags,                     /* True if this is likely an append */
-
  int seekResult                 /* Result of prior MovetoUnpacked() call */
+
  int seekResult                 /* Result of prior IndexMoveto() call */
){
  int rc;
  int loc = seekResult;          /* -1: before desired location  +1: after */
@@ -75031,7 +77103,12 @@ SQLITE_PRIVATE int sqlite3BtreeInsert(
    }
  }

+
  /* Ensure that the cursor is not in the CURSOR_FAULT state and that it
+
  ** points to a valid cell.
+
  */
  if( pCur->eState>=CURSOR_REQUIRESEEK ){
+
    testcase( pCur->eState==CURSOR_REQUIRESEEK );
+
    testcase( pCur->eState==CURSOR_FAULT );
    rc = moveToRoot(pCur);
    if( rc && rc!=SQLITE_EMPTY ) return rc;
  }
@@ -75143,7 +77220,8 @@ SQLITE_PRIVATE int sqlite3BtreeInsert(
  assert( pPage->intKey || pX->nKey>=0 || (flags & BTREE_PREFORMAT) );
  assert( pPage->leaf || !pPage->intKey );
  if( pPage->nFree<0 ){
-
    if( pCur->eState>CURSOR_INVALID ){
+
    if( NEVER(pCur->eState>CURSOR_INVALID) ){
+
     /* ^^^^^--- due to the moveToRoot() call above */
      rc = SQLITE_CORRUPT_BKPT;
    }else{
      rc = btreeComputeFreeSpace(pPage);
@@ -75154,7 +77232,7 @@ SQLITE_PRIVATE int sqlite3BtreeInsert(
  TRACE(("INSERT: table=%d nkey=%lld ndata=%d page=%d %s\n",
          pCur->pgnoRoot, pX->nKey, pX->nData, pPage->pgno,
          loc==0 ? "overwrite" : "new entry"));
-
  assert( pPage->isInit );
+
  assert( pPage->isInit || CORRUPT_DB );
  newCell = pBt->pTmpSpace;
  assert( newCell!=0 );
  if( flags & BTREE_PREFORMAT ){
@@ -75305,7 +77383,11 @@ SQLITE_PRIVATE int sqlite3BtreeTransferRow(BtCursor *pDest, BtCursor *pSrc, i64
  u32 nRem;                     /* Bytes of data still to copy */

  getCellInfo(pSrc);
-
  aOut += putVarint32(aOut, pSrc->info.nPayload);
+
  if( pSrc->info.nPayload<0x80 ){
+
    *(aOut++) = pSrc->info.nPayload;
+
  }else{
+
    aOut += sqlite3PutVarint(aOut, pSrc->info.nPayload);
+
  }
  if( pDest->pKeyInfo==0 ) aOut += putVarint(aOut, iKey);
  nIn = pSrc->info.nLocal;
  aIn = pSrc->info.pPayload;
@@ -75465,7 +77547,8 @@ SQLITE_PRIVATE int sqlite3BtreeDelete(BtCursor *pCur, u8 flags){
  bPreserve = (flags & BTREE_SAVEPOSITION)!=0;
  if( bPreserve ){
    if( !pPage->leaf
-
     || (pPage->nFree+cellSizePtr(pPage,pCell)+2)>(int)(pBt->usableSize*2/3)
+
     || (pPage->nFree+pPage->xCellSize(pPage,pCell)+2) >
+
                                                   (int)(pBt->usableSize*2/3)
     || pPage->nCell==1  /* See dbfuzz001.test for a test case */
    ){
      /* A b-tree rebalance will be required after deleting this entry.
@@ -75561,7 +77644,15 @@ SQLITE_PRIVATE int sqlite3BtreeDelete(BtCursor *pCur, u8 flags){
  ** been corrected, so be it. Otherwise, after balancing the leaf node,
  ** walk the cursor up the tree to the internal node and balance it as
  ** well.  */
-
  rc = balance(pCur);
+
  assert( pCur->pPage->nOverflow==0 );
+
  assert( pCur->pPage->nFree>=0 );
+
  if( pCur->pPage->nFree*3<=(int)pCur->pBt->usableSize*2 ){
+
    /* Optimization: If the free space is less than 2/3rds of the page,
+
    ** then balance() will always be a no-op.  No need to invoke it. */
+
    rc = SQLITE_OK;
+
  }else{
+
    rc = balance(pCur);
+
  }
  if( rc==SQLITE_OK && pCur->iPage>iCellDepth ){
    releasePageNotNull(pCur->pPage);
    pCur->iPage--;
@@ -78309,9 +80400,10 @@ SQLITE_PRIVATE int sqlite3VdbeMemFinalize(Mem *pMem, FuncDef *pFunc){
  Mem t;
  assert( pFunc!=0 );
  assert( pMem!=0 );
+
  assert( pMem->db!=0 );
  assert( pFunc->xFinalize!=0 );
  assert( (pMem->flags & MEM_Null)!=0 || pFunc==pMem->u.pDef );
-
  assert( pMem->db==0 || sqlite3_mutex_held(pMem->db->mutex) );
+
  assert( sqlite3_mutex_held(pMem->db->mutex) );
  memset(&ctx, 0, sizeof(ctx));
  memset(&t, 0, sizeof(t));
  t.flags = MEM_Null;
@@ -78319,6 +80411,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemFinalize(Mem *pMem, FuncDef *pFunc){
  ctx.pOut = &t;
  ctx.pMem = pMem;
  ctx.pFunc = pFunc;
+
  ctx.enc = ENC(t.db);
  pFunc->xFinalize(&ctx); /* IMP: R-24505-23230 */
  assert( (pMem->flags & MEM_Dyn)==0 );
  if( pMem->szMalloc>0 ) sqlite3DbFreeNN(pMem->db, pMem->zMalloc);
@@ -78340,12 +80433,14 @@ SQLITE_PRIVATE int sqlite3VdbeMemAggValue(Mem *pAccum, Mem *pOut, FuncDef *pFunc
  assert( pFunc!=0 );
  assert( pFunc->xValue!=0 );
  assert( (pAccum->flags & MEM_Null)!=0 || pFunc==pAccum->u.pDef );
-
  assert( pAccum->db==0 || sqlite3_mutex_held(pAccum->db->mutex) );
+
  assert( pAccum->db!=0 );
+
  assert( sqlite3_mutex_held(pAccum->db->mutex) );
  memset(&ctx, 0, sizeof(ctx));
  sqlite3VdbeMemSetNull(pOut);
  ctx.pOut = pOut;
  ctx.pMem = pAccum;
  ctx.pFunc = pFunc;
+
  ctx.enc = ENC(pAccum->db);
  pFunc->xValue(&ctx);
  return ctx.isError;
}
@@ -78411,6 +80506,14 @@ SQLITE_PRIVATE void sqlite3VdbeMemRelease(Mem *p){
  }
}

+
/* Like sqlite3VdbeMemRelease() but faster for cases where we
+
** know in advance that the Mem is not MEM_Dyn or MEM_Agg.
+
*/
+
SQLITE_PRIVATE void sqlite3VdbeMemReleaseMalloc(Mem *p){
+
  assert( !VdbeMemDynamic(p) );
+
  if( p->szMalloc ) vdbeMemClear(p);
+
}
+

/*
** Convert a 64-bit IEEE double into a 64-bit signed integer.
** If the double is out of range of a 64-bit signed integer then
@@ -78589,6 +80692,16 @@ SQLITE_PRIVATE int sqlite3RealSameAsInt(double r1, sqlite3_int64 i){
          && i >= -2251799813685248LL && i < 2251799813685248LL);
}

+
/* Convert a floating point value to its closest integer.  Do so in
+
** a way that avoids 'outside the range of representable values' warnings
+
** from UBSAN.
+
*/
+
SQLITE_PRIVATE i64 sqlite3RealToI64(double r){
+
  if( r<=(double)SMALLEST_INT64 ) return SMALLEST_INT64;
+
  if( r>=(double)LARGEST_INT64) return LARGEST_INT64;
+
  return (i64)r;
+
}
+

/*
** Convert pMem so that it has type MEM_Real or MEM_Int.
** Invalidate any prior representations.
@@ -78610,7 +80723,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemNumerify(Mem *pMem){
    assert( pMem->db==0 || sqlite3_mutex_held(pMem->db->mutex) );
    rc = sqlite3AtoF(pMem->z, &pMem->u.r, pMem->n, pMem->enc);
    if( ((rc==0 || rc==1) && sqlite3Atoi64(pMem->z, &ix, pMem->n, pMem->enc)<=1)
-
     || sqlite3RealSameAsInt(pMem->u.r, (ix = (i64)pMem->u.r))
+
     || sqlite3RealSameAsInt(pMem->u.r, (ix = sqlite3RealToI64(pMem->u.r)))
    ){
      pMem->u.i = ix;
      MemSetTypeFlag(pMem, MEM_Int);
@@ -78662,6 +80775,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemCast(Mem *pMem, u8 aff, u8 encoding){
      sqlite3ValueApplyAffinity(pMem, SQLITE_AFF_TEXT, encoding);
      assert( pMem->flags & MEM_Str || pMem->db->mallocFailed );
      pMem->flags &= ~(MEM_Int|MEM_Real|MEM_IntReal|MEM_Blob|MEM_Zero);
+
      if( encoding!=SQLITE_UTF8 ) pMem->n &= ~1;
      return sqlite3VdbeChangeEncoding(pMem, encoding);
    }
  }
@@ -78955,6 +81069,13 @@ SQLITE_PRIVATE void sqlite3VdbeMemMove(Mem *pTo, Mem *pFrom){
** stored without allocating memory, then it is.  If a memory allocation
** is required to store the string, then value of pMem is unchanged.  In
** either case, SQLITE_TOOBIG is returned.
+
**
+
** The "enc" parameter is the text encoding for the string, or zero
+
** to store a blob.
+
**
+
** If n is negative, then the string consists of all bytes up to but
+
** excluding the first zero character.  The n parameter must be
+
** non-negative for blobs.
*/
SQLITE_PRIVATE int sqlite3VdbeMemSetStr(
  Mem *pMem,          /* Memory cell to set to string value */
@@ -78965,11 +81086,12 @@ SQLITE_PRIVATE int sqlite3VdbeMemSetStr(
){
  i64 nByte = n;      /* New value for pMem->n */
  int iLimit;         /* Maximum allowed string or blob size */
-
  u16 flags = 0;      /* New value for pMem->flags */
+
  u16 flags;          /* New value for pMem->flags */

  assert( pMem!=0 );
  assert( pMem->db==0 || sqlite3_mutex_held(pMem->db->mutex) );
  assert( !sqlite3VdbeMemIsRowSet(pMem) );
+
  assert( enc!=0 || n>=0 );

  /* If z is a NULL pointer, set pMem to contain an SQL NULL. */
  if( !z ){
@@ -78982,7 +81104,6 @@ SQLITE_PRIVATE int sqlite3VdbeMemSetStr(
  }else{
    iLimit = SQLITE_MAX_LENGTH;
  }
-
  flags = (enc==0?MEM_Blob:MEM_Str);
  if( nByte<0 ){
    assert( enc!=0 );
    if( enc==SQLITE_UTF8 ){
@@ -78990,7 +81111,23 @@ SQLITE_PRIVATE int sqlite3VdbeMemSetStr(
    }else{
      for(nByte=0; nByte<=iLimit && (z[nByte] | z[nByte+1]); nByte+=2){}
    }
-
    flags |= MEM_Term;
+
    flags= MEM_Str|MEM_Term;
+
  }else if( enc==0 ){
+
    flags = MEM_Blob;
+
    enc = SQLITE_UTF8;
+
  }else{
+
    flags = MEM_Str;
+
  }
+
  if( nByte>iLimit ){
+
    if( xDel && xDel!=SQLITE_TRANSIENT ){
+
      if( xDel==SQLITE_DYNAMIC ){
+
        sqlite3DbFree(pMem->db, (void*)z);
+
      }else{
+
        xDel((void*)z);
+
      }
+
    }
+
    sqlite3VdbeMemSetNull(pMem);
+
    return sqlite3ErrorToParser(pMem->db, SQLITE_TOOBIG);
  }

  /* The following block sets the new values of Mem.z and Mem.xDel. It
@@ -79002,9 +81139,6 @@ SQLITE_PRIVATE int sqlite3VdbeMemSetStr(
    if( flags&MEM_Term ){
      nAlloc += (enc==SQLITE_UTF8?1:2);
    }
-
    if( nByte>iLimit ){
-
      return sqlite3ErrorToParser(pMem->db, SQLITE_TOOBIG);
-
    }
    testcase( nAlloc==0 );
    testcase( nAlloc==31 );
    testcase( nAlloc==32 );
@@ -79026,16 +81160,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemSetStr(

  pMem->n = (int)(nByte & 0x7fffffff);
  pMem->flags = flags;
-
  if( enc ){
-
    pMem->enc = enc;
-
#ifdef SQLITE_ENABLE_SESSION
-
  }else if( pMem->db==0 ){
-
    pMem->enc = SQLITE_UTF8;
-
#endif
-
  }else{
-
    assert( pMem->db!=0 );
-
    pMem->enc = ENC(pMem->db);
-
  }
+
  pMem->enc = enc;

#ifndef SQLITE_OMIT_UTF16
  if( enc>SQLITE_UTF8 && sqlite3VdbeMemHandleBom(pMem) ){
@@ -79043,9 +81168,6 @@ SQLITE_PRIVATE int sqlite3VdbeMemSetStr(
  }
#endif

-
  if( nByte>iLimit ){
-
    return sqlite3ErrorToParser(pMem->db, SQLITE_TOOBIG);
-
  }

  return SQLITE_OK;
}
@@ -79323,10 +81445,12 @@ static int valueFromFunction(
    goto value_from_function_out;
  }

-
  assert( pCtx->pParse->rc==SQLITE_OK );
+
  testcase( pCtx->pParse->rc==SQLITE_ERROR );
+
  testcase( pCtx->pParse->rc==SQLITE_OK );
  memset(&ctx, 0, sizeof(ctx));
  ctx.pOut = pVal;
  ctx.pFunc = pFunc;
+
  ctx.enc = ENC(db);
  pFunc->xSFunc(&ctx, nVal, apVal);
  if( ctx.isError ){
    rc = ctx.isError;
@@ -79402,8 +81526,8 @@ static int valueFromExpr(
    rc = valueFromExpr(db, pExpr->pLeft, enc, aff, ppVal, pCtx);
    testcase( rc!=SQLITE_OK );
    if( *ppVal ){
-
      sqlite3VdbeMemCast(*ppVal, aff, SQLITE_UTF8);
-
      sqlite3ValueApplyAffinity(*ppVal, affinity, SQLITE_UTF8);
+
      sqlite3VdbeMemCast(*ppVal, aff, enc);
+
      sqlite3ValueApplyAffinity(*ppVal, affinity, enc);
    }
    return rc;
  }
@@ -79787,6 +81911,9 @@ SQLITE_PRIVATE int sqlite3ValueBytes(sqlite3_value *pVal, u8 enc){
  if( (p->flags & MEM_Str)!=0 && pVal->enc==enc ){
    return p->n;
  }
+
  if( (p->flags & MEM_Str)!=0 && enc!=SQLITE_UTF8 && pVal->enc!=SQLITE_UTF8 ){
+
    return p->n;
+
  }
  if( (p->flags & MEM_Blob)!=0 ){
    if( p->flags & MEM_Zero ){
      return p->n + p->u.nZero;
@@ -79832,12 +81959,12 @@ SQLITE_PRIVATE Vdbe *sqlite3VdbeCreate(Parse *pParse){
  memset(&p->aOp, 0, sizeof(Vdbe)-offsetof(Vdbe,aOp));
  p->db = db;
  if( db->pVdbe ){
-
    db->pVdbe->pPrev = p;
+
    db->pVdbe->ppVPrev = &p->pVNext;
  }
-
  p->pNext = db->pVdbe;
-
  p->pPrev = 0;
+
  p->pVNext = db->pVdbe;
+
  p->ppVPrev = &db->pVdbe;
  db->pVdbe = p;
-
  p->iVdbeMagic = VDBE_MAGIC_INIT;
+
  assert( p->eVdbeState==VDBE_INIT_STATE );
  p->pParse = pParse;
  pParse->pVdbe = p;
  assert( pParse->aLabel==0 );
@@ -79917,21 +82044,28 @@ SQLITE_PRIVATE int sqlite3VdbeUsesDoubleQuotedString(
#endif

/*
-
** Swap all content between two VDBE structures.
+
** Swap byte-code between two VDBE structures.
+
**
+
** This happens after pB was previously run and returned
+
** SQLITE_SCHEMA.  The statement was then reprepared in pA.
+
** This routine transfers the new bytecode in pA over to pB
+
** so that pB can be run again.  The old pB byte code is
+
** moved back to pA so that it will be cleaned up when pA is
+
** finalized.
*/
SQLITE_PRIVATE void sqlite3VdbeSwap(Vdbe *pA, Vdbe *pB){
-
  Vdbe tmp, *pTmp;
+
  Vdbe tmp, *pTmp, **ppTmp;
  char *zTmp;
  assert( pA->db==pB->db );
  tmp = *pA;
  *pA = *pB;
  *pB = tmp;
-
  pTmp = pA->pNext;
-
  pA->pNext = pB->pNext;
-
  pB->pNext = pTmp;
-
  pTmp = pA->pPrev;
-
  pA->pPrev = pB->pPrev;
-
  pB->pPrev = pTmp;
+
  pTmp = pA->pVNext;
+
  pA->pVNext = pB->pVNext;
+
  pB->pVNext = pTmp;
+
  ppTmp = pA->ppVPrev;
+
  pA->ppVPrev = pB->ppVPrev;
+
  pB->ppVPrev = ppTmp;
  zTmp = pA->zSql;
  pA->zSql = pB->zSql;
  pB->zSql = zTmp;
@@ -79982,7 +82116,7 @@ static int growOpArray(Vdbe *v, int nOp){
    return SQLITE_NOMEM;
  }

-
  assert( nOp<=(1024/sizeof(Op)) );
+
  assert( nOp<=(int)(1024/sizeof(Op)) );
  assert( nNew>=(v->nOpAlloc+nOp) );
  pNew = sqlite3DbRealloc(p->db, v->aOp, nNew*sizeof(Op));
  if( pNew ){
@@ -80038,7 +82172,7 @@ SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe *p, int op, int p1, int p2, int p3){
  VdbeOp *pOp;

  i = p->nOp;
-
  assert( p->iVdbeMagic==VDBE_MAGIC_INIT );
+
  assert( p->eVdbeState==VDBE_INIT_STATE );
  assert( op>=0 && op<0xff );
  if( p->nOpAlloc<=i ){
    return growOp3(p, op, p1, p2, p3);
@@ -80183,6 +82317,7 @@ SQLITE_PRIVATE int sqlite3VdbeAddFunctionCall(
  addr = sqlite3VdbeAddOp4(v, eCallCtx ? OP_PureFunc : OP_Function,
                           p1, p2, p3, (char*)pCtx, P4_FUNCCTX);
  sqlite3VdbeChangeP5(v, eCallCtx & NC_SelfRef);
+
  sqlite3MayAbort(pParse);
  return addr;
}

@@ -80251,7 +82386,7 @@ SQLITE_PRIVATE void sqlite3VdbeExplain(Parse *pParse, u8 bPush, const char *zFmt
    iThis = v->nOp;
    sqlite3VdbeAddOp4(v, OP_Explain, iThis, pParse->addrExplain, 0,
                      zMsg, P4_DYNAMIC);
-
    sqlite3ExplainBreakpoint(bPush?"PUSH":"", sqlite3VdbeGetOp(v,-1)->p4.z);
+
    sqlite3ExplainBreakpoint(bPush?"PUSH":"", sqlite3VdbeGetLastOp(v)->p4.z);
    if( bPush){
      pParse->addrExplain = iThis;
    }
@@ -80370,7 +82505,7 @@ static SQLITE_NOINLINE void resizeResolveLabel(Parse *p, Vdbe *v, int j){
SQLITE_PRIVATE void sqlite3VdbeResolveLabel(Vdbe *v, int x){
  Parse *p = v->pParse;
  int j = ADDR(x);
-
  assert( v->iVdbeMagic==VDBE_MAGIC_INIT );
+
  assert( v->eVdbeState==VDBE_INIT_STATE );
  assert( j<-p->nLabel );
  assert( j>=0 );
#ifdef SQLITE_DEBUG
@@ -80390,14 +82525,20 @@ SQLITE_PRIVATE void sqlite3VdbeResolveLabel(Vdbe *v, int x){
** Mark the VDBE as one that can only be run one time.
*/
SQLITE_PRIVATE void sqlite3VdbeRunOnlyOnce(Vdbe *p){
-
  p->runOnlyOnce = 1;
+
  sqlite3VdbeAddOp2(p, OP_Expire, 1, 1);
}

/*
-
** Mark the VDBE as one that can only be run multiple times.
+
** Mark the VDBE as one that can be run multiple times.
*/
SQLITE_PRIVATE void sqlite3VdbeReusable(Vdbe *p){
-
  p->runOnlyOnce = 0;
+
  int i;
+
  for(i=1; ALWAYS(i<p->nOp); i++){
+
    if( ALWAYS(p->aOp[i].opcode==OP_Expire) ){
+
      p->aOp[1].opcode = OP_Noop;
+
      break;
+
    }
+
  }
}

#ifdef SQLITE_DEBUG /* sqlite3AssertMayAbort() logic */
@@ -80501,6 +82642,8 @@ SQLITE_PRIVATE int sqlite3VdbeAssertMayAbort(Vdbe *v, int mayAbort){
  int hasInitCoroutine = 0;
  Op *pOp;
  VdbeOpIter sIter;
+

+
  if( v==0 ) return 0;
  memset(&sIter, 0, sizeof(sIter));
  sIter.v = v;

@@ -80510,6 +82653,7 @@ SQLITE_PRIVATE int sqlite3VdbeAssertMayAbort(Vdbe *v, int mayAbort){
     || opcode==OP_VDestroy
     || opcode==OP_VCreate
     || opcode==OP_ParseSchema
+
     || opcode==OP_Function || opcode==OP_PureFunc
     || ((opcode==OP_Halt || opcode==OP_HaltIfNull)
      && ((pOp->p1)!=SQLITE_OK && pOp->p2==OE_Abort))
    ){
@@ -80584,7 +82728,7 @@ SQLITE_PRIVATE void sqlite3VdbeAssertAbortable(Vdbe *p){
** (3) Update the Vdbe.readOnly and Vdbe.bIsReader flags to accurately
**     indicate what the prepared statement actually does.
**
-
** (4) Initialize the p4.xAdvance pointer on opcodes that use it.
+
** (4) (discontinued)
**
** (5) Reclaim the memory allocated for storing labels.
**
@@ -80600,8 +82744,8 @@ static void resolveP2Values(Vdbe *p, int *pMaxFuncArgs){
  p->readOnly = 1;
  p->bIsReader = 0;
  pOp = &p->aOp[p->nOp-1];
-
  while(1){
-

+
  assert( p->aOp[0].opcode==OP_Init );
+
  while( 1 /* Loop termates when it reaches the OP_Init opcode */ ){
    /* Only JUMP opcodes and the short list of special opcodes in the switch
    ** below need to be considered.  The mkopcodeh.tcl generator script groups
    ** all these opcodes together near the front of the opcode list.  Skip
@@ -80630,24 +82774,9 @@ static void resolveP2Values(Vdbe *p, int *pMaxFuncArgs){
          p->bIsReader = 1;
          break;
        }
-
        case OP_Next:
-
        case OP_SorterNext: {
-
          pOp->p4.xAdvance = sqlite3BtreeNext;
-
          pOp->p4type = P4_ADVANCE;
-
          /* The code generator never codes any of these opcodes as a jump
-
          ** to a label.  They are always coded as a jump backwards to a
-
          ** known address */
+
        case OP_Init: {
          assert( pOp->p2>=0 );
-
          break;
-
        }
-
        case OP_Prev: {
-
          pOp->p4.xAdvance = sqlite3BtreePrevious;
-
          pOp->p4type = P4_ADVANCE;
-
          /* The code generator never codes any of these opcodes as a jump
-
          ** to a label.  They are always coded as a jump backwards to a
-
          ** known address */
-
          assert( pOp->p2>=0 );
-
          break;
+
          goto resolve_p2_values_loop_exit;
        }
#ifndef SQLITE_OMIT_VIRTUALTABLE
        case OP_VUpdate: {
@@ -80681,21 +82810,108 @@ static void resolveP2Values(Vdbe *p, int *pMaxFuncArgs){
      ** have non-negative values for P2. */
      assert( (sqlite3OpcodeProperty[pOp->opcode]&OPFLG_JUMP)==0 || pOp->p2>=0);
    }
-
    if( pOp==p->aOp ) break;
+
    assert( pOp>p->aOp );
    pOp--;
  }
-
  sqlite3DbFree(p->db, pParse->aLabel);
-
  pParse->aLabel = 0;
+
resolve_p2_values_loop_exit:
+
  if( aLabel ){
+
    sqlite3DbNNFreeNN(p->db, pParse->aLabel);
+
    pParse->aLabel = 0;
+
  }
  pParse->nLabel = 0;
  *pMaxFuncArgs = nMaxArgs;
  assert( p->bIsReader!=0 || DbMaskAllZero(p->btreeMask) );
}

+
#ifdef SQLITE_DEBUG
+
/*
+
** Check to see if a subroutine contains a jump to a location outside of
+
** the subroutine.  If a jump outside the subroutine is detected, add code
+
** that will cause the program to halt with an error message.
+
**
+
** The subroutine consists of opcodes between iFirst and iLast.  Jumps to
+
** locations within the subroutine are acceptable.  iRetReg is a register
+
** that contains the return address.  Jumps to outside the range of iFirst
+
** through iLast are also acceptable as long as the jump destination is
+
** an OP_Return to iReturnAddr.
+
**
+
** A jump to an unresolved label means that the jump destination will be
+
** beyond the current address.  That is normally a jump to an early
+
** termination and is consider acceptable.
+
**
+
** This routine only runs during debug builds.  The purpose is (of course)
+
** to detect invalid escapes out of a subroutine.  The OP_Halt opcode
+
** is generated rather than an assert() or other error, so that ".eqp full"
+
** will still work to show the original bytecode, to aid in debugging.
+
*/
+
SQLITE_PRIVATE void sqlite3VdbeNoJumpsOutsideSubrtn(
+
  Vdbe *v,          /* The byte-code program under construction */
+
  int iFirst,       /* First opcode of the subroutine */
+
  int iLast,        /* Last opcode of the subroutine */
+
  int iRetReg       /* Subroutine return address register */
+
){
+
  VdbeOp *pOp;
+
  Parse *pParse;
+
  int i;
+
  sqlite3_str *pErr = 0;
+
  assert( v!=0 );
+
  pParse = v->pParse;
+
  assert( pParse!=0 );
+
  if( pParse->nErr ) return;
+
  assert( iLast>=iFirst );
+
  assert( iLast<v->nOp );
+
  pOp = &v->aOp[iFirst];
+
  for(i=iFirst; i<=iLast; i++, pOp++){
+
    if( (sqlite3OpcodeProperty[pOp->opcode] & OPFLG_JUMP)!=0 ){
+
      int iDest = pOp->p2;   /* Jump destination */
+
      if( iDest==0 ) continue;
+
      if( pOp->opcode==OP_Gosub ) continue;
+
      if( iDest<0 ){
+
        int j = ADDR(iDest);
+
        assert( j>=0 );
+
        if( j>=-pParse->nLabel || pParse->aLabel[j]<0 ){
+
          continue;
+
        }
+
        iDest = pParse->aLabel[j];
+
      }
+
      if( iDest<iFirst || iDest>iLast ){
+
        int j = iDest;
+
        for(; j<v->nOp; j++){
+
          VdbeOp *pX = &v->aOp[j];
+
          if( pX->opcode==OP_Return ){
+
            if( pX->p1==iRetReg ) break;
+
            continue;
+
          }
+
          if( pX->opcode==OP_Noop ) continue;
+
          if( pX->opcode==OP_Explain ) continue;
+
          if( pErr==0 ){
+
            pErr = sqlite3_str_new(0);
+
          }else{
+
            sqlite3_str_appendchar(pErr, 1, '\n');
+
          }
+
          sqlite3_str_appendf(pErr,
+
              "Opcode at %d jumps to %d which is outside the "
+
              "subroutine at %d..%d",
+
              i, iDest, iFirst, iLast);
+
          break;
+
        }
+
      }
+
    }
+
  }
+
  if( pErr ){
+
    char *zErr = sqlite3_str_finish(pErr);
+
    sqlite3VdbeAddOp4(v, OP_Halt, SQLITE_INTERNAL, OE_Abort, 0, zErr, 0);
+
    sqlite3_free(zErr);
+
    sqlite3MayAbort(pParse);
+
  }
+
}
+
#endif /* SQLITE_DEBUG */
+

/*
** Return the address of the next instruction to be inserted.
*/
SQLITE_PRIVATE int sqlite3VdbeCurrentAddr(Vdbe *p){
-
  assert( p->iVdbeMagic==VDBE_MAGIC_INIT );
+
  assert( p->eVdbeState==VDBE_INIT_STATE );
  return p->nOp;
}

@@ -80780,7 +82996,7 @@ SQLITE_PRIVATE VdbeOp *sqlite3VdbeAddOpList(
  int i;
  VdbeOp *pOut, *pFirst;
  assert( nOp>0 );
-
  assert( p->iVdbeMagic==VDBE_MAGIC_INIT );
+
  assert( p->eVdbeState==VDBE_INIT_STATE );
  if( p->nOp + nOp > p->nOpAlloc && growOpArray(p, nOp) ){
    return 0;
  }
@@ -80848,15 +83064,19 @@ SQLITE_PRIVATE void sqlite3VdbeScanStatus(
** for a specific instruction.
*/
SQLITE_PRIVATE void sqlite3VdbeChangeOpcode(Vdbe *p, int addr, u8 iNewOpcode){
+
  assert( addr>=0 );
  sqlite3VdbeGetOp(p,addr)->opcode = iNewOpcode;
}
SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe *p, int addr, int val){
+
  assert( addr>=0 );
  sqlite3VdbeGetOp(p,addr)->p1 = val;
}
SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe *p, int addr, int val){
+
  assert( addr>=0 || p->db->mallocFailed );
  sqlite3VdbeGetOp(p,addr)->p2 = val;
}
SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe *p, int addr, int val){
+
  assert( addr>=0 );
  sqlite3VdbeGetOp(p,addr)->p3 = val;
}
SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe *p, u16 p5){
@@ -80865,6 +83085,18 @@ SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe *p, u16 p5){
}

/*
+
** If the previous opcode is an OP_Column that delivers results
+
** into register iDest, then add the OPFLAG_TYPEOFARG flag to that
+
** opcode.
+
*/
+
SQLITE_PRIVATE void sqlite3VdbeTypeofColumn(Vdbe *p, int iDest){
+
  VdbeOp *pOp = sqlite3VdbeGetLastOp(p);
+
  if( pOp->p3==iDest && pOp->opcode==OP_Column ){
+
    pOp->p5 |= OPFLAG_TYPEOFARG;
+
  }
+
}
+

+
/*
** Change the P2 operand of instruction addr so that it points to
** the address of the next instruction to be coded.
*/
@@ -80892,7 +83124,7 @@ SQLITE_PRIVATE void sqlite3VdbeJumpHereOrPopInst(Vdbe *p, int addr){
         || p->aOp[addr].opcode==OP_FkIfZero );
    assert( p->aOp[addr].p4type==0 );
#ifdef SQLITE_VDBE_COVERAGE
-
    sqlite3VdbeGetOp(p,-1)->iSrcLine = 0;  /* Erase VdbeCoverage() macros */
+
    sqlite3VdbeGetLastOp(p)->iSrcLine = 0;  /* Erase VdbeCoverage() macros */
#endif
    p->nOp--;
  }else{
@@ -80906,8 +83138,9 @@ SQLITE_PRIVATE void sqlite3VdbeJumpHereOrPopInst(Vdbe *p, int addr){
** the FuncDef is not ephermal, then do nothing.
*/
static void freeEphemeralFunction(sqlite3 *db, FuncDef *pDef){
+
  assert( db!=0 );
  if( (pDef->funcFlags & SQLITE_FUNC_EPHEM)!=0 ){
-
    sqlite3DbFreeNN(db, pDef);
+
    sqlite3DbNNFreeNN(db, pDef);
  }
}

@@ -80916,11 +83149,12 @@ static void freeEphemeralFunction(sqlite3 *db, FuncDef *pDef){
*/
static SQLITE_NOINLINE void freeP4Mem(sqlite3 *db, Mem *p){
  if( p->szMalloc ) sqlite3DbFree(db, p->zMalloc);
-
  sqlite3DbFreeNN(db, p);
+
  sqlite3DbNNFreeNN(db, p);
}
static SQLITE_NOINLINE void freeP4FuncCtx(sqlite3 *db, sqlite3_context *p){
+
  assert( db!=0 );
  freeEphemeralFunction(db, p->pFunc);
-
  sqlite3DbFreeNN(db, p);
+
  sqlite3DbNNFreeNN(db, p);
}
static void freeP4(sqlite3 *db, int p4type, void *p4){
  assert( db );
@@ -80932,9 +83166,8 @@ static void freeP4(sqlite3 *db, int p4type, void *p4){
    case P4_REAL:
    case P4_INT64:
    case P4_DYNAMIC:
-
    case P4_DYNBLOB:
    case P4_INTARRAY: {
-
      sqlite3DbFree(db, p4);
+
      if( p4 ) sqlite3DbNNFreeNN(db, p4);
      break;
    }
    case P4_KEYINFO: {
@@ -80972,15 +83205,19 @@ static void freeP4(sqlite3 *db, int p4type, void *p4){
** nOp entries.
*/
static void vdbeFreeOpArray(sqlite3 *db, Op *aOp, int nOp){
+
  assert( nOp>=0 );
+
  assert( db!=0 );
  if( aOp ){
-
    Op *pOp;
-
    for(pOp=&aOp[nOp-1]; pOp>=aOp; pOp--){
+
    Op *pOp = &aOp[nOp-1];
+
    while(1){  /* Exit via break */
      if( pOp->p4type <= P4_FREE_IF_LE ) freeP4(db, pOp->p4type, pOp->p4.p);
#ifdef SQLITE_ENABLE_EXPLAIN_COMMENTS
      sqlite3DbFree(db, pOp->zComment);
#endif
+
      if( pOp==aOp ) break;
+
      pOp--;
    }
-
    sqlite3DbFreeNN(db, aOp);
+
    sqlite3DbNNFreeNN(db, aOp);
  }
}

@@ -81040,7 +83277,7 @@ SQLITE_PRIVATE void sqlite3VdbeReleaseRegisters(
  u32 mask,            /* Mask of registers to NOT release */
  int bUndefine        /* If true, mark registers as undefined */
){
-
  if( N==0 ) return;
+
  if( N==0 || OptimizationDisabled(pParse->db, SQLITE_ReleaseReg) ) return;
  assert( pParse->pVdbe );
  assert( iFirst>=1 );
  assert( iFirst+N-1<=pParse->nMem );
@@ -81104,7 +83341,7 @@ SQLITE_PRIVATE void sqlite3VdbeChangeP4(Vdbe *p, int addr, const char *zP4, int
  sqlite3 *db;
  assert( p!=0 );
  db = p->db;
-
  assert( p->iVdbeMagic==VDBE_MAGIC_INIT );
+
  assert( p->eVdbeState==VDBE_INIT_STATE );
  assert( p->aOp!=0 || db->mallocFailed );
  if( db->mallocFailed ){
    if( n!=P4_VTAB ) freeP4(db, n, (void*)*(char**)&zP4);
@@ -81211,13 +83448,13 @@ SQLITE_PRIVATE void sqlite3VdbeNoopComment(Vdbe *p, const char *zFormat, ...){
** Set the value if the iSrcLine field for the previously coded instruction.
*/
SQLITE_PRIVATE void sqlite3VdbeSetLineNumber(Vdbe *v, int iLine){
-
  sqlite3VdbeGetOp(v,-1)->iSrcLine = iLine;
+
  sqlite3VdbeGetLastOp(v)->iSrcLine = iLine;
}
#endif /* SQLITE_VDBE_COVERAGE */

/*
-
** Return the opcode for a given address.  If the address is -1, then
-
** return the most recently inserted opcode.
+
** Return the opcode for a given address.  The address must be non-negative.
+
** See sqlite3VdbeGetLastOp() to get the most recently added opcode.
**
** If a memory allocation error has occurred prior to the calling of this
** routine, then a pointer to a dummy VdbeOp will be returned.  That opcode
@@ -81232,10 +83469,7 @@ SQLITE_PRIVATE VdbeOp *sqlite3VdbeGetOp(Vdbe *p, int addr){
  /* C89 specifies that the constant "dummy" will be initialized to all
  ** zeros, which is correct.  MSVC generates a warning, nevertheless. */
  static VdbeOp dummy;  /* Ignore the MSVC warning about no initializer */
-
  assert( p->iVdbeMagic==VDBE_MAGIC_INIT );
-
  if( addr<0 ){
-
    addr = p->nOp - 1;
-
  }
+
  assert( p->eVdbeState==VDBE_INIT_STATE );
  assert( (addr>=0 && addr<p->nOp) || p->db->mallocFailed );
  if( p->db->mallocFailed ){
    return (VdbeOp*)&dummy;
@@ -81244,6 +83478,12 @@ SQLITE_PRIVATE VdbeOp *sqlite3VdbeGetOp(Vdbe *p, int addr){
  }
}

+
/* Return the most recently added opcode
+
*/
+
VdbeOp * sqlite3VdbeGetLastOp(Vdbe *p){
+
  return sqlite3VdbeGetOp(p, p->nOp - 1);
+
}
+

#if defined(SQLITE_ENABLE_EXPLAIN_COMMENTS)
/*
** Return an integer value for one of the parameters to the opcode pOp
@@ -81299,8 +83539,11 @@ SQLITE_PRIVATE char *sqlite3VdbeDisplayComment(
        if( c=='4' ){
          sqlite3_str_appendall(&x, zP4);
        }else if( c=='X' ){
-
          sqlite3_str_appendall(&x, pOp->zComment);
-
          seenCom = 1;
+
          if( pOp->zComment && pOp->zComment[0] ){
+
            sqlite3_str_appendall(&x, pOp->zComment);
+
            seenCom = 1;
+
            break;
+
          }
        }else{
          int v1 = translateP(c, pOp);
          int v2;
@@ -81529,10 +83772,6 @@ SQLITE_PRIVATE char *sqlite3VdbeDisplayP4(sqlite3 *db, Op *pOp){
      zP4 = "program";
      break;
    }
-
    case P4_DYNBLOB:
-
    case P4_ADVANCE: {
-
      break;
-
    }
    case P4_TABLE: {
      zP4 = pOp->p4.pTab->zName;
      break;
@@ -81664,21 +83903,40 @@ SQLITE_PRIVATE void sqlite3VdbePrintOp(FILE *pOut, int pc, VdbeOp *pOp){

/*
** Initialize an array of N Mem element.
+
**
+
** This is a high-runner, so only those fields that really do need to
+
** be initialized are set.  The Mem structure is organized so that
+
** the fields that get initialized are nearby and hopefully on the same
+
** cache line.
+
**
+
**    Mem.flags = flags
+
**    Mem.db = db
+
**    Mem.szMalloc = 0
+
**
+
** All other fields of Mem can safely remain uninitialized for now.  They
+
** will be initialized before use.
*/
static void initMemArray(Mem *p, int N, sqlite3 *db, u16 flags){
-
  while( (N--)>0 ){
-
    p->db = db;
-
    p->flags = flags;
-
    p->szMalloc = 0;
+
  if( N>0 ){
+
    do{
+
      p->flags = flags;
+
      p->db = db;
+
      p->szMalloc = 0;
#ifdef SQLITE_DEBUG
-
    p->pScopyFrom = 0;
+
      p->pScopyFrom = 0;
#endif
-
    p++;
+
      p++;
+
    }while( (--N)>0 );
  }
}

/*
-
** Release an array of N Mem elements
+
** Release auxiliary memory held in an array of N Mem elements.
+
**
+
** After this routine returns, all Mem elements in the array will still
+
** be valid.  Those Mem elements that were not holding auxiliary resources
+
** will be unchanged.  Mem elements which had something freed will be
+
** set to MEM_Undefined.
*/
static void releaseMemArray(Mem *p, int N){
  if( p && N ){
@@ -81711,12 +83969,17 @@ static void releaseMemArray(Mem *p, int N){
      if( p->flags&(MEM_Agg|MEM_Dyn) ){
        testcase( (p->flags & MEM_Dyn)!=0 && p->xDel==sqlite3VdbeFrameMemDel );
        sqlite3VdbeMemRelease(p);
+
        p->flags = MEM_Undefined;
      }else if( p->szMalloc ){
-
        sqlite3DbFreeNN(db, p->zMalloc);
+
        sqlite3DbNNFreeNN(db, p->zMalloc);
        p->szMalloc = 0;
+
        p->flags = MEM_Undefined;
      }
-

-
      p->flags = MEM_Undefined;
+
#ifdef SQLITE_DEBUG
+
      else{
+
        p->flags = MEM_Undefined;
+
      }
+
#endif
    }while( (++p)<pEnd );
  }
}
@@ -81875,7 +84138,7 @@ SQLITE_PRIVATE void sqlite3VdbeFrameDelete(VdbeFrame *p){
  VdbeCursor **apCsr = (VdbeCursor **)&aMem[p->nChildMem];
  assert( sqlite3VdbeFrameIsValid(p) );
  for(i=0; i<p->nChildCsr; i++){
-
    sqlite3VdbeFreeCursor(p->v, apCsr[i]);
+
    if( apCsr[i] ) sqlite3VdbeFreeCursorNN(p->v, apCsr[i]);
  }
  releaseMemArray(aMem, p->nChildMem);
  sqlite3VdbeDeleteAuxData(p->v->db, &p->pAuxData, -1, 0);
@@ -81914,7 +84177,7 @@ SQLITE_PRIVATE int sqlite3VdbeList(
  Op *pOp;                             /* Current opcode */

  assert( p->explain );
-
  assert( p->iVdbeMagic==VDBE_MAGIC_RUN );
+
  assert( p->eVdbeState==VDBE_RUN_STATE );
  assert( p->rc==SQLITE_OK || p->rc==SQLITE_BUSY || p->rc==SQLITE_NOMEM );

  /* Even though this opcode does not use dynamic strings for
@@ -82069,11 +84332,11 @@ struct ReusableSpace {
static void *allocSpace(
  struct ReusableSpace *p,  /* Bulk memory available for allocation */
  void *pBuf,               /* Pointer to a prior allocation */
-
  sqlite3_int64 nByte       /* Bytes of memory needed */
+
  sqlite3_int64 nByte       /* Bytes of memory needed. */
){
  assert( EIGHT_BYTE_ALIGNMENT(p->pSpace) );
  if( pBuf==0 ){
-
    nByte = ROUND8(nByte);
+
    nByte = ROUND8P(nByte);
    if( nByte <= p->nFree ){
      p->nFree -= nByte;
      pBuf = &p->pSpace[p->nFree];
@@ -82094,14 +84357,15 @@ SQLITE_PRIVATE void sqlite3VdbeRewind(Vdbe *p){
  int i;
#endif
  assert( p!=0 );
-
  assert( p->iVdbeMagic==VDBE_MAGIC_INIT || p->iVdbeMagic==VDBE_MAGIC_RESET );
+
  assert( p->eVdbeState==VDBE_INIT_STATE
+
       || p->eVdbeState==VDBE_READY_STATE
+
       || p->eVdbeState==VDBE_HALT_STATE );

  /* There should be at least one opcode.
  */
  assert( p->nOp>0 );

-
  /* Set the magic to VDBE_MAGIC_RUN sooner rather than later. */
-
  p->iVdbeMagic = VDBE_MAGIC_RUN;
+
  p->eVdbeState = VDBE_READY_STATE;

#ifdef SQLITE_DEBUG
  for(i=0; i<p->nMem; i++){
@@ -82157,7 +84421,7 @@ SQLITE_PRIVATE void sqlite3VdbeMakeReady(
  assert( p!=0 );
  assert( p->nOp>0 );
  assert( pParse!=0 );
-
  assert( p->iVdbeMagic==VDBE_MAGIC_INIT );
+
  assert( p->eVdbeState==VDBE_INIT_STATE );
  assert( pParse==p->pParse );
  p->pVList = pParse->pVList;
  pParse->pVList =  0;
@@ -82180,7 +84444,7 @@ SQLITE_PRIVATE void sqlite3VdbeMakeReady(
  ** opcode array.  This extra memory will be reallocated for other elements
  ** of the prepared statement.
  */
-
  n = ROUND8(sizeof(Op)*p->nOp);              /* Bytes of opcode memory used */
+
  n = ROUND8P(sizeof(Op)*p->nOp);             /* Bytes of opcode memory used */
  x.pSpace = &((u8*)p->aOp)[n];               /* Unused opcode memory */
  assert( EIGHT_BYTE_ALIGNMENT(x.pSpace) );
  x.nFree = ROUNDDOWN8(pParse->szOpAlloc - n);  /* Bytes of unused memory */
@@ -82268,9 +84532,9 @@ SQLITE_PRIVATE void sqlite3VdbeMakeReady(
** happens to hold.
*/
SQLITE_PRIVATE void sqlite3VdbeFreeCursor(Vdbe *p, VdbeCursor *pCx){
-
  if( pCx==0 ){
-
    return;
-
  }
+
  if( pCx ) sqlite3VdbeFreeCursorNN(p,pCx);
+
}
+
SQLITE_PRIVATE void sqlite3VdbeFreeCursorNN(Vdbe *p, VdbeCursor *pCx){
  switch( pCx->eCurType ){
    case CURTYPE_SORTER: {
      sqlite3VdbeSorterClose(p->db, pCx);
@@ -82298,14 +84562,12 @@ SQLITE_PRIVATE void sqlite3VdbeFreeCursor(Vdbe *p, VdbeCursor *pCx){
** Close all cursors in the current frame.
*/
static void closeCursorsInFrame(Vdbe *p){
-
  if( p->apCsr ){
-
    int i;
-
    for(i=0; i<p->nCursor; i++){
-
      VdbeCursor *pC = p->apCsr[i];
-
      if( pC ){
-
        sqlite3VdbeFreeCursor(p, pC);
-
        p->apCsr[i] = 0;
-
      }
+
  int i;
+
  for(i=0; i<p->nCursor; i++){
+
    VdbeCursor *pC = p->apCsr[i];
+
    if( pC ){
+
      sqlite3VdbeFreeCursorNN(p, pC);
+
      p->apCsr[i] = 0;
    }
  }
}
@@ -82354,9 +84616,7 @@ static void closeAllCursors(Vdbe *p){
  }
  assert( p->nFrame==0 );
  closeCursorsInFrame(p);
-
  if( p->aMem ){
-
    releaseMemArray(p->aMem, p->nMem);
-
  }
+
  releaseMemArray(p->aMem, p->nMem);
  while( p->pDelFrame ){
    VdbeFrame *pDel = p->pDelFrame;
    p->pDelFrame = pDel->pParent;
@@ -82703,7 +84963,7 @@ static void checkActiveVdbeCnt(sqlite3 *db){
      if( p->readOnly==0 ) nWrite++;
      if( p->bIsReader ) nRead++;
    }
-
    p = p->pNext;
+
    p = p->pVNext;
  }
  assert( cnt==db->nVdbeActive );
  assert( nWrite==db->nVdbeWrite );
@@ -82796,7 +85056,8 @@ SQLITE_PRIVATE int sqlite3VdbeCheckFk(Vdbe *p, int deferred){
    p->rc = SQLITE_CONSTRAINT_FOREIGNKEY;
    p->errorAction = OE_Abort;
    sqlite3VdbeError(p, "FOREIGN KEY constraint failed");
-
    return SQLITE_ERROR;
+
    if( (p->prepFlags & SQLITE_PREPARE_SAVESQL)==0 ) return SQLITE_ERROR;
+
    return SQLITE_CONSTRAINT_FOREIGNKEY;
  }
  return SQLITE_OK;
}
@@ -82835,9 +85096,7 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){
  ** one, or the complete transaction if there is no statement transaction.
  */

-
  if( p->iVdbeMagic!=VDBE_MAGIC_RUN ){
-
    return SQLITE_OK;
-
  }
+
  assert( p->eVdbeState==VDBE_RUN_STATE );
  if( db->mallocFailed ){
    p->rc = SQLITE_NOMEM_BKPT;
  }
@@ -82846,7 +85105,7 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){

  /* No commit or rollback needed if the program never started or if the
  ** SQL statement does not read or write a database file.  */
-
  if( p->pc>=0 && p->bIsReader ){
+
  if( p->bIsReader ){
    int mrc;   /* Primary error code from p->rc */
    int eStatementOp = 0;
    int isSpecialError;            /* Set to true if a 'special' error */
@@ -82994,15 +85253,13 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){
  }

  /* We have successfully halted and closed the VM.  Record this fact. */
-
  if( p->pc>=0 ){
-
    db->nVdbeActive--;
-
    if( !p->readOnly ) db->nVdbeWrite--;
-
    if( p->bIsReader ) db->nVdbeRead--;
-
    assert( db->nVdbeActive>=db->nVdbeRead );
-
    assert( db->nVdbeRead>=db->nVdbeWrite );
-
    assert( db->nVdbeWrite>=0 );
-
  }
-
  p->iVdbeMagic = VDBE_MAGIC_HALT;
+
  db->nVdbeActive--;
+
  if( !p->readOnly ) db->nVdbeWrite--;
+
  if( p->bIsReader ) db->nVdbeRead--;
+
  assert( db->nVdbeActive>=db->nVdbeRead );
+
  assert( db->nVdbeRead>=db->nVdbeWrite );
+
  assert( db->nVdbeWrite>=0 );
+
  p->eVdbeState = VDBE_HALT_STATE;
  checkActiveVdbeCnt(db);
  if( db->mallocFailed ){
    p->rc = SQLITE_NOMEM_BKPT;
@@ -83084,8 +85341,8 @@ static void vdbeInvokeSqllog(Vdbe *v){
** again.
**
** To look at it another way, this routine resets the state of the
-
** virtual machine from VDBE_MAGIC_RUN or VDBE_MAGIC_HALT back to
-
** VDBE_MAGIC_INIT.
+
** virtual machine from VDBE_RUN_STATE or VDBE_HALT_STATE back to
+
** VDBE_READY_STATE.
*/
SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe *p){
#if defined(SQLITE_DEBUG) || defined(VDBE_PROFILE)
@@ -83099,7 +85356,7 @@ SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe *p){
  ** error, then it might not have been halted properly.  So halt
  ** it now.
  */
-
  sqlite3VdbeHalt(p);
+
  if( p->eVdbeState==VDBE_RUN_STATE ) sqlite3VdbeHalt(p);

  /* If the VDBE has been run even partially, then transfer the error code
  ** and error message from the VDBE into the main database structure.  But
@@ -83113,13 +85370,6 @@ SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe *p){
    }else{
      db->errCode = p->rc;
    }
-
    if( p->runOnlyOnce ) p->expired = 1;
-
  }else if( p->rc && p->expired ){
-
    /* The expired flag was set on the VDBE before the first call
-
    ** to sqlite3_step(). For consistency (since sqlite3_step() was
-
    ** called), set the database error in this case as well.
-
    */
-
    sqlite3ErrorWithMsg(db, p->rc, p->zErrMsg ? "%s" : 0, p->zErrMsg);
  }

  /* Reset register contents and reclaim error message memory.
@@ -83176,7 +85426,6 @@ SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe *p){
    }
  }
#endif
-
  p->iVdbeMagic = VDBE_MAGIC_RESET;
  return p->rc & db->errMask;
}

@@ -83186,7 +85435,10 @@ SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe *p){
*/
SQLITE_PRIVATE int sqlite3VdbeFinalize(Vdbe *p){
  int rc = SQLITE_OK;
-
  if( p->iVdbeMagic==VDBE_MAGIC_RUN || p->iVdbeMagic==VDBE_MAGIC_HALT ){
+
  assert( VDBE_RUN_STATE>VDBE_READY_STATE );
+
  assert( VDBE_HALT_STATE>VDBE_READY_STATE );
+
  assert( VDBE_INIT_STATE<VDBE_READY_STATE );
+
  if( p->eVdbeState>=VDBE_READY_STATE ){
    rc = sqlite3VdbeReset(p);
    assert( (rc & p->db->errMask)==rc );
  }
@@ -83238,23 +85490,26 @@ SQLITE_PRIVATE void sqlite3VdbeDeleteAuxData(sqlite3 *db, AuxData **pp, int iOp,
** VdbeDelete() also unlinks the Vdbe from the list of VMs associated with
** the database connection and frees the object itself.
*/
-
SQLITE_PRIVATE void sqlite3VdbeClearObject(sqlite3 *db, Vdbe *p){
+
static void sqlite3VdbeClearObject(sqlite3 *db, Vdbe *p){
  SubProgram *pSub, *pNext;
+
  assert( db!=0 );
  assert( p->db==0 || p->db==db );
-
  releaseMemArray(p->aColName, p->nResColumn*COLNAME_N);
+
  if( p->aColName ){
+
    releaseMemArray(p->aColName, p->nResColumn*COLNAME_N);
+
    sqlite3DbNNFreeNN(db, p->aColName);
+
  }
  for(pSub=p->pProgram; pSub; pSub=pNext){
    pNext = pSub->pNext;
    vdbeFreeOpArray(db, pSub->aOp, pSub->nOp);
    sqlite3DbFree(db, pSub);
  }
-
  if( p->iVdbeMagic!=VDBE_MAGIC_INIT ){
+
  if( p->eVdbeState!=VDBE_INIT_STATE ){
    releaseMemArray(p->aVar, p->nVar);
-
    sqlite3DbFree(db, p->pVList);
-
    sqlite3DbFree(db, p->pFree);
+
    if( p->pVList ) sqlite3DbNNFreeNN(db, p->pVList);
+
    if( p->pFree ) sqlite3DbNNFreeNN(db, p->pFree);
  }
  vdbeFreeOpArray(db, p->aOp, p->nOp);
-
  sqlite3DbFree(db, p->aColName);
-
  sqlite3DbFree(db, p->zSql);
+
  if( p->zSql ) sqlite3DbNNFreeNN(db, p->zSql);
#ifdef SQLITE_ENABLE_NORMALIZE
  sqlite3DbFree(db, p->zNormSql);
  {
@@ -83284,20 +85539,17 @@ SQLITE_PRIVATE void sqlite3VdbeDelete(Vdbe *p){

  assert( p!=0 );
  db = p->db;
+
  assert( db!=0 );
  assert( sqlite3_mutex_held(db->mutex) );
  sqlite3VdbeClearObject(db, p);
-
  if( p->pPrev ){
-
    p->pPrev->pNext = p->pNext;
-
  }else{
-
    assert( db->pVdbe==p );
-
    db->pVdbe = p->pNext;
-
  }
-
  if( p->pNext ){
-
    p->pNext->pPrev = p->pPrev;
+
  if( db->pnBytesFreed==0 ){
+
    assert( p->ppVPrev!=0 );
+
    *p->ppVPrev = p->pVNext;
+
    if( p->pVNext ){
+
      p->pVNext->ppVPrev = p->ppVPrev;
+
    }
  }
-
  p->iVdbeMagic = VDBE_MAGIC_DEAD;
-
  p->db = 0;
-
  sqlite3DbFreeNN(db, p);
+
  sqlite3DbNNFreeNN(db, p);
}

/*
@@ -83331,7 +85583,7 @@ SQLITE_PRIVATE int SQLITE_NOINLINE sqlite3VdbeFinishMoveto(VdbeCursor *p){
** is supposed to be pointing.  If the row was deleted out from under the
** cursor, set the cursor to point to a NULL row.
*/
-
static int SQLITE_NOINLINE handleMovedCursor(VdbeCursor *p){
+
SQLITE_PRIVATE int SQLITE_NOINLINE sqlite3VdbeHandleMovedCursor(VdbeCursor *p){
  int isDifferentRow, rc;
  assert( p->eCurType==CURTYPE_BTREE );
  assert( p->uc.pCursor!=0 );
@@ -83347,41 +85599,9 @@ static int SQLITE_NOINLINE handleMovedCursor(VdbeCursor *p){
** if need be.  Return any I/O error from the restore operation.
*/
SQLITE_PRIVATE int sqlite3VdbeCursorRestore(VdbeCursor *p){
-
  assert( p->eCurType==CURTYPE_BTREE );
-
  if( sqlite3BtreeCursorHasMoved(p->uc.pCursor) ){
-
    return handleMovedCursor(p);
-
  }
-
  return SQLITE_OK;
-
}
-

-
/*
-
** Make sure the cursor p is ready to read or write the row to which it
-
** was last positioned.  Return an error code if an OOM fault or I/O error
-
** prevents us from positioning the cursor to its correct position.
-
**
-
** If a MoveTo operation is pending on the given cursor, then do that
-
** MoveTo now.  If no move is pending, check to see if the row has been
-
** deleted out from under the cursor and if it has, mark the row as
-
** a NULL row.
-
**
-
** If the cursor is already pointing to the correct row and that row has
-
** not been deleted out from under the cursor, then this routine is a no-op.
-
*/
-
SQLITE_PRIVATE int sqlite3VdbeCursorMoveto(VdbeCursor **pp, u32 *piCol){
-
  VdbeCursor *p = *pp;
-
  assert( p->eCurType==CURTYPE_BTREE || p->eCurType==CURTYPE_PSEUDO );
-
  if( p->deferredMoveto ){
-
    u32 iMap;
-
    assert( !p->isEphemeral );
-
    if( p->ub.aAltMap && (iMap = p->ub.aAltMap[1+*piCol])>0 && !p->nullRow ){
-
      *pp = p->pAltCursor;
-
      *piCol = iMap - 1;
-
      return SQLITE_OK;
-
    }
-
    return sqlite3VdbeFinishMoveto(p);
-
  }
+
  assert( p->eCurType==CURTYPE_BTREE || IsNullCursor(p) );
  if( sqlite3BtreeCursorHasMoved(p->uc.pCursor) ){
-
    return handleMovedCursor(p);
+
    return sqlite3VdbeHandleMovedCursor(p);
  }
  return SQLITE_OK;
}
@@ -83392,7 +85612,7 @@ SQLITE_PRIVATE int sqlite3VdbeCursorMoveto(VdbeCursor **pp, u32 *piCol){
** sqlite3VdbeSerialType()
** sqlite3VdbeSerialTypeLen()
** sqlite3VdbeSerialLen()
-
** sqlite3VdbeSerialPut()
+
** sqlite3VdbeSerialPut()  <--- in-lined into OP_MakeRecord as of 2022-04-02
** sqlite3VdbeSerialGet()
**
** encapsulate the code that serializes values for storage in SQLite
@@ -83504,7 +85724,7 @@ SQLITE_PRIVATE u32 sqlite3VdbeSerialType(Mem *pMem, int file_format, u32 *pLen){
/*
** The sizes for serial types less than 128
*/
-
static const u8 sqlite3SmallTypeSizes[] = {
+
SQLITE_PRIVATE const u8 sqlite3SmallTypeSizes[128] = {
        /*  0   1   2   3   4   5   6   7   8   9 */
/*   0 */   0,  1,  2,  3,  4,  6,  8,  8,  0,  0,
/*  10 */   0,  0,  0,  0,  1,  1,  2,  2,  3,  3,
@@ -83573,7 +85793,7 @@ SQLITE_PRIVATE u8 sqlite3VdbeOneByteSerialTypeLen(u8 serial_type){
** so we trust him.
*/
#ifdef SQLITE_MIXED_ENDIAN_64BIT_FLOAT
-
static u64 floatSwap(u64 in){
+
SQLITE_PRIVATE u64 sqlite3FloatSwap(u64 in){
  union {
    u64 r;
    u32 i[2];
@@ -83586,59 +85806,8 @@ static u64 floatSwap(u64 in){
  u.i[1] = t;
  return u.r;
}
-
# define swapMixedEndianFloat(X)  X = floatSwap(X)
-
#else
-
# define swapMixedEndianFloat(X)
-
#endif
-

-
/*
-
** Write the serialized data blob for the value stored in pMem into
-
** buf. It is assumed that the caller has allocated sufficient space.
-
** Return the number of bytes written.
-
**
-
** nBuf is the amount of space left in buf[].  The caller is responsible
-
** for allocating enough space to buf[] to hold the entire field, exclusive
-
** of the pMem->u.nZero bytes for a MEM_Zero value.
-
**
-
** Return the number of bytes actually written into buf[].  The number
-
** of bytes in the zero-filled tail is included in the return value only
-
** if those bytes were zeroed in buf[].
-
*/
-
SQLITE_PRIVATE u32 sqlite3VdbeSerialPut(u8 *buf, Mem *pMem, u32 serial_type){
-
  u32 len;
-

-
  /* Integer and Real */
-
  if( serial_type<=7 && serial_type>0 ){
-
    u64 v;
-
    u32 i;
-
    if( serial_type==7 ){
-
      assert( sizeof(v)==sizeof(pMem->u.r) );
-
      memcpy(&v, &pMem->u.r, sizeof(v));
-
      swapMixedEndianFloat(v);
-
    }else{
-
      v = pMem->u.i;
-
    }
-
    len = i = sqlite3SmallTypeSizes[serial_type];
-
    assert( i>0 );
-
    do{
-
      buf[--i] = (u8)(v&0xFF);
-
      v >>= 8;
-
    }while( i );
-
    return len;
-
  }
-

-
  /* String or blob */
-
  if( serial_type>=12 ){
-
    assert( pMem->n + ((pMem->flags & MEM_Zero)?pMem->u.nZero:0)
-
             == (int)sqlite3VdbeSerialTypeLen(serial_type) );
-
    len = pMem->n;
-
    if( len>0 ) memcpy(buf, pMem->z, len);
-
    return len;
-
  }
+
#endif /* SQLITE_MIXED_ENDIAN_64BIT_FLOAT */

-
  /* NULL or constants 0 or 1 */
-
  return 0;
-
}

/* Input "x" is a sequence of unsigned characters that represent a
** big-endian integer.  Return the equivalent native integer
@@ -83804,10 +85973,10 @@ SQLITE_PRIVATE UnpackedRecord *sqlite3VdbeAllocUnpackedRecord(
){
  UnpackedRecord *p;              /* Unpacked record to return */
  int nByte;                      /* Number of bytes required for *p */
-
  nByte = ROUND8(sizeof(UnpackedRecord)) + sizeof(Mem)*(pKeyInfo->nKeyField+1);
+
  nByte = ROUND8P(sizeof(UnpackedRecord)) + sizeof(Mem)*(pKeyInfo->nKeyField+1);
  p = (UnpackedRecord *)sqlite3DbMallocRaw(pKeyInfo->db, nByte);
  if( !p ) return 0;
-
  p->aMem = (Mem*)&((char*)p)[ROUND8(sizeof(UnpackedRecord))];
+
  p->aMem = (Mem*)&((char*)p)[ROUND8P(sizeof(UnpackedRecord))];
  assert( pKeyInfo->aSortFlags!=0 );
  p->pKeyInfo = pKeyInfo;
  p->nField = pKeyInfo->nKeyField + 1;
@@ -84043,8 +86212,8 @@ static int vdbeCompareMemString(
    }else{
      rc = pColl->xCmp(pColl->pUser, c1.n, v1, c2.n, v2);
    }
-
    sqlite3VdbeMemRelease(&c1);
-
    sqlite3VdbeMemRelease(&c2);
+
    sqlite3VdbeMemReleaseMalloc(&c1);
+
    sqlite3VdbeMemReleaseMalloc(&c2);
    return rc;
  }
}
@@ -84305,14 +86474,22 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompareWithSkip(
  ** two elements in the keys are equal. Fix the various stack variables so
  ** that this routine begins comparing at the second field. */
  if( bSkip ){
-
    u32 s1;
-
    idx1 = 1 + getVarint32(&aKey1[1], s1);
+
    u32 s1 = aKey1[1];
+
    if( s1<0x80 ){
+
      idx1 = 2;
+
    }else{
+
      idx1 = 1 + sqlite3GetVarint32(&aKey1[1], &s1);
+
    }
    szHdr1 = aKey1[0];
    d1 = szHdr1 + sqlite3VdbeSerialTypeLen(s1);
    i = 1;
    pRhs++;
  }else{
-
    idx1 = getVarint32(aKey1, szHdr1);
+
    if( (szHdr1 = aKey1[0])<0x80 ){
+
      idx1 = 1;
+
    }else{
+
      idx1 = sqlite3GetVarint32(aKey1, &szHdr1);
+
    }
    d1 = szHdr1;
    i = 0;
  }
@@ -84327,7 +86504,7 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompareWithSkip(
  assert( pPKey2->pKeyInfo->aSortFlags!=0 );
  assert( pPKey2->pKeyInfo->nKeyField>0 );
  assert( idx1<=szHdr1 || CORRUPT_DB );
-
  do{
+
  while( 1 /*exit-by-break*/ ){
    u32 serial_type;

    /* RHS is an integer */
@@ -84337,7 +86514,7 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompareWithSkip(
      serial_type = aKey1[idx1];
      testcase( serial_type==12 );
      if( serial_type>=10 ){
-
        rc = +1;
+
        rc = serial_type==10 ? -1 : +1;
      }else if( serial_type==0 ){
        rc = -1;
      }else if( serial_type==7 ){
@@ -84362,7 +86539,7 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompareWithSkip(
        ** numbers). Types 10 and 11 are currently "reserved for future
        ** use", so it doesn't really matter what the results of comparing
        ** them to numberic values are.  */
-
        rc = +1;
+
        rc = serial_type==10 ? -1 : +1;
      }else if( serial_type==0 ){
        rc = -1;
      }else{
@@ -84443,7 +86620,7 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompareWithSkip(
    /* RHS is null */
    else{
      serial_type = aKey1[idx1];
-
      rc = (serial_type!=0);
+
      rc = (serial_type!=0 && serial_type!=10);
    }

    if( rc!=0 ){
@@ -84465,8 +86642,13 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompareWithSkip(
    if( i==pPKey2->nField ) break;
    pRhs++;
    d1 += sqlite3VdbeSerialTypeLen(serial_type);
+
    if( d1>(unsigned)nKey1 ) break;
    idx1 += sqlite3VarintLen(serial_type);
-
  }while( idx1<(unsigned)szHdr1 && d1<=(unsigned)nKey1 );
+
    if( idx1>=(unsigned)szHdr1 ){
+
      pPKey2->errCode = (u8)SQLITE_CORRUPT_BKPT;
+
      return 0;  /* Corrupt index */
+
    }
+
  }

  /* No memory allocation is ever used on mem1.  Prove this using
  ** the following assert().  If the assert() fails, it indicates a
@@ -84568,7 +86750,8 @@ static int vdbeRecordCompareInt(
      return sqlite3VdbeRecordCompare(nKey1, pKey1, pPKey2);
  }

-
  v = pPKey2->aMem[0].u.i;
+
  assert( pPKey2->u.i == pPKey2->aMem[0].u.i );
+
  v = pPKey2->u.i;
  if( v>lhs ){
    res = pPKey2->r1;
  }else if( v<lhs ){
@@ -84603,12 +86786,18 @@ static int vdbeRecordCompareString(
  int res;

  assert( pPKey2->aMem[0].flags & MEM_Str );
+
  assert( pPKey2->aMem[0].n == pPKey2->n );
+
  assert( pPKey2->aMem[0].z == pPKey2->u.z );
  vdbeAssertFieldCountWithinLimits(nKey1, pKey1, pPKey2->pKeyInfo);
-
  serial_type = (u8)(aKey1[1]);
-
  if( serial_type >= 0x80 ){
-
    sqlite3GetVarint32(&aKey1[1], (u32*)&serial_type);
-
  }
+
  serial_type = (signed char)(aKey1[1]);
+

+
vrcs_restart:
  if( serial_type<12 ){
+
    if( serial_type<0 ){
+
      sqlite3GetVarint32(&aKey1[1], (u32*)&serial_type);
+
      if( serial_type>=12 ) goto vrcs_restart;
+
      assert( CORRUPT_DB );
+
    }
    res = pPKey2->r1;      /* (pKey1/nKey1) is a number or a null */
  }else if( !(serial_type & 0x01) ){
    res = pPKey2->r2;      /* (pKey1/nKey1) is a blob */
@@ -84622,15 +86811,15 @@ static int vdbeRecordCompareString(
      pPKey2->errCode = (u8)SQLITE_CORRUPT_BKPT;
      return 0;    /* Corruption */
    }
-
    nCmp = MIN( pPKey2->aMem[0].n, nStr );
-
    res = memcmp(&aKey1[szHdr], pPKey2->aMem[0].z, nCmp);
+
    nCmp = MIN( pPKey2->n, nStr );
+
    res = memcmp(&aKey1[szHdr], pPKey2->u.z, nCmp);

    if( res>0 ){
      res = pPKey2->r2;
    }else if( res<0 ){
      res = pPKey2->r1;
    }else{
-
      res = nStr - pPKey2->aMem[0].n;
+
      res = nStr - pPKey2->n;
      if( res==0 ){
        if( pPKey2->nField>1 ){
          res = sqlite3VdbeRecordCompareWithSkip(nKey1, pKey1, pPKey2, 1);
@@ -84685,6 +86874,7 @@ SQLITE_PRIVATE RecordCompare sqlite3VdbeFindCompare(UnpackedRecord *p){
      p->r2 = 1;
    }
    if( (flags & MEM_Int) ){
+
      p->u.i = p->aMem[0].u.i;
      return vdbeRecordCompareInt;
    }
    testcase( flags & MEM_Real );
@@ -84694,6 +86884,8 @@ SQLITE_PRIVATE RecordCompare sqlite3VdbeFindCompare(UnpackedRecord *p){
     && p->pKeyInfo->aColl[0]==0
    ){
      assert( flags & MEM_Str );
+
      p->u.z = p->aMem[0].z;
+
      p->n = p->aMem[0].n;
      return vdbeRecordCompareString;
    }
  }
@@ -84766,14 +86958,14 @@ SQLITE_PRIVATE int sqlite3VdbeIdxRowid(sqlite3 *db, BtCursor *pCur, i64 *rowid){
  /* Fetch the integer off the end of the index record */
  sqlite3VdbeSerialGet((u8*)&m.z[m.n-lenRowid], typeRowid, &v);
  *rowid = v.u.i;
-
  sqlite3VdbeMemRelease(&m);
+
  sqlite3VdbeMemReleaseMalloc(&m);
  return SQLITE_OK;

  /* Jump here if database corruption is detected after m has been
  ** allocated.  Free the m object and return SQLITE_CORRUPT. */
idx_rowid_corruption:
  testcase( m.szMalloc!=0 );
-
  sqlite3VdbeMemRelease(&m);
+
  sqlite3VdbeMemReleaseMalloc(&m);
  return SQLITE_CORRUPT_BKPT;
}

@@ -84815,7 +87007,7 @@ SQLITE_PRIVATE int sqlite3VdbeIdxKeyCompare(
    return rc;
  }
  *res = sqlite3VdbeRecordCompareWithSkip(m.n, m.z, pUnpacked, 0);
-
  sqlite3VdbeMemRelease(&m);
+
  sqlite3VdbeMemReleaseMalloc(&m);
  return SQLITE_OK;
}

@@ -84857,7 +87049,7 @@ SQLITE_PRIVATE void sqlite3VdbeCountChanges(Vdbe *v){
*/
SQLITE_PRIVATE void sqlite3ExpirePreparedStatements(sqlite3 *db, int iCode){
  Vdbe *p;
-
  for(p = db->pVdbe; p; p=p->pNext){
+
  for(p = db->pVdbe; p; p=p->pVNext){
    p->expired = iCode+1;
  }
}
@@ -84978,13 +87170,14 @@ SQLITE_PRIVATE void sqlite3VtabImportErrmsg(Vdbe *p, sqlite3_vtab *pVtab){
** the vdbeUnpackRecord() function found in vdbeapi.c.
*/
static void vdbeFreeUnpacked(sqlite3 *db, int nField, UnpackedRecord *p){
+
  assert( db!=0 );
  if( p ){
    int i;
    for(i=0; i<nField; i++){
      Mem *pMem = &p->aMem[i];
-
      if( pMem->zMalloc ) sqlite3VdbeMemRelease(pMem);
+
      if( pMem->zMalloc ) sqlite3VdbeMemReleaseMalloc(pMem);
    }
-
    sqlite3DbFreeNN(db, p);
+
    sqlite3DbNNFreeNN(db, p);
  }
}
#endif /* SQLITE_ENABLE_PREUPDATE_HOOK */
@@ -85055,7 +87248,7 @@ SQLITE_PRIVATE void sqlite3VdbePreUpdateHook(
    for(i=0; i<pCsr->nField; i++){
      sqlite3VdbeMemRelease(&preupdate.aNew[i]);
    }
-
    sqlite3DbFreeNN(db, preupdate.aNew);
+
    sqlite3DbNNFreeNN(db, preupdate.aNew);
  }
}
#endif /* SQLITE_ENABLE_PREUPDATE_HOOK */
@@ -85172,7 +87365,9 @@ SQLITE_API int sqlite3_finalize(sqlite3_stmt *pStmt){
    if( vdbeSafety(v) ) return SQLITE_MISUSE_BKPT;
    sqlite3_mutex_enter(db->mutex);
    checkProfileCallback(db, v);
-
    rc = sqlite3VdbeFinalize(v);
+
    assert( v->eVdbeState>=VDBE_READY_STATE );
+
    rc = sqlite3VdbeReset(v);
+
    sqlite3VdbeDelete(v);
    rc = sqlite3ApiExit(db, rc);
    sqlite3LeaveMutexAndCloseZombie(db);
  }
@@ -85380,6 +87575,9 @@ SQLITE_API int sqlite3_value_type(sqlite3_value* pVal){
#endif
  return aType[pVal->flags&MEM_AffMask];
}
+
SQLITE_API int sqlite3_value_encoding(sqlite3_value *pVal){
+
  return pVal->enc;
+
}

/* Return true if a parameter to xUpdate represents an unchanged column */
SQLITE_API int sqlite3_value_nochange(sqlite3_value *pVal){
@@ -85409,6 +87607,9 @@ SQLITE_API sqlite3_value *sqlite3_value_dup(const sqlite3_value *pOrig){
      sqlite3ValueFree(pNew);
      pNew = 0;
    }
+
  }else if( pNew->flags & MEM_Null ){
+
    /* Do not duplicate pointer values */
+
    pNew->flags &= ~(MEM_Term|MEM_Subtype);
  }
  return pNew;
}
@@ -85439,7 +87640,8 @@ static void setResultStrOrError(
  u8 enc,                 /* Encoding of z.  0 for BLOBs */
  void (*xDel)(void*)     /* Destructor function */
){
-
  int rc = sqlite3VdbeMemSetStr(pCtx->pOut, z, n, enc, xDel);
+
  Mem *pOut = pCtx->pOut;
+
  int rc = sqlite3VdbeMemSetStr(pOut, z, n, enc, xDel);
  if( rc ){
    if( rc==SQLITE_TOOBIG ){
      sqlite3_result_error_toobig(pCtx);
@@ -85449,6 +87651,11 @@ static void setResultStrOrError(
      assert( rc==SQLITE_NOMEM );
      sqlite3_result_error_nomem(pCtx);
    }
+
    return;
+
  }
+
  sqlite3VdbeChangeEncoding(pOut, pCtx->enc);
+
  if( sqlite3VdbeMemTooBig(pOut) ){
+
    sqlite3_result_error_toobig(pCtx);
  }
}
static int invokeValueDestructor(
@@ -85592,17 +87799,22 @@ SQLITE_API void sqlite3_result_text16le(
}
#endif /* SQLITE_OMIT_UTF16 */
SQLITE_API void sqlite3_result_value(sqlite3_context *pCtx, sqlite3_value *pValue){
+
  Mem *pOut = pCtx->pOut;
  assert( sqlite3_mutex_held(pCtx->pOut->db->mutex) );
-
  sqlite3VdbeMemCopy(pCtx->pOut, pValue);
+
  sqlite3VdbeMemCopy(pOut, pValue);
+
  sqlite3VdbeChangeEncoding(pOut, pCtx->enc);
+
  if( sqlite3VdbeMemTooBig(pOut) ){
+
    sqlite3_result_error_toobig(pCtx);
+
  }
}
SQLITE_API void sqlite3_result_zeroblob(sqlite3_context *pCtx, int n){
-
  assert( sqlite3_mutex_held(pCtx->pOut->db->mutex) );
-
  sqlite3VdbeMemSetZeroBlob(pCtx->pOut, n);
+
  sqlite3_result_zeroblob64(pCtx, n>0 ? n : 0);
}
SQLITE_API int sqlite3_result_zeroblob64(sqlite3_context *pCtx, u64 n){
  Mem *pOut = pCtx->pOut;
  assert( sqlite3_mutex_held(pOut->db->mutex) );
  if( n>(u64)pOut->db->aLimit[SQLITE_LIMIT_LENGTH] ){
+
    sqlite3_result_error_toobig(pCtx);
    return SQLITE_TOOBIG;
  }
#ifndef SQLITE_OMIT_INCRBLOB
@@ -85618,8 +87830,8 @@ SQLITE_API void sqlite3_result_error_code(sqlite3_context *pCtx, int errCode){
  if( pCtx->pVdbe ) pCtx->pVdbe->rcApp = errCode;
#endif
  if( pCtx->pOut->flags & MEM_Null ){
-
    sqlite3VdbeMemSetStr(pCtx->pOut, sqlite3ErrStr(errCode), -1,
-
                         SQLITE_UTF8, SQLITE_STATIC);
+
    setResultStrOrError(pCtx, sqlite3ErrStr(errCode), -1, SQLITE_UTF8,
+
                        SQLITE_STATIC);
  }
}

@@ -85693,80 +87905,83 @@ static int sqlite3Step(Vdbe *p){
  int rc;

  assert(p);
-
  if( p->iVdbeMagic!=VDBE_MAGIC_RUN ){
-
    /* We used to require that sqlite3_reset() be called before retrying
-
    ** sqlite3_step() after any error or after SQLITE_DONE.  But beginning
-
    ** with version 3.7.0, we changed this so that sqlite3_reset() would
-
    ** be called automatically instead of throwing the SQLITE_MISUSE error.
-
    ** This "automatic-reset" change is not technically an incompatibility,
-
    ** since any application that receives an SQLITE_MISUSE is broken by
-
    ** definition.
-
    **
-
    ** Nevertheless, some published applications that were originally written
-
    ** for version 3.6.23 or earlier do in fact depend on SQLITE_MISUSE
-
    ** returns, and those were broken by the automatic-reset change.  As a
-
    ** a work-around, the SQLITE_OMIT_AUTORESET compile-time restores the
-
    ** legacy behavior of returning SQLITE_MISUSE for cases where the
-
    ** previous sqlite3_step() returned something other than a SQLITE_LOCKED
-
    ** or SQLITE_BUSY error.
-
    */
-
#ifdef SQLITE_OMIT_AUTORESET
-
    if( (rc = p->rc&0xff)==SQLITE_BUSY || rc==SQLITE_LOCKED ){
-
      sqlite3_reset((sqlite3_stmt*)p);
-
    }else{
-
      return SQLITE_MISUSE_BKPT;
-
    }
-
#else
-
    sqlite3_reset((sqlite3_stmt*)p);
-
#endif
-
  }
-

-
  /* Check that malloc() has not failed. If it has, return early. */
  db = p->db;
-
  if( db->mallocFailed ){
-
    p->rc = SQLITE_NOMEM;
-
    return SQLITE_NOMEM_BKPT;
-
  }
+
  if( p->eVdbeState!=VDBE_RUN_STATE ){
+
    restart_step:
+
    if( p->eVdbeState==VDBE_READY_STATE ){
+
      if( p->expired ){
+
        p->rc = SQLITE_SCHEMA;
+
        rc = SQLITE_ERROR;
+
        if( (p->prepFlags & SQLITE_PREPARE_SAVESQL)!=0 ){
+
          /* If this statement was prepared using saved SQL and an
+
          ** error has occurred, then return the error code in p->rc to the
+
          ** caller. Set the error code in the database handle to the same
+
          ** value.
+
          */
+
          rc = sqlite3VdbeTransferError(p);
+
        }
+
        goto end_of_step;
+
      }

-
  if( p->pc<0 && p->expired ){
-
    p->rc = SQLITE_SCHEMA;
-
    rc = SQLITE_ERROR;
-
    if( (p->prepFlags & SQLITE_PREPARE_SAVESQL)!=0 ){
-
      /* If this statement was prepared using saved SQL and an
-
      ** error has occurred, then return the error code in p->rc to the
-
      ** caller. Set the error code in the database handle to the same value.
+
      /* If there are no other statements currently running, then
+
      ** reset the interrupt flag.  This prevents a call to sqlite3_interrupt
+
      ** from interrupting a statement that has not yet started.
      */
-
      rc = sqlite3VdbeTransferError(p);
-
    }
-
    goto end_of_step;
-
  }
-
  if( p->pc<0 ){
-
    /* If there are no other statements currently running, then
-
    ** reset the interrupt flag.  This prevents a call to sqlite3_interrupt
-
    ** from interrupting a statement that has not yet started.
-
    */
-
    if( db->nVdbeActive==0 ){
-
      AtomicStore(&db->u1.isInterrupted, 0);
-
    }
+
      if( db->nVdbeActive==0 ){
+
        AtomicStore(&db->u1.isInterrupted, 0);
+
      }

-
    assert( db->nVdbeWrite>0 || db->autoCommit==0
-
        || (db->nDeferredCons==0 && db->nDeferredImmCons==0)
-
    );
+
      assert( db->nVdbeWrite>0 || db->autoCommit==0
+
          || (db->nDeferredCons==0 && db->nDeferredImmCons==0)
+
      );

#ifndef SQLITE_OMIT_TRACE
-
    if( (db->mTrace & (SQLITE_TRACE_PROFILE|SQLITE_TRACE_XPROFILE))!=0
-
        && !db->init.busy && p->zSql ){
-
      sqlite3OsCurrentTimeInt64(db->pVfs, &p->startTime);
-
    }else{
-
      assert( p->startTime==0 );
-
    }
+
      if( (db->mTrace & (SQLITE_TRACE_PROFILE|SQLITE_TRACE_XPROFILE))!=0
+
          && !db->init.busy && p->zSql ){
+
        sqlite3OsCurrentTimeInt64(db->pVfs, &p->startTime);
+
      }else{
+
        assert( p->startTime==0 );
+
      }
#endif

-
    db->nVdbeActive++;
-
    if( p->readOnly==0 ) db->nVdbeWrite++;
-
    if( p->bIsReader ) db->nVdbeRead++;
-
    p->pc = 0;
+
      db->nVdbeActive++;
+
      if( p->readOnly==0 ) db->nVdbeWrite++;
+
      if( p->bIsReader ) db->nVdbeRead++;
+
      p->pc = 0;
+
      p->eVdbeState = VDBE_RUN_STATE;
+
    }else
+

+
    if( ALWAYS(p->eVdbeState==VDBE_HALT_STATE) ){
+
      /* We used to require that sqlite3_reset() be called before retrying
+
      ** sqlite3_step() after any error or after SQLITE_DONE.  But beginning
+
      ** with version 3.7.0, we changed this so that sqlite3_reset() would
+
      ** be called automatically instead of throwing the SQLITE_MISUSE error.
+
      ** This "automatic-reset" change is not technically an incompatibility,
+
      ** since any application that receives an SQLITE_MISUSE is broken by
+
      ** definition.
+
      **
+
      ** Nevertheless, some published applications that were originally written
+
      ** for version 3.6.23 or earlier do in fact depend on SQLITE_MISUSE
+
      ** returns, and those were broken by the automatic-reset change.  As a
+
      ** a work-around, the SQLITE_OMIT_AUTORESET compile-time restores the
+
      ** legacy behavior of returning SQLITE_MISUSE for cases where the
+
      ** previous sqlite3_step() returned something other than a SQLITE_LOCKED
+
      ** or SQLITE_BUSY error.
+
      */
+
#ifdef SQLITE_OMIT_AUTORESET
+
      if( (rc = p->rc&0xff)==SQLITE_BUSY || rc==SQLITE_LOCKED ){
+
        sqlite3_reset((sqlite3_stmt*)p);
+
      }else{
+
        return SQLITE_MISUSE_BKPT;
+
      }
+
#else
+
      sqlite3_reset((sqlite3_stmt*)p);
+
#endif
+
      assert( p->eVdbeState==VDBE_READY_STATE );
+
      goto restart_step;
+
    }
  }
+

#ifdef SQLITE_DEBUG
  p->rcApp = SQLITE_OK;
#endif
@@ -85781,7 +87996,12 @@ static int sqlite3Step(Vdbe *p){
    db->nVdbeExec--;
  }

-
  if( rc!=SQLITE_ROW ){
+
  if( rc==SQLITE_ROW ){
+
    assert( p->rc==SQLITE_OK );
+
    assert( db->mallocFailed==0 );
+
    db->errCode = SQLITE_ROW;
+
    return SQLITE_ROW;
+
  }else{
#ifndef SQLITE_OMIT_TRACE
    /* If the statement completed successfully, invoke the profile callback */
    checkProfileCallback(db, p);
@@ -85833,7 +88053,6 @@ SQLITE_API int sqlite3_step(sqlite3_stmt *pStmt){
  }
  db = v->db;
  sqlite3_mutex_enter(db->mutex);
-
  v->doingRerun = 0;
  while( (rc = sqlite3Step(v))==SQLITE_SCHEMA
         && cnt++ < SQLITE_MAX_SCHEMA_RETRY ){
    int savedPc = v->pc;
@@ -85859,7 +88078,13 @@ SQLITE_API int sqlite3_step(sqlite3_stmt *pStmt){
      break;
    }
    sqlite3_reset(pStmt);
-
    if( savedPc>=0 ) v->doingRerun = 1;
+
    if( savedPc>=0 ){
+
      /* Setting minWriteFileFormat to 254 is a signal to the OP_Init and
+
      ** OP_Trace opcodes to *not* perform SQLITE_TRACE_STMT because it has
+
      ** already been done once on a prior invocation that failed due to
+
      ** SQLITE_SCHEMA.   tag-20220401a  */
+
      v->minWriteFileFormat = 254;
+
    }
    assert( v->expired==0 );
  }
  sqlite3_mutex_leave(db->mutex);
@@ -86168,15 +88393,15 @@ static const Mem *columnNullValue(void){
#endif
    = {
        /* .u          = */ {0},
+
        /* .z          = */ (char*)0,
+
        /* .n          = */ (int)0,
        /* .flags      = */ (u16)MEM_Null,
        /* .enc        = */ (u8)0,
        /* .eSubtype   = */ (u8)0,
-
        /* .n          = */ (int)0,
-
        /* .z          = */ (char*)0,
-
        /* .zMalloc    = */ (char*)0,
+
        /* .db         = */ (sqlite3*)0,
        /* .szMalloc   = */ (int)0,
        /* .uTemp      = */ (u32)0,
-
        /* .db         = */ (sqlite3*)0,
+
        /* .zMalloc    = */ (char*)0,
        /* .xDel       = */ (void(*)(void*))0,
#ifdef SQLITE_DEBUG
        /* .pScopyFrom = */ (Mem*)0,
@@ -86467,25 +88692,24 @@ SQLITE_API const void *sqlite3_column_origin_name16(sqlite3_stmt *pStmt, int N){
** The error code stored in database p->db is overwritten with the return
** value in any case.
*/
-
static int vdbeUnbind(Vdbe *p, int i){
+
static int vdbeUnbind(Vdbe *p, unsigned int i){
  Mem *pVar;
  if( vdbeSafetyNotNull(p) ){
    return SQLITE_MISUSE_BKPT;
  }
  sqlite3_mutex_enter(p->db->mutex);
-
  if( p->iVdbeMagic!=VDBE_MAGIC_RUN || p->pc>=0 ){
+
  if( p->eVdbeState!=VDBE_READY_STATE ){
    sqlite3Error(p->db, SQLITE_MISUSE);
    sqlite3_mutex_leave(p->db->mutex);
    sqlite3_log(SQLITE_MISUSE,
        "bind on a busy prepared statement: [%s]", p->zSql);
    return SQLITE_MISUSE_BKPT;
  }
-
  if( i<1 || i>p->nVar ){
+
  if( i>=(unsigned int)p->nVar ){
    sqlite3Error(p->db, SQLITE_RANGE);
    sqlite3_mutex_leave(p->db->mutex);
    return SQLITE_RANGE;
  }
-
  i--;
  pVar = &p->aVar[i];
  sqlite3VdbeMemRelease(pVar);
  pVar->flags = MEM_Null;
@@ -86522,7 +88746,7 @@ static int bindText(
  Mem *pVar;
  int rc;

-
  rc = vdbeUnbind(p, i);
+
  rc = vdbeUnbind(p, (u32)(i-1));
  if( rc==SQLITE_OK ){
    if( zData!=0 ){
      pVar = &p->aVar[i-1];
@@ -86571,7 +88795,7 @@ SQLITE_API int sqlite3_bind_blob64(
SQLITE_API int sqlite3_bind_double(sqlite3_stmt *pStmt, int i, double rValue){
  int rc;
  Vdbe *p = (Vdbe *)pStmt;
-
  rc = vdbeUnbind(p, i);
+
  rc = vdbeUnbind(p, (u32)(i-1));
  if( rc==SQLITE_OK ){
    sqlite3VdbeMemSetDouble(&p->aVar[i-1], rValue);
    sqlite3_mutex_leave(p->db->mutex);
@@ -86584,7 +88808,7 @@ SQLITE_API int sqlite3_bind_int(sqlite3_stmt *p, int i, int iValue){
SQLITE_API int sqlite3_bind_int64(sqlite3_stmt *pStmt, int i, sqlite_int64 iValue){
  int rc;
  Vdbe *p = (Vdbe *)pStmt;
-
  rc = vdbeUnbind(p, i);
+
  rc = vdbeUnbind(p, (u32)(i-1));
  if( rc==SQLITE_OK ){
    sqlite3VdbeMemSetInt64(&p->aVar[i-1], iValue);
    sqlite3_mutex_leave(p->db->mutex);
@@ -86594,7 +88818,7 @@ SQLITE_API int sqlite3_bind_int64(sqlite3_stmt *pStmt, int i, sqlite_int64 iValu
SQLITE_API int sqlite3_bind_null(sqlite3_stmt *pStmt, int i){
  int rc;
  Vdbe *p = (Vdbe*)pStmt;
-
  rc = vdbeUnbind(p, i);
+
  rc = vdbeUnbind(p, (u32)(i-1));
  if( rc==SQLITE_OK ){
    sqlite3_mutex_leave(p->db->mutex);
  }
@@ -86609,7 +88833,7 @@ SQLITE_API int sqlite3_bind_pointer(
){
  int rc;
  Vdbe *p = (Vdbe*)pStmt;
-
  rc = vdbeUnbind(p, i);
+
  rc = vdbeUnbind(p, (u32)(i-1));
  if( rc==SQLITE_OK ){
    sqlite3VdbeMemSetPointer(&p->aVar[i-1], pPtr, zPTtype, xDestructor);
    sqlite3_mutex_leave(p->db->mutex);
@@ -86687,7 +88911,7 @@ SQLITE_API int sqlite3_bind_value(sqlite3_stmt *pStmt, int i, const sqlite3_valu
SQLITE_API int sqlite3_bind_zeroblob(sqlite3_stmt *pStmt, int i, int n){
  int rc;
  Vdbe *p = (Vdbe *)pStmt;
-
  rc = vdbeUnbind(p, i);
+
  rc = vdbeUnbind(p, (u32)(i-1));
  if( rc==SQLITE_OK ){
#ifndef SQLITE_OMIT_INCRBLOB
    sqlite3VdbeMemSetZeroBlob(&p->aVar[i-1], n);
@@ -86826,7 +89050,7 @@ SQLITE_API int sqlite3_stmt_isexplain(sqlite3_stmt *pStmt){
*/
SQLITE_API int sqlite3_stmt_busy(sqlite3_stmt *pStmt){
  Vdbe *v = (Vdbe*)pStmt;
-
  return v!=0 && v->iVdbeMagic==VDBE_MAGIC_RUN && v->pc>=0;
+
  return v!=0 && v->eVdbeState==VDBE_RUN_STATE;
}

/*
@@ -86847,7 +89071,7 @@ SQLITE_API sqlite3_stmt *sqlite3_next_stmt(sqlite3 *pDb, sqlite3_stmt *pStmt){
  if( pStmt==0 ){
    pNext = (sqlite3_stmt*)pDb->pVdbe;
  }else{
-
    pNext = (sqlite3_stmt*)((Vdbe*)pStmt)->pNext;
+
    pNext = (sqlite3_stmt*)((Vdbe*)pStmt)->pVNext;
  }
  sqlite3_mutex_leave(pDb->mutex);
  return pNext;
@@ -86872,9 +89096,11 @@ SQLITE_API int sqlite3_stmt_status(sqlite3_stmt *pStmt, int op, int resetFlag){
    sqlite3_mutex_enter(db->mutex);
    v = 0;
    db->pnBytesFreed = (int*)&v;
-
    sqlite3VdbeClearObject(db, pVdbe);
-
    sqlite3DbFree(db, pVdbe);
+
    assert( db->lookaside.pEnd==db->lookaside.pTrueEnd );
+
    db->lookaside.pEnd = db->lookaside.pStart;
+
    sqlite3VdbeDelete(pVdbe);
    db->pnBytesFreed = 0;
+
    db->lookaside.pEnd = db->lookaside.pTrueEnd;
    sqlite3_mutex_leave(db->mutex);
  }else{
    v = pVdbe->aCounter[op];
@@ -87666,12 +89892,12 @@ static VdbeCursor *allocateCursor(
  int nByte;
  VdbeCursor *pCx = 0;
  nByte =
-
      ROUND8(sizeof(VdbeCursor)) + 2*sizeof(u32)*nField +
+
      ROUND8P(sizeof(VdbeCursor)) + 2*sizeof(u32)*nField +
      (eCurType==CURTYPE_BTREE?sqlite3BtreeCursorSize():0);

  assert( iCur>=0 && iCur<p->nCursor );
  if( p->apCsr[iCur] ){ /*OPTIMIZATION-IF-FALSE*/
-
    sqlite3VdbeFreeCursor(p, p->apCsr[iCur]);
+
    sqlite3VdbeFreeCursorNN(p, p->apCsr[iCur]);
    p->apCsr[iCur] = 0;
  }

@@ -87701,7 +89927,7 @@ static VdbeCursor *allocateCursor(
  pCx->aOffset = &pCx->aType[nField];
  if( eCurType==CURTYPE_BTREE ){
    pCx->uc.pCursor = (BtCursor*)
-
        &pMem->z[ROUND8(sizeof(VdbeCursor))+2*sizeof(u32)*nField];
+
        &pMem->z[ROUND8P(sizeof(VdbeCursor))+2*sizeof(u32)*nField];
    sqlite3BtreeCursorZero(pCx->uc.pCursor);
  }
  return pCx;
@@ -87714,7 +89940,8 @@ static VdbeCursor *allocateCursor(
** return false.
*/
static int alsoAnInt(Mem *pRec, double rValue, i64 *piValue){
-
  i64 iValue = (double)rValue;
+
  i64 iValue;
+
  iValue = sqlite3RealToI64(rValue);
  if( sqlite3RealSameAsInt(rValue,iValue) ){
    *piValue = iValue;
    return 1;
@@ -87876,17 +90103,18 @@ static u16 SQLITE_NOINLINE computeNumericType(Mem *pMem){
** But it does set pMem->u.r and pMem->u.i appropriately.
*/
static u16 numericType(Mem *pMem){
-
  if( pMem->flags & (MEM_Int|MEM_Real|MEM_IntReal) ){
+
  assert( (pMem->flags & MEM_Null)==0
+
       || pMem->db==0 || pMem->db->mallocFailed );
+
  if( pMem->flags & (MEM_Int|MEM_Real|MEM_IntReal|MEM_Null) ){
    testcase( pMem->flags & MEM_Int );
    testcase( pMem->flags & MEM_Real );
    testcase( pMem->flags & MEM_IntReal );
-
    return pMem->flags & (MEM_Int|MEM_Real|MEM_IntReal);
-
  }
-
  if( pMem->flags & (MEM_Str|MEM_Blob) ){
-
    testcase( pMem->flags & MEM_Str );
-
    testcase( pMem->flags & MEM_Blob );
-
    return computeNumericType(pMem);
+
    return pMem->flags & (MEM_Int|MEM_Real|MEM_IntReal|MEM_Null);
  }
+
  assert( pMem->flags & (MEM_Str|MEM_Blob) );
+
  testcase( pMem->flags & MEM_Str );
+
  testcase( pMem->flags & MEM_Blob );
+
  return computeNumericType(pMem);
  return 0;
}

@@ -88140,7 +90368,7 @@ SQLITE_PRIVATE int sqlite3VdbeExec(
#endif
  /*** INSERT STACK UNION HERE ***/

-
  assert( p->iVdbeMagic==VDBE_MAGIC_RUN );  /* sqlite3_step() verifies this */
+
  assert( p->eVdbeState==VDBE_RUN_STATE );  /* sqlite3_step() verifies this */
  sqlite3VdbeEnter(p);
#ifndef SQLITE_OMIT_PROGRESS_CALLBACK
  if( db->xProgress ){
@@ -88383,26 +90611,39 @@ case OP_Gosub: { /* jump */
  pIn1->flags = MEM_Int;
  pIn1->u.i = (int)(pOp-aOp);
  REGISTER_TRACE(pOp->p1, pIn1);
-

-
  /* Most jump operations do a goto to this spot in order to update
-
  ** the pOp pointer. */
-
jump_to_p2:
-
  assert( pOp->p2>0 );       /* There are never any jumps to instruction 0 */
-
  assert( pOp->p2<p->nOp );  /* Jumps must be in range */
-
  pOp = &aOp[pOp->p2 - 1];
-
  break;
+
  goto jump_to_p2_and_check_for_interrupt;
}

-
/* Opcode:  Return P1 * * * *
+
/* Opcode:  Return P1 P2 P3 * *
**
-
** Jump to the next instruction after the address in register P1.  After
-
** the jump, register P1 becomes undefined.
+
** Jump to the address stored in register P1.  If P1 is a return address
+
** register, then this accomplishes a return from a subroutine.
+
**
+
** If P3 is 1, then the jump is only taken if register P1 holds an integer
+
** values, otherwise execution falls through to the next opcode, and the
+
** OP_Return becomes a no-op. If P3 is 0, then register P1 must hold an
+
** integer or else an assert() is raised.  P3 should be set to 1 when
+
** this opcode is used in combination with OP_BeginSubrtn, and set to 0
+
** otherwise.
+
**
+
** The value in register P1 is unchanged by this opcode.
+
**
+
** P2 is not used by the byte-code engine.  However, if P2 is positive
+
** and also less than the current address, then the "EXPLAIN" output
+
** formatter in the CLI will indent all opcodes from the P2 opcode up
+
** to be not including the current Return.   P2 should be the first opcode
+
** in the subroutine from which this opcode is returning.  Thus the P2
+
** value is a byte-code indentation hint.  See tag-20220407a in
+
** wherecode.c and shell.c.
*/
case OP_Return: {           /* in1 */
  pIn1 = &aMem[pOp->p1];
-
  assert( pIn1->flags==MEM_Int );
-
  pOp = &aOp[pIn1->u.i];
-
  pIn1->flags = MEM_Undefined;
+
  if( pIn1->flags & MEM_Int ){
+
    if( pOp->p3 ){ VdbeBranchTaken(1, 2); }
+
    pOp = &aOp[pIn1->u.i];
+
  }else if( ALWAYS(pOp->p3) ){
+
    VdbeBranchTaken(0, 2);
+
  }
  break;
}

@@ -88425,7 +90666,14 @@ case OP_InitCoroutine: { /* jump */
  assert( !VdbeMemDynamic(pOut) );
  pOut->u.i = pOp->p3 - 1;
  pOut->flags = MEM_Int;
-
  if( pOp->p2 ) goto jump_to_p2;
+
  if( pOp->p2==0 ) break;
+

+
  /* Most jump operations do a goto to this spot in order to update
+
  ** the pOp pointer. */
+
jump_to_p2:
+
  assert( pOp->p2>0 );       /* There are never any jumps to instruction 0 */
+
  assert( pOp->p2<p->nOp );  /* Jumps must be in range */
+
  pOp = &aOp[pOp->p2 - 1];
  break;
}

@@ -88527,11 +90775,10 @@ case OP_Halt: {
  VdbeFrame *pFrame;
  int pcx;

-
  pcx = (int)(pOp - aOp);
#ifdef SQLITE_DEBUG
  if( pOp->p2==OE_Abort ){ sqlite3VdbeAssertAbortable(p); }
#endif
-
  if( pOp->p1==SQLITE_OK && p->pFrame ){
+
  if( p->pFrame && pOp->p1==SQLITE_OK ){
    /* Halt the sub-program. Return control to the parent frame. */
    pFrame = p->pFrame;
    p->pFrame = pFrame->pParent;
@@ -88553,7 +90800,6 @@ case OP_Halt: {
  }
  p->rc = pOp->p1;
  p->errorAction = (u8)pOp->p2;
-
  p->pc = pcx;
  assert( pOp->p5<=4 );
  if( p->rc ){
    if( pOp->p5 ){
@@ -88570,6 +90816,7 @@ case OP_Halt: {
    }else{
      sqlite3VdbeError(p, "%s", pOp->p4.z);
    }
+
    pcx = (int)(pOp - aOp);
    sqlite3_log(pOp->p1, "abort at %d in [%s]: %s", pcx, p->zSql, p->zErrMsg);
  }
  rc = sqlite3VdbeHalt(p);
@@ -88695,6 +90942,28 @@ case OP_String: { /* out2 */
  break;
}

+
/* Opcode: BeginSubrtn * P2 * * *
+
** Synopsis: r[P2]=NULL
+
**
+
** Mark the beginning of a subroutine that can be entered in-line
+
** or that can be called using OP_Gosub.  The subroutine should
+
** be terminated by an OP_Return instruction that has a P1 operand that
+
** is the same as the P2 operand to this opcode and that has P3 set to 1.
+
** If the subroutine is entered in-line, then the OP_Return will simply
+
** fall through.  But if the subroutine is entered using OP_Gosub, then
+
** the OP_Return will jump back to the first instruction after the OP_Gosub.
+
**
+
** This routine works by loading a NULL into the P2 register.  When the
+
** return address register contains a NULL, the OP_Return instruction is
+
** a no-op that simply falls through to the next instruction (assuming that
+
** the OP_Return opcode has a P3 value of 1).  Thus if the subroutine is
+
** entered in-line, then the OP_Return will cause in-line execution to
+
** continue.  But if the subroutine is entered via OP_Gosub, then the
+
** OP_Return will cause a return to the address following the OP_Gosub.
+
**
+
** This opcode is identical to OP_Null.  It has a different name
+
** only to make the byte code easier to read and verify.
+
*/
/* Opcode: Null P1 P2 P3 * *
** Synopsis: r[P2..P3]=NULL
**
@@ -88707,6 +90976,7 @@ case OP_String: { /* out2 */
** NULL values will not compare equal even if SQLITE_NULLEQ is set on
** OP_Ne or OP_Eq.
*/
+
case OP_BeginSubrtn:
case OP_Null: {           /* out2 */
  int cnt;
  u16 nullFlag;
@@ -88837,11 +91107,16 @@ case OP_Move: {
  break;
}

-
/* Opcode: Copy P1 P2 P3 * *
+
/* Opcode: Copy P1 P2 P3 * P5
** Synopsis: r[P2@P3+1]=r[P1@P3+1]
**
** Make a copy of registers P1..P1+P3 into registers P2..P2+P3.
**
+
** If the 0x0002 bit of P5 is set then also clear the MEM_Subtype flag in the
+
** destination.  The 0x0001 bit of P5 indicates that this Copy opcode cannot
+
** be merged.  The 0x0001 bit is used by the query planner and does not
+
** come into play during query execution.
+
**
** This instruction makes a deep copy of the value.  A duplicate
** is made of any string or blob constant.  See also OP_SCopy.
*/
@@ -88856,6 +91131,9 @@ case OP_Copy: {
    memAboutToChange(p, pOut);
    sqlite3VdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
    Deephemeralize(pOut);
+
    if( (pOut->flags & MEM_Subtype)!=0 &&  (pOp->p5 & 0x0002)!=0 ){
+
      pOut->flags &= ~MEM_Subtype;
+
    }
#ifdef SQLITE_DEBUG
    pOut->pScopyFrom = 0;
#endif
@@ -88936,45 +91214,32 @@ case OP_FkCheck: {
** the result row.
*/
case OP_ResultRow: {
-
  Mem *pMem;
-
  int i;
  assert( p->nResColumn==pOp->p2 );
  assert( pOp->p1>0 || CORRUPT_DB );
  assert( pOp->p1+pOp->p2<=(p->nMem+1 - p->nCursor)+1 );

-
  /* Invalidate all ephemeral cursor row caches */
  p->cacheCtr = (p->cacheCtr + 2)|1;
-

-
  /* Make sure the results of the current row are \000 terminated
-
  ** and have an assigned type.  The results are de-ephemeralized as
-
  ** a side effect.
-
  */
-
  pMem = p->pResultSet = &aMem[pOp->p1];
-
  for(i=0; i<pOp->p2; i++){
-
    assert( memIsValid(&pMem[i]) );
-
    Deephemeralize(&pMem[i]);
-
    assert( (pMem[i].flags & MEM_Ephem)==0
-
            || (pMem[i].flags & (MEM_Str|MEM_Blob))==0 );
-
    sqlite3VdbeMemNulTerminate(&pMem[i]);
-
    REGISTER_TRACE(pOp->p1+i, &pMem[i]);
+
  p->pResultSet = &aMem[pOp->p1];
#ifdef SQLITE_DEBUG
-
    /* The registers in the result will not be used again when the
-
    ** prepared statement restarts.  This is because sqlite3_column()
-
    ** APIs might have caused type conversions of made other changes to
-
    ** the register values.  Therefore, we can go ahead and break any
-
    ** OP_SCopy dependencies. */
-
    pMem[i].pScopyFrom = 0;
-
#endif
+
  {
+
    Mem *pMem = p->pResultSet;
+
    int i;
+
    for(i=0; i<pOp->p2; i++){
+
      assert( memIsValid(&pMem[i]) );
+
      REGISTER_TRACE(pOp->p1+i, &pMem[i]);
+
      /* The registers in the result will not be used again when the
+
      ** prepared statement restarts.  This is because sqlite3_column()
+
      ** APIs might have caused type conversions of made other changes to
+
      ** the register values.  Therefore, we can go ahead and break any
+
      ** OP_SCopy dependencies. */
+
      pMem[i].pScopyFrom = 0;
+
    }
  }
+
#endif
  if( db->mallocFailed ) goto no_mem;
-

  if( db->mTrace & SQLITE_TRACE_ROW ){
    db->trace.xV2(SQLITE_TRACE_ROW, db->pTraceArg, p, 0);
  }
-

-

-
  /* Return SQLITE_ROW
-
  */
  p->pc = (int)(pOp - aOp) + 1;
  rc = SQLITE_ROW;
  goto vdbe_return;
@@ -89029,7 +91294,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
  if( nByte>db->aLimit[SQLITE_LIMIT_LENGTH] ){
    goto too_big;
  }
-
  if( sqlite3VdbeMemGrow(pOut, (int)nByte+3, pOut==pIn2) ){
+
  if( sqlite3VdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2) ){
    goto no_mem;
  }
  MemSetTypeFlag(pOut, MEM_Str);
@@ -89041,9 +91306,9 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
  memcpy(&pOut->z[pIn2->n], pIn1->z, pIn1->n);
  assert( (pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn) );
  pIn1->flags = flags1;
+
  if( encoding>SQLITE_UTF8 ) nByte &= ~1;
  pOut->z[nByte]=0;
  pOut->z[nByte+1] = 0;
-
  pOut->z[nByte+2] = 0;
  pOut->flags |= MEM_Term;
  pOut->n = (int)nByte;
  pOut->enc = encoding;
@@ -89094,7 +91359,6 @@ case OP_Subtract: /* same as TK_MINUS, in1, in2, out3 */
case OP_Multiply:              /* same as TK_STAR, in1, in2, out3 */
case OP_Divide:                /* same as TK_SLASH, in1, in2, out3 */
case OP_Remainder: {           /* same as TK_REM, in1, in2, out3 */
-
  u16 flags;      /* Combined MEM_* flags from both inputs */
  u16 type1;      /* Numeric type of left operand */
  u16 type2;      /* Numeric type of right operand */
  i64 iA;         /* Integer value of left operand */
@@ -89103,12 +91367,12 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
  double rB;      /* Real value of right operand */

  pIn1 = &aMem[pOp->p1];
-
  type1 = numericType(pIn1);
+
  type1 = pIn1->flags;
  pIn2 = &aMem[pOp->p2];
-
  type2 = numericType(pIn2);
+
  type2 = pIn2->flags;
  pOut = &aMem[pOp->p3];
-
  flags = pIn1->flags | pIn2->flags;
  if( (type1 & type2 & MEM_Int)!=0 ){
+
int_math:
    iA = pIn1->u.i;
    iB = pIn2->u.i;
    switch( pOp->opcode ){
@@ -89130,9 +91394,12 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
    }
    pOut->u.i = iB;
    MemSetTypeFlag(pOut, MEM_Int);
-
  }else if( (flags & MEM_Null)!=0 ){
+
  }else if( ((type1 | type2) & MEM_Null)!=0 ){
    goto arithmetic_result_is_null;
  }else{
+
    type1 = numericType(pIn1);
+
    type2 = numericType(pIn2);
+
    if( (type1 & type2 & MEM_Int)!=0 ) goto int_math;
fp_math:
    rA = sqlite3VdbeRealValue(pIn1);
    rB = sqlite3VdbeRealValue(pIn2);
@@ -89488,23 +91755,23 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
    assert( (pOp->p5 & SQLITE_AFF_MASK)!=SQLITE_AFF_TEXT || CORRUPT_DB );
    /* Common case of comparison of two integers */
    if( pIn3->u.i > pIn1->u.i ){
-
      iCompare = +1;
      if( sqlite3aGTb[pOp->opcode] ){
        VdbeBranchTaken(1, (pOp->p5 & SQLITE_NULLEQ)?2:3);
        goto jump_to_p2;
      }
+
      iCompare = +1;
    }else if( pIn3->u.i < pIn1->u.i ){
-
      iCompare = -1;
      if( sqlite3aLTb[pOp->opcode] ){
        VdbeBranchTaken(1, (pOp->p5 & SQLITE_NULLEQ)?2:3);
        goto jump_to_p2;
      }
+
      iCompare = -1;
    }else{
-
      iCompare = 0;
      if( sqlite3aEQb[pOp->opcode] ){
        VdbeBranchTaken(1, (pOp->p5 & SQLITE_NULLEQ)?2:3);
        goto jump_to_p2;
      }
+
      iCompare = 0;
    }
    VdbeBranchTaken(0, (pOp->p5 & SQLITE_NULLEQ)?2:3);
    break;
@@ -89531,11 +91798,11 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
      ** then the result is always NULL.
      ** The jump is taken if the SQLITE_JUMPIFNULL bit is set.
      */
-
      iCompare = 1;    /* Operands are not equal */
      VdbeBranchTaken(2,3);
      if( pOp->p5 & SQLITE_JUMPIFNULL ){
        goto jump_to_p2;
      }
+
      iCompare = 1;    /* Operands are not equal */
      break;
    }
  }else{
@@ -89641,9 +91908,8 @@ case OP_ElseEq: { /* same as TK_ESCAPE, jump */
** Set the permutation used by the OP_Compare operator in the next
** instruction.  The permutation is stored in the P4 operand.
**
-
** The permutation is only valid until the next OP_Compare that has
-
** the OPFLAG_PERMUTE bit set in P5. Typically the OP_Permutation should
-
** occur immediately prior to the OP_Compare.
+
** The permutation is only valid for the next opcode which must be
+
** an OP_Compare that has the OPFLAG_PERMUTE bit set in P5.
**
** The first integer in the P4 integer array is the length of the array
** and does not become part of the permutation.
@@ -89675,6 +91941,8 @@ case OP_Permutation: {
** The comparison is a sort comparison, so NULLs compare equal,
** NULLs are less than numbers, numbers are less than strings,
** and strings are less than blobs.
+
**
+
** This opcode must be immediately followed by an OP_Jump opcode.
*/
case OP_Compare: {
  int n;
@@ -89733,6 +92001,7 @@ case OP_Compare: {
      break;
    }
  }
+
  assert( pOp[1].opcode==OP_Jump );
  break;
}

@@ -89741,8 +92010,11 @@ case OP_Compare: {
** Jump to the instruction at address P1, P2, or P3 depending on whether
** in the most recent OP_Compare instruction the P1 vector was less than
** equal to, or greater than the P2 vector, respectively.
+
**
+
** This opcode must immediately follow an OP_Compare opcode.
*/
case OP_Jump: {             /* jump */
+
  assert( pOp>aOp && pOp[-1].opcode==OP_Compare );
  if( iCompare<0 ){
    VdbeBranchTaken(0,4); pOp = &aOp[pOp->p1 - 1];
  }else if( iCompare==0 ){
@@ -89942,19 +92214,90 @@ case OP_IsNull: { /* same as TK_ISNULL, jump, in1 */
  break;
}

-
/* Opcode: IsNullOrType P1 P2 P3 * *
-
** Synopsis: if typeof(r[P1]) IN (P3,5) goto P2
+
/* Opcode: IsType P1 P2 P3 P4 P5
+
** Synopsis: if typeof(P1.P3) in P5 goto P2
+
**
+
** Jump to P2 if the type of a column in a btree is one of the types specified
+
** by the P5 bitmask.
+
**
+
** P1 is normally a cursor on a btree for which the row decode cache is
+
** valid through at least column P3.  In other words, there should have been
+
** a prior OP_Column for column P3 or greater.  If the cursor is not valid,
+
** then this opcode might give spurious results.
+
** The the btree row has fewer than P3 columns, then use P4 as the
+
** datatype.
+
**
+
** If P1 is -1, then P3 is a register number and the datatype is taken
+
** from the value in that register.
+
**
+
** P5 is a bitmask of data types.  SQLITE_INTEGER is the least significant
+
** (0x01) bit. SQLITE_FLOAT is the 0x02 bit. SQLITE_TEXT is 0x04.
+
** SQLITE_BLOB is 0x08.  SQLITE_NULL is 0x10.
+
**
+
** Take the jump to address P2 if and only if the datatype of the
+
** value determined by P1 and P3 corresponds to one of the bits in the
+
** P5 bitmask.
**
-
** Jump to P2 if the value in register P1 is NULL or has a datatype P3.
-
** P3 is an integer which should be one of SQLITE_INTEGER, SQLITE_FLOAT,
-
** SQLITE_BLOB, SQLITE_NULL, or SQLITE_TEXT.
*/
-
case OP_IsNullOrType: {      /* jump, in1 */
-
  int doTheJump;
-
  pIn1 = &aMem[pOp->p1];
-
  doTheJump = (pIn1->flags & MEM_Null)!=0 || sqlite3_value_type(pIn1)==pOp->p3;
-
  VdbeBranchTaken( doTheJump, 2);
-
  if( doTheJump ) goto jump_to_p2;
+
case OP_IsType: {        /* jump */
+
  VdbeCursor *pC;
+
  u16 typeMask;
+
  u32 serialType;
+

+
  assert( pOp->p1>=(-1) && pOp->p1<p->nCursor );
+
  assert( pOp->p1>=0 || (pOp->p3>=0 && pOp->p3<=(p->nMem+1 - p->nCursor)) );
+
  if( pOp->p1>=0 ){
+
    pC = p->apCsr[pOp->p1];
+
    assert( pC!=0 );
+
    assert( pOp->p3>=0 );
+
    if( pOp->p3<pC->nHdrParsed ){
+
      serialType = pC->aType[pOp->p3];
+
      if( serialType>=12 ){
+
        if( serialType&1 ){
+
          typeMask = 0x04;   /* SQLITE_TEXT */
+
        }else{
+
          typeMask = 0x08;   /* SQLITE_BLOB */
+
        }
+
      }else{
+
        static const unsigned char aMask[] = {
+
           0x10, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x2,
+
           0x01, 0x01, 0x10, 0x10
+
        };
+
        testcase( serialType==0 );
+
        testcase( serialType==1 );
+
        testcase( serialType==2 );
+
        testcase( serialType==3 );
+
        testcase( serialType==4 );
+
        testcase( serialType==5 );
+
        testcase( serialType==6 );
+
        testcase( serialType==7 );
+
        testcase( serialType==8 );
+
        testcase( serialType==9 );
+
        testcase( serialType==10 );
+
        testcase( serialType==11 );
+
        typeMask = aMask[serialType];
+
      }
+
    }else{
+
      typeMask = 1 << (pOp->p4.i - 1);
+
      testcase( typeMask==0x01 );
+
      testcase( typeMask==0x02 );
+
      testcase( typeMask==0x04 );
+
      testcase( typeMask==0x08 );
+
      testcase( typeMask==0x10 );
+
    }
+
  }else{
+
    assert( memIsValid(&aMem[pOp->p3]) );
+
    typeMask = 1 << (sqlite3_value_type((sqlite3_value*)&aMem[pOp->p3])-1);
+
    testcase( typeMask==0x01 );
+
    testcase( typeMask==0x02 );
+
    testcase( typeMask==0x04 );
+
    testcase( typeMask==0x08 );
+
    testcase( typeMask==0x10 );
+
  }
+
  VdbeBranchTaken( (typeMask & pOp->p5)!=0, 2);
+
  if( typeMask & pOp->p5 ){
+
    goto jump_to_p2;
+
  }
  break;
}

@@ -89997,11 +92340,14 @@ case OP_NotNull: { /* same as TK_NOTNULL, jump, in1 */
** If it is, then set register P3 to NULL and jump immediately to P2.
** If P1 is not on a NULL row, then fall through without making any
** changes.
+
**
+
** If P1 is not an open cursor, then this opcode is a no-op.
*/
case OP_IfNullRow: {         /* jump */
+
  VdbeCursor *pC;
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
-
  assert( p->apCsr[pOp->p1]!=0 );
-
  if( p->apCsr[pOp->p1]->nullRow ){
+
  pC = p->apCsr[pOp->p1];
+
  if( ALWAYS(pC) && pC->nullRow ){
    sqlite3VdbeMemSetNull(aMem + pOp->p3);
    goto jump_to_p2;
  }
@@ -90047,12 +92393,12 @@ case OP_Offset: { /* out3 */
#endif /* SQLITE_ENABLE_OFFSET_SQL_FUNC */

/* Opcode: Column P1 P2 P3 P4 P5
-
** Synopsis: r[P3]=PX
+
** Synopsis: r[P3]=PX cursor P1 column P2
**
** Interpret the data that cursor P1 points to as a structure built using
** the MakeRecord instruction.  (See the MakeRecord opcode for additional
** information about the format of the data.)  Extract the P2-th column
-
** from this record.  If there are less that (P2+1)
+
** from this record.  If there are less than (P2+1)
** values in the record, extract a NULL.
**
** The value extracted is stored in register P3.
@@ -90061,15 +92407,17 @@ case OP_Offset: { /* out3 */
** if the P4 argument is a P4_MEM use the value of the P4 argument as
** the result.
**
-
** If the OPFLAG_LENGTHARG and OPFLAG_TYPEOFARG bits are set on P5 then
-
** the result is guaranteed to only be used as the argument of a length()
-
** or typeof() function, respectively.  The loading of large blobs can be
-
** skipped for length() and all content loading can be skipped for typeof().
+
** If the OPFLAG_LENGTHARG bit is set in P5 then the result is guaranteed
+
** to only be used by the length() function or the equivalent.  The content
+
** of large blobs is not loaded, thus saving CPU cycles.  If the
+
** OPFLAG_TYPEOFARG bit is set then the result will only be used by the
+
** typeof() function or the IS NULL or IS NOT NULL operators or the
+
** equivalent.  In this case, all content loading can be omitted.
*/
case OP_Column: {
  u32 p2;            /* column number to retrieve */
  VdbeCursor *pC;    /* The VDBE cursor */
-
  BtCursor *pCrsr;   /* The BTree cursor */
+
  BtCursor *pCrsr;   /* The B-Tree cursor corresponding to pC */
  u32 *aOffset;      /* aOffset[i] is offset to start of data for i-th column */
  int len;           /* The length of the serialized data for the column */
  int i;             /* Loop counter */
@@ -90083,21 +92431,14 @@ case OP_Column: {
  Mem *pReg;         /* PseudoTable input register */

  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
+
  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pC = p->apCsr[pOp->p1];
-
  assert( pC!=0 );
  p2 = (u32)pOp->p2;

-
  /* If the cursor cache is stale (meaning it is not currently point at
-
  ** the correct row) then bring it up-to-date by doing the necessary
-
  ** B-Tree seek. */
-
  rc = sqlite3VdbeCursorMoveto(&pC, &p2);
-
  if( rc ) goto abort_due_to_error;
-

-
  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
-
  pDest = &aMem[pOp->p3];
-
  memAboutToChange(p, pDest);
+
op_column_restart:
  assert( pC!=0 );
-
  assert( p2<(u32)pC->nField );
+
  assert( p2<(u32)pC->nField
+
       || (pC->eCurType==CURTYPE_PSEUDO && pC->seekResult==0) );
  aOffset = pC->aOffset;
  assert( aOffset==pC->aType+pC->nField );
  assert( pC->eCurType!=CURTYPE_VTAB );
@@ -90106,21 +92447,37 @@ case OP_Column: {

  if( pC->cacheStatus!=p->cacheCtr ){                /*OPTIMIZATION-IF-FALSE*/
    if( pC->nullRow ){
-
      if( pC->eCurType==CURTYPE_PSEUDO ){
+
      if( pC->eCurType==CURTYPE_PSEUDO && pC->seekResult>0 ){
        /* For the special case of as pseudo-cursor, the seekResult field
        ** identifies the register that holds the record */
-
        assert( pC->seekResult>0 );
        pReg = &aMem[pC->seekResult];
        assert( pReg->flags & MEM_Blob );
        assert( memIsValid(pReg) );
        pC->payloadSize = pC->szRow = pReg->n;
        pC->aRow = (u8*)pReg->z;
      }else{
+
        pDest = &aMem[pOp->p3];
+
        memAboutToChange(p, pDest);
        sqlite3VdbeMemSetNull(pDest);
        goto op_column_out;
      }
    }else{
      pCrsr = pC->uc.pCursor;
+
      if( pC->deferredMoveto ){
+
        u32 iMap;
+
        assert( !pC->isEphemeral );
+
        if( pC->ub.aAltMap && (iMap = pC->ub.aAltMap[1+p2])>0  ){
+
          pC = pC->pAltCursor;
+
          p2 = iMap - 1;
+
          goto op_column_restart;
+
        }
+
        rc = sqlite3VdbeFinishMoveto(pC);
+
        if( rc ) goto abort_due_to_error;
+
      }else if( sqlite3BtreeCursorHasMoved(pCrsr) ){
+
        rc = sqlite3VdbeHandleMovedCursor(pC);
+
        if( rc ) goto abort_due_to_error;
+
        goto op_column_restart;
+
      }
      assert( pC->eCurType==CURTYPE_BTREE );
      assert( pCrsr );
      assert( sqlite3BtreeCursorIsValid(pCrsr) );
@@ -90128,15 +92485,15 @@ case OP_Column: {
      pC->aRow = sqlite3BtreePayloadFetch(pCrsr, &pC->szRow);
      assert( pC->szRow<=pC->payloadSize );
      assert( pC->szRow<=65536 );  /* Maximum page size is 64KiB */
-
      if( pC->payloadSize > (u32)db->aLimit[SQLITE_LIMIT_LENGTH] ){
-
        goto too_big;
-
      }
    }
    pC->cacheStatus = p->cacheCtr;
-
    pC->iHdrOffset = getVarint32(pC->aRow, aOffset[0]);
+
    if( (aOffset[0] = pC->aRow[0])<0x80 ){
+
      pC->iHdrOffset = 1;
+
    }else{
+
      pC->iHdrOffset = sqlite3GetVarint32(pC->aRow, aOffset);
+
    }
    pC->nHdrParsed = 0;

-

    if( pC->szRow<aOffset[0] ){      /*OPTIMIZATION-IF-FALSE*/
      /* pC->aRow does not have to hold the entire row, but it does at least
      ** need to cover the header of the record.  If pC->aRow does not contain
@@ -90176,6 +92533,10 @@ case OP_Column: {
      testcase( aOffset[0]==0 );
      goto op_column_read_header;
    }
+
  }else if( sqlite3BtreeCursorHasMoved(pC->uc.pCursor) ){
+
    rc = sqlite3VdbeHandleMovedCursor(pC);
+
    if( rc ) goto abort_due_to_error;
+
    goto op_column_restart;
  }

  /* Make sure at least the first p2+1 entries of the header have been
@@ -90244,6 +92605,8 @@ case OP_Column: {
    ** columns.  So the result will be either the default value or a NULL.
    */
    if( pC->nHdrParsed<=p2 ){
+
      pDest = &aMem[pOp->p3];
+
      memAboutToChange(p, pDest);
      if( pOp->p4type==P4_MEM ){
        sqlite3VdbeMemShallowCopy(pDest, pOp->p4.pMem, MEM_Static);
      }else{
@@ -90261,6 +92624,8 @@ case OP_Column: {
  */
  assert( p2<pC->nHdrParsed );
  assert( rc==SQLITE_OK );
+
  pDest = &aMem[pOp->p3];
+
  memAboutToChange(p, pDest);
  assert( sqlite3VdbeCheckMemInvariants(pDest) );
  if( VdbeMemDynamic(pDest) ){
    sqlite3VdbeMemSetNull(pDest);
@@ -90281,6 +92646,7 @@ case OP_Column: {
      pDest->n = len = (t-12)/2;
      pDest->enc = encoding;
      if( pDest->szMalloc < len+2 ){
+
        if( len>db->aLimit[SQLITE_LIMIT_LENGTH] ) goto too_big;
        pDest->flags = MEM_Null;
        if( sqlite3VdbeMemGrow(pDest, len+2, 0) ) goto no_mem;
      }else{
@@ -90313,6 +92679,7 @@ case OP_Column: {
      */
      sqlite3VdbeSerialGet((u8*)sqlite3CtypeMap, t, pDest);
    }else{
+
      if( len>db->aLimit[SQLITE_LIMIT_LENGTH] ) goto too_big;
      rc = sqlite3VdbeMemFromBtree(pC->uc.pCursor, aOffset[p2], len, pDest);
      if( rc!=SQLITE_OK ) goto abort_due_to_error;
      sqlite3VdbeSerialGet((const u8*)pDest->z, t, pDest);
@@ -90525,7 +92892,6 @@ case OP_MakeRecord: {
  Mem *pLast;            /* Last field of the record */
  int nField;            /* Number of fields in the record */
  char *zAffinity;       /* The affinity string for the record */
-
  int file_format;       /* File format to use for encoding */
  u32 len;               /* Length of a field */
  u8 *zHdr;              /* Where to write next byte of the header */
  u8 *zPayload;          /* Where to write next byte of the payload */
@@ -90554,7 +92920,6 @@ case OP_MakeRecord: {
  pData0 = &aMem[nField];
  nField = pOp->p2;
  pLast = &pData0[nField-1];
-
  file_format = p->minWriteFileFormat;

  /* Identify the output register */
  assert( pOp->p3<pOp->p1 || pOp->p3>=pOp->p1+pOp->p2 );
@@ -90656,7 +93021,7 @@ case OP_MakeRecord: {
      testcase( uu==2147483647 );        testcase( uu==2147483648LL );
      testcase( uu==140737488355327LL ); testcase( uu==140737488355328LL );
      if( uu<=127 ){
-
        if( (i&1)==i && file_format>=4 ){
+
        if( (i&1)==i && p->minWriteFileFormat>=4 ){
          pRec->uTemp = 8+(u32)uu;
        }else{
          nData++;
@@ -90761,18 +93126,60 @@ case OP_MakeRecord: {
  zPayload = zHdr + nHdr;

  /* Write the record */
-
  zHdr += putVarint32(zHdr, nHdr);
+
  if( nHdr<0x80 ){
+
    *(zHdr++) = nHdr;
+
  }else{
+
    zHdr += sqlite3PutVarint(zHdr,nHdr);
+
  }
  assert( pData0<=pLast );
  pRec = pData0;
-
  do{
+
  while( 1 /*exit-by-break*/ ){
    serial_type = pRec->uTemp;
    /* EVIDENCE-OF: R-06529-47362 Following the size varint are one or more
-
    ** additional varints, one per column. */
-
    zHdr += putVarint32(zHdr, serial_type);            /* serial type */
-
    /* EVIDENCE-OF: R-64536-51728 The values for each column in the record
+
    ** additional varints, one per column.
+
    ** EVIDENCE-OF: R-64536-51728 The values for each column in the record
    ** immediately follow the header. */
-
    zPayload += sqlite3VdbeSerialPut(zPayload, pRec, serial_type); /* content */
-
  }while( (++pRec)<=pLast );
+
    if( serial_type<=7 ){
+
      *(zHdr++) = serial_type;
+
      if( serial_type==0 ){
+
        /* NULL value.  No change in zPayload */
+
      }else{
+
        u64 v;
+
        u32 i;
+
        if( serial_type==7 ){
+
          assert( sizeof(v)==sizeof(pRec->u.r) );
+
          memcpy(&v, &pRec->u.r, sizeof(v));
+
          swapMixedEndianFloat(v);
+
        }else{
+
          v = pRec->u.i;
+
        }
+
        len = i = sqlite3SmallTypeSizes[serial_type];
+
        assert( i>0 );
+
        while( 1 /*exit-by-break*/ ){
+
          zPayload[--i] = (u8)(v&0xFF);
+
          if( i==0 ) break;
+
          v >>= 8;
+
        }
+
        zPayload += len;
+
      }
+
    }else if( serial_type<0x80 ){
+
      *(zHdr++) = serial_type;
+
      if( serial_type>=14 && pRec->n>0 ){
+
        assert( pRec->z!=0 );
+
        memcpy(zPayload, pRec->z, pRec->n);
+
        zPayload += pRec->n;
+
      }
+
    }else{
+
      zHdr += sqlite3PutVarint(zHdr, serial_type);
+
      if( pRec->n ){
+
        assert( pRec->z!=0 );
+
        memcpy(zPayload, pRec->z, pRec->n);
+
        zPayload += pRec->n;
+
      }
+
    }
+
    if( pRec==pLast ) break;
+
    pRec++;
+
  }
  assert( nHdr==(int)(zHdr - (u8*)pOut->z) );
  assert( nByte==(int)(zPayload - (u8*)pOut->z) );

@@ -90991,7 +93398,10 @@ case OP_Savepoint: {
    }
  }
  if( rc ) goto abort_due_to_error;
-

+
  if( p->eVdbeState==VDBE_HALT_STATE ){
+
    rc = SQLITE_DONE;
+
    goto vdbe_return;
+
  }
  break;
}

@@ -91095,6 +93505,7 @@ case OP_AutoCommit: {
*/
case OP_Transaction: {
  Btree *pBt;
+
  Db *pDb;
  int iMeta = 0;

  assert( p->bIsReader );
@@ -91114,7 +93525,8 @@ case OP_Transaction: {
    }
    goto abort_due_to_error;
  }
-
  pBt = db->aDb[pOp->p1].pBt;
+
  pDb = &db->aDb[pOp->p1];
+
  pBt = pDb->pBt;

  if( pBt ){
    rc = sqlite3BtreeBeginTrans(pBt, pOp->p2, &iMeta);
@@ -91155,8 +93567,7 @@ case OP_Transaction: {
  assert( pOp->p5==0 || pOp->p4type==P4_INT32 );
  if( rc==SQLITE_OK
   && pOp->p5
-
   && (iMeta!=pOp->p3
-
      || db->aDb[pOp->p1].pSchema->iGeneration!=pOp->p4.i)
+
   && (iMeta!=pOp->p3 || pDb->pSchema->iGeneration!=pOp->p4.i)
  ){
    /*
    ** IMPLEMENTATION-OF: R-03189-51135 As each SQL statement runs, the schema
@@ -91183,6 +93594,11 @@ case OP_Transaction: {
    }
    p->expired = 1;
    rc = SQLITE_SCHEMA;
+

+
    /* Set changeCntOn to 0 to prevent the value returned by sqlite3_changes()
+
    ** from being modified in sqlite3VdbeHalt(). If this statement is
+
    ** reprepared, changeCntOn will be set again. */
+
    p->changeCntOn = 0;
  }
  if( rc ) goto abort_due_to_error;
  break;
@@ -91249,7 +93665,7 @@ case OP_SetCookie: {
  rc = sqlite3BtreeUpdateMeta(pDb->pBt, pOp->p2, pOp->p3);
  if( pOp->p2==BTREE_SCHEMA_VERSION ){
    /* When the schema cookie changes, record the new cookie internally */
-
    pDb->pSchema->schema_cookie = pOp->p3 - pOp->p5;
+
    *(u32*)&pDb->pSchema->schema_cookie = *(u32*)&pOp->p3 - pOp->p5;
    db->mDbFlags |= DBFLAG_SchemaChange;
    sqlite3FkClearTriggerCache(db, pOp->p1);
  }else if( pOp->p2==BTREE_FILE_FORMAT ){
@@ -91482,8 +93898,8 @@ case OP_OpenDup: {
  pCx->pgnoRoot = pOrig->pgnoRoot;
  pCx->isOrdered = pOrig->isOrdered;
  pCx->ub.pBtx = pOrig->ub.pBtx;
-
  pCx->hasBeenDuped = 1;
-
  pOrig->hasBeenDuped = 1;
+
  pCx->noReuse = 1;
+
  pOrig->noReuse = 1;
  rc = sqlite3BtreeCursor(pCx->ub.pBtx, pCx->pgnoRoot, BTREE_WRCSR,
                          pCx->pKeyInfo, pCx->uc.pCursor);
  /* The sqlite3BtreeCursor() routine can only fail for the first cursor
@@ -91550,7 +93966,7 @@ case OP_OpenEphemeral: {
    aMem[pOp->p3].z = "";
  }
  pCx = p->apCsr[pOp->p1];
-
  if( pCx && !pCx->hasBeenDuped &&  ALWAYS(pOp->p2<=pCx->nField) ){
+
  if( pCx && !pCx->noReuse &&  ALWAYS(pOp->p2<=pCx->nField) ){
    /* If the ephermeral table is already open and has no duplicates from
    ** OP_OpenDup, then erase all existing content so that the table is
    ** empty again, rather than creating a new table. */
@@ -91935,7 +94351,13 @@ case OP_SeekGT: { /* jump, in3, group */

    r.aMem = &aMem[pOp->p3];
#ifdef SQLITE_DEBUG
-
    { int i; for(i=0; i<r.nField; i++) assert( memIsValid(&r.aMem[i]) ); }
+
    {
+
      int i;
+
      for(i=0; i<r.nField; i++){
+
        assert( memIsValid(&r.aMem[i]) );
+
        if( i>0 ) REGISTER_TRACE(pOp->p3+i, &r.aMem[i]);
+
      }
+
    }
#endif
    r.eqSeen = 0;
    rc = sqlite3BtreeIndexMoveto(pC->uc.pCursor, &r, &res);
@@ -91998,7 +94420,7 @@ seek_not_found:
}


-
/* Opcode: SeekScan  P1 P2 * * *
+
/* Opcode: SeekScan  P1 P2 * * P5
** Synopsis: Scan-ahead up to P1 rows
**
** This opcode is a prefix opcode to OP_SeekGE.  In other words, this
@@ -92008,8 +94430,8 @@ seek_not_found:
** This opcode uses the P1 through P4 operands of the subsequent
** OP_SeekGE.  In the text that follows, the operands of the subsequent
** OP_SeekGE opcode are denoted as SeekOP.P1 through SeekOP.P4.   Only
-
** the P1 and P2 operands of this opcode are also used, and  are called
-
** This.P1 and This.P2.
+
** the P1, P2 and P5 operands of this opcode are also used, and  are called
+
** This.P1, This.P2 and This.P5.
**
** This opcode helps to optimize IN operators on a multi-column index
** where the IN operator is on the later terms of the index by avoiding
@@ -92019,29 +94441,51 @@ seek_not_found:
**
** The SeekGE.P3 and SeekGE.P4 operands identify an unpacked key which
** is the desired entry that we want the cursor SeekGE.P1 to be pointing
-
** to.  Call this SeekGE.P4/P5 row the "target".
+
** to.  Call this SeekGE.P3/P4 row the "target".
**
** If the SeekGE.P1 cursor is not currently pointing to a valid row,
** then this opcode is a no-op and control passes through into the OP_SeekGE.
**
** If the SeekGE.P1 cursor is pointing to a valid row, then that row
** might be the target row, or it might be near and slightly before the
-
** target row.  This opcode attempts to position the cursor on the target
-
** row by, perhaps by invoking sqlite3BtreeStep() on the cursor
-
** between 0 and This.P1 times.
-
**
-
** There are three possible outcomes from this opcode:<ol>
-
**
-
** <li> If after This.P1 steps, the cursor is still pointing to a place that
-
**      is earlier in the btree than the target row, then fall through
-
**      into the subsquence OP_SeekGE opcode.
-
**
-
** <li> If the cursor is successfully moved to the target row by 0 or more
-
**      sqlite3BtreeNext() calls, then jump to This.P2, which will land just
-
**      past the OP_IdxGT or OP_IdxGE opcode that follows the OP_SeekGE.
-
**
-
** <li> If the cursor ends up past the target row (indicating the the target
-
**      row does not exist in the btree) then jump to SeekOP.P2.
+
** target row, or it might be after the target row.  If the cursor is
+
** currently before the target row, then this opcode attempts to position
+
** the cursor on or after the target row by invoking sqlite3BtreeStep()
+
** on the cursor between 1 and This.P1 times.
+
**
+
** The This.P5 parameter is a flag that indicates what to do if the
+
** cursor ends up pointing at a valid row that is past the target
+
** row.  If This.P5 is false (0) then a jump is made to SeekGE.P2.  If
+
** This.P5 is true (non-zero) then a jump is made to This.P2.  The P5==0
+
** case occurs when there are no inequality constraints to the right of
+
** the IN constraing.  The jump to SeekGE.P2 ends the loop.  The P5!=0 case
+
** occurs when there are inequality constraints to the right of the IN
+
** operator.  In that case, the This.P2 will point either directly to or
+
** to setup code prior to the OP_IdxGT or OP_IdxGE opcode that checks for
+
** loop terminate.
+
**
+
** Possible outcomes from this opcode:<ol>
+
**
+
** <li> If the cursor is initally not pointed to any valid row, then
+
**      fall through into the subsequent OP_SeekGE opcode.
+
**
+
** <li> If the cursor is left pointing to a row that is before the target
+
**      row, even after making as many as This.P1 calls to
+
**      sqlite3BtreeNext(), then also fall through into OP_SeekGE.
+
**
+
** <li> If the cursor is left pointing at the target row, either because it
+
**      was at the target row to begin with or because one or more
+
**      sqlite3BtreeNext() calls moved the cursor to the target row,
+
**      then jump to This.P2..,
+
**
+
** <li> If the cursor started out before the target row and a call to
+
**      to sqlite3BtreeNext() moved the cursor off the end of the index
+
**      (indicating that the target row definitely does not exist in the
+
**      btree) then jump to SeekGE.P2, ending the loop.
+
**
+
** <li> If the cursor ends up on a valid row that is past the target row
+
**      (indicating that the target row does not exist in the btree) then
+
**      jump to SeekOP.P2 if This.P5==0 or to This.P2 if This.P5>0.
** </ol>
*/
case OP_SeekScan: {
@@ -92052,14 +94496,25 @@ case OP_SeekScan: {

  assert( pOp[1].opcode==OP_SeekGE );

-
  /* pOp->p2 points to the first instruction past the OP_IdxGT that
-
  ** follows the OP_SeekGE.  */
+
  /* If pOp->p5 is clear, then pOp->p2 points to the first instruction past the
+
  ** OP_IdxGT that follows the OP_SeekGE. Otherwise, it points to the first
+
  ** opcode past the OP_SeekGE itself.  */
  assert( pOp->p2>=(int)(pOp-aOp)+2 );
-
  assert( aOp[pOp->p2-1].opcode==OP_IdxGT || aOp[pOp->p2-1].opcode==OP_IdxGE );
-
  testcase( aOp[pOp->p2-1].opcode==OP_IdxGE );
-
  assert( pOp[1].p1==aOp[pOp->p2-1].p1 );
-
  assert( pOp[1].p2==aOp[pOp->p2-1].p2 );
-
  assert( pOp[1].p3==aOp[pOp->p2-1].p3 );
+
#ifdef SQLITE_DEBUG
+
  if( pOp->p5==0 ){
+
    /* There are no inequality constraints following the IN constraint. */
+
    assert( pOp[1].p1==aOp[pOp->p2-1].p1 );
+
    assert( pOp[1].p2==aOp[pOp->p2-1].p2 );
+
    assert( pOp[1].p3==aOp[pOp->p2-1].p3 );
+
    assert( aOp[pOp->p2-1].opcode==OP_IdxGT
+
         || aOp[pOp->p2-1].opcode==OP_IdxGE );
+
    testcase( aOp[pOp->p2-1].opcode==OP_IdxGE );
+
  }else{
+
    /* There are inequality constraints.  */
+
    assert( pOp->p2==(int)(pOp-aOp)+2 );
+
    assert( aOp[pOp->p2-1].opcode==OP_SeekGE );
+
  }
+
#endif

  assert( pOp->p1>0 );
  pC = p->apCsr[pOp[1].p1];
@@ -92093,8 +94548,9 @@ case OP_SeekScan: {
  while(1){
    rc = sqlite3VdbeIdxKeyCompare(db, pC, &r, &res);
    if( rc ) goto abort_due_to_error;
-
    if( res>0 ){
+
    if( res>0 && pOp->p5==0 ){
      seekscan_search_fail:
+
      /* Jump to SeekGE.P2, ending the loop */
#ifdef SQLITE_DEBUG
      if( db->flags&SQLITE_VdbeTrace ){
        printf("... %d steps and then skip\n", pOp->p1 - nStep);
@@ -92104,7 +94560,8 @@ case OP_SeekScan: {
      pOp++;
      goto jump_to_p2;
    }
-
    if( res==0 ){
+
    if( res>=0 ){
+
      /* Jump to This.P2, bypassing the OP_SeekGE opcode */
#ifdef SQLITE_DEBUG
      if( db->flags&SQLITE_VdbeTrace ){
        printf("... %d steps and then success\n", pOp->p1 - nStep);
@@ -92180,12 +94637,16 @@ case OP_SeekHit: {
/* Opcode: IfNotOpen P1 P2 * * *
** Synopsis: if( !csr[P1] ) goto P2
**
-
** If cursor P1 is not open, jump to instruction P2. Otherwise, fall through.
+
** If cursor P1 is not open or if P1 is set to a NULL row using the
+
** OP_NullRow opcode, then jump to instruction P2. Otherwise, fall through.
*/
case OP_IfNotOpen: {        /* jump */
+
  VdbeCursor *pCur;
+

  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
-
  VdbeBranchTaken(p->apCsr[pOp->p1]==0, 2);
-
  if( !p->apCsr[pOp->p1] ){
+
  pCur = p->apCsr[pOp->p1];
+
  VdbeBranchTaken(pCur==0 || pCur->nullRow, 2);
+
  if( pCur==0 || pCur->nullRow ){
    goto jump_to_p2_and_check_for_interrupt;
  }
  break;
@@ -92299,11 +94760,8 @@ case OP_NoConflict: /* jump, in3 */
case OP_NotFound:       /* jump, in3 */
case OP_Found: {        /* jump, in3 */
  int alreadyExists;
-
  int takeJump;
  int ii;
  VdbeCursor *pC;
-
  int res;
-
  UnpackedRecord *pFree;
  UnpackedRecord *pIdxKey;
  UnpackedRecord r;

@@ -92318,14 +94776,15 @@ case OP_Found: { /* jump, in3 */
#ifdef SQLITE_DEBUG
  pC->seekOp = pOp->opcode;
#endif
-
  pIn3 = &aMem[pOp->p3];
+
  r.aMem = &aMem[pOp->p3];
  assert( pC->eCurType==CURTYPE_BTREE );
  assert( pC->uc.pCursor!=0 );
  assert( pC->isTable==0 );
-
  if( pOp->p4.i>0 ){
+
  r.nField = (u16)pOp->p4.i;
+
  if( r.nField>0 ){
+
    /* Key values in an array of registers */
    r.pKeyInfo = pC->pKeyInfo;
-
    r.nField = (u16)pOp->p4.i;
-
    r.aMem = pIn3;
+
    r.default_rc = 0;
#ifdef SQLITE_DEBUG
    for(ii=0; ii<r.nField; ii++){
      assert( memIsValid(&r.aMem[ii]) );
@@ -92333,37 +94792,25 @@ case OP_Found: { /* jump, in3 */
      if( ii ) REGISTER_TRACE(pOp->p3+ii, &r.aMem[ii]);
    }
#endif
-
    pIdxKey = &r;
-
    pFree = 0;
+
    rc = sqlite3BtreeIndexMoveto(pC->uc.pCursor, &r, &pC->seekResult);
  }else{
-
    assert( pIn3->flags & MEM_Blob );
-
    rc = ExpandBlob(pIn3);
+
    /* Composite key generated by OP_MakeRecord */
+
    assert( r.aMem->flags & MEM_Blob );
+
    assert( pOp->opcode!=OP_NoConflict );
+
    rc = ExpandBlob(r.aMem);
    assert( rc==SQLITE_OK || rc==SQLITE_NOMEM );
    if( rc ) goto no_mem;
-
    pFree = pIdxKey = sqlite3VdbeAllocUnpackedRecord(pC->pKeyInfo);
+
    pIdxKey = sqlite3VdbeAllocUnpackedRecord(pC->pKeyInfo);
    if( pIdxKey==0 ) goto no_mem;
-
    sqlite3VdbeRecordUnpack(pC->pKeyInfo, pIn3->n, pIn3->z, pIdxKey);
-
  }
-
  pIdxKey->default_rc = 0;
-
  takeJump = 0;
-
  if( pOp->opcode==OP_NoConflict ){
-
    /* For the OP_NoConflict opcode, take the jump if any of the
-
    ** input fields are NULL, since any key with a NULL will not
-
    ** conflict */
-
    for(ii=0; ii<pIdxKey->nField; ii++){
-
      if( pIdxKey->aMem[ii].flags & MEM_Null ){
-
        takeJump = 1;
-
        break;
-
      }
-
    }
+
    sqlite3VdbeRecordUnpack(pC->pKeyInfo, r.aMem->n, r.aMem->z, pIdxKey);
+
    pIdxKey->default_rc = 0;
+
    rc = sqlite3BtreeIndexMoveto(pC->uc.pCursor, pIdxKey, &pC->seekResult);
+
    sqlite3DbFreeNN(db, pIdxKey);
  }
-
  rc = sqlite3BtreeIndexMoveto(pC->uc.pCursor, pIdxKey, &res);
-
  if( pFree ) sqlite3DbFreeNN(db, pFree);
  if( rc!=SQLITE_OK ){
    goto abort_due_to_error;
  }
-
  pC->seekResult = res;
-
  alreadyExists = (res==0);
+
  alreadyExists = (pC->seekResult==0);
  pC->nullRow = 1-alreadyExists;
  pC->deferredMoveto = 0;
  pC->cacheStatus = CACHE_STALE;
@@ -92371,9 +94818,25 @@ case OP_Found: { /* jump, in3 */
    VdbeBranchTaken(alreadyExists!=0,2);
    if( alreadyExists ) goto jump_to_p2;
  }else{
-
    VdbeBranchTaken(takeJump||alreadyExists==0,2);
-
    if( takeJump || !alreadyExists ) goto jump_to_p2;
-
    if( pOp->opcode==OP_IfNoHope ) pC->seekHit = pOp->p4.i;
+
    if( !alreadyExists ){
+
      VdbeBranchTaken(1,2);
+
      goto jump_to_p2;
+
    }
+
    if( pOp->opcode==OP_NoConflict ){
+
      /* For the OP_NoConflict opcode, take the jump if any of the
+
      ** input fields are NULL, since any key with a NULL will not
+
      ** conflict */
+
      for(ii=0; ii<r.nField; ii++){
+
        if( r.aMem[ii].flags & MEM_Null ){
+
          VdbeBranchTaken(1,2);
+
          goto jump_to_p2;
+
        }
+
      }
+
    }
+
    VdbeBranchTaken(0,2);
+
    if( pOp->opcode==OP_IfNoHope ){
+
      pC->seekHit = pOp->p4.i;
+
    }
  }
  break;
}
@@ -93064,7 +95527,7 @@ case OP_RowData: {
}

/* Opcode: Rowid P1 P2 * * *
-
** Synopsis: r[P2]=rowid
+
** Synopsis: r[P2]=PX rowid of P1
**
** Store in register P2 an integer which is the key of the table entry that
** P1 is currently point to.
@@ -93120,16 +95583,24 @@ case OP_Rowid: { /* out2 */
** that occur while the cursor is on the null row will always
** write a NULL.
**
-
** Or, if P1 is a Pseudo-Cursor (a cursor opened using OP_OpenPseudo)
-
** just reset the cache for that cursor.  This causes the row of
-
** content held by the pseudo-cursor to be reparsed.
+
** If cursor P1 is not previously opened, open it now to a special
+
** pseudo-cursor that always returns NULL for every column.
*/
case OP_NullRow: {
  VdbeCursor *pC;

  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  pC = p->apCsr[pOp->p1];
-
  assert( pC!=0 );
+
  if( pC==0 ){
+
    /* If the cursor is not already open, create a special kind of
+
    ** pseudo-cursor that always gives null rows. */
+
    pC = allocateCursor(p, pOp->p1, 1, CURTYPE_PSEUDO);
+
    if( pC==0 ) goto no_mem;
+
    pC->seekResult = 0;
+
    pC->isTable = 1;
+
    pC->noReuse = 1;
+
    pC->uc.pCursor = sqlite3BtreeFakeValidCursor();
+
  }
  pC->nullRow = 1;
  pC->cacheStatus = CACHE_STALE;
  if( pC->eCurType==CURTYPE_BTREE ){
@@ -93302,7 +95773,7 @@ case OP_Rewind: { /* jump */
  break;
}

-
/* Opcode: Next P1 P2 P3 P4 P5
+
/* Opcode: Next P1 P2 P3 * P5
**
** Advance cursor P1 so that it points to the next key/data pair in its
** table or index.  If there are no more key/value pairs then fall through
@@ -93321,15 +95792,12 @@ case OP_Rewind: { /* jump */
** omitted if that index had been unique.  P3 is usually 0.  P3 is
** always either 0 or 1.
**
-
** P4 is always of type P4_ADVANCE. The function pointer points to
-
** sqlite3BtreeNext().
-
**
** If P5 is positive and the jump is taken, then event counter
** number P5-1 in the prepared statement is incremented.
**
** See also: Prev
*/
-
/* Opcode: Prev P1 P2 P3 P4 P5
+
/* Opcode: Prev P1 P2 P3 * P5
**
** Back up cursor P1 so that it points to the previous key/data pair in its
** table or index.  If there is no previous key/value pairs then fall through
@@ -93349,9 +95817,6 @@ case OP_Rewind: { /* jump */
** omitted if that index had been unique.  P3 is usually 0.  P3 is
** always either 0 or 1.
**
-
** P4 is always of type P4_ADVANCE. The function pointer points to
-
** sqlite3BtreePrevious().
-
**
** If P5 is positive and the jump is taken, then event counter
** number P5-1 in the prepared statement is incremented.
*/
@@ -93369,30 +95834,37 @@ case OP_SorterNext: { /* jump */
  assert( isSorter(pC) );
  rc = sqlite3VdbeSorterNext(db, pC);
  goto next_tail;
+

case OP_Prev:          /* jump */
-
case OP_Next:          /* jump */
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
-
  assert( pOp->p5<ArraySize(p->aCounter) );
+
  assert( pOp->p5==0
+
       || pOp->p5==SQLITE_STMTSTATUS_FULLSCAN_STEP
+
       || pOp->p5==SQLITE_STMTSTATUS_AUTOINDEX);
  pC = p->apCsr[pOp->p1];
  assert( pC!=0 );
  assert( pC->deferredMoveto==0 );
  assert( pC->eCurType==CURTYPE_BTREE );
-
  assert( pOp->opcode!=OP_Next || pOp->p4.xAdvance==sqlite3BtreeNext );
-
  assert( pOp->opcode!=OP_Prev || pOp->p4.xAdvance==sqlite3BtreePrevious );
+
  assert( pC->seekOp==OP_SeekLT || pC->seekOp==OP_SeekLE
+
       || pC->seekOp==OP_Last   || pC->seekOp==OP_IfNoHope
+
       || pC->seekOp==OP_NullRow);
+
  rc = sqlite3BtreePrevious(pC->uc.pCursor, pOp->p3);
+
  goto next_tail;

-
  /* The Next opcode is only used after SeekGT, SeekGE, Rewind, and Found.
-
  ** The Prev opcode is only used after SeekLT, SeekLE, and Last. */
-
  assert( pOp->opcode!=OP_Next
-
       || pC->seekOp==OP_SeekGT || pC->seekOp==OP_SeekGE
+
case OP_Next:          /* jump */
+
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
+
  assert( pOp->p5==0
+
       || pOp->p5==SQLITE_STMTSTATUS_FULLSCAN_STEP
+
       || pOp->p5==SQLITE_STMTSTATUS_AUTOINDEX);
+
  pC = p->apCsr[pOp->p1];
+
  assert( pC!=0 );
+
  assert( pC->deferredMoveto==0 );
+
  assert( pC->eCurType==CURTYPE_BTREE );
+
  assert( pC->seekOp==OP_SeekGT || pC->seekOp==OP_SeekGE
       || pC->seekOp==OP_Rewind || pC->seekOp==OP_Found
       || pC->seekOp==OP_NullRow|| pC->seekOp==OP_SeekRowid
       || pC->seekOp==OP_IfNoHope);
-
  assert( pOp->opcode!=OP_Prev
-
       || pC->seekOp==OP_SeekLT || pC->seekOp==OP_SeekLE
-
       || pC->seekOp==OP_Last   || pC->seekOp==OP_IfNoHope
-
       || pC->seekOp==OP_NullRow);
+
  rc = sqlite3BtreeNext(pC->uc.pCursor, pOp->p3);

-
  rc = pOp->p4.xAdvance(pC->uc.pCursor, pOp->p3);
next_tail:
  pC->cacheStatus = CACHE_STALE;
  VdbeBranchTaken(rc==SQLITE_OK,2);
@@ -93579,9 +96051,9 @@ case OP_IdxRowid: { /* out2 */
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  pC = p->apCsr[pOp->p1];
  assert( pC!=0 );
-
  assert( pC->eCurType==CURTYPE_BTREE );
+
  assert( pC->eCurType==CURTYPE_BTREE || IsNullCursor(pC) );
  assert( pC->uc.pCursor!=0 );
-
  assert( pC->isTable==0 );
+
  assert( pC->isTable==0 || IsNullCursor(pC) );
  assert( pC->deferredMoveto==0 );
  assert( !pC->nullRow || pOp->opcode==OP_IdxRowid );

@@ -93589,10 +96061,10 @@ case OP_IdxRowid: { /* out2 */
  ** of sqlite3VdbeCursorRestore() and sqlite3VdbeIdxRowid(). */
  rc = sqlite3VdbeCursorRestore(pC);

-
  /* sqlite3VbeCursorRestore() can only fail if the record has been deleted
-
  ** out from under the cursor.  That will never happens for an IdxRowid
-
  ** or Seek opcode */
-
  if( NEVER(rc!=SQLITE_OK) ) goto abort_due_to_error;
+
  /* sqlite3VdbeCursorRestore() may fail if the cursor has been disturbed
+
  ** since it was last positioned and an error (e.g. OOM or an IO error)
+
  ** occurs while trying to reposition it. */
+
  if( rc!=SQLITE_OK ) goto abort_due_to_error;

  if( !pC->nullRow ){
    rowid = 0;  /* Not needed.  Only used to silence a warning. */
@@ -93610,6 +96082,7 @@ case OP_IdxRowid: { /* out2 */
      pTabCur->nullRow = 0;
      pTabCur->movetoTarget = rowid;
      pTabCur->deferredMoveto = 1;
+
      pTabCur->cacheStatus = CACHE_STALE;
      assert( pOp->p4type==P4_INTARRAY || pOp->p4.ai==0 );
      assert( !pTabCur->isEphemeral );
      pTabCur->ub.aAltMap = pOp->p4.ai;
@@ -93744,7 +96217,7 @@ case OP_IdxGE: { /* jump */
    rc = sqlite3VdbeMemFromBtreeZeroOffset(pCur, (u32)nCellKey, &m);
    if( rc ) goto abort_due_to_error;
    res = sqlite3VdbeRecordCompareWithSkip(m.n, m.z, &r, 0);
-
    sqlite3VdbeMemRelease(&m);
+
    sqlite3VdbeMemReleaseMalloc(&m);
  }
  /* End of inlined sqlite3VdbeIdxKeyCompare() */

@@ -94493,7 +96966,7 @@ case OP_IfPos: { /* jump, in1 */
** Synopsis: if r[P1]>0 then r[P2]=r[P1]+max(0,r[P3]) else r[P2]=(-1)
**
** This opcode performs a commonly used computation associated with
-
** LIMIT and OFFSET process.  r[P1] holds the limit counter.  r[P3]
+
** LIMIT and OFFSET processing.  r[P1] holds the limit counter.  r[P3]
** holds the offset counter.  The opcode computes the combined value
** of the LIMIT and OFFSET and stores that value in r[P2].  The r[P2]
** value computed is the total number of rows that will need to be
@@ -94625,6 +97098,7 @@ case OP_AggStep: {
  pCtx->pVdbe = p;
  pCtx->skipFlag = 0;
  pCtx->isError = 0;
+
  pCtx->enc = encoding;
  pCtx->argc = n;
  pOp->p4type = P4_FUNCCTX;
  pOp->p4.pCtx = pCtx;
@@ -94754,9 +97228,6 @@ case OP_AggFinal: {
  }
  sqlite3VdbeChangeEncoding(pMem, encoding);
  UPDATE_MAX_BLOBSIZE(pMem);
-
  if( sqlite3VdbeMemTooBig(pMem) ){
-
    goto too_big;
-
  }
  break;
}

@@ -95264,7 +97735,6 @@ case OP_VColumn: {

  VdbeCursor *pCur = p->apCsr[pOp->p1];
  assert( pCur!=0 );
-
  assert( pCur->eCurType==CURTYPE_VTAB );
  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pDest = &aMem[pOp->p3];
  memAboutToChange(p, pDest);
@@ -95272,11 +97742,13 @@ case OP_VColumn: {
    sqlite3VdbeMemSetNull(pDest);
    break;
  }
+
  assert( pCur->eCurType==CURTYPE_VTAB );
  pVtab = pCur->uc.pVCur->pVtab;
  pModule = pVtab->pModule;
  assert( pModule->xColumn );
  memset(&sContext, 0, sizeof(sContext));
  sContext.pOut = pDest;
+
  sContext.enc = encoding;
  assert( pOp->p5==OPFLAG_NOCHNG || pOp->p5==0 );
  if( pOp->p5 & OPFLAG_NOCHNG ){
    sqlite3VdbeMemSetNull(pDest);
@@ -95295,9 +97767,6 @@ case OP_VColumn: {
  REGISTER_TRACE(pOp->p3, pDest);
  UPDATE_MAX_BLOBSIZE(pDest);

-
  if( sqlite3VdbeMemTooBig(pDest) ){
-
    goto too_big;
-
  }
  if( rc ) goto abort_due_to_error;
  break;
}
@@ -95564,6 +98033,7 @@ case OP_Function: { /* group */
  if( pCtx->pOut != pOut ){
    pCtx->pVdbe = p;
    pCtx->pOut = pOut;
+
    pCtx->enc = encoding;
    for(i=pCtx->argc-1; i>=0; i--) pCtx->argv[i] = &aMem[pOp->p2+i];
  }
  assert( pCtx->pVdbe==p );
@@ -95590,17 +98060,27 @@ case OP_Function: { /* group */
    if( rc ) goto abort_due_to_error;
  }

-
  /* Copy the result of the function into register P3 */
-
  if( pOut->flags & (MEM_Str|MEM_Blob) ){
-
    sqlite3VdbeChangeEncoding(pOut, encoding);
-
    if( sqlite3VdbeMemTooBig(pOut) ) goto too_big;
-
  }
+
  assert( (pOut->flags&MEM_Str)==0
+
       || pOut->enc==encoding
+
       || db->mallocFailed );
+
  assert( !sqlite3VdbeMemTooBig(pOut) );

  REGISTER_TRACE(pOp->p3, pOut);
  UPDATE_MAX_BLOBSIZE(pOut);
  break;
}

+
/* Opcode: ClrSubtype P1 * * * *
+
** Synopsis:  r[P1].subtype = 0
+
**
+
** Clear the subtype from register P1.
+
*/
+
case OP_ClrSubtype: {   /* in1 */
+
  pIn1 = &aMem[pOp->p1];
+
  pIn1->flags &= ~MEM_Subtype;
+
  break;
+
}
+

/* Opcode: FilterAdd P1 * P3 P4 *
** Synopsis: filter(P1) += key(P3@P4)
**
@@ -95720,7 +98200,7 @@ case OP_Init: { /* jump */

#ifndef SQLITE_OMIT_TRACE
  if( (db->mTrace & (SQLITE_TRACE_STMT|SQLITE_TRACE_LEGACY))!=0
-
   && !p->doingRerun
+
   && p->minWriteFileFormat!=254  /* tag-20220401a */
   && (zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql))!=0
  ){
#ifndef SQLITE_OMIT_DEPRECATED
@@ -95949,7 +98429,7 @@ abort_due_to_error:
  testcase( sqlite3GlobalConfig.xLog!=0 );
  sqlite3_log(rc, "statement aborts at %d: [%s] %s",
                   (int)(pOp - aOp), p->zSql, p->zErrMsg);
-
  sqlite3VdbeHalt(p);
+
  if( p->eVdbeState==VDBE_RUN_STATE ) sqlite3VdbeHalt(p);
  if( rc==SQLITE_IOERR_NOMEM ) sqlite3OomFault(db);
  if( rc==SQLITE_CORRUPT && db->autoCommit==0 ){
    db->flags |= SQLITE_CorruptRdOnly;
@@ -100084,6 +102564,8 @@ SQLITE_PRIVATE int sqlite3JournalOpen(
){
  MemJournal *p = (MemJournal*)pJfd;

+
  assert( zName || nSpill<0 || (flags & SQLITE_OPEN_EXCLUSIVE) );
+

  /* Zero the file-handle object. If nSpill was passed zero, initialize
  ** it using the sqlite3OsOpen() function of the underlying VFS. In this
  ** case none of the code in this module is executed as a result of calls
@@ -100511,53 +102993,24 @@ static void resolveAlias(
    sqlite3ExprDelete(db, pDup);
    pDup = 0;
  }else{
+
    Expr temp;
    incrAggFunctionDepth(pDup, nSubquery);
    if( pExpr->op==TK_COLLATE ){
      assert( !ExprHasProperty(pExpr, EP_IntValue) );
      pDup = sqlite3ExprAddCollateString(pParse, pDup, pExpr->u.zToken);
    }
-

-
    /* Before calling sqlite3ExprDelete(), set the EP_Static flag. This
-
    ** prevents ExprDelete() from deleting the Expr structure itself,
-
    ** allowing it to be repopulated by the memcpy() on the following line.
-
    ** The pExpr->u.zToken might point into memory that will be freed by the
-
    ** sqlite3DbFree(db, pDup) on the last line of this block, so be sure to
-
    ** make a copy of the token before doing the sqlite3DbFree().
-
    */
-
    ExprSetProperty(pExpr, EP_Static);
-
    sqlite3ExprDelete(db, pExpr);
-
    memcpy(pExpr, pDup, sizeof(*pExpr));
-
    if( !ExprHasProperty(pExpr, EP_IntValue) && pExpr->u.zToken!=0 ){
-
      assert( (pExpr->flags & (EP_Reduced|EP_TokenOnly))==0 );
-
      pExpr->u.zToken = sqlite3DbStrDup(db, pExpr->u.zToken);
-
      pExpr->flags |= EP_MemToken;
-
    }
+
    memcpy(&temp, pDup, sizeof(Expr));
+
    memcpy(pDup, pExpr, sizeof(Expr));
+
    memcpy(pExpr, &temp, sizeof(Expr));
    if( ExprHasProperty(pExpr, EP_WinFunc) ){
      if( ALWAYS(pExpr->y.pWin!=0) ){
        pExpr->y.pWin->pOwner = pExpr;
      }
    }
-
    sqlite3DbFree(db, pDup);
+
    sqlite3ExprDeferredDelete(pParse, pDup);
  }
}

-

-
/*
-
** Return TRUE if the name zCol occurs anywhere in the USING clause.
-
**
-
** Return FALSE if the USING clause is NULL or if it does not contain
-
** zCol.
-
*/
-
static int nameInUsingClause(IdList *pUsing, const char *zCol){
-
  if( pUsing ){
-
    int k;
-
    for(k=0; k<pUsing->nId; k++){
-
      if( sqlite3StrICmp(pUsing->a[k].zName, zCol)==0 ) return 1;
-
    }
-
  }
-
  return 0;
-
}
-

/*
** Subqueries stores the original database, table and column names for their
** result sets in ExprList.a[].zSpan, in the form "DATABASE.TABLE.COLUMN".
@@ -100573,7 +103026,7 @@ SQLITE_PRIVATE int sqlite3MatchEName(
){
  int n;
  const char *zSpan;
-
  if( pItem->eEName!=ENAME_TAB ) return 0;
+
  if( pItem->fg.eEName!=ENAME_TAB ) return 0;
  zSpan = pItem->zEName;
  for(n=0; ALWAYS(zSpan[n]) && zSpan[n]!='.'; n++){}
  if( zDb && (sqlite3StrNICmp(zSpan, zDb, n)!=0 || zDb[n]!=0) ){
@@ -100635,6 +103088,29 @@ SQLITE_PRIVATE Bitmask sqlite3ExprColUsed(Expr *pExpr){
}

/*
+
** Create a new expression term for the column specified by pMatch and
+
** iColumn.  Append this new expression term to the FULL JOIN Match set
+
** in *ppList.  Create a new *ppList if this is the first term in the
+
** set.
+
*/
+
static void extendFJMatch(
+
  Parse *pParse,          /* Parsing context */
+
  ExprList **ppList,      /* ExprList to extend */
+
  SrcItem *pMatch,        /* Source table containing the column */
+
  i16 iColumn             /* The column number */
+
){
+
  Expr *pNew = sqlite3ExprAlloc(pParse->db, TK_COLUMN, 0, 0);
+
  if( pNew ){
+
    pNew->iTable = pMatch->iCursor;
+
    pNew->iColumn = iColumn;
+
    pNew->y.pTab = pMatch->pTab;
+
    assert( (pMatch->fg.jointype & (JT_LEFT|JT_LTORJ))!=0 );
+
    ExprSetProperty(pNew, EP_CanBeNull);
+
    *ppList = sqlite3ExprListAppend(pParse, *ppList, pNew);
+
  }
+
}
+

+
/*
** Given the name of a column of the form X.Y.Z or Y.Z or just Z, look up
** that name in the set of source tables in pSrcList and make the pExpr
** expression node refer back to that source column.  The following changes
@@ -100679,11 +103155,13 @@ static int lookupName(
  NameContext *pTopNC = pNC;        /* First namecontext in the list */
  Schema *pSchema = 0;              /* Schema of the expression */
  int eNewExprOp = TK_COLUMN;       /* New value for pExpr->op on success */
-
  Table *pTab = 0;                  /* Table hold the row */
+
  Table *pTab = 0;                  /* Table holding the row */
  Column *pCol;                     /* A column of pTab */
+
  ExprList *pFJMatch = 0;           /* Matches for FULL JOIN .. USING */

  assert( pNC );     /* the name context cannot be NULL. */
  assert( zCol );    /* The Z in X.Y.Z cannot be NULL */
+
  assert( zDb==0 || zTab!=0 );
  assert( !ExprHasProperty(pExpr, EP_TokenOnly|EP_Reduced) );

  /* Initialize the node to no-match */
@@ -100732,26 +103210,65 @@ static int lookupName(
        pTab = pItem->pTab;
        assert( pTab!=0 && pTab->zName!=0 );
        assert( pTab->nCol>0 || pParse->nErr );
-
        if( pItem->pSelect && (pItem->pSelect->selFlags & SF_NestedFrom)!=0 ){
+
        assert( (int)pItem->fg.isNestedFrom == IsNestedFrom(pItem->pSelect) );
+
        if( pItem->fg.isNestedFrom ){
+
          /* In this case, pItem is a subquery that has been formed from a
+
          ** parenthesized subset of the FROM clause terms.  Example:
+
          **   .... FROM t1 LEFT JOIN (t2 RIGHT JOIN t3 USING(x)) USING(y) ...
+
          **                          \_________________________/
+
          **             This pItem -------------^
+
          */
          int hit = 0;
+
          assert( pItem->pSelect!=0 );
          pEList = pItem->pSelect->pEList;
+
          assert( pEList!=0 );
+
          assert( pEList->nExpr==pTab->nCol );
          for(j=0; j<pEList->nExpr; j++){
-
            if( sqlite3MatchEName(&pEList->a[j], zCol, zTab, zDb) ){
-
              cnt++;
-
              cntTab = 2;
-
              pMatch = pItem;
-
              pExpr->iColumn = j;
-
              hit = 1;
+
            if( !sqlite3MatchEName(&pEList->a[j], zCol, zTab, zDb) ){
+
              continue;
+
            }
+
            if( cnt>0 ){
+
              if( pItem->fg.isUsing==0
+
               || sqlite3IdListIndex(pItem->u3.pUsing, zCol)<0
+
              ){
+
                /* Two or more tables have the same column name which is
+
                ** not joined by USING.  This is an error.  Signal as much
+
                ** by clearing pFJMatch and letting cnt go above 1. */
+
                sqlite3ExprListDelete(db, pFJMatch);
+
                pFJMatch = 0;
+
              }else
+
              if( (pItem->fg.jointype & JT_RIGHT)==0 ){
+
                /* An INNER or LEFT JOIN.  Use the left-most table */
+
                continue;
+
              }else
+
              if( (pItem->fg.jointype & JT_LEFT)==0 ){
+
                /* A RIGHT JOIN.  Use the right-most table */
+
                cnt = 0;
+
                sqlite3ExprListDelete(db, pFJMatch);
+
                pFJMatch = 0;
+
              }else{
+
                /* For a FULL JOIN, we must construct a coalesce() func */
+
                extendFJMatch(pParse, &pFJMatch, pMatch, pExpr->iColumn);
+
              }
            }
+
            cnt++;
+
            cntTab = 2;
+
            pMatch = pItem;
+
            pExpr->iColumn = j;
+
            pEList->a[j].fg.bUsed = 1;
+
            hit = 1;
+
            if( pEList->a[j].fg.bUsingTerm ) break;
          }
          if( hit || zTab==0 ) continue;
        }
-
        if( zDb ){
-
          if( pTab->pSchema!=pSchema ) continue;
-
          if( pSchema==0 && strcmp(zDb,"*")!=0 ) continue;
-
        }
+
        assert( zDb==0 || zTab!=0 );
        if( zTab ){
-
          const char *zTabName = pItem->zAlias ? pItem->zAlias : pTab->zName;
+
          const char *zTabName;
+
          if( zDb ){
+
            if( pTab->pSchema!=pSchema ) continue;
+
            if( pSchema==0 && strcmp(zDb,"*")!=0 ) continue;
+
          }
+
          zTabName = pItem->zAlias ? pItem->zAlias : pTab->zName;
          assert( zTabName!=0 );
          if( sqlite3StrICmp(zTabName, zTab)!=0 ){
            continue;
@@ -100766,18 +103283,37 @@ static int lookupName(
          if( pCol->hName==hCol
           && sqlite3StrICmp(pCol->zCnName, zCol)==0
          ){
-
            /* If there has been exactly one prior match and this match
-
            ** is for the right-hand table of a NATURAL JOIN or is in a
-
            ** USING clause, then skip this match.
-
            */
-
            if( cnt==1 ){
-
              if( pItem->fg.jointype & JT_NATURAL ) continue;
-
              if( nameInUsingClause(pItem->pUsing, zCol) ) continue;
+
            if( cnt>0 ){
+
              if( pItem->fg.isUsing==0
+
               || sqlite3IdListIndex(pItem->u3.pUsing, zCol)<0
+
              ){
+
                /* Two or more tables have the same column name which is
+
                ** not joined by USING.  This is an error.  Signal as much
+
                ** by clearing pFJMatch and letting cnt go above 1. */
+
                sqlite3ExprListDelete(db, pFJMatch);
+
                pFJMatch = 0;
+
              }else
+
              if( (pItem->fg.jointype & JT_RIGHT)==0 ){
+
                /* An INNER or LEFT JOIN.  Use the left-most table */
+
                continue;
+
              }else
+
              if( (pItem->fg.jointype & JT_LEFT)==0 ){
+
                /* A RIGHT JOIN.  Use the right-most table */
+
                cnt = 0;
+
                sqlite3ExprListDelete(db, pFJMatch);
+
                pFJMatch = 0;
+
              }else{
+
                /* For a FULL JOIN, we must construct a coalesce() func */
+
                extendFJMatch(pParse, &pFJMatch, pMatch, pExpr->iColumn);
+
              }
            }
            cnt++;
            pMatch = pItem;
            /* Substitute the rowid (column -1) for the INTEGER PRIMARY KEY */
            pExpr->iColumn = j==pTab->iPKey ? -1 : (i16)j;
+
            if( pItem->fg.isNestedFrom ){
+
              sqlite3SrcItemColumnUsed(pItem, j);
+
            }
            break;
          }
        }
@@ -100790,9 +103326,7 @@ static int lookupName(
        pExpr->iTable = pMatch->iCursor;
        assert( ExprUseYTab(pExpr) );
        pExpr->y.pTab = pMatch->pTab;
-
        /* RIGHT JOIN not (yet) supported */
-
        assert( (pMatch->fg.jointype & JT_RIGHT)==0 );
-
        if( (pMatch->fg.jointype & JT_LEFT)!=0 ){
+
        if( (pMatch->fg.jointype & (JT_LEFT|JT_LTORJ))!=0 ){
          ExprSetProperty(pExpr, EP_CanBeNull);
        }
        pSchema = pExpr->y.pTab->pSchema;
@@ -100946,7 +103480,7 @@ static int lookupName(
      assert( pEList!=0 );
      for(j=0; j<pEList->nExpr; j++){
        char *zAs = pEList->a[j].zEName;
-
        if( pEList->a[j].eEName==ENAME_NAME
+
        if( pEList->a[j].fg.eEName==ENAME_NAME
         && sqlite3_stricmp(zAs, zCol)==0
        ){
          Expr *pOrig;
@@ -101033,11 +103567,37 @@ static int lookupName(
  }

  /*
-
  ** cnt==0 means there was not match.  cnt>1 means there were two or
-
  ** more matches.  Either way, we have an error.
+
  ** cnt==0 means there was not match.
+
  ** cnt>1 means there were two or more matches.
+
  **
+
  ** cnt==0 is always an error.  cnt>1 is often an error, but might
+
  ** be multiple matches for a NATURAL LEFT JOIN or a LEFT JOIN USING.
  */
+
  assert( pFJMatch==0 || cnt>0 );
+
  assert( !ExprHasProperty(pExpr, EP_xIsSelect|EP_IntValue) );
  if( cnt!=1 ){
    const char *zErr;
+
    if( pFJMatch ){
+
      if( pFJMatch->nExpr==cnt-1 ){
+
        if( ExprHasProperty(pExpr,EP_Leaf) ){
+
          ExprClearProperty(pExpr,EP_Leaf);
+
        }else{
+
          sqlite3ExprDelete(db, pExpr->pLeft);
+
          pExpr->pLeft = 0;
+
          sqlite3ExprDelete(db, pExpr->pRight);
+
          pExpr->pRight = 0;
+
        }
+
        extendFJMatch(pParse, &pFJMatch, pMatch, pExpr->iColumn);
+
        pExpr->op = TK_FUNCTION;
+
        pExpr->u.zToken = "coalesce";
+
        pExpr->x.pList = pFJMatch;
+
        cnt = 1;
+
        goto lookupname_end;
+
      }else{
+
        sqlite3ExprListDelete(db, pFJMatch);
+
        pFJMatch = 0;
+
      }
+
    }
    zErr = cnt==0 ? "no such column" : "ambiguous column name";
    if( zDb ){
      sqlite3ErrorMsg(pParse, "%s: %s.%s.%s", zErr, zDb, zTab, zCol);
@@ -101050,6 +103610,16 @@ static int lookupName(
    pParse->checkSchema = 1;
    pTopNC->nNcErr++;
  }
+
  assert( pFJMatch==0 );
+

+
  /* Remove all substructure from pExpr */
+
  if( !ExprHasProperty(pExpr,(EP_TokenOnly|EP_Leaf)) ){
+
    sqlite3ExprDelete(db, pExpr->pLeft);
+
    pExpr->pLeft = 0;
+
    sqlite3ExprDelete(db, pExpr->pRight);
+
    pExpr->pRight = 0;
+
    ExprSetProperty(pExpr, EP_Leaf);
+
  }

  /* If a column from a table in pSrcList is referenced, then record
  ** this fact in the pSrcList.a[].colUsed bitmask.  Column 0 causes
@@ -101069,16 +103639,7 @@ static int lookupName(
    pMatch->colUsed |= sqlite3ExprColUsed(pExpr);
  }

-
  /* Clean up and return
-
  */
-
  if( !ExprHasProperty(pExpr,(EP_TokenOnly|EP_Leaf)) ){
-
    sqlite3ExprDelete(db, pExpr->pLeft);
-
    pExpr->pLeft = 0;
-
    sqlite3ExprDelete(db, pExpr->pRight);
-
    pExpr->pRight = 0;
-
  }
  pExpr->op = eNewExprOp;
-
  ExprSetProperty(pExpr, EP_Leaf);
lookupname_end:
  if( cnt==1 ){
    assert( pNC!=0 );
@@ -101263,7 +103824,7 @@ static int resolveExprStep(Walker *pWalker, Expr *pExpr){
      }
      sqlite3WalkExpr(pWalker, pExpr->pLeft);
      if( 0==sqlite3ExprCanBeNull(pExpr->pLeft) && !IN_RENAME_OBJECT ){
-
        testcase( ExprHasProperty(pExpr, EP_FromJoin) );
+
        testcase( ExprHasProperty(pExpr, EP_OuterON) );
        assert( !ExprHasProperty(pExpr, EP_IntValue) );
        if( pExpr->op==TK_NOTNULL ){
          pExpr->u.zToken = "true";
@@ -101672,7 +104233,7 @@ static int resolveAsName(
    assert( !ExprHasProperty(pE, EP_IntValue) );
    zCol = pE->u.zToken;
    for(i=0; i<pEList->nExpr; i++){
-
      if( pEList->a[i].eEName==ENAME_NAME
+
      if( pEList->a[i].fg.eEName==ENAME_NAME
       && sqlite3_stricmp(pEList->a[i].zEName, zCol)==0
      ){
        return i+1;
@@ -101793,7 +104354,7 @@ static int resolveCompoundOrderBy(
    return 1;
  }
  for(i=0; i<pOrderBy->nExpr; i++){
-
    pOrderBy->a[i].done = 0;
+
    pOrderBy->a[i].fg.done = 0;
  }
  pSelect->pNext = 0;
  while( pSelect->pPrior ){
@@ -101808,7 +104369,7 @@ static int resolveCompoundOrderBy(
    for(i=0, pItem=pOrderBy->a; i<pOrderBy->nExpr; i++, pItem++){
      int iCol = -1;
      Expr *pE, *pDup;
-
      if( pItem->done ) continue;
+
      if( pItem->fg.done ) continue;
      pE = sqlite3ExprSkipCollateAndLikely(pItem->pExpr);
      if( NEVER(pE==0) ) continue;
      if( sqlite3ExprIsInteger(pE, &iCol) ){
@@ -101861,7 +104422,7 @@ static int resolveCompoundOrderBy(
          sqlite3ExprDelete(db, pE);
          pItem->u.x.iOrderByCol = (u16)iCol;
        }
-
        pItem->done = 1;
+
        pItem->fg.done = 1;
      }else{
        moreToDo = 1;
      }
@@ -101869,7 +104430,7 @@ static int resolveCompoundOrderBy(
    pSelect = pSelect->pNext;
  }
  for(i=0; i<pOrderBy->nExpr; i++){
-
    if( pOrderBy->a[i].done==0 ){
+
    if( pOrderBy->a[i].fg.done==0 ){
      sqlite3ErrorMsg(pParse, "%r ORDER BY term does not match any "
            "column in the result set", i+1);
      return 1;
@@ -102159,8 +104720,8 @@ static int resolveSelectStep(Walker *pWalker, Select *p){
    sNC.uNC.pEList = p->pEList;
    sNC.ncFlags |= NC_UEList;
    if( p->pHaving ){
-
      if( !pGroupBy ){
-
        sqlite3ErrorMsg(pParse, "a GROUP BY clause is required before HAVING");
+
      if( (p->selFlags & SF_Aggregate)==0 ){
+
        sqlite3ErrorMsg(pParse, "HAVING clause on a non-aggregate query");
        return WRC_Abort;
      }
      if( sqlite3ResolveExprNames(&sNC, p->pHaving) ) return WRC_Abort;
@@ -102540,9 +105101,8 @@ SQLITE_PRIVATE char sqlite3ExprAffinity(const Expr *pExpr){
  if( op==TK_REGISTER ) op = pExpr->op2;
  if( op==TK_COLUMN || op==TK_AGG_COLUMN ){
    assert( ExprUseYTab(pExpr) );
-
    if( pExpr->y.pTab ){
-
      return sqlite3TableColumnAffinity(pExpr->y.pTab, pExpr->iColumn);
-
    }
+
    assert( pExpr->y.pTab!=0 );
+
    return sqlite3TableColumnAffinity(pExpr->y.pTab, pExpr->iColumn);
  }
  if( op==TK_SELECT ){
    assert( ExprUseXSelect(pExpr) );
@@ -102660,17 +105220,14 @@ SQLITE_PRIVATE CollSeq *sqlite3ExprCollSeq(Parse *pParse, const Expr *pExpr){
    int op = p->op;
    if( op==TK_REGISTER ) op = p->op2;
    if( op==TK_AGG_COLUMN || op==TK_COLUMN || op==TK_TRIGGER ){
+
      int j;
      assert( ExprUseYTab(p) );
-
      if( p->y.pTab!=0 ){
-
        /* op==TK_REGISTER && p->y.pTab!=0 happens when pExpr was originally
-
        ** a TK_COLUMN but was previously evaluated and cached in a register */
-
        int j = p->iColumn;
-
        if( j>=0 ){
-
          const char *zColl = sqlite3ColumnColl(&p->y.pTab->aCol[j]);
-
          pColl = sqlite3FindCollSeq(db, ENC(db), zColl, 0);
-
        }
-
        break;
+
      assert( p->y.pTab!=0 );
+
      if( (j = p->iColumn)>=0 ){
+
        const char *zColl = sqlite3ColumnColl(&p->y.pTab->aCol[j]);
+
        pColl = sqlite3FindCollSeq(db, ENC(db), zColl, 0);
      }
+
      break;
    }
    if( op==TK_CAST || op==TK_UPLUS ){
      p = p->pLeft;
@@ -103255,7 +105812,9 @@ static void heightOfSelect(const Select *pSelect, int *pnHeight){
*/
static void exprSetHeight(Expr *p){
  int nHeight = p->pLeft ? p->pLeft->nHeight : 0;
-
  if( p->pRight && p->pRight->nHeight>nHeight ) nHeight = p->pRight->nHeight;
+
  if( NEVER(p->pRight) && p->pRight->nHeight>nHeight ){
+
    nHeight = p->pRight->nHeight;
+
  }
  if( ExprUseXSelect(p) ){
    heightOfSelect(p->x.pSelect, &nHeight);
  }else if( p->x.pList ){
@@ -103398,15 +105957,26 @@ SQLITE_PRIVATE void sqlite3ExprAttachSubtrees(
    sqlite3ExprDelete(db, pLeft);
    sqlite3ExprDelete(db, pRight);
  }else{
+
    assert( ExprUseXList(pRoot) );
+
    assert( pRoot->x.pSelect==0 );
    if( pRight ){
      pRoot->pRight = pRight;
      pRoot->flags |= EP_Propagate & pRight->flags;
+
#if SQLITE_MAX_EXPR_DEPTH>0
+
      pRoot->nHeight = pRight->nHeight+1;
+
    }else{
+
      pRoot->nHeight = 1;
+
#endif
    }
    if( pLeft ){
      pRoot->pLeft = pLeft;
      pRoot->flags |= EP_Propagate & pLeft->flags;
+
#if SQLITE_MAX_EXPR_DEPTH>0
+
      if( pLeft->nHeight>=pRoot->nHeight ){
+
        pRoot->nHeight = pLeft->nHeight+1;
+
      }
+
#endif
    }
-
    exprSetHeight(pRoot);
  }
}

@@ -103554,6 +106124,7 @@ SQLITE_PRIVATE Expr *sqlite3ExprFunction(
    sqlite3ExprListDelete(db, pList); /* Avoid memory leak when malloc fails */
    return 0;
  }
+
  assert( !ExprHasProperty(pNew, EP_InnerON|EP_OuterON) );
  pNew->w.iOfst = (int)(pToken->z - pParse->zTail);
  if( pList
   && pList->nExpr > pParse->db->aLimit[SQLITE_LIMIT_FUNCTION_ARG]
@@ -103691,6 +106262,7 @@ SQLITE_PRIVATE void sqlite3ExprAssignVarNumber(Parse *pParse, Expr *pExpr, u32 n
*/
static SQLITE_NOINLINE void sqlite3ExprDeleteNN(sqlite3 *db, Expr *p){
  assert( p!=0 );
+
  assert( db!=0 );
  assert( !ExprUseUValue(p) || p->u.iValue>=0 );
  assert( !ExprUseYWin(p) || !ExprUseYSub(p) );
  assert( !ExprUseYWin(p) || p->y.pWin!=0 || db->mallocFailed );
@@ -103722,18 +106294,26 @@ static SQLITE_NOINLINE void sqlite3ExprDeleteNN(sqlite3 *db, Expr *p){
#endif
    }
  }
-
  if( ExprHasProperty(p, EP_MemToken) ){
-
    assert( !ExprHasProperty(p, EP_IntValue) );
-
    sqlite3DbFree(db, p->u.zToken);
-
  }
  if( !ExprHasProperty(p, EP_Static) ){
-
    sqlite3DbFreeNN(db, p);
+
    sqlite3DbNNFreeNN(db, p);
  }
}
SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3 *db, Expr *p){
  if( p ) sqlite3ExprDeleteNN(db, p);
}

+
/*
+
** Clear both elements of an OnOrUsing object
+
*/
+
SQLITE_PRIVATE void sqlite3ClearOnOrUsing(sqlite3 *db, OnOrUsing *p){
+
  if( p==0 ){
+
    /* Nothing to clear */
+
  }else if( p->pOn ){
+
    sqlite3ExprDeleteNN(db, p->pOn);
+
  }else if( p->pUsing ){
+
    sqlite3IdListDelete(db, p->pUsing);
+
  }
+
}

/*
** Arrange to cause pExpr to be deleted when the pParse is deleted.
@@ -103746,8 +106326,9 @@ SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3 *db, Expr *p){
** pExpr to the pParse->pConstExpr list with a register number of 0.
*/
SQLITE_PRIVATE void sqlite3ExprDeferredDelete(Parse *pParse, Expr *pExpr){
-
  pParse->pConstExpr =
-
      sqlite3ExprListAppend(pParse, pParse->pConstExpr, pExpr);
+
  sqlite3ParserAddCleanup(pParse,
+
    (void(*)(sqlite3*,void*))sqlite3ExprDelete,
+
    pExpr);
}

/* Invoke sqlite3RenameExprUnmap() and sqlite3ExprDelete() on the
@@ -103820,8 +106401,7 @@ static int dupedExprStructSize(const Expr *p, int flags){
    nSize = EXPR_FULLSIZE;
  }else{
    assert( !ExprHasProperty(p, EP_TokenOnly|EP_Reduced) );
-
    assert( !ExprHasProperty(p, EP_FromJoin) );
-
    assert( !ExprHasProperty(p, EP_MemToken) );
+
    assert( !ExprHasProperty(p, EP_OuterON) );
    assert( !ExprHasVVAProperty(p, EP_NoReduce) );
    if( p->pLeft || p->x.pList ){
      nSize = EXPR_REDUCEDSIZE | EP_Reduced;
@@ -103925,7 +106505,7 @@ static Expr *exprDup(sqlite3 *db, const Expr *p, int dupFlags, u8 **pzBuffer){
    }

    /* Set the EP_Reduced, EP_TokenOnly, and EP_Static flags appropriately. */
-
    pNew->flags &= ~(EP_Reduced|EP_TokenOnly|EP_Static|EP_MemToken);
+
    pNew->flags &= ~(EP_Reduced|EP_TokenOnly|EP_Static);
    pNew->flags |= nStructSize & (EP_Reduced|EP_TokenOnly);
    pNew->flags |= staticFlag;
    ExprClearVVAProperties(pNew);
@@ -104000,6 +106580,7 @@ SQLITE_PRIVATE With *sqlite3WithDup(sqlite3 *db, With *p){
        pRet->a[i].pSelect = sqlite3SelectDup(db, p->a[i].pSelect, 0);
        pRet->a[i].pCols = sqlite3ExprListDup(db, p->a[i].pCols, 0);
        pRet->a[i].zName = sqlite3DbStrDup(db, p->a[i].zName);
+
        pRet->a[i].eM10d = p->a[i].eM10d;
      }
    }
  }
@@ -104100,11 +106681,8 @@ SQLITE_PRIVATE ExprList *sqlite3ExprListDup(sqlite3 *db, const ExprList *p, int
      }
    }
    pItem->zEName = sqlite3DbStrDup(db, pOldItem->zEName);
-
    pItem->sortFlags = pOldItem->sortFlags;
-
    pItem->eEName = pOldItem->eEName;
-
    pItem->done = 0;
-
    pItem->bNulls = pOldItem->bNulls;
-
    pItem->bSorterRef = pOldItem->bSorterRef;
+
    pItem->fg = pOldItem->fg;
+
    pItem->fg.done = 0;
    pItem->u = pOldItem->u;
  }
  return pNew;
@@ -104156,8 +106734,12 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListDup(sqlite3 *db, const SrcList *p, int fla
      pTab->nTabRef++;
    }
    pNewItem->pSelect = sqlite3SelectDup(db, pOldItem->pSelect, flags);
-
    pNewItem->pOn = sqlite3ExprDup(db, pOldItem->pOn, flags);
-
    pNewItem->pUsing = sqlite3IdListDup(db, pOldItem->pUsing);
+
    if( pOldItem->fg.isUsing ){
+
      assert( pNewItem->fg.isUsing );
+
      pNewItem->u3.pUsing = sqlite3IdListDup(db, pOldItem->u3.pUsing);
+
    }else{
+
      pNewItem->u3.pOn = sqlite3ExprDup(db, pOldItem->u3.pOn, flags);
+
    }
    pNewItem->colUsed = pOldItem->colUsed;
  }
  return pNew;
@@ -104167,22 +106749,16 @@ SQLITE_PRIVATE IdList *sqlite3IdListDup(sqlite3 *db, const IdList *p){
  int i;
  assert( db!=0 );
  if( p==0 ) return 0;
-
  pNew = sqlite3DbMallocRawNN(db, sizeof(*pNew) );
+
  assert( p->eU4!=EU4_EXPR );
+
  pNew = sqlite3DbMallocRawNN(db, sizeof(*pNew)+(p->nId-1)*sizeof(p->a[0]) );
  if( pNew==0 ) return 0;
  pNew->nId = p->nId;
-
  pNew->a = sqlite3DbMallocRawNN(db, p->nId*sizeof(p->a[0]) );
-
  if( pNew->a==0 ){
-
    sqlite3DbFreeNN(db, pNew);
-
    return 0;
-
  }
-
  /* Note that because the size of the allocation for p->a[] is not
-
  ** necessarily a power of two, sqlite3IdListAppend() may not be called
-
  ** on the duplicate created by this function. */
+
  pNew->eU4 = p->eU4;
  for(i=0; i<p->nId; i++){
    struct IdList_item *pNewItem = &pNew->a[i];
-
    struct IdList_item *pOldItem = &p->a[i];
+
    const struct IdList_item *pOldItem = &p->a[i];
    pNewItem->zName = sqlite3DbStrDup(db, pOldItem->zName);
-
    pNewItem->idx = pOldItem->idx;
+
    pNewItem->u4 = pOldItem->u4;
  }
  return pNew;
}
@@ -104406,16 +106982,16 @@ SQLITE_PRIVATE void sqlite3ExprListSetSortOrder(ExprList *p, int iSortOrder, int
  );

  pItem = &p->a[p->nExpr-1];
-
  assert( pItem->bNulls==0 );
+
  assert( pItem->fg.bNulls==0 );
  if( iSortOrder==SQLITE_SO_UNDEFINED ){
    iSortOrder = SQLITE_SO_ASC;
  }
-
  pItem->sortFlags = (u8)iSortOrder;
+
  pItem->fg.sortFlags = (u8)iSortOrder;

  if( eNulls!=SQLITE_SO_UNDEFINED ){
-
    pItem->bNulls = 1;
+
    pItem->fg.bNulls = 1;
    if( iSortOrder!=eNulls ){
-
      pItem->sortFlags |= KEYINFO_ORDER_BIGNULL;
+
      pItem->fg.sortFlags |= KEYINFO_ORDER_BIGNULL;
    }
  }
}
@@ -104441,7 +107017,7 @@ SQLITE_PRIVATE void sqlite3ExprListSetName(
    assert( pList->nExpr>0 );
    pItem = &pList->a[pList->nExpr-1];
    assert( pItem->zEName==0 );
-
    assert( pItem->eEName==ENAME_NAME );
+
    assert( pItem->fg.eEName==ENAME_NAME );
    pItem->zEName = sqlite3DbStrNDup(pParse->db, pName->z, pName->n);
    if( dequote ){
      /* If dequote==0, then pName->z does not point to part of a DDL
@@ -104476,7 +107052,7 @@ SQLITE_PRIVATE void sqlite3ExprListSetSpan(
    assert( pList->nExpr>0 );
    if( pItem->zEName==0 ){
      pItem->zEName = sqlite3DbSpanDup(db, zStart, zEnd);
-
      pItem->eEName = ENAME_SPAN;
+
      pItem->fg.eEName = ENAME_SPAN;
    }
  }
}
@@ -104505,12 +107081,13 @@ static SQLITE_NOINLINE void exprListDeleteNN(sqlite3 *db, ExprList *pList){
  int i = pList->nExpr;
  struct ExprList_item *pItem =  pList->a;
  assert( pList->nExpr>0 );
+
  assert( db!=0 );
  do{
    sqlite3ExprDelete(db, pItem->pExpr);
-
    sqlite3DbFree(db, pItem->zEName);
+
    if( pItem->zEName ) sqlite3DbNNFreeNN(db, pItem->zEName);
    pItem++;
  }while( --i>0 );
-
  sqlite3DbFreeNN(db, pList);
+
  sqlite3DbNNFreeNN(db, pList);
}
SQLITE_PRIVATE void sqlite3ExprListDelete(sqlite3 *db, ExprList *pList){
  if( pList ) exprListDeleteNN(db, pList);
@@ -104648,9 +107225,9 @@ SQLITE_PRIVATE Expr *sqlite3ExprSimplifiedAndOr(Expr *pExpr){
static int exprNodeIsConstant(Walker *pWalker, Expr *pExpr){

  /* If pWalker->eCode is 2 then any term of the expression that comes from
-
  ** the ON or USING clauses of a left join disqualifies the expression
+
  ** the ON or USING clauses of an outer join disqualifies the expression
  ** from being considered constant. */
-
  if( pWalker->eCode==2 && ExprHasProperty(pExpr, EP_FromJoin) ){
+
  if( pWalker->eCode==2 && ExprHasProperty(pExpr, EP_OuterON) ){
    pWalker->eCode = 0;
    return WRC_Abort;
  }
@@ -104773,7 +107350,7 @@ SQLITE_PRIVATE int sqlite3ExprIsTableConstant(Expr *p, int iCur){
** Check pExpr to see if it is an invariant constraint on data source pSrc.
** This is an optimization.  False negatives will perhaps cause slower
** queries, but false positives will yield incorrect answers.  So when in
-
** double, return 0.
+
** doubt, return 0.
**
** To be an invariant constraint, the following must be true:
**
@@ -104781,24 +107358,28 @@ SQLITE_PRIVATE int sqlite3ExprIsTableConstant(Expr *p, int iCur){
**
**   (2)  pExpr cannot use subqueries or non-deterministic functions.
**
-
**   (*)  ** Not applicable to this branch **
+
**   (3)  pSrc cannot be part of the left operand for a RIGHT JOIN.
+
**        (Is there some way to relax this constraint?)
**
**   (4)  If pSrc is the right operand of a LEFT JOIN, then...
**         (4a)  pExpr must come from an ON clause..
-
**         (4b)  and specifically the ON clause associated with the LEFT JOIN.
+
           (4b)  and specifically the ON clause associated with the LEFT JOIN.
**
**   (5)  If pSrc is not the right operand of a LEFT JOIN or the left
**        operand of a RIGHT JOIN, then pExpr must be from the WHERE
**        clause, not an ON clause.
*/
SQLITE_PRIVATE int sqlite3ExprIsTableConstraint(Expr *pExpr, const SrcItem *pSrc){
+
  if( pSrc->fg.jointype & JT_LTORJ ){
+
    return 0;  /* rule (3) */
+
  }
  if( pSrc->fg.jointype & JT_LEFT ){
-
    if( !ExprHasProperty(pExpr, EP_FromJoin) ) return 0;    /* rule (4a) */
-
    if( pExpr->w.iRightJoinTable!=pSrc->iCursor ) return 0; /* rule (4b) */
+
    if( !ExprHasProperty(pExpr, EP_OuterON) ) return 0;   /* rule (4a) */
+
    if( pExpr->w.iJoin!=pSrc->iCursor ) return 0;         /* rule (4b) */
  }else{
-
    if( ExprHasProperty(pExpr, EP_FromJoin) ) return 0;     /* rule (5) */
+
    if( ExprHasProperty(pExpr, EP_OuterON) ) return 0;    /* rule (5) */
  }
-
  return sqlite3ExprIsTableConstant(pExpr, pSrc->iCursor);  /* rules (1), (2) */
+
  return sqlite3ExprIsTableConstant(pExpr, pSrc->iCursor); /* rules (1), (2) */
}


@@ -105128,7 +107709,7 @@ static int sqlite3InRhsIsConstant(Expr *pIn){
** all members of the RHS set, skipping duplicates.
**
** A cursor is opened on the b-tree object that is the RHS of the IN operator
-
** and pX->iTable is set to the index of that cursor.
+
** and the *piTab parameter is set to the index of that cursor.
**
** The returned value of this function indicates the b-tree type, as follows:
**
@@ -105148,7 +107729,10 @@ static int sqlite3InRhsIsConstant(Expr *pIn){
** If the RHS of the IN operator is a list or a more complex subquery, then
** an ephemeral table might need to be generated from the RHS and then
** pX->iTable made to point to the ephemeral table instead of an
-
** existing table.
+
** existing table.  In this case, the creation and initialization of the
+
** ephmeral table might be put inside of a subroutine, the EP_Subrtn flag
+
** will be set on pX and the pX->y.sub fields will be set to show where
+
** the subroutine is coded.
**
** The inFlags parameter must contain, at a minimum, one of the bits
** IN_INDEX_MEMBERSHIP or IN_INDEX_LOOP but not both.  If inFlags contains
@@ -105209,12 +107793,13 @@ SQLITE_PRIVATE int sqlite3FindInIndex(
){
  Select *p;                            /* SELECT to the right of IN operator */
  int eType = 0;                        /* Type of RHS table. IN_INDEX_* */
-
  int iTab = pParse->nTab++;            /* Cursor of the RHS table */
+
  int iTab;                             /* Cursor of the RHS table */
  int mustBeUnique;                     /* True if RHS must be unique */
  Vdbe *v = sqlite3GetVdbe(pParse);     /* Virtual machine being coded */

  assert( pX->op==TK_IN );
  mustBeUnique = (inFlags & IN_INDEX_LOOP)!=0;
+
  iTab = pParse->nTab++;

  /* If the RHS of this IN(...) operator is a SELECT, and if it matters
  ** whether or not the SELECT result contains NULL values, check whether
@@ -105380,6 +107965,8 @@ SQLITE_PRIVATE int sqlite3FindInIndex(
   && ExprUseXList(pX)
   && (!sqlite3InRhsIsConstant(pX) || pX->x.pList->nExpr<=2)
  ){
+
    pParse->nTab--;  /* Back out the allocation of the unused cursor */
+
    iTab = -1;       /* Cursor is not allocated */
    eType = IN_INDEX_NOOP;
  }

@@ -105546,6 +108133,7 @@ SQLITE_PRIVATE void sqlite3CodeRhsOfIN(
      assert( ExprUseYSub(pExpr) );
      sqlite3VdbeAddOp2(v, OP_Gosub, pExpr->y.sub.regReturn,
                        pExpr->y.sub.iAddr);
+
      assert( iTab!=pExpr->iTable );
      sqlite3VdbeAddOp2(v, OP_OpenDup, iTab, pExpr->iTable);
      sqlite3VdbeJumpHere(v, addrOnce);
      return;
@@ -105557,8 +108145,7 @@ SQLITE_PRIVATE void sqlite3CodeRhsOfIN(
    assert( !ExprHasProperty(pExpr, EP_TokenOnly|EP_Reduced) );
    pExpr->y.sub.regReturn = ++pParse->nMem;
    pExpr->y.sub.iAddr =
-
      sqlite3VdbeAddOp2(v, OP_Integer, 0, pExpr->y.sub.regReturn) + 1;
-
    VdbeComment((v, "return address"));
+
      sqlite3VdbeAddOp2(v, OP_BeginSubrtn, 0, pExpr->y.sub.regReturn) + 1;

    addrOnce = sqlite3VdbeAddOp0(v, OP_Once); VdbeCoverage(v);
  }
@@ -105660,6 +108247,7 @@ SQLITE_PRIVATE void sqlite3CodeRhsOfIN(
      ** expression we need to rerun this code each time.
      */
      if( addrOnce && !sqlite3ExprIsConstant(pE2) ){
+
        sqlite3VdbeChangeToNoop(v, addrOnce-1);
        sqlite3VdbeChangeToNoop(v, addrOnce);
        ExprClearProperty(pExpr, EP_Subrtn);
        addrOnce = 0;
@@ -105677,11 +108265,15 @@ SQLITE_PRIVATE void sqlite3CodeRhsOfIN(
    sqlite3VdbeChangeP4(v, addr, (void *)pKeyInfo, P4_KEYINFO);
  }
  if( addrOnce ){
+
    sqlite3VdbeAddOp1(v, OP_NullRow, iTab);
    sqlite3VdbeJumpHere(v, addrOnce);
    /* Subroutine return */
    assert( ExprUseYSub(pExpr) );
-
    sqlite3VdbeAddOp1(v, OP_Return, pExpr->y.sub.regReturn);
-
    sqlite3VdbeChangeP1(v, pExpr->y.sub.iAddr-1, sqlite3VdbeCurrentAddr(v)-1);
+
    assert( sqlite3VdbeGetOp(v,pExpr->y.sub.iAddr-1)->opcode==OP_BeginSubrtn
+
            || pParse->nErr );
+
    sqlite3VdbeAddOp3(v, OP_Return, pExpr->y.sub.regReturn,
+
                      pExpr->y.sub.iAddr, 1);
+
    VdbeCoverage(v);
    sqlite3ClearTempRegCache(pParse);
  }
}
@@ -105735,9 +108327,7 @@ SQLITE_PRIVATE int sqlite3CodeSubselect(Parse *pParse, Expr *pExpr){
  ExprSetProperty(pExpr, EP_Subrtn);
  pExpr->y.sub.regReturn = ++pParse->nMem;
  pExpr->y.sub.iAddr =
-
    sqlite3VdbeAddOp2(v, OP_Integer, 0, pExpr->y.sub.regReturn) + 1;
-
  VdbeComment((v, "return address"));
-

+
    sqlite3VdbeAddOp2(v, OP_BeginSubrtn, 0, pExpr->y.sub.regReturn) + 1;

  /* The evaluation of the EXISTS/SELECT must be repeated every time it
  ** is encountered if any of the following is true:
@@ -105789,7 +108379,7 @@ SQLITE_PRIVATE int sqlite3CodeSubselect(Parse *pParse, Expr *pExpr){
      pLimit = sqlite3PExpr(pParse, TK_NE,
                            sqlite3ExprDup(db, pSel->pLimit->pLeft, 0), pLimit);
    }
-
    sqlite3ExprDelete(db, pSel->pLimit->pLeft);
+
    sqlite3ExprDeferredDelete(pParse, pSel->pLimit->pLeft);
    pSel->pLimit->pLeft = pLimit;
  }else{
    /* If there is no pre-existing limit add a limit of 1 */
@@ -105810,8 +108400,11 @@ SQLITE_PRIVATE int sqlite3CodeSubselect(Parse *pParse, Expr *pExpr){

  /* Subroutine return */
  assert( ExprUseYSub(pExpr) );
-
  sqlite3VdbeAddOp1(v, OP_Return, pExpr->y.sub.regReturn);
-
  sqlite3VdbeChangeP1(v, pExpr->y.sub.iAddr-1, sqlite3VdbeCurrentAddr(v)-1);
+
  assert( sqlite3VdbeGetOp(v,pExpr->y.sub.iAddr-1)->opcode==OP_BeginSubrtn
+
          || pParse->nErr );
+
  sqlite3VdbeAddOp3(v, OP_Return, pExpr->y.sub.regReturn,
+
                    pExpr->y.sub.iAddr, 1);
+
  VdbeCoverage(v);
  sqlite3ClearTempRegCache(pParse);
  return rReg;
}
@@ -106239,12 +108832,10 @@ SQLITE_PRIVATE void sqlite3ExprCodeGetColumnOfTable(
){
  Column *pCol;
  assert( v!=0 );
-
  if( pTab==0 ){
-
    sqlite3VdbeAddOp3(v, OP_Column, iTabCur, iCol, regOut);
-
    return;
-
  }
+
  assert( pTab!=0 );
  if( iCol<0 || iCol==pTab->iPKey ){
    sqlite3VdbeAddOp2(v, OP_Rowid, iTabCur, regOut);
+
    VdbeComment((v, "%s.rowid", pTab->zName));
  }else{
    int op;
    int x;
@@ -106299,7 +108890,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeGetColumn(
  assert( pParse->pVdbe!=0 );
  sqlite3ExprCodeGetColumnOfTable(pParse->pVdbe, pTab, iTable, iColumn, iReg);
  if( p5 ){
-
    VdbeOp *pOp = sqlite3VdbeGetOp(pParse->pVdbe,-1);
+
    VdbeOp *pOp = sqlite3VdbeGetLastOp(pParse->pVdbe);
    if( pOp->opcode==OP_Column ) pOp->p5 = p5;
  }
  return iReg;
@@ -106368,7 +108959,7 @@ static int exprCodeVector(Parse *pParse, Expr *p, int *piFreeable){
** so that a subsequent copy will not be merged into this one.
*/
static void setDoNotMergeFlagOnCopy(Vdbe *v){
-
  if( sqlite3VdbeGetOp(v, -1)->opcode==OP_Copy ){
+
  if( sqlite3VdbeGetLastOp(v)->opcode==OP_Copy ){
    sqlite3VdbeChangeP5(v, 1);  /* Tag trailing OP_Copy as not mergable */
  }
}
@@ -106415,7 +109006,17 @@ static int exprCodeInlineFunction(
      caseExpr.x.pList = pFarg;
      return sqlite3ExprCodeTarget(pParse, &caseExpr, target);
    }
-

+
#ifdef SQLITE_ENABLE_OFFSET_SQL_FUNC
+
    case INLINEFUNC_sqlite_offset: {
+
      Expr *pArg = pFarg->a[0].pExpr;
+
      if( pArg->op==TK_COLUMN && pArg->iTable>=0 ){
+
        sqlite3VdbeAddOp3(v, OP_Offset, pArg->iTable, pArg->iColumn, target);
+
      }else{
+
        sqlite3VdbeAddOp2(v, OP_Null, 0, target);
+
      }
+
      break;
+
    }
+
#endif
    default: {
      /* The UNLIKELY() function is a no-op.  The result is the value
      ** of the first argument.
@@ -106481,6 +109082,53 @@ static int exprCodeInlineFunction(
  return target;
}

+
/*
+
** Check to see if pExpr is one of the indexed expressions on pParse->pIdxExpr.
+
** If it is, then resolve the expression by reading from the index and
+
** return the register into which the value has been read.  If pExpr is
+
** not an indexed expression, then return negative.
+
*/
+
static SQLITE_NOINLINE int sqlite3IndexedExprLookup(
+
  Parse *pParse,   /* The parsing context */
+
  Expr *pExpr,     /* The expression to potentially bypass */
+
  int target       /* Where to store the result of the expression */
+
){
+
  IndexedExpr *p;
+
  Vdbe *v;
+
  for(p=pParse->pIdxExpr; p; p=p->pIENext){
+
    int iDataCur = p->iDataCur;
+
    if( iDataCur<0 ) continue;
+
    if( pParse->iSelfTab ){
+
      if( p->iDataCur!=pParse->iSelfTab-1 ) continue;
+
      iDataCur = -1;
+
    }
+
    if( sqlite3ExprCompare(0, pExpr, p->pExpr, iDataCur)!=0 ) continue;
+
    v = pParse->pVdbe;
+
    assert( v!=0 );
+
    if( p->bMaybeNullRow ){
+
      /* If the index is on a NULL row due to an outer join, then we
+
      ** cannot extract the value from the index.  The value must be
+
      ** computed using the original expression. */
+
      int addr = sqlite3VdbeCurrentAddr(v);
+
      sqlite3VdbeAddOp3(v, OP_IfNullRow, p->iIdxCur, addr+3, target);
+
      VdbeCoverage(v);
+
      sqlite3VdbeAddOp3(v, OP_Column, p->iIdxCur, p->iIdxCol, target);
+
      VdbeComment((v, "%s expr-column %d", p->zIdxName, p->iIdxCol));
+
      sqlite3VdbeGoto(v, 0);
+
      p = pParse->pIdxExpr;
+
      pParse->pIdxExpr = 0;
+
      sqlite3ExprCode(pParse, pExpr, target);
+
      pParse->pIdxExpr = p;
+
      sqlite3VdbeJumpHere(v, addr+2);
+
    }else{
+
      sqlite3VdbeAddOp3(v, OP_Column, p->iIdxCur, p->iIdxCol, target);
+
      VdbeComment((v, "%s expr-column %d", p->zIdxName, p->iIdxCol));
+
    }
+
    return target;
+
  }
+
  return -1;  /* Not found */
+
}
+


/*
** Generate code into the current Vdbe to evaluate the given
@@ -106509,6 +109157,11 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target)
expr_code_doover:
  if( pExpr==0 ){
    op = TK_NULL;
+
  }else if( pParse->pIdxExpr!=0
+
   && !ExprHasProperty(pExpr, EP_Leaf)
+
   && (r1 = sqlite3IndexedExprLookup(pParse, pExpr, target))>=0
+
  ){
+
    return r1;
  }else{
    assert( !ExprHasVVAProperty(pExpr,EP_Immutable) );
    op = pExpr->op;
@@ -106529,7 +109182,7 @@ expr_code_doover:
                              pCol->iSorterColumn, target);
        if( pCol->iColumn<0 ){
          VdbeComment((v,"%s.rowid",pTab->zName));
-
        }else{
+
        }else if( ALWAYS(pTab!=0) ){
          VdbeComment((v,"%s.%s",
              pTab->zName, pTab->aCol[pCol->iColumn].zCnName));
          if( pTab->aCol[pCol->iColumn].affinity==SQLITE_AFF_REAL ){
@@ -106554,11 +109207,8 @@ expr_code_doover:
        int aff;
        iReg = sqlite3ExprCodeTarget(pParse, pExpr->pLeft,target);
        assert( ExprUseYTab(pExpr) );
-
        if( pExpr->y.pTab ){
-
          aff = sqlite3TableColumnAffinity(pExpr->y.pTab, pExpr->iColumn);
-
        }else{
-
          aff = pExpr->affExpr;
-
        }
+
        assert( pExpr->y.pTab!=0 );
+
        aff = sqlite3TableColumnAffinity(pExpr->y.pTab, pExpr->iColumn);
        if( aff>SQLITE_AFF_BLOB ){
          static const char zAff[] = "B\000C\000D\000E";
          assert( SQLITE_AFF_BLOB=='A' );
@@ -106620,12 +109270,10 @@ expr_code_doover:
        }
      }
      assert( ExprUseYTab(pExpr) );
+
      assert( pExpr->y.pTab!=0 );
      iReg = sqlite3ExprCodeGetColumn(pParse, pExpr->y.pTab,
                               pExpr->iColumn, iTab, target,
                               pExpr->op2);
-
      if( pExpr->y.pTab==0 && pExpr->affExpr==SQLITE_AFF_REAL ){
-
        sqlite3VdbeAddOp1(v, OP_RealAffinity, iReg);
-
      }
      return iReg;
    }
    case TK_INTEGER: {
@@ -106954,20 +109602,8 @@ expr_code_doover:
        if( !pColl ) pColl = db->pDfltColl;
        sqlite3VdbeAddOp4(v, OP_CollSeq, 0, 0, 0, (char *)pColl, P4_COLLSEQ);
      }
-
#ifdef SQLITE_ENABLE_OFFSET_SQL_FUNC
-
      if( (pDef->funcFlags & SQLITE_FUNC_OFFSET)!=0 && ALWAYS(pFarg!=0) ){
-
        Expr *pArg = pFarg->a[0].pExpr;
-
        if( pArg->op==TK_COLUMN ){
-
          sqlite3VdbeAddOp3(v, OP_Offset, pArg->iTable, pArg->iColumn, target);
-
        }else{
-
          sqlite3VdbeAddOp2(v, OP_Null, 0, target);
-
        }
-
      }else
-
#endif
-
      {
-
        sqlite3VdbeAddFunctionCall(pParse, constMask, r1, target, nFarg,
-
                                   pDef, pExpr->op2);
-
      }
+
      sqlite3VdbeAddFunctionCall(pParse, constMask, r1, target, nFarg,
+
                                 pDef, pExpr->op2);
      if( nFarg ){
        if( constMask==0 ){
          sqlite3ReleaseTempRange(pParse, r1, nFarg);
@@ -106997,16 +109633,18 @@ expr_code_doover:
    }
    case TK_SELECT_COLUMN: {
      int n;
-
      if( pExpr->pLeft->iTable==0 ){
-
        pExpr->pLeft->iTable = sqlite3CodeSubselect(pParse, pExpr->pLeft);
+
      Expr *pLeft = pExpr->pLeft;
+
      if( pLeft->iTable==0 || pParse->withinRJSubrtn > pLeft->op2 ){
+
        pLeft->iTable = sqlite3CodeSubselect(pParse, pLeft);
+
        pLeft->op2 = pParse->withinRJSubrtn;
      }
-
      assert( pExpr->pLeft->op==TK_SELECT || pExpr->pLeft->op==TK_ERROR );
-
      n = sqlite3ExprVectorSize(pExpr->pLeft);
+
      assert( pLeft->op==TK_SELECT || pLeft->op==TK_ERROR );
+
      n = sqlite3ExprVectorSize(pLeft);
      if( pExpr->iTable!=n ){
        sqlite3ErrorMsg(pParse, "%d columns assigned %d values",
                                pExpr->iTable, n);
      }
-
      return pExpr->pLeft->iTable + pExpr->iColumn;
+
      return pLeft->iTable + pExpr->iColumn;
    }
    case TK_IN: {
      int destIfFalse = sqlite3VdbeMakeLabel(pParse);
@@ -107037,8 +109675,24 @@ expr_code_doover:
      exprCodeBetween(pParse, pExpr, target, 0, 0);
      return target;
    }
+
    case TK_COLLATE: {
+
      if( !ExprHasProperty(pExpr, EP_Collate)
+
       && ALWAYS(pExpr->pLeft)
+
       && pExpr->pLeft->op==TK_FUNCTION
+
      ){
+
        inReg = sqlite3ExprCodeTarget(pParse, pExpr->pLeft, target);
+
        if( inReg!=target ){
+
          sqlite3VdbeAddOp2(v, OP_SCopy, inReg, target);
+
          inReg = target;
+
        }
+
        sqlite3VdbeAddOp1(v, OP_ClrSubtype, inReg);
+
        return inReg;
+
      }else{
+
        pExpr = pExpr->pLeft;
+
        goto expr_code_doover; /* 2018-04-28: Prevent deep recursion. */
+
      }
+
    }
    case TK_SPAN:
-
    case TK_COLLATE:
    case TK_UPLUS: {
      pExpr = pExpr->pLeft;
      goto expr_code_doover; /* 2018-04-28: Prevent deep recursion. OSSFuzz. */
@@ -107118,6 +109772,21 @@ expr_code_doover:
    case TK_IF_NULL_ROW: {
      int addrINR;
      u8 okConstFactor = pParse->okConstFactor;
+
      AggInfo *pAggInfo = pExpr->pAggInfo;
+
      if( pAggInfo ){
+
        assert( pExpr->iAgg>=0 && pExpr->iAgg<pAggInfo->nColumn );
+
        if( !pAggInfo->directMode ){
+
          inReg = pAggInfo->aCol[pExpr->iAgg].iMem;
+
          break;
+
        }
+
        if( pExpr->pAggInfo->useSortingIdx ){
+
          sqlite3VdbeAddOp3(v, OP_Column, pAggInfo->sortingIdxPTab,
+
                            pAggInfo->aCol[pExpr->iAgg].iSorterColumn,
+
                            target);
+
          inReg = target;
+
          break;
+
        }
+
      }
      addrINR = sqlite3VdbeAddOp1(v, OP_IfNullRow, pExpr->iTable);
      /* Temporarily disable factoring of constant expressions, since
      ** even though expressions may appear to be constant, they are not
@@ -107279,7 +109948,9 @@ SQLITE_PRIVATE int sqlite3ExprCodeRunJustOnce(
    struct ExprList_item *pItem;
    int i;
    for(pItem=p->a, i=p->nExpr; i>0; pItem++, i--){
-
      if( pItem->reusable && sqlite3ExprCompare(0,pItem->pExpr,pExpr,-1)==0 ){
+
      if( pItem->fg.reusable
+
       && sqlite3ExprCompare(0,pItem->pExpr,pExpr,-1)==0
+
      ){
        return pItem->u.iConstExprReg;
      }
    }
@@ -107302,7 +109973,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeRunJustOnce(
    p = sqlite3ExprListAppend(pParse, p, pExpr);
    if( p ){
       struct ExprList_item *pItem = &p->a[p->nExpr-1];
-
       pItem->reusable = regDest<0;
+
       pItem->fg.reusable = regDest<0;
       if( regDest<0 ) regDest = ++pParse->nMem;
       pItem->u.iConstExprReg = regDest;
    }
@@ -107436,7 +110107,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeExprList(
  for(pItem=pList->a, i=0; i<n; i++, pItem++){
    Expr *pExpr = pItem->pExpr;
#ifdef SQLITE_ENABLE_SORTER_REFERENCES
-
    if( pItem->bSorterRef ){
+
    if( pItem->fg.bSorterRef ){
      i--;
      n--;
    }else
@@ -107457,7 +110128,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeExprList(
      if( inReg!=target+i ){
        VdbeOp *pOp;
        if( copyOp==OP_Copy
-
         && (pOp=sqlite3VdbeGetOp(v, -1))->opcode==OP_Copy
+
         && (pOp=sqlite3VdbeGetLastOp(v))->opcode==OP_Copy
         && pOp->p1+pOp->p3+1==inReg
         && pOp->p2+pOp->p3+1==target+i
         && pOp->p5==0  /* The do-not-merge flag must be clear */
@@ -107530,8 +110201,8 @@ static void exprCodeBetween(
      ** so that the sqlite3ExprCodeTarget() routine will not attempt to move
      ** it into the Parse.pConstExpr list.  We should use a new bit for this,
      ** for clarity, but we are out of bits in the Expr.flags field so we
-
      ** have to reuse the EP_FromJoin bit.  Bummer. */
-
      pDel->flags |= EP_FromJoin;
+
      ** have to reuse the EP_OuterON bit.  Bummer. */
+
      pDel->flags |= EP_OuterON;
      sqlite3ExprCodeTarget(pParse, &exprAnd, dest);
    }
    sqlite3ReleaseTempReg(pParse, regFree1);
@@ -107656,6 +110327,7 @@ SQLITE_PRIVATE void sqlite3ExprIfTrue(Parse *pParse, Expr *pExpr, int dest, int
      assert( TK_ISNULL==OP_IsNull );   testcase( op==TK_ISNULL );
      assert( TK_NOTNULL==OP_NotNull ); testcase( op==TK_NOTNULL );
      r1 = sqlite3ExprCodeTemp(pParse, pExpr->pLeft, &regFree1);
+
      sqlite3VdbeTypeofColumn(v, r1);
      sqlite3VdbeAddOp2(v, op, r1, dest);
      VdbeCoverageIf(v, op==TK_ISNULL);
      VdbeCoverageIf(v, op==TK_NOTNULL);
@@ -107830,6 +110502,7 @@ SQLITE_PRIVATE void sqlite3ExprIfFalse(Parse *pParse, Expr *pExpr, int dest, int
    case TK_ISNULL:
    case TK_NOTNULL: {
      r1 = sqlite3ExprCodeTemp(pParse, pExpr->pLeft, &regFree1);
+
      sqlite3VdbeTypeofColumn(v, r1);
      sqlite3VdbeAddOp2(v, op, r1, dest);
      testcase( op==TK_ISNULL );   VdbeCoverageIf(v, op==TK_ISNULL);
      testcase( op==TK_NOTNULL );  VdbeCoverageIf(v, op==TK_NOTNULL);
@@ -107983,7 +110656,13 @@ SQLITE_PRIVATE int sqlite3ExprCompare(
    if( pB->op==TK_COLLATE && sqlite3ExprCompare(pParse, pA,pB->pLeft,iTab)<2 ){
      return 1;
    }
-
    return 2;
+
    if( pA->op==TK_AGG_COLUMN && pB->op==TK_COLUMN
+
     && pB->iTable<0 && pA->iTable==iTab
+
    ){
+
      /* fall through */
+
    }else{
+
      return 2;
+
    }
  }
  assert( !ExprHasProperty(pA, EP_IntValue) );
  assert( !ExprHasProperty(pB, EP_IntValue) );
@@ -108061,7 +110740,7 @@ SQLITE_PRIVATE int sqlite3ExprListCompare(const ExprList *pA, const ExprList *pB
    int res;
    Expr *pExprA = pA->a[i].pExpr;
    Expr *pExprB = pB->a[i].pExpr;
-
    if( pA->a[i].sortFlags!=pB->a[i].sortFlags ) return 1;
+
    if( pA->a[i].fg.sortFlags!=pB->a[i].fg.sortFlags ) return 1;
    if( (res = sqlite3ExprCompare(0, pExprA, pExprB, iTab)) ) return res;
  }
  return 0;
@@ -108216,7 +110895,7 @@ SQLITE_PRIVATE int sqlite3ExprImpliesExpr(
static int impliesNotNullRow(Walker *pWalker, Expr *pExpr){
  testcase( pExpr->op==TK_AGG_COLUMN );
  testcase( pExpr->op==TK_AGG_FUNCTION );
-
  if( ExprHasProperty(pExpr, EP_FromJoin) ) return WRC_Prune;
+
  if( ExprHasProperty(pExpr, EP_OuterON) ) return WRC_Prune;
  switch( pExpr->op ){
    case TK_ISNOT:
    case TK_ISNULL:
@@ -108285,10 +110964,10 @@ static int impliesNotNullRow(Walker *pWalker, Expr *pExpr){
      assert( pLeft->op!=TK_COLUMN || ExprUseYTab(pLeft) );
      assert( pRight->op!=TK_COLUMN || ExprUseYTab(pRight) );
      if( (pLeft->op==TK_COLUMN
-
           && pLeft->y.pTab!=0
+
           && ALWAYS(pLeft->y.pTab!=0)
           && IsVirtual(pLeft->y.pTab))
       || (pRight->op==TK_COLUMN
-
           && pRight->y.pTab!=0
+
           && ALWAYS(pRight->y.pTab!=0)
           && IsVirtual(pRight->y.pTab))
      ){
        return WRC_Prune;
@@ -108313,8 +110992,8 @@ static int impliesNotNullRow(Walker *pWalker, Expr *pExpr){
** False positives are not allowed, however.  A false positive may result
** in an incorrect answer.
**
-
** Terms of p that are marked with EP_FromJoin (and hence that come from
-
** the ON or USING clauses of LEFT JOINS) are excluded from the analysis.
+
** Terms of p that are marked with EP_OuterON (and hence that come from
+
** the ON or USING clauses of OUTER JOINS) are excluded from the analysis.
**
** This routine is used to check if a LEFT JOIN can be converted into
** an ordinary JOIN.  The p argument is the WHERE clause.  If the WHERE
@@ -108493,6 +111172,7 @@ static int exprRefToSrcList(Walker *pWalker, Expr *pExpr){
SQLITE_PRIVATE int sqlite3ReferencesSrcList(Parse *pParse, Expr *pExpr, SrcList *pSrcList){
  Walker w;
  struct RefSrcList x;
+
  assert( pParse->db!=0 );
  memset(&w, 0, sizeof(w));
  memset(&x, 0, sizeof(x));
  w.xExprCallback = exprRefToSrcList;
@@ -108509,7 +111189,7 @@ SQLITE_PRIVATE int sqlite3ReferencesSrcList(Parse *pParse, Expr *pExpr, SrcList
    sqlite3WalkExpr(&w, pExpr->y.pWin->pFilter);
  }
#endif
-
  sqlite3DbFree(pParse->db, x.aiExclude);
+
  if( x.aiExclude ) sqlite3DbNNFreeNN(pParse->db, x.aiExclude);
  if( w.eCode & 0x01 ){
    return 1;
  }else if( w.eCode ){
@@ -108540,8 +111220,8 @@ static int agginfoPersistExprCb(Walker *pWalker, Expr *pExpr){
    int iAgg = pExpr->iAgg;
    Parse *pParse = pWalker->pParse;
    sqlite3 *db = pParse->db;
-
    assert( pExpr->op==TK_AGG_COLUMN || pExpr->op==TK_AGG_FUNCTION );
-
    if( pExpr->op==TK_AGG_COLUMN ){
+
    if( pExpr->op!=TK_AGG_FUNCTION ){
+
      assert( pExpr->op==TK_AGG_COLUMN || pExpr->op==TK_IF_NULL_ROW );
      assert( iAgg>=0 && iAgg<pAggInfo->nColumn );
      if( pAggInfo->aCol[iAgg].pCExpr==pExpr ){
        pExpr = sqlite3ExprDup(db, pExpr, 0);
@@ -108551,6 +111231,7 @@ static int agginfoPersistExprCb(Walker *pWalker, Expr *pExpr){
        }
      }
    }else{
+
      assert( pExpr->op==TK_AGG_FUNCTION );
      assert( iAgg>=0 && iAgg<pAggInfo->nFunc );
      if( pAggInfo->aFunc[iAgg].pFExpr==pExpr ){
        pExpr = sqlite3ExprDup(db, pExpr, 0);
@@ -108621,10 +111302,12 @@ static int analyzeAggregate(Walker *pWalker, Expr *pExpr){

  assert( pNC->ncFlags & NC_UAggInfo );
  switch( pExpr->op ){
+
    case TK_IF_NULL_ROW:
    case TK_AGG_COLUMN:
    case TK_COLUMN: {
      testcase( pExpr->op==TK_AGG_COLUMN );
      testcase( pExpr->op==TK_COLUMN );
+
      testcase( pExpr->op==TK_IF_NULL_ROW );
      /* Check to see if the column is in one of the tables in the FROM
      ** clause of the aggregate query */
      if( ALWAYS(pSrcList!=0) ){
@@ -108642,8 +111325,10 @@ static int analyzeAggregate(Walker *pWalker, Expr *pExpr){
            int k;
            pCol = pAggInfo->aCol;
            for(k=0; k<pAggInfo->nColumn; k++, pCol++){
-
              if( pCol->iTable==pExpr->iTable &&
-
                  pCol->iColumn==pExpr->iColumn ){
+
              if( pCol->iTable==pExpr->iTable
+
               && pCol->iColumn==pExpr->iColumn
+
               && pExpr->op!=TK_IF_NULL_ROW
+
              ){
                break;
              }
            }
@@ -108658,15 +111343,17 @@ static int analyzeAggregate(Walker *pWalker, Expr *pExpr){
              pCol->iMem = ++pParse->nMem;
              pCol->iSorterColumn = -1;
              pCol->pCExpr = pExpr;
-
              if( pAggInfo->pGroupBy ){
+
              if( pAggInfo->pGroupBy && pExpr->op!=TK_IF_NULL_ROW ){
                int j, n;
                ExprList *pGB = pAggInfo->pGroupBy;
                struct ExprList_item *pTerm = pGB->a;
                n = pGB->nExpr;
                for(j=0; j<n; j++, pTerm++){
                  Expr *pE = pTerm->pExpr;
-
                  if( pE->op==TK_COLUMN && pE->iTable==pExpr->iTable &&
-
                      pE->iColumn==pExpr->iColumn ){
+
                  if( pE->op==TK_COLUMN
+
                   && pE->iTable==pExpr->iTable
+
                   && pE->iColumn==pExpr->iColumn
+
                  ){
                    pCol->iSorterColumn = j;
                    break;
                  }
@@ -108683,7 +111370,9 @@ static int analyzeAggregate(Walker *pWalker, Expr *pExpr){
            */
            ExprSetVVAProperty(pExpr, EP_NoReduce);
            pExpr->pAggInfo = pAggInfo;
-
            pExpr->op = TK_AGG_COLUMN;
+
            if( pExpr->op==TK_COLUMN ){
+
              pExpr->op = TK_AGG_COLUMN;
+
            }
            pExpr->iAgg = (i16)k;
            break;
          } /* endif pExpr->iTable==pItem->iCursor */
@@ -109728,11 +112417,10 @@ static void unmapColumnIdlistNames(
  Parse *pParse,
  const IdList *pIdList
){
-
  if( pIdList ){
-
    int ii;
-
    for(ii=0; ii<pIdList->nId; ii++){
-
      sqlite3RenameTokenRemap(pParse, 0, (const void*)pIdList->a[ii].zName);
-
    }
+
  int ii;
+
  assert( pIdList!=0 );
+
  for(ii=0; ii<pIdList->nId; ii++){
+
    sqlite3RenameTokenRemap(pParse, 0, (const void*)pIdList->a[ii].zName);
  }
}

@@ -109751,7 +112439,7 @@ static int renameUnmapSelectCb(Walker *pWalker, Select *p){
  if( ALWAYS(p->pEList) ){
    ExprList *pList = p->pEList;
    for(i=0; i<pList->nExpr; i++){
-
      if( pList->a[i].zEName && pList->a[i].eEName==ENAME_NAME ){
+
      if( pList->a[i].zEName && pList->a[i].fg.eEName==ENAME_NAME ){
        sqlite3RenameTokenRemap(pParse, 0, (void*)pList->a[i].zEName);
      }
    }
@@ -109760,8 +112448,11 @@ static int renameUnmapSelectCb(Walker *pWalker, Select *p){
    SrcList *pSrc = p->pSrc;
    for(i=0; i<pSrc->nSrc; i++){
      sqlite3RenameTokenRemap(pParse, 0, (void*)pSrc->a[i].zName);
-
      sqlite3WalkExpr(pWalker, pSrc->a[i].pOn);
-
      unmapColumnIdlistNames(pParse, pSrc->a[i].pUsing);
+
      if( pSrc->a[i].fg.isUsing==0 ){
+
        sqlite3WalkExpr(pWalker, pSrc->a[i].u3.pOn);
+
      }else{
+
        unmapColumnIdlistNames(pParse, pSrc->a[i].u3.pUsing);
+
      }
    }
  }

@@ -109797,7 +112488,7 @@ SQLITE_PRIVATE void sqlite3RenameExprlistUnmap(Parse *pParse, ExprList *pEList){
    sWalker.xExprCallback = renameUnmapExprCb;
    sqlite3WalkExprList(&sWalker, pEList);
    for(i=0; i<pEList->nExpr; i++){
-
      if( ALWAYS(pEList->a[i].eEName==ENAME_NAME) ){
+
      if( ALWAYS(pEList->a[i].fg.eEName==ENAME_NAME) ){
        sqlite3RenameTokenRemap(pParse, 0, (void*)pEList->a[i].zEName);
      }
    }
@@ -109955,7 +112646,7 @@ static void renameColumnElistNames(
    int i;
    for(i=0; i<pEList->nExpr; i++){
      const char *zName = pEList->a[i].zEName;
-
      if( ALWAYS(pEList->a[i].eEName==ENAME_NAME)
+
      if( ALWAYS(pEList->a[i].fg.eEName==ENAME_NAME)
       && ALWAYS(zName!=0)
       && 0==sqlite3_stricmp(zName, zOld)
      ){
@@ -110185,27 +112876,33 @@ static int renameResolveTrigger(Parse *pParse){
    if( rc==SQLITE_OK && pStep->zTarget ){
      SrcList *pSrc = sqlite3TriggerStepSrc(pParse, pStep);
      if( pSrc ){
-
        int i;
-
        for(i=0; i<pSrc->nSrc && rc==SQLITE_OK; i++){
-
          SrcItem *p = &pSrc->a[i];
-
          p->iCursor = pParse->nTab++;
-
          if( p->pSelect ){
-
            sqlite3SelectPrep(pParse, p->pSelect, 0);
-
            sqlite3ExpandSubquery(pParse, p);
-
            assert( i>0 );
-
            assert( pStep->pFrom->a[i-1].pSelect );
-
            sqlite3SelectPrep(pParse, pStep->pFrom->a[i-1].pSelect, 0);
-
          }else{
-
            p->pTab = sqlite3LocateTableItem(pParse, 0, p);
-
            if( p->pTab==0 ){
-
              rc = SQLITE_ERROR;
-
            }else{
-
              p->pTab->nTabRef++;
-
              rc = sqlite3ViewGetColumnNames(pParse, p->pTab);
+
        Select *pSel = sqlite3SelectNew(
+
            pParse, pStep->pExprList, pSrc, 0, 0, 0, 0, 0, 0
+
        );
+
        if( pSel==0 ){
+
          pStep->pExprList = 0;
+
          pSrc = 0;
+
          rc = SQLITE_NOMEM;
+
        }else{
+
          sqlite3SelectPrep(pParse, pSel, 0);
+
          rc = pParse->nErr ? SQLITE_ERROR : SQLITE_OK;
+
          assert( pStep->pExprList==0 || pStep->pExprList==pSel->pEList );
+
          assert( pSrc==pSel->pSrc );
+
          if( pStep->pExprList ) pSel->pEList = 0;
+
          pSel->pSrc = 0;
+
          sqlite3SelectDelete(db, pSel);
+
        }
+
        if( pStep->pFrom ){
+
          int i;
+
          for(i=0; i<pStep->pFrom->nSrc && rc==SQLITE_OK; i++){
+
            SrcItem *p = &pStep->pFrom->a[i];
+
            if( p->pSelect ){
+
              sqlite3SelectPrep(pParse, p->pSelect, 0);
            }
          }
        }
-
        if( rc==SQLITE_OK && db->mallocFailed ){
+

+
        if(  db->mallocFailed ){
          rc = SQLITE_NOMEM;
        }
        sNC.pSrcList = pSrc;
@@ -110657,6 +113354,15 @@ static void renameTableFunc(
              if( pStep->zTarget && 0==sqlite3_stricmp(pStep->zTarget, zOld) ){
                renameTokenFind(&sParse, &sCtx, pStep->zTarget);
              }
+
              if( pStep->pFrom ){
+
                int i;
+
                for(i=0; i<pStep->pFrom->nSrc; i++){
+
                  SrcItem *pItem = &pStep->pFrom->a[i];
+
                  if( 0==sqlite3_stricmp(pItem->zName, zOld) ){
+
                    renameTokenFind(&sParse, &sCtx, pItem->zName);
+
                  }
+
                }
+
              }
            }
          }
        }
@@ -111980,9 +114686,14 @@ static void statGet(
    **   * "WHERE a=? AND b=?" matches 2 rows.
    **
    ** If D is the count of distinct values and K is the total number of
-
    ** rows, then each estimate is computed as:
+
    ** rows, then each estimate is usually computed as:
    **
    **        I = (K+D-1)/D
+
    **
+
    ** In other words, I is K/D rounded up to the next whole integer.
+
    ** However, if I is between 1.0 and 1.1 (in other words if I is
+
    ** close to 1.0 but just a little larger) then do not round up but
+
    ** instead keep the I value at 1.0.
    */
    sqlite3_str sStat;   /* Text of the constructed "stat" line */
    int i;               /* Loop counter */
@@ -111993,6 +114704,7 @@ static void statGet(
    for(i=0; i<p->nKeyCol; i++){
      u64 nDistinct = p->current.anDLt[i] + 1;
      u64 iVal = (p->nRow + nDistinct - 1) / nDistinct;
+
      if( iVal==2 && p->nRow*10 <= nDistinct*11 ) iVal = 1;
      sqlite3_str_appendf(&sStat, " %llu", iVal);
      assert( p->current.anEq[i] );
    }
@@ -112080,6 +114792,7 @@ static void analyzeVdbeCommentIndexWithColumnName(
  if( NEVER(i==XN_ROWID) ){
    VdbeComment((v,"%s.rowid",pIdx->zName));
  }else if( i==XN_EXPR ){
+
    assert( pIdx->bHasExpr );
    VdbeComment((v,"%s.expr(%d)",pIdx->zName, k));
  }else{
    VdbeComment((v,"%s.%s", pIdx->zName, pIdx->pTable->aCol[i].zCnName));
@@ -112156,7 +114869,7 @@ static void analyzeOneTable(
    memcpy(pStat1->zName, "sqlite_stat1", 13);
    pStat1->nCol = 3;
    pStat1->iPKey = -1;
-
    sqlite3VdbeAddOp4(pParse->pVdbe, OP_Noop, 0, 0, 0,(char*)pStat1,P4_DYNBLOB);
+
    sqlite3VdbeAddOp4(pParse->pVdbe, OP_Noop, 0, 0, 0,(char*)pStat1,P4_DYNAMIC);
  }
#endif

@@ -113546,7 +116259,11 @@ static int fixSelectCb(Walker *p, Select *pSelect){
      pItem->fg.fromDDL = 1;
    }
#if !defined(SQLITE_OMIT_VIEW) || !defined(SQLITE_OMIT_TRIGGER)
-
    if( sqlite3WalkExpr(&pFix->w, pList->a[i].pOn) ) return WRC_Abort;
+
    if( pList->a[i].fg.isUsing==0
+
     && sqlite3WalkExpr(&pFix->w, pList->a[i].u3.pOn)
+
    ){
+
      return WRC_Abort;
+
    }
#endif
  }
  if( pSelect->pWith ){
@@ -114078,6 +116795,7 @@ SQLITE_PRIVATE int sqlite3DbMaskAllZero(yDbMask m){
SQLITE_PRIVATE void sqlite3FinishCoding(Parse *pParse){
  sqlite3 *db;
  Vdbe *v;
+
  int iDb, i;

  assert( pParse->pToplevel==0 );
  db = pParse->db;
@@ -114107,12 +116825,9 @@ SQLITE_PRIVATE void sqlite3FinishCoding(Parse *pParse){
    if( pParse->bReturning ){
      Returning *pReturning = pParse->u1.pReturning;
      int addrRewind;
-
      int i;
      int reg;

-
      if( NEVER(pReturning->nRetCol==0) ){
-
        assert( CORRUPT_DB );
-
      }else{
+
      if( pReturning->nRetCol ){
        sqlite3VdbeAddOp0(v, OP_FkCheck);
        addrRewind =
           sqlite3VdbeAddOp1(v, OP_Rewind, pReturning->iRetCur);
@@ -114146,76 +116861,69 @@ SQLITE_PRIVATE void sqlite3FinishCoding(Parse *pParse){
    ** transaction on each used database and to verify the schema cookie
    ** on each used database.
    */
-
    if( db->mallocFailed==0
-
     && (DbMaskNonZero(pParse->cookieMask) || pParse->pConstExpr)
-
    ){
-
      int iDb, i;
-
      assert( sqlite3VdbeGetOp(v, 0)->opcode==OP_Init );
-
      sqlite3VdbeJumpHere(v, 0);
-
      for(iDb=0; iDb<db->nDb; iDb++){
-
        Schema *pSchema;
-
        if( DbMaskTest(pParse->cookieMask, iDb)==0 ) continue;
-
        sqlite3VdbeUsesBtree(v, iDb);
-
        pSchema = db->aDb[iDb].pSchema;
-
        sqlite3VdbeAddOp4Int(v,
-
          OP_Transaction,                    /* Opcode */
-
          iDb,                               /* P1 */
-
          DbMaskTest(pParse->writeMask,iDb), /* P2 */
-
          pSchema->schema_cookie,            /* P3 */
-
          pSchema->iGeneration               /* P4 */
-
        );
-
        if( db->init.busy==0 ) sqlite3VdbeChangeP5(v, 1);
-
        VdbeComment((v,
-
              "usesStmtJournal=%d", pParse->mayAbort && pParse->isMultiWrite));
-
      }
+
    assert( pParse->nErr>0 || sqlite3VdbeGetOp(v, 0)->opcode==OP_Init );
+
    sqlite3VdbeJumpHere(v, 0);
+
    assert( db->nDb>0 );
+
    iDb = 0;
+
    do{
+
      Schema *pSchema;
+
      if( DbMaskTest(pParse->cookieMask, iDb)==0 ) continue;
+
      sqlite3VdbeUsesBtree(v, iDb);
+
      pSchema = db->aDb[iDb].pSchema;
+
      sqlite3VdbeAddOp4Int(v,
+
        OP_Transaction,                    /* Opcode */
+
        iDb,                               /* P1 */
+
        DbMaskTest(pParse->writeMask,iDb), /* P2 */
+
        pSchema->schema_cookie,            /* P3 */
+
        pSchema->iGeneration               /* P4 */
+
      );
+
      if( db->init.busy==0 ) sqlite3VdbeChangeP5(v, 1);
+
      VdbeComment((v,
+
            "usesStmtJournal=%d", pParse->mayAbort && pParse->isMultiWrite));
+
    }while( ++iDb<db->nDb );
#ifndef SQLITE_OMIT_VIRTUALTABLE
-
      for(i=0; i<pParse->nVtabLock; i++){
-
        char *vtab = (char *)sqlite3GetVTable(db, pParse->apVtabLock[i]);
-
        sqlite3VdbeAddOp4(v, OP_VBegin, 0, 0, 0, vtab, P4_VTAB);
-
      }
-
      pParse->nVtabLock = 0;
+
    for(i=0; i<pParse->nVtabLock; i++){
+
      char *vtab = (char *)sqlite3GetVTable(db, pParse->apVtabLock[i]);
+
      sqlite3VdbeAddOp4(v, OP_VBegin, 0, 0, 0, vtab, P4_VTAB);
+
    }
+
    pParse->nVtabLock = 0;
#endif

-
      /* Once all the cookies have been verified and transactions opened,
-
      ** obtain the required table-locks. This is a no-op unless the
-
      ** shared-cache feature is enabled.
-
      */
-
      codeTableLocks(pParse);
+
    /* Once all the cookies have been verified and transactions opened,
+
    ** obtain the required table-locks. This is a no-op unless the
+
    ** shared-cache feature is enabled.
+
    */
+
    codeTableLocks(pParse);

-
      /* Initialize any AUTOINCREMENT data structures required.
-
      */
-
      sqlite3AutoincrementBegin(pParse);
+
    /* Initialize any AUTOINCREMENT data structures required.
+
    */
+
    sqlite3AutoincrementBegin(pParse);

-
      /* Code constant expressions that where factored out of inner loops.
-
      **
-
      ** The pConstExpr list might also contain expressions that we simply
-
      ** want to keep around until the Parse object is deleted.  Such
-
      ** expressions have iConstExprReg==0.  Do not generate code for
-
      ** those expressions, of course.
-
      */
-
      if( pParse->pConstExpr ){
-
        ExprList *pEL = pParse->pConstExpr;
-
        pParse->okConstFactor = 0;
-
        for(i=0; i<pEL->nExpr; i++){
-
          int iReg = pEL->a[i].u.iConstExprReg;
-
          if( iReg>0 ){
-
            sqlite3ExprCode(pParse, pEL->a[i].pExpr, iReg);
-
          }
-
        }
+
    /* Code constant expressions that where factored out of inner loops.
+
    **
+
    ** The pConstExpr list might also contain expressions that we simply
+
    ** want to keep around until the Parse object is deleted.  Such
+
    ** expressions have iConstExprReg==0.  Do not generate code for
+
    ** those expressions, of course.
+
    */
+
    if( pParse->pConstExpr ){
+
      ExprList *pEL = pParse->pConstExpr;
+
      pParse->okConstFactor = 0;
+
      for(i=0; i<pEL->nExpr; i++){
+
        int iReg = pEL->a[i].u.iConstExprReg;
+
        sqlite3ExprCode(pParse, pEL->a[i].pExpr, iReg);
      }
+
    }

-
      if( pParse->bReturning ){
-
        Returning *pRet = pParse->u1.pReturning;
-
        if( NEVER(pRet->nRetCol==0) ){
-
          assert( CORRUPT_DB );
-
        }else{
-
          sqlite3VdbeAddOp2(v, OP_OpenEphemeral, pRet->iRetCur, pRet->nRetCol);
-
        }
+
    if( pParse->bReturning ){
+
      Returning *pRet = pParse->u1.pReturning;
+
      if( pRet->nRetCol ){
+
        sqlite3VdbeAddOp2(v, OP_OpenEphemeral, pRet->iRetCur, pRet->nRetCol);
      }
-

-
      /* Finally, jump back to the beginning of the executable code. */
-
      sqlite3VdbeGoto(v, 1);
    }
+

+
    /* Finally, jump back to the beginning of the executable code. */
+
    sqlite3VdbeGoto(v, 1);
  }

  /* Get the VDBE program ready for execution
@@ -114271,8 +116979,6 @@ SQLITE_PRIVATE void sqlite3NestedParse(Parse *pParse, const char *zFormat, ...){
  memset(PARSE_TAIL(pParse), 0, PARSE_TAIL_SZ);
  db->mDbFlags |= DBFLAG_PreferBuiltin;
  sqlite3RunParser(pParse, zSql);
-
  sqlite3DbFree(db, pParse->zErrMsg);
-
  pParse->zErrMsg = 0;
  db->mDbFlags = savedDbFlags;
  sqlite3DbFree(db, zSql);
  memcpy(PARSE_TAIL(pParse), saveBuf, PARSE_TAIL_SZ);
@@ -114402,7 +117108,7 @@ SQLITE_PRIVATE Table *sqlite3LocateTable(
    /* If zName is the not the name of a table in the schema created using
    ** CREATE, then check to see if it is the name of an virtual table that
    ** can be an eponymous virtual table. */
-
    if( pParse->disableVtab==0 && db->init.busy==0 ){
+
    if( (pParse->prepFlags & SQLITE_PREPARE_NO_VTAB)==0 && db->init.busy==0 ){
      Module *pMod = (Module*)sqlite3HashFind(&db->aModule, zName);
      if( pMod==0 && sqlite3_strnicmp(zName, "pragma_", 7)==0 ){
        pMod = sqlite3PragmaVtabRegister(db, zName);
@@ -114415,7 +117121,7 @@ SQLITE_PRIVATE Table *sqlite3LocateTable(
#endif
    if( flags & LOCATE_NOERR ) return 0;
    pParse->checkSchema = 1;
-
  }else if( IsVirtual(p) && pParse->disableVtab ){
+
  }else if( IsVirtual(p) && (pParse->prepFlags & SQLITE_PREPARE_NO_VTAB)!=0 ){
    p = 0;
  }

@@ -114724,16 +117430,17 @@ SQLITE_PRIVATE void sqlite3DeleteColumnNames(sqlite3 *db, Table *pTable){
  int i;
  Column *pCol;
  assert( pTable!=0 );
+
  assert( db!=0 );
  if( (pCol = pTable->aCol)!=0 ){
    for(i=0; i<pTable->nCol; i++, pCol++){
      assert( pCol->zCnName==0 || pCol->hName==sqlite3StrIHash(pCol->zCnName) );
      sqlite3DbFree(db, pCol->zCnName);
    }
-
    sqlite3DbFree(db, pTable->aCol);
+
    sqlite3DbNNFreeNN(db, pTable->aCol);
    if( IsOrdinaryTable(pTable) ){
      sqlite3ExprListDelete(db, pTable->u.tab.pDfltList);
    }
-
    if( db==0 || db->pnBytesFreed==0 ){
+
    if( db->pnBytesFreed==0 ){
      pTable->aCol = 0;
      pTable->nCol = 0;
      if( IsOrdinaryTable(pTable) ){
@@ -114770,7 +117477,8 @@ static void SQLITE_NOINLINE deleteTable(sqlite3 *db, Table *pTable){
  ** a Table object that was going to be marked ephemeral. So do not check
  ** that no lookaside memory is used in this case either. */
  int nLookaside = 0;
-
  if( db && !db->mallocFailed && (pTable->tabFlags & TF_Ephemeral)==0 ){
+
  assert( db!=0 );
+
  if( !db->mallocFailed && (pTable->tabFlags & TF_Ephemeral)==0 ){
    nLookaside = sqlite3LookasideUsed(db, 0);
  }
#endif
@@ -114780,7 +117488,7 @@ static void SQLITE_NOINLINE deleteTable(sqlite3 *db, Table *pTable){
    pNext = pIndex->pNext;
    assert( pIndex->pSchema==pTable->pSchema
         || (IsVirtual(pTable) && pIndex->idxType!=SQLITE_IDXTYPE_APPDEF) );
-
    if( (db==0 || db->pnBytesFreed==0) && !IsVirtual(pTable) ){
+
    if( db->pnBytesFreed==0 && !IsVirtual(pTable) ){
      char *zName = pIndex->zName;
      TESTONLY ( Index *pOld = ) sqlite3HashInsert(
         &pIndex->pSchema->idxHash, zName, 0
@@ -114817,8 +117525,9 @@ static void SQLITE_NOINLINE deleteTable(sqlite3 *db, Table *pTable){
}
SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3 *db, Table *pTable){
  /* Do not delete the table until the reference count reaches zero. */
+
  assert( db!=0 );
  if( !pTable ) return;
-
  if( ((!db || db->pnBytesFreed==0) && (--pTable->nTabRef)>0) ) return;
+
  if( db->pnBytesFreed==0 && (--pTable->nTabRef)>0 ) return;
  deleteTable(db, pTable);
}

@@ -115830,7 +118539,7 @@ SQLITE_PRIVATE void sqlite3AddPrimaryKey(
    pTab->keyConf = (u8)onError;
    assert( autoInc==0 || autoInc==1 );
    pTab->tabFlags |= autoInc*TF_Autoincrement;
-
    if( pList ) pParse->iPkSortOrder = pList->a[0].sortFlags;
+
    if( pList ) pParse->iPkSortOrder = pList->a[0].fg.sortFlags;
    (void)sqlite3HasExplicitNulls(pParse, pList);
  }else if( autoInc ){
#ifndef SQLITE_OMIT_AUTOINCREMENT
@@ -116222,7 +118931,8 @@ static int isDupColumn(Index *pIdx, int nKey, Index *pPk, int iCol){
/* Recompute the colNotIdxed field of the Index.
**
** colNotIdxed is a bitmask that has a 0 bit representing each indexed
-
** columns that are within the first 63 columns of the table.  The
+
** columns that are within the first 63 columns of the table and a 1 for
+
** all other bits (all columns that are not in the index).  The
** high-order bit of colNotIdxed is always 1.  All unindexed columns
** of the table have a 1.
**
@@ -116250,7 +118960,7 @@ static void recomputeColumnsNotIndexed(Index *pIdx){
    }
  }
  pIdx->colNotIdxed = ~m;
-
  assert( (pIdx->colNotIdxed>>63)==1 );
+
  assert( (pIdx->colNotIdxed>>63)==1 );  /* See note-20221022-a */
}

/*
@@ -116324,7 +119034,7 @@ static void convertToWithoutRowidTable(Parse *pParse, Table *pTab){
    if( IN_RENAME_OBJECT ){
      sqlite3RenameTokenRemap(pParse, pList->a[0].pExpr, &pTab->iPKey);
    }
-
    pList->a[0].sortFlags = pParse->iPkSortOrder;
+
    pList->a[0].fg.sortFlags = pParse->iPkSortOrder;
    assert( pParse->pNewTable==pTab );
    pTab->iPKey = -1;
    sqlite3CreateIndex(pParse, 0, 0, 0, pList, pTab->keyConf, 0, 0, 0, 0,
@@ -116991,11 +119701,10 @@ create_view_fail:
** the columns of the view in the pTable structure.  Return the number
** of errors.  If an error is seen leave an error message in pParse->zErrMsg.
*/
-
SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){
+
static SQLITE_NOINLINE int viewGetColumnNames(Parse *pParse, Table *pTable){
  Table *pSelTab;   /* A fake table from which we get the result set */
  Select *pSel;     /* Copy of the SELECT that implements the view */
  int nErr = 0;     /* Number of errors encountered */
-
  int n;            /* Temporarily holds the number of cursors assigned */
  sqlite3 *db = pParse->db;  /* Database connection for malloc errors */
#ifndef SQLITE_OMIT_VIRTUALTABLE
  int rc;
@@ -117017,9 +119726,10 @@ SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){

#ifndef SQLITE_OMIT_VIEW
  /* A positive nCol means the columns names for this view are
-
  ** already known.
+
  ** already known.  This routine is not called unless either the
+
  ** table is virtual or nCol is zero.
  */
-
  if( pTable->nCol>0 ) return 0;
+
  assert( pTable->nCol<=0 );

  /* A negative nCol is a special marker meaning that we are currently
  ** trying to compute the column names.  If we enter this routine with
@@ -117053,8 +119763,9 @@ SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){
  pSel = sqlite3SelectDup(db, pTable->u.view.pSelect, 0);
  if( pSel ){
    u8 eParseMode = pParse->eParseMode;
+
    int nTab = pParse->nTab;
+
    int nSelect = pParse->nSelect;
    pParse->eParseMode = PARSE_MODE_NORMAL;
-
    n = pParse->nTab;
    sqlite3SrcListAssignCursors(pParse, pSel->pSrc);
    pTable->nCol = -1;
    DisableLookaside;
@@ -117066,7 +119777,8 @@ SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){
#else
    pSelTab = sqlite3ResultSetOfSelect(pParse, pSel, SQLITE_AFF_NONE);
#endif
-
    pParse->nTab = n;
+
    pParse->nTab = nTab;
+
    pParse->nSelect = nSelect;
    if( pSelTab==0 ){
      pTable->nCol = 0;
      nErr++;
@@ -117113,6 +119825,11 @@ SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){
#endif /* SQLITE_OMIT_VIEW */
  return nErr;
}
+
SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){
+
  assert( pTable!=0 );
+
  if( !IsVirtual(pTable) && pTable->nCol>0 ) return 0;
+
  return viewGetColumnNames(pParse, pTable);
+
}
#endif /* !defined(SQLITE_OMIT_VIEW) || !defined(SQLITE_OMIT_VIRTUALTABLE) */

#ifndef SQLITE_OMIT_VIEW
@@ -117811,8 +120528,8 @@ SQLITE_PRIVATE int sqlite3HasExplicitNulls(Parse *pParse, ExprList *pList){
  if( pList ){
    int i;
    for(i=0; i<pList->nExpr; i++){
-
      if( pList->a[i].bNulls ){
-
        u8 sf = pList->a[i].sortFlags;
+
      if( pList->a[i].fg.bNulls ){
+
        u8 sf = pList->a[i].fg.sortFlags;
        sqlite3ErrorMsg(pParse, "unsupported use of NULLS %s",
            (sf==0 || sf==3) ? "FIRST" : "LAST"
        );
@@ -117978,7 +120695,7 @@ SQLITE_PRIVATE void sqlite3CreateIndex(
    }
    if( !IN_RENAME_OBJECT ){
      if( !db->init.busy ){
-
        if( sqlite3FindTable(db, zName, 0)!=0 ){
+
        if( sqlite3FindTable(db, zName, pDb->zDbSName)!=0 ){
          sqlite3ErrorMsg(pParse, "there is already a table named %s", zName);
          goto exit_create_index;
        }
@@ -118131,6 +120848,7 @@ SQLITE_PRIVATE void sqlite3CreateIndex(
      j = XN_EXPR;
      pIndex->aiColumn[i] = XN_EXPR;
      pIndex->uniqNotNull = 0;
+
      pIndex->bHasExpr = 1;
    }else{
      j = pCExpr->iColumn;
      assert( j<=0x7fff );
@@ -118142,6 +120860,7 @@ SQLITE_PRIVATE void sqlite3CreateIndex(
        }
        if( pTab->aCol[j].colFlags & COLFLAG_VIRTUAL ){
          pIndex->bHasVCol = 1;
+
          pIndex->bHasExpr = 1;
        }
      }
      pIndex->aiColumn[i] = (i16)j;
@@ -118165,7 +120884,7 @@ SQLITE_PRIVATE void sqlite3CreateIndex(
      goto exit_create_index;
    }
    pIndex->azColl[i] = zColl;
-
    requestedSortOrder = pListItem->sortFlags & sortOrderMask;
+
    requestedSortOrder = pListItem->fg.sortFlags & sortOrderMask;
    pIndex->aSortOrder[i] = (u8)requestedSortOrder;
  }

@@ -118608,18 +121327,17 @@ SQLITE_PRIVATE IdList *sqlite3IdListAppend(Parse *pParse, IdList *pList, Token *
  if( pList==0 ){
    pList = sqlite3DbMallocZero(db, sizeof(IdList) );
    if( pList==0 ) return 0;
+
  }else{
+
    IdList *pNew;
+
    pNew = sqlite3DbRealloc(db, pList,
+
                 sizeof(IdList) + pList->nId*sizeof(pList->a));
+
    if( pNew==0 ){
+
      sqlite3IdListDelete(db, pList);
+
      return 0;
+
    }
+
    pList = pNew;
  }
-
  pList->a = sqlite3ArrayAllocate(
-
      db,
-
      pList->a,
-
      sizeof(pList->a[0]),
-
      &pList->nId,
-
      &i
-
  );
-
  if( i<0 ){
-
    sqlite3IdListDelete(db, pList);
-
    return 0;
-
  }
+
  i = pList->nId++;
  pList->a[i].zName = sqlite3NameFromToken(db, pToken);
  if( IN_RENAME_OBJECT && pList->a[i].zName ){
    sqlite3RenameTokenMap(pParse, (void*)pList->a[i].zName, pToken);
@@ -118632,12 +121350,13 @@ SQLITE_PRIVATE IdList *sqlite3IdListAppend(Parse *pParse, IdList *pList, Token *
*/
SQLITE_PRIVATE void sqlite3IdListDelete(sqlite3 *db, IdList *pList){
  int i;
+
  assert( db!=0 );
  if( pList==0 ) return;
+
  assert( pList->eU4!=EU4_EXPR ); /* EU4_EXPR mode is not currently used */
  for(i=0; i<pList->nId; i++){
    sqlite3DbFree(db, pList->a[i].zName);
  }
-
  sqlite3DbFree(db, pList->a);
-
  sqlite3DbFreeNN(db, pList);
+
  sqlite3DbNNFreeNN(db, pList);
}

/*
@@ -118646,7 +121365,7 @@ SQLITE_PRIVATE void sqlite3IdListDelete(sqlite3 *db, IdList *pList){
*/
SQLITE_PRIVATE int sqlite3IdListIndex(IdList *pList, const char *zName){
  int i;
-
  if( pList==0 ) return -1;
+
  assert( pList!=0 );
  for(i=0; i<pList->nId; i++){
    if( sqlite3StrICmp(pList->a[i].zName, zName)==0 ) return i;
  }
@@ -118840,19 +121559,23 @@ SQLITE_PRIVATE void sqlite3SrcListAssignCursors(Parse *pParse, SrcList *pList){
SQLITE_PRIVATE void sqlite3SrcListDelete(sqlite3 *db, SrcList *pList){
  int i;
  SrcItem *pItem;
+
  assert( db!=0 );
  if( pList==0 ) return;
  for(pItem=pList->a, i=0; i<pList->nSrc; i++, pItem++){
-
    if( pItem->zDatabase ) sqlite3DbFreeNN(db, pItem->zDatabase);
-
    sqlite3DbFree(db, pItem->zName);
-
    if( pItem->zAlias ) sqlite3DbFreeNN(db, pItem->zAlias);
+
    if( pItem->zDatabase ) sqlite3DbNNFreeNN(db, pItem->zDatabase);
+
    if( pItem->zName ) sqlite3DbNNFreeNN(db, pItem->zName);
+
    if( pItem->zAlias ) sqlite3DbNNFreeNN(db, pItem->zAlias);
    if( pItem->fg.isIndexedBy ) sqlite3DbFree(db, pItem->u1.zIndexedBy);
    if( pItem->fg.isTabFunc ) sqlite3ExprListDelete(db, pItem->u1.pFuncArg);
    sqlite3DeleteTable(db, pItem->pTab);
    if( pItem->pSelect ) sqlite3SelectDelete(db, pItem->pSelect);
-
    if( pItem->pOn ) sqlite3ExprDelete(db, pItem->pOn);
-
    if( pItem->pUsing ) sqlite3IdListDelete(db, pItem->pUsing);
+
    if( pItem->fg.isUsing ){
+
      sqlite3IdListDelete(db, pItem->u3.pUsing);
+
    }else if( pItem->u3.pOn ){
+
      sqlite3ExprDelete(db, pItem->u3.pOn);
+
    }
  }
-
  sqlite3DbFreeNN(db, pList);
+
  sqlite3DbNNFreeNN(db, pList);
}

/*
@@ -118878,14 +121601,13 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListAppendFromTerm(
  Token *pDatabase,       /* Name of the database containing pTable */
  Token *pAlias,          /* The right-hand side of the AS subexpression */
  Select *pSubquery,      /* A subquery used in place of a table name */
-
  Expr *pOn,              /* The ON clause of a join */
-
  IdList *pUsing          /* The USING clause of a join */
+
  OnOrUsing *pOnUsing     /* Either the ON clause or the USING clause */
){
  SrcItem *pItem;
  sqlite3 *db = pParse->db;
-
  if( !p && (pOn || pUsing) ){
+
  if( !p && pOnUsing!=0 && (pOnUsing->pOn || pOnUsing->pUsing) ){
    sqlite3ErrorMsg(pParse, "a JOIN clause is required before %s",
-
      (pOn ? "ON" : "USING")
+
      (pOnUsing->pOn ? "ON" : "USING")
    );
    goto append_from_error;
  }
@@ -118905,15 +121627,27 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListAppendFromTerm(
  if( pAlias->n ){
    pItem->zAlias = sqlite3NameFromToken(db, pAlias);
  }
-
  pItem->pSelect = pSubquery;
-
  pItem->pOn = pOn;
-
  pItem->pUsing = pUsing;
+
  if( pSubquery ){
+
    pItem->pSelect = pSubquery;
+
    if( pSubquery->selFlags & SF_NestedFrom ){
+
      pItem->fg.isNestedFrom = 1;
+
    }
+
  }
+
  assert( pOnUsing==0 || pOnUsing->pOn==0 || pOnUsing->pUsing==0 );
+
  assert( pItem->fg.isUsing==0 );
+
  if( pOnUsing==0 ){
+
    pItem->u3.pOn = 0;
+
  }else if( pOnUsing->pUsing ){
+
    pItem->fg.isUsing = 1;
+
    pItem->u3.pUsing = pOnUsing->pUsing;
+
  }else{
+
    pItem->u3.pOn = pOnUsing->pOn;
+
  }
  return p;

append_from_error:
  assert( p==0 );
-
  sqlite3ExprDelete(db, pOn);
-
  sqlite3IdListDelete(db, pUsing);
+
  sqlite3ClearOnOrUsing(db, pOnUsing);
  sqlite3SelectDelete(db, pSubquery);
  return 0;
}
@@ -118958,6 +121692,7 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListAppendList(Parse *pParse, SrcList *p1, Src
      p1 = pNew;
      memcpy(&p1->a[1], p2->a, p2->nSrc*sizeof(SrcItem));
      sqlite3DbFree(pParse->db, p2);
+
      p1->a[0].fg.jointype |= (JT_LTORJ & p1->a[1].fg.jointype);
    }
  }
  return p1;
@@ -118994,14 +121729,34 @@ SQLITE_PRIVATE void sqlite3SrcListFuncArgs(Parse *pParse, SrcList *p, ExprList *
** The operator is "natural cross join".  The A and B operands are stored
** in p->a[0] and p->a[1], respectively.  The parser initially stores the
** operator with A.  This routine shifts that operator over to B.
+
**
+
** Additional changes:
+
**
+
**   *   All tables to the left of the right-most RIGHT JOIN are tagged with
+
**       JT_LTORJ (mnemonic: Left Table Of Right Join) so that the
+
**       code generator can easily tell that the table is part of
+
**       the left operand of at least one RIGHT JOIN.
*/
-
SQLITE_PRIVATE void sqlite3SrcListShiftJoinType(SrcList *p){
-
  if( p ){
-
    int i;
-
    for(i=p->nSrc-1; i>0; i--){
-
      p->a[i].fg.jointype = p->a[i-1].fg.jointype;
-
    }
+
SQLITE_PRIVATE void sqlite3SrcListShiftJoinType(Parse *pParse, SrcList *p){
+
  (void)pParse;
+
  if( p && p->nSrc>1 ){
+
    int i = p->nSrc-1;
+
    u8 allFlags = 0;
+
    do{
+
      allFlags |= p->a[i].fg.jointype = p->a[i-1].fg.jointype;
+
    }while( (--i)>0 );
    p->a[0].fg.jointype = 0;
+

+
    /* All terms to the left of a RIGHT JOIN should be tagged with the
+
    ** JT_LTORJ flags */
+
    if( allFlags & JT_RIGHT ){
+
      for(i=p->nSrc-1; ALWAYS(i>0) && (p->a[i].fg.jointype&JT_RIGHT)==0; i--){}
+
      i--;
+
      assert( i>=0 );
+
      do{
+
        p->a[i].fg.jointype |= JT_LTORJ;
+
      }while( (--i)>=0 );
+
    }
  }
}

@@ -120072,19 +122827,21 @@ SQLITE_PRIVATE void sqlite3SchemaClear(void *p){
  Hash temp2;
  HashElem *pElem;
  Schema *pSchema = (Schema *)p;
+
  sqlite3 xdb;

+
  memset(&xdb, 0, sizeof(xdb));
  temp1 = pSchema->tblHash;
  temp2 = pSchema->trigHash;
  sqlite3HashInit(&pSchema->trigHash);
  sqlite3HashClear(&pSchema->idxHash);
  for(pElem=sqliteHashFirst(&temp2); pElem; pElem=sqliteHashNext(pElem)){
-
    sqlite3DeleteTrigger(0, (Trigger*)sqliteHashData(pElem));
+
    sqlite3DeleteTrigger(&xdb, (Trigger*)sqliteHashData(pElem));
  }
  sqlite3HashClear(&temp2);
  sqlite3HashInit(&pSchema->tblHash);
  for(pElem=sqliteHashFirst(&temp1); pElem; pElem=sqliteHashNext(pElem)){
    Table *pTab = sqliteHashData(pElem);
-
    sqlite3DeleteTable(0, pTab);
+
    sqlite3DeleteTable(&xdb, pTab);
  }
  sqlite3HashClear(&temp1);
  sqlite3HashClear(&pSchema->fkeyHash);
@@ -120183,18 +122940,42 @@ SQLITE_PRIVATE void sqlite3CodeChangeCount(Vdbe *v, int regCounter, const char *
**   1) It is a virtual table and no implementation of the xUpdate method
**      has been provided
**
-
**   2) It is a system table (i.e. sqlite_schema), this call is not
+
**   2) A trigger is currently being coded and the table is a virtual table
+
**      that is SQLITE_VTAB_DIRECTONLY or if PRAGMA trusted_schema=OFF and
+
**      the table is not SQLITE_VTAB_INNOCUOUS.
+
**
+
**   3) It is a system table (i.e. sqlite_schema), this call is not
**      part of a nested parse and writable_schema pragma has not
**      been specified
**
-
**   3) The table is a shadow table, the database connection is in
+
**   4) The table is a shadow table, the database connection is in
**      defensive mode, and the current sqlite3_prepare()
**      is for a top-level SQL statement.
*/
+
static int vtabIsReadOnly(Parse *pParse, Table *pTab){
+
  if( sqlite3GetVTable(pParse->db, pTab)->pMod->pModule->xUpdate==0 ){
+
    return 1;
+
  }
+

+
  /* Within triggers:
+
  **   *  Do not allow DELETE, INSERT, or UPDATE of SQLITE_VTAB_DIRECTONLY
+
  **      virtual tables
+
  **   *  Only allow DELETE, INSERT, or UPDATE of non-SQLITE_VTAB_INNOCUOUS
+
  **      virtual tables if PRAGMA trusted_schema=ON.
+
  */
+
  if( pParse->pToplevel!=0
+
   && pTab->u.vtab.p->eVtabRisk >
+
           ((pParse->db->flags & SQLITE_TrustedSchema)!=0)
+
  ){
+
    sqlite3ErrorMsg(pParse, "unsafe use of virtual table \"%s\"",
+
      pTab->zName);
+
  }
+
  return 0;
+
}
static int tabIsReadOnly(Parse *pParse, Table *pTab){
  sqlite3 *db;
  if( IsVirtual(pTab) ){
-
    return sqlite3GetVTable(pParse->db, pTab)->pMod->pModule->xUpdate==0;
+
    return vtabIsReadOnly(pParse, pTab);
  }
  if( (pTab->tabFlags & (TF_Readonly|TF_Shadow))==0 ) return 0;
  db = pParse->db;
@@ -120206,9 +122987,11 @@ static int tabIsReadOnly(Parse *pParse, Table *pTab){
}

/*
-
** Check to make sure the given table is writable.  If it is not
-
** writable, generate an error message and return 1.  If it is
-
** writable return 0;
+
** Check to make sure the given table is writable.
+
**
+
** If pTab is not writable  ->  generate an error message and return 1.
+
** If pTab is writable but other errors have occurred -> return 1.
+
** If pTab is writable and no prior errors -> return 0;
*/
SQLITE_PRIVATE int sqlite3IsReadOnly(Parse *pParse, Table *pTab, int viewOk){
  if( tabIsReadOnly(pParse, pTab) ){
@@ -120250,8 +123033,8 @@ SQLITE_PRIVATE void sqlite3MaterializeView(
    assert( pFrom->nSrc==1 );
    pFrom->a[0].zName = sqlite3DbStrDup(db, pView->zName);
    pFrom->a[0].zDatabase = sqlite3DbStrDup(db, db->aDb[iDb].zDbSName);
-
    assert( pFrom->a[0].pOn==0 );
-
    assert( pFrom->a[0].pUsing==0 );
+
    assert( pFrom->a[0].fg.isUsing==0 );
+
    assert( pFrom->a[0].u3.pOn==0 );
  }
  pSel = sqlite3SelectNew(pParse, 0, pFrom, pWhere, 0, 0, pOrderBy,
                          SF_IncludeHidden, pLimit);
@@ -120422,7 +123205,6 @@ SQLITE_PRIVATE void sqlite3DeleteFrom(
  assert( db->mallocFailed==0 );
  assert( pTabList->nSrc==1 );

-

  /* Locate the table which we want to delete.  This table has to be
  ** put in an SrcList structure because some of the subroutines we
  ** will be calling are designed to work with multiple tables and expect
@@ -120447,6 +123229,14 @@ SQLITE_PRIVATE void sqlite3DeleteFrom(
# define isView 0
#endif

+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x10000 ){
+
    sqlite3TreeViewLine(0, "In sqlite3Delete() at %s:%d", __FILE__, __LINE__);
+
    sqlite3TreeViewDelete(pParse->pWith, pTabList, pWhere,
+
                          pOrderBy, pLimit, pTrigger);
+
  }
+
#endif
+

#ifdef SQLITE_ENABLE_UPDATE_DELETE_LIMIT
  if( !isView ){
    pWhere = sqlite3LimitWhere(
@@ -120562,9 +123352,10 @@ SQLITE_PRIVATE void sqlite3DeleteFrom(
    }
    for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
      assert( pIdx->pSchema==pTab->pSchema );
-
      sqlite3VdbeAddOp2(v, OP_Clear, pIdx->tnum, iDb);
      if( IsPrimaryKeyIndex(pIdx) && !HasRowid(pTab) ){
-
        sqlite3VdbeChangeP3(v, -1, memCnt ? memCnt : -1);
+
        sqlite3VdbeAddOp3(v, OP_Clear, pIdx->tnum, iDb, memCnt ? memCnt : -1);
+
      }else{
+
        sqlite3VdbeAddOp2(v, OP_Clear, pIdx->tnum, iDb);
      }
    }
  }else
@@ -120764,7 +123555,7 @@ delete_from_cleanup:
  sqlite3ExprListDelete(db, pOrderBy);
  sqlite3ExprDelete(db, pLimit);
#endif
-
  sqlite3DbFree(db, aToOpen);
+
  if( aToOpen ) sqlite3DbNNFreeNN(db, aToOpen);
  return;
}
/* Make sure "isView" and other macros defined above are undefined. Otherwise
@@ -121847,7 +124638,7 @@ static int patternCompare(
      ** c but in the other case and search the input string for either
      ** c or cx.
      */
-
      if( c<=0x80 ){
+
      if( c<0x80 ){
        char zStop[3];
        int bMatch;
        if( noCase ){
@@ -121930,7 +124721,13 @@ static int patternCompare(
** non-zero if there is no match.
*/
SQLITE_API int sqlite3_strglob(const char *zGlobPattern, const char *zString){
-
  return patternCompare((u8*)zGlobPattern, (u8*)zString, &globInfo, '[');
+
  if( zString==0 ){
+
    return zGlobPattern!=0;
+
  }else if( zGlobPattern==0 ){
+
    return 1;
+
  }else {
+
    return patternCompare((u8*)zGlobPattern, (u8*)zString, &globInfo, '[');
+
  }
}

/*
@@ -121938,7 +124735,13 @@ SQLITE_API int sqlite3_strglob(const char *zGlobPattern, const char *zString){
** a miss - like strcmp().
*/
SQLITE_API int sqlite3_strlike(const char *zPattern, const char *zStr, unsigned int esc){
-
  return patternCompare((u8*)zPattern, (u8*)zStr, &likeInfoNorm, esc);
+
  if( zStr==0 ){
+
    return zPattern!=0;
+
  }else if( zPattern==0 ){
+
    return 1;
+
  }else{
+
    return patternCompare((u8*)zPattern, (u8*)zStr, &likeInfoNorm, esc);
+
  }
}

/*
@@ -123205,11 +126008,11 @@ static void logFunc(
    switch( SQLITE_PTR_TO_INT(sqlite3_user_data(context)) ){
      case 1:
        /* Convert from natural logarithm to log base 10 */
-
        ans *= 1.0/M_LN10;
+
        ans /= M_LN10;
        break;
      case 2:
        /* Convert from natural logarithm to log base 2 */
-
        ans *= 1.0/M_LN2;
+
        ans /= M_LN2;
        break;
      default:
        break;
@@ -123348,8 +126151,7 @@ SQLITE_PRIVATE void sqlite3RegisterBuiltinFunctions(void){
    INLINE_FUNC(likelihood,      2, INLINEFUNC_unlikely, SQLITE_FUNC_UNLIKELY),
    INLINE_FUNC(likely,          1, INLINEFUNC_unlikely, SQLITE_FUNC_UNLIKELY),
#ifdef SQLITE_ENABLE_OFFSET_SQL_FUNC
-
    {1, SQLITE_FUNC_BUILTIN|SQLITE_UTF8|SQLITE_FUNC_OFFSET|SQLITE_FUNC_TYPEOF,
-
     0, 0, noopFunc, 0, 0, 0, "sqlite_offset", {0} },
+
    INLINE_FUNC(sqlite_offset,   1, INLINEFUNC_sqlite_offset, 0 ),
#endif
    FUNCTION(ltrim,              1, 1, 0, trimFunc         ),
    FUNCTION(ltrim,              2, 1, 0, trimFunc         ),
@@ -123884,7 +126686,6 @@ static void fkLookupParent(
    }else{
      int nCol = pFKey->nCol;
      int regTemp = sqlite3GetTempRange(pParse, nCol);
-
      int regRec = sqlite3GetTempReg(pParse);

      sqlite3VdbeAddOp3(v, OP_OpenRead, iCur, pIdx->tnum, iDb);
      sqlite3VdbeSetP4KeyInfo(pParse, pIdx);
@@ -123924,11 +126725,10 @@ static void fkLookupParent(
        sqlite3VdbeGoto(v, iOk);
      }

-
      sqlite3VdbeAddOp4(v, OP_MakeRecord, regTemp, nCol, regRec,
+
      sqlite3VdbeAddOp4(v, OP_Affinity, regTemp, nCol, 0,
                        sqlite3IndexAffinityStr(pParse->db,pIdx), nCol);
-
      sqlite3VdbeAddOp4Int(v, OP_Found, iCur, iOk, regRec, 0); VdbeCoverage(v);
-

-
      sqlite3ReleaseTempReg(pParse, regRec);
+
      sqlite3VdbeAddOp4Int(v, OP_Found, iCur, iOk, regTemp, nCol);
+
      VdbeCoverage(v);
      sqlite3ReleaseTempRange(pParse, regTemp, nCol);
    }
  }
@@ -124030,14 +126830,10 @@ static Expr *exprTableColumn(
**   Operation | FK type   | Action taken
**   --------------------------------------------------------------------------
**   DELETE      immediate   Increment the "immediate constraint counter".
-
**                           Or, if the ON (UPDATE|DELETE) action is RESTRICT,
-
**                           throw a "FOREIGN KEY constraint failed" exception.
**
**   INSERT      immediate   Decrement the "immediate constraint counter".
**
**   DELETE      deferred    Increment the "deferred constraint counter".
-
**                           Or, if the ON (UPDATE|DELETE) action is RESTRICT,
-
**                           throw a "FOREIGN KEY constraint failed" exception.
**
**   INSERT      deferred    Decrement the "deferred constraint counter".
**
@@ -124685,9 +127481,9 @@ SQLITE_PRIVATE int sqlite3FkRequired(
**
** It returns a pointer to a Trigger structure containing a trigger
** equivalent to the ON UPDATE or ON DELETE action specified by pFKey.
-
** If the action is "NO ACTION" or "RESTRICT", then a NULL pointer is
-
** returned (these actions require no special handling by the triggers
-
** sub-system, code for them is created by fkScanChildren()).
+
** If the action is "NO ACTION" then a NULL pointer is returned (these actions
+
** require no special handling by the triggers sub-system, code for them is
+
** created by fkScanChildren()).
**
** For example, if pFKey is the foreign key and pTab is table "p" in
** the following schema:
@@ -124816,18 +127612,23 @@ static Trigger *fkActionTrigger(
    nFrom = sqlite3Strlen30(zFrom);

    if( action==OE_Restrict ){
+
      int iDb = sqlite3SchemaToIndex(db, pTab->pSchema);
      Token tFrom;
+
      Token tDb;
      Expr *pRaise;

      tFrom.z = zFrom;
      tFrom.n = nFrom;
+
      tDb.z = db->aDb[iDb].zDbSName;
+
      tDb.n = sqlite3Strlen30(tDb.z);
+

      pRaise = sqlite3Expr(db, TK_RAISE, "FOREIGN KEY constraint failed");
      if( pRaise ){
        pRaise->affExpr = OE_Abort;
      }
      pSelect = sqlite3SelectNew(pParse,
          sqlite3ExprListAppend(pParse, 0, pRaise),
-
          sqlite3SrcListAppend(pParse, 0, &tFrom, 0),
+
          sqlite3SrcListAppend(pParse, 0, &tDb, &tFrom),
          pWhere,
          0, 0, 0, 0, 0
      );
@@ -124934,11 +127735,12 @@ SQLITE_PRIVATE void sqlite3FkDelete(sqlite3 *db, Table *pTab){
  FKey *pNext;                    /* Copy of pFKey->pNextFrom */

  assert( IsOrdinaryTable(pTab) );
+
  assert( db!=0 );
  for(pFKey=pTab->u.tab.pFKey; pFKey; pFKey=pNext){
    assert( db==0 || sqlite3SchemaMutexHeld(db, 0, pTab->pSchema) );

    /* Remove the FK from the fkeyHash hash table. */
-
    if( !db || db->pnBytesFreed==0 ){
+
    if( db->pnBytesFreed==0 ){
      if( pFKey->pPrevTo ){
        pFKey->pPrevTo->pNextTo = pFKey->pNextTo;
      }else{
@@ -125068,6 +127870,7 @@ SQLITE_PRIVATE const char *sqlite3IndexAffinityStr(sqlite3 *db, Index *pIdx){
        aff = SQLITE_AFF_INTEGER;
      }else{
        assert( x==XN_EXPR );
+
        assert( pIdx->bHasExpr );
        assert( pIdx->aColExpr!=0 );
        aff = sqlite3ExprAffinity(pIdx->aColExpr->a[n].pExpr);
      }
@@ -125082,6 +127885,28 @@ SQLITE_PRIVATE const char *sqlite3IndexAffinityStr(sqlite3 *db, Index *pIdx){
}

/*
+
** Compute an affinity string for a table.   Space is obtained
+
** from sqlite3DbMalloc().  The caller is responsible for freeing
+
** the space when done.
+
*/
+
SQLITE_PRIVATE char *sqlite3TableAffinityStr(sqlite3 *db, const Table *pTab){
+
  char *zColAff;
+
  zColAff = (char *)sqlite3DbMallocRaw(db, pTab->nCol+1);
+
  if( zColAff ){
+
    int i, j;
+
    for(i=j=0; i<pTab->nCol; i++){
+
      if( (pTab->aCol[i].colFlags & COLFLAG_VIRTUAL)==0 ){
+
        zColAff[j++] = pTab->aCol[i].affinity;
+
      }
+
    }
+
    do{
+
      zColAff[j--] = 0;
+
    }while( j>=0 && zColAff[j]<=SQLITE_AFF_BLOB );
+
  }
+
  return zColAff;
+
}
+

+
/*
** Make changes to the evolving bytecode to do affinity transformations
** of values that are about to be gathered into a row for table pTab.
**
@@ -125122,7 +127947,7 @@ SQLITE_PRIVATE const char *sqlite3IndexAffinityStr(sqlite3 *db, Index *pIdx){
** Apply the type checking to that array of registers.
*/
SQLITE_PRIVATE void sqlite3TableAffinity(Vdbe *v, Table *pTab, int iReg){
-
  int i, j;
+
  int i;
  char *zColAff;
  if( pTab->tabFlags & TF_Strict ){
    if( iReg==0 ){
@@ -125131,7 +127956,7 @@ SQLITE_PRIVATE void sqlite3TableAffinity(Vdbe *v, Table *pTab, int iReg){
      ** OP_MakeRecord is found */
      VdbeOp *pPrev;
      sqlite3VdbeAppendP4(v, pTab, P4_TABLE);
-
      pPrev = sqlite3VdbeGetOp(v, -1);
+
      pPrev = sqlite3VdbeGetLastOp(v);
      assert( pPrev!=0 );
      assert( pPrev->opcode==OP_MakeRecord || sqlite3VdbeDb(v)->mallocFailed );
      pPrev->opcode = OP_TypeCheck;
@@ -125145,22 +127970,11 @@ SQLITE_PRIVATE void sqlite3TableAffinity(Vdbe *v, Table *pTab, int iReg){
  }
  zColAff = pTab->zColAff;
  if( zColAff==0 ){
-
    sqlite3 *db = sqlite3VdbeDb(v);
-
    zColAff = (char *)sqlite3DbMallocRaw(0, pTab->nCol+1);
+
    zColAff = sqlite3TableAffinityStr(0, pTab);
    if( !zColAff ){
-
      sqlite3OomFault(db);
+
      sqlite3OomFault(sqlite3VdbeDb(v));
      return;
    }
-

-
    for(i=j=0; i<pTab->nCol; i++){
-
      assert( pTab->aCol[i].affinity!=0 || sqlite3VdbeParser(v)->nErr>0 );
-
      if( (pTab->aCol[i].colFlags & COLFLAG_VIRTUAL)==0 ){
-
        zColAff[j++] = pTab->aCol[i].affinity;
-
      }
-
    }
-
    do{
-
      zColAff[j--] = 0;
-
    }while( j>=0 && zColAff[j]<=SQLITE_AFF_BLOB );
    pTab->zColAff = zColAff;
  }
  assert( zColAff!=0 );
@@ -125169,7 +127983,7 @@ SQLITE_PRIVATE void sqlite3TableAffinity(Vdbe *v, Table *pTab, int iReg){
    if( iReg ){
      sqlite3VdbeAddOp4(v, OP_Affinity, iReg, i, 0, zColAff, i);
    }else{
-
      assert( sqlite3VdbeGetOp(v, -1)->opcode==OP_MakeRecord
+
      assert( sqlite3VdbeGetLastOp(v)->opcode==OP_MakeRecord
              || sqlite3VdbeDb(v)->mallocFailed );
      sqlite3VdbeChangeP4(v, -1, zColAff, i);
    }
@@ -125255,7 +128069,7 @@ SQLITE_PRIVATE void sqlite3ComputeGeneratedColumns(
  */
  sqlite3TableAffinity(pParse->pVdbe, pTab, iRegStore);
  if( (pTab->tabFlags & TF_HasStored)!=0 ){
-
    pOp = sqlite3VdbeGetOp(pParse->pVdbe,-1);
+
    pOp = sqlite3VdbeGetLastOp(pParse->pVdbe);
    if( pOp->opcode==OP_Affinity ){
      /* Change the OP_Affinity argument to '@' (NONE) for all stored
      ** columns.  '@' is the no-op affinity and those columns have not
@@ -125737,6 +128551,14 @@ SQLITE_PRIVATE void sqlite3Insert(
#endif
  assert( (pTrigger && tmask) || (pTrigger==0 && tmask==0) );

+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x10000 ){
+
    sqlite3TreeViewLine(0, "In sqlite3Insert() at %s:%d", __FILE__, __LINE__);
+
    sqlite3TreeViewInsert(pParse->pWith, pTabList, pColumn, pSelect, pList,
+
                          onError, pUpsert, pTrigger);
+
  }
+
#endif
+

  /* If pTab is really a view, make sure it has been initialized.
  ** ViewGetColumnNames() is a no-op if pTab is not a view.
  */
@@ -125815,13 +128637,15 @@ SQLITE_PRIVATE void sqlite3Insert(
  */
  bIdListInOrder = (pTab->tabFlags & (TF_OOOHidden|TF_HasStored))==0;
  if( pColumn ){
+
    assert( pColumn->eU4!=EU4_EXPR );
+
    pColumn->eU4 = EU4_IDX;
    for(i=0; i<pColumn->nId; i++){
-
      pColumn->a[i].idx = -1;
+
      pColumn->a[i].u4.idx = -1;
    }
    for(i=0; i<pColumn->nId; i++){
      for(j=0; j<pTab->nCol; j++){
        if( sqlite3StrICmp(pColumn->a[i].zName, pTab->aCol[j].zCnName)==0 ){
-
          pColumn->a[i].idx = j;
+
          pColumn->a[i].u4.idx = j;
          if( i!=j ) bIdListInOrder = 0;
          if( j==pTab->iPKey ){
            ipkColumn = i;  assert( !withoutRowid );
@@ -126123,7 +128947,8 @@ SQLITE_PRIVATE void sqlite3Insert(
      }
    }
    if( pColumn ){
-
      for(j=0; j<pColumn->nId && pColumn->a[j].idx!=i; j++){}
+
      assert( pColumn->eU4==EU4_IDX );
+
      for(j=0; j<pColumn->nId && pColumn->a[j].u4.idx!=i; j++){}
      if( j>=pColumn->nId ){
        /* A column not named in the insert column list gets its
        ** default value */
@@ -126150,7 +128975,12 @@ SQLITE_PRIVATE void sqlite3Insert(
        sqlite3VdbeAddOp2(v, OP_SCopy, regFromSelect+k, iRegStore);
      }
    }else{
-
      sqlite3ExprCode(pParse, pList->a[k].pExpr, iRegStore);
+
      Expr *pX = pList->a[k].pExpr;
+
      int y = sqlite3ExprCodeTarget(pParse, pX, iRegStore);
+
      if( y!=iRegStore ){
+
        sqlite3VdbeAddOp2(v,
+
          ExprHasProperty(pX, EP_Subquery) ? OP_Copy : OP_SCopy, y, iRegStore);
+
      }
    }
  }

@@ -126287,7 +129117,9 @@ SQLITE_PRIVATE void sqlite3Insert(
      sqlite3GenerateConstraintChecks(pParse, pTab, aRegIdx, iDataCur, iIdxCur,
          regIns, 0, ipkColumn>=0, onError, endOfLoop, &isReplace, 0, pUpsert
      );
-
      sqlite3FkCheck(pParse, pTab, 0, regIns, 0, 0);
+
      if( db->flags & SQLITE_ForeignKeys ){
+
        sqlite3FkCheck(pParse, pTab, 0, regIns, 0, 0);
+
      }

      /* Set the OPFLAG_USESEEKRESULT flag if either (a) there are no REPLACE
      ** constraints or (b) there are no triggers and this table is not a
@@ -126371,7 +129203,7 @@ insert_cleanup:
  sqlite3UpsertDelete(db, pUpsert);
  sqlite3SelectDelete(db, pSelect);
  sqlite3IdListDelete(db, pColumn);
-
  sqlite3DbFree(db, aRegIdx);
+
  if( aRegIdx ) sqlite3DbNNFreeNN(db, aRegIdx);
}

/* Make sure "isView" and other macros defined above are undefined. Otherwise
@@ -127254,7 +130086,7 @@ SQLITE_PRIVATE void sqlite3GenerateConstraintChecks(
        if( isUpdate ){
          /* If currently processing the PRIMARY KEY of a WITHOUT ROWID
          ** table, only conflict if the new PRIMARY KEY values are actually
-
          ** different from the old.
+
          ** different from the old.  See TH3 withoutrowid04.test.
          **
          ** For a UNIQUE index, only conflict if the PRIMARY KEY values
          ** of the matched index row are different from the original PRIMARY
@@ -128598,9 +131430,9 @@ struct sqlite3_api_routines {
  const char *(*filename_journal)(const char*);
  const char *(*filename_wal)(const char*);
  /* Version 3.32.0 and later */
-
  char *(*create_filename)(const char*,const char*,const char*,
+
  const char *(*create_filename)(const char*,const char*,const char*,
                           int,const char**);
-
  void (*free_filename)(char*);
+
  void (*free_filename)(const char*);
  sqlite3_file *(*database_file_object)(const char*);
  /* Version 3.34.0 and later */
  int (*txn_state)(sqlite3*,const char*);
@@ -128618,6 +131450,14 @@ struct sqlite3_api_routines {
  int (*vtab_in)(sqlite3_index_info*,int,int);
  int (*vtab_in_first)(sqlite3_value*,sqlite3_value**);
  int (*vtab_in_next)(sqlite3_value*,sqlite3_value**);
+
  /* Version 3.39.0 and later */
+
  int (*deserialize)(sqlite3*,const char*,unsigned char*,
+
                     sqlite3_int64,sqlite3_int64,unsigned);
+
  unsigned char *(*serialize)(sqlite3*,const char *,sqlite3_int64*,
+
                              unsigned int);
+
  const char *(*db_name)(sqlite3*,int);
+
  /* Version 3.40.0 and later */
+
  int (*value_encoding)(sqlite3_value*);
};

/*
@@ -128936,6 +131776,14 @@ typedef int (*sqlite3_loadext_entry)(
#define sqlite3_vtab_in                sqlite3_api->vtab_in
#define sqlite3_vtab_in_first          sqlite3_api->vtab_in_first
#define sqlite3_vtab_in_next           sqlite3_api->vtab_in_next
+
/* Version 3.39.0 and later */
+
#ifndef SQLITE_OMIT_DESERIALIZE
+
#define sqlite3_deserialize            sqlite3_api->deserialize
+
#define sqlite3_serialize              sqlite3_api->serialize
+
#endif
+
#define sqlite3_db_name                sqlite3_api->db_name
+
/* Version 3.40.0 and later */
+
#define sqlite3_value_encoding         sqlite3_api->value_encoding
#endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */

#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
@@ -129427,11 +132275,30 @@ static const sqlite3_api_routines sqlite3Apis = {
  sqlite3_autovacuum_pages,
  /* Version 3.38.0 and later */
  sqlite3_error_offset,
+
#ifndef SQLITE_OMIT_VIRTUALTABLE
  sqlite3_vtab_rhs_value,
  sqlite3_vtab_distinct,
  sqlite3_vtab_in,
  sqlite3_vtab_in_first,
-
  sqlite3_vtab_in_next
+
  sqlite3_vtab_in_next,
+
#else
+
  0,
+
  0,
+
  0,
+
  0,
+
  0,
+
#endif
+
  /* Version 3.39.0 and later */
+
#ifndef SQLITE_OMIT_DESERIALIZE
+
  sqlite3_deserialize,
+
  sqlite3_serialize,
+
#else
+
  0,
+
  0,
+
#endif
+
  sqlite3_db_name,
+
  /* Version 3.40.0 and later */
+
  sqlite3_value_type
};

/* True if x is the directory separator character
@@ -130102,7 +132969,7 @@ static const PragmaName aPragmaName[] = {
#if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS)
 {/* zName:     */ "database_list",
  /* ePragTyp:  */ PragTyp_DATABASE_LIST,
-
  /* ePragFlg:  */ PragFlg_NeedSchema|PragFlg_Result0,
+
  /* ePragFlg:  */ PragFlg_Result0,
  /* ColNames:  */ 47, 3,
  /* iArg:      */ 0 },
#endif
@@ -130790,15 +133657,16 @@ static void pragmaFunclistLine(
  int isBuiltin,         /* True if this is a built-in function */
  int showInternFuncs    /* True if showing internal functions */
){
+
  u32 mask =
+
      SQLITE_DETERMINISTIC |
+
      SQLITE_DIRECTONLY |
+
      SQLITE_SUBTYPE |
+
      SQLITE_INNOCUOUS |
+
      SQLITE_FUNC_INTERNAL
+
  ;
+
  if( showInternFuncs ) mask = 0xffffffff;
  for(; p; p=p->pNext){
    const char *zType;
-
    static const u32 mask =
-
        SQLITE_DETERMINISTIC |
-
        SQLITE_DIRECTONLY |
-
        SQLITE_SUBTYPE |
-
        SQLITE_INNOCUOUS |
-
        SQLITE_FUNC_INTERNAL
-
    ;
    static const char *azEnc[] = { 0, "utf8", "utf16le", "utf16be" };

    assert( SQLITE_FUNC_ENCMASK==0x3 );
@@ -131290,7 +134158,7 @@ SQLITE_PRIVATE void sqlite3Pragma(
  */
#ifndef SQLITE_OMIT_AUTOVACUUM
  case PragTyp_INCREMENTAL_VACUUM: {
-
    int iLimit, addr;
+
    int iLimit = 0, addr;
    if( zRight==0 || !sqlite3GetInt32(zRight, &iLimit) || iLimit<=0 ){
      iLimit = 0x7fffffff;
    }
@@ -131447,6 +134315,7 @@ SQLITE_PRIVATE void sqlite3Pragma(
  **
  */
  case PragTyp_TEMP_STORE_DIRECTORY: {
+
    sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
    if( !zRight ){
      returnSingleText(v, sqlite3_temp_directory);
    }else{
@@ -131456,6 +134325,7 @@ SQLITE_PRIVATE void sqlite3Pragma(
        rc = sqlite3OsAccess(db->pVfs, zRight, SQLITE_ACCESS_READWRITE, &res);
        if( rc!=SQLITE_OK || res==0 ){
          sqlite3ErrorMsg(pParse, "not a writable directory");
+
          sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
          goto pragma_out;
        }
      }
@@ -131473,6 +134343,7 @@ SQLITE_PRIVATE void sqlite3Pragma(
      }
#endif /* SQLITE_OMIT_WSD */
    }
+
    sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
    break;
  }

@@ -131491,6 +134362,7 @@ SQLITE_PRIVATE void sqlite3Pragma(
  **
  */
  case PragTyp_DATA_STORE_DIRECTORY: {
+
    sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
    if( !zRight ){
      returnSingleText(v, sqlite3_data_directory);
    }else{
@@ -131500,6 +134372,7 @@ SQLITE_PRIVATE void sqlite3Pragma(
        rc = sqlite3OsAccess(db->pVfs, zRight, SQLITE_ACCESS_READWRITE, &res);
        if( rc!=SQLITE_OK || res==0 ){
          sqlite3ErrorMsg(pParse, "not a writable directory");
+
          sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
          goto pragma_out;
        }
      }
@@ -131511,6 +134384,7 @@ SQLITE_PRIVATE void sqlite3Pragma(
      }
#endif /* SQLITE_OMIT_WSD */
    }
+
    sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_TEMPDIR));
    break;
  }
#endif
@@ -131978,7 +134852,6 @@ SQLITE_PRIVATE void sqlite3Pragma(
    HashElem *k;           /* Loop counter:  Next table in schema */
    int x;                 /* result variable */
    int regResult;         /* 3 registers to hold a result row */
-
    int regKey;            /* Register to hold key for checking the FK */
    int regRow;            /* Registers to hold a row from pTab */
    int addrTop;           /* Top of a loop checking foreign keys */
    int addrOk;            /* Jump here if the key is OK */
@@ -131986,7 +134859,6 @@ SQLITE_PRIVATE void sqlite3Pragma(

    regResult = pParse->nMem+1;
    pParse->nMem += 4;
-
    regKey = ++pParse->nMem;
    regRow = ++pParse->nMem;
    k = sqliteHashFirst(&db->aDb[iDb].pSchema->tblHash);
    while( k ){
@@ -132053,9 +134925,9 @@ SQLITE_PRIVATE void sqlite3Pragma(
        /* Generate code to query the parent index for a matching parent
        ** key. If a match is found, jump to addrOk. */
        if( pIdx ){
-
          sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, pFK->nCol, regKey,
+
          sqlite3VdbeAddOp4(v, OP_Affinity, regRow, pFK->nCol, 0,
              sqlite3IndexAffinityStr(db,pIdx), pFK->nCol);
-
          sqlite3VdbeAddOp4Int(v, OP_Found, i, addrOk, regKey, 0);
+
          sqlite3VdbeAddOp4Int(v, OP_Found, i, addrOk, regRow, pFK->nCol);
          VdbeCoverage(v);
        }else if( pParent ){
          int jmp = sqlite3VdbeCurrentAddr(v)+2;
@@ -132226,15 +135098,24 @@ SQLITE_PRIVATE void sqlite3Pragma(
      for(x=sqliteHashFirst(pTbls); x; x=sqliteHashNext(x)){
        Table *pTab = sqliteHashData(x);
        Index *pIdx, *pPk;
-
        Index *pPrior = 0;
+
        Index *pPrior = 0;      /* Previous index */
        int loopTop;
        int iDataCur, iIdxCur;
        int r1 = -1;
-
        int bStrict;
+
        int bStrict;            /* True for a STRICT table */
+
        int r2;                 /* Previous key for WITHOUT ROWID tables */
+
        int mxCol;              /* Maximum non-virtual column number */

        if( !IsOrdinaryTable(pTab) ) continue;
        if( pObjTab && pObjTab!=pTab ) continue;
-
        pPk = HasRowid(pTab) ? 0 : sqlite3PrimaryKeyIndex(pTab);
+
        if( isQuick || HasRowid(pTab) ){
+
          pPk = 0;
+
          r2 = 0;
+
        }else{
+
          pPk = sqlite3PrimaryKeyIndex(pTab);
+
          r2 = sqlite3GetTempRange(pParse, pPk->nKeyCol);
+
          sqlite3VdbeAddOp3(v, OP_Null, 1, r2, r2+pPk->nKeyCol-1);
+
        }
        sqlite3OpenTableAndIndices(pParse, pTab, OP_OpenRead, 0,
                                   1, 0, &iDataCur, &iIdxCur);
        /* reg[7] counts the number of entries in the table.
@@ -132248,52 +135129,157 @@ SQLITE_PRIVATE void sqlite3Pragma(
        assert( sqlite3NoTempsInRange(pParse,1,7+j) );
        sqlite3VdbeAddOp2(v, OP_Rewind, iDataCur, 0); VdbeCoverage(v);
        loopTop = sqlite3VdbeAddOp2(v, OP_AddImm, 7, 1);
+

+
        /* Fetch the right-most column from the table.  This will cause
+
        ** the entire record header to be parsed and sanity checked.  It
+
        ** will also prepopulate the cursor column cache that is used
+
        ** by the OP_IsType code, so it is a required step.
+
        */
+
        mxCol = pTab->nCol-1;
+
        while( mxCol>=0
+
            && ((pTab->aCol[mxCol].colFlags & COLFLAG_VIRTUAL)!=0
+
                || pTab->iPKey==mxCol) ) mxCol--;
+
        if( mxCol>=0 ){
+
          sqlite3ExprCodeGetColumnOfTable(v, pTab, iDataCur, mxCol, 3);
+
          sqlite3VdbeTypeofColumn(v, 3);
+
        }
+

        if( !isQuick ){
-
          /* Sanity check on record header decoding */
-
          sqlite3VdbeAddOp3(v, OP_Column, iDataCur, pTab->nNVCol-1,3);
-
          sqlite3VdbeChangeP5(v, OPFLAG_TYPEOFARG);
-
          VdbeComment((v, "(right-most column)"));
+
          if( pPk ){
+
            /* Verify WITHOUT ROWID keys are in ascending order */
+
            int a1;
+
            char *zErr;
+
            a1 = sqlite3VdbeAddOp4Int(v, OP_IdxGT, iDataCur, 0,r2,pPk->nKeyCol);
+
            VdbeCoverage(v);
+
            sqlite3VdbeAddOp1(v, OP_IsNull, r2); VdbeCoverage(v);
+
            zErr = sqlite3MPrintf(db,
+
                   "row not in PRIMARY KEY order for %s",
+
                    pTab->zName);
+
            sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0, zErr, P4_DYNAMIC);
+
            integrityCheckResultRow(v);
+
            sqlite3VdbeJumpHere(v, a1);
+
            sqlite3VdbeJumpHere(v, a1+1);
+
            for(j=0; j<pPk->nKeyCol; j++){
+
              sqlite3ExprCodeLoadIndexColumn(pParse, pPk, iDataCur, j, r2+j);
+
            }
+
          }
        }
-
        /* Verify that all NOT NULL columns really are NOT NULL.  At the
-
        ** same time verify the type of the content of STRICT tables */
+
        /* Verify datatypes for all columns:
+
        **
+
        **   (1) NOT NULL columns may not contain a NULL
+
        **   (2) Datatype must be exact for non-ANY columns in STRICT tables
+
        **   (3) Datatype for TEXT columns in non-STRICT tables must be
+
        **       NULL, TEXT, or BLOB.
+
        **   (4) Datatype for numeric columns in non-STRICT tables must not
+
        **       be a TEXT value that can be losslessly converted to numeric.
+
        */
        bStrict = (pTab->tabFlags & TF_Strict)!=0;
        for(j=0; j<pTab->nCol; j++){
          char *zErr;
-
          Column *pCol = pTab->aCol + j;
-
          int doError, jmp2;
+
          Column *pCol = pTab->aCol + j;  /* The column to be checked */
+
          int labelError;               /* Jump here to report an error */
+
          int labelOk;                  /* Jump here if all looks ok */
+
          int p1, p3, p4;               /* Operands to the OP_IsType opcode */
+
          int doTypeCheck;              /* Check datatypes (besides NOT NULL) */
+

          if( j==pTab->iPKey ) continue;
-
          if( pCol->notNull==0 && !bStrict ) continue;
-
          doError = bStrict ? sqlite3VdbeMakeLabel(pParse) : 0;
-
          sqlite3ExprCodeGetColumnOfTable(v, pTab, iDataCur, j, 3);
-
          if( sqlite3VdbeGetOp(v,-1)->opcode==OP_Column ){
-
            sqlite3VdbeChangeP5(v, OPFLAG_TYPEOFARG);
+
          if( bStrict ){
+
            doTypeCheck = pCol->eCType>COLTYPE_ANY;
+
          }else{
+
            doTypeCheck = pCol->affinity>SQLITE_AFF_BLOB;
          }
+
          if( pCol->notNull==0 && !doTypeCheck ) continue;
+

+
          /* Compute the operands that will be needed for OP_IsType */
+
          p4 = SQLITE_NULL;
+
          if( pCol->colFlags & COLFLAG_VIRTUAL ){
+
            sqlite3ExprCodeGetColumnOfTable(v, pTab, iDataCur, j, 3);
+
            p1 = -1;
+
            p3 = 3;
+
          }else{
+
            if( pCol->iDflt ){
+
              sqlite3_value *pDfltValue = 0;
+
              sqlite3ValueFromExpr(db, sqlite3ColumnExpr(pTab,pCol), ENC(db),
+
                                   pCol->affinity, &pDfltValue);
+
              if( pDfltValue ){
+
                p4 = sqlite3_value_type(pDfltValue);
+
                sqlite3ValueFree(pDfltValue);
+
              }
+
            }
+
            p1 = iDataCur;
+
            if( !HasRowid(pTab) ){
+
              testcase( j!=sqlite3TableColumnToStorage(pTab, j) );
+
              p3 = sqlite3TableColumnToIndex(sqlite3PrimaryKeyIndex(pTab), j);
+
            }else{
+
              p3 = sqlite3TableColumnToStorage(pTab,j);
+
              testcase( p3!=j);
+
            }
+
          }
+

+
          labelError = sqlite3VdbeMakeLabel(pParse);
+
          labelOk = sqlite3VdbeMakeLabel(pParse);
          if( pCol->notNull ){
-
            jmp2 = sqlite3VdbeAddOp1(v, OP_NotNull, 3); VdbeCoverage(v);
+
            /* (1) NOT NULL columns may not contain a NULL */
+
            int jmp2 = sqlite3VdbeAddOp4Int(v, OP_IsType, p1, labelOk, p3, p4);
+
            sqlite3VdbeChangeP5(v, 0x0f);
+
            VdbeCoverage(v);
            zErr = sqlite3MPrintf(db, "NULL value in %s.%s", pTab->zName,
                                pCol->zCnName);
            sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0, zErr, P4_DYNAMIC);
-
            if( bStrict && pCol->eCType!=COLTYPE_ANY ){
-
              sqlite3VdbeGoto(v, doError);
+
            if( doTypeCheck ){
+
              sqlite3VdbeGoto(v, labelError);
+
              sqlite3VdbeJumpHere(v, jmp2);
            }else{
-
              integrityCheckResultRow(v);
+
              /* VDBE byte code will fall thru */
            }
-
            sqlite3VdbeJumpHere(v, jmp2);
          }
-
          if( (pTab->tabFlags & TF_Strict)!=0
-
           && pCol->eCType!=COLTYPE_ANY
-
          ){
-
            jmp2 = sqlite3VdbeAddOp3(v, OP_IsNullOrType, 3, 0,
-
                                     sqlite3StdTypeMap[pCol->eCType-1]);
+
          if( bStrict && doTypeCheck ){
+
            /* (2) Datatype must be exact for non-ANY columns in STRICT tables*/
+
            static unsigned char aStdTypeMask[] = {
+
               0x1f,    /* ANY */
+
               0x18,    /* BLOB */
+
               0x11,    /* INT */
+
               0x11,    /* INTEGER */
+
               0x13,    /* REAL */
+
               0x14     /* TEXT */
+
            };
+
            sqlite3VdbeAddOp4Int(v, OP_IsType, p1, labelOk, p3, p4);
+
            assert( pCol->eCType>=1 && pCol->eCType<=sizeof(aStdTypeMask) );
+
            sqlite3VdbeChangeP5(v, aStdTypeMask[pCol->eCType-1]);
            VdbeCoverage(v);
            zErr = sqlite3MPrintf(db, "non-%s value in %s.%s",
                                  sqlite3StdType[pCol->eCType-1],
                                  pTab->zName, pTab->aCol[j].zCnName);
            sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0, zErr, P4_DYNAMIC);
-
            sqlite3VdbeResolveLabel(v, doError);
-
            integrityCheckResultRow(v);
-
            sqlite3VdbeJumpHere(v, jmp2);
+
          }else if( !bStrict && pCol->affinity==SQLITE_AFF_TEXT ){
+
            /* (3) Datatype for TEXT columns in non-STRICT tables must be
+
            **     NULL, TEXT, or BLOB. */
+
            sqlite3VdbeAddOp4Int(v, OP_IsType, p1, labelOk, p3, p4);
+
            sqlite3VdbeChangeP5(v, 0x1c); /* NULL, TEXT, or BLOB */
+
            VdbeCoverage(v);
+
            zErr = sqlite3MPrintf(db, "NUMERIC value in %s.%s",
+
                                  pTab->zName, pTab->aCol[j].zCnName);
+
            sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0, zErr, P4_DYNAMIC);
+
          }else if( !bStrict && pCol->affinity>=SQLITE_AFF_NUMERIC ){
+
            /* (4) Datatype for numeric columns in non-STRICT tables must not
+
            **     be a TEXT value that can be converted to numeric. */
+
            sqlite3VdbeAddOp4Int(v, OP_IsType, p1, labelOk, p3, p4);
+
            sqlite3VdbeChangeP5(v, 0x1b); /* NULL, INT, FLOAT, or BLOB */
+
            VdbeCoverage(v);
+
            if( p1>=0 ){
+
              sqlite3ExprCodeGetColumnOfTable(v, pTab, iDataCur, j, 3);
+
            }
+
            sqlite3VdbeAddOp4(v, OP_Affinity, 3, 1, 0, "C", P4_STATIC);
+
            sqlite3VdbeAddOp4Int(v, OP_IsType, -1, labelOk, 3, p4);
+
            sqlite3VdbeChangeP5(v, 0x1c); /* NULL, TEXT, or BLOB */
+
            VdbeCoverage(v);
+
            zErr = sqlite3MPrintf(db, "TEXT value in %s.%s",
+
                                  pTab->zName, pTab->aCol[j].zCnName);
+
            sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0, zErr, P4_DYNAMIC);
          }
+
          sqlite3VdbeResolveLabel(v, labelError);
+
          integrityCheckResultRow(v);
+
          sqlite3VdbeResolveLabel(v, labelOk);
        }
        /* Verify CHECK constraints */
        if( pTab->pCheck && (db->flags & SQLITE_IgnoreChecks)==0 ){
@@ -132381,6 +135367,9 @@ SQLITE_PRIVATE void sqlite3Pragma(
            integrityCheckResultRow(v);
            sqlite3VdbeJumpHere(v, addr);
          }
+
          if( pPk ){
+
            sqlite3ReleaseTempRange(pParse, r2, pPk->nKeyCol);
+
          }
        }
      }
    }
@@ -132531,6 +135520,11 @@ SQLITE_PRIVATE void sqlite3Pragma(
      aOp[1].p2 = iCookie;
      aOp[1].p3 = sqlite3Atoi(zRight);
      aOp[1].p5 = 1;
+
      if( iCookie==BTREE_SCHEMA_VERSION && (db->flags & SQLITE_Defensive)!=0 ){
+
        /* Do not allow the use of PRAGMA schema_version=VALUE in defensive
+
        ** mode.  Change the OP_SetCookie opcode into a no-op.  */
+
        aOp[1].opcode = OP_Noop;
+
      }
    }else{
      /* Read the specified cookie value */
      static const VdbeOpList readCookie[] = {
@@ -133779,15 +136773,15 @@ SQLITE_PRIVATE void sqlite3ParseObjectReset(Parse *pParse){
  assert( db->pParse==pParse );
  assert( pParse->nested==0 );
#ifndef SQLITE_OMIT_SHARED_CACHE
-
  sqlite3DbFree(db, pParse->aTableLock);
+
  if( pParse->aTableLock ) sqlite3DbNNFreeNN(db, pParse->aTableLock);
#endif
  while( pParse->pCleanup ){
    ParseCleanup *pCleanup = pParse->pCleanup;
    pParse->pCleanup = pCleanup->pNext;
    pCleanup->xCleanup(db, pCleanup->pPtr);
-
    sqlite3DbFreeNN(db, pCleanup);
+
    sqlite3DbNNFreeNN(db, pCleanup);
  }
-
  sqlite3DbFree(db, pParse->aLabel);
+
  if( pParse->aLabel ) sqlite3DbNNFreeNN(db, pParse->aLabel);
  if( pParse->pConstExpr ){
    sqlite3ExprListDelete(db, pParse->pConstExpr);
  }
@@ -133910,7 +136904,7 @@ static int sqlite3Prepare(
    sParse.disableLookaside++;
    DisableLookaside;
  }
-
  sParse.disableVtab = (prepFlags & SQLITE_PREPARE_NO_VTAB)!=0;
+
  sParse.prepFlags = prepFlags & 0xff;

  /* Check to verify that it is possible to get a read lock on all
  ** database schemas.  The inability to get a read lock indicates that
@@ -133951,7 +136945,9 @@ static int sqlite3Prepare(
    }
  }

-
  sqlite3VtabUnlockList(db);
+
#ifndef SQLITE_OMIT_VIRTUALTABLE
+
  if( db->pDisconnect ) sqlite3VtabUnlockList(db);
+
#endif

  if( nBytes>=0 && (nBytes==0 || zSql[nBytes-1]!=0) ){
    char *zSqlCopy;
@@ -134291,7 +137287,7 @@ SQLITE_API int sqlite3_prepare16_v3(
*/
typedef struct DistinctCtx DistinctCtx;
struct DistinctCtx {
-
  u8 isTnct;      /* True if the DISTINCT keyword is present */
+
  u8 isTnct;      /* 0: Not distinct. 1: DISTICT  2: DISTINCT and ORDER BY */
  u8 eTnctType;   /* One of the WHERE_DISTINCT_* operators */
  int tabTnct;    /* Ephemeral table used for DISTINCT processing */
  int addrTnct;   /* Address of OP_OpenEphemeral opcode for tabTnct */
@@ -134346,6 +137342,7 @@ struct SortCtx {
** If bFree==0, Leave the first Select object unfreed
*/
static void clearSelect(sqlite3 *db, Select *p, int bFree){
+
  assert( db!=0 );
  while( p ){
    Select *pPrior = p->pPrior;
    sqlite3ExprListDelete(db, p->pEList);
@@ -134365,7 +137362,7 @@ static void clearSelect(sqlite3 *db, Select *p, int bFree){
      sqlite3WindowUnlinkFromSelect(p->pWin);
    }
#endif
-
    if( bFree ) sqlite3DbFreeNN(db, p);
+
    if( bFree ) sqlite3DbNNFreeNN(db, p);
    p = pPrior;
    bFree = 1;
  }
@@ -134474,6 +137471,52 @@ static Select *findRightmost(Select *p){
**
** If an illegal or unsupported join type is seen, then still return
** a join type, but put an error in the pParse structure.
+
**
+
** These are the valid join types:
+
**
+
**
+
**      pA       pB       pC               Return Value
+
**     -------  -----    -----             ------------
+
**     CROSS      -        -                 JT_CROSS
+
**     INNER      -        -                 JT_INNER
+
**     LEFT       -        -                 JT_LEFT|JT_OUTER
+
**     LEFT     OUTER      -                 JT_LEFT|JT_OUTER
+
**     RIGHT      -        -                 JT_RIGHT|JT_OUTER
+
**     RIGHT    OUTER      -                 JT_RIGHT|JT_OUTER
+
**     FULL       -        -                 JT_LEFT|JT_RIGHT|JT_OUTER
+
**     FULL     OUTER      -                 JT_LEFT|JT_RIGHT|JT_OUTER
+
**     NATURAL  INNER      -                 JT_NATURAL|JT_INNER
+
**     NATURAL  LEFT       -                 JT_NATURAL|JT_LEFT|JT_OUTER
+
**     NATURAL  LEFT     OUTER               JT_NATURAL|JT_LEFT|JT_OUTER
+
**     NATURAL  RIGHT      -                 JT_NATURAL|JT_RIGHT|JT_OUTER
+
**     NATURAL  RIGHT    OUTER               JT_NATURAL|JT_RIGHT|JT_OUTER
+
**     NATURAL  FULL       -                 JT_NATURAL|JT_LEFT|JT_RIGHT
+
**     NATURAL  FULL     OUTER               JT_NATRUAL|JT_LEFT|JT_RIGHT
+
**
+
** To preserve historical compatibly, SQLite also accepts a variety
+
** of other non-standard and in many cases non-sensical join types.
+
** This routine makes as much sense at it can from the nonsense join
+
** type and returns a result.  Examples of accepted nonsense join types
+
** include but are not limited to:
+
**
+
**          INNER CROSS JOIN        ->   same as JOIN
+
**          NATURAL CROSS JOIN      ->   same as NATURAL JOIN
+
**          OUTER LEFT JOIN         ->   same as LEFT JOIN
+
**          LEFT NATURAL JOIN       ->   same as NATURAL LEFT JOIN
+
**          LEFT RIGHT JOIN         ->   same as FULL JOIN
+
**          RIGHT OUTER FULL JOIN   ->   same as FULL JOIN
+
**          CROSS CROSS CROSS JOIN  ->   same as JOIN
+
**
+
** The only restrictions on the join type name are:
+
**
+
**    *   "INNER" cannot appear together with "OUTER", "LEFT", "RIGHT",
+
**        or "FULL".
+
**
+
**    *   "CROSS" cannot appear together with "OUTER", "LEFT", "RIGHT,
+
**        or "FULL".
+
**
+
**    *   If "OUTER" is present then there must also be one of
+
**        "LEFT", "RIGHT", or "FULL"
*/
SQLITE_PRIVATE int sqlite3JoinType(Parse *pParse, Token *pA, Token *pB, Token *pC){
  int jointype = 0;
@@ -134486,13 +137529,13 @@ SQLITE_PRIVATE int sqlite3JoinType(Parse *pParse, Token *pA, Token *pB, Token *p
    u8 nChar;    /* Length of the keyword in characters */
    u8 code;     /* Join type mask */
  } aKeyword[] = {
-
    /* natural */ { 0,  7, JT_NATURAL                },
-
    /* left    */ { 6,  4, JT_LEFT|JT_OUTER          },
-
    /* outer   */ { 10, 5, JT_OUTER                  },
-
    /* right   */ { 14, 5, JT_RIGHT|JT_OUTER         },
-
    /* full    */ { 19, 4, JT_LEFT|JT_RIGHT|JT_OUTER },
-
    /* inner   */ { 23, 5, JT_INNER                  },
-
    /* cross   */ { 28, 5, JT_INNER|JT_CROSS         },
+
    /* (0) natural */ { 0,  7, JT_NATURAL                },
+
    /* (1) left    */ { 6,  4, JT_LEFT|JT_OUTER          },
+
    /* (2) outer   */ { 10, 5, JT_OUTER                  },
+
    /* (3) right   */ { 14, 5, JT_RIGHT|JT_OUTER         },
+
    /* (4) full    */ { 19, 4, JT_LEFT|JT_RIGHT|JT_OUTER },
+
    /* (5) inner   */ { 23, 5, JT_INNER                  },
+
    /* (6) cross   */ { 28, 5, JT_INNER|JT_CROSS         },
  };
  int i, j;
  apAll[0] = pA;
@@ -134515,18 +137558,15 @@ SQLITE_PRIVATE int sqlite3JoinType(Parse *pParse, Token *pA, Token *pB, Token *p
  }
  if(
     (jointype & (JT_INNER|JT_OUTER))==(JT_INNER|JT_OUTER) ||
-
     (jointype & JT_ERROR)!=0
+
     (jointype & JT_ERROR)!=0 ||
+
     (jointype & (JT_OUTER|JT_LEFT|JT_RIGHT))==JT_OUTER
  ){
-
    const char *zSp = " ";
-
    assert( pB!=0 );
-
    if( pC==0 ){ zSp++; }
-
    sqlite3ErrorMsg(pParse, "unknown or unsupported join type: "
-
       "%T %T%s%T", pA, pB, zSp, pC);
-
    jointype = JT_INNER;
-
  }else if( (jointype & JT_OUTER)!=0
-
         && (jointype & (JT_LEFT|JT_RIGHT))!=JT_LEFT ){
-
    sqlite3ErrorMsg(pParse,
-
      "RIGHT and FULL OUTER JOINs are not currently supported");
+
    const char *zSp1 = " ";
+
    const char *zSp2 = " ";
+
    if( pB==0 ){ zSp1++; }
+
    if( pC==0 ){ zSp2++; }
+
    sqlite3ErrorMsg(pParse, "unknown join type: "
+
       "%T%s%T%s%T", pA, zSp1, pB, zSp2, pC);
    jointype = JT_INNER;
  }
  return jointype;
@@ -134547,8 +137587,25 @@ SQLITE_PRIVATE int sqlite3ColumnIndex(Table *pTab, const char *zCol){
}

/*
-
** Search the first N tables in pSrc, from left to right, looking for a
-
** table that has a column named zCol.
+
** Mark a subquery result column as having been used.
+
*/
+
SQLITE_PRIVATE void sqlite3SrcItemColumnUsed(SrcItem *pItem, int iCol){
+
  assert( pItem!=0 );
+
  assert( (int)pItem->fg.isNestedFrom == IsNestedFrom(pItem->pSelect) );
+
  if( pItem->fg.isNestedFrom ){
+
    ExprList *pResults;
+
    assert( pItem->pSelect!=0 );
+
    pResults = pItem->pSelect->pEList;
+
    assert( pResults!=0 );
+
    assert( iCol>=0 && iCol<pResults->nExpr );
+
    pResults->a[iCol].fg.bUsed = 1;
+
  }
+
}
+

+
/*
+
** Search the tables iStart..iEnd (inclusive) in pSrc, looking for a
+
** table that has a column named zCol.  The search is left-to-right.
+
** The first match found is returned.
**
** When found, set *piTab and *piCol to the table index and column index
** of the matching column and return TRUE.
@@ -134557,22 +137614,27 @@ SQLITE_PRIVATE int sqlite3ColumnIndex(Table *pTab, const char *zCol){
*/
static int tableAndColumnIndex(
  SrcList *pSrc,       /* Array of tables to search */
-
  int N,               /* Number of tables in pSrc->a[] to search */
+
  int iStart,          /* First member of pSrc->a[] to check */
+
  int iEnd,            /* Last member of pSrc->a[] to check */
  const char *zCol,    /* Name of the column we are looking for */
  int *piTab,          /* Write index of pSrc->a[] here */
  int *piCol,          /* Write index of pSrc->a[*piTab].pTab->aCol[] here */
-
  int bIgnoreHidden    /* True to ignore hidden columns */
+
  int bIgnoreHidden    /* Ignore hidden columns */
){
  int i;               /* For looping over tables in pSrc */
  int iCol;            /* Index of column matching zCol */

+
  assert( iEnd<pSrc->nSrc );
+
  assert( iStart>=0 );
  assert( (piTab==0)==(piCol==0) );  /* Both or neither are NULL */
-
  for(i=0; i<N; i++){
+

+
  for(i=iStart; i<=iEnd; i++){
    iCol = sqlite3ColumnIndex(pSrc->a[i].pTab, zCol);
    if( iCol>=0
     && (bIgnoreHidden==0 || IsHiddenColumn(&pSrc->a[i].pTab->aCol[iCol])==0)
    ){
      if( piTab ){
+
        sqlite3SrcItemColumnUsed(&pSrc->a[i], iCol);
        *piTab = i;
        *piCol = iCol;
      }
@@ -134583,66 +137645,19 @@ static int tableAndColumnIndex(
}

/*
-
** This function is used to add terms implied by JOIN syntax to the
-
** WHERE clause expression of a SELECT statement. The new term, which
-
** is ANDed with the existing WHERE clause, is of the form:
-
**
-
**    (tab1.col1 = tab2.col2)
-
**
-
** where tab1 is the iSrc'th table in SrcList pSrc and tab2 is the
-
** (iSrc+1)'th. Column col1 is column iColLeft of tab1, and col2 is
-
** column iColRight of tab2.
-
*/
-
static void addWhereTerm(
-
  Parse *pParse,                  /* Parsing context */
-
  SrcList *pSrc,                  /* List of tables in FROM clause */
-
  int iLeft,                      /* Index of first table to join in pSrc */
-
  int iColLeft,                   /* Index of column in first table */
-
  int iRight,                     /* Index of second table in pSrc */
-
  int iColRight,                  /* Index of column in second table */
-
  int isOuterJoin,                /* True if this is an OUTER join */
-
  Expr **ppWhere                  /* IN/OUT: The WHERE clause to add to */
-
){
-
  sqlite3 *db = pParse->db;
-
  Expr *pE1;
-
  Expr *pE2;
-
  Expr *pEq;
-

-
  assert( iLeft<iRight );
-
  assert( pSrc->nSrc>iRight );
-
  assert( pSrc->a[iLeft].pTab );
-
  assert( pSrc->a[iRight].pTab );
-

-
  pE1 = sqlite3CreateColumnExpr(db, pSrc, iLeft, iColLeft);
-
  pE2 = sqlite3CreateColumnExpr(db, pSrc, iRight, iColRight);
-

-
  pEq = sqlite3PExpr(pParse, TK_EQ, pE1, pE2);
-
  assert( pE2!=0 || pEq==0 );  /* Due to db->mallocFailed test
-
                               ** in sqlite3DbMallocRawNN() called from
-
                               ** sqlite3PExpr(). */
-
  if( pEq && isOuterJoin ){
-
    ExprSetProperty(pEq, EP_FromJoin);
-
    assert( !ExprHasProperty(pEq, EP_TokenOnly|EP_Reduced) );
-
    ExprSetVVAProperty(pEq, EP_NoReduce);
-
    pEq->w.iRightJoinTable = pE2->iTable;
-
  }
-
  *ppWhere = sqlite3ExprAnd(pParse, *ppWhere, pEq);
-
}
-

-
/*
-
** Set the EP_FromJoin property on all terms of the given expression.
-
** And set the Expr.w.iRightJoinTable to iTable for every term in the
+
** Set the EP_OuterON property on all terms of the given expression.
+
** And set the Expr.w.iJoin to iTable for every term in the
** expression.
**
-
** The EP_FromJoin property is used on terms of an expression to tell
-
** the LEFT OUTER JOIN processing logic that this term is part of the
+
** The EP_OuterON property is used on terms of an expression to tell
+
** the OUTER JOIN processing logic that this term is part of the
** join restriction specified in the ON or USING clause and not a part
** of the more general WHERE clause.  These terms are moved over to the
** WHERE clause during join processing but we need to remember that they
** originated in the ON or USING clause.
**
-
** The Expr.w.iRightJoinTable tells the WHERE clause processing that the
-
** expression depends on table w.iRightJoinTable even if that table is not
+
** The Expr.w.iJoin tells the WHERE clause processing that the
+
** expression depends on table w.iJoin even if that table is not
** explicitly mentioned in the expression.  That information is needed
** for cases like this:
**
@@ -134655,39 +137670,48 @@ static void addWhereTerm(
** after the t1 loop and rows with t1.x!=5 will never appear in
** the output, which is incorrect.
*/
-
SQLITE_PRIVATE void sqlite3SetJoinExpr(Expr *p, int iTable){
+
SQLITE_PRIVATE void sqlite3SetJoinExpr(Expr *p, int iTable, u32 joinFlag){
+
  assert( joinFlag==EP_OuterON || joinFlag==EP_InnerON );
  while( p ){
-
    ExprSetProperty(p, EP_FromJoin);
+
    ExprSetProperty(p, joinFlag);
    assert( !ExprHasProperty(p, EP_TokenOnly|EP_Reduced) );
    ExprSetVVAProperty(p, EP_NoReduce);
-
    p->w.iRightJoinTable = iTable;
+
    p->w.iJoin = iTable;
    if( p->op==TK_FUNCTION ){
      assert( ExprUseXList(p) );
      if( p->x.pList ){
        int i;
        for(i=0; i<p->x.pList->nExpr; i++){
-
          sqlite3SetJoinExpr(p->x.pList->a[i].pExpr, iTable);
+
          sqlite3SetJoinExpr(p->x.pList->a[i].pExpr, iTable, joinFlag);
        }
      }
    }
-
    sqlite3SetJoinExpr(p->pLeft, iTable);
+
    sqlite3SetJoinExpr(p->pLeft, iTable, joinFlag);
    p = p->pRight;
  }
}

-
/* Undo the work of sqlite3SetJoinExpr(). In the expression p, convert every
-
** term that is marked with EP_FromJoin and w.iRightJoinTable==iTable into
-
** an ordinary term that omits the EP_FromJoin mark.
+
/* Undo the work of sqlite3SetJoinExpr().  This is used when a LEFT JOIN
+
** is simplified into an ordinary JOIN, and when an ON expression is
+
** "pushed down" into the WHERE clause of a subquery.
**
-
** This happens when a LEFT JOIN is simplified into an ordinary JOIN.
+
** Convert every term that is marked with EP_OuterON and w.iJoin==iTable into
+
** an ordinary term that omits the EP_OuterON mark.  Or if iTable<0, then
+
** just clear every EP_OuterON and EP_InnerON mark from the expression tree.
+
**
+
** If nullable is true, that means that Expr p might evaluate to NULL even
+
** if it is a reference to a NOT NULL column.  This can happen, for example,
+
** if the table that p references is on the left side of a RIGHT JOIN.
+
** If nullable is true, then take care to not remove the EP_CanBeNull bit.
+
** See forum thread https://sqlite.org/forum/forumpost/b40696f50145d21c
*/
-
static void unsetJoinExpr(Expr *p, int iTable){
+
static void unsetJoinExpr(Expr *p, int iTable, int nullable){
  while( p ){
-
    if( ExprHasProperty(p, EP_FromJoin)
-
     && (iTable<0 || p->w.iRightJoinTable==iTable) ){
-
      ExprClearProperty(p, EP_FromJoin);
+
    if( iTable<0 || (ExprHasProperty(p, EP_OuterON) && p->w.iJoin==iTable) ){
+
      ExprClearProperty(p, EP_OuterON|EP_InnerON);
+
      if( iTable>=0 ) ExprSetProperty(p, EP_InnerON);
    }
-
    if( p->op==TK_COLUMN && p->iTable==iTable ){
+
    if( p->op==TK_COLUMN && p->iTable==iTable && !nullable ){
      ExprClearProperty(p, EP_CanBeNull);
    }
    if( p->op==TK_FUNCTION ){
@@ -134695,30 +137719,37 @@ static void unsetJoinExpr(Expr *p, int iTable){
      if( p->x.pList ){
        int i;
        for(i=0; i<p->x.pList->nExpr; i++){
-
          unsetJoinExpr(p->x.pList->a[i].pExpr, iTable);
+
          unsetJoinExpr(p->x.pList->a[i].pExpr, iTable, nullable);
        }
      }
    }
-
    unsetJoinExpr(p->pLeft, iTable);
+
    unsetJoinExpr(p->pLeft, iTable, nullable);
    p = p->pRight;
  }
}

/*
** This routine processes the join information for a SELECT statement.
-
** ON and USING clauses are converted into extra terms of the WHERE clause.
-
** NATURAL joins also create extra WHERE clause terms.
+
**
+
**   *  A NATURAL join is converted into a USING join.  After that, we
+
**      do not need to be concerned with NATURAL joins and we only have
+
**      think about USING joins.
+
**
+
**   *  ON and USING clauses result in extra terms being added to the
+
**      WHERE clause to enforce the specified constraints.  The extra
+
**      WHERE clause terms will be tagged with EP_OuterON or
+
**      EP_InnerON so that we know that they originated in ON/USING.
**
** The terms of a FROM clause are contained in the Select.pSrc structure.
** The left most table is the first entry in Select.pSrc.  The right-most
** table is the last entry.  The join operator is held in the entry to
-
** the left.  Thus entry 0 contains the join operator for the join between
+
** the right.  Thus entry 1 contains the join operator for the join between
** entries 0 and 1.  Any ON or USING clauses associated with the join are
-
** also attached to the left entry.
+
** also attached to the right entry.
**
** This routine returns the number of errors encountered.
*/
-
static int sqliteProcessJoin(Parse *pParse, Select *p){
+
static int sqlite3ProcessJoin(Parse *pParse, Select *p){
  SrcList *pSrc;                  /* All tables in the FROM clause */
  int i, j;                       /* Loop counters */
  SrcItem *pLeft;                 /* Left table being joined */
@@ -134729,49 +137760,41 @@ static int sqliteProcessJoin(Parse *pParse, Select *p){
  pRight = &pLeft[1];
  for(i=0; i<pSrc->nSrc-1; i++, pRight++, pLeft++){
    Table *pRightTab = pRight->pTab;
-
    int isOuter;
+
    u32 joinType;

    if( NEVER(pLeft->pTab==0 || pRightTab==0) ) continue;
-
    isOuter = (pRight->fg.jointype & JT_OUTER)!=0;
+
    joinType = (pRight->fg.jointype & JT_OUTER)!=0 ? EP_OuterON : EP_InnerON;

-
    /* When the NATURAL keyword is present, add WHERE clause terms for
-
    ** every column that the two tables have in common.
+
    /* If this is a NATURAL join, synthesize an approprate USING clause
+
    ** to specify which columns should be joined.
    */
    if( pRight->fg.jointype & JT_NATURAL ){
-
      if( pRight->pOn || pRight->pUsing ){
+
      IdList *pUsing = 0;
+
      if( pRight->fg.isUsing || pRight->u3.pOn ){
        sqlite3ErrorMsg(pParse, "a NATURAL join may not have "
           "an ON or USING clause", 0);
        return 1;
      }
      for(j=0; j<pRightTab->nCol; j++){
        char *zName;   /* Name of column in the right table */
-
        int iLeft;     /* Matching left table */
-
        int iLeftCol;  /* Matching column in the left table */

        if( IsHiddenColumn(&pRightTab->aCol[j]) ) continue;
        zName = pRightTab->aCol[j].zCnName;
-
        if( tableAndColumnIndex(pSrc, i+1, zName, &iLeft, &iLeftCol, 1) ){
-
          addWhereTerm(pParse, pSrc, iLeft, iLeftCol, i+1, j,
-
                isOuter, &p->pWhere);
+
        if( tableAndColumnIndex(pSrc, 0, i, zName, 0, 0, 1) ){
+
          pUsing = sqlite3IdListAppend(pParse, pUsing, 0);
+
          if( pUsing ){
+
            assert( pUsing->nId>0 );
+
            assert( pUsing->a[pUsing->nId-1].zName==0 );
+
            pUsing->a[pUsing->nId-1].zName = sqlite3DbStrDup(pParse->db, zName);
+
          }
        }
      }
-
    }
-

-
    /* Disallow both ON and USING clauses in the same join
-
    */
-
    if( pRight->pOn && pRight->pUsing ){
-
      sqlite3ErrorMsg(pParse, "cannot have both ON and USING "
-
        "clauses in the same join");
-
      return 1;
-
    }
-

-
    /* Add the ON clause to the end of the WHERE clause, connected by
-
    ** an AND operator.
-
    */
-
    if( pRight->pOn ){
-
      if( isOuter ) sqlite3SetJoinExpr(pRight->pOn, pRight->iCursor);
-
      p->pWhere = sqlite3ExprAnd(pParse, p->pWhere, pRight->pOn);
-
      pRight->pOn = 0;
+
      if( pUsing ){
+
        pRight->fg.isUsing = 1;
+
        pRight->fg.isSynthUsing = 1;
+
        pRight->u3.pUsing = pUsing;
+
      }
+
      if( pParse->nErr ) return 1;
    }

    /* Create extra terms on the WHERE clause for each column named
@@ -134781,27 +137804,88 @@ static int sqliteProcessJoin(Parse *pParse, Select *p){
    ** Report an error if any column mentioned in the USING clause is
    ** not contained in both tables to be joined.
    */
-
    if( pRight->pUsing ){
-
      IdList *pList = pRight->pUsing;
+
    if( pRight->fg.isUsing ){
+
      IdList *pList = pRight->u3.pUsing;
+
      sqlite3 *db = pParse->db;
+
      assert( pList!=0 );
      for(j=0; j<pList->nId; j++){
        char *zName;     /* Name of the term in the USING clause */
        int iLeft;       /* Table on the left with matching column name */
        int iLeftCol;    /* Column number of matching column on the left */
        int iRightCol;   /* Column number of matching column on the right */
+
        Expr *pE1;       /* Reference to the column on the LEFT of the join */
+
        Expr *pE2;       /* Reference to the column on the RIGHT of the join */
+
        Expr *pEq;       /* Equality constraint.  pE1 == pE2 */

        zName = pList->a[j].zName;
        iRightCol = sqlite3ColumnIndex(pRightTab, zName);
        if( iRightCol<0
-
         || !tableAndColumnIndex(pSrc, i+1, zName, &iLeft, &iLeftCol, 0)
+
         || tableAndColumnIndex(pSrc, 0, i, zName, &iLeft, &iLeftCol,
+
                                pRight->fg.isSynthUsing)==0
        ){
          sqlite3ErrorMsg(pParse, "cannot join using column %s - column "
            "not present in both tables", zName);
          return 1;
        }
-
        addWhereTerm(pParse, pSrc, iLeft, iLeftCol, i+1, iRightCol,
-
                     isOuter, &p->pWhere);
+
        pE1 = sqlite3CreateColumnExpr(db, pSrc, iLeft, iLeftCol);
+
        sqlite3SrcItemColumnUsed(&pSrc->a[iLeft], iLeftCol);
+
        if( (pSrc->a[0].fg.jointype & JT_LTORJ)!=0 ){
+
          /* This branch runs if the query contains one or more RIGHT or FULL
+
          ** JOINs.  If only a single table on the left side of this join
+
          ** contains the zName column, then this branch is a no-op.
+
          ** But if there are two or more tables on the left side
+
          ** of the join, construct a coalesce() function that gathers all
+
          ** such tables.  Raise an error if more than one of those references
+
          ** to zName is not also within a prior USING clause.
+
          **
+
          ** We really ought to raise an error if there are two or more
+
          ** non-USING references to zName on the left of an INNER or LEFT
+
          ** JOIN.  But older versions of SQLite do not do that, so we avoid
+
          ** adding a new error so as to not break legacy applications.
+
          */
+
          ExprList *pFuncArgs = 0;   /* Arguments to the coalesce() */
+
          static const Token tkCoalesce = { "coalesce", 8 };
+
          while( tableAndColumnIndex(pSrc, iLeft+1, i, zName, &iLeft, &iLeftCol,
+
                                     pRight->fg.isSynthUsing)!=0 ){
+
            if( pSrc->a[iLeft].fg.isUsing==0
+
             || sqlite3IdListIndex(pSrc->a[iLeft].u3.pUsing, zName)<0
+
            ){
+
              sqlite3ErrorMsg(pParse, "ambiguous reference to %s in USING()",
+
                              zName);
+
              break;
+
            }
+
            pFuncArgs = sqlite3ExprListAppend(pParse, pFuncArgs, pE1);
+
            pE1 = sqlite3CreateColumnExpr(db, pSrc, iLeft, iLeftCol);
+
            sqlite3SrcItemColumnUsed(&pSrc->a[iLeft], iLeftCol);
+
          }
+
          if( pFuncArgs ){
+
            pFuncArgs = sqlite3ExprListAppend(pParse, pFuncArgs, pE1);
+
            pE1 = sqlite3ExprFunction(pParse, pFuncArgs, &tkCoalesce, 0);
+
          }
+
        }
+
        pE2 = sqlite3CreateColumnExpr(db, pSrc, i+1, iRightCol);
+
        sqlite3SrcItemColumnUsed(pRight, iRightCol);
+
        pEq = sqlite3PExpr(pParse, TK_EQ, pE1, pE2);
+
        assert( pE2!=0 || pEq==0 );
+
        if( pEq ){
+
          ExprSetProperty(pEq, joinType);
+
          assert( !ExprHasProperty(pEq, EP_TokenOnly|EP_Reduced) );
+
          ExprSetVVAProperty(pEq, EP_NoReduce);
+
          pEq->w.iJoin = pE2->iTable;
+
        }
+
        p->pWhere = sqlite3ExprAnd(pParse, p->pWhere, pEq);
      }
    }
+

+
    /* Add the ON clause to the end of the WHERE clause, connected by
+
    ** an AND operator.
+
    */
+
    else if( pRight->u3.pOn ){
+
      sqlite3SetJoinExpr(pRight->u3.pOn, pRight->iCursor, joinType);
+
      p->pWhere = sqlite3ExprAnd(pParse, p->pWhere, pRight->u3.pOn);
+
      pRight->u3.pOn = 0;
+
      pRight->fg.isOn = 1;
+
    }
  }
  return 0;
}
@@ -135190,7 +138274,7 @@ static void fixDistinctOpenEph(
** retrieved directly from table t1. If the values are very large, this
** can be more efficient than storing them directly in the sorter records.
**
-
** The ExprList_item.bSorterRef flag is set for each expression in pEList
+
** The ExprList_item.fg.bSorterRef flag is set for each expression in pEList
** for which the sorter-reference optimization should be enabled.
** Additionally, the pSort->aDefer[] array is populated with entries
** for all cursors required to evaluate all selected expressions. Finally.
@@ -135250,7 +138334,7 @@ static void selectExprDefer(
            nDefer++;
          }
        }
-
        pItem->bSorterRef = 1;
+
        pItem->fg.bSorterRef = 1;
      }
    }
  }
@@ -135381,7 +138465,7 @@ static void selectInnerLoop(
      for(i=0; i<pEList->nExpr; i++){
        if( pEList->a[i].u.x.iOrderByCol>0
#ifdef SQLITE_ENABLE_SORTER_REFERENCES
-
         || pEList->a[i].bSorterRef
+
         || pEList->a[i].fg.bSorterRef
#endif
        ){
          nResultCol--;
@@ -135469,6 +138553,9 @@ static void selectInnerLoop(
      testcase( eDest==SRT_Fifo );
      testcase( eDest==SRT_DistFifo );
      sqlite3VdbeAddOp3(v, OP_MakeRecord, regResult, nResultCol, r1+nPrefixReg);
+
      if( pDest->zAffSdst ){
+
        sqlite3VdbeChangeP4(v, -1, pDest->zAffSdst, nResultCol);
+
      }
#ifndef SQLITE_OMIT_CTE
      if( eDest==SRT_DistFifo ){
        /* If the destination is DistFifo, then cursor (iParm+1) is open
@@ -135684,9 +138771,10 @@ SQLITE_PRIVATE KeyInfo *sqlite3KeyInfoAlloc(sqlite3 *db, int N, int X){
*/
SQLITE_PRIVATE void sqlite3KeyInfoUnref(KeyInfo *p){
  if( p ){
+
    assert( p->db!=0 );
    assert( p->nRef>0 );
    p->nRef--;
-
    if( p->nRef==0 ) sqlite3DbFreeNN(p->db, p);
+
    if( p->nRef==0 ) sqlite3DbNNFreeNN(p->db, p);
  }
}

@@ -135743,7 +138831,7 @@ SQLITE_PRIVATE KeyInfo *sqlite3KeyInfoFromExprList(
    assert( sqlite3KeyInfoIsWriteable(pInfo) );
    for(i=iStart, pItem=pList->a+iStart; i<nExpr; i++, pItem++){
      pInfo->aColl[i-iStart] = sqlite3ExprNNCollSeq(pParse, pItem->pExpr);
-
      pInfo->aSortFlags[i-iStart] = pItem->sortFlags;
+
      pInfo->aSortFlags[i-iStart] = pItem->fg.sortFlags;
    }
  }
  return pInfo;
@@ -135871,7 +138959,7 @@ static void generateSortTail(
    if( addrOnce ) sqlite3VdbeJumpHere(v, addrOnce);
    addr = 1 + sqlite3VdbeAddOp2(v, OP_SorterSort, iTab, addrBreak);
    VdbeCoverage(v);
-
    codeOffset(v, p->iOffset, addrContinue);
+
    assert( p->iLimit==0 && p->iOffset==0 );
    sqlite3VdbeAddOp3(v, OP_SorterData, iTab, regSortOut, iSortTab);
    bSeq = 0;
  }else{
@@ -135879,10 +138967,13 @@ static void generateSortTail(
    codeOffset(v, p->iOffset, addrContinue);
    iSortTab = iTab;
    bSeq = 1;
+
    if( p->iOffset>0 ){
+
      sqlite3VdbeAddOp2(v, OP_AddImm, p->iLimit, -1);
+
    }
  }
  for(i=0, iCol=nKey+bSeq-1; i<nColumn; i++){
#ifdef SQLITE_ENABLE_SORTER_REFERENCES
-
    if( aOutEx[i].bSorterRef ) continue;
+
    if( aOutEx[i].fg.bSorterRef ) continue;
#endif
    if( aOutEx[i].u.x.iOrderByCol==0 ) iCol++;
  }
@@ -135919,7 +139010,7 @@ static void generateSortTail(
#endif
  for(i=nColumn-1; i>=0; i--){
#ifdef SQLITE_ENABLE_SORTER_REFERENCES
-
    if( aOutEx[i].bSorterRef ){
+
    if( aOutEx[i].fg.bSorterRef ){
      sqlite3ExprCode(pParse, aOutEx[i].pExpr, regRow+i);
    }else
#endif
@@ -136003,9 +139094,6 @@ static void generateSortTail(
** Return a pointer to a string containing the 'declaration type' of the
** expression pExpr. The string may be treated as static by the caller.
**
-
** Also try to estimate the size of the returned value and return that
-
** result in *pEstWidth.
-
**
** The declaration type is the exact datatype definition extracted from the
** original CREATE TABLE statement if the expression is a column. The
** declaration type for a ROWID field is INTEGER. Exactly when an expression
@@ -136285,7 +139373,7 @@ SQLITE_PRIVATE void sqlite3GenerateColumnNames(
    assert( p->op!=TK_AGG_COLUMN );  /* Agg processing has not run yet */
    assert( p->op!=TK_COLUMN
        || (ExprUseYTab(p) && p->y.pTab!=0) ); /* Covering idx not yet coded */
-
    if( pEList->a[i].zEName && pEList->a[i].eEName==ENAME_NAME ){
+
    if( pEList->a[i].zEName && pEList->a[i].fg.eEName==ENAME_NAME ){
      /* An AS clause always takes first priority */
      char *zName = pEList->a[i].zEName;
      sqlite3VdbeSetColName(v, i, COLNAME_NAME, zName, SQLITE_TRANSIENT);
@@ -136370,22 +139458,25 @@ SQLITE_PRIVATE int sqlite3ColumnsFromExprList(
  *paCol = aCol;

  for(i=0, pCol=aCol; i<nCol && !db->mallocFailed; i++, pCol++){
+
    struct ExprList_item *pX = &pEList->a[i];
+
    struct ExprList_item *pCollide;
    /* Get an appropriate name for the column
    */
-
    if( (zName = pEList->a[i].zEName)!=0 && pEList->a[i].eEName==ENAME_NAME ){
+
    if( (zName = pX->zEName)!=0 && pX->fg.eEName==ENAME_NAME ){
      /* If the column contains an "AS <name>" phrase, use <name> as the name */
    }else{
-
      Expr *pColExpr = sqlite3ExprSkipCollateAndLikely(pEList->a[i].pExpr);
+
      Expr *pColExpr = sqlite3ExprSkipCollateAndLikely(pX->pExpr);
      while( ALWAYS(pColExpr!=0) && pColExpr->op==TK_DOT ){
        pColExpr = pColExpr->pRight;
        assert( pColExpr!=0 );
      }
      if( pColExpr->op==TK_COLUMN
       && ALWAYS( ExprUseYTab(pColExpr) )
-
       && (pTab = pColExpr->y.pTab)!=0
+
       && ALWAYS( pColExpr->y.pTab!=0 )
      ){
        /* For columns use the column name name */
        int iCol = pColExpr->iColumn;
+
        pTab = pColExpr->y.pTab;
        if( iCol<0 ) iCol = pTab->iPKey;
        zName = iCol>=0 ? pTab->aCol[iCol].zCnName : "rowid";
      }else if( pColExpr->op==TK_ID ){
@@ -136393,7 +139484,7 @@ SQLITE_PRIVATE int sqlite3ColumnsFromExprList(
        zName = pColExpr->u.zToken;
      }else{
        /* Use the original text of the column expression as its name */
-
        zName = pEList->a[i].zEName;
+
        assert( zName==pX->zEName );  /* pointer comparison intended */
      }
    }
    if( zName && !sqlite3IsTrueOrFalse(zName) ){
@@ -136406,7 +139497,10 @@ SQLITE_PRIVATE int sqlite3ColumnsFromExprList(
    ** append an integer to the name so that it becomes unique.
    */
    cnt = 0;
-
    while( zName && sqlite3HashFind(&ht, zName)!=0 ){
+
    while( zName && (pCollide = sqlite3HashFind(&ht, zName))!=0 ){
+
      if( pCollide->fg.bUsingTerm ){
+
        pCol->colFlags |= COLFLAG_NOEXPAND;
+
      }
      nName = sqlite3Strlen30(zName);
      if( nName>0 ){
        for(j=nName-1; j>0 && sqlite3Isdigit(zName[j]); j--){}
@@ -136417,8 +139511,11 @@ SQLITE_PRIVATE int sqlite3ColumnsFromExprList(
    }
    pCol->zCnName = zName;
    pCol->hName = sqlite3StrIHash(zName);
+
    if( pX->fg.bNoExpand ){
+
      pCol->colFlags |= COLFLAG_NOEXPAND;
+
    }
    sqlite3ColumnPropertiesFromName(0, pCol);
-
    if( zName && sqlite3HashInsert(&ht, zName, pCol)==pCol ){
+
    if( zName && sqlite3HashInsert(&ht, zName, pX)==pX ){
      sqlite3OomFault(db);
    }
  }
@@ -136675,7 +139772,7 @@ static KeyInfo *multiSelectOrderByKeyInfo(Parse *pParse, Select *p, int nExtra){
      }
      assert( sqlite3KeyInfoIsWriteable(pRet) );
      pRet->aColl[i] = pColl;
-
      pRet->aSortFlags[i] = pOrderBy->a[i].sortFlags;
+
      pRet->aSortFlags[i] = pOrderBy->a[i].fg.sortFlags;
    }
  }

@@ -136893,7 +139990,7 @@ static int multiSelectOrderBy(
** The "LIMIT of exactly 1" case of condition (1) comes about when a VALUES
** clause occurs within scalar expression (ex: "SELECT (VALUES(1),(2),(3))").
** The sqlite3CodeSubselect will have added the LIMIT 1 clause in tht case.
-
** Since the limit is exactly 1, we only need to evalutes the left-most VALUES.
+
** Since the limit is exactly 1, we only need to evaluate the left-most VALUES.
*/
static int multiSelectValues(
  Parse *pParse,        /* Parsing context */
@@ -137862,10 +140959,11 @@ static int multiSelectOrderBy(
  */
  sqlite3VdbeResolveLabel(v, labelEnd);

-
  /* Reassembly the compound query so that it will be freed correctly
-
  ** by the calling function */
+
  /* Make arrangements to free the 2nd and subsequent arms of the compound
+
  ** after the parse has finished */
  if( pSplit->pPrior ){
-
    sqlite3SelectDelete(db, pSplit->pPrior);
+
    sqlite3ParserAddCleanup(pParse,
+
       (void(*)(sqlite3*,void*))sqlite3SelectDelete, pSplit->pPrior);
  }
  pSplit->pPrior = pPrior;
  pPrior->pNext = pSplit;
@@ -137886,13 +140984,42 @@ static int multiSelectOrderBy(
**
** All references to columns in table iTable are to be replaced by corresponding
** expressions in pEList.
+
**
+
** ## About "isOuterJoin":
+
**
+
** The isOuterJoin column indicates that the replacement will occur into a
+
** position in the parent that NULL-able due to an OUTER JOIN.  Either the
+
** target slot in the parent is the right operand of a LEFT JOIN, or one of
+
** the left operands of a RIGHT JOIN.  In either case, we need to potentially
+
** bypass the substituted expression with OP_IfNullRow.
+
**
+
** Suppose the original expression is an integer constant. Even though the table
+
** has the nullRow flag set, because the expression is an integer constant,
+
** it will not be NULLed out.  So instead, we insert an OP_IfNullRow opcode
+
** that checks to see if the nullRow flag is set on the table.  If the nullRow
+
** flag is set, then the value in the register is set to NULL and the original
+
** expression is bypassed.  If the nullRow flag is not set, then the original
+
** expression runs to populate the register.
+
**
+
** Example where this is needed:
+
**
+
**      CREATE TABLE t1(a INTEGER PRIMARY KEY, b INT);
+
**      CREATE TABLE t2(x INT UNIQUE);
+
**
+
**      SELECT a,b,m,x FROM t1 LEFT JOIN (SELECT 59 AS m,x FROM t2) ON b=x;
+
**
+
** When the subquery on the right side of the LEFT JOIN is flattened, we
+
** have to add OP_IfNullRow in front of the OP_Integer that implements the
+
** "m" value of the subquery so that a NULL will be loaded instead of 59
+
** when processing a non-matched row of the left.
*/
typedef struct SubstContext {
  Parse *pParse;            /* The parsing context */
  int iTable;               /* Replace references to this table */
  int iNewTable;            /* New table number */
-
  int isLeftJoin;           /* Add TK_IF_NULL_ROW opcodes on each replacement */
+
  int isOuterJoin;          /* Add TK_IF_NULL_ROW opcodes on each replacement */
  ExprList *pEList;         /* Replacement expressions */
+
  ExprList *pCList;         /* Collation sequences for replacement expr */
} SubstContext;

/* Forward Declarations */
@@ -137917,10 +141044,11 @@ static Expr *substExpr(
  Expr *pExpr            /* Expr in which substitution occurs */
){
  if( pExpr==0 ) return 0;
-
  if( ExprHasProperty(pExpr, EP_FromJoin)
-
   && pExpr->w.iRightJoinTable==pSubst->iTable
+
  if( ExprHasProperty(pExpr, EP_OuterON|EP_InnerON)
+
   && pExpr->w.iJoin==pSubst->iTable
  ){
-
    pExpr->w.iRightJoinTable = pSubst->iNewTable;
+
    testcase( ExprHasProperty(pExpr, EP_InnerON) );
+
    pExpr->w.iJoin = pSubst->iNewTable;
  }
  if( pExpr->op==TK_COLUMN
   && pExpr->iTable==pSubst->iTable
@@ -137933,19 +141061,21 @@ static Expr *substExpr(
#endif
    {
      Expr *pNew;
-
      Expr *pCopy = pSubst->pEList->a[pExpr->iColumn].pExpr;
+
      int iColumn = pExpr->iColumn;
+
      Expr *pCopy = pSubst->pEList->a[iColumn].pExpr;
      Expr ifNullRow;
-
      assert( pSubst->pEList!=0 && pExpr->iColumn<pSubst->pEList->nExpr );
+
      assert( pSubst->pEList!=0 && iColumn<pSubst->pEList->nExpr );
      assert( pExpr->pRight==0 );
      if( sqlite3ExprIsVector(pCopy) ){
        sqlite3VectorErrorMsg(pSubst->pParse, pCopy);
      }else{
        sqlite3 *db = pSubst->pParse->db;
-
        if( pSubst->isLeftJoin && pCopy->op!=TK_COLUMN ){
+
        if( pSubst->isOuterJoin && pCopy->op!=TK_COLUMN ){
          memset(&ifNullRow, 0, sizeof(ifNullRow));
          ifNullRow.op = TK_IF_NULL_ROW;
          ifNullRow.pLeft = pCopy;
          ifNullRow.iTable = pSubst->iNewTable;
+
          ifNullRow.iColumn = -99;
          ifNullRow.flags = EP_IfNullRow;
          pCopy = &ifNullRow;
        }
@@ -137955,22 +141085,33 @@ static Expr *substExpr(
          sqlite3ExprDelete(db, pNew);
          return pExpr;
        }
-
        if( pSubst->isLeftJoin ){
+
        if( pSubst->isOuterJoin ){
          ExprSetProperty(pNew, EP_CanBeNull);
        }
-
        if( ExprHasProperty(pExpr,EP_FromJoin) ){
-
          sqlite3SetJoinExpr(pNew, pExpr->w.iRightJoinTable);
+
        if( ExprHasProperty(pExpr,EP_OuterON|EP_InnerON) ){
+
          sqlite3SetJoinExpr(pNew, pExpr->w.iJoin,
+
                             pExpr->flags & (EP_OuterON|EP_InnerON));
        }
        sqlite3ExprDelete(db, pExpr);
        pExpr = pNew;
+
        if( pExpr->op==TK_TRUEFALSE ){
+
          pExpr->u.iValue = sqlite3ExprTruthValue(pExpr);
+
          pExpr->op = TK_INTEGER;
+
          ExprSetProperty(pExpr, EP_IntValue);
+
        }

        /* Ensure that the expression now has an implicit collation sequence,
        ** just as it did when it was a column of a view or sub-query. */
-
        if( pExpr->op!=TK_COLUMN && pExpr->op!=TK_COLLATE ){
-
          CollSeq *pColl = sqlite3ExprCollSeq(pSubst->pParse, pExpr);
-
          pExpr = sqlite3ExprAddCollateString(pSubst->pParse, pExpr,
-
              (pColl ? pColl->zName : "BINARY")
+
        {
+
          CollSeq *pNat = sqlite3ExprCollSeq(pSubst->pParse, pExpr);
+
          CollSeq *pColl = sqlite3ExprCollSeq(pSubst->pParse,
+
                pSubst->pCList->a[iColumn].pExpr
          );
+
          if( pNat!=pColl || (pExpr->op!=TK_COLUMN && pExpr->op!=TK_COLLATE) ){
+
            pExpr = sqlite3ExprAddCollateString(pSubst->pParse, pExpr,
+
                (pColl ? pColl->zName : "BINARY")
+
            );
+
          }
        }
        ExprClearProperty(pExpr, EP_Collate);
      }
@@ -138123,8 +141264,8 @@ static int renumberCursorsCb(Walker *pWalker, Expr *pExpr){
  if( op==TK_COLUMN || op==TK_IF_NULL_ROW ){
    renumberCursorDoMapping(pWalker, &pExpr->iTable);
  }
-
  if( ExprHasProperty(pExpr, EP_FromJoin) ){
-
    renumberCursorDoMapping(pWalker, &pExpr->w.iRightJoinTable);
+
  if( ExprHasProperty(pExpr, EP_OuterON) ){
+
    renumberCursorDoMapping(pWalker, &pExpr->w.iJoin);
  }
  return WRC_Continue;
}
@@ -138163,6 +141304,18 @@ static void renumberCursors(
}
#endif /* !defined(SQLITE_OMIT_SUBQUERY) || !defined(SQLITE_OMIT_VIEW) */

+
/*
+
** If pSel is not part of a compound SELECT, return a pointer to its
+
** expression list. Otherwise, return a pointer to the expression list
+
** of the leftmost SELECT in the compound.
+
*/
+
static ExprList *findLeftmostExprlist(Select *pSel){
+
  while( pSel->pPrior ){
+
    pSel = pSel->pPrior;
+
  }
+
  return pSel->pEList;
+
}
+

#if !defined(SQLITE_OMIT_SUBQUERY) || !defined(SQLITE_OMIT_VIEW)
/*
** This routine attempts to flatten subqueries as a performance optimization.
@@ -138207,8 +141360,10 @@ static void renumberCursors(
**        (3a) the subquery may not be a join and
**        (3b) the FROM clause of the subquery may not contain a virtual
**             table and
-
**        (3c) the outer query may not be an aggregate.
+
**        (**) Was: "The outer query may not have a GROUP BY." This case
+
**             is now managed correctly
**        (3d) the outer query may not be DISTINCT.
+
**        See also (26) for restrictions on RIGHT JOIN.
**
**   (4)  The subquery can not be DISTINCT.
**
@@ -138260,6 +141415,11 @@ static void renumberCursors(
**              (17d2) DISTINCT
**        (17e) the subquery may not contain window functions, and
**        (17f) the subquery must not be the RHS of a LEFT JOIN.
+
**        (17g) either the subquery is the first element of the outer
+
**              query or there are no RIGHT or FULL JOINs in any arm
+
**              of the subquery.  (This is a duplicate of condition (27b).)
+
**        (17h) The corresponding result set expressions in all arms of the
+
**              compound must have the same affinity.
**
**        The parent and sub-query may contain WHERE clauses. Subject to
**        rules (11), (13) and (14), they may also contain ORDER BY,
@@ -138307,6 +141467,17 @@ static void renumberCursors(
**        function in the select list or ORDER BY clause, flattening
**        is not attempted.
**
+
**  (26)  The subquery may not be the right operand of a RIGHT JOIN.
+
**        See also (3) for restrictions on LEFT JOIN.
+
**
+
**  (27)  The subquery may not contain a FULL or RIGHT JOIN unless it
+
**        is the first element of the parent query.  Two subcases:
+
**        (27a) the subquery is not a compound query.
+
**        (27b) the subquery is a compound query and the RIGHT JOIN occurs
+
**              in any arm of the compound query.  (See also (17g).)
+
**
+
**  (28)  The subquery is not a MATERIALIZED CTE.
+
**
**
** In this routine, the "p" parameter is a pointer to the outer query.
** The subquery is p->pSrc->a[iFrom].  isAgg is true if the outer query
@@ -138332,7 +141503,7 @@ static int flattenSubquery(
  SrcList *pSubSrc;   /* The FROM clause of the subquery */
  int iParent;        /* VDBE cursor number of the pSub result set temp table */
  int iNewParent = -1;/* Replacement table for iParent */
-
  int isLeftJoin = 0; /* True if pSub is the right side of a LEFT JOIN */
+
  int isOuterJoin = 0; /* True if pSub is the right side of a LEFT JOIN */
  int i;              /* Loop counter */
  Expr *pWhere;                    /* The WHERE clause */
  SrcItem *pSubitem;               /* The subquery */
@@ -138398,32 +141569,26 @@ static int flattenSubquery(
  **
  ** which is not at all the same thing.
  **
-
  ** If the subquery is the right operand of a LEFT JOIN, then the outer
-
  ** query cannot be an aggregate. (3c)  This is an artifact of the way
-
  ** aggregates are processed - there is no mechanism to determine if
-
  ** the LEFT JOIN table should be all-NULL.
-
  **
  ** See also tickets #306, #350, and #3300.
  */
-
  if( (pSubitem->fg.jointype & JT_OUTER)!=0 ){
-
    isLeftJoin = 1;
-
    if( pSubSrc->nSrc>1                   /* (3a) */
-
     || isAgg                             /* (3b) */
-
     || IsVirtual(pSubSrc->a[0].pTab)     /* (3c) */
-
     || (p->selFlags & SF_Distinct)!=0    /* (3d) */
+
  if( (pSubitem->fg.jointype & (JT_OUTER|JT_LTORJ))!=0 ){
+
    if( pSubSrc->nSrc>1                        /* (3a) */
+
     || IsVirtual(pSubSrc->a[0].pTab)          /* (3b) */
+
     || (p->selFlags & SF_Distinct)!=0         /* (3d) */
+
     || (pSubitem->fg.jointype & JT_RIGHT)!=0  /* (26) */
    ){
      return 0;
    }
+
    isOuterJoin = 1;
  }
-
#ifdef SQLITE_EXTRA_IFNULLROW
-
  else if( iFrom>0 && !isAgg ){
-
    /* Setting isLeftJoin to -1 causes OP_IfNullRow opcodes to be generated for
-
    ** every reference to any result column from subquery in a join, even
-
    ** though they are not necessary.  This will stress-test the OP_IfNullRow
-
    ** opcode. */
-
    isLeftJoin = -1;
+

+
  assert( pSubSrc->nSrc>0 );  /* True by restriction (7) */
+
  if( iFrom>0 && (pSubSrc->a[0].fg.jointype & JT_LTORJ)!=0 ){
+
    return 0;   /* Restriction (27a) */
+
  }
+
  if( pSubitem->fg.isCte && pSubitem->u2.pCteUse->eM10d==M10d_Yes ){
+
    return 0;       /* (28) */
  }
-
#endif

  /* Restriction (17): If the sub-query is a compound SELECT, then it must
  ** use only the UNION ALL operator. And none of the simple select queries
@@ -138431,10 +141596,11 @@ static int flattenSubquery(
  ** queries.
  */
  if( pSub->pPrior ){
+
    int ii;
    if( pSub->pOrderBy ){
      return 0;  /* Restriction (20) */
    }
-
    if( isAgg || (p->selFlags & SF_Distinct)!=0 || isLeftJoin>0 ){
+
    if( isAgg || (p->selFlags & SF_Distinct)!=0 || isOuterJoin>0 ){
      return 0; /* (17d1), (17d2), or (17f) */
    }
    for(pSub1=pSub; pSub1; pSub1=pSub1->pPrior){
@@ -138452,12 +141618,17 @@ static int flattenSubquery(
      ){
        return 0;
      }
+
      if( iFrom>0 && (pSub1->pSrc->a[0].fg.jointype & JT_LTORJ)!=0 ){
+
        /* Without this restriction, the JT_LTORJ flag would end up being
+
        ** omitted on left-hand tables of the right join that is being
+
        ** flattened. */
+
        return 0;   /* Restrictions (17g), (27b) */
+
      }
      testcase( pSub1->pSrc->nSrc>1 );
    }

    /* Restriction (18). */
    if( p->pOrderBy ){
-
      int ii;
      for(ii=0; ii<p->pOrderBy->nExpr; ii++){
        if( p->pOrderBy->a[ii].u.x.iOrderByCol==0 ) return 0;
      }
@@ -138466,8 +141637,24 @@ static int flattenSubquery(
    /* Restriction (23) */
    if( (p->selFlags & SF_Recursive) ) return 0;

+
    /* Restriction (17h) */
+
    for(ii=0; ii<pSub->pEList->nExpr; ii++){
+
      char aff;
+
      assert( pSub->pEList->a[ii].pExpr!=0 );
+
      aff = sqlite3ExprAffinity(pSub->pEList->a[ii].pExpr);
+
      for(pSub1=pSub->pPrior; pSub1; pSub1=pSub1->pPrior){
+
        assert( pSub1->pEList!=0 );
+
        assert( pSub1->pEList->nExpr>ii );
+
        assert( pSub1->pEList->a[ii].pExpr!=0 );
+
        if( sqlite3ExprAffinity(pSub1->pEList->a[ii].pExpr)!=aff ){
+
          return 0;
+
        }
+
      }
+
    }
+

    if( pSrc->nSrc>1 ){
      if( pParse->nSelect>500 ) return 0;
+
      if( OptimizationDisabled(db, SQLITE_FlttnUnionAll) ) return 0;
      aCsrMap = sqlite3DbMallocZero(db, ((i64)pParse->nTab+1)*sizeof(int));
      if( aCsrMap ) aCsrMap[0] = pParse->nTab;
    }
@@ -138492,7 +141679,7 @@ static int flattenSubquery(
  pSubitem->zName = 0;
  pSubitem->zAlias = 0;
  pSubitem->pSelect = 0;
-
  assert( pSubitem->pOn==0 );
+
  assert( pSubitem->fg.isUsing!=0 || pSubitem->u3.pOn==0 );

  /* If the sub-query is a compound SELECT statement, then (by restrictions
  ** 17 and 18 above) it must be a UNION ALL and the parent query must
@@ -138602,6 +141789,7 @@ static int flattenSubquery(
  for(pParent=p; pParent; pParent=pParent->pPrior, pSub=pSub->pPrior){
    int nSubSrc;
    u8 jointype = 0;
+
    u8 ltorj = pSrc->a[iFrom].fg.jointype & JT_LTORJ;
    assert( pSub!=0 );
    pSubSrc = pSub->pSrc;     /* FROM clause of subquery */
    nSubSrc = pSubSrc->nSrc;  /* Number of terms in subquery FROM clause */
@@ -138636,13 +141824,16 @@ static int flattenSubquery(
    ** outer query.
    */
    for(i=0; i<nSubSrc; i++){
-
      sqlite3IdListDelete(db, pSrc->a[i+iFrom].pUsing);
-
      assert( pSrc->a[i+iFrom].fg.isTabFunc==0 );
-
      pSrc->a[i+iFrom] = pSubSrc->a[i];
+
      SrcItem *pItem = &pSrc->a[i+iFrom];
+
      if( pItem->fg.isUsing ) sqlite3IdListDelete(db, pItem->u3.pUsing);
+
      assert( pItem->fg.isTabFunc==0 );
+
      *pItem = pSubSrc->a[i];
+
      pItem->fg.jointype |= ltorj;
      iNewParent = pSubSrc->a[i].iCursor;
      memset(&pSubSrc->a[i], 0, sizeof(pSubSrc->a[i]));
    }
-
    pSrc->a[iFrom].fg.jointype = jointype;
+
    pSrc->a[iFrom].fg.jointype &= JT_LTORJ;
+
    pSrc->a[iFrom].fg.jointype |= jointype | ltorj;

    /* Now begin substituting subquery result set expressions for
    ** references to the iParent in the outer query.
@@ -138677,8 +141868,8 @@ static int flattenSubquery(
    }
    pWhere = pSub->pWhere;
    pSub->pWhere = 0;
-
    if( isLeftJoin>0 ){
-
      sqlite3SetJoinExpr(pWhere, iNewParent);
+
    if( isOuterJoin>0 ){
+
      sqlite3SetJoinExpr(pWhere, iNewParent, EP_OuterON);
    }
    if( pWhere ){
      if( pParent->pWhere ){
@@ -138692,8 +141883,9 @@ static int flattenSubquery(
      x.pParse = pParse;
      x.iTable = iParent;
      x.iNewTable = iNewParent;
-
      x.isLeftJoin = isLeftJoin;
+
      x.isOuterJoin = isOuterJoin;
      x.pEList = pSub->pEList;
+
      x.pCList = findLeftmostExprlist(pSub);
      substSelect(&x, pParent, 0);
    }

@@ -138713,7 +141905,7 @@ static int flattenSubquery(
      pSub->pLimit = 0;
    }

-
    /* Recompute the SrcList_item.colUsed masks for the flattened
+
    /* Recompute the SrcItem.colUsed masks for the flattened
    ** tables. */
    for(i=0; i<nSubSrc; i++){
      recomputeColumnsUsed(pParent, &pSrc->a[i+iFrom]);
@@ -138727,8 +141919,8 @@ static int flattenSubquery(
  sqlite3WalkSelect(&w,pSub1);
  sqlite3SelectDelete(db, pSub1);

-
#if SELECTTRACE_ENABLED
-
  if( sqlite3SelectTrace & 0x100 ){
+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x100 ){
    SELECTTRACE(0x100,pParse,p,("After flattening:\n"));
    sqlite3TreeViewSelect(0, p, 0);
  }
@@ -138749,6 +141941,8 @@ struct WhereConst {
  int nConst;      /* Number for COLUMN=CONSTANT terms */
  int nChng;       /* Number of times a constant is propagated */
  int bHasAffBlob; /* At least one column in apExpr[] as affinity BLOB */
+
  u32 mExcludeOn;  /* Which ON expressions to exclude from considertion.
+
                   ** Either EP_OuterON or EP_InnerON|EP_OuterON */
  Expr **apExpr;   /* [i*2] is COLUMN and [i*2+1] is VALUE */
};

@@ -138811,7 +142005,11 @@ static void constInsert(
static void findConstInWhere(WhereConst *pConst, Expr *pExpr){
  Expr *pRight, *pLeft;
  if( NEVER(pExpr==0) ) return;
-
  if( ExprHasProperty(pExpr, EP_FromJoin) ) return;
+
  if( ExprHasProperty(pExpr, pConst->mExcludeOn) ){
+
    testcase( ExprHasProperty(pExpr, EP_OuterON) );
+
    testcase( ExprHasProperty(pExpr, EP_InnerON) );
+
    return;
+
  }
  if( pExpr->op==TK_AND ){
    findConstInWhere(pConst, pExpr->pRight);
    findConstInWhere(pConst, pExpr->pLeft);
@@ -138847,9 +142045,10 @@ static int propagateConstantExprRewriteOne(
  int i;
  if( pConst->pOomFault[0] ) return WRC_Prune;
  if( pExpr->op!=TK_COLUMN ) return WRC_Continue;
-
  if( ExprHasProperty(pExpr, EP_FixedCol|EP_FromJoin) ){
+
  if( ExprHasProperty(pExpr, EP_FixedCol|pConst->mExcludeOn) ){
    testcase( ExprHasProperty(pExpr, EP_FixedCol) );
-
    testcase( ExprHasProperty(pExpr, EP_FromJoin) );
+
    testcase( ExprHasProperty(pExpr, EP_OuterON) );
+
    testcase( ExprHasProperty(pExpr, EP_InnerON) );
    return WRC_Continue;
  }
  for(i=0; i<pConst->nConst; i++){
@@ -138973,6 +142172,17 @@ static int propagateConstants(
    x.nChng = 0;
    x.apExpr = 0;
    x.bHasAffBlob = 0;
+
    if( ALWAYS(p->pSrc!=0)
+
     && p->pSrc->nSrc>0
+
     && (p->pSrc->a[0].fg.jointype & JT_LTORJ)!=0
+
    ){
+
      /* Do not propagate constants on any ON clause if there is a
+
      ** RIGHT JOIN anywhere in the query */
+
      x.mExcludeOn = EP_InnerON | EP_OuterON;
+
    }else{
+
      /* Do not propagate constants through the ON clause of a LEFT JOIN */
+
      x.mExcludeOn = EP_OuterON;
+
    }
    findConstInWhere(&x, p->pWhere);
    if( x.nConst ){
      memset(&w, 0, sizeof(w));
@@ -139085,6 +142295,13 @@ static int pushDownWindowCheck(Parse *pParse, Select *pSubq, Expr *pExpr){
**       be materialized.  (This restriction is implemented in the calling
**       routine.)
**
+
**   (8) The subquery may not be a compound that uses UNION, INTERSECT,
+
**       or EXCEPT.  (We could, perhaps, relax this restriction to allow
+
**       this case if none of the comparisons operators between left and
+
**       right arms of the compound use a collation other than BINARY.
+
**       But it is a lot of work to check that case for an obscure and
+
**       minor optimization, so we omit it for now.)
+
**
** Return 0 if no changes are made and non-zero if one or more WHERE clause
** terms are duplicated into the subquery.
*/
@@ -139098,11 +142315,16 @@ static int pushDownWhereTerms(
  int nChng = 0;
  if( pWhere==0 ) return 0;
  if( pSubq->selFlags & (SF_Recursive|SF_MultiPart) ) return 0;
+
  if( pSrc->fg.jointype & (JT_LTORJ|JT_RIGHT) ) return 0;

#ifndef SQLITE_OMIT_WINDOWFUNC
  if( pSubq->pPrior ){
    Select *pSel;
    for(pSel=pSubq; pSel; pSel=pSel->pPrior){
+
      u8 op = pSel->op;
+
      assert( op==TK_ALL || op==TK_SELECT
+
           || op==TK_UNION || op==TK_INTERSECT || op==TK_EXCEPT );
+
      if( op!=TK_ALL && op!=TK_SELECT ) return 0;  /* restriction (8) */
      if( pSel->pWin ) return 0;    /* restriction (6b) */
    }
  }else{
@@ -139133,13 +142355,13 @@ static int pushDownWhereTerms(

#if 0  /* Legacy code. Checks now done by sqlite3ExprIsTableConstraint() */
  if( isLeftJoin
-
   && (ExprHasProperty(pWhere,EP_FromJoin)==0
-
         || pWhere->w.iRightJoinTable!=iCursor)
+
   && (ExprHasProperty(pWhere,EP_OuterON)==0
+
         || pWhere->w.iJoin!=iCursor)
  ){
    return 0; /* restriction (4) */
  }
-
  if( ExprHasProperty(pWhere,EP_FromJoin)
-
   && pWhere->w.iRightJoinTable!=iCursor
+
  if( ExprHasProperty(pWhere,EP_OuterON)
+
   && pWhere->w.iJoin!=iCursor
  ){
    return 0; /* restriction (5) */
  }
@@ -139151,12 +142373,13 @@ static int pushDownWhereTerms(
    while( pSubq ){
      SubstContext x;
      pNew = sqlite3ExprDup(pParse->db, pWhere, 0);
-
      unsetJoinExpr(pNew, -1);
+
      unsetJoinExpr(pNew, -1, 1);
      x.pParse = pParse;
      x.iTable = pSrc->iCursor;
      x.iNewTable = pSrc->iCursor;
-
      x.isLeftJoin = 0;
+
      x.isOuterJoin = 0;
      x.pEList = pSubq->pEList;
+
      x.pCList = findLeftmostExprlist(pSubq);
      pNew = substExpr(&x, pNew);
#ifndef SQLITE_OMIT_WINDOWFUNC
      if( pSubq->pWin && 0==pushDownWindowCheck(pParse, pSubq, pNew) ){
@@ -139228,7 +142451,7 @@ static u8 minMaxQuery(sqlite3 *db, Expr *pFunc, ExprList **ppMinMax){
  }
  *ppMinMax = pOrderBy = sqlite3ExprListDup(db, pEList, 0);
  assert( pOrderBy!=0 || db->mallocFailed );
-
  if( pOrderBy ) pOrderBy->a[0].sortFlags = sortFlags;
+
  if( pOrderBy ) pOrderBy->a[0].fg.sortFlags = sortFlags;
  return eRet;
}

@@ -139260,6 +142483,7 @@ static Table *isSimpleCount(Select *p, AggInfo *pAggInfo){
   || p->pSrc->nSrc!=1
   || p->pSrc->a[0].pSelect
   || pAggInfo->nFunc!=1
+
   || p->pHaving
  ){
    return 0;
  }
@@ -139364,7 +142588,7 @@ static int convertCompoundSelectToSubquery(Walker *pWalker, Select *p){
  pNew = sqlite3DbMallocZero(db, sizeof(*pNew) );
  if( pNew==0 ) return WRC_Abort;
  memset(&dummy, 0, sizeof(dummy));
-
  pNewSrc = sqlite3SrcListAppendFromTerm(pParse,0,0,0,&dummy,pNew,0,0);
+
  pNewSrc = sqlite3SrcListAppendFromTerm(pParse,0,0,0,&dummy,pNew,0);
  if( pNewSrc==0 ) return WRC_Abort;
  *pNew = *p;
  p->pSrc = pNewSrc;
@@ -139680,9 +142904,9 @@ SQLITE_PRIVATE void sqlite3SelectPopWith(Walker *pWalker, Select *p){
#endif

/*
-
** The SrcList_item structure passed as the second argument represents a
+
** The SrcItem structure passed as the second argument represents a
** sub-query in the FROM clause of a SELECT statement. This function
-
** allocates and populates the SrcList_item.pTab object. If successful,
+
** allocates and populates the SrcItem.pTab object. If successful,
** SQLITE_OK is returned. Otherwise, if an OOM error is encountered,
** SQLITE_NOMEM.
*/
@@ -139697,7 +142921,7 @@ SQLITE_PRIVATE int sqlite3ExpandSubquery(Parse *pParse, SrcItem *pFrom){
  if( pFrom->zAlias ){
    pTab->zName = sqlite3DbStrDup(pParse->db, pFrom->zAlias);
  }else{
-
    pTab->zName = sqlite3MPrintf(pParse->db, "subquery_%u", pSel->selId);
+
    pTab->zName = sqlite3MPrintf(pParse->db, "%!S", pFrom);
  }
  while( pSel->pPrior ){ pSel = pSel->pPrior; }
  sqlite3ColumnsFromExprList(pParse, pSel->pEList,&pTab->nCol,&pTab->aCol);
@@ -139709,11 +142933,35 @@ SQLITE_PRIVATE int sqlite3ExpandSubquery(Parse *pParse, SrcItem *pFrom){
#else
  pTab->tabFlags |= TF_Ephemeral;  /* Legacy compatibility mode */
#endif
+
  return pParse->nErr ? SQLITE_ERROR : SQLITE_OK;
+
}


-
  return pParse->nErr ? SQLITE_ERROR : SQLITE_OK;
+
/*
+
** Check the N SrcItem objects to the right of pBase.  (N might be zero!)
+
** If any of those SrcItem objects have a USING clause containing zName
+
** then return true.
+
**
+
** If N is zero, or none of the N SrcItem objects to the right of pBase
+
** contains a USING clause, or if none of the USING clauses contain zName,
+
** then return false.
+
*/
+
static int inAnyUsingClause(
+
  const char *zName, /* Name we are looking for */
+
  SrcItem *pBase,    /* The base SrcItem.  Looking at pBase[1] and following */
+
  int N              /* How many SrcItems to check */
+
){
+
  while( N>0 ){
+
    N--;
+
    pBase++;
+
    if( pBase->fg.isUsing==0 ) continue;
+
    if( NEVER(pBase->u3.pUsing==0) ) continue;
+
    if( sqlite3IdListIndex(pBase->u3.pUsing, zName)>=0 ) return 1;
+
  }
+
  return 0;
}

+

/*
** This routine is a Walker callback for "expanding" a SELECT statement.
** "Expanding" means to do the following:
@@ -139863,7 +143111,7 @@ static int selectExpander(Walker *pWalker, Select *p){
  /* Process NATURAL keywords, and ON and USING clauses of joins.
  */
  assert( db->mallocFailed==0 || pParse->nErr!=0 );
-
  if( pParse->nErr || sqliteProcessJoin(pParse, p) ){
+
  if( pParse->nErr || sqlite3ProcessJoin(pParse, p) ){
    return WRC_Abort;
  }

@@ -139911,7 +143159,7 @@ static int selectExpander(Walker *pWalker, Select *p){
        pNew = sqlite3ExprListAppend(pParse, pNew, a[k].pExpr);
        if( pNew ){
          pNew->a[pNew->nExpr-1].zEName = a[k].zEName;
-
          pNew->a[pNew->nExpr-1].eEName = a[k].eEName;
+
          pNew->a[pNew->nExpr-1].fg.eEName = a[k].fg.eEName;
          a[k].zEName = 0;
        }
        a[k].pExpr = 0;
@@ -139926,32 +143174,60 @@ static int selectExpander(Walker *pWalker, Select *p){
          zTName = pE->pLeft->u.zToken;
        }
        for(i=0, pFrom=pTabList->a; i<pTabList->nSrc; i++, pFrom++){
-
          Table *pTab = pFrom->pTab;
-
          Select *pSub = pFrom->pSelect;
-
          char *zTabName = pFrom->zAlias;
-
          const char *zSchemaName = 0;
-
          int iDb;
-
          if( zTabName==0 ){
+
          Table *pTab = pFrom->pTab;   /* Table for this data source */
+
          ExprList *pNestedFrom;       /* Result-set of a nested FROM clause */
+
          char *zTabName;              /* AS name for this data source */
+
          const char *zSchemaName = 0; /* Schema name for this data source */
+
          int iDb;                     /* Schema index for this data src */
+
          IdList *pUsing;              /* USING clause for pFrom[1] */
+

+
          if( (zTabName = pFrom->zAlias)==0 ){
            zTabName = pTab->zName;
          }
          if( db->mallocFailed ) break;
-
          if( pSub==0 || (pSub->selFlags & SF_NestedFrom)==0 ){
-
            pSub = 0;
+
          assert( (int)pFrom->fg.isNestedFrom == IsNestedFrom(pFrom->pSelect) );
+
          if( pFrom->fg.isNestedFrom ){
+
            assert( pFrom->pSelect!=0 );
+
            pNestedFrom = pFrom->pSelect->pEList;
+
            assert( pNestedFrom!=0 );
+
            assert( pNestedFrom->nExpr==pTab->nCol );
+
          }else{
            if( zTName && sqlite3StrICmp(zTName, zTabName)!=0 ){
              continue;
            }
+
            pNestedFrom = 0;
            iDb = sqlite3SchemaToIndex(db, pTab->pSchema);
            zSchemaName = iDb>=0 ? db->aDb[iDb].zDbSName : "*";
          }
+
          if( i+1<pTabList->nSrc
+
           && pFrom[1].fg.isUsing
+
           && (selFlags & SF_NestedFrom)!=0
+
          ){
+
            int ii;
+
            pUsing = pFrom[1].u3.pUsing;
+
            for(ii=0; ii<pUsing->nId; ii++){
+
              const char *zUName = pUsing->a[ii].zName;
+
              pRight = sqlite3Expr(db, TK_ID, zUName);
+
              pNew = sqlite3ExprListAppend(pParse, pNew, pRight);
+
              if( pNew ){
+
                struct ExprList_item *pX = &pNew->a[pNew->nExpr-1];
+
                assert( pX->zEName==0 );
+
                pX->zEName = sqlite3MPrintf(db,"..%s", zUName);
+
                pX->fg.eEName = ENAME_TAB;
+
                pX->fg.bUsingTerm = 1;
+
              }
+
            }
+
          }else{
+
            pUsing = 0;
+
          }
          for(j=0; j<pTab->nCol; j++){
            char *zName = pTab->aCol[j].zCnName;
-
            char *zColname;  /* The computed column name */
-
            char *zToFree;   /* Malloced string that needs to be freed */
-
            Token sColname;  /* Computed column name as a token */
+
            struct ExprList_item *pX; /* Newly added ExprList term */

            assert( zName );
-
            if( zTName && pSub
-
             && sqlite3MatchEName(&pSub->pEList->a[j], 0, zTName, 0)==0
+
            if( zTName
+
             && pNestedFrom
+
             && sqlite3MatchEName(&pNestedFrom->a[j], 0, zTName, 0)==0
            ){
              continue;
            }
@@ -139965,57 +143241,75 @@ static int selectExpander(Walker *pWalker, Select *p){
            ){
              continue;
            }
+
            if( (pTab->aCol[j].colFlags & COLFLAG_NOEXPAND)!=0
+
             && zTName==0
+
             && (selFlags & (SF_NestedFrom))==0
+
            ){
+
              continue;
+
            }
            tableSeen = 1;

-
            if( i>0 && zTName==0 ){
-
              if( (pFrom->fg.jointype & JT_NATURAL)!=0
-
                && tableAndColumnIndex(pTabList, i, zName, 0, 0, 1)
+
            if( i>0 && zTName==0 && (selFlags & SF_NestedFrom)==0 ){
+
              if( pFrom->fg.isUsing
+
               && sqlite3IdListIndex(pFrom->u3.pUsing, zName)>=0
              ){
-
                /* In a NATURAL join, omit the join columns from the
-
                ** table to the right of the join */
-
                continue;
-
              }
-
              if( sqlite3IdListIndex(pFrom->pUsing, zName)>=0 ){
                /* In a join with a USING clause, omit columns in the
                ** using clause from the table on the right. */
                continue;
              }
            }
            pRight = sqlite3Expr(db, TK_ID, zName);
-
            zColname = zName;
-
            zToFree = 0;
-
            if( longNames || pTabList->nSrc>1 ){
+
            if( (pTabList->nSrc>1
+
                 && (  (pFrom->fg.jointype & JT_LTORJ)==0
+
                     || (selFlags & SF_NestedFrom)!=0
+
                     || !inAnyUsingClause(zName,pFrom,pTabList->nSrc-i-1)
+
                    )
+
                )
+
             || IN_RENAME_OBJECT
+
            ){
              Expr *pLeft;
              pLeft = sqlite3Expr(db, TK_ID, zTabName);
              pExpr = sqlite3PExpr(pParse, TK_DOT, pLeft, pRight);
+
              if( IN_RENAME_OBJECT && pE->pLeft ){
+
                sqlite3RenameTokenRemap(pParse, pLeft, pE->pLeft);
+
              }
              if( zSchemaName ){
                pLeft = sqlite3Expr(db, TK_ID, zSchemaName);
                pExpr = sqlite3PExpr(pParse, TK_DOT, pLeft, pExpr);
              }
-
              if( longNames ){
-
                zColname = sqlite3MPrintf(db, "%s.%s", zTabName, zName);
-
                zToFree = zColname;
-
              }
            }else{
              pExpr = pRight;
            }
            pNew = sqlite3ExprListAppend(pParse, pNew, pExpr);
-
            sqlite3TokenInit(&sColname, zColname);
-
            sqlite3ExprListSetName(pParse, pNew, &sColname, 0);
-
            if( pNew && (p->selFlags & SF_NestedFrom)!=0 && !IN_RENAME_OBJECT ){
-
              struct ExprList_item *pX = &pNew->a[pNew->nExpr-1];
-
              sqlite3DbFree(db, pX->zEName);
-
              if( pSub ){
-
                pX->zEName = sqlite3DbStrDup(db, pSub->pEList->a[j].zEName);
+
            if( pNew==0 ){
+
              break;  /* OOM */
+
            }
+
            pX = &pNew->a[pNew->nExpr-1];
+
            assert( pX->zEName==0 );
+
            if( (selFlags & SF_NestedFrom)!=0 && !IN_RENAME_OBJECT ){
+
              if( pNestedFrom ){
+
                pX->zEName = sqlite3DbStrDup(db, pNestedFrom->a[j].zEName);
                testcase( pX->zEName==0 );
              }else{
                pX->zEName = sqlite3MPrintf(db, "%s.%s.%s",
-
                                           zSchemaName, zTabName, zColname);
+
                                           zSchemaName, zTabName, zName);
                testcase( pX->zEName==0 );
              }
-
              pX->eEName = ENAME_TAB;
+
              pX->fg.eEName = ENAME_TAB;
+
              if( (pFrom->fg.isUsing
+
                   && sqlite3IdListIndex(pFrom->u3.pUsing, zName)>=0)
+
               || (pUsing && sqlite3IdListIndex(pUsing, zName)>=0)
+
               || (pTab->aCol[j].colFlags & COLFLAG_NOEXPAND)!=0
+
              ){
+
                pX->fg.bNoExpand = 1;
+
              }
+
            }else if( longNames ){
+
              pX->zEName = sqlite3MPrintf(db, "%s.%s", zTabName, zName);
+
              pX->fg.eEName = ENAME_NAME;
+
            }else{
+
              pX->zEName = sqlite3DbStrDup(db, zName);
+
              pX->fg.eEName = ENAME_NAME;
            }
-
            sqlite3DbFree(db, zToFree);
          }
        }
        if( !tableSeen ){
@@ -140039,6 +143333,12 @@ static int selectExpander(Walker *pWalker, Select *p){
      p->selFlags |= SF_ComplexResult;
    }
  }
+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x100 ){
+
    SELECTTRACE(0x100,pParse,p,("After result-set wildcard expansion:\n"));
+
    sqlite3TreeViewSelect(0, p, 0);
+
  }
+
#endif
  return WRC_Continue;
}

@@ -140429,8 +143729,8 @@ static void havingToWhere(Parse *pParse, Select *p){
  sWalker.xExprCallback = havingToWhereExprCb;
  sWalker.u.pSelect = p;
  sqlite3WalkExpr(&sWalker, p->pHaving);
-
#if SELECTTRACE_ENABLED
-
  if( sWalker.eCode && (sqlite3SelectTrace & 0x100)!=0 ){
+
#if TREETRACE_ENABLED
+
  if( sWalker.eCode && (sqlite3TreeTrace & 0x100)!=0 ){
    SELECTTRACE(0x100,pParse,p,("Move HAVING terms into WHERE:\n"));
    sqlite3TreeViewSelect(0, p, 0);
  }
@@ -140439,7 +143739,7 @@ static void havingToWhere(Parse *pParse, Select *p){

/*
** Check to see if the pThis entry of pTabList is a self-join of a prior view.
-
** If it is, then return the SrcList_item for the prior view.  If it is not,
+
** If it is, then return the SrcItem for the prior view.  If it is not,
** then return 0.
*/
static SrcItem *isSelfJoinView(
@@ -140562,8 +143862,8 @@ static int countOfViewOptimization(Parse *pParse, Select *p){
  p->pEList->a[0].pExpr = pExpr;
  p->selFlags &= ~SF_Aggregate;

-
#if SELECTTRACE_ENABLED
-
  if( sqlite3SelectTrace & 0x400 ){
+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x400 ){
    SELECTTRACE(0x400,pParse,p,("After count-of-view optimization:\n"));
    sqlite3TreeViewSelect(0, p, 0);
  }
@@ -140573,6 +143873,29 @@ static int countOfViewOptimization(Parse *pParse, Select *p){
#endif /* SQLITE_COUNTOFVIEW_OPTIMIZATION */

/*
+
** If any term of pSrc, or any SF_NestedFrom sub-query, is not the same
+
** as pSrcItem but has the same alias as p0, then return true.
+
** Otherwise return false.
+
*/
+
static int sameSrcAlias(SrcItem *p0, SrcList *pSrc){
+
  int i;
+
  for(i=0; i<pSrc->nSrc; i++){
+
    SrcItem *p1 = &pSrc->a[i];
+
    if( p1==p0 ) continue;
+
    if( p0->pTab==p1->pTab && 0==sqlite3_stricmp(p0->zAlias, p1->zAlias) ){
+
      return 1;
+
    }
+
    if( p1->pSelect
+
     && (p1->pSelect->selFlags & SF_NestedFrom)!=0
+
     && sameSrcAlias(p0, p1->pSelect->pSrc)
+
    ){
+
      return 1;
+
    }
+
  }
+
  return 0;
+
}
+

+
/*
** Generate code for the SELECT statement given in the p argument.
**
** The results are returned according to the SelectDest structure.
@@ -140616,10 +143939,14 @@ SQLITE_PRIVATE int sqlite3Select(
  }
  assert( db->mallocFailed==0 );
  if( sqlite3AuthCheck(pParse, SQLITE_SELECT, 0, 0, 0) ) return 1;
-
#if SELECTTRACE_ENABLED
+
#if TREETRACE_ENABLED
  SELECTTRACE(1,pParse,p, ("begin processing:\n", pParse->addrExplain));
-
  if( sqlite3SelectTrace & 0x100 ){
-
    sqlite3TreeViewSelect(0, p, 0);
+
  if( sqlite3TreeTrace & 0x10100 ){
+
    if( (sqlite3TreeTrace & 0x10001)==0x10000 ){
+
      sqlite3TreeViewLine(0, "In sqlite3Select() at %s:%d",
+
                           __FILE__, __LINE__);
+
    }
+
    sqlite3ShowSelect(p);
  }
#endif

@@ -140633,9 +143960,9 @@ SQLITE_PRIVATE int sqlite3Select(
           pDest->eDest==SRT_DistQueue  || pDest->eDest==SRT_DistFifo );
    /* All of these destinations are also able to ignore the ORDER BY clause */
    if( p->pOrderBy ){
-
#if SELECTTRACE_ENABLED
+
#if TREETRACE_ENABLED
      SELECTTRACE(1,pParse,p, ("dropping superfluous ORDER BY:\n"));
-
      if( sqlite3SelectTrace & 0x100 ){
+
      if( sqlite3TreeTrace & 0x100 ){
        sqlite3TreeViewExprList(0, p->pOrderBy, 0, "ORDERBY");
      }
#endif
@@ -140654,8 +143981,8 @@ SQLITE_PRIVATE int sqlite3Select(
  }
  assert( db->mallocFailed==0 );
  assert( p->pEList!=0 );
-
#if SELECTTRACE_ENABLED
-
  if( sqlite3SelectTrace & 0x104 ){
+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x104 ){
    SELECTTRACE(0x104,pParse,p, ("after name resolution:\n"));
    sqlite3TreeViewSelect(0, p, 0);
  }
@@ -140672,15 +143999,12 @@ SQLITE_PRIVATE int sqlite3Select(
  ** disallow it altogether.  */
  if( p->selFlags & SF_UFSrcCheck ){
    SrcItem *p0 = &p->pSrc->a[0];
-
    for(i=1; i<p->pSrc->nSrc; i++){
-
      SrcItem *p1 = &p->pSrc->a[i];
-
      if( p0->pTab==p1->pTab && 0==sqlite3_stricmp(p0->zAlias, p1->zAlias) ){
-
        sqlite3ErrorMsg(pParse,
-
            "target object/alias may not appear in FROM clause: %s",
-
            p0->zAlias ? p0->zAlias : p0->pTab->zName
-
        );
-
        goto select_end;
-
      }
+
    if( sameSrcAlias(p0, p->pSrc) ){
+
      sqlite3ErrorMsg(pParse,
+
          "target object/alias may not appear in FROM clause: %s",
+
          p0->zAlias ? p0->zAlias : p0->pTab->zName
+
      );
+
      goto select_end;
    }

    /* Clear the SF_UFSrcCheck flag. The check has already been performed,
@@ -140699,8 +144023,8 @@ SQLITE_PRIVATE int sqlite3Select(
    assert( pParse->nErr );
    goto select_end;
  }
-
#if SELECTTRACE_ENABLED
-
  if( p->pWin && (sqlite3SelectTrace & 0x108)!=0 ){
+
#if TREETRACE_ENABLED
+
  if( p->pWin && (sqlite3TreeTrace & 0x108)!=0 ){
    SELECTTRACE(0x104,pParse,p, ("after window rewrite:\n"));
    sqlite3TreeViewSelect(0, p, 0);
  }
@@ -140728,14 +144052,16 @@ SQLITE_PRIVATE int sqlite3Select(
    /* Convert LEFT JOIN into JOIN if there are terms of the right table
    ** of the LEFT JOIN used in the WHERE clause.
    */
-
    if( (pItem->fg.jointype & JT_LEFT)!=0
+
    if( (pItem->fg.jointype & (JT_LEFT|JT_RIGHT))==JT_LEFT
     && sqlite3ExprImpliesNonNullRow(p->pWhere, pItem->iCursor)
     && OptimizationEnabled(db, SQLITE_SimplifyJoin)
    ){
      SELECTTRACE(0x100,pParse,p,
                ("LEFT-JOIN simplifies to JOIN on term %d\n",i));
      pItem->fg.jointype &= ~(JT_LEFT|JT_OUTER);
-
      unsetJoinExpr(p->pWhere, pItem->iCursor);
+
      assert( pItem->iCursor>=0 );
+
      unsetJoinExpr(p->pWhere, pItem->iCursor,
+
                    pTabList->a[0].fg.jointype & JT_LTORJ);
    }

    /* No futher action if this term of the FROM clause is no a subquery */
@@ -140788,7 +144114,9 @@ SQLITE_PRIVATE int sqlite3Select(
    ){
      SELECTTRACE(0x100,pParse,p,
                ("omit superfluous ORDER BY on %r FROM-clause subquery\n",i+1));
-
      sqlite3ExprListDelete(db, pSub->pOrderBy);
+
      sqlite3ParserAddCleanup(pParse,
+
         (void(*)(sqlite3*,void*))sqlite3ExprListDelete,
+
         pSub->pOrderBy);
      pSub->pOrderBy = 0;
    }

@@ -140814,7 +144142,7 @@ SQLITE_PRIVATE int sqlite3Select(
     && i==0
     && (p->selFlags & SF_ComplexResult)!=0
     && (pTabList->nSrc==1
-
         || (pTabList->a[1].fg.jointype&(JT_LEFT|JT_CROSS))!=0)
+
         || (pTabList->a[1].fg.jointype&(JT_OUTER|JT_CROSS))!=0)
    ){
      continue;
    }
@@ -140838,9 +144166,9 @@ SQLITE_PRIVATE int sqlite3Select(
  */
  if( p->pPrior ){
    rc = multiSelect(pParse, p, pDest);
-
#if SELECTTRACE_ENABLED
+
#if TREETRACE_ENABLED
    SELECTTRACE(0x1,pParse,p,("end compound-select processing\n"));
-
    if( (sqlite3SelectTrace & 0x2000)!=0 && ExplainQueryPlanParent(pParse)==0 ){
+
    if( (sqlite3TreeTrace & 0x2000)!=0 && ExplainQueryPlanParent(pParse)==0 ){
      sqlite3TreeViewSelect(0, p, 0);
    }
#endif
@@ -140859,8 +144187,8 @@ SQLITE_PRIVATE int sqlite3Select(
   && OptimizationEnabled(db, SQLITE_PropagateConst)
   && propagateConstants(pParse, p)
  ){
-
#if SELECTTRACE_ENABLED
-
    if( sqlite3SelectTrace & 0x100 ){
+
#if TREETRACE_ENABLED
+
    if( sqlite3TreeTrace & 0x100 ){
      SELECTTRACE(0x100,pParse,p,("After constant propagation:\n"));
      sqlite3TreeViewSelect(0, p, 0);
    }
@@ -140938,8 +144266,8 @@ SQLITE_PRIVATE int sqlite3Select(
         || (pItem->u2.pCteUse->eM10d!=M10d_Yes && pItem->u2.pCteUse->nUse<2))
     && pushDownWhereTerms(pParse, pSub, p->pWhere, pItem)
    ){
-
#if SELECTTRACE_ENABLED
-
      if( sqlite3SelectTrace & 0x100 ){
+
#if TREETRACE_ENABLED
+
      if( sqlite3TreeTrace & 0x100 ){
        SELECTTRACE(0x100,pParse,p,
            ("After WHERE-clause push-down into subquery %d:\n", pSub->selId));
        sqlite3TreeViewSelect(0, p, 0);
@@ -140955,18 +144283,19 @@ SQLITE_PRIVATE int sqlite3Select(

    /* Generate code to implement the subquery
    **
-
    ** The subquery is implemented as a co-routine if:
+
    ** The subquery is implemented as a co-routine if all of the following are
+
    ** true:
+
    **
    **    (1)  the subquery is guaranteed to be the outer loop (so that
    **         it does not need to be computed more than once), and
    **    (2)  the subquery is not a CTE that should be materialized
-
    **
-
    ** TODO: Are there other reasons beside (1) and (2) to use a co-routine
-
    ** implementation?
+
    **    (3)  the subquery is not part of a left operand for a RIGHT JOIN
    */
    if( i==0
     && (pTabList->nSrc==1
-
            || (pTabList->a[1].fg.jointype&(JT_LEFT|JT_CROSS))!=0)  /* (1) */
-
     && (pItem->fg.isCte==0 || pItem->u2.pCteUse->eM10d!=M10d_Yes)  /* (2) */
+
            || (pTabList->a[1].fg.jointype&(JT_OUTER|JT_CROSS))!=0)  /* (1) */
+
     && (pItem->fg.isCte==0 || pItem->u2.pCteUse->eM10d!=M10d_Yes)   /* (2) */
+
     && (pTabList->a[0].fg.jointype & JT_LTORJ)==0                   /* (3) */
    ){
      /* Implement a co-routine that will return a single row of the result
      ** set on each invocation.
@@ -141012,11 +144341,11 @@ SQLITE_PRIVATE int sqlite3Select(
      ** the same view can reuse the materialization. */
      int topAddr;
      int onceAddr = 0;
-
      int retAddr;

      pItem->regReturn = ++pParse->nMem;
-
      topAddr = sqlite3VdbeAddOp2(v, OP_Integer, 0, pItem->regReturn);
+
      topAddr = sqlite3VdbeAddOp0(v, OP_Goto);
      pItem->addrFillSub = topAddr+1;
+
      pItem->fg.isMaterialized = 1;
      if( pItem->fg.isCorrelated==0 ){
        /* If the subquery is not correlated and if we are not inside of
        ** a trigger, then we only need to compute the value of the subquery
@@ -141028,12 +144357,15 @@ SQLITE_PRIVATE int sqlite3Select(
      }
      sqlite3SelectDestInit(&dest, SRT_EphemTab, pItem->iCursor);
      ExplainQueryPlan((pParse, 1, "MATERIALIZE %!S", pItem));
+
      dest.zAffSdst = sqlite3TableAffinityStr(db, pItem->pTab);
      sqlite3Select(pParse, pSub, &dest);
+
      sqlite3DbFree(db, dest.zAffSdst);
+
      dest.zAffSdst = 0;
      pItem->pTab->nRowLogEst = pSub->nSelectRow;
      if( onceAddr ) sqlite3VdbeJumpHere(v, onceAddr);
-
      retAddr = sqlite3VdbeAddOp1(v, OP_Return, pItem->regReturn);
+
      sqlite3VdbeAddOp2(v, OP_Return, pItem->regReturn, topAddr+1);
      VdbeComment((v, "end %!S", pItem));
-
      sqlite3VdbeChangeP1(v, topAddr, retAddr);
+
      sqlite3VdbeJumpHere(v, topAddr);
      sqlite3ClearTempRegCache(pParse);
      if( pItem->fg.isCte && pItem->fg.isCorrelated==0 ){
        CteUse *pCteUse = pItem->u2.pCteUse;
@@ -141057,8 +144389,8 @@ SQLITE_PRIVATE int sqlite3Select(
  pHaving = p->pHaving;
  sDistinct.isTnct = (p->selFlags & SF_Distinct)!=0;

-
#if SELECTTRACE_ENABLED
-
  if( sqlite3SelectTrace & 0x400 ){
+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x400 ){
    SELECTTRACE(0x400,pParse,p,("After all FROM-clause analysis:\n"));
    sqlite3TreeViewSelect(0, p, 0);
  }
@@ -141092,9 +144424,10 @@ SQLITE_PRIVATE int sqlite3Select(
    ** the sDistinct.isTnct is still set.  Hence, isTnct represents the
    ** original setting of the SF_Distinct flag, not the current setting */
    assert( sDistinct.isTnct );
+
    sDistinct.isTnct = 2;

-
#if SELECTTRACE_ENABLED
-
    if( sqlite3SelectTrace & 0x400 ){
+
#if TREETRACE_ENABLED
+
    if( sqlite3TreeTrace & 0x400 ){
      SELECTTRACE(0x400,pParse,p,("Transform DISTINCT into GROUP BY:\n"));
      sqlite3TreeViewSelect(0, p, 0);
    }
@@ -141127,6 +144460,18 @@ SQLITE_PRIVATE int sqlite3Select(
  */
  if( pDest->eDest==SRT_EphemTab ){
    sqlite3VdbeAddOp2(v, OP_OpenEphemeral, pDest->iSDParm, pEList->nExpr);
+
    if( p->selFlags & SF_NestedFrom ){
+
      /* Delete or NULL-out result columns that will never be used */
+
      int ii;
+
      for(ii=pEList->nExpr-1; ii>0 && pEList->a[ii].fg.bUsed==0; ii--){
+
        sqlite3ExprDelete(db, pEList->a[ii].pExpr);
+
        sqlite3DbFree(db, pEList->a[ii].zEName);
+
        pEList->nExpr--;
+
      }
+
      for(ii=0; ii<pEList->nExpr; ii++){
+
        if( pEList->a[ii].fg.bUsed==0 ) pEList->a[ii].pExpr->op = TK_NULL;
+
      }
+
    }
  }

  /* Set the limiter.
@@ -141135,7 +144480,7 @@ SQLITE_PRIVATE int sqlite3Select(
  if( (p->selFlags & SF_FixedLimit)==0 ){
    p->nSelectRow = 320;  /* 4 billion rows */
  }
-
  computeLimitRegisters(pParse, p, iEnd);
+
  if( p->pLimit ) computeLimitRegisters(pParse, p, iEnd);
  if( p->iLimit==0 && sSort.addrSortIndex>=0 ){
    sqlite3VdbeChangeOpcode(v, sSort.addrSortIndex, OP_SorterOpen);
    sSort.sortFlags |= SORTFLAG_UseSorter;
@@ -141276,8 +144621,9 @@ SQLITE_PRIVATE int sqlite3Select(
        ** ORDER BY to maximize the chances of rows being delivered in an
        ** order that makes the ORDER BY redundant.  */
        for(ii=0; ii<pGroupBy->nExpr; ii++){
-
          u8 sortFlags = sSort.pOrderBy->a[ii].sortFlags & KEYINFO_ORDER_DESC;
-
          pGroupBy->a[ii].sortFlags = sortFlags;
+
          u8 sortFlags;
+
          sortFlags = sSort.pOrderBy->a[ii].fg.sortFlags & KEYINFO_ORDER_DESC;
+
          pGroupBy->a[ii].fg.sortFlags = sortFlags;
        }
        if( sqlite3ExprListCompare(pGroupBy, sSort.pOrderBy, -1)==0 ){
          orderByGrp = 1;
@@ -141346,8 +144692,8 @@ SQLITE_PRIVATE int sqlite3Select(
    }
    pAggInfo->mxReg = pParse->nMem;
    if( db->mallocFailed ) goto select_end;
-
#if SELECTTRACE_ENABLED
-
    if( sqlite3SelectTrace & 0x400 ){
+
#if TREETRACE_ENABLED
+
    if( sqlite3TreeTrace & 0x400 ){
      int ii;
      SELECTTRACE(0x400,pParse,p,("After aggregate analysis %p:\n", pAggInfo));
      sqlite3TreeViewSelect(0, p, 0);
@@ -141356,8 +144702,13 @@ SQLITE_PRIVATE int sqlite3Select(
        sqlite3TreeViewExprList(0, pMinMaxOrderBy, 0, "ORDERBY");
      }
      for(ii=0; ii<pAggInfo->nColumn; ii++){
-
        sqlite3DebugPrintf("agg-column[%d] iMem=%d\n",
-
            ii, pAggInfo->aCol[ii].iMem);
+
        struct AggInfo_col *pCol = &pAggInfo->aCol[ii];
+
        sqlite3DebugPrintf(
+
           "agg-column[%d] pTab=%s iTable=%d iColumn=%d iMem=%d"
+
           " iSorterColumn=%d\n",
+
           ii, pCol->pTab ? pCol->pTab->zName : "NULL",
+
           pCol->iTable, pCol->iColumn, pCol->iMem,
+
           pCol->iSorterColumn);
        sqlite3TreeViewExpr(0, pAggInfo->aCol[ii].pCExpr, 0);
      }
      for(ii=0; ii<pAggInfo->nFunc; ii++){
@@ -141435,7 +144786,8 @@ SQLITE_PRIVATE int sqlite3Select(
      sqlite3VdbeAddOp2(v, OP_Gosub, regReset, addrReset);
      SELECTTRACE(1,pParse,p,("WhereBegin\n"));
      pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, pGroupBy, pDistinct,
-
          0, (WHERE_GROUPBY|(orderByGrp ? WHERE_SORTBYGROUP : 0)|distFlag), 0
+
          p, (sDistinct.isTnct==2 ? WHERE_DISTINCTBY : WHERE_GROUPBY)
+
          |  (orderByGrp ? WHERE_SORTBYGROUP : 0) | distFlag, 0
      );
      if( pWInfo==0 ){
        sqlite3ExprListDelete(db, pDistinct);
@@ -141477,15 +144829,15 @@ SQLITE_PRIVATE int sqlite3Select(
        regBase = sqlite3GetTempRange(pParse, nCol);
        sqlite3ExprCodeExprList(pParse, pGroupBy, regBase, 0, 0);
        j = nGroupBy;
+
        pAggInfo->directMode = 1;
        for(i=0; i<pAggInfo->nColumn; i++){
          struct AggInfo_col *pCol = &pAggInfo->aCol[i];
          if( pCol->iSorterColumn>=j ){
-
            int r1 = j + regBase;
-
            sqlite3ExprCodeGetColumnOfTable(v,
-
                               pCol->pTab, pCol->iTable, pCol->iColumn, r1);
+
            sqlite3ExprCode(pParse, pCol->pCExpr, j + regBase);
            j++;
          }
        }
+
        pAggInfo->directMode = 0;
        regRecord = sqlite3GetTempReg(pParse);
        sqlite3VdbeAddOp3(v, OP_MakeRecord, regBase, nCol, regRecord);
        sqlite3VdbeAddOp2(v, OP_SorterInsert, pAggInfo->sortingIdx, regRecord);
@@ -141617,7 +144969,7 @@ SQLITE_PRIVATE int sqlite3Select(
      VdbeComment((v, "indicate accumulator empty"));
      sqlite3VdbeAddOp1(v, OP_Return, regReset);

-
      if( eDist!=WHERE_DISTINCT_NOOP ){
+
      if( distFlag!=0 && eDist!=WHERE_DISTINCT_NOOP ){
        struct AggInfo_func *pF = &pAggInfo->aFunc[0];
        fixDistinctOpenEph(pParse, eDist, pF->iDistinct, pF->iDistAddr);
      }
@@ -141733,7 +145085,7 @@ SQLITE_PRIVATE int sqlite3Select(

        SELECTTRACE(1,pParse,p,("WhereBegin\n"));
        pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, pMinMaxOrderBy,
-
                                   pDistinct, 0, minMaxFlag|distFlag, 0);
+
                                   pDistinct, p, minMaxFlag|distFlag, 0);
        if( pWInfo==0 ){
          goto select_end;
        }
@@ -141741,8 +145093,10 @@ SQLITE_PRIVATE int sqlite3Select(
        eDist = sqlite3WhereIsDistinct(pWInfo);
        updateAccumulator(pParse, regAcc, pAggInfo, eDist);
        if( eDist!=WHERE_DISTINCT_NOOP ){
-
          struct AggInfo_func *pF = &pAggInfo->aFunc[0];
-
          fixDistinctOpenEph(pParse, eDist, pF->iDistinct, pF->iDistAddr);
+
          struct AggInfo_func *pF = pAggInfo->aFunc;
+
          if( pF ){
+
            fixDistinctOpenEph(pParse, eDist, pF->iDistinct, pF->iDistAddr);
+
          }
        }

        if( regAcc ) sqlite3VdbeAddOp2(v, OP_Integer, 1, regAcc);
@@ -141809,9 +145163,9 @@ select_end:
  }
#endif

-
#if SELECTTRACE_ENABLED
+
#if TREETRACE_ENABLED
  SELECTTRACE(0x1,pParse,p,("end processing\n"));
-
  if( (sqlite3SelectTrace & 0x2000)!=0 && ExplainQueryPlanParent(pParse)==0 ){
+
  if( (sqlite3TreeTrace & 0x2000)!=0 && ExplainQueryPlanParent(pParse)==0 ){
    sqlite3TreeViewSelect(0, p, 0);
  }
#endif
@@ -142076,9 +145430,7 @@ SQLITE_PRIVATE Trigger *sqlite3TriggerList(Parse *pParse, Table *pTab){
  Trigger *pList;           /* List of triggers to return */
  HashElem *p;              /* Loop variable for TEMP triggers */

-
  if( pParse->disableTriggers ){
-
    return 0;
-
  }
+
  assert( pParse->disableTriggers==0 );
  pTmpSchema = pParse->db->aDb[1].pSchema;
  p = sqliteHashFirst(&pTmpSchema->trigHash);
  pList = pTab->pTrigger;
@@ -142091,11 +145443,10 @@ SQLITE_PRIVATE Trigger *sqlite3TriggerList(Parse *pParse, Table *pTab){
    ){
      pTrig->pNext = pList;
      pList = pTrig;
-
    }else if( pTrig->op==TK_RETURNING
+
    }else if( pTrig->op==TK_RETURNING ){
#ifndef SQLITE_OMIT_VIRTUALTABLE
-
              && pParse->db->pVtabCtx==0
+
      assert( pParse->db->pVtabCtx==0 );
#endif
-
    ){
      assert( pParse->bReturning );
      assert( &(pParse->u1.pReturning->retTrig) == pTrig );
      pTrig->table = pTab->zName;
@@ -142378,6 +145729,23 @@ SQLITE_PRIVATE void sqlite3FinishTrigger(
    Vdbe *v;
    char *z;

+
    /* If this is a new CREATE TABLE statement, and if shadow tables
+
    ** are read-only, and the trigger makes a change to a shadow table,
+
    ** then raise an error - do not allow the trigger to be created. */
+
    if( sqlite3ReadOnlyShadowTables(db) ){
+
      TriggerStep *pStep;
+
      for(pStep=pTrig->step_list; pStep; pStep=pStep->pNext){
+
        if( pStep->zTarget!=0
+
         && sqlite3ShadowTableName(db, pStep->zTarget)
+
        ){
+
          sqlite3ErrorMsg(pParse,
+
            "trigger \"%s\" may not write to shadow table \"%s\"",
+
            pTrig->zName, pStep->zTarget);
+
          goto triggerfinish_cleanup;
+
        }
+
      }
+
    }
+

    /* Make an entry in the sqlite_schema table */
    v = sqlite3GetVdbe(pParse);
    if( v==0 ) goto triggerfinish_cleanup;
@@ -142541,7 +145909,7 @@ SQLITE_PRIVATE TriggerStep *sqlite3TriggerInsertStep(
SQLITE_PRIVATE TriggerStep *sqlite3TriggerUpdateStep(
  Parse *pParse,          /* Parser */
  Token *pTableName,   /* Name of the table to be updated */
-
  SrcList *pFrom,
+
  SrcList *pFrom,      /* FROM clause for an UPDATE-FROM, or NULL */
  ExprList *pEList,    /* The SET clause: list of column and new values */
  Expr *pWhere,        /* The WHERE clause */
  u8 orconf,           /* The conflict algorithm. (OE_Abort, OE_Ignore, etc) */
@@ -142755,12 +146123,21 @@ static int checkColumnOverlap(IdList *pIdList, ExprList *pEList){
}

/*
+
** Return true if any TEMP triggers exist
+
*/
+
static int tempTriggersExist(sqlite3 *db){
+
  if( NEVER(db->aDb[1].pSchema==0) ) return 0;
+
  if( sqliteHashFirst(&db->aDb[1].pSchema->trigHash)==0 ) return 0;
+
  return 1;
+
}
+

+
/*
** Return a list of all triggers on table pTab if there exists at least
** one trigger that must be fired when an operation of type 'op' is
** performed on the table, and, if that operation is an UPDATE, if at
** least one of the columns in pChanges is being modified.
*/
-
SQLITE_PRIVATE Trigger *sqlite3TriggersExist(
+
static SQLITE_NOINLINE Trigger *triggersReallyExist(
  Parse *pParse,          /* Parse context */
  Table *pTab,            /* The table the contains the triggers */
  int op,                 /* one of TK_DELETE, TK_INSERT, TK_UPDATE */
@@ -142823,6 +146200,22 @@ exit_triggers_exist:
  }
  return (mask ? pList : 0);
}
+
SQLITE_PRIVATE Trigger *sqlite3TriggersExist(
+
  Parse *pParse,          /* Parse context */
+
  Table *pTab,            /* The table the contains the triggers */
+
  int op,                 /* one of TK_DELETE, TK_INSERT, TK_UPDATE */
+
  ExprList *pChanges,     /* Columns that change in an UPDATE statement */
+
  int *pMask              /* OUT: Mask of TRIGGER_BEFORE|TRIGGER_AFTER */
+
){
+
  assert( pTab!=0 );
+
  if( (pTab->pTrigger==0 && !tempTriggersExist(pParse->db))
+
   || pParse->disableTriggers
+
  ){
+
    if( pMask ) *pMask = 0;
+
    return 0;
+
  }
+
  return triggersReallyExist(pParse,pTab,op,pChanges,pMask);
+
}

/*
** Convert the pStep->zTarget string into a SrcList and return a pointer
@@ -142852,6 +146245,14 @@ SQLITE_PRIVATE SrcList *sqlite3TriggerStepSrc(
    }
    if( pStep->pFrom ){
      SrcList *pDup = sqlite3SrcListDup(db, pStep->pFrom, 0);
+
      if( pDup && pDup->nSrc>1 && !IN_RENAME_OBJECT ){
+
        Select *pSubquery;
+
        Token as;
+
        pSubquery = sqlite3SelectNew(pParse,0,pDup,0,0,0,0,SF_NestedFrom,0);
+
        as.n = 0;
+
        as.z = 0;
+
        pDup = sqlite3SrcListAppendFromTerm(pParse,0,0,0,&as,pSubquery,0);
+
      }
      pSrc = sqlite3SrcListAppendList(pParse, pSrc, pDup);
    }
  }else{
@@ -142907,7 +146308,7 @@ static ExprList *sqlite3ExpandReturning(
        if( !db->mallocFailed ){
          struct ExprList_item *pItem = &pNew->a[pNew->nExpr-1];
          pItem->zEName = sqlite3DbStrDup(db, pTab->aCol[jj].zCnName);
-
          pItem->eEName = ENAME_NAME;
+
          pItem->fg.eEName = ENAME_NAME;
        }
      }
    }else{
@@ -142916,7 +146317,7 @@ static ExprList *sqlite3ExpandReturning(
      if( !db->mallocFailed && ALWAYS(pList->a[i].zEName!=0) ){
        struct ExprList_item *pItem = &pNew->a[pNew->nExpr-1];
        pItem->zEName = sqlite3DbStrDup(db, pList->a[i].zEName);
-
        pItem->eEName = pList->a[i].eEName;
+
        pItem->fg.eEName = pList->a[i].fg.eEName;
      }
    }
  }
@@ -143168,7 +146569,7 @@ static TriggerPrg *codeRowTrigger(
  sSubParse.zAuthContext = pTrigger->zName;
  sSubParse.eTriggerOp = pTrigger->op;
  sSubParse.nQueryLoop = pParse->nQueryLoop;
-
  sSubParse.disableVtab = pParse->disableVtab;
+
  sSubParse.prepFlags = pParse->prepFlags;

  v = sqlite3GetVdbe(&sSubParse);
  if( v ){
@@ -143514,11 +146915,14 @@ static void updateVirtualTable(
** it has been converted into REAL.
*/
SQLITE_PRIVATE void sqlite3ColumnDefault(Vdbe *v, Table *pTab, int i, int iReg){
+
  Column *pCol;
  assert( pTab!=0 );
-
  if( !IsView(pTab) ){
+
  assert( pTab->nCol>i );
+
  pCol = &pTab->aCol[i];
+
  if( pCol->iDflt ){
    sqlite3_value *pValue = 0;
    u8 enc = ENC(sqlite3VdbeDb(v));
-
    Column *pCol = &pTab->aCol[i];
+
    assert( !IsView(pTab) );
    VdbeComment((v, "%s.%s", pTab->zName, pCol->zCnName));
    assert( i<pTab->nCol );
    sqlite3ValueFromExpr(sqlite3VdbeDb(v),
@@ -143529,7 +146933,7 @@ SQLITE_PRIVATE void sqlite3ColumnDefault(Vdbe *v, Table *pTab, int i, int iReg){
    }
  }
#ifndef SQLITE_OMIT_FLOATING_POINT
-
  if( pTab->aCol[i].affinity==SQLITE_AFF_REAL && !IsVirtual(pTab) ){
+
  if( pCol->affinity==SQLITE_AFF_REAL && !IsVirtual(pTab) ){
    sqlite3VdbeAddOp1(v, OP_RealAffinity, iReg);
  }
#endif
@@ -143831,6 +147235,14 @@ SQLITE_PRIVATE void sqlite3Update(
# define isView 0
#endif

+
#if TREETRACE_ENABLED
+
  if( sqlite3TreeTrace & 0x10000 ){
+
    sqlite3TreeViewLine(0, "In sqlite3Update() at %s:%d", __FILE__, __LINE__);
+
    sqlite3TreeViewUpdate(pParse->pWith, pTabList, pChanges, pWhere,
+
                          onError, pOrderBy, pLimit, pUpsert, pTrigger);
+
  }
+
#endif
+

  /* If there was a FROM clause, set nChangeFrom to the number of expressions
  ** in the change-list. Otherwise, set it to 0. There cannot be a FROM
  ** clause if this function is being called to generate code for part of
@@ -144475,7 +147887,7 @@ SQLITE_PRIVATE void sqlite3Update(
      }else{
        sqlite3VdbeAddOp3(v, OP_NotExists, iDataCur, labelContinue,regOldRowid);
      }
-
      VdbeCoverageNeverTaken(v);
+
      VdbeCoverage(v);
    }

    /* Do FK constraint checks. */
@@ -144961,6 +148373,7 @@ SQLITE_PRIVATE int sqlite3UpsertAnalyzeTarget(
        if( pIdx->aiColumn[ii]==XN_EXPR ){
          assert( pIdx->aColExpr!=0 );
          assert( pIdx->aColExpr->nExpr>ii );
+
          assert( pIdx->bHasExpr );
          pExpr = pIdx->aColExpr->a[ii].pExpr;
          if( pExpr->op!=TK_COLLATE ){
            sCol[0].pLeft = pExpr;
@@ -145274,6 +148687,7 @@ SQLITE_PRIVATE SQLITE_NOINLINE int sqlite3RunVacuum(
  int nDb;                /* Number of attached databases */
  const char *zDbMain;    /* Schema name of database to vacuum */
  const char *zOut;       /* Name of output file */
+
  u32 pgflags = PAGER_SYNCHRONOUS_OFF; /* sync flags for output db */

  if( !db->autoCommit ){
    sqlite3SetString(pzErrMsg, db, "cannot VACUUM from within a transaction");
@@ -145345,12 +148759,17 @@ SQLITE_PRIVATE SQLITE_NOINLINE int sqlite3RunVacuum(
      goto end_of_vacuum;
    }
    db->mDbFlags |= DBFLAG_VacuumInto;
+

+
    /* For a VACUUM INTO, the pager-flags are set to the same values as
+
    ** they are for the database being vacuumed, except that PAGER_CACHESPILL
+
    ** is always set. */
+
    pgflags = db->aDb[iDb].safety_level | (db->flags & PAGER_FLAGS_MASK);
  }
  nRes = sqlite3BtreeGetRequestedReserve(pMain);

  sqlite3BtreeSetCacheSize(pTemp, db->aDb[iDb].pSchema->cache_size);
  sqlite3BtreeSetSpillSize(pTemp, sqlite3BtreeSetSpillSize(pMain,0));
-
  sqlite3BtreeSetPagerFlags(pTemp, PAGER_SYNCHRONOUS_OFF|PAGER_CACHESPILL);
+
  sqlite3BtreeSetPagerFlags(pTemp, pgflags|PAGER_CACHESPILL);

  /* Begin a transaction and take an exclusive lock on the main database
  ** file. This is done before the sqlite3BtreeGetPageSize(pMain) call below,
@@ -145481,6 +148900,7 @@ SQLITE_PRIVATE SQLITE_NOINLINE int sqlite3RunVacuum(

  assert( rc==SQLITE_OK );
  if( pOut==0 ){
+
    nRes = sqlite3BtreeGetRequestedReserve(pTemp);
    rc = sqlite3BtreeSetPageSize(pMain, sqlite3BtreeGetPageSize(pTemp), nRes,1);
  }

@@ -145862,7 +149282,8 @@ SQLITE_PRIVATE void sqlite3VtabUnlockList(sqlite3 *db){
*/
SQLITE_PRIVATE void sqlite3VtabClear(sqlite3 *db, Table *p){
  assert( IsVirtual(p) );
-
  if( !db || db->pnBytesFreed==0 ) vtabDisconnectAll(0, p);
+
  assert( db!=0 );
+
  if( db->pnBytesFreed==0 ) vtabDisconnectAll(0, p);
  if( p->u.vtab.azArg ){
    int i;
    for(i=0; i<p->u.vtab.nArg; i++){
@@ -146662,7 +150083,7 @@ SQLITE_PRIVATE FuncDef *sqlite3VtabOverloadFunction(
  if( pExpr->op!=TK_COLUMN ) return pDef;
  assert( ExprUseYTab(pExpr) );
  pTab = pExpr->y.pTab;
-
  if( pTab==0 ) return pDef;
+
  if( NEVER(pTab==0) ) return pDef;
  if( !IsVirtual(pTab) ) return pDef;
  pVtab = sqlite3GetVTable(db, pTab)->pVtab;
  assert( pVtab!=0 );
@@ -146923,6 +150344,28 @@ typedef struct WhereLoopBuilder WhereLoopBuilder;
typedef struct WhereScan WhereScan;
typedef struct WhereOrCost WhereOrCost;
typedef struct WhereOrSet WhereOrSet;
+
typedef struct WhereMemBlock WhereMemBlock;
+
typedef struct WhereRightJoin WhereRightJoin;
+

+
/*
+
** This object is a header on a block of allocated memory that will be
+
** automatically freed when its WInfo oject is destructed.
+
*/
+
struct WhereMemBlock {
+
  WhereMemBlock *pNext;      /* Next block in the chain */
+
  u64 sz;                    /* Bytes of space */
+
};
+

+
/*
+
** Extra information attached to a WhereLevel that is a RIGHT JOIN.
+
*/
+
struct WhereRightJoin {
+
  int iMatch;          /* Cursor used to determine prior matched rows */
+
  int regBloom;        /* Bloom filter for iRJMatch */
+
  int regReturn;       /* Return register for the interior subroutine */
+
  int addrSubrtn;      /* Starting address for the interior subroutine */
+
  int endSubrtn;       /* The last opcode in the interior subroutine */
+
};

/*
** This object contains information needed to implement a single nested
@@ -146956,6 +150399,7 @@ struct WhereLevel {
  int addrLikeRep;      /* LIKE range processing address */
#endif
  int regFilter;        /* Bloom filter */
+
  WhereRightJoin *pRJ;  /* Extra information for RIGHT JOIN */
  u8 iFrom;             /* Which entry in the FROM clause */
  u8 op, p3, p5;        /* Opcode, P3 & P5 of the opcode that ends the loop */
  int p1, p2;           /* Operands of the opcode used to end the loop */
@@ -147246,7 +150690,7 @@ struct WhereAndInfo {
** between VDBE cursor numbers and bits of the bitmasks in WhereTerm.
**
** The VDBE cursor numbers are small integers contained in
-
** SrcList_item.iCursor and Expr.iTable fields.  For any given WHERE
+
** SrcItem.iCursor and Expr.iTable fields.  For any given WHERE
** clause, the cursor numbers might not begin with 0 and they might
** contain gaps in the numbering sequence.  But we want to make maximum
** use of the bits in our bitmasks.  This structure provides a mapping
@@ -147318,20 +150762,6 @@ struct WhereLoopBuilder {
#endif

/*
-
** Each instance of this object records a change to a single node
-
** in an expression tree to cause that node to point to a column
-
** of an index rather than an expression or a virtual column.  All
-
** such transformations need to be undone at the end of WHERE clause
-
** processing.
-
*/
-
typedef struct WhereExprMod WhereExprMod;
-
struct WhereExprMod {
-
  WhereExprMod *pNext;  /* Next translation on a list of them all */
-
  Expr *pExpr;          /* The Expr node that was transformed */
-
  Expr orig;            /* Original value of the Expr node */
-
};
-

-
/*
** The WHERE clause processing routine has two halves.  The
** first part does the start of the WHERE loop and the second
** half does the tail of the WHERE loop.  An instance of
@@ -147346,10 +150776,10 @@ struct WhereInfo {
  SrcList *pTabList;        /* List of tables in the join */
  ExprList *pOrderBy;       /* The ORDER BY clause or NULL */
  ExprList *pResultSet;     /* Result set of the query */
+
#if WHERETRACE_ENABLED
  Expr *pWhere;             /* The complete WHERE clause */
-
#ifndef SQLITE_OMIT_VIRTUALTABLE
-
  Select *pLimit;           /* Used to access LIMIT expr/registers for vtabs */
#endif
+
  Select *pSelect;          /* The entire SELECT statement containing WHERE */
  int aiCurOnePass[2];      /* OP_OpenWrite cursors for the ONEPASS opt */
  int iContinue;            /* Jump here to continue with next record */
  int iBreak;               /* Jump here to break out of the loop */
@@ -147368,7 +150798,7 @@ struct WhereInfo {
  int iTop;                 /* The very beginning of the WHERE loop */
  int iEndWhere;            /* End of the WHERE clause itself */
  WhereLoop *pLoops;        /* List of all WhereLoop objects */
-
  WhereExprMod *pExprMods;  /* Expression modifications */
+
  WhereMemBlock *pMemToFree;/* Memory to free when this object destroyed */
  Bitmask revMask;          /* Mask of ORDER BY terms that need reversing */
  WhereClause sWC;          /* Decomposition of the WHERE clause */
  WhereMaskSet sMaskSet;    /* Map cursor numbers to bitmasks */
@@ -147394,6 +150824,8 @@ SQLITE_PRIVATE WhereTerm *sqlite3WhereFindTerm(
  u32 op,               /* Mask of WO_xx values describing operator */
  Index *pIdx           /* Must be compatible with this index, if not NULL */
);
+
SQLITE_PRIVATE void *sqlite3WhereMalloc(WhereInfo *pWInfo, u64 nByte);
+
SQLITE_PRIVATE void *sqlite3WhereRealloc(WhereInfo *pWInfo, void *pOld, u64 nByte);

/* wherecode.c: */
#ifndef SQLITE_OMIT_EXPLAIN
@@ -147430,6 +150862,11 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
  WhereLevel *pLevel,  /* The current level pointer */
  Bitmask notReady     /* Which tables are currently available */
);
+
SQLITE_PRIVATE SQLITE_NOINLINE void sqlite3WhereRightJoinLoop(
+
  WhereInfo *pWInfo,
+
  int iLevel,
+
  WhereLevel *pLevel
+
);

/* whereexpr.c: */
SQLITE_PRIVATE void sqlite3WhereClauseInit(WhereClause*,WhereInfo*);
@@ -147472,8 +150909,9 @@ SQLITE_PRIVATE void sqlite3WhereTabFuncArgs(Parse*, SrcItem*, WhereClause*);
#define WO_AND    0x0400       /* Two or more AND-connected terms */
#define WO_EQUIV  0x0800       /* Of the form A==B, both columns */
#define WO_NOOP   0x1000       /* This term does not restrict search space */
+
#define WO_ROWVAL 0x2000       /* A row-value term */

-
#define WO_ALL    0x1fff       /* Mask of all possible WO_* values */
+
#define WO_ALL    0x3fff       /* Mask of all possible WO_* values */
#define WO_SINGLE 0x01ff       /* Mask of all non-compound WO_* values */

/*
@@ -147507,6 +150945,7 @@ SQLITE_PRIVATE void sqlite3WhereTabFuncArgs(Parse*, SrcItem*, WhereClause*);
#define WHERE_BLOOMFILTER  0x00400000  /* Consider using a Bloom-filter */
#define WHERE_SELFCULL     0x00800000  /* nOut reduced by extra WHERE terms */
#define WHERE_OMIT_OFFSET  0x01000000  /* Set offset counter to zero */
+
#define WHERE_VIEWSCAN     0x02000000  /* A full-scan of a VIEW or subquery */

#endif /* !defined(SQLITE_WHEREINT_H) */

@@ -147697,6 +151136,9 @@ SQLITE_PRIVATE int sqlite3WhereExplainOneScan(
                  pLoop->u.vtab.idxNum, pLoop->u.vtab.idxStr);
    }
#endif
+
    if( pItem->fg.jointype & JT_LEFT ){
+
      sqlite3_str_appendf(&str, " LEFT-JOIN");
+
    }
#ifdef SQLITE_EXPLAIN_ESTIMATED_ROWS
    if( pLoop->nOut>=10 ){
      sqlite3_str_appendf(&str, " (~%llu rows)",
@@ -147840,7 +151282,7 @@ static void disableTerm(WhereLevel *pLevel, WhereTerm *pTerm){
  int nLoop = 0;
  assert( pTerm!=0 );
  while( (pTerm->wtFlags & TERM_CODED)==0
-
      && (pLevel->iLeftJoin==0 || ExprHasProperty(pTerm->pExpr, EP_FromJoin))
+
      && (pLevel->iLeftJoin==0 || ExprHasProperty(pTerm->pExpr, EP_OuterON))
      && (pLevel->notReady & pTerm->prereqAll)==0
  ){
    if( nLoop && (pTerm->wtFlags & TERM_LIKE)!=0 ){
@@ -148101,16 +151543,22 @@ static int codeEqualityTerm(
    if( !ExprUseXSelect(pX) || pX->x.pSelect->pEList->nExpr==1 ){
      eType = sqlite3FindInIndex(pParse, pX, IN_INDEX_LOOP, 0, 0, &iTab);
    }else{
-
      sqlite3 *db = pParse->db;
-
      pX = removeUnindexableInClauseTerms(pParse, iEq, pLoop, pX);
-

-
      if( !db->mallocFailed ){
-
        aiMap = (int*)sqlite3DbMallocZero(pParse->db, sizeof(int)*nEq);
+
      Expr *pExpr = pTerm->pExpr;
+
      if( pExpr->iTable==0 || !ExprHasProperty(pExpr, EP_Subrtn) ){
+
        sqlite3 *db = pParse->db;
+
        pX = removeUnindexableInClauseTerms(pParse, iEq, pLoop, pX);
+
        if( !db->mallocFailed ){
+
          aiMap = (int*)sqlite3DbMallocZero(pParse->db, sizeof(int)*nEq);
+
          eType = sqlite3FindInIndex(pParse, pX, IN_INDEX_LOOP, 0, aiMap,&iTab);
+
          pExpr->iTable = iTab;
+
        }
+
        sqlite3ExprDelete(db, pX);
+
      }else{
+
        int n = sqlite3ExprVectorSize(pX->pLeft);
+
        aiMap = (int*)sqlite3DbMallocZero(pParse->db, sizeof(int)*MAX(nEq,n));
        eType = sqlite3FindInIndex(pParse, pX, IN_INDEX_LOOP, 0, aiMap, &iTab);
-
        pTerm->pExpr->iTable = iTab;
      }
-
      sqlite3ExprDelete(db, pX);
-
      pX = pTerm->pExpr;
+
      pX = pExpr;
    }

    if( eType==IN_INDEX_INDEX_DESC ){
@@ -148133,8 +151581,9 @@ static int codeEqualityTerm(
    i = pLevel->u.in.nIn;
    pLevel->u.in.nIn += nEq;
    pLevel->u.in.aInLoop =
-
       sqlite3DbReallocOrFree(pParse->db, pLevel->u.in.aInLoop,
-
                              sizeof(pLevel->u.in.aInLoop[0])*pLevel->u.in.nIn);
+
       sqlite3WhereRealloc(pTerm->pWC->pWInfo,
+
                           pLevel->u.in.aInLoop,
+
                           sizeof(pLevel->u.in.aInLoop[0])*pLevel->u.in.nIn);
    pIn = pLevel->u.in.aInLoop;
    if( pIn ){
      int iMap = 0;               /* Index in aiMap[] */
@@ -148376,7 +151825,7 @@ static void whereLikeOptimizationStringFixup(
  if( pTerm->wtFlags & TERM_LIKEOPT ){
    VdbeOp *pOp;
    assert( pLevel->iLikeRepCntr>0 );
-
    pOp = sqlite3VdbeGetOp(v, -1);
+
    pOp = sqlite3VdbeGetLastOp(v);
    assert( pOp!=0 );
    assert( pOp->opcode==OP_String8
            || pTerm->pWC->pWInfo->pParse->db->mallocFailed );
@@ -148555,8 +152004,8 @@ static void codeCursorHint(
    */
    if( pTabItem->fg.jointype & JT_LEFT ){
      Expr *pExpr = pTerm->pExpr;
-
      if( !ExprHasProperty(pExpr, EP_FromJoin)
-
       || pExpr->w.iRightJoinTable!=pTabItem->iCursor
+
      if( !ExprHasProperty(pExpr, EP_OuterON)
+
       || pExpr->w.iJoin!=pTabItem->iCursor
      ){
        sWalker.eCode = 0;
        sWalker.xExprCallback = codeCursorHintIsOrFunction;
@@ -148564,7 +152013,7 @@ static void codeCursorHint(
        if( sWalker.eCode ) continue;
      }
    }else{
-
      if( ExprHasProperty(pTerm->pExpr, EP_FromJoin) ) continue;
+
      if( ExprHasProperty(pTerm->pExpr, EP_OuterON) ) continue;
    }

    /* All terms in pWLoop->aLTerm[] except pEndRange are used to initialize
@@ -148612,13 +152061,21 @@ static void codeCursorHint(
**
**   OP_DeferredSeek $iCur $iRowid
**
+
** Which causes a seek on $iCur to the row with rowid $iRowid.
+
**
** However, if the scan currently being coded is a branch of an OR-loop and
-
** the statement currently being coded is a SELECT, then P3 of OP_DeferredSeek
-
** is set to iIdxCur and P4 is set to point to an array of integers
-
** containing one entry for each column of the table cursor iCur is open
-
** on. For each table column, if the column is the i'th column of the
-
** index, then the corresponding array entry is set to (i+1). If the column
-
** does not appear in the index at all, the array entry is set to 0.
+
** the statement currently being coded is a SELECT, then additional information
+
** is added that might allow OP_Column to omit the seek and instead do its
+
** lookup on the index, thus avoiding an expensive seek operation.  To
+
** enable this optimization, the P3 of OP_DeferredSeek is set to iIdxCur
+
** and P4 is set to an array of integers containing one entry for each column
+
** in the table.  For each table column, if the column is the i'th
+
** column of the index, then the corresponding array entry is set to (i+1).
+
** If the column does not appear in the index at all, the array entry is set
+
** to 0.  The OP_Column opcode can check this array to see if the column it
+
** wants is in the index and if it is, it will substitute the index cursor
+
** and column number and continue with those new values, rather than seeking
+
** the table cursor.
*/
static void codeDeferredSeek(
  WhereInfo *pWInfo,              /* Where clause context */
@@ -148634,7 +152091,7 @@ static void codeDeferredSeek(

  pWInfo->bDeferredSeek = 1;
  sqlite3VdbeAddOp3(v, OP_DeferredSeek, iIdxCur, 0, iCur);
-
  if( (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)
+
  if( (pWInfo->wctrlFlags & (WHERE_OR_SUBCLAUSE|WHERE_RIGHT_JOIN))
   && DbMaskAllZero(sqlite3ParseToplevel(pParse)->writeMask)
  ){
    int i;
@@ -148692,144 +152149,6 @@ static void codeExprOrVector(Parse *pParse, Expr *p, int iReg, int nReg){
  }
}

-
/* An instance of the IdxExprTrans object carries information about a
-
** mapping from an expression on table columns into a column in an index
-
** down through the Walker.
-
*/
-
typedef struct IdxExprTrans {
-
  Expr *pIdxExpr;    /* The index expression */
-
  int iTabCur;       /* The cursor of the corresponding table */
-
  int iIdxCur;       /* The cursor for the index */
-
  int iIdxCol;       /* The column for the index */
-
  int iTabCol;       /* The column for the table */
-
  WhereInfo *pWInfo; /* Complete WHERE clause information */
-
  sqlite3 *db;       /* Database connection (for malloc()) */
-
} IdxExprTrans;
-

-
/*
-
** Preserve pExpr on the WhereETrans list of the WhereInfo.
-
*/
-
static void preserveExpr(IdxExprTrans *pTrans, Expr *pExpr){
-
  WhereExprMod *pNew;
-
  pNew = sqlite3DbMallocRaw(pTrans->db, sizeof(*pNew));
-
  if( pNew==0 ) return;
-
  pNew->pNext = pTrans->pWInfo->pExprMods;
-
  pTrans->pWInfo->pExprMods = pNew;
-
  pNew->pExpr = pExpr;
-
  memcpy(&pNew->orig, pExpr, sizeof(*pExpr));
-
}
-

-
/* The walker node callback used to transform matching expressions into
-
** a reference to an index column for an index on an expression.
-
**
-
** If pExpr matches, then transform it into a reference to the index column
-
** that contains the value of pExpr.
-
*/
-
static int whereIndexExprTransNode(Walker *p, Expr *pExpr){
-
  IdxExprTrans *pX = p->u.pIdxTrans;
-
  if( sqlite3ExprCompare(0, pExpr, pX->pIdxExpr, pX->iTabCur)==0 ){
-
    pExpr = sqlite3ExprSkipCollate(pExpr);
-
    preserveExpr(pX, pExpr);
-
    pExpr->affExpr = sqlite3ExprAffinity(pExpr);
-
    pExpr->op = TK_COLUMN;
-
    pExpr->iTable = pX->iIdxCur;
-
    pExpr->iColumn = pX->iIdxCol;
-
    testcase( ExprHasProperty(pExpr, EP_Skip) );
-
    testcase( ExprHasProperty(pExpr, EP_Unlikely) );
-
    ExprClearProperty(pExpr, EP_Skip|EP_Unlikely|EP_WinFunc|EP_Subrtn);
-
    pExpr->y.pTab = 0;
-
    return WRC_Prune;
-
  }else{
-
    return WRC_Continue;
-
  }
-
}
-

-
#ifndef SQLITE_OMIT_GENERATED_COLUMNS
-
/* A walker node callback that translates a column reference to a table
-
** into a corresponding column reference of an index.
-
*/
-
static int whereIndexExprTransColumn(Walker *p, Expr *pExpr){
-
  if( pExpr->op==TK_COLUMN ){
-
    IdxExprTrans *pX = p->u.pIdxTrans;
-
    if( pExpr->iTable==pX->iTabCur && pExpr->iColumn==pX->iTabCol ){
-
      assert( ExprUseYTab(pExpr) && pExpr->y.pTab!=0 );
-
      preserveExpr(pX, pExpr);
-
      pExpr->affExpr = sqlite3TableColumnAffinity(pExpr->y.pTab,pExpr->iColumn);
-
      pExpr->iTable = pX->iIdxCur;
-
      pExpr->iColumn = pX->iIdxCol;
-
      pExpr->y.pTab = 0;
-
    }
-
  }
-
  return WRC_Continue;
-
}
-
#endif /* SQLITE_OMIT_GENERATED_COLUMNS */
-

-
/*
-
** For an indexes on expression X, locate every instance of expression X
-
** in pExpr and change that subexpression into a reference to the appropriate
-
** column of the index.
-
**
-
** 2019-10-24: Updated to also translate references to a VIRTUAL column in
-
** the table into references to the corresponding (stored) column of the
-
** index.
-
*/
-
static void whereIndexExprTrans(
-
  Index *pIdx,      /* The Index */
-
  int iTabCur,      /* Cursor of the table that is being indexed */
-
  int iIdxCur,      /* Cursor of the index itself */
-
  WhereInfo *pWInfo /* Transform expressions in this WHERE clause */
-
){
-
  int iIdxCol;               /* Column number of the index */
-
  ExprList *aColExpr;        /* Expressions that are indexed */
-
  Table *pTab;
-
  Walker w;
-
  IdxExprTrans x;
-
  aColExpr = pIdx->aColExpr;
-
  if( aColExpr==0 && !pIdx->bHasVCol ){
-
    /* The index does not reference any expressions or virtual columns
-
    ** so no translations are needed. */
-
    return;
-
  }
-
  pTab = pIdx->pTable;
-
  memset(&w, 0, sizeof(w));
-
  w.u.pIdxTrans = &x;
-
  x.iTabCur = iTabCur;
-
  x.iIdxCur = iIdxCur;
-
  x.pWInfo = pWInfo;
-
  x.db = pWInfo->pParse->db;
-
  for(iIdxCol=0; iIdxCol<pIdx->nColumn; iIdxCol++){
-
    i16 iRef = pIdx->aiColumn[iIdxCol];
-
    if( iRef==XN_EXPR ){
-
      assert( aColExpr!=0 && aColExpr->a[iIdxCol].pExpr!=0 );
-
      x.pIdxExpr = aColExpr->a[iIdxCol].pExpr;
-
      if( sqlite3ExprIsConstant(x.pIdxExpr) ) continue;
-
      w.xExprCallback = whereIndexExprTransNode;
-
#ifndef SQLITE_OMIT_GENERATED_COLUMNS
-
    }else if( iRef>=0
-
       && (pTab->aCol[iRef].colFlags & COLFLAG_VIRTUAL)!=0
-
       && ((pTab->aCol[iRef].colFlags & COLFLAG_HASCOLL)==0
-
           || sqlite3StrICmp(sqlite3ColumnColl(&pTab->aCol[iRef]),
-
                                               sqlite3StrBINARY)==0)
-
    ){
-
      /* Check to see if there are direct references to generated columns
-
      ** that are contained in the index.  Pulling the generated column
-
      ** out of the index is an optimization only - the main table is always
-
      ** available if the index cannot be used.  To avoid unnecessary
-
      ** complication, omit this optimization if the collating sequence for
-
      ** the column is non-standard */
-
      x.iTabCol = iRef;
-
      w.xExprCallback = whereIndexExprTransColumn;
-
#endif /* SQLITE_OMIT_GENERATED_COLUMNS */
-
    }else{
-
      continue;
-
    }
-
    x.iIdxCol = iIdxCol;
-
    sqlite3WalkExpr(&w, pWInfo->pWhere);
-
    sqlite3WalkExprList(&w, pWInfo->pOrderBy);
-
    sqlite3WalkExprList(&w, pWInfo->pResultSet);
-
  }
-
}
-

/*
** The pTruth expression is always true because it is the WHERE clause
** a partial index that is driving a query loop.  Look through all of the
@@ -148898,6 +152217,8 @@ static SQLITE_NOINLINE void filterPullDown(
      testcase( pTerm->wtFlags & TERM_VIRTUAL );
      regRowid = sqlite3GetTempReg(pParse);
      regRowid = codeEqualityTerm(pParse, pTerm, pLevel, 0, 0, regRowid);
+
      sqlite3VdbeAddOp2(pParse->pVdbe, OP_MustBeInt, regRowid, addrNxt);
+
      VdbeCoverage(pParse->pVdbe);
      sqlite3VdbeAddOp4Int(pParse->pVdbe, OP_Filter, pLevel->regFilter,
                           addrNxt, regRowid, 1);
      VdbeCoverage(pParse->pVdbe);
@@ -148990,7 +152311,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
  ** initialize a memory cell that records if this table matches any
  ** row of the left table of the join.
  */
-
  assert( (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)
+
  assert( (pWInfo->wctrlFlags & (WHERE_OR_SUBCLAUSE|WHERE_RIGHT_JOIN))
       || pLevel->iFrom>0 || (pTabItem[0].fg.jointype & JT_LEFT)==0
  );
  if( pLevel->iFrom>0 && (pTabItem[0].fg.jointype & JT_LEFT)!=0 ){
@@ -149001,7 +152322,10 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(

  /* Compute a safe address to jump to if we discover that the table for
  ** this loop is empty and can never contribute content. */
-
  for(j=iLevel; j>0 && pWInfo->a[j].iLeftJoin==0; j--){}
+
  for(j=iLevel; j>0; j--){
+
    if( pWInfo->a[j].iLeftJoin ) break;
+
    if( pWInfo->a[j].pRJ ) break;
+
  }
  addrHalt = pWInfo->a[j].addrBrk;

  /* Special case of a FROM clause subquery implemented as a co-routine */
@@ -149046,9 +152370,9 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
         && pLoop->u.vtab.bOmitOffset
        ){
          assert( pTerm->eOperator==WO_AUX );
-
          assert( pWInfo->pLimit!=0 );
-
          assert( pWInfo->pLimit->iOffset>0 );
-
          sqlite3VdbeAddOp2(v, OP_Integer, 0, pWInfo->pLimit->iOffset);
+
          assert( pWInfo->pSelect!=0 );
+
          assert( pWInfo->pSelect->iOffset>0 );
+
          sqlite3VdbeAddOp2(v, OP_Integer, 0, pWInfo->pSelect->iOffset);
          VdbeComment((v,"Zero OFFSET counter"));
        }
      }
@@ -149156,6 +152480,8 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
    if( iRowidReg!=iReleaseReg ) sqlite3ReleaseTempReg(pParse, iReleaseReg);
    addrNxt = pLevel->addrNxt;
    if( pLevel->regFilter ){
+
      sqlite3VdbeAddOp2(v, OP_MustBeInt, iRowidReg, addrNxt);
+
      VdbeCoverage(v);
      sqlite3VdbeAddOp4Int(v, OP_Filter, pLevel->regFilter, addrNxt,
                           iRowidReg, 1);
      VdbeCoverage(v);
@@ -149507,6 +152833,11 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
        ** guess. */
        addrSeekScan = sqlite3VdbeAddOp1(v, OP_SeekScan,
                                         (pIdx->aiRowLogEst[0]+9)/10);
+
        if( pRangeStart ){
+
          sqlite3VdbeChangeP5(v, 1);
+
          sqlite3VdbeChangeP2(v, addrSeekScan, sqlite3VdbeCurrentAddr(v)+1);
+
          addrSeekScan = 0;
+
        }
        VdbeCoverage(v);
      }
      sqlite3VdbeAddOp4Int(v, op, iIdxCur, addrNxt, regBase, nConstraint);
@@ -149582,8 +152913,8 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
      }
      nConstraint++;
    }
-
    sqlite3DbFree(db, zStartAff);
-
    sqlite3DbFree(db, zEndAff);
+
    if( zStartAff ) sqlite3DbNNFreeNN(db, zStartAff);
+
    if( zEndAff ) sqlite3DbNNFreeNN(db, zEndAff);

    /* Top of the loop body */
    if( pLevel->p2==0 ) pLevel->p2 = sqlite3VdbeCurrentAddr(v);
@@ -149628,7 +152959,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(

    /* Seek the table cursor, if required */
    omitTable = (pLoop->wsFlags & WHERE_IDX_ONLY)!=0
-
           && (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)==0;
+
           && (pWInfo->wctrlFlags & (WHERE_OR_SUBCLAUSE|WHERE_RIGHT_JOIN))==0;
    if( omitTable ){
      /* pIdx is a covering index.  No need to access the main table. */
    }else if( HasRowid(pIdx->pTable) ){
@@ -149645,27 +152976,6 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
    }

    if( pLevel->iLeftJoin==0 ){
-
      /* If pIdx is an index on one or more expressions, then look through
-
      ** all the expressions in pWInfo and try to transform matching expressions
-
      ** into reference to index columns.  Also attempt to translate references
-
      ** to virtual columns in the table into references to (stored) columns
-
      ** of the index.
-
      **
-
      ** Do not do this for the RHS of a LEFT JOIN. This is because the
-
      ** expression may be evaluated after OP_NullRow has been executed on
-
      ** the cursor. In this case it is important to do the full evaluation,
-
      ** as the result of the expression may not be NULL, even if all table
-
      ** column values are.  https://www.sqlite.org/src/info/7fa8049685b50b5a
-
      **
-
      ** Also, do not do this when processing one index an a multi-index
-
      ** OR clause, since the transformation will become invalid once we
-
      ** move forward to the next index.
-
      ** https://sqlite.org/src/info/4e8e4857d32d401f
-
      */
-
      if( (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)==0 ){
-
        whereIndexExprTrans(pIdx, iCur, iIdxCur, pWInfo);
-
      }
-

      /* If a partial index is driving the loop, try to eliminate WHERE clause
      ** terms from the query that must be true due to the WHERE clause of
      ** the partial index.
@@ -149681,7 +152991,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
      /* The following assert() is not a requirement, merely an observation:
      ** The OR-optimization doesn't work for the right hand table of
      ** a LEFT JOIN: */
-
      assert( (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)==0 );
+
      assert( (pWInfo->wctrlFlags & (WHERE_OR_SUBCLAUSE|WHERE_RIGHT_JOIN))==0 );
    }

    /* Record the instruction used to terminate the loop. */
@@ -149778,7 +153088,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
      int nNotReady;                 /* The number of notReady tables */
      SrcItem *origSrc;              /* Original list of tables */
      nNotReady = pWInfo->nLevel - iLevel - 1;
-
      pOrTab = sqlite3StackAllocRaw(db,
+
      pOrTab = sqlite3DbMallocRawNN(db,
                            sizeof(*pOrTab)+ nNotReady*sizeof(pOrTab->a[0]));
      if( pOrTab==0 ) return notReady;
      pOrTab->nAlloc = (u8)(nNotReady + 1);
@@ -149885,7 +153195,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
        Expr *pDelete;                  /* Local copy of OR clause term */
        int jmp1 = 0;                   /* Address of jump operation */
        testcase( (pTabItem[0].fg.jointype & JT_LEFT)!=0
-
               && !ExprHasProperty(pOrExpr, EP_FromJoin)
+
               && !ExprHasProperty(pOrExpr, EP_OuterON)
        ); /* See TH3 vtab25.400 and ticket 614b25314c766238 */
        pDelete = pOrExpr = sqlite3ExprDup(db, pOrExpr, 0);
        if( db->mallocFailed ){
@@ -150023,7 +153333,15 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
    sqlite3VdbeGoto(v, pLevel->addrBrk);
    sqlite3VdbeResolveLabel(v, iLoopBody);

-
    if( pWInfo->nLevel>1 ){ sqlite3StackFree(db, pOrTab); }
+
    /* Set the P2 operand of the OP_Return opcode that will end the current
+
    ** loop to point to this spot, which is the top of the next containing
+
    ** loop.  The byte-code formatter will use that P2 value as a hint to
+
    ** indent everything in between the this point and the final OP_Return.
+
    ** See tag-20220407a in vdbe.c and shell.c */
+
    assert( pLevel->op==OP_Return );
+
    pLevel->p2 = sqlite3VdbeCurrentAddr(v);
+

+
    if( pWInfo->nLevel>1 ){ sqlite3DbFreeNN(db, pOrTab); }
    if( !untestedTerms ) disableTerm(pLevel, pTerm);
  }else
#endif /* SQLITE_OMIT_OR_OPTIMIZATION */
@@ -150085,10 +153403,22 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
      }
      pE = pTerm->pExpr;
      assert( pE!=0 );
-
      if( (pTabItem->fg.jointype&JT_LEFT) && !ExprHasProperty(pE,EP_FromJoin) ){
-
        continue;
+
      if( pTabItem->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT) ){
+
        if( !ExprHasProperty(pE,EP_OuterON|EP_InnerON) ){
+
          /* Defer processing WHERE clause constraints until after outer
+
          ** join processing.  tag-20220513a */
+
          continue;
+
        }else if( (pTabItem->fg.jointype & JT_LEFT)==JT_LEFT
+
               && !ExprHasProperty(pE,EP_OuterON) ){
+
          continue;
+
        }else{
+
          Bitmask m = sqlite3WhereGetMask(&pWInfo->sMaskSet, pE->w.iJoin);
+
          if( m & pLevel->notReady ){
+
            /* An ON clause that is not ripe */
+
            continue;
+
          }
+
        }
      }
-

      if( iLoop==1 && !sqlite3ExprCoveredByIndex(pE, pLevel->iTabCur, pIdx) ){
        iNext = 2;
        continue;
@@ -150147,7 +153477,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
    if( (pTerm->eOperator & (WO_EQ|WO_IS))==0 ) continue;
    if( (pTerm->eOperator & WO_EQUIV)==0 ) continue;
    if( pTerm->leftCursor!=iCur ) continue;
-
    if( pTabItem->fg.jointype & JT_LEFT ) continue;
+
    if( pTabItem->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT) ) continue;
    pE = pTerm->pExpr;
#ifdef WHERETRACE_ENABLED /* 0x800 */
    if( sqlite3WhereTrace & 0x800 ){
@@ -150155,7 +153485,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
      sqlite3WhereTermPrint(pTerm, pWC->nTerm-j);
    }
#endif
-
    assert( !ExprHasProperty(pE, EP_FromJoin) );
+
    assert( !ExprHasProperty(pE, EP_OuterON) );
    assert( (pTerm->prereqRight & pLevel->notReady)!=0 );
    assert( (pTerm->eOperator & (WO_OR|WO_AND))==0 );
    pAlt = sqlite3WhereFindTerm(pWC, iCur, pTerm->u.x.leftColumn, notReady,
@@ -150178,6 +153508,47 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
    pAlt->wtFlags |= TERM_CODED;
  }

+
  /* For a RIGHT OUTER JOIN, record the fact that the current row has
+
  ** been matched at least once.
+
  */
+
  if( pLevel->pRJ ){
+
    Table *pTab;
+
    int nPk;
+
    int r;
+
    int jmp1 = 0;
+
    WhereRightJoin *pRJ = pLevel->pRJ;
+

+
    /* pTab is the right-hand table of the RIGHT JOIN.  Generate code that
+
    ** will record that the current row of that table has been matched at
+
    ** least once.  This is accomplished by storing the PK for the row in
+
    ** both the iMatch index and the regBloom Bloom filter.
+
    */
+
    pTab = pWInfo->pTabList->a[pLevel->iFrom].pTab;
+
    if( HasRowid(pTab) ){
+
      r = sqlite3GetTempRange(pParse, 2);
+
      sqlite3ExprCodeGetColumnOfTable(v, pTab, pLevel->iTabCur, -1, r+1);
+
      nPk = 1;
+
    }else{
+
      int iPk;
+
      Index *pPk = sqlite3PrimaryKeyIndex(pTab);
+
      nPk = pPk->nKeyCol;
+
      r = sqlite3GetTempRange(pParse, nPk+1);
+
      for(iPk=0; iPk<nPk; iPk++){
+
        int iCol = pPk->aiColumn[iPk];
+
        sqlite3ExprCodeGetColumnOfTable(v, pTab, iCur, iCol,r+1+iPk);
+
      }
+
    }
+
    jmp1 = sqlite3VdbeAddOp4Int(v, OP_Found, pRJ->iMatch, 0, r+1, nPk);
+
    VdbeCoverage(v);
+
    VdbeComment((v, "match against %s", pTab->zName));
+
    sqlite3VdbeAddOp3(v, OP_MakeRecord, r+1, nPk, r);
+
    sqlite3VdbeAddOp4Int(v, OP_IdxInsert, pRJ->iMatch, r, r+1, nPk);
+
    sqlite3VdbeAddOp4Int(v, OP_FilterAdd, pRJ->regBloom, 0, r+1, nPk);
+
    sqlite3VdbeChangeP5(v, OPFLAG_USESEEKRESULT);
+
    sqlite3VdbeJumpHere(v, jmp1);
+
    sqlite3ReleaseTempRange(pParse, r, nPk+1);
+
  }
+

  /* For a LEFT OUTER JOIN, generate code that will record the fact that
  ** at least one row of the right table has matched the left table.
  */
@@ -150185,6 +153556,30 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
    pLevel->addrFirst = sqlite3VdbeCurrentAddr(v);
    sqlite3VdbeAddOp2(v, OP_Integer, 1, pLevel->iLeftJoin);
    VdbeComment((v, "record LEFT JOIN hit"));
+
    if( pLevel->pRJ==0 ){
+
      goto code_outer_join_constraints; /* WHERE clause constraints */
+
    }
+
  }
+

+
  if( pLevel->pRJ ){
+
    /* Create a subroutine used to process all interior loops and code
+
    ** of the RIGHT JOIN.  During normal operation, the subroutine will
+
    ** be in-line with the rest of the code.  But at the end, a separate
+
    ** loop will run that invokes this subroutine for unmatched rows
+
    ** of pTab, with all tables to left begin set to NULL.
+
    */
+
    WhereRightJoin *pRJ = pLevel->pRJ;
+
    sqlite3VdbeAddOp2(v, OP_BeginSubrtn, 0, pRJ->regReturn);
+
    pRJ->addrSubrtn = sqlite3VdbeCurrentAddr(v);
+
    assert( pParse->withinRJSubrtn < 255 );
+
    pParse->withinRJSubrtn++;
+

+
    /* WHERE clause constraints must be deferred until after outer join
+
    ** row elimination has completed, since WHERE clause constraints apply
+
    ** to the results of the OUTER JOIN.  The following loop generates the
+
    ** appropriate WHERE clause constraint checks.  tag-20220513a.
+
    */
+
  code_outer_join_constraints:
    for(pTerm=pWC->a, j=0; j<pWC->nBase; j++, pTerm++){
      testcase( pTerm->wtFlags & TERM_VIRTUAL );
      testcase( pTerm->wtFlags & TERM_CODED );
@@ -150193,6 +153588,7 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
        assert( pWInfo->untestedTerms );
        continue;
      }
+
      if( pTabItem->fg.jointype & JT_LTORJ ) continue;
      assert( pTerm->pExpr );
      sqlite3ExprIfFalse(pParse, pTerm->pExpr, addrCont, SQLITE_JUMPIFNULL);
      pTerm->wtFlags |= TERM_CODED;
@@ -150213,6 +153609,96 @@ SQLITE_PRIVATE Bitmask sqlite3WhereCodeOneLoopStart(
  return pLevel->notReady;
}

+
/*
+
** Generate the code for the loop that finds all non-matched terms
+
** for a RIGHT JOIN.
+
*/
+
SQLITE_PRIVATE SQLITE_NOINLINE void sqlite3WhereRightJoinLoop(
+
  WhereInfo *pWInfo,
+
  int iLevel,
+
  WhereLevel *pLevel
+
){
+
  Parse *pParse = pWInfo->pParse;
+
  Vdbe *v = pParse->pVdbe;
+
  WhereRightJoin *pRJ = pLevel->pRJ;
+
  Expr *pSubWhere = 0;
+
  WhereClause *pWC = &pWInfo->sWC;
+
  WhereInfo *pSubWInfo;
+
  WhereLoop *pLoop = pLevel->pWLoop;
+
  SrcItem *pTabItem = &pWInfo->pTabList->a[pLevel->iFrom];
+
  SrcList sFrom;
+
  Bitmask mAll = 0;
+
  int k;
+

+
  ExplainQueryPlan((pParse, 1, "RIGHT-JOIN %s", pTabItem->pTab->zName));
+
  sqlite3VdbeNoJumpsOutsideSubrtn(v, pRJ->addrSubrtn, pRJ->endSubrtn,
+
                                  pRJ->regReturn);
+
  for(k=0; k<iLevel; k++){
+
    int iIdxCur;
+
    mAll |= pWInfo->a[k].pWLoop->maskSelf;
+
    sqlite3VdbeAddOp1(v, OP_NullRow, pWInfo->a[k].iTabCur);
+
    iIdxCur = pWInfo->a[k].iIdxCur;
+
    if( iIdxCur ){
+
      sqlite3VdbeAddOp1(v, OP_NullRow, iIdxCur);
+
    }
+
  }
+
  if( (pTabItem->fg.jointype & JT_LTORJ)==0 ){
+
    mAll |= pLoop->maskSelf;
+
    for(k=0; k<pWC->nTerm; k++){
+
      WhereTerm *pTerm = &pWC->a[k];
+
      if( (pTerm->wtFlags & (TERM_VIRTUAL|TERM_SLICE))!=0
+
       && pTerm->eOperator!=WO_ROWVAL
+
      ){
+
        break;
+
      }
+
      if( pTerm->prereqAll & ~mAll ) continue;
+
      if( ExprHasProperty(pTerm->pExpr, EP_OuterON|EP_InnerON) ) continue;
+
      pSubWhere = sqlite3ExprAnd(pParse, pSubWhere,
+
                                 sqlite3ExprDup(pParse->db, pTerm->pExpr, 0));
+
    }
+
  }
+
  sFrom.nSrc = 1;
+
  sFrom.nAlloc = 1;
+
  memcpy(&sFrom.a[0], pTabItem, sizeof(SrcItem));
+
  sFrom.a[0].fg.jointype = 0;
+
  assert( pParse->withinRJSubrtn < 100 );
+
  pParse->withinRJSubrtn++;
+
  pSubWInfo = sqlite3WhereBegin(pParse, &sFrom, pSubWhere, 0, 0, 0,
+
                                WHERE_RIGHT_JOIN, 0);
+
  if( pSubWInfo ){
+
    int iCur = pLevel->iTabCur;
+
    int r = ++pParse->nMem;
+
    int nPk;
+
    int jmp;
+
    int addrCont = sqlite3WhereContinueLabel(pSubWInfo);
+
    Table *pTab = pTabItem->pTab;
+
    if( HasRowid(pTab) ){
+
      sqlite3ExprCodeGetColumnOfTable(v, pTab, iCur, -1, r);
+
      nPk = 1;
+
    }else{
+
      int iPk;
+
      Index *pPk = sqlite3PrimaryKeyIndex(pTab);
+
      nPk = pPk->nKeyCol;
+
      pParse->nMem += nPk - 1;
+
      for(iPk=0; iPk<nPk; iPk++){
+
        int iCol = pPk->aiColumn[iPk];
+
        sqlite3ExprCodeGetColumnOfTable(v, pTab, iCur, iCol,r+iPk);
+
      }
+
    }
+
    jmp = sqlite3VdbeAddOp4Int(v, OP_Filter, pRJ->regBloom, 0, r, nPk);
+
    VdbeCoverage(v);
+
    sqlite3VdbeAddOp4Int(v, OP_Found, pRJ->iMatch, addrCont, r, nPk);
+
    VdbeCoverage(v);
+
    sqlite3VdbeJumpHere(v, jmp);
+
    sqlite3VdbeAddOp2(v, OP_Gosub, pRJ->regReturn, pRJ->addrSubrtn);
+
    sqlite3WhereEnd(pSubWInfo);
+
  }
+
  sqlite3ExprDelete(pParse->db, pSubWhere);
+
  ExplainQueryPlanPop(pParse);
+
  assert( pParse->withinRJSubrtn>0 );
+
  pParse->withinRJSubrtn--;
+
}
+

/************** End of wherecode.c *******************************************/
/************** Begin file whereexpr.c ***************************************/
/*
@@ -150281,7 +153767,7 @@ static int whereClauseInsert(WhereClause *pWC, Expr *p, u16 wtFlags){
  if( pWC->nTerm>=pWC->nSlot ){
    WhereTerm *pOld = pWC->a;
    sqlite3 *db = pWC->pWInfo->pParse->db;
-
    pWC->a = sqlite3DbMallocRawNN(db, sizeof(pWC->a[0])*pWC->nSlot*2 );
+
    pWC->a = sqlite3WhereMalloc(pWC->pWInfo, sizeof(pWC->a[0])*pWC->nSlot*2 );
    if( pWC->a==0 ){
      if( wtFlags & TERM_DYNAMIC ){
        sqlite3ExprDelete(db, p);
@@ -150290,10 +153776,7 @@ static int whereClauseInsert(WhereClause *pWC, Expr *p, u16 wtFlags){
      return 0;
    }
    memcpy(pWC->a, pOld, sizeof(pWC->a[0])*pWC->nTerm);
-
    if( pOld!=pWC->aStatic ){
-
      sqlite3DbFree(db, pOld);
-
    }
-
    pWC->nSlot = sqlite3DbMallocSize(db, pWC->a)/sizeof(pWC->a[0]);
+
    pWC->nSlot = pWC->nSlot*2;
  }
  pTerm = &pWC->a[idx = pWC->nTerm++];
  if( (wtFlags & TERM_VIRTUAL)==0 ) pWC->nBase = pWC->nTerm;
@@ -150486,7 +153969,7 @@ static int isLikeOrGlob(
        if( pLeft->op!=TK_COLUMN
         || sqlite3ExprAffinity(pLeft)!=SQLITE_AFF_TEXT
         || (ALWAYS( ExprUseYTab(pLeft) )
-
             && pLeft->y.pTab
+
             && ALWAYS(pLeft->y.pTab)
             && IsVirtual(pLeft->y.pTab))  /* Might be numeric */
        ){
          int isNum;
@@ -150603,8 +154086,7 @@ static int isAuxiliaryVtabOperator(
    **       MATCH(expression,vtab_column)
    */
    pCol = pList->a[1].pExpr;
-
    assert( pCol->op!=TK_COLUMN || ExprUseYTab(pCol) );
-
    testcase( pCol->op==TK_COLUMN && pCol->y.pTab==0 );
+
    assert( pCol->op!=TK_COLUMN || (ExprUseYTab(pCol) && pCol->y.pTab!=0) );
    if( ExprIsVtab(pCol) ){
      for(i=0; i<ArraySize(aOp); i++){
        assert( !ExprHasProperty(pExpr, EP_IntValue) );
@@ -150629,7 +154111,7 @@ static int isAuxiliaryVtabOperator(
    */
    pCol = pList->a[0].pExpr;
    assert( pCol->op!=TK_COLUMN || ExprUseYTab(pCol) );
-
    testcase( pCol->op==TK_COLUMN && pCol->y.pTab==0 );
+
    assert( pCol->op!=TK_COLUMN || (ExprUseYTab(pCol) && pCol->y.pTab!=0) );
    if( ExprIsVtab(pCol) ){
      sqlite3_vtab *pVtab;
      sqlite3_module *pMod;
@@ -150654,13 +154136,12 @@ static int isAuxiliaryVtabOperator(
    int res = 0;
    Expr *pLeft = pExpr->pLeft;
    Expr *pRight = pExpr->pRight;
-
    assert( pLeft->op!=TK_COLUMN || ExprUseYTab(pLeft) );
-
    testcase( pLeft->op==TK_COLUMN && pLeft->y.pTab==0 );
+
    assert( pLeft->op!=TK_COLUMN || (ExprUseYTab(pLeft) && pLeft->y.pTab!=0) );
    if( ExprIsVtab(pLeft) ){
      res++;
    }
-
    assert( pRight==0 || pRight->op!=TK_COLUMN || ExprUseYTab(pRight) );
-
    testcase( pRight && pRight->op==TK_COLUMN && pRight->y.pTab==0 );
+
    assert( pRight==0 || pRight->op!=TK_COLUMN
+
            || (ExprUseYTab(pRight) && pRight->y.pTab!=0) );
    if( pRight && ExprIsVtab(pRight) ){
      res++;
      SWAP(Expr*, pLeft, pRight);
@@ -150681,9 +154162,9 @@ static int isAuxiliaryVtabOperator(
** a join, then transfer the appropriate markings over to derived.
*/
static void transferJoinMarkings(Expr *pDerived, Expr *pBase){
-
  if( pDerived ){
-
    pDerived->flags |= pBase->flags & EP_FromJoin;
-
    pDerived->w.iRightJoinTable = pBase->w.iRightJoinTable;
+
  if( pDerived && ExprHasProperty(pBase, EP_OuterON|EP_InnerON) ){
+
    pDerived->flags |= pBase->flags & (EP_OuterON|EP_InnerON);
+
    pDerived->w.iJoin = pBase->w.iJoin;
  }
}

@@ -151137,7 +154618,7 @@ static int termIsEquivalence(Parse *pParse, Expr *pExpr){
  CollSeq *pColl;
  if( !OptimizationEnabled(pParse->db, SQLITE_Transitive) ) return 0;
  if( pExpr->op!=TK_EQ && pExpr->op!=TK_IS ) return 0;
-
  if( ExprHasProperty(pExpr, EP_FromJoin) ) return 0;
+
  if( ExprHasProperty(pExpr, EP_OuterON) ) return 0;
  aff1 = sqlite3ExprAffinity(pExpr->pLeft);
  aff2 = sqlite3ExprAffinity(pExpr->pRight);
  if( aff1!=aff2
@@ -151168,7 +154649,9 @@ static Bitmask exprSelectUsage(WhereMaskSet *pMaskSet, Select *pS){
      int i;
      for(i=0; i<pSrc->nSrc; i++){
        mask |= exprSelectUsage(pMaskSet, pSrc->a[i].pSelect);
-
        mask |= sqlite3WhereExprUsage(pMaskSet, pSrc->a[i].pOn);
+
        if( pSrc->a[i].fg.isUsing==0 ){
+
          mask |= sqlite3WhereExprUsage(pMaskSet, pSrc->a[i].u3.pOn);
+
        }
        if( pSrc->a[i].fg.isTabFunc ){
          mask |= sqlite3WhereExprListUsage(pMaskSet, pSrc->a[i].u1.pFuncArg);
        }
@@ -151207,6 +154690,7 @@ static SQLITE_NOINLINE int exprMightBeIndexed2(
    if( pIdx->aColExpr==0 ) continue;
    for(i=0; i<pIdx->nKeyCol; i++){
      if( pIdx->aiColumn[i]!=XN_EXPR ) continue;
+
      assert( pIdx->bHasExpr );
      if( sqlite3ExprCompareSkip(pExpr, pIdx->aColExpr->a[i].pExpr, iCur)==0 ){
        aiCurCol[0] = iCur;
        aiCurCol[1] = XN_EXPR;
@@ -151323,18 +154807,32 @@ static void exprAnalyze(
  if( prereqAll!=sqlite3WhereExprUsageNN(pMaskSet, pExpr) ){
    printf("\n*** Incorrect prereqAll computed for:\n");
    sqlite3TreeViewExpr(0,pExpr,0);
-
    abort();
+
    assert( 0 );
  }
#endif

-
  if( ExprHasProperty(pExpr, EP_FromJoin) ){
-
    Bitmask x = sqlite3WhereGetMask(pMaskSet, pExpr->w.iRightJoinTable);
-
    prereqAll |= x;
-
    extraRight = x-1;  /* ON clause terms may not be used with an index
-
                       ** on left table of a LEFT JOIN.  Ticket #3015 */
-
    if( (prereqAll>>1)>=x ){
-
      sqlite3ErrorMsg(pParse, "ON clause references tables to its right");
-
      return;
+
  if( ExprHasProperty(pExpr, EP_OuterON|EP_InnerON) ){
+
    Bitmask x = sqlite3WhereGetMask(pMaskSet, pExpr->w.iJoin);
+
    if( ExprHasProperty(pExpr, EP_OuterON) ){
+
      prereqAll |= x;
+
      extraRight = x-1;  /* ON clause terms may not be used with an index
+
                         ** on left table of a LEFT JOIN.  Ticket #3015 */
+
      if( (prereqAll>>1)>=x ){
+
        sqlite3ErrorMsg(pParse, "ON clause references tables to its right");
+
        return;
+
      }
+
    }else if( (prereqAll>>1)>=x ){
+
      /* The ON clause of an INNER JOIN references a table to its right.
+
      ** Most other SQL database engines raise an error.  But SQLite versions
+
      ** 3.0 through 3.38 just put the ON clause constraint into the WHERE
+
      ** clause and carried on.   Beginning with 3.39, raise an error only
+
      ** if there is a RIGHT or FULL JOIN in the query.  This makes SQLite
+
      ** more like other systems, and also preserves legacy. */
+
      if( ALWAYS(pSrc->nSrc>0) && (pSrc->a[0].fg.jointype & JT_LTORJ)!=0 ){
+
        sqlite3ErrorMsg(pParse, "ON clause references tables to its right");
+
        return;
+
      }
+
      ExprClearProperty(pExpr, EP_InnerON);
    }
  }
  pTerm->prereqAll = prereqAll;
@@ -151402,7 +154900,7 @@ static void exprAnalyze(
      pNew->eOperator = (operatorMask(pDup->op) + eExtraOp) & opMask;
    }else
    if( op==TK_ISNULL
-
     && !ExprHasProperty(pExpr,EP_FromJoin)
+
     && !ExprHasProperty(pExpr,EP_OuterON)
     && 0==sqlite3ExprCanBeNull(pLeft)
    ){
      assert( !ExprHasProperty(pExpr, EP_IntValue) );
@@ -151473,7 +154971,7 @@ static void exprAnalyze(
  else if( pExpr->op==TK_NOTNULL ){
    if( pExpr->pLeft->op==TK_COLUMN
     && pExpr->pLeft->iColumn>=0
-
     && !ExprHasProperty(pExpr, EP_FromJoin)
+
     && !ExprHasProperty(pExpr, EP_OuterON)
    ){
      Expr *pNewExpr;
      Expr *pLeft = pExpr->pLeft;
@@ -151621,7 +155119,7 @@ static void exprAnalyze(
    }
    pTerm = &pWC->a[idxTerm];
    pTerm->wtFlags |= TERM_CODED|TERM_VIRTUAL;  /* Disable the original */
-
    pTerm->eOperator = 0;
+
    pTerm->eOperator = WO_ROWVAL;
  }

  /* If there is a vector IN term - e.g. "(a, b) IN (SELECT ...)" - create
@@ -151677,9 +155175,9 @@ static void exprAnalyze(
        Expr *pNewExpr;
        pNewExpr = sqlite3PExpr(pParse, TK_MATCH,
            0, sqlite3ExprDup(db, pRight, 0));
-
        if( ExprHasProperty(pExpr, EP_FromJoin) && pNewExpr ){
-
          ExprSetProperty(pNewExpr, EP_FromJoin);
-
          pNewExpr->w.iRightJoinTable = pExpr->w.iRightJoinTable;
+
        if( ExprHasProperty(pExpr, EP_OuterON) && pNewExpr ){
+
          ExprSetProperty(pNewExpr, EP_OuterON);
+
          pNewExpr->w.iJoin = pExpr->w.iJoin;
        }
        idxNew = whereClauseInsert(pWC, pNewExpr, TERM_VIRTUAL|TERM_DYNAMIC);
        testcase( idxNew==0 );
@@ -151806,9 +155304,9 @@ static void whereAddLimitExpr(
** exist only so that they may be passed to the xBestIndex method of the
** single virtual table in the FROM clause of the SELECT.
*/
-
SQLITE_PRIVATE void sqlite3WhereAddLimit(WhereClause *pWC, Select *p){
-
  assert( p==0 || (p->pGroupBy==0 && (p->selFlags & SF_Aggregate)==0) );
-
  if( (p && p->pLimit)                                          /* 1 */
+
SQLITE_PRIVATE void SQLITE_NOINLINE sqlite3WhereAddLimit(WhereClause *pWC, Select *p){
+
  assert( p!=0 && p->pLimit!=0 );                 /* 1 -- checked by caller */
+
  if( p->pGroupBy==0
   && (p->selFlags & (SF_Distinct|SF_Aggregate))==0             /* 2 */
   && (p->pSrc->nSrc==1 && IsVirtual(p->pSrc->a[0].pTab))       /* 3 */
  ){
@@ -151822,7 +155320,7 @@ SQLITE_PRIVATE void sqlite3WhereAddLimit(WhereClause *pWC, Select *p){
        /* This term is a vector operation that has been decomposed into
        ** other, subsequent terms.  It can be ignored. See tag-20220128a */
        assert( pWC->a[ii].wtFlags & TERM_VIRTUAL );
-
        assert( pWC->a[ii].eOperator==0 );
+
        assert( pWC->a[ii].eOperator==WO_ROWVAL );
        continue;
      }
      if( pWC->a[ii].leftCursor!=iCsr ) return;
@@ -151834,7 +155332,7 @@ SQLITE_PRIVATE void sqlite3WhereAddLimit(WhereClause *pWC, Select *p){
        Expr *pExpr = pOrderBy->a[ii].pExpr;
        if( pExpr->op!=TK_COLUMN ) return;
        if( pExpr->iTable!=iCsr ) return;
-
        if( pOrderBy->a[ii].sortFlags & KEYINFO_ORDER_BIGNULL ) return;
+
        if( pOrderBy->a[ii].fg.sortFlags & KEYINFO_ORDER_BIGNULL ) return;
      }
    }

@@ -151901,9 +155399,6 @@ SQLITE_PRIVATE void sqlite3WhereClauseClear(WhereClause *pWC){
      a++;
    }
  }
-
  if( pWC->a!=pWC->aStatic ){
-
    sqlite3DbFree(db, pWC->a);
-
  }
}


@@ -152030,6 +155525,7 @@ SQLITE_PRIVATE void sqlite3WhereTabFuncArgs(
  if( pArgs==0 ) return;
  for(j=k=0; j<pArgs->nExpr; j++){
    Expr *pRhs;
+
    u32 joinType;
    while( k<pTab->nCol && (pTab->aCol[k].colFlags & COLFLAG_HIDDEN)==0 ){k++;}
    if( k>=pTab->nCol ){
      sqlite3ErrorMsg(pParse, "too many arguments on %s() - max %d",
@@ -152046,9 +155542,12 @@ SQLITE_PRIVATE void sqlite3WhereTabFuncArgs(
    pRhs = sqlite3PExpr(pParse, TK_UPLUS,
        sqlite3ExprDup(pParse->db, pArgs->a[j].pExpr, 0), 0);
    pTerm = sqlite3PExpr(pParse, TK_EQ, pColRef, pRhs);
-
    if( pItem->fg.jointype & JT_LEFT ){
-
      sqlite3SetJoinExpr(pTerm, pItem->iCursor);
+
    if( pItem->fg.jointype & (JT_LEFT|JT_LTORJ) ){
+
      joinType = EP_OuterON;
+
    }else{
+
      joinType = EP_InnerON;
    }
+
    sqlite3SetJoinExpr(pTerm, pItem->iCursor, joinType);
    whereClauseInsert(pWC, pTerm, TERM_DYNAMIC);
  }
}
@@ -152159,7 +155658,7 @@ SQLITE_PRIVATE int sqlite3WhereOrderByLimitOptLabel(WhereInfo *pWInfo){
  }
  pInner = &pWInfo->a[pWInfo->nLevel-1];
  assert( pInner->addrNxt!=0 );
-
  return pInner->addrNxt;
+
  return pInner->pRJ ? pWInfo->iContinue : pInner->addrNxt;
}

/*
@@ -152310,6 +155809,30 @@ SQLITE_PRIVATE Bitmask sqlite3WhereGetMask(WhereMaskSet *pMaskSet, int iCursor){
  return 0;
}

+
/* Allocate memory that is automatically freed when pWInfo is freed.
+
*/
+
SQLITE_PRIVATE void *sqlite3WhereMalloc(WhereInfo *pWInfo, u64 nByte){
+
  WhereMemBlock *pBlock;
+
  pBlock = sqlite3DbMallocRawNN(pWInfo->pParse->db, nByte+sizeof(*pBlock));
+
  if( pBlock ){
+
    pBlock->pNext = pWInfo->pMemToFree;
+
    pBlock->sz = nByte;
+
    pWInfo->pMemToFree = pBlock;
+
    pBlock++;
+
  }
+
  return (void*)pBlock;
+
}
+
SQLITE_PRIVATE void *sqlite3WhereRealloc(WhereInfo *pWInfo, void *pOld, u64 nByte){
+
  void *pNew = sqlite3WhereMalloc(pWInfo, nByte);
+
  if( pNew && pOld ){
+
    WhereMemBlock *pOldBlk = (WhereMemBlock*)pOld;
+
    pOldBlk--;
+
    assert( pOldBlk->sz<nByte );
+
    memcpy(pNew, pOld, pOldBlk->sz);
+
  }
+
  return pNew;
+
}
+

/*
** Create a new mask for cursor iCursor.
**
@@ -152363,7 +155886,7 @@ static WhereTerm *whereScanNext(WhereScan *pScan){
         && (iColumn!=XN_EXPR
             || sqlite3ExprCompareSkip(pTerm->pExpr->pLeft,
                                       pScan->pIdxExpr,iCur)==0)
-
         && (pScan->iEquiv<=1 || !ExprHasProperty(pTerm->pExpr, EP_FromJoin))
+
         && (pScan->iEquiv<=1 || !ExprHasProperty(pTerm->pExpr, EP_OuterON))
        ){
          if( (pTerm->eOperator & WO_EQUIV)!=0
           && pScan->nEquiv<ArraySize(pScan->aiCur)
@@ -152715,6 +156238,7 @@ static void translateColumnToCopy(
      pOp->p1 = pOp->p2 + iRegister;
      pOp->p2 = pOp->p3;
      pOp->p3 = 0;
+
      pOp->p5 = 2;  /* Cause the MEM_Subtype flag to be cleared */
    }else if( pOp->opcode==OP_Rowid ){
      pOp->opcode = OP_Sequence;
      pOp->p1 = iAutoidxCur;
@@ -152775,6 +156299,43 @@ static void whereTraceIndexInfoOutputs(sqlite3_index_info *p){
#define whereTraceIndexInfoOutputs(A)
#endif

+
/*
+
** We know that pSrc is an operand of an outer join.  Return true if
+
** pTerm is a constraint that is compatible with that join.
+
**
+
** pTerm must be EP_OuterON if pSrc is the right operand of an
+
** outer join.  pTerm can be either EP_OuterON or EP_InnerON if pSrc
+
** is the left operand of a RIGHT join.
+
**
+
** See https://sqlite.org/forum/forumpost/206d99a16dd9212f
+
** for an example of a WHERE clause constraints that may not be used on
+
** the right table of a RIGHT JOIN because the constraint implies a
+
** not-NULL condition on the left table of the RIGHT JOIN.
+
*/
+
static int constraintCompatibleWithOuterJoin(
+
  const WhereTerm *pTerm,       /* WHERE clause term to check */
+
  const SrcItem *pSrc           /* Table we are trying to access */
+
){
+
  assert( (pSrc->fg.jointype&(JT_LEFT|JT_LTORJ|JT_RIGHT))!=0 ); /* By caller */
+
  testcase( (pSrc->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT))==JT_LEFT );
+
  testcase( (pSrc->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT))==JT_LTORJ );
+
  testcase( ExprHasProperty(pTerm->pExpr, EP_OuterON) )
+
  testcase( ExprHasProperty(pTerm->pExpr, EP_InnerON) );
+
  if( !ExprHasProperty(pTerm->pExpr, EP_OuterON|EP_InnerON)
+
   || pTerm->pExpr->w.iJoin != pSrc->iCursor
+
  ){
+
    return 0;
+
  }
+
  if( (pSrc->fg.jointype & (JT_LEFT|JT_RIGHT))!=0
+
   && ExprHasProperty(pTerm->pExpr, EP_InnerON)
+
  ){
+
    return 0;
+
  }
+
  return 1;
+
}
+

+

+

#ifndef SQLITE_OMIT_AUTOMATIC_INDEX
/*
** Return TRUE if the WHERE clause term pTerm is of a form where it
@@ -152789,14 +156350,11 @@ static int termCanDriveIndex(
  char aff;
  if( pTerm->leftCursor!=pSrc->iCursor ) return 0;
  if( (pTerm->eOperator & (WO_EQ|WO_IS))==0 ) return 0;
-
  if( (pSrc->fg.jointype & JT_LEFT)
-
   && !ExprHasProperty(pTerm->pExpr, EP_FromJoin)
-
   && (pTerm->eOperator & WO_IS)
+
  assert( (pSrc->fg.jointype & JT_RIGHT)==0 );
+
  if( (pSrc->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT))!=0
+
   && !constraintCompatibleWithOuterJoin(pTerm,pSrc)
  ){
-
    /* Cannot use an IS term from the WHERE clause as an index driver for
-
    ** the RHS of a LEFT JOIN. Such a term can only be used if it is from
-
    ** the ON clause.  */
-
    return 0;
+
    return 0;  /* See https://sqlite.org/forum/forumpost/51e6959f61 */
  }
  if( (pTerm->prereqRight & notReady)!=0 ) return 0;
  assert( (pTerm->eOperator & (WO_OR|WO_AND))==0 );
@@ -153137,7 +156695,7 @@ static SQLITE_NOINLINE void sqlite3ConstructBloomFilter(
      const SrcItem *pTabItem;
      pLevel = &pWInfo->a[iLevel];
      pTabItem = &pWInfo->pTabList->a[pLevel->iFrom];
-
      if( pTabItem->fg.jointype & JT_LEFT ) continue;
+
      if( pTabItem->fg.jointype & (JT_LEFT|JT_LTORJ) ) continue;
      pLoop = pLevel->pWLoop;
      if( NEVER(pLoop==0) ) continue;
      if( pLoop->prereq & notReady ) continue;
@@ -153208,12 +156766,8 @@ static sqlite3_index_info *allocateIndexInfo(
    assert( (pTerm->eOperator & (WO_OR|WO_AND))==0 );
    assert( pTerm->u.x.leftColumn>=XN_ROWID );
    assert( pTerm->u.x.leftColumn<pTab->nCol );
-

-
    /* tag-20191211-002: WHERE-clause constraints are not useful to the
-
    ** right-hand table of a LEFT JOIN.  See tag-20191211-001 for the
-
    ** equivalent restriction for ordinary tables. */
-
    if( (pSrc->fg.jointype & JT_LEFT)!=0
-
     && !ExprHasProperty(pTerm->pExpr, EP_FromJoin)
+
    if( (pSrc->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT))!=0
+
     && !constraintCompatibleWithOuterJoin(pTerm,pSrc)
    ){
      continue;
    }
@@ -153238,7 +156792,7 @@ static sqlite3_index_info *allocateIndexInfo(
      }

      /* Virtual tables are unable to deal with NULLS FIRST */
-
      if( pOrderBy->a[i].sortFlags & KEYINFO_ORDER_BIGNULL ) break;
+
      if( pOrderBy->a[i].fg.sortFlags & KEYINFO_ORDER_BIGNULL ) break;

      /* First case - a direct column references without a COLLATE operator */
      if( pExpr->op==TK_COLUMN && pExpr->iTable==pSrc->iCursor ){
@@ -153268,8 +156822,10 @@ static sqlite3_index_info *allocateIndexInfo(
    }
    if( i==n ){
      nOrderBy = n;
-
      if( (pWInfo->wctrlFlags & (WHERE_GROUPBY|WHERE_DISTINCTBY)) ){
-
        eDistinct = 1 + ((pWInfo->wctrlFlags & WHERE_DISTINCTBY)!=0);
+
      if( (pWInfo->wctrlFlags & WHERE_DISTINCTBY) ){
+
        eDistinct = 2 + ((pWInfo->wctrlFlags & WHERE_SORTBYGROUP)!=0);
+
      }else if( pWInfo->wctrlFlags & WHERE_GROUPBY ){
+
        eDistinct = 1;
      }
    }
  }
@@ -153348,7 +156904,7 @@ static sqlite3_index_info *allocateIndexInfo(
         || (pExpr->op==TK_COLLATE && pExpr->pLeft->op==TK_COLUMN
              && pExpr->iColumn==pExpr->pLeft->iColumn) );
    pIdxOrderBy[j].iColumn = pExpr->iColumn;
-
    pIdxOrderBy[j].desc = pOrderBy->a[i].sortFlags & KEYINFO_ORDER_DESC;
+
    pIdxOrderBy[j].desc = pOrderBy->a[i].fg.sortFlags & KEYINFO_ORDER_DESC;
    j++;
  }
  pIdxInfo->nOrderBy = j;
@@ -153452,7 +157008,7 @@ static int whereKeyStats(
#endif
  assert( pRec!=0 );
  assert( pIdx->nSample>0 );
-
  assert( pRec->nField>0 && pRec->nField<=pIdx->nSampleCol );
+
  assert( pRec->nField>0 );

  /* Do a binary search to find the first sample greater than or equal
  ** to pRec. If pRec contains a single field, the set of samples to search
@@ -153498,7 +157054,7 @@ static int whereKeyStats(
  ** it is extended to two fields. The duplicates that this creates do not
  ** cause any problems.
  */
-
  nField = pRec->nField;
+
  nField = MIN(pRec->nField, pIdx->nSample);
  iCol = 0;
  iSample = pIdx->nSample * nField;
  do{
@@ -153586,7 +157142,7 @@ static int whereKeyStats(
    ** is larger than all samples in the array. */
    tRowcnt iUpper, iGap;
    if( i>=pIdx->nSample ){
-
      iUpper = sqlite3LogEstToInt(pIdx->aiRowLogEst[0]);
+
      iUpper = pIdx->nRowEst0;
    }else{
      iUpper = aSample[i].anLt[iCol];
    }
@@ -154089,7 +157645,7 @@ SQLITE_PRIVATE void sqlite3WhereTermPrint(WhereTerm *pTerm, int iTerm){
    memcpy(zType, "....", 5);
    if( pTerm->wtFlags & TERM_VIRTUAL ) zType[0] = 'V';
    if( pTerm->eOperator & WO_EQUIV  ) zType[1] = 'E';
-
    if( ExprHasProperty(pTerm->pExpr, EP_FromJoin) ) zType[2] = 'L';
+
    if( ExprHasProperty(pTerm->pExpr, EP_OuterON) ) zType[2] = 'L';
    if( pTerm->wtFlags & TERM_CODED  ) zType[3] = 'C';
    if( pTerm->eOperator & WO_SINGLE ){
      assert( (pTerm->eOperator & (WO_OR|WO_AND))==0 );
@@ -154215,12 +157771,18 @@ static void whereLoopClearUnion(sqlite3 *db, WhereLoop *p){
}

/*
-
** Deallocate internal memory used by a WhereLoop object
+
** Deallocate internal memory used by a WhereLoop object.  Leave the
+
** object in an initialized state, as if it had been newly allocated.
*/
static void whereLoopClear(sqlite3 *db, WhereLoop *p){
-
  if( p->aLTerm!=p->aLTermSpace ) sqlite3DbFreeNN(db, p->aLTerm);
+
  if( p->aLTerm!=p->aLTermSpace ){
+
    sqlite3DbFreeNN(db, p->aLTerm);
+
    p->aLTerm = p->aLTermSpace;
+
    p->nLSlot = ArraySize(p->aLTermSpace);
+
  }
  whereLoopClearUnion(db, p);
-
  whereLoopInit(p);
+
  p->nLTerm = 0;
+
  p->wsFlags = 0;
}

/*
@@ -154244,7 +157806,9 @@ static int whereLoopResize(sqlite3 *db, WhereLoop *p, int n){
*/
static int whereLoopXfer(sqlite3 *db, WhereLoop *pTo, WhereLoop *pFrom){
  whereLoopClearUnion(db, pTo);
-
  if( whereLoopResize(db, pTo, pFrom->nLTerm) ){
+
  if( pFrom->nLTerm > pTo->nLSlot
+
   && whereLoopResize(db, pTo, pFrom->nLTerm)
+
  ){
    memset(pTo, 0, WHERE_LOOP_XFER_SZ);
    return SQLITE_NOMEM_BKPT;
  }
@@ -154262,42 +157826,29 @@ static int whereLoopXfer(sqlite3 *db, WhereLoop *pTo, WhereLoop *pFrom){
** Delete a WhereLoop object
*/
static void whereLoopDelete(sqlite3 *db, WhereLoop *p){
+
  assert( db!=0 );
  whereLoopClear(db, p);
-
  sqlite3DbFreeNN(db, p);
+
  sqlite3DbNNFreeNN(db, p);
}

/*
** Free a WhereInfo structure
*/
static void whereInfoFree(sqlite3 *db, WhereInfo *pWInfo){
-
  int i;
  assert( pWInfo!=0 );
-
  for(i=0; i<pWInfo->nLevel; i++){
-
    WhereLevel *pLevel = &pWInfo->a[i];
-
    if( pLevel->pWLoop && (pLevel->pWLoop->wsFlags & WHERE_IN_ABLE)!=0 ){
-
      assert( (pLevel->pWLoop->wsFlags & WHERE_MULTI_OR)==0 );
-
      sqlite3DbFree(db, pLevel->u.in.aInLoop);
-
    }
-
  }
+
  assert( db!=0 );
  sqlite3WhereClauseClear(&pWInfo->sWC);
  while( pWInfo->pLoops ){
    WhereLoop *p = pWInfo->pLoops;
    pWInfo->pLoops = p->pNextLoop;
    whereLoopDelete(db, p);
  }
-
  assert( pWInfo->pExprMods==0 );
-
  sqlite3DbFreeNN(db, pWInfo);
-
}
-

-
/* Undo all Expr node modifications
-
*/
-
static void whereUndoExprMods(WhereInfo *pWInfo){
-
  while( pWInfo->pExprMods ){
-
    WhereExprMod *p = pWInfo->pExprMods;
-
    pWInfo->pExprMods = p->pNext;
-
    memcpy(p->pExpr, &p->orig, sizeof(p->orig));
-
    sqlite3DbFree(pWInfo->pParse->db, p);
+
  while( pWInfo->pMemToFree ){
+
    WhereMemBlock *pNext = pWInfo->pMemToFree->pNext;
+
    sqlite3DbNNFreeNN(db, pWInfo->pMemToFree);
+
    pWInfo->pMemToFree = pNext;
  }
+
  sqlite3DbNNFreeNN(db, pWInfo);
}

/*
@@ -154654,10 +158205,11 @@ static void whereLoopOutputAdjust(
        **
        ** 2022-03-24:  Self-culling only applies if either the extra terms
        ** are straight comparison operators that are non-true with NULL
-
        ** operand, or if the loop is not a LEFT JOIN.
+
        ** operand, or if the loop is not an OUTER JOIN.
        */
        if( (pTerm->eOperator & 0x3f)!=0
-
         || (pWC->pWInfo->pTabList->a[pLoop->iTab].fg.jointype & JT_LEFT)==0
+
         || (pWC->pWInfo->pTabList->a[pLoop->iTab].fg.jointype
+
                  & (JT_LEFT|JT_LTORJ))==0
        ){
          pLoop->wsFlags |= WHERE_SELFCULL;
        }
@@ -154863,15 +158415,11 @@ static int whereLoopAddBtreeIndex(
    ** to mix with a lower range bound from some other source */
    if( pTerm->wtFlags & TERM_LIKEOPT && pTerm->eOperator==WO_LT ) continue;

-
    /* tag-20191211-001:  Do not allow constraints from the WHERE clause to
-
    ** be used by the right table of a LEFT JOIN.  Only constraints in the
-
    ** ON clause are allowed.  See tag-20191211-002 for the vtab equivalent. */
-
    if( (pSrc->fg.jointype & JT_LEFT)!=0
-
     && !ExprHasProperty(pTerm->pExpr, EP_FromJoin)
+
    if( (pSrc->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT))!=0
+
     && !constraintCompatibleWithOuterJoin(pTerm,pSrc)
    ){
      continue;
    }
-

    if( IsUniqueIndex(pProbe) && saved_nEq==pProbe->nKeyCol-1 ){
      pBuilder->bldFlags1 |= SQLITE_BLDF1_UNIQUE;
    }else{
@@ -154882,7 +158430,11 @@ static int whereLoopAddBtreeIndex(
    pNew->u.btree.nBtm = saved_nBtm;
    pNew->u.btree.nTop = saved_nTop;
    pNew->nLTerm = saved_nLTerm;
-
    if( whereLoopResize(db, pNew, pNew->nLTerm+1) ) break; /* OOM */
+
    if( pNew->nLTerm>=pNew->nLSlot
+
     && whereLoopResize(db, pNew, pNew->nLTerm+1)
+
    ){
+
       break; /* OOM while trying to enlarge the pNew->aLTerm array */
+
    }
    pNew->aLTerm[pNew->nLTerm++] = pTerm;
    pNew->prereq = (saved_prereq | pTerm->prereqRight) & ~pNew->maskSelf;

@@ -154975,38 +158527,39 @@ static int whereLoopAddBtreeIndex(
      if( scan.iEquiv>1 ) pNew->wsFlags |= WHERE_TRANSCONS;
    }else if( eOp & WO_ISNULL ){
      pNew->wsFlags |= WHERE_COLUMN_NULL;
-
    }else if( eOp & (WO_GT|WO_GE) ){
-
      testcase( eOp & WO_GT );
-
      testcase( eOp & WO_GE );
-
      pNew->wsFlags |= WHERE_COLUMN_RANGE|WHERE_BTM_LIMIT;
-
      pNew->u.btree.nBtm = whereRangeVectorLen(
-
          pParse, pSrc->iCursor, pProbe, saved_nEq, pTerm
-
      );
-
      pBtm = pTerm;
-
      pTop = 0;
-
      if( pTerm->wtFlags & TERM_LIKEOPT ){
-
        /* Range constraints that come from the LIKE optimization are
-
        ** always used in pairs. */
-
        pTop = &pTerm[1];
-
        assert( (pTop-(pTerm->pWC->a))<pTerm->pWC->nTerm );
-
        assert( pTop->wtFlags & TERM_LIKEOPT );
-
        assert( pTop->eOperator==WO_LT );
-
        if( whereLoopResize(db, pNew, pNew->nLTerm+1) ) break; /* OOM */
-
        pNew->aLTerm[pNew->nLTerm++] = pTop;
-
        pNew->wsFlags |= WHERE_TOP_LIMIT;
-
        pNew->u.btree.nTop = 1;
-
      }
-
    }else{
-
      assert( eOp & (WO_LT|WO_LE) );
-
      testcase( eOp & WO_LT );
-
      testcase( eOp & WO_LE );
-
      pNew->wsFlags |= WHERE_COLUMN_RANGE|WHERE_TOP_LIMIT;
-
      pNew->u.btree.nTop = whereRangeVectorLen(
+
    }else{
+
      int nVecLen = whereRangeVectorLen(
          pParse, pSrc->iCursor, pProbe, saved_nEq, pTerm
      );
-
      pTop = pTerm;
-
      pBtm = (pNew->wsFlags & WHERE_BTM_LIMIT)!=0 ?
-
                     pNew->aLTerm[pNew->nLTerm-2] : 0;
+
      if( eOp & (WO_GT|WO_GE) ){
+
        testcase( eOp & WO_GT );
+
        testcase( eOp & WO_GE );
+
        pNew->wsFlags |= WHERE_COLUMN_RANGE|WHERE_BTM_LIMIT;
+
        pNew->u.btree.nBtm = nVecLen;
+
        pBtm = pTerm;
+
        pTop = 0;
+
        if( pTerm->wtFlags & TERM_LIKEOPT ){
+
          /* Range constraints that come from the LIKE optimization are
+
          ** always used in pairs. */
+
          pTop = &pTerm[1];
+
          assert( (pTop-(pTerm->pWC->a))<pTerm->pWC->nTerm );
+
          assert( pTop->wtFlags & TERM_LIKEOPT );
+
          assert( pTop->eOperator==WO_LT );
+
          if( whereLoopResize(db, pNew, pNew->nLTerm+1) ) break; /* OOM */
+
          pNew->aLTerm[pNew->nLTerm++] = pTop;
+
          pNew->wsFlags |= WHERE_TOP_LIMIT;
+
          pNew->u.btree.nTop = 1;
+
        }
+
      }else{
+
        assert( eOp & (WO_LT|WO_LE) );
+
        testcase( eOp & WO_LT );
+
        testcase( eOp & WO_LE );
+
        pNew->wsFlags |= WHERE_COLUMN_RANGE|WHERE_TOP_LIMIT;
+
        pNew->u.btree.nTop = nVecLen;
+
        pTop = pTerm;
+
        pBtm = (pNew->wsFlags & WHERE_BTM_LIMIT)!=0 ?
+
                       pNew->aLTerm[pNew->nLTerm-2] : 0;
+
      }
    }

    /* At this point pNew->nOut is set to the number of rows expected to
@@ -155221,23 +158774,26 @@ static int indexMightHelpWithOrderBy(
*/
static int whereUsablePartialIndex(
  int iTab,             /* The table for which we want an index */
-
  int isLeft,           /* True if iTab is the right table of a LEFT JOIN */
+
  u8 jointype,          /* The JT_* flags on the join */
  WhereClause *pWC,     /* The WHERE clause of the query */
  Expr *pWhere          /* The WHERE clause from the partial index */
){
  int i;
  WhereTerm *pTerm;
-
  Parse *pParse = pWC->pWInfo->pParse;
+
  Parse *pParse;
+

+
  if( jointype & JT_LTORJ ) return 0;
+
  pParse = pWC->pWInfo->pParse;
  while( pWhere->op==TK_AND ){
-
    if( !whereUsablePartialIndex(iTab,isLeft,pWC,pWhere->pLeft) ) return 0;
+
    if( !whereUsablePartialIndex(iTab,jointype,pWC,pWhere->pLeft) ) return 0;
    pWhere = pWhere->pRight;
  }
  if( pParse->db->flags & SQLITE_EnableQPSG ) pParse = 0;
  for(i=0, pTerm=pWC->a; i<pWC->nTerm; i++, pTerm++){
    Expr *pExpr;
    pExpr = pTerm->pExpr;
-
    if( (!ExprHasProperty(pExpr, EP_FromJoin) || pExpr->w.iRightJoinTable==iTab)
-
     && (isLeft==0 || ExprHasProperty(pExpr, EP_FromJoin))
+
    if( (!ExprHasProperty(pExpr, EP_OuterON) || pExpr->w.iJoin==iTab)
+
     && ((jointype & JT_OUTER)==0 || ExprHasProperty(pExpr, EP_OuterON))
     && sqlite3ExprImpliesExpr(pParse, pExpr, pWhere, iTab)
     && (pTerm->wtFlags & TERM_VNULL)==0
    ){
@@ -155248,6 +158804,94 @@ static int whereUsablePartialIndex(
}

/*
+
** Structure passed to the whereIsCoveringIndex Walker callback.
+
*/
+
struct CoveringIndexCheck {
+
  Index *pIdx;       /* The index */
+
  int iTabCur;       /* Cursor number for the corresponding table */
+
};
+

+
/*
+
** Information passed in is pWalk->u.pCovIdxCk.  Call is pCk.
+
**
+
** If the Expr node references the table with cursor pCk->iTabCur, then
+
** make sure that column is covered by the index pCk->pIdx.  We know that
+
** all columns less than 63 (really BMS-1) are covered, so we don't need
+
** to check them.  But we do need to check any column at 63 or greater.
+
**
+
** If the index does not cover the column, then set pWalk->eCode to
+
** non-zero and return WRC_Abort to stop the search.
+
**
+
** If this node does not disprove that the index can be a covering index,
+
** then just return WRC_Continue, to continue the search.
+
*/
+
static int whereIsCoveringIndexWalkCallback(Walker *pWalk, Expr *pExpr){
+
  int i;                  /* Loop counter */
+
  const Index *pIdx;      /* The index of interest */
+
  const i16 *aiColumn;    /* Columns contained in the index */
+
  u16 nColumn;            /* Number of columns in the index */
+
  if( pExpr->op!=TK_COLUMN && pExpr->op!=TK_AGG_COLUMN ) return WRC_Continue;
+
  if( pExpr->iColumn<(BMS-1) ) return WRC_Continue;
+
  if( pExpr->iTable!=pWalk->u.pCovIdxCk->iTabCur ) return WRC_Continue;
+
  pIdx = pWalk->u.pCovIdxCk->pIdx;
+
  aiColumn = pIdx->aiColumn;
+
  nColumn = pIdx->nColumn;
+
  for(i=0; i<nColumn; i++){
+
    if( aiColumn[i]==pExpr->iColumn ) return WRC_Continue;
+
  }
+
  pWalk->eCode = 1;
+
  return WRC_Abort;
+
}
+

+

+
/*
+
** pIdx is an index that covers all of the low-number columns used by
+
** pWInfo->pSelect (columns from 0 through 62).  But there are columns
+
** in pWInfo->pSelect beyond 62.  This routine tries to answer the question
+
** of whether pIdx covers *all* columns in the query.
+
**
+
** Return 0 if pIdx is a covering index.   Return non-zero if pIdx is
+
** not a covering index or if we are unable to determine if pIdx is a
+
** covering index.
+
**
+
** This routine is an optimization.  It is always safe to return non-zero.
+
** But returning zero when non-zero should have been returned can lead to
+
** incorrect bytecode and assertion faults.
+
*/
+
static SQLITE_NOINLINE u32 whereIsCoveringIndex(
+
  WhereInfo *pWInfo,     /* The WHERE clause context */
+
  Index *pIdx,           /* Index that is being tested */
+
  int iTabCur            /* Cursor for the table being indexed */
+
){
+
  int i;
+
  struct CoveringIndexCheck ck;
+
  Walker w;
+
  if( pWInfo->pSelect==0 ){
+
    /* We don't have access to the full query, so we cannot check to see
+
    ** if pIdx is covering.  Assume it is not. */
+
    return 1;
+
  }
+
  for(i=0; i<pIdx->nColumn; i++){
+
    if( pIdx->aiColumn[i]>=BMS-1 ) break;
+
  }
+
  if( i>=pIdx->nColumn ){
+
    /* pIdx does not index any columns greater than 62, but we know from
+
    ** colMask that columns greater than 62 are used, so this is not a
+
    ** covering index */
+
    return 1;
+
  }
+
  ck.pIdx = pIdx;
+
  ck.iTabCur = iTabCur;
+
  memset(&w, 0, sizeof(w));
+
  w.xExprCallback = whereIsCoveringIndexWalkCallback;
+
  w.xSelectCallback = sqlite3SelectWalkNoop;
+
  w.u.pCovIdxCk = &ck;
+
  w.eCode = 0;
+
  sqlite3WalkSelect(&w, pWInfo->pSelect);
+
  return w.eCode;
+
}
+

+
/*
** Add all WhereLoop objects for a single table of the join where the table
** is identified by pBuilder->pNew->iTab.  That table is guaranteed to be
** a b-tree table, not a virtual table.
@@ -155346,13 +158990,14 @@ static int whereLoopAddBtree(
#ifndef SQLITE_OMIT_AUTOMATIC_INDEX
  /* Automatic indexes */
  if( !pBuilder->pOrSet      /* Not part of an OR optimization */
-
   && (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)==0
+
   && (pWInfo->wctrlFlags & (WHERE_RIGHT_JOIN|WHERE_OR_SUBCLAUSE))==0
   && (pWInfo->pParse->db->flags & SQLITE_AutoIndex)!=0
   && !pSrc->fg.isIndexedBy  /* Has no INDEXED BY clause */
   && !pSrc->fg.notIndexed   /* Has no NOT INDEXED clause */
   && HasRowid(pTab)         /* Not WITHOUT ROWID table. (FIXME: Why not?) */
   && !pSrc->fg.isCorrelated /* Not a correlated subquery */
   && !pSrc->fg.isRecursive  /* Not a recursive common table expression. */
+
   && (pSrc->fg.jointype & JT_RIGHT)==0 /* Not the right tab of a RIGHT JOIN */
  ){
    /* Generate auto-index WhereLoops */
    LogEst rLogSize;         /* Logarithm of the number of rows in the table */
@@ -155402,9 +159047,8 @@ static int whereLoopAddBtree(
  for(; rc==SQLITE_OK && pProbe;
      pProbe=(pSrc->fg.isIndexedBy ? 0 : pProbe->pNext), iSortIdx++
  ){
-
    int isLeft = (pSrc->fg.jointype & JT_OUTER)!=0;
    if( pProbe->pPartIdxWhere!=0
-
     && !whereUsablePartialIndex(pSrc->iCursor, isLeft, pWC,
+
     && !whereUsablePartialIndex(pSrc->iCursor, pSrc->fg.jointype, pWC,
                                 pProbe->pPartIdxWhere)
    ){
      testcase( pNew->iTab!=pSrc->iCursor );  /* See ticket [98d973b8f5] */
@@ -155449,6 +159093,9 @@ static int whereLoopAddBtree(
#else
      pNew->rRun = rSize + 16;
#endif
+
      if( IsView(pTab) || (pTab->tabFlags & TF_Ephemeral)!=0 ){
+
        pNew->wsFlags |= WHERE_VIEWSCAN;
+
      }
      ApplyCostMultiplier(pNew->rRun, pTab->costMult);
      whereLoopOutputAdjust(pWC, pNew, rSize);
      rc = whereLoopInsert(pBuilder, pNew);
@@ -155461,6 +159108,9 @@ static int whereLoopAddBtree(
        m = 0;
      }else{
        m = pSrc->colUsed & pProbe->colNotIdxed;
+
        if( m==TOPBIT ){
+
          m = whereIsCoveringIndex(pWInfo, pProbe, pSrc->iCursor);
+
        }
        pNew->wsFlags = (m==0) ? (WHERE_IDX_ONLY|WHERE_INDEXED) : WHERE_INDEXED;
      }

@@ -155512,7 +159162,14 @@ static int whereLoopAddBtree(
        }
        ApplyCostMultiplier(pNew->rRun, pTab->costMult);
        whereLoopOutputAdjust(pWC, pNew, rSize);
-
        rc = whereLoopInsert(pBuilder, pNew);
+
        if( (pSrc->fg.jointype & JT_RIGHT)!=0 && pProbe->aColExpr ){
+
          /* Do not do an SCAN of a index-on-expression in a RIGHT JOIN
+
          ** because the cursor used to access the index might not be
+
          ** positioned to the correct row during the right-join no-match
+
          ** loop. */
+
        }else{
+
          rc = whereLoopInsert(pBuilder, pNew);
+
        }
        pNew->nOut = rSize;
        if( rc ) break;
      }
@@ -155687,6 +159344,7 @@ static int whereLoopAddVirtualOne(
        *pbIn = 1; assert( (mExclude & WO_IN)==0 );
      }

+
      assert( pbRetryLimit || !isLimitTerm(pTerm) );
      if( isLimitTerm(pTerm) && *pbIn ){
        /* If there is an IN(...) term handled as an == (separate call to
        ** xFilter for each value on the RHS of the IN) and a LIMIT or
@@ -155834,9 +159492,7 @@ SQLITE_API int sqlite3_vtab_rhs_value(
*/
SQLITE_API int sqlite3_vtab_distinct(sqlite3_index_info *pIdxInfo){
  HiddenIndexInfo *pHidden = (HiddenIndexInfo*)&pIdxInfo[1];
-
  assert( pHidden->eDistinct==0
-
       || pHidden->eDistinct==1
-
       || pHidden->eDistinct==2 );
+
  assert( pHidden->eDistinct>=0 && pHidden->eDistinct<=3 );
  return pHidden->eDistinct;
}

@@ -155844,15 +159500,26 @@ SQLITE_API int sqlite3_vtab_distinct(sqlite3_index_info *pIdxInfo){
    && !defined(SQLITE_OMIT_VIRTUALTABLE)
/*
** Cause the prepared statement that is associated with a call to
-
** xBestIndex to open write transactions on all attached schemas.
+
** xBestIndex to potentiall use all schemas.  If the statement being
+
** prepared is read-only, then just start read transactions on all
+
** schemas.  But if this is a write operation, start writes on all
+
** schemas.
+
**
** This is used by the (built-in) sqlite_dbpage virtual table.
*/
-
SQLITE_PRIVATE void sqlite3VtabWriteAll(sqlite3_index_info *pIdxInfo){
+
SQLITE_PRIVATE void sqlite3VtabUsesAllSchemas(sqlite3_index_info *pIdxInfo){
  HiddenIndexInfo *pHidden = (HiddenIndexInfo*)&pIdxInfo[1];
  Parse *pParse = pHidden->pParse;
  int nDb = pParse->db->nDb;
  int i;
-
  for(i=0; i<nDb; i++) sqlite3BeginWriteOperation(pParse, 0, i);
+
  for(i=0; i<nDb; i++){
+
    sqlite3CodeVerifySchema(pParse, i);
+
  }
+
  if( pParse->writeMask ){
+
    for(i=0; i<nDb; i++){
+
      sqlite3BeginWriteOperation(pParse, 0, i);
+
    }
+
  }
}
#endif

@@ -156035,6 +159702,9 @@ static int whereLoopAddOr(
  pItem = pWInfo->pTabList->a + pNew->iTab;
  iCur = pItem->iCursor;

+
  /* The multi-index OR optimization does not work for RIGHT and FULL JOIN */
+
  if( pItem->fg.jointype & JT_RIGHT ) return SQLITE_OK;
+

  for(pTerm=pWC->a; pTerm<pWCEnd && rc==SQLITE_OK; pTerm++){
    if( (pTerm->eOperator & WO_OR)!=0
     && (pTerm->u.pOrInfo->indexable & pNew->maskSelf)!=0
@@ -156148,29 +159818,50 @@ static int whereLoopAddAll(WhereLoopBuilder *pBuilder){
  SrcItem *pEnd = &pTabList->a[pWInfo->nLevel];
  sqlite3 *db = pWInfo->pParse->db;
  int rc = SQLITE_OK;
+
  int bFirstPastRJ = 0;
+
  int hasRightJoin = 0;
  WhereLoop *pNew;

+

  /* Loop over the tables in the join, from left to right */
  pNew = pBuilder->pNew;
-
  whereLoopInit(pNew);
+

+
  /* Verify that pNew has already been initialized */
+
  assert( pNew->nLTerm==0 );
+
  assert( pNew->wsFlags==0 );
+
  assert( pNew->nLSlot>=ArraySize(pNew->aLTermSpace) );
+
  assert( pNew->aLTerm!=0 );
+

  pBuilder->iPlanLimit = SQLITE_QUERY_PLANNER_LIMIT;
  for(iTab=0, pItem=pTabList->a; pItem<pEnd; iTab++, pItem++){
    Bitmask mUnusable = 0;
    pNew->iTab = iTab;
    pBuilder->iPlanLimit += SQLITE_QUERY_PLANNER_LIMIT_INCR;
    pNew->maskSelf = sqlite3WhereGetMask(&pWInfo->sMaskSet, pItem->iCursor);
-
    if( (pItem->fg.jointype & (JT_LEFT|JT_CROSS))!=0 ){
-
      /* This condition is true when pItem is the FROM clause term on the
-
      ** right-hand-side of a LEFT or CROSS JOIN.  */
-
      mPrereq = mPrior;
-
    }else{
+
    if( bFirstPastRJ
+
     || (pItem->fg.jointype & (JT_OUTER|JT_CROSS|JT_LTORJ))!=0
+
    ){
+
      /* Add prerequisites to prevent reordering of FROM clause terms
+
      ** across CROSS joins and outer joins.  The bFirstPastRJ boolean
+
      ** prevents the right operand of a RIGHT JOIN from being swapped with
+
      ** other elements even further to the right.
+
      **
+
      ** The JT_LTORJ case and the hasRightJoin flag work together to
+
      ** prevent FROM-clause terms from moving from the right side of
+
      ** a LEFT JOIN over to the left side of that join if the LEFT JOIN
+
      ** is itself on the left side of a RIGHT JOIN.
+
      */
+
      if( pItem->fg.jointype & JT_LTORJ ) hasRightJoin = 1;
+
      mPrereq |= mPrior;
+
      bFirstPastRJ = (pItem->fg.jointype & JT_RIGHT)!=0;
+
    }else if( !hasRightJoin ){
      mPrereq = 0;
    }
#ifndef SQLITE_OMIT_VIRTUALTABLE
    if( IsVirtual(pItem->pTab) ){
      SrcItem *p;
      for(p=&pItem[1]; p<pEnd; p++){
-
        if( mUnusable || (p->fg.jointype & (JT_LEFT|JT_CROSS)) ){
+
        if( mUnusable || (p->fg.jointype & (JT_OUTER|JT_CROSS)) ){
          mUnusable |= sqlite3WhereGetMask(&pWInfo->sMaskSet, p->iCursor);
        }
      }
@@ -156295,7 +159986,9 @@ static i8 wherePathSatisfiesOrderBy(
      pLoop = pLast;
    }
    if( pLoop->wsFlags & WHERE_VIRTUALTABLE ){
-
      if( pLoop->u.vtab.isOrdered && (wctrlFlags & WHERE_DISTINCTBY)==0 ){
+
      if( pLoop->u.vtab.isOrdered
+
       && ((wctrlFlags&(WHERE_DISTINCTBY|WHERE_SORTBYGROUP))!=WHERE_DISTINCTBY)
+
      ){
        obSat = obDone;
      }
      break;
@@ -156473,16 +160166,18 @@ static i8 wherePathSatisfiesOrderBy(
          /* Make sure the sort order is compatible in an ORDER BY clause.
          ** Sort order is irrelevant for a GROUP BY clause. */
          if( revSet ){
-
            if( (rev ^ revIdx)!=(pOrderBy->a[i].sortFlags&KEYINFO_ORDER_DESC) ){
+
            if( (rev ^ revIdx)
+
                           != (pOrderBy->a[i].fg.sortFlags&KEYINFO_ORDER_DESC)
+
            ){
              isMatch = 0;
            }
          }else{
-
            rev = revIdx ^ (pOrderBy->a[i].sortFlags & KEYINFO_ORDER_DESC);
+
            rev = revIdx ^ (pOrderBy->a[i].fg.sortFlags & KEYINFO_ORDER_DESC);
            if( rev ) *pRevMask |= MASKBIT(iLoop);
            revSet = 1;
          }
        }
-
        if( isMatch && (pOrderBy->a[i].sortFlags & KEYINFO_ORDER_BIGNULL) ){
+
        if( isMatch && (pOrderBy->a[i].fg.sortFlags & KEYINFO_ORDER_BIGNULL) ){
          if( j==pLoop->u.btree.nEq ){
            pLoop->wsFlags |= WHERE_BIGNULL_SORT;
          }else{
@@ -156562,7 +160257,7 @@ static i8 wherePathSatisfiesOrderBy(
**   SELECT * FROM t1 GROUP BY y,x ORDER BY y,x;   -- IsSorted()==0
*/
SQLITE_PRIVATE int sqlite3WhereIsSorted(WhereInfo *pWInfo){
-
  assert( pWInfo->wctrlFlags & WHERE_GROUPBY );
+
  assert( pWInfo->wctrlFlags & (WHERE_GROUPBY|WHERE_DISTINCTBY) );
  assert( pWInfo->wctrlFlags & WHERE_SORTBYGROUP );
  return pWInfo->sorted;
}
@@ -156641,7 +160336,6 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
  int mxChoice;             /* Maximum number of simultaneous paths tracked */
  int nLoop;                /* Number of terms in the join */
  Parse *pParse;            /* Parsing context */
-
  sqlite3 *db;              /* The database connection */
  int iLoop;                /* Loop counter over the terms of the join */
  int ii, jj;               /* Loop counters */
  int mxI = 0;              /* Index of next entry to replace */
@@ -156660,7 +160354,6 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
  int nSpace;               /* Bytes of space allocated at pSpace */

  pParse = pWInfo->pParse;
-
  db = pParse->db;
  nLoop = pWInfo->nLevel;
  /* TUNING: For simple queries, only the best path is tracked.
  ** For 2-way joins, the 5 best paths are followed.
@@ -156683,7 +160376,7 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
  /* Allocate and initialize space for aTo, aFrom and aSortCost[] */
  nSpace = (sizeof(WherePath)+sizeof(WhereLoop*)*nLoop)*mxChoice*2;
  nSpace += sizeof(LogEst) * nOrderBy;
-
  pSpace = sqlite3DbMallocRawNN(db, nSpace);
+
  pSpace = sqlite3StackAllocRawNN(pParse->db, nSpace);
  if( pSpace==0 ) return SQLITE_NOMEM_BKPT;
  aTo = (WherePath*)pSpace;
  aFrom = aTo+mxChoice;
@@ -156733,9 +160426,9 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
        LogEst nOut;                      /* Rows visited by (pFrom+pWLoop) */
        LogEst rCost;                     /* Cost of path (pFrom+pWLoop) */
        LogEst rUnsorted;                 /* Unsorted cost of (pFrom+pWLoop) */
-
        i8 isOrdered = pFrom->isOrdered;  /* isOrdered for (pFrom+pWLoop) */
+
        i8 isOrdered;                     /* isOrdered for (pFrom+pWLoop) */
        Bitmask maskNew;                  /* Mask of src visited by (..) */
-
        Bitmask revMask = 0;              /* Mask of rev-order loops for (..) */
+
        Bitmask revMask;                  /* Mask of rev-order loops for (..) */

        if( (pWLoop->prereq & ~pFrom->maskLoop)!=0 ) continue;
        if( (pWLoop->maskSelf & pFrom->maskLoop)!=0 ) continue;
@@ -156754,7 +160447,9 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
        rUnsorted = sqlite3LogEstAdd(rUnsorted, pFrom->rUnsorted);
        nOut = pFrom->nRow + pWLoop->nOut;
        maskNew = pFrom->maskLoop | pWLoop->maskSelf;
+
        isOrdered = pFrom->isOrdered;
        if( isOrdered<0 ){
+
          revMask = 0;
          isOrdered = wherePathSatisfiesOrderBy(pWInfo,
                       pWInfo->pOrderBy, pFrom, pWInfo->wctrlFlags,
                       iLoop, pWLoop, &revMask);
@@ -156782,6 +160477,13 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
          rUnsorted -= 2;  /* TUNING:  Slight bias in favor of no-sort plans */
        }

+
        /* TUNING:  A full-scan of a VIEW or subquery in the outer loop
+
        ** is not so bad. */
+
        if( iLoop==0 && (pWLoop->wsFlags & WHERE_VIEWSCAN)!=0 ){
+
          rCost += -10;
+
          nOut += -30;
+
        }
+

        /* Check to see if pWLoop should be added to the set of
        ** mxChoice best-so-far paths.
        **
@@ -156932,7 +160634,7 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){

  if( nFrom==0 ){
    sqlite3ErrorMsg(pParse, "no query solution");
-
    sqlite3DbFreeNN(db, pSpace);
+
    sqlite3StackFreeNN(pParse->db, pSpace);
    return SQLITE_ERROR;
  }

@@ -156963,12 +160665,12 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
  }
  pWInfo->bOrderedInnerLoop = 0;
  if( pWInfo->pOrderBy ){
+
    pWInfo->nOBSat = pFrom->isOrdered;
    if( pWInfo->wctrlFlags & WHERE_DISTINCTBY ){
      if( pFrom->isOrdered==pWInfo->pOrderBy->nExpr ){
        pWInfo->eDistinct = WHERE_DISTINCT_ORDERED;
      }
    }else{
-
      pWInfo->nOBSat = pFrom->isOrdered;
      pWInfo->revMask = pFrom->revLoop;
      if( pWInfo->nOBSat<=0 ){
        pWInfo->nOBSat = 0;
@@ -157014,7 +160716,7 @@ static int wherePathSolver(WhereInfo *pWInfo, LogEst nRowEst){
  pWInfo->nRowOut = pFrom->nRow;

  /* Free temporary memory and return success */
-
  sqlite3DbFreeNN(db, pSpace);
+
  sqlite3StackFreeNN(pParse->db, pSpace);
  return SQLITE_OK;
}

@@ -157047,7 +160749,11 @@ static int whereShortCut(WhereLoopBuilder *pBuilder){
  pItem = pWInfo->pTabList->a;
  pTab = pItem->pTab;
  if( IsVirtual(pTab) ) return 0;
-
  if( pItem->fg.isIndexedBy ) return 0;
+
  if( pItem->fg.isIndexedBy || pItem->fg.notIndexed ){
+
    testcase( pItem->fg.isIndexedBy );
+
    testcase( pItem->fg.notIndexed );
+
    return 0;
+
  }
  iCur = pItem->iCursor;
  pWC = &pWInfo->sWC;
  pLoop = pBuilder->pNew;
@@ -157220,7 +160926,7 @@ static SQLITE_NOINLINE Bitmask whereOmitNoopJoin(
    WhereLoop *pLoop;
    pLoop = pWInfo->a[i].pWLoop;
    pItem = &pWInfo->pTabList->a[pLoop->iTab];
-
    if( (pItem->fg.jointype & JT_LEFT)==0 ) continue;
+
    if( (pItem->fg.jointype & (JT_LEFT|JT_RIGHT))!=JT_LEFT ) continue;
    if( (pWInfo->wctrlFlags & WHERE_WANT_DISTINCT)==0
     && (pLoop->wsFlags & WHERE_ONEROW)==0
    ){
@@ -157230,8 +160936,8 @@ static SQLITE_NOINLINE Bitmask whereOmitNoopJoin(
    pEnd = pWInfo->sWC.a + pWInfo->sWC.nTerm;
    for(pTerm=pWInfo->sWC.a; pTerm<pEnd; pTerm++){
      if( (pTerm->prereqAll & pLoop->maskSelf)!=0 ){
-
        if( !ExprHasProperty(pTerm->pExpr, EP_FromJoin)
-
         || pTerm->pExpr->w.iRightJoinTable!=pItem->iCursor
+
        if( !ExprHasProperty(pTerm->pExpr, EP_OuterON)
+
         || pTerm->pExpr->w.iJoin!=pItem->iCursor
        ){
          break;
        }
@@ -157310,6 +161016,77 @@ static SQLITE_NOINLINE void whereCheckIfBloomFilterIsUseful(
}

/*
+
** This is an sqlite3ParserAddCleanup() callback that is invoked to
+
** free the Parse->pIdxExpr list when the Parse object is destroyed.
+
*/
+
static void whereIndexedExprCleanup(sqlite3 *db, void *pObject){
+
  Parse *pParse = (Parse*)pObject;
+
  while( pParse->pIdxExpr!=0 ){
+
    IndexedExpr *p = pParse->pIdxExpr;
+
    pParse->pIdxExpr = p->pIENext;
+
    sqlite3ExprDelete(db, p->pExpr);
+
    sqlite3DbFreeNN(db, p);
+
  }
+
}
+

+
/*
+
** The index pIdx is used by a query and contains one or more expressions.
+
** In other words pIdx is an index on an expression.  iIdxCur is the cursor
+
** number for the index and iDataCur is the cursor number for the corresponding
+
** table.
+
**
+
** This routine adds IndexedExpr entries to the Parse->pIdxExpr field for
+
** each of the expressions in the index so that the expression code generator
+
** will know to replace occurrences of the indexed expression with
+
** references to the corresponding column of the index.
+
*/
+
static SQLITE_NOINLINE void whereAddIndexedExpr(
+
  Parse *pParse,     /* Add IndexedExpr entries to pParse->pIdxExpr */
+
  Index *pIdx,       /* The index-on-expression that contains the expressions */
+
  int iIdxCur,       /* Cursor number for pIdx */
+
  SrcItem *pTabItem  /* The FROM clause entry for the table */
+
){
+
  int i;
+
  IndexedExpr *p;
+
  Table *pTab;
+
  assert( pIdx->bHasExpr );
+
  pTab = pIdx->pTable;
+
  for(i=0; i<pIdx->nColumn; i++){
+
    Expr *pExpr;
+
    int j = pIdx->aiColumn[i];
+
    int bMaybeNullRow;
+
    if( j==XN_EXPR ){
+
      pExpr = pIdx->aColExpr->a[i].pExpr;
+
      testcase( pTabItem->fg.jointype & JT_LEFT );
+
      testcase( pTabItem->fg.jointype & JT_RIGHT );
+
      testcase( pTabItem->fg.jointype & JT_LTORJ );
+
      bMaybeNullRow = (pTabItem->fg.jointype & (JT_LEFT|JT_LTORJ|JT_RIGHT))!=0;
+
    }else if( j>=0 && (pTab->aCol[j].colFlags & COLFLAG_VIRTUAL)!=0 ){
+
      pExpr = sqlite3ColumnExpr(pTab, &pTab->aCol[j]);
+
      bMaybeNullRow = 0;
+
    }else{
+
      continue;
+
    }
+
    if( sqlite3ExprIsConstant(pExpr) ) continue;
+
    p = sqlite3DbMallocRaw(pParse->db,  sizeof(IndexedExpr));
+
    if( p==0 ) break;
+
    p->pIENext = pParse->pIdxExpr;
+
    p->pExpr = sqlite3ExprDup(pParse->db, pExpr, 0);
+
    p->iDataCur = pTabItem->iCursor;
+
    p->iIdxCur = iIdxCur;
+
    p->iIdxCol = i;
+
    p->bMaybeNullRow = bMaybeNullRow;
+
#ifdef SQLITE_ENABLE_EXPLAIN_COMMENTS
+
    p->zIdxName = pIdx->zName;
+
#endif
+
    pParse->pIdxExpr = p;
+
    if( p->pIENext==0 ){
+
      sqlite3ParserAddCleanup(pParse, whereIndexedExprCleanup, pParse);
+
    }
+
  }
+
}
+

+
/*
** Generate the beginning of the loop used for WHERE clause processing.
** The return value is a pointer to an opaque structure that contains
** information needed to terminate the loop.  Later, the calling routine
@@ -157403,7 +161180,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
  Expr *pWhere,           /* The WHERE clause */
  ExprList *pOrderBy,     /* An ORDER BY (or GROUP BY) clause, or NULL */
  ExprList *pResultSet,   /* Query result set.  Req'd for DISTINCT */
-
  Select *pLimit,         /* Use this LIMIT/OFFSET clause, if any */
+
  Select *pSelect,        /* The entire SELECT statement */
  u16 wctrlFlags,         /* The WHERE_* flags defined in sqliteInt.h */
  int iAuxArg             /* If WHERE_OR_SUBCLAUSE is set, index cursor number
                          ** If WHERE_USE_LIMIT, then the limit amount */
@@ -157462,7 +161239,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
  ** field (type Bitmask) it must be aligned on an 8-byte boundary on
  ** some architectures. Hence the ROUND8() below.
  */
-
  nByteWInfo = ROUND8(sizeof(WhereInfo)+(nTabList-1)*sizeof(WhereLevel));
+
  nByteWInfo = ROUND8P(sizeof(WhereInfo)+(nTabList-1)*sizeof(WhereLevel));
  pWInfo = sqlite3DbMallocRawNN(db, nByteWInfo + sizeof(WhereLoop));
  if( db->mallocFailed ){
    sqlite3DbFree(db, pWInfo);
@@ -157472,7 +161249,9 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
  pWInfo->pParse = pParse;
  pWInfo->pTabList = pTabList;
  pWInfo->pOrderBy = pOrderBy;
+
#if WHERETRACE_ENABLED
  pWInfo->pWhere = pWhere;
+
#endif
  pWInfo->pResultSet = pResultSet;
  pWInfo->aiCurOnePass[0] = pWInfo->aiCurOnePass[1] = -1;
  pWInfo->nLevel = nTabList;
@@ -157480,9 +161259,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
  pWInfo->wctrlFlags = wctrlFlags;
  pWInfo->iLimit = iAuxArg;
  pWInfo->savedNQueryLoop = pParse->nQueryLoop;
-
#ifndef SQLITE_OMIT_VIRTUALTABLE
-
  pWInfo->pLimit = pLimit;
-
#endif
+
  pWInfo->pSelect = pSelect;
  memset(&pWInfo->nOBSat, 0,
         offsetof(WhereInfo,sWC) - offsetof(WhereInfo,nOBSat));
  memset(&pWInfo->a[0], 0, sizeof(WhereLoop)+nTabList*sizeof(WhereLevel));
@@ -157551,8 +161328,10 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(

  /* Analyze all of the subexpressions. */
  sqlite3WhereExprAnalyze(pTabList, &pWInfo->sWC);
-
  sqlite3WhereAddLimit(&pWInfo->sWC, pLimit);
-
  if( db->mallocFailed ) goto whereBeginError;
+
  if( pSelect && pSelect->pLimit ){
+
    sqlite3WhereAddLimit(&pWInfo->sWC, pSelect);
+
  }
+
  if( pParse->nErr ) goto whereBeginError;

  /* Special case: WHERE terms that do not refer to any tables in the join
  ** (constant expressions). Evaluate each such term, and jump over all the
@@ -157784,8 +161563,10 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
      /* noop */
    }else
#endif
-
    if( (pLoop->wsFlags & WHERE_IDX_ONLY)==0
-
         && (wctrlFlags & WHERE_OR_SUBCLAUSE)==0 ){
+
    if( ((pLoop->wsFlags & WHERE_IDX_ONLY)==0
+
         && (wctrlFlags & WHERE_OR_SUBCLAUSE)==0)
+
     || (pTabItem->fg.jointype & (JT_LTORJ|JT_RIGHT))!=0
+
    ){
      int op = OP_OpenRead;
      if( pWInfo->eOnePass!=ONEPASS_OFF ){
        op = OP_OpenWrite;
@@ -157852,8 +161633,12 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
        op = OP_ReopenIdx;
      }else{
        iIndexCur = pParse->nTab++;
+
        if( pIx->bHasExpr && OptimizationEnabled(db, SQLITE_IndexedExpr) ){
+
          whereAddIndexedExpr(pParse, pIx, iIndexCur, pTabItem);
+
        }
      }
      pLevel->iIdxCur = iIndexCur;
+
      assert( pIx!=0 );
      assert( pIx->pSchema==pTab->pSchema );
      assert( iIndexCur>=0 );
      if( op ){
@@ -157887,6 +161672,37 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
      }
    }
    if( iDb>=0 ) sqlite3CodeVerifySchema(pParse, iDb);
+
    if( (pTabItem->fg.jointype & JT_RIGHT)!=0
+
     && (pLevel->pRJ = sqlite3WhereMalloc(pWInfo, sizeof(WhereRightJoin)))!=0
+
    ){
+
      WhereRightJoin *pRJ = pLevel->pRJ;
+
      pRJ->iMatch = pParse->nTab++;
+
      pRJ->regBloom = ++pParse->nMem;
+
      sqlite3VdbeAddOp2(v, OP_Blob, 65536, pRJ->regBloom);
+
      pRJ->regReturn = ++pParse->nMem;
+
      sqlite3VdbeAddOp2(v, OP_Null, 0, pRJ->regReturn);
+
      assert( pTab==pTabItem->pTab );
+
      if( HasRowid(pTab) ){
+
        KeyInfo *pInfo;
+
        sqlite3VdbeAddOp2(v, OP_OpenEphemeral, pRJ->iMatch, 1);
+
        pInfo = sqlite3KeyInfoAlloc(pParse->db, 1, 0);
+
        if( pInfo ){
+
          pInfo->aColl[0] = 0;
+
          pInfo->aSortFlags[0] = 0;
+
          sqlite3VdbeAppendP4(v, pInfo, P4_KEYINFO);
+
        }
+
      }else{
+
        Index *pPk = sqlite3PrimaryKeyIndex(pTab);
+
        sqlite3VdbeAddOp2(v, OP_OpenEphemeral, pRJ->iMatch, pPk->nKeyCol);
+
        sqlite3VdbeSetP4KeyInfo(pParse, pPk);
+
      }
+
      pLoop->wsFlags &= ~WHERE_IDX_ONLY;
+
      /* The nature of RIGHT JOIN processing is such that it messes up
+
      ** the output order.  So omit any ORDER BY/GROUP BY elimination
+
      ** optimizations.  We need to do an actual sort for RIGHT JOIN. */
+
      pWInfo->nOBSat = 0;
+
      pWInfo->eDistinct = WHERE_DISTINCT_UNORDERED;
+
    }
  }
  pWInfo->iTop = sqlite3VdbeCurrentAddr(v);
  if( db->mallocFailed ) goto whereBeginError;
@@ -157898,9 +161714,20 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
  for(ii=0; ii<nTabList; ii++){
    int addrExplain;
    int wsFlags;
+
    SrcItem *pSrc;
    if( pParse->nErr ) goto whereBeginError;
    pLevel = &pWInfo->a[ii];
    wsFlags = pLevel->pWLoop->wsFlags;
+
    pSrc = &pTabList->a[pLevel->iFrom];
+
    if( pSrc->fg.isMaterialized ){
+
      if( pSrc->fg.isCorrelated ){
+
        sqlite3VdbeAddOp2(v, OP_Gosub, pSrc->regReturn, pSrc->addrFillSub);
+
      }else{
+
        int iOnce = sqlite3VdbeAddOp0(v, OP_Once);  VdbeCoverage(v);
+
        sqlite3VdbeAddOp2(v, OP_Gosub, pSrc->regReturn, pSrc->addrFillSub);
+
        sqlite3VdbeJumpHere(v, iOnce);
+
      }
+
    }
    if( (wsFlags & (WHERE_AUTO_INDEX|WHERE_BLOOMFILTER))!=0 ){
      if( (wsFlags & WHERE_AUTO_INDEX)!=0 ){
#ifndef SQLITE_OMIT_AUTOMATIC_INDEX
@@ -157931,8 +161758,6 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
  /* Jump here if malloc fails */
whereBeginError:
  if( pWInfo ){
-
    testcase( pWInfo->pExprMods!=0 );
-
    whereUndoExprMods(pWInfo);
    pParse->nQueryLoop = pWInfo->savedNQueryLoop;
    whereInfoFree(db, pWInfo);
  }
@@ -157992,6 +161817,7 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
  SrcList *pTabList = pWInfo->pTabList;
  sqlite3 *db = pParse->db;
  int iEnd = sqlite3VdbeCurrentAddr(v);
+
  int nRJ = 0;

  /* Generate loop termination code.
  */
@@ -157999,6 +161825,17 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
  for(i=pWInfo->nLevel-1; i>=0; i--){
    int addr;
    pLevel = &pWInfo->a[i];
+
    if( pLevel->pRJ ){
+
      /* Terminate the subroutine that forms the interior of the loop of
+
      ** the RIGHT JOIN table */
+
      WhereRightJoin *pRJ = pLevel->pRJ;
+
      sqlite3VdbeResolveLabel(v, pLevel->addrCont);
+
      pLevel->addrCont = 0;
+
      pRJ->endSubrtn = sqlite3VdbeCurrentAddr(v);
+
      sqlite3VdbeAddOp3(v, OP_Return, pRJ->regReturn, pRJ->addrSubrtn, 1);
+
      VdbeCoverage(v);
+
      nRJ++;
+
    }
    pLoop = pLevel->pWLoop;
    if( pLevel->op!=OP_Noop ){
#ifndef SQLITE_DISABLE_SKIPAHEAD_DISTINCT
@@ -158026,7 +161863,7 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
      }
#endif /* SQLITE_DISABLE_SKIPAHEAD_DISTINCT */
      /* The common case: Advance to the next row */
-
      sqlite3VdbeResolveLabel(v, pLevel->addrCont);
+
      if( pLevel->addrCont ) sqlite3VdbeResolveLabel(v, pLevel->addrCont);
      sqlite3VdbeAddOp3(v, pLevel->op, pLevel->p1, pLevel->p2, pLevel->p3);
      sqlite3VdbeChangeP5(v, pLevel->p5);
      VdbeCoverage(v);
@@ -158041,7 +161878,7 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
#ifndef SQLITE_DISABLE_SKIPAHEAD_DISTINCT
      if( addrSeek ) sqlite3VdbeJumpHere(v, addrSeek);
#endif
-
    }else{
+
    }else if( pLevel->addrCont ){
      sqlite3VdbeResolveLabel(v, pLevel->addrCont);
    }
    if( (pLoop->wsFlags & WHERE_IN_ABLE)!=0 && pLevel->u.in.nIn>0 ){
@@ -158091,6 +161928,10 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
      }
    }
    sqlite3VdbeResolveLabel(v, pLevel->addrBrk);
+
    if( pLevel->pRJ ){
+
      sqlite3VdbeAddOp3(v, OP_Return, pLevel->pRJ->regReturn, 0, 1);
+
      VdbeCoverage(v);
+
    }
    if( pLevel->addrSkip ){
      sqlite3VdbeGoto(v, pLevel->addrSkip);
      VdbeComment((v, "next skip-scan on %s", pLoop->u.btree.pIndex->zName));
@@ -158134,11 +161975,6 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
                     pWInfo->pTabList->a[pLevel->iFrom].pTab->zName));
  }

-
  /* The "break" point is here, just past the end of the outer loop.
-
  ** Set it.
-
  */
-
  sqlite3VdbeResolveLabel(v, pWInfo->iBreak);
-

  assert( pWInfo->nLevel<=pTabList->nSrc );
  for(i=0, pLevel=pWInfo->a; i<pWInfo->nLevel; i++, pLevel++){
    int k, last;
@@ -158149,6 +161985,15 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
    assert( pTab!=0 );
    pLoop = pLevel->pWLoop;

+
    /* Do RIGHT JOIN processing.  Generate code that will output the
+
    ** unmatched rows of the right operand of the RIGHT JOIN with
+
    ** all of the columns of the left operand set to NULL.
+
    */
+
    if( pLevel->pRJ ){
+
      sqlite3WhereRightJoinLoop(pWInfo, i, pLevel);
+
      continue;
+
    }
+

    /* For a co-routine, change all OP_Column references to the table of
    ** the co-routine into OP_Copy of result contained in a register.
    ** OP_Rowid becomes OP_Null.
@@ -158160,29 +162005,6 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
      continue;
    }

-
#ifdef SQLITE_ENABLE_EARLY_CURSOR_CLOSE
-
    /* Close all of the cursors that were opened by sqlite3WhereBegin.
-
    ** Except, do not close cursors that will be reused by the OR optimization
-
    ** (WHERE_OR_SUBCLAUSE).  And do not close the OP_OpenWrite cursors
-
    ** created for the ONEPASS optimization.
-
    */
-
    if( (pTab->tabFlags & TF_Ephemeral)==0
-
     && !IsView(pTab)
-
     && (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)==0
-
    ){
-
      int ws = pLoop->wsFlags;
-
      if( pWInfo->eOnePass==ONEPASS_OFF && (ws & WHERE_IDX_ONLY)==0 ){
-
        sqlite3VdbeAddOp1(v, OP_Close, pTabItem->iCursor);
-
      }
-
      if( (ws & WHERE_INDEXED)!=0
-
       && (ws & (WHERE_IPK|WHERE_AUTO_INDEX))==0
-
       && pLevel->iIdxCur!=pWInfo->aiCurOnePass[1]
-
      ){
-
        sqlite3VdbeAddOp1(v, OP_Close, pLevel->iIdxCur);
-
      }
-
    }
-
#endif
-

    /* If this scan uses an index, make VDBE code substitutions to read data
    ** from the index instead of from the table where possible.  In some cases
    ** this optimization prevents the table from ever being read, which can
@@ -158207,6 +162029,16 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
      }else{
        last = pWInfo->iEndWhere;
      }
+
      if( pIdx->bHasExpr ){
+
        IndexedExpr *p = pParse->pIdxExpr;
+
        while( p ){
+
          if( p->iIdxCur==pLevel->iIdxCur ){
+
            p->iDataCur = -1;
+
            p->iIdxCur = -1;
+
          }
+
          p = p->pIENext;
+
        }
+
      }
      k = pLevel->addrBody + 1;
#ifdef SQLITE_DEBUG
      if( db->flags & SQLITE_VdbeAddopTrace ){
@@ -158283,11 +162115,16 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){
    }
  }

+
  /* The "break" point is here, just past the end of the outer loop.
+
  ** Set it.
+
  */
+
  sqlite3VdbeResolveLabel(v, pWInfo->iBreak);
+

  /* Final cleanup
  */
-
  if( pWInfo->pExprMods ) whereUndoExprMods(pWInfo);
  pParse->nQueryLoop = pWInfo->savedNQueryLoop;
  whereInfoFree(db, pWInfo);
+
  pParse->withinRJSubrtn -= nRJ;
  return;
}

@@ -159019,7 +162856,7 @@ SQLITE_PRIVATE void sqlite3WindowUpdate(
      }
    }
  }
-
  pWin->pFunc = pFunc;
+
  pWin->pWFunc = pFunc;
}

/*
@@ -159195,7 +163032,6 @@ static ExprList *exprListAppendList(
    for(i=0; i<pAppend->nExpr; i++){
      sqlite3 *db = pParse->db;
      Expr *pDup = sqlite3ExprDup(db, pAppend->a[i].pExpr, 0);
-
      assert( pDup==0 || !ExprHasProperty(pDup, EP_MemToken) );
      if( db->mallocFailed ){
        sqlite3ExprDelete(db, pDup);
        break;
@@ -159211,7 +163047,7 @@ static ExprList *exprListAppendList(
        }
      }
      pList = sqlite3ExprListAppend(pParse, pList, pDup);
-
      if( pList ) pList->a[nInit+i].sortFlags = pAppend->a[i].sortFlags;
+
      if( pList ) pList->a[nInit+i].fg.sortFlags = pAppend->a[i].fg.sortFlags;
    }
  }
  return pList;
@@ -159331,9 +163167,9 @@ SQLITE_PRIVATE int sqlite3WindowRewrite(Parse *pParse, Select *p){
    for(pWin=pMWin; pWin; pWin=pWin->pNextWin){
      ExprList *pArgs;
      assert( ExprUseXList(pWin->pOwner) );
-
      assert( pWin->pFunc!=0 );
+
      assert( pWin->pWFunc!=0 );
      pArgs = pWin->pOwner->x.pList;
-
      if( pWin->pFunc->funcFlags & SQLITE_FUNC_SUBTYPE ){
+
      if( pWin->pWFunc->funcFlags & SQLITE_FUNC_SUBTYPE ){
        selectWindowRewriteEList(pParse, pMWin, pSrc, pArgs, pTab, &pSublist);
        pWin->iArgCol = (pSublist ? pSublist->nExpr : 0);
        pWin->bExprArgs = 1;
@@ -159715,7 +163551,7 @@ SQLITE_PRIVATE void sqlite3WindowCodeInit(Parse *pParse, Select *pSelect){
  }

  for(pWin=pMWin; pWin; pWin=pWin->pNextWin){
-
    FuncDef *p = pWin->pFunc;
+
    FuncDef *p = pWin->pWFunc;
    if( (p->funcFlags & SQLITE_FUNC_MINMAX) && pWin->eStart!=TK_UNBOUNDED ){
      /* The inline versions of min() and max() require a single ephemeral
      ** table and 3 registers. The registers are used as follows:
@@ -159732,7 +163568,7 @@ SQLITE_PRIVATE void sqlite3WindowCodeInit(Parse *pParse, Select *pSelect){
      pWin->csrApp = pParse->nTab++;
      pWin->regApp = pParse->nMem+1;
      pParse->nMem += 3;
-
      if( pKeyInfo && pWin->pFunc->zName[1]=='i' ){
+
      if( pKeyInfo && pWin->pWFunc->zName[1]=='i' ){
        assert( pKeyInfo->aSortFlags[0]==0 );
        pKeyInfo->aSortFlags[0] = KEYINFO_ORDER_DESC;
      }
@@ -159955,7 +163791,7 @@ static void windowAggStep(
  Vdbe *v = sqlite3GetVdbe(pParse);
  Window *pWin;
  for(pWin=pMWin; pWin; pWin=pWin->pNextWin){
-
    FuncDef *pFunc = pWin->pFunc;
+
    FuncDef *pFunc = pWin->pWFunc;
    int regArg;
    int nArg = pWin->bExprArgs ? 0 : windowArgCount(pWin);
    int i;
@@ -160069,7 +163905,7 @@ static void windowAggFinal(WindowCodeArg *p, int bFin){

  for(pWin=pMWin; pWin; pWin=pWin->pNextWin){
    if( pMWin->regStartRowid==0
-
     && (pWin->pFunc->funcFlags & SQLITE_FUNC_MINMAX)
+
     && (pWin->pWFunc->funcFlags & SQLITE_FUNC_MINMAX)
     && (pWin->eStart!=TK_UNBOUNDED)
    ){
      sqlite3VdbeAddOp2(v, OP_Null, 0, pWin->regResult);
@@ -160083,12 +163919,12 @@ static void windowAggFinal(WindowCodeArg *p, int bFin){
      int nArg = windowArgCount(pWin);
      if( bFin ){
        sqlite3VdbeAddOp2(v, OP_AggFinal, pWin->regAccum, nArg);
-
        sqlite3VdbeAppendP4(v, pWin->pFunc, P4_FUNCDEF);
+
        sqlite3VdbeAppendP4(v, pWin->pWFunc, P4_FUNCDEF);
        sqlite3VdbeAddOp2(v, OP_Copy, pWin->regAccum, pWin->regResult);
        sqlite3VdbeAddOp2(v, OP_Null, 0, pWin->regAccum);
      }else{
        sqlite3VdbeAddOp3(v, OP_AggValue,pWin->regAccum,nArg,pWin->regResult);
-
        sqlite3VdbeAppendP4(v, pWin->pFunc, P4_FUNCDEF);
+
        sqlite3VdbeAppendP4(v, pWin->pWFunc, P4_FUNCDEF);
      }
    }
  }
@@ -160217,7 +164053,7 @@ static void windowReturnOneRow(WindowCodeArg *p){
    Window *pWin;

    for(pWin=pMWin; pWin; pWin=pWin->pNextWin){
-
      FuncDef *pFunc = pWin->pFunc;
+
      FuncDef *pFunc = pWin->pWFunc;
      assert( ExprUseXList(pWin->pOwner) );
      if( pFunc->zName==nth_valueName
       || pFunc->zName==first_valueName
@@ -160289,7 +164125,7 @@ static int windowInitAccum(Parse *pParse, Window *pMWin){
  int nArg = 0;
  Window *pWin;
  for(pWin=pMWin; pWin; pWin=pWin->pNextWin){
-
    FuncDef *pFunc = pWin->pFunc;
+
    FuncDef *pFunc = pWin->pWFunc;
    assert( pWin->regAccum );
    sqlite3VdbeAddOp2(v, OP_Null, 0, pWin->regAccum);
    nArg = MAX(nArg, windowArgCount(pWin));
@@ -160319,7 +164155,7 @@ static int windowCacheFrame(Window *pMWin){
  Window *pWin;
  if( pMWin->regStartRowid ) return 1;
  for(pWin=pMWin; pWin; pWin=pWin->pNextWin){
-
    FuncDef *pFunc = pWin->pFunc;
+
    FuncDef *pFunc = pWin->pWFunc;
    if( (pFunc->zName==nth_valueName)
     || (pFunc->zName==first_valueName)
     || (pFunc->zName==leadName)
@@ -160412,7 +164248,7 @@ static void windowCodeRangeTest(

  assert( op==OP_Ge || op==OP_Gt || op==OP_Le );
  assert( pOrderBy && pOrderBy->nExpr==1 );
-
  if( pOrderBy->a[0].sortFlags & KEYINFO_ORDER_DESC ){
+
  if( pOrderBy->a[0].fg.sortFlags & KEYINFO_ORDER_DESC ){
    switch( op ){
      case OP_Ge: op = OP_Le; break;
      case OP_Gt: op = OP_Lt; break;
@@ -160445,7 +164281,7 @@ static void windowCodeRangeTest(
  ** Additionally, if either reg1 or reg2 are NULL but the jump to lbl is
  ** not taken, control jumps over the comparison operator coded below this
  ** block.  */
-
  if( pOrderBy->a[0].sortFlags & KEYINFO_ORDER_BIGNULL ){
+
  if( pOrderBy->a[0].fg.sortFlags & KEYINFO_ORDER_BIGNULL ){
    /* This block runs if reg1 contains a NULL. */
    int addr = sqlite3VdbeAddOp1(v, OP_NotNull, reg1); VdbeCoverage(v);
    switch( op ){
@@ -160466,10 +164302,9 @@ static void windowCodeRangeTest(

    /* This block runs if reg1 is not NULL, but reg2 is. */
    sqlite3VdbeJumpHere(v, addr);
-
    sqlite3VdbeAddOp2(v, OP_IsNull, reg2, lbl); VdbeCoverage(v);
-
    if( op==OP_Gt || op==OP_Ge ){
-
      sqlite3VdbeChangeP2(v, -1, addrDone);
-
    }
+
    sqlite3VdbeAddOp2(v, OP_IsNull, reg2,
+
                      (op==OP_Gt || op==OP_Ge) ? addrDone : lbl);
+
    VdbeCoverage(v);
  }

  /* Register reg1 currently contains csr1.peerVal (the peer-value from csr1).
@@ -160677,7 +164512,7 @@ SQLITE_PRIVATE Window *sqlite3WindowDup(sqlite3 *db, Expr *pOwner, Window *p){
      pNew->zName = sqlite3DbStrDup(db, p->zName);
      pNew->zBase = sqlite3DbStrDup(db, p->zBase);
      pNew->pFilter = sqlite3ExprDup(db, p->pFilter, 0);
-
      pNew->pFunc = p->pFunc;
+
      pNew->pWFunc = p->pWFunc;
      pNew->pPartition = sqlite3ExprListDup(db, p->pPartition, 0);
      pNew->pOrderBy = sqlite3ExprListDup(db, p->pOrderBy, 0);
      pNew->eFrmType = p->eFrmType;
@@ -161563,7 +165398,7 @@ static void updateDeleteLimitError(
      p->affExpr = 0;
      p->flags = EP_Leaf;
      ExprClearVVAProperties(p);
-
      p->iAgg = -1;
+
      /* p->iAgg = -1; // Not required */
      p->pLeft = p->pRight = 0;
      p->pAggInfo = 0;
      memset(&p->x, 0, sizeof(p->x));
@@ -161896,6 +165731,7 @@ typedef union {
  With* yy521;
  const char* yy522;
  Expr* yy528;
+
  OnOrUsing yy561;
  struct FrameBound yy595;
} YYMINORTYPE;
#ifndef YYSTACKDEPTH
@@ -161912,18 +165748,18 @@ typedef union {
#define sqlite3ParserCTX_FETCH Parse *pParse=yypParser->pParse;
#define sqlite3ParserCTX_STORE yypParser->pParse=pParse;
#define YYFALLBACK 1
-
#define YYNSTATE             574
-
#define YYNRULE              402
-
#define YYNRULE_WITH_ACTION  340
+
#define YYNSTATE             576
+
#define YYNRULE              405
+
#define YYNRULE_WITH_ACTION  342
#define YYNTOKEN             185
-
#define YY_MAX_SHIFT         573
-
#define YY_MIN_SHIFTREDUCE   831
-
#define YY_MAX_SHIFTREDUCE   1232
-
#define YY_ERROR_ACTION      1233
-
#define YY_ACCEPT_ACTION     1234
-
#define YY_NO_ACTION         1235
-
#define YY_MIN_REDUCE        1236
-
#define YY_MAX_REDUCE        1637
+
#define YY_MAX_SHIFT         575
+
#define YY_MIN_SHIFTREDUCE   835
+
#define YY_MAX_SHIFTREDUCE   1239
+
#define YY_ERROR_ACTION      1240
+
#define YY_ACCEPT_ACTION     1241
+
#define YY_NO_ACTION         1242
+
#define YY_MIN_REDUCE        1243
+
#define YY_MAX_REDUCE        1647
/************* End control #defines *******************************************/
#define YY_NLOOKAHEAD ((int)(sizeof(yy_lookahead)/sizeof(yy_lookahead[0])))

@@ -161990,427 +165826,430 @@ typedef union {
**  yy_default[]       Default action for each state.
**
*********** Begin parsing tables **********************************************/
-
#define YY_ACTTAB_COUNT (2070)
+
#define YY_ACTTAB_COUNT (2098)
static const YYACTIONTYPE yy_action[] = {
-
 /*     0 */   566, 1307,  566, 1286,  201,  201,  566,  116,  112,  222,
-
 /*    10 */   566, 1307,  377,  566,  116,  112,  222,  397,  408,  409,
-
 /*    20 */  1260,  378, 1269,   41,   41,   41,   41, 1412, 1517,   71,
-
 /*    30 */    71,  967, 1258,   41,   41,  491,   71,   71,  272,  968,
-
 /*    40 */   298,  476,  298,  123,  124,  114, 1210, 1210, 1044, 1047,
-
 /*    50 */  1036, 1036,  121,  121,  122,  122,  122,  122,  543,  409,
-
 /*    60 */  1234,    1,    1,  573,    2, 1238,  548,  116,  112,  222,
-
 /*    70 */   309,  480,  142,  548, 1272,  524,  116,  112,  222, 1320,
-
 /*    80 */   417,  523,  547,  123,  124,  114, 1210, 1210, 1044, 1047,
-
 /*    90 */  1036, 1036,  121,  121,  122,  122,  122,  122,  424,  116,
-
 /*   100 */   112,  222,  120,  120,  120,  120,  119,  119,  118,  118,
-
 /*   110 */   118,  117,  113,  444,  277,  277,  277,  277,  560,  560,
-
 /*   120 */   560, 1558,  376, 1560, 1186,  375, 1157,  563, 1157,  563,
-
 /*   130 */   409, 1558,  537,  252,  219, 1553,   99,  141,  449,    6,
-
 /*   140 */   365,  233,  120,  120,  120,  120,  119,  119,  118,  118,
-
 /*   150 */   118,  117,  113,  444,  123,  124,  114, 1210, 1210, 1044,
-
 /*   160 */  1047, 1036, 1036,  121,  121,  122,  122,  122,  122,  138,
-
 /*   170 */   289, 1186, 1546,  448,  118,  118,  118,  117,  113,  444,
-
 /*   180 */   125, 1186, 1187, 1188,  144,  465,  334,  566,  150,  127,
-
 /*   190 */   444,  122,  122,  122,  122,  115,  120,  120,  120,  120,
-
 /*   200 */   119,  119,  118,  118,  118,  117,  113,  444,  454,  419,
-
 /*   210 */    13,   13,  215,  120,  120,  120,  120,  119,  119,  118,
-
 /*   220 */   118,  118,  117,  113,  444,  422,  308,  557, 1186, 1187,
-
 /*   230 */  1188,  441,  440,  409, 1271,  122,  122,  122,  122,  120,
-
 /*   240 */   120,  120,  120,  119,  119,  118,  118,  118,  117,  113,
-
 /*   250 */   444, 1543,   98, 1033, 1033, 1045, 1048,  123,  124,  114,
-
 /*   260 */  1210, 1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,
-
 /*   270 */   122,  122,  566,  406,  405, 1186,  566,  409, 1217,  319,
-
 /*   280 */  1217,   80,   81,  120,  120,  120,  120,  119,  119,  118,
-
 /*   290 */   118,  118,  117,  113,  444,   70,   70, 1186, 1604,   71,
-
 /*   300 */    71,  123,  124,  114, 1210, 1210, 1044, 1047, 1036, 1036,
-
 /*   310 */   121,  121,  122,  122,  122,  122,  120,  120,  120,  120,
-
 /*   320 */   119,  119,  118,  118,  118,  117,  113,  444, 1037,  210,
-
 /*   330 */  1186,  365, 1186, 1187, 1188,  245,  548,  399,  504,  501,
-
 /*   340 */   500,  108,  558,  138,    4,  516,  933,  433,  499,  217,
-
 /*   350 */   514,  522,  352,  879, 1186, 1187, 1188,  383,  561,  566,
-
 /*   360 */   120,  120,  120,  120,  119,  119,  118,  118,  118,  117,
-
 /*   370 */   113,  444,  277,  277,   16,   16, 1598,  441,  440,  153,
-
 /*   380 */   409,  445,   13,   13, 1279,  563, 1214, 1186, 1187, 1188,
-
 /*   390 */  1003, 1216,  264,  555, 1574,  186,  566,  427,  138, 1215,
-
 /*   400 */   308,  557,  472,  138,  123,  124,  114, 1210, 1210, 1044,
-
 /*   410 */  1047, 1036, 1036,  121,  121,  122,  122,  122,  122,   55,
-
 /*   420 */    55,  413, 1023,  507, 1217, 1186, 1217,  474,  106,  106,
-
 /*   430 */  1312, 1312, 1186,  171,  566,  384,  107,  380,  445,  568,
-
 /*   440 */   567,  430, 1543, 1013,  332,  549,  565,  263,  280,  360,
-
 /*   450 */   510,  355,  509,  250,  491,  308,  557,   71,   71,  351,
-
 /*   460 */   308,  557,  374,  120,  120,  120,  120,  119,  119,  118,
-
 /*   470 */   118,  118,  117,  113,  444, 1013, 1013, 1015, 1016,   27,
-
 /*   480 */   277,  277, 1186, 1187, 1188, 1152,  566,  528,  409, 1186,
-
 /*   490 */  1187, 1188,  348,  563,  548, 1260,  533,  517, 1152, 1516,
-
 /*   500 */   317, 1152,  285,  550,  485,  569,  566,  569,  482,   51,
-
 /*   510 */    51,  207,  123,  124,  114, 1210, 1210, 1044, 1047, 1036,
-
 /*   520 */  1036,  121,  121,  122,  122,  122,  122,  171, 1412,   13,
-
 /*   530 */    13,  409,  277,  277, 1186,  505,  119,  119,  118,  118,
-
 /*   540 */   118,  117,  113,  444,  429,  563,  518,  220,  515, 1552,
-
 /*   550 */   365,  546, 1186,    6,  532,  123,  124,  114, 1210, 1210,
-
 /*   560 */  1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,  122,
-
 /*   570 */   145,  120,  120,  120,  120,  119,  119,  118,  118,  118,
-
 /*   580 */   117,  113,  444,  245,  566,  474,  504,  501,  500,  566,
-
 /*   590 */  1481, 1186, 1187, 1188, 1310, 1310,  499, 1186,  149,  425,
-
 /*   600 */  1186,  480,  409,  274,  365,  952,  872,   56,   56, 1186,
-
 /*   610 */  1187, 1188,   71,   71,  120,  120,  120,  120,  119,  119,
-
 /*   620 */   118,  118,  118,  117,  113,  444,  123,  124,  114, 1210,
-
 /*   630 */  1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,
-
 /*   640 */   122,  409,  541, 1552,   83,  865,   98,    6,  928,  529,
-
 /*   650 */   848,  543,  151,  927, 1186, 1187, 1188, 1186, 1187, 1188,
-
 /*   660 */   290, 1543,  187, 1633,  395,  123,  124,  114, 1210, 1210,
-
 /*   670 */  1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,  122,
-
 /*   680 */   566,  954,  566,  453,  953,  120,  120,  120,  120,  119,
-
 /*   690 */   119,  118,  118,  118,  117,  113,  444, 1152,  221, 1186,
-
 /*   700 */   331,  453,  452,   13,   13,   13,   13, 1003,  365,  463,
-
 /*   710 */  1152,  193,  409, 1152,  382, 1543, 1170,   32,  297,  474,
-
 /*   720 */   195, 1527,    5,  952,  120,  120,  120,  120,  119,  119,
-
 /*   730 */   118,  118,  118,  117,  113,  444,  123,  124,  114, 1210,
-
 /*   740 */  1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,
-
 /*   750 */   122,  409, 1067,  419, 1186, 1024, 1186, 1187, 1188, 1186,
-
 /*   760 */   419,  332,  460,  320,  544, 1545,  442,  442,  442,  566,
-
 /*   770 */     3,  117,  113,  444,  453,  123,  124,  114, 1210, 1210,
-
 /*   780 */  1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,  122,
-
 /*   790 */  1473,  566,   15,   15,  293,  120,  120,  120,  120,  119,
-
 /*   800 */   119,  118,  118,  118,  117,  113,  444, 1186,  566, 1486,
-
 /*   810 */  1412, 1186, 1187, 1188,   13,   13, 1186, 1187, 1188, 1544,
-
 /*   820 */   271,  271,  409,  286,  308,  557, 1008, 1486, 1488,  196,
-
 /*   830 */   288,   71,   71,  563,  120,  120,  120,  120,  119,  119,
-
 /*   840 */   118,  118,  118,  117,  113,  444,  123,  124,  114, 1210,
-
 /*   850 */  1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,
-
 /*   860 */   122,  409,  201, 1087, 1186, 1187, 1188, 1324,  304, 1529,
-
 /*   870 */   388,  278,  278,  450,  564,  402,  922,  922,  566,  563,
-
 /*   880 */   566,  426,  491,  480,  563,  123,  124,  114, 1210, 1210,
-
 /*   890 */  1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,  122,
-
 /*   900 */  1486,   71,   71,   13,   13,  120,  120,  120,  120,  119,
-
 /*   910 */   119,  118,  118,  118,  117,  113,  444,  566,  545,  566,
-
 /*   920 */  1577,  573,    2, 1238, 1092, 1092,  488, 1480,  309, 1525,
-
 /*   930 */   142,  324,  409,  836,  837,  838,  312, 1320,  305,  363,
-
 /*   940 */    43,   43,   57,   57,  120,  120,  120,  120,  119,  119,
-
 /*   950 */   118,  118,  118,  117,  113,  444,  123,  124,  114, 1210,
-
 /*   960 */  1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,
-
 /*   970 */   122,   12,  277,  277,  566, 1152,  409,  572,  428, 1238,
-
 /*   980 */   465,  334,  296,  474,  309,  563,  142,  249, 1152,  308,
-
 /*   990 */   557, 1152,  321, 1320,  323,  491,  455,   71,   71,  233,
-
 /*  1000 */   283,  101,  114, 1210, 1210, 1044, 1047, 1036, 1036,  121,
-
 /*  1010 */   121,  122,  122,  122,  122,  120,  120,  120,  120,  119,
-
 /*  1020 */   119,  118,  118,  118,  117,  113,  444, 1108,  277,  277,
-
 /*  1030 */  1412,  448,  394, 1230,  439,  277,  277,  248,  247,  246,
-
 /*  1040 */  1319,  563, 1109,  313,  198,  294,  491, 1318,  563,  464,
-
 /*  1050 */   566, 1427,  394, 1130, 1023,  233,  414, 1110,  295,  120,
-
 /*  1060 */   120,  120,  120,  119,  119,  118,  118,  118,  117,  113,
-
 /*  1070 */   444, 1014,  104,   71,   71, 1013,  322,  496,  908,  566,
-
 /*  1080 */   277,  277,  277,  277, 1108, 1261,  415,  448,  909,  361,
-
 /*  1090 */  1571, 1315,  409,  563,  952,  563,    9,  202,  255, 1109,
-
 /*  1100 */   316,  487,   44,   44,  249,  559,  415, 1013, 1013, 1015,
-
 /*  1110 */   443, 1231,  409, 1603, 1110,  897,  123,  124,  114, 1210,
-
 /*  1120 */  1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,
-
 /*  1130 */   122, 1231,  409, 1207,  215,  554,  123,  124,  114, 1210,
-
 /*  1140 */  1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,
-
 /*  1150 */   122, 1131, 1631,  470, 1631,  255,  123,  111,  114, 1210,
-
 /*  1160 */  1210, 1044, 1047, 1036, 1036,  121,  121,  122,  122,  122,
-
 /*  1170 */   122, 1131, 1632,  414, 1632,  120,  120,  120,  120,  119,
-
 /*  1180 */   119,  118,  118,  118,  117,  113,  444,  221,  209,  351,
-
 /*  1190 */  1207, 1207,  147, 1426,  491,  120,  120,  120,  120,  119,
-
 /*  1200 */   119,  118,  118,  118,  117,  113,  444, 1256,  539,  519,
-
 /*  1210 */   888,  551,  952,   12,  566,  120,  120,  120,  120,  119,
-
 /*  1220 */   119,  118,  118,  118,  117,  113,  444,  538,  566,  860,
-
 /*  1230 */  1129,  361, 1571,  346, 1356,  409, 1163,   58,   58,  339,
-
 /*  1240 */  1355,  508,  277,  277,  277,  277,  277,  277, 1207,  889,
-
 /*  1250 */  1129,   59,   59,  459,  363,  563,  566,  563,   96,  563,
-
 /*  1260 */   124,  114, 1210, 1210, 1044, 1047, 1036, 1036,  121,  121,
-
 /*  1270 */   122,  122,  122,  122,  566, 1412,  566,  281, 1186,   60,
-
 /*  1280 */    60,  110,  392,  392,  391,  266,  389,  860, 1163,  845,
-
 /*  1290 */   566,  481,  566,  436,  341, 1152,  344,   61,   61,   62,
-
 /*  1300 */    62,  967,  227, 1550,  315,  431,  540,    6, 1152,  968,
-
 /*  1310 */   566, 1152,  314,   45,   45,   46,   46,  512,  120,  120,
-
 /*  1320 */   120,  120,  119,  119,  118,  118,  118,  117,  113,  444,
-
 /*  1330 */   416,  173, 1532,   47,   47, 1186, 1187, 1188,  108,  558,
-
 /*  1340 */   325,    4,  229, 1551,  928,  566,  437,    6,  566,  927,
-
 /*  1350 */   164,  566, 1290,  137, 1190,  561,  566, 1549,  566, 1089,
-
 /*  1360 */   566,    6,  566, 1089,  531,  566,  868,    8,   49,   49,
-
 /*  1370 */   228,   50,   50,  566,   63,   63,  566,  457,  445,   64,
-
 /*  1380 */    64,   65,   65,   14,   14,   66,   66,  407,  129,  129,
-
 /*  1390 */   555,  566,  458,  566, 1505,  486,   67,   67,  566,   52,
-
 /*  1400 */    52,  546,  407,  467,  535,  410,  226, 1023,  566,  534,
-
 /*  1410 */   308,  557, 1190,  407,   68,   68,   69,   69,  566, 1023,
-
 /*  1420 */   566,   53,   53,  868, 1014,  106,  106,  525, 1013,  566,
-
 /*  1430 */  1504,  159,  159,  107,  451,  445,  568,  567,  471,  307,
-
 /*  1440 */  1013,  160,  160,   76,   76,  566, 1548,  466,  407,  407,
-
 /*  1450 */     6, 1225,   54,   54,  478,  276,  219,  566,  887,  886,
-
 /*  1460 */  1013, 1013, 1015,   84,  206, 1206,  230,  282,   72,   72,
-
 /*  1470 */   329,  483, 1013, 1013, 1015, 1016,   27, 1576, 1174,  447,
-
 /*  1480 */   130,  130,  281,  148,  105,   38,  103,  392,  392,  391,
-
 /*  1490 */   266,  389,  566, 1126,  845,  396,  566,  108,  558,  566,
-
 /*  1500 */     4,  311,  566,   30,   17,  566,  279,  227,  566,  315,
-
 /*  1510 */   108,  558,  468,    4,  561,   73,   73,  314,  566,  157,
-
 /*  1520 */   157,  566,  131,  131,  526,  132,  132,  561,  128,  128,
-
 /*  1530 */   566,  158,  158,  566,   31,  291,  566,  445,  330,  521,
-
 /*  1540 */    98,  152,  152,  420,  136,  136, 1005,  229,  254,  555,
-
 /*  1550 */   445,  479,  336,  135,  135,  164,  133,  133,  137,  134,
-
 /*  1560 */   134,  875,  555,  535,  566,  473,  566,  254,  536,  475,
-
 /*  1570 */   335,  254,   98,  894,  895,  228,  535,  566, 1023,  566,
-
 /*  1580 */  1074,  534,  210,  232,  106,  106, 1352,   75,   75,   77,
-
 /*  1590 */    77, 1023,  107,  340,  445,  568,  567,  106,  106, 1013,
-
 /*  1600 */    74,   74,   42,   42,  566,  107,  343,  445,  568,  567,
-
 /*  1610 */   410,  497, 1013,  251,  359,  308,  557, 1135,  349,  875,
-
 /*  1620 */    98, 1070,  345,  251,  358, 1591,  347,   48,   48, 1017,
-
 /*  1630 */  1303, 1013, 1013, 1015, 1016,   27, 1289, 1287, 1074,  451,
-
 /*  1640 */   961,  925,  254,  110, 1013, 1013, 1015, 1016,   27, 1174,
-
 /*  1650 */   447,  970,  971,  281,  108,  558, 1288,    4,  392,  392,
-
 /*  1660 */   391,  266,  389, 1343, 1086,  845, 1086, 1085,  858, 1085,
-
 /*  1670 */   146,  561,  926,  354,  110,  303,  364,  553,  227, 1364,
-
 /*  1680 */   315,  108,  558, 1411,    4, 1339,  492, 1017,  314, 1350,
-
 /*  1690 */  1565,  552, 1417, 1268,  445,  204, 1259, 1247,  561, 1246,
-
 /*  1700 */  1248, 1584,  269, 1336,  367,  369,  555,  371,   11,  212,
-
 /*  1710 */   393,  225, 1393,  284, 1398,  456,  287,  327,  229,  328,
-
 /*  1720 */   292,  445, 1386,  216,  333, 1403,  164,  477,  373,  137,
-
 /*  1730 */  1402,  400,  502,  555, 1286, 1023,  357, 1477,  199, 1587,
-
 /*  1740 */   211,  106,  106,  932, 1476, 1225,  228,  556,  175,  107,
-
 /*  1750 */   200,  445,  568,  567,  258,  387, 1013, 1524, 1522,  223,
-
 /*  1760 */  1222,  418, 1023,   83,  208,   79,   82,  184,  106,  106,
-
 /*  1770 */  1482,  169,  177,  461,  179,  462,  107, 1399,  445,  568,
-
 /*  1780 */   567,  410,  180, 1013,  495,  181,  308,  557, 1013, 1013,
-
 /*  1790 */  1015, 1016,   27,  182,   35,  235,  100,  558,  398,    4,
-
 /*  1800 */    96, 1405, 1404,   36,  484,  469, 1407,  188,  401, 1471,
-
 /*  1810 */   451,   89, 1493,  561,  239, 1013, 1013, 1015, 1016,   27,
-
 /*  1820 */   490,  338,  270,  241,  192,  342,  493,  242,  403, 1249,
-
 /*  1830 */   243,  511,  432, 1297, 1306,   91,  445, 1305, 1304,  879,
-
 /*  1840 */   217,  434,  435, 1570, 1276, 1602,  520, 1601,  555,  301,
-
 /*  1850 */   527,  404, 1275,  302,  356, 1274, 1600,   95, 1347,  366,
-
 /*  1860 */  1296,  362, 1348,  368,  256,  257, 1556, 1555,  438, 1346,
-
 /*  1870 */   370,  126, 1345,   10, 1371,  546,  381, 1023,  102, 1457,
-
 /*  1880 */    97,  530,   34,  106,  106,  570, 1180,  372,  265, 1329,
-
 /*  1890 */   379,  107,  203,  445,  568,  567, 1328,  385, 1013, 1370,
-
 /*  1900 */   386,  267,  268,  571, 1244,  161, 1239,  162, 1509, 1510,
-
 /*  1910 */  1508,  143, 1507,  299,  832,  213,  214,   78,  446,  205,
-
 /*  1920 */   310,  306,  163,  224, 1084,  140, 1082,  318,  165,  176,
-
 /*  1930 */  1013, 1013, 1015, 1016,   27,  178, 1206,  231,  911,  234,
-
 /*  1940 */   326, 1098,  183,  421,  166,  167,  411,  185,   85,  423,
-
 /*  1950 */   412,   86,  174,   87,  168,   88, 1101,  236, 1097,  237,
-
 /*  1960 */   154,   18,  238,  254,  337, 1219,  489, 1090,  240,  190,
-
 /*  1970 */    37,  847,  189,  494,  358,  244,  350,  506,  191,  877,
-
 /*  1980 */    90,  498,   19,   20,  503,   92,  353,  890,  300,  170,
-
 /*  1990 */   155,   93,  513,   94, 1168,  156, 1050, 1137,   39,  218,
-
 /*  2000 */   273,  275, 1136,  960,  194,  955,  110, 1154, 1158,  253,
-
 /*  2010 */     7, 1162, 1156,   21,   22, 1161, 1142,   23,   24,   25,
-
 /*  2020 */    33,  542,   26,  260,  197,   98, 1065, 1051, 1049, 1053,
-
 /*  2030 */  1107, 1054, 1106,  259,   28,   40,  562, 1018,  859,  109,
-
 /*  2040 */    29,  921,  390, 1176,  172,  139, 1175, 1235,  261, 1235,
-
 /*  2050 */  1235, 1235, 1235, 1235, 1235, 1235, 1235,  262, 1235, 1235,
-
 /*  2060 */  1235, 1235, 1235, 1235, 1235, 1235, 1235, 1235, 1593, 1592,
+
 /*     0 */   568,  208,  568,  118,  115,  229,  568,  118,  115,  229,
+
 /*    10 */   568, 1314,  377, 1293,  408,  562,  562,  562,  568,  409,
+
 /*    20 */   378, 1314, 1276,   41,   41,   41,   41,  208, 1526,   71,
+
 /*    30 */    71,  971,  419,   41,   41,  491,  303,  279,  303,  972,
+
 /*    40 */   397,   71,   71,  125,  126,   80, 1217, 1217, 1050, 1053,
+
 /*    50 */  1040, 1040,  123,  123,  124,  124,  124,  124,  476,  409,
+
 /*    60 */  1241,    1,    1,  575,    2, 1245,  550,  118,  115,  229,
+
 /*    70 */   317,  480,  146,  480,  524,  118,  115,  229,  529, 1327,
+
 /*    80 */   417,  523,  142,  125,  126,   80, 1217, 1217, 1050, 1053,
+
 /*    90 */  1040, 1040,  123,  123,  124,  124,  124,  124,  118,  115,
+
 /*   100 */   229,  327,  122,  122,  122,  122,  121,  121,  120,  120,
+
 /*   110 */   120,  119,  116,  444,  284,  284,  284,  284,  442,  442,
+
 /*   120 */   442, 1567,  376, 1569, 1192,  375, 1163,  565, 1163,  565,
+
 /*   130 */   409, 1567,  537,  259,  226,  444,  101,  145,  449,  316,
+
 /*   140 */   559,  240,  122,  122,  122,  122,  121,  121,  120,  120,
+
 /*   150 */   120,  119,  116,  444,  125,  126,   80, 1217, 1217, 1050,
+
 /*   160 */  1053, 1040, 1040,  123,  123,  124,  124,  124,  124,  142,
+
 /*   170 */   294, 1192,  339,  448,  120,  120,  120,  119,  116,  444,
+
 /*   180 */   127, 1192, 1193, 1194,  148,  441,  440,  568,  119,  116,
+
 /*   190 */   444,  124,  124,  124,  124,  117,  122,  122,  122,  122,
+
 /*   200 */   121,  121,  120,  120,  120,  119,  116,  444,  454,  113,
+
 /*   210 */    13,   13,  546,  122,  122,  122,  122,  121,  121,  120,
+
 /*   220 */   120,  120,  119,  116,  444,  422,  316,  559, 1192, 1193,
+
 /*   230 */  1194,  149, 1224,  409, 1224,  124,  124,  124,  124,  122,
+
 /*   240 */   122,  122,  122,  121,  121,  120,  120,  120,  119,  116,
+
 /*   250 */   444,  465,  342, 1037, 1037, 1051, 1054,  125,  126,   80,
+
 /*   260 */  1217, 1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,
+
 /*   270 */   124,  124, 1279,  522,  222, 1192,  568,  409,  224,  514,
+
 /*   280 */   175,   82,   83,  122,  122,  122,  122,  121,  121,  120,
+
 /*   290 */   120,  120,  119,  116,  444, 1007,   16,   16, 1192,  133,
+
 /*   300 */   133,  125,  126,   80, 1217, 1217, 1050, 1053, 1040, 1040,
+
 /*   310 */   123,  123,  124,  124,  124,  124,  122,  122,  122,  122,
+
 /*   320 */   121,  121,  120,  120,  120,  119,  116,  444, 1041,  546,
+
 /*   330 */  1192,  373, 1192, 1193, 1194,  252, 1434,  399,  504,  501,
+
 /*   340 */   500,  111,  560,  566,    4,  926,  926,  433,  499,  340,
+
 /*   350 */   460,  328,  360,  394, 1237, 1192, 1193, 1194,  563,  568,
+
 /*   360 */   122,  122,  122,  122,  121,  121,  120,  120,  120,  119,
+
 /*   370 */   116,  444,  284,  284,  369, 1580, 1607,  441,  440,  154,
+
 /*   380 */   409,  445,   71,   71, 1286,  565, 1221, 1192, 1193, 1194,
+
 /*   390 */    85, 1223,  271,  557,  543,  515, 1561,  568,   98, 1222,
+
 /*   400 */     6, 1278,  472,  142,  125,  126,   80, 1217, 1217, 1050,
+
 /*   410 */  1053, 1040, 1040,  123,  123,  124,  124,  124,  124,  550,
+
 /*   420 */    13,   13, 1027,  507, 1224, 1192, 1224,  549,  109,  109,
+
 /*   430 */   222,  568, 1238,  175,  568,  427,  110,  197,  445,  570,
+
 /*   440 */   569,  430, 1552, 1017,  325,  551, 1192,  270,  287,  368,
+
 /*   450 */   510,  363,  509,  257,   71,   71,  543,   71,   71,  359,
+
 /*   460 */   316,  559, 1613,  122,  122,  122,  122,  121,  121,  120,
+
 /*   470 */   120,  120,  119,  116,  444, 1017, 1017, 1019, 1020,   27,
+
 /*   480 */   284,  284, 1192, 1193, 1194, 1158,  568, 1612,  409,  901,
+
 /*   490 */   190,  550,  356,  565,  550,  937,  533,  517, 1158,  516,
+
 /*   500 */   413, 1158,  552, 1192, 1193, 1194,  568,  544, 1554,   51,
+
 /*   510 */    51,  214,  125,  126,   80, 1217, 1217, 1050, 1053, 1040,
+
 /*   520 */  1040,  123,  123,  124,  124,  124,  124, 1192,  474,  135,
+
 /*   530 */   135,  409,  284,  284, 1490,  505,  121,  121,  120,  120,
+
 /*   540 */   120,  119,  116,  444, 1007,  565,  518,  217,  541, 1561,
+
 /*   550 */   316,  559,  142,    6,  532,  125,  126,   80, 1217, 1217,
+
 /*   560 */  1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,  124,
+
 /*   570 */  1555,  122,  122,  122,  122,  121,  121,  120,  120,  120,
+
 /*   580 */   119,  116,  444,  485, 1192, 1193, 1194,  482,  281, 1267,
+
 /*   590 */   957,  252, 1192,  373,  504,  501,  500, 1192,  340,  571,
+
 /*   600 */  1192,  571,  409,  292,  499,  957,  876,  191,  480,  316,
+
 /*   610 */   559,  384,  290,  380,  122,  122,  122,  122,  121,  121,
+
 /*   620 */   120,  120,  120,  119,  116,  444,  125,  126,   80, 1217,
+
 /*   630 */  1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,
+
 /*   640 */   124,  409,  394, 1136, 1192,  869,  100,  284,  284, 1192,
+
 /*   650 */  1193, 1194,  373, 1093, 1192, 1193, 1194, 1192, 1193, 1194,
+
 /*   660 */   565,  455,   32,  373,  233,  125,  126,   80, 1217, 1217,
+
 /*   670 */  1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,  124,
+
 /*   680 */  1433,  959,  568,  228,  958,  122,  122,  122,  122,  121,
+
 /*   690 */   121,  120,  120,  120,  119,  116,  444, 1158,  228, 1192,
+
 /*   700 */   157, 1192, 1193, 1194, 1553,   13,   13,  301,  957, 1232,
+
 /*   710 */  1158,  153,  409, 1158,  373, 1583, 1176,    5,  369, 1580,
+
 /*   720 */   429, 1238,    3,  957,  122,  122,  122,  122,  121,  121,
+
 /*   730 */   120,  120,  120,  119,  116,  444,  125,  126,   80, 1217,
+
 /*   740 */  1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,
+
 /*   750 */   124,  409,  208,  567, 1192, 1028, 1192, 1193, 1194, 1192,
+
 /*   760 */   388,  852,  155, 1552,  286,  402, 1098, 1098,  488,  568,
+
 /*   770 */   465,  342, 1319, 1319, 1552,  125,  126,   80, 1217, 1217,
+
 /*   780 */  1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,  124,
+
 /*   790 */   129,  568,   13,   13,  374,  122,  122,  122,  122,  121,
+
 /*   800 */   121,  120,  120,  120,  119,  116,  444,  302,  568,  453,
+
 /*   810 */   528, 1192, 1193, 1194,   13,   13, 1192, 1193, 1194, 1297,
+
 /*   820 */   463, 1267,  409, 1317, 1317, 1552, 1012,  453,  452,  200,
+
 /*   830 */   299,   71,   71, 1265,  122,  122,  122,  122,  121,  121,
+
 /*   840 */   120,  120,  120,  119,  116,  444,  125,  126,   80, 1217,
+
 /*   850 */  1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,
+
 /*   860 */   124,  409,  227, 1073, 1158,  284,  284,  419,  312,  278,
+
 /*   870 */   278,  285,  285, 1419,  406,  405,  382, 1158,  565,  568,
+
 /*   880 */  1158, 1196,  565, 1600,  565,  125,  126,   80, 1217, 1217,
+
 /*   890 */  1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,  124,
+
 /*   900 */   453, 1482,   13,   13, 1536,  122,  122,  122,  122,  121,
+
 /*   910 */   121,  120,  120,  120,  119,  116,  444,  201,  568,  354,
+
 /*   920 */  1586,  575,    2, 1245,  840,  841,  842, 1562,  317, 1212,
+
 /*   930 */   146,    6,  409,  255,  254,  253,  206, 1327,    9, 1196,
+
 /*   940 */   262,   71,   71,  424,  122,  122,  122,  122,  121,  121,
+
 /*   950 */   120,  120,  120,  119,  116,  444,  125,  126,   80, 1217,
+
 /*   960 */  1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,
+
 /*   970 */   124,  568,  284,  284,  568, 1213,  409,  574,  313, 1245,
+
 /*   980 */   349, 1296,  352,  419,  317,  565,  146,  491,  525, 1643,
+
 /*   990 */   395,  371,  491, 1327,   70,   70, 1295,   71,   71,  240,
+
 /*  1000 */  1325,  104,   80, 1217, 1217, 1050, 1053, 1040, 1040,  123,
+
 /*  1010 */   123,  124,  124,  124,  124,  122,  122,  122,  122,  121,
+
 /*  1020 */   121,  120,  120,  120,  119,  116,  444, 1114,  284,  284,
+
 /*  1030 */   428,  448, 1525, 1213,  439,  284,  284, 1489, 1352,  311,
+
 /*  1040 */   474,  565, 1115,  971,  491,  491,  217, 1263,  565, 1538,
+
 /*  1050 */   568,  972,  207,  568, 1027,  240,  383, 1116,  519,  122,
+
 /*  1060 */   122,  122,  122,  121,  121,  120,  120,  120,  119,  116,
+
 /*  1070 */   444, 1018,  107,   71,   71, 1017,   13,   13,  912,  568,
+
 /*  1080 */  1495,  568,  284,  284,   97,  526,  491,  448,  913, 1326,
+
 /*  1090 */  1322,  545,  409,  284,  284,  565,  151,  209, 1495, 1497,
+
 /*  1100 */   262,  450,   55,   55,   56,   56,  565, 1017, 1017, 1019,
+
 /*  1110 */   443,  332,  409,  527,   12,  295,  125,  126,   80, 1217,
+
 /*  1120 */  1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,
+
 /*  1130 */   124,  347,  409,  864, 1534, 1213,  125,  126,   80, 1217,
+
 /*  1140 */  1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,
+
 /*  1150 */   124, 1137, 1641,  474, 1641,  371,  125,  114,   80, 1217,
+
 /*  1160 */  1217, 1050, 1053, 1040, 1040,  123,  123,  124,  124,  124,
+
 /*  1170 */   124, 1495,  329,  474,  331,  122,  122,  122,  122,  121,
+
 /*  1180 */   121,  120,  120,  120,  119,  116,  444,  203, 1419,  568,
+
 /*  1190 */  1294,  864,  464, 1213,  436,  122,  122,  122,  122,  121,
+
 /*  1200 */   121,  120,  120,  120,  119,  116,  444,  553, 1137, 1642,
+
 /*  1210 */   539, 1642,   15,   15,  892,  122,  122,  122,  122,  121,
+
 /*  1220 */   121,  120,  120,  120,  119,  116,  444,  568,  298,  538,
+
 /*  1230 */  1135, 1419, 1559, 1560, 1331,  409,    6,    6, 1169, 1268,
+
 /*  1240 */   415,  320,  284,  284, 1419,  508,  565,  525,  300,  457,
+
 /*  1250 */    43,   43,  568,  893,   12,  565,  330,  478,  425,  407,
+
 /*  1260 */   126,   80, 1217, 1217, 1050, 1053, 1040, 1040,  123,  123,
+
 /*  1270 */   124,  124,  124,  124,  568,   57,   57,  288, 1192, 1419,
+
 /*  1280 */   496,  458,  392,  392,  391,  273,  389, 1135, 1558,  849,
+
 /*  1290 */  1169,  407,    6,  568,  321, 1158,  470,   44,   44, 1557,
+
 /*  1300 */  1114,  426,  234,    6,  323,  256,  540,  256, 1158,  431,
+
 /*  1310 */   568, 1158,  322,   17,  487, 1115,   58,   58,  122,  122,
+
 /*  1320 */   122,  122,  121,  121,  120,  120,  120,  119,  116,  444,
+
 /*  1330 */  1116,  216,  481,   59,   59, 1192, 1193, 1194,  111,  560,
+
 /*  1340 */   324,    4,  236,  456,  526,  568,  237,  456,  568,  437,
+
 /*  1350 */   168,  556,  420,  141,  479,  563,  568,  293,  568, 1095,
+
 /*  1360 */   568,  293,  568, 1095,  531,  568,  872,    8,   60,   60,
+
 /*  1370 */   235,   61,   61,  568,  414,  568,  414,  568,  445,   62,
+
 /*  1380 */    62,   45,   45,   46,   46,   47,   47,  199,   49,   49,
+
 /*  1390 */   557,  568,  359,  568,  100,  486,   50,   50,   63,   63,
+
 /*  1400 */    64,   64,  561,  415,  535,  410,  568, 1027,  568,  534,
+
 /*  1410 */   316,  559,  316,  559,   65,   65,   14,   14,  568, 1027,
+
 /*  1420 */   568,  512,  932,  872, 1018,  109,  109,  931, 1017,   66,
+
 /*  1430 */    66,  131,  131,  110,  451,  445,  570,  569,  416,  177,
+
 /*  1440 */  1017,  132,  132,   67,   67,  568,  467,  568,  932,  471,
+
 /*  1450 */  1364,  283,  226,  931,  315, 1363,  407,  568,  459,  407,
+
 /*  1460 */  1017, 1017, 1019,  239,  407,   86,  213, 1350,   52,   52,
+
 /*  1470 */    68,   68, 1017, 1017, 1019, 1020,   27, 1585, 1180,  447,
+
 /*  1480 */    69,   69,  288,   97,  108, 1541,  106,  392,  392,  391,
+
 /*  1490 */   273,  389,  568,  879,  849,  883,  568,  111,  560,  466,
+
 /*  1500 */     4,  568,  152,   30,   38,  568, 1132,  234,  396,  323,
+
 /*  1510 */   111,  560,  527,    4,  563,   53,   53,  322,  568,  163,
+
 /*  1520 */   163,  568,  337,  468,  164,  164,  333,  563,   76,   76,
+
 /*  1530 */   568,  289, 1514,  568,   31, 1513,  568,  445,  338,  483,
+
 /*  1540 */   100,   54,   54,  344,   72,   72,  296,  236, 1080,  557,
+
 /*  1550 */   445,  879, 1360,  134,  134,  168,   73,   73,  141,  161,
+
 /*  1560 */   161, 1574,  557,  535,  568,  319,  568,  348,  536, 1009,
+
 /*  1570 */   473,  261,  261,  891,  890,  235,  535,  568, 1027,  568,
+
 /*  1580 */   475,  534,  261,  367,  109,  109,  521,  136,  136,  130,
+
 /*  1590 */   130, 1027,  110,  366,  445,  570,  569,  109,  109, 1017,
+
 /*  1600 */   162,  162,  156,  156,  568,  110, 1080,  445,  570,  569,
+
 /*  1610 */   410,  351, 1017,  568,  353,  316,  559,  568,  343,  568,
+
 /*  1620 */   100,  497,  357,  258,  100,  898,  899,  140,  140,  355,
+
 /*  1630 */  1310, 1017, 1017, 1019, 1020,   27,  139,  139,  362,  451,
+
 /*  1640 */   137,  137,  138,  138, 1017, 1017, 1019, 1020,   27, 1180,
+
 /*  1650 */   447,  568,  372,  288,  111,  560, 1021,    4,  392,  392,
+
 /*  1660 */   391,  273,  389,  568, 1141,  849,  568, 1076,  568,  258,
+
 /*  1670 */   492,  563,  568,  211,   75,   75,  555,  962,  234,  261,
+
 /*  1680 */   323,  111,  560,  929,    4,  113,   77,   77,  322,   74,
+
 /*  1690 */    74,   42,   42, 1373,  445,   48,   48, 1418,  563,  974,
+
 /*  1700 */   975, 1092, 1091, 1092, 1091,  862,  557,  150,  930, 1346,
+
 /*  1710 */   113, 1358,  554, 1424, 1021, 1275, 1266, 1254,  236, 1253,
+
 /*  1720 */  1255,  445, 1593, 1343,  308,  276,  168,  309,   11,  141,
+
 /*  1730 */   393,  310,  232,  557, 1405, 1027,  335,  291, 1400,  219,
+
 /*  1740 */   336,  109,  109,  936,  297, 1410,  235,  341,  477,  110,
+
 /*  1750 */   502,  445,  570,  569, 1393, 1409, 1017,  400, 1293,  365,
+
 /*  1760 */   223, 1486, 1027, 1485, 1355, 1356, 1354, 1353,  109,  109,
+
 /*  1770 */   204, 1596, 1232,  558,  265,  218,  110,  205,  445,  570,
+
 /*  1780 */   569,  410,  387, 1017, 1533,  179,  316,  559, 1017, 1017,
+
 /*  1790 */  1019, 1020,   27,  230, 1531, 1229,   79,  560,   85,    4,
+
 /*  1800 */   418,  215,  548,   81,   84,  188, 1406,  173,  181,  461,
+
 /*  1810 */   451,   35,  462,  563,  183, 1017, 1017, 1019, 1020,   27,
+
 /*  1820 */   184, 1491,  185,  186,  495,  242,   98,  398, 1412,   36,
+
 /*  1830 */  1411,  484,   91,  469,  401, 1414,  445,  192, 1480,  246,
+
 /*  1840 */  1502,  490,  346,  277,  248,  196,  493,  511,  557,  350,
+
 /*  1850 */  1256,  249,  250,  403, 1313, 1312,  111,  560,  432,    4,
+
 /*  1860 */  1311, 1304,   93, 1611,  883, 1610,  224,  404,  434,  520,
+
 /*  1870 */   263,  435, 1579,  563, 1283, 1282,  364, 1027,  306, 1281,
+
 /*  1880 */   264, 1609, 1565,  109,  109,  370, 1303,  307, 1564,  438,
+
 /*  1890 */   128,  110, 1378,  445,  570,  569,  445,  546, 1017,   10,
+
 /*  1900 */  1466,  105,  381, 1377,   34,  572,   99, 1336,  557,  314,
+
 /*  1910 */  1186,  530,  272,  274,  379,  210, 1335,  547,  385,  386,
+
 /*  1920 */   275,  573, 1251, 1246,  411,  412, 1518,  165,  178, 1519,
+
 /*  1930 */  1017, 1017, 1019, 1020,   27, 1517, 1516, 1027,   78,  147,
+
 /*  1940 */   166,  220,  221,  109,  109,  836,  304,  167,  446,  212,
+
 /*  1950 */   318,  110,  231,  445,  570,  569,  144, 1090, 1017, 1088,
+
 /*  1960 */   326,  180,  169, 1212,  182,  334,  238,  915,  241, 1104,
+
 /*  1970 */   187,  170,  171,  421,   87,   88,  423,  189,   89,   90,
+
 /*  1980 */   172, 1107,  243, 1103,  244,  158,   18,  245,  345,  247,
+
 /*  1990 */  1017, 1017, 1019, 1020,   27,  261, 1096,  193, 1226,  489,
+
 /*  2000 */   194,   37,  366,  851,  494,  251,  195,  506,   92,   19,
+
 /*  2010 */   498,  358,   20,  503,  881,  361,   94,  894,  305,  159,
+
 /*  2020 */   513,   39,   95, 1174,  160, 1056,  966, 1143,   96,  174,
+
 /*  2030 */  1142,  225,  280,  282,  198,  960,  113, 1164, 1160,  260,
+
 /*  2040 */    21,   22,   23, 1162, 1168, 1167, 1148,   24,   33,   25,
+
 /*  2050 */   202,  542,   26,  100, 1071,  102, 1057,  103,    7, 1055,
+
 /*  2060 */  1059, 1113, 1060, 1112,  266,  267,   28,   40,  390, 1022,
+
 /*  2070 */   863,  112,   29,  564, 1182, 1181,  268,  176,  143,  925,
+
 /*  2080 */  1242, 1242, 1242, 1242, 1242, 1242, 1242, 1242, 1242, 1242,
+
 /*  2090 */  1242, 1242, 1242, 1242,  269, 1602, 1242, 1601,
};
static const YYCODETYPE yy_lookahead[] = {
-
 /*     0 */   193,  223,  193,  225,  193,  193,  193,  274,  275,  276,
-
 /*    10 */   193,  233,  219,  193,  274,  275,  276,  206,  206,   19,
-
 /*    20 */   193,  219,  216,  216,  217,  216,  217,  193,  295,  216,
-
 /*    30 */   217,   31,  205,  216,  217,  193,  216,  217,  213,   39,
-
 /*    40 */   228,  193,  230,   43,   44,   45,   46,   47,   48,   49,
+
 /*     0 */   193,  193,  193,  274,  275,  276,  193,  274,  275,  276,
+
 /*    10 */   193,  223,  219,  225,  206,  210,  211,  212,  193,   19,
+
 /*    20 */   219,  233,  216,  216,  217,  216,  217,  193,  295,  216,
+
 /*    30 */   217,   31,  193,  216,  217,  193,  228,  213,  230,   39,
+
 /*    40 */   206,  216,  217,   43,   44,   45,   46,   47,   48,   49,
 /*    50 */    50,   51,   52,   53,   54,   55,   56,   57,  193,   19,
 /*    60 */   185,  186,  187,  188,  189,  190,  253,  274,  275,  276,
-
 /*    70 */   195,  193,  197,  253,  216,  262,  274,  275,  276,  204,
-
 /*    80 */   238,  204,  262,   43,   44,   45,   46,   47,   48,   49,
-
 /*    90 */    50,   51,   52,   53,   54,   55,   56,   57,  264,  274,
-
 /*   100 */   275,  276,  102,  103,  104,  105,  106,  107,  108,  109,
+
 /*    70 */   195,  193,  197,  193,  261,  274,  275,  276,  253,  204,
+
 /*    80 */   238,  204,   81,   43,   44,   45,   46,   47,   48,   49,
+
 /*    90 */    50,   51,   52,   53,   54,   55,   56,   57,  274,  275,
+
 /*   100 */   276,  262,  102,  103,  104,  105,  106,  107,  108,  109,
 /*   110 */   110,  111,  112,  113,  239,  240,  239,  240,  210,  211,
 /*   120 */   212,  314,  315,  314,   59,  316,   86,  252,   88,  252,
-
 /*   130 */    19,  314,  315,  256,  257,  309,   25,   72,  296,  313,
-
 /*   140 */   193,  266,  102,  103,  104,  105,  106,  107,  108,  109,
+
 /*   130 */    19,  314,  315,  256,  257,  113,   25,   72,  296,  138,
+
 /*   140 */   139,  266,  102,  103,  104,  105,  106,  107,  108,  109,
 /*   150 */   110,  111,  112,  113,   43,   44,   45,   46,   47,   48,
 /*   160 */    49,   50,   51,   52,   53,   54,   55,   56,   57,   81,
-
 /*   170 */   292,   59,  307,  298,  108,  109,  110,  111,  112,  113,
-
 /*   180 */    69,  116,  117,  118,   72,  128,  129,  193,  241,   22,
+
 /*   170 */   292,   59,  292,  298,  108,  109,  110,  111,  112,  113,
+
 /*   180 */    69,  116,  117,  118,   72,  106,  107,  193,  111,  112,
 /*   190 */   113,   54,   55,   56,   57,   58,  102,  103,  104,  105,
-
 /*   200 */   106,  107,  108,  109,  110,  111,  112,  113,  120,  193,
-
 /*   210 */   216,  217,   25,  102,  103,  104,  105,  106,  107,  108,
+
 /*   200 */   106,  107,  108,  109,  110,  111,  112,  113,  120,   25,
+
 /*   210 */   216,  217,  145,  102,  103,  104,  105,  106,  107,  108,
 /*   220 */   109,  110,  111,  112,  113,  231,  138,  139,  116,  117,
-
 /*   230 */   118,  106,  107,   19,  216,   54,   55,   56,   57,  102,
+
 /*   230 */   118,  164,  153,   19,  155,   54,   55,   56,   57,  102,
 /*   240 */   103,  104,  105,  106,  107,  108,  109,  110,  111,  112,
-
 /*   250 */   113,  304,   25,   46,   47,   48,   49,   43,   44,   45,
+
 /*   250 */   113,  128,  129,   46,   47,   48,   49,   43,   44,   45,
 /*   260 */    46,   47,   48,   49,   50,   51,   52,   53,   54,   55,
-
 /*   270 */    56,   57,  193,  106,  107,   59,  193,   19,  153,  263,
-
 /*   280 */   155,   67,   24,  102,  103,  104,  105,  106,  107,  108,
-
 /*   290 */   109,  110,  111,  112,  113,  216,  217,   59,  230,  216,
+
 /*   270 */    56,   57,  216,  193,   25,   59,  193,   19,  165,  166,
+
 /*   280 */   193,   67,   24,  102,  103,  104,  105,  106,  107,  108,
+
 /*   290 */   109,  110,  111,  112,  113,   73,  216,  217,   59,  216,
 /*   300 */   217,   43,   44,   45,   46,   47,   48,   49,   50,   51,
 /*   310 */    52,   53,   54,   55,   56,   57,  102,  103,  104,  105,
-
 /*   320 */   106,  107,  108,  109,  110,  111,  112,  113,  121,  142,
-
 /*   330 */    59,  193,  116,  117,  118,  119,  253,  204,  122,  123,
-
 /*   340 */   124,   19,   20,   81,   22,  262,  108,   19,  132,  165,
-
 /*   350 */   166,  193,   24,  126,  116,  117,  118,  278,   36,  193,
+
 /*   320 */   106,  107,  108,  109,  110,  111,  112,  113,  121,  145,
+
 /*   330 */    59,  193,  116,  117,  118,  119,  273,  204,  122,  123,
+
 /*   340 */   124,   19,   20,  134,   22,  136,  137,   19,  132,  127,
+
 /*   350 */   128,  129,   24,   22,   23,  116,  117,  118,   36,  193,
 /*   360 */   102,  103,  104,  105,  106,  107,  108,  109,  110,  111,
-
 /*   370 */   112,  113,  239,  240,  216,  217,  215,  106,  107,  241,
+
 /*   370 */   112,  113,  239,  240,  311,  312,  215,  106,  107,  241,
 /*   380 */    19,   59,  216,  217,  223,  252,  115,  116,  117,  118,
-
 /*   390 */    73,  120,   26,   71,  193,   22,  193,  231,   81,  128,
-
 /*   400 */   138,  139,  269,   81,   43,   44,   45,   46,   47,   48,
-
 /*   410 */    49,   50,   51,   52,   53,   54,   55,   56,   57,  216,
-
 /*   420 */   217,  198,  100,   95,  153,   59,  155,  193,  106,  107,
-
 /*   430 */   235,  236,   59,  193,  193,  249,  114,  251,  116,  117,
-
 /*   440 */   118,  113,  304,  121,  127,  204,  193,  119,  120,  121,
-
 /*   450 */   122,  123,  124,  125,  193,  138,  139,  216,  217,  131,
-
 /*   460 */   138,  139,  193,  102,  103,  104,  105,  106,  107,  108,
+
 /*   390 */   151,  120,   26,   71,  193,  308,  309,  193,  149,  128,
+
 /*   400 */   313,  216,  269,   81,   43,   44,   45,   46,   47,   48,
+
 /*   410 */    49,   50,   51,   52,   53,   54,   55,   56,   57,  253,
+
 /*   420 */   216,  217,  100,   95,  153,   59,  155,  261,  106,  107,
+
 /*   430 */    25,  193,  101,  193,  193,  231,  114,   25,  116,  117,
+
 /*   440 */   118,  113,  304,  121,  193,  204,   59,  119,  120,  121,
+
 /*   450 */   122,  123,  124,  125,  216,  217,  193,  216,  217,  131,
+
 /*   460 */   138,  139,  230,  102,  103,  104,  105,  106,  107,  108,
 /*   470 */   109,  110,  111,  112,  113,  153,  154,  155,  156,  157,
-
 /*   480 */   239,  240,  116,  117,  118,   76,  193,  193,   19,  116,
-
 /*   490 */   117,  118,   23,  252,  253,  193,   87,  204,   89,  238,
-
 /*   500 */   193,   92,  268,  262,  281,  203,  193,  205,  285,  216,
+
 /*   480 */   239,  240,  116,  117,  118,   76,  193,   23,   19,   25,
+
 /*   490 */    22,  253,   23,  252,  253,  108,   87,  204,   89,  261,
+
 /*   500 */   198,   92,  261,  116,  117,  118,  193,  306,  307,  216,
 /*   510 */   217,  150,   43,   44,   45,   46,   47,   48,   49,   50,
-
 /*   520 */    51,   52,   53,   54,   55,   56,   57,  193,  193,  216,
-
 /*   530 */   217,   19,  239,  240,   59,   23,  106,  107,  108,  109,
-
 /*   540 */   110,  111,  112,  113,  231,  252,  253,  193,  308,  309,
-
 /*   550 */   193,  145,   59,  313,  145,   43,   44,   45,   46,   47,
+
 /*   520 */    51,   52,   53,   54,   55,   56,   57,   59,  193,  216,
+
 /*   530 */   217,   19,  239,  240,  283,   23,  106,  107,  108,  109,
+
 /*   540 */   110,  111,  112,  113,   73,  252,  253,  142,  308,  309,
+
 /*   550 */   138,  139,   81,  313,  145,   43,   44,   45,   46,   47,
 /*   560 */    48,   49,   50,   51,   52,   53,   54,   55,   56,   57,
-
 /*   570 */   164,  102,  103,  104,  105,  106,  107,  108,  109,  110,
-
 /*   580 */   111,  112,  113,  119,  193,  193,  122,  123,  124,  193,
-
 /*   590 */   283,  116,  117,  118,  235,  236,  132,   59,  241,  264,
-
 /*   600 */    59,  193,   19,   23,  193,   25,   23,  216,  217,  116,
-
 /*   610 */   117,  118,  216,  217,  102,  103,  104,  105,  106,  107,
+
 /*   570 */   307,  102,  103,  104,  105,  106,  107,  108,  109,  110,
+
 /*   580 */   111,  112,  113,  281,  116,  117,  118,  285,   23,  193,
+
 /*   590 */    25,  119,   59,  193,  122,  123,  124,   59,  127,  203,
+
 /*   600 */    59,  205,   19,  268,  132,   25,   23,   22,  193,  138,
+
 /*   610 */   139,  249,  204,  251,  102,  103,  104,  105,  106,  107,
 /*   620 */   108,  109,  110,  111,  112,  113,   43,   44,   45,   46,
 /*   630 */    47,   48,   49,   50,   51,   52,   53,   54,   55,   56,
-
 /*   640 */    57,   19,  308,  309,  151,   23,   25,  313,  135,  253,
-
 /*   650 */    21,  193,  241,  140,  116,  117,  118,  116,  117,  118,
-
 /*   660 */   268,  304,   22,  301,  302,   43,   44,   45,   46,   47,
+
 /*   640 */    57,   19,   22,   23,   59,   23,   25,  239,  240,  116,
+
 /*   650 */   117,  118,  193,   11,  116,  117,  118,  116,  117,  118,
+
 /*   660 */   252,  269,   22,  193,   15,   43,   44,   45,   46,   47,
 /*   670 */    48,   49,   50,   51,   52,   53,   54,   55,   56,   57,
-
 /*   680 */   193,  143,  193,  193,  143,  102,  103,  104,  105,  106,
+
 /*   680 */   273,  143,  193,  118,  143,  102,  103,  104,  105,  106,
 /*   690 */   107,  108,  109,  110,  111,  112,  113,   76,  118,   59,
-
 /*   700 */   292,  211,  212,  216,  217,  216,  217,   73,  193,   80,
-
 /*   710 */    89,   25,   19,   92,  193,  304,   23,   22,  231,  193,
-
 /*   720 */   231,  193,   22,  143,  102,  103,  104,  105,  106,  107,
+
 /*   700 */   241,  116,  117,  118,  304,  216,  217,  292,  143,   60,
+
 /*   710 */    89,  241,   19,   92,  193,  193,   23,   22,  311,  312,
+
 /*   720 */   231,  101,   22,  143,  102,  103,  104,  105,  106,  107,
 /*   730 */   108,  109,  110,  111,  112,  113,   43,   44,   45,   46,
 /*   740 */    47,   48,   49,   50,   51,   52,   53,   54,   55,   56,
-
 /*   750 */    57,   19,  123,  193,   59,   23,  116,  117,  118,   59,
-
 /*   760 */   193,  127,  128,  129,  306,  307,  210,  211,  212,  193,
-
 /*   770 */    22,  111,  112,  113,  284,   43,   44,   45,   46,   47,
+
 /*   750 */    57,   19,  193,  193,   59,   23,  116,  117,  118,   59,
+
 /*   760 */   201,   21,  241,  304,   22,  206,  127,  128,  129,  193,
+
 /*   770 */   128,  129,  235,  236,  304,   43,   44,   45,   46,   47,
 /*   780 */    48,   49,   50,   51,   52,   53,   54,   55,   56,   57,
-
 /*   790 */   161,  193,  216,  217,  268,  102,  103,  104,  105,  106,
-
 /*   800 */   107,  108,  109,  110,  111,  112,  113,   59,  193,  193,
-
 /*   810 */   193,  116,  117,  118,  216,  217,  116,  117,  118,  304,
-
 /*   820 */   239,  240,   19,  263,  138,  139,   23,  211,  212,  231,
-
 /*   830 */   263,  216,  217,  252,  102,  103,  104,  105,  106,  107,
+
 /*   790 */    22,  193,  216,  217,  193,  102,  103,  104,  105,  106,
+
 /*   800 */   107,  108,  109,  110,  111,  112,  113,  231,  193,  193,
+
 /*   810 */   193,  116,  117,  118,  216,  217,  116,  117,  118,  226,
+
 /*   820 */    80,  193,   19,  235,  236,  304,   23,  211,  212,  231,
+
 /*   830 */   204,  216,  217,  205,  102,  103,  104,  105,  106,  107,
 /*   840 */   108,  109,  110,  111,  112,  113,   43,   44,   45,   46,
 /*   850 */    47,   48,   49,   50,   51,   52,   53,   54,   55,   56,
-
 /*   860 */    57,   19,  193,   11,  116,  117,  118,  240,  253,  193,
-
 /*   870 */   201,  239,  240,  193,  134,  206,  136,  137,  193,  252,
-
 /*   880 */   193,  264,  193,  193,  252,   43,   44,   45,   46,   47,
+
 /*   860 */    57,   19,  193,  123,   76,  239,  240,  193,  253,  239,
+
 /*   870 */   240,  239,  240,  193,  106,  107,  193,   89,  252,  193,
+
 /*   880 */    92,   59,  252,  141,  252,   43,   44,   45,   46,   47,
 /*   890 */    48,   49,   50,   51,   52,   53,   54,   55,   56,   57,
-
 /*   900 */   284,  216,  217,  216,  217,  102,  103,  104,  105,  106,
-
 /*   910 */   107,  108,  109,  110,  111,  112,  113,  193,  231,  193,
-
 /*   920 */   187,  188,  189,  190,  127,  128,  129,  238,  195,  193,
-
 /*   930 */   197,   16,   19,    7,    8,    9,  193,  204,  253,  193,
-
 /*   940 */   216,  217,  216,  217,  102,  103,  104,  105,  106,  107,
+
 /*   900 */   284,  161,  216,  217,  193,  102,  103,  104,  105,  106,
+
 /*   910 */   107,  108,  109,  110,  111,  112,  113,  231,  193,   16,
+
 /*   920 */   187,  188,  189,  190,    7,    8,    9,  309,  195,   25,
+
 /*   930 */   197,  313,   19,  127,  128,  129,  262,  204,   22,  117,
+
 /*   940 */    24,  216,  217,  263,  102,  103,  104,  105,  106,  107,
 /*   950 */   108,  109,  110,  111,  112,  113,   43,   44,   45,   46,
 /*   960 */    47,   48,   49,   50,   51,   52,   53,   54,   55,   56,
-
 /*   970 */    57,  213,  239,  240,  193,   76,   19,  188,  232,  190,
-
 /*   980 */   128,  129,  292,  193,  195,  252,  197,   46,   89,  138,
-
 /*   990 */   139,   92,   77,  204,   79,  193,  269,  216,  217,  266,
+
 /*   970 */    57,  193,  239,  240,  193,   59,   19,  188,  253,  190,
+
 /*   980 */    77,  226,   79,  193,  195,  252,  197,  193,   19,  301,
+
 /*   990 */   302,  193,  193,  204,  216,  217,  226,  216,  217,  266,
 /*  1000 */   204,  159,   45,   46,   47,   48,   49,   50,   51,   52,
 /*  1010 */    53,   54,   55,   56,   57,  102,  103,  104,  105,  106,
 /*  1020 */   107,  108,  109,  110,  111,  112,  113,   12,  239,  240,
-
 /*  1030 */   193,  298,   22,   23,  253,  239,  240,  127,  128,  129,
-
 /*  1040 */   238,  252,   27,  193,  286,  204,  193,  204,  252,  291,
-
 /*  1050 */   193,  273,   22,   23,  100,  266,  115,   42,  268,  102,
+
 /*  1030 */   232,  298,  238,  117,  253,  239,  240,  238,  259,  260,
+
 /*  1040 */   193,  252,   27,   31,  193,  193,  142,  204,  252,  193,
+
 /*  1050 */   193,   39,  262,  193,  100,  266,  278,   42,  204,  102,
 /*  1060 */   103,  104,  105,  106,  107,  108,  109,  110,  111,  112,
-
 /*  1070 */   113,  117,  159,  216,  217,  121,  161,   19,   63,  193,
-
 /*  1080 */   239,  240,  239,  240,   12,  208,  209,  298,   73,  311,
-
 /*  1090 */   312,  238,   19,  252,   25,  252,   22,   24,   24,   27,
-
 /*  1100 */   193,  264,  216,  217,   46,  208,  209,  153,  154,  155,
-
 /*  1110 */   253,  101,   19,   23,   42,   25,   43,   44,   45,   46,
+
 /*  1070 */   113,  117,  159,  216,  217,  121,  216,  217,   63,  193,
+
 /*  1080 */   193,  193,  239,  240,  115,  116,  193,  298,   73,  238,
+
 /*  1090 */   238,  231,   19,  239,  240,  252,   22,   24,  211,  212,
+
 /*  1100 */    24,  193,  216,  217,  216,  217,  252,  153,  154,  155,
+
 /*  1110 */   253,   16,   19,  144,  213,  268,   43,   44,   45,   46,
 /*  1120 */    47,   48,   49,   50,   51,   52,   53,   54,   55,   56,
-
 /*  1130 */    57,  101,   19,   59,   25,   63,   43,   44,   45,   46,
+
 /*  1130 */    57,  238,   19,   59,  193,   59,   43,   44,   45,   46,
 /*  1140 */    47,   48,   49,   50,   51,   52,   53,   54,   55,   56,
-
 /*  1150 */    57,   22,   23,  115,   25,   24,   43,   44,   45,   46,
+
 /*  1150 */    57,   22,   23,  193,   25,  193,   43,   44,   45,   46,
 /*  1160 */    47,   48,   49,   50,   51,   52,   53,   54,   55,   56,
-
 /*  1170 */    57,   22,   23,  115,   25,  102,  103,  104,  105,  106,
-
 /*  1180 */   107,  108,  109,  110,  111,  112,  113,  118,  150,  131,
-
 /*  1190 */    59,  117,   22,  273,  193,  102,  103,  104,  105,  106,
-
 /*  1200 */   107,  108,  109,  110,  111,  112,  113,  204,   66,  204,
-
 /*  1210 */    35,  204,  143,  213,  193,  102,  103,  104,  105,  106,
-
 /*  1220 */   107,  108,  109,  110,  111,  112,  113,   85,  193,   59,
-
 /*  1230 */   101,  311,  312,   16,  193,   19,   94,  216,  217,  238,
-
 /*  1240 */   193,   66,  239,  240,  239,  240,  239,  240,  117,   74,
-
 /*  1250 */   101,  216,  217,  193,  193,  252,  193,  252,  149,  252,
+
 /*  1170 */    57,  284,   77,  193,   79,  102,  103,  104,  105,  106,
+
 /*  1180 */   107,  108,  109,  110,  111,  112,  113,  286,  193,  193,
+
 /*  1190 */   193,  117,  291,  117,  232,  102,  103,  104,  105,  106,
+
 /*  1200 */   107,  108,  109,  110,  111,  112,  113,  204,   22,   23,
+
 /*  1210 */    66,   25,  216,  217,   35,  102,  103,  104,  105,  106,
+
 /*  1220 */   107,  108,  109,  110,  111,  112,  113,  193,  268,   85,
+
 /*  1230 */   101,  193,  309,  309,  240,   19,  313,  313,   94,  208,
+
 /*  1240 */   209,  193,  239,  240,  193,   66,  252,   19,  268,  244,
+
 /*  1250 */   216,  217,  193,   74,  213,  252,  161,   19,  263,  254,
 /*  1260 */    44,   45,   46,   47,   48,   49,   50,   51,   52,   53,
-
 /*  1270 */    54,   55,   56,   57,  193,  193,  193,    5,   59,  216,
-
 /*  1280 */   217,   25,   10,   11,   12,   13,   14,  117,  146,   17,
-
 /*  1290 */   193,  291,  193,  232,   77,   76,   79,  216,  217,  216,
-
 /*  1300 */   217,   31,   30,  309,   32,  130,   87,  313,   89,   39,
-
 /*  1310 */   193,   92,   40,  216,  217,  216,  217,  108,  102,  103,
+
 /*  1270 */    54,   55,   56,   57,  193,  216,  217,    5,   59,  193,
+
 /*  1280 */    19,  244,   10,   11,   12,   13,   14,  101,  309,   17,
+
 /*  1290 */   146,  254,  313,  193,  193,   76,  115,  216,  217,  309,
+
 /*  1300 */    12,  263,   30,  313,   32,   46,   87,   46,   89,  130,
+
 /*  1310 */   193,   92,   40,   22,  263,   27,  216,  217,  102,  103,
 /*  1320 */   104,  105,  106,  107,  108,  109,  110,  111,  112,  113,
-
 /*  1330 */   299,  300,  193,  216,  217,  116,  117,  118,   19,   20,
-
 /*  1340 */   193,   22,   70,  309,  135,  193,  264,  313,  193,  140,
-
 /*  1350 */    78,  193,  226,   81,   59,   36,  193,  309,  193,   29,
-
 /*  1360 */   193,  313,  193,   33,  145,  193,   59,   48,  216,  217,
-
 /*  1370 */    98,  216,  217,  193,  216,  217,  193,  244,   59,  216,
-
 /*  1380 */   217,  216,  217,  216,  217,  216,  217,  254,  216,  217,
-
 /*  1390 */    71,  193,  244,  193,  193,   65,  216,  217,  193,  216,
-
 /*  1400 */   217,  145,  254,  244,   85,  133,   15,  100,  193,   90,
-
 /*  1410 */   138,  139,  117,  254,  216,  217,  216,  217,  193,  100,
-
 /*  1420 */   193,  216,  217,  116,  117,  106,  107,   19,  121,  193,
-
 /*  1430 */   193,  216,  217,  114,  162,  116,  117,  118,  244,  244,
-
 /*  1440 */   121,  216,  217,  216,  217,  193,  309,  129,  254,  254,
-
 /*  1450 */   313,   60,  216,  217,   19,  256,  257,  193,  120,  121,
-
 /*  1460 */   153,  154,  155,  149,  150,   25,   24,   99,  216,  217,
-
 /*  1470 */   152,  193,  153,  154,  155,  156,  157,    0,    1,    2,
-
 /*  1480 */   216,  217,    5,   22,  158,   24,  160,   10,   11,   12,
-
 /*  1490 */    13,   14,  193,   23,   17,   25,  193,   19,   20,  193,
-
 /*  1500 */    22,  133,  193,   22,   22,  193,   22,   30,  193,   32,
-
 /*  1510 */    19,   20,  129,   22,   36,  216,  217,   40,  193,  216,
-
 /*  1520 */   217,  193,  216,  217,  116,  216,  217,   36,  216,  217,
-
 /*  1530 */   193,  216,  217,  193,   53,  152,  193,   59,   23,   19,
-
 /*  1540 */    25,  216,  217,   61,  216,  217,   23,   70,   25,   71,
-
 /*  1550 */    59,  116,  193,  216,  217,   78,  216,  217,   81,  216,
-
 /*  1560 */   217,   59,   71,   85,  193,   23,  193,   25,   90,   23,
-
 /*  1570 */    23,   25,   25,    7,    8,   98,   85,  193,  100,  193,
-
 /*  1580 */    59,   90,  142,  141,  106,  107,  193,  216,  217,  216,
-
 /*  1590 */   217,  100,  114,  193,  116,  117,  118,  106,  107,  121,
-
 /*  1600 */   216,  217,  216,  217,  193,  114,  193,  116,  117,  118,
-
 /*  1610 */   133,   23,  121,   25,  121,  138,  139,   97,   23,  117,
-
 /*  1620 */    25,   23,  193,   25,  131,  141,  193,  216,  217,   59,
-
 /*  1630 */   193,  153,  154,  155,  156,  157,  226,  193,  117,  162,
-
 /*  1640 */    23,   23,   25,   25,  153,  154,  155,  156,  157,    1,
-
 /*  1650 */     2,   83,   84,    5,   19,   20,  226,   22,   10,   11,
-
 /*  1660 */    12,   13,   14,  258,  153,   17,  155,  153,   23,  155,
-
 /*  1670 */    25,   36,   23,  193,   25,  255,  193,  236,   30,  193,
-
 /*  1680 */    32,   19,   20,  193,   22,  193,  288,  117,   40,  193,
-
 /*  1690 */   318,  193,  193,  193,   59,  242,  193,  193,   36,  193,
-
 /*  1700 */   193,  193,  287,  255,  255,  255,   71,  255,  243,  214,
-
 /*  1710 */   191,  297,  267,  245,  271,  259,  259,  293,   70,  246,
-
 /*  1720 */   246,   59,  267,  229,  245,  271,   78,  293,  259,   81,
-
 /*  1730 */   271,  271,  220,   71,  225,  100,  219,  219,  249,  196,
-
 /*  1740 */   243,  106,  107,  108,  219,   60,   98,  280,  297,  114,
-
 /*  1750 */   249,  116,  117,  118,  141,  245,  121,  200,  200,  297,
-
 /*  1760 */    38,  200,  100,  151,  150,  294,  294,   22,  106,  107,
-
 /*  1770 */   283,   43,  234,   18,  237,  200,  114,  272,  116,  117,
-
 /*  1780 */   118,  133,  237,  121,   18,  237,  138,  139,  153,  154,
-
 /*  1790 */   155,  156,  157,  237,  270,  199,   19,   20,  246,   22,
-
 /*  1800 */   149,  272,  272,  270,  200,  246,  234,  234,  246,  246,
-
 /*  1810 */   162,  158,  290,   36,  199,  153,  154,  155,  156,  157,
-
 /*  1820 */    62,  289,  200,  199,   22,  200,  221,  199,  221,  200,
-
 /*  1830 */   199,  115,   64,  227,  218,   22,   59,  218,  218,  126,
-
 /*  1840 */   165,   24,  113,  312,  218,  224,  305,  224,   71,  282,
-
 /*  1850 */   144,  221,  220,  282,  218,  218,  218,  115,  261,  260,
-
 /*  1860 */   227,  221,  261,  260,  200,   91,  317,  317,   82,  261,
-
 /*  1870 */   260,  148,  261,   22,  265,  145,  200,  100,  158,  277,
-
 /*  1880 */   147,  146,   25,  106,  107,  202,   13,  260,  194,  250,
-
 /*  1890 */   249,  114,  248,  116,  117,  118,  250,  247,  121,  265,
-
 /*  1900 */   246,  194,    6,  192,  192,  207,  192,  207,  213,  213,
-
 /*  1910 */   213,  222,  213,  222,    4,  214,  214,  213,    3,   22,
-
 /*  1920 */   163,  279,  207,   15,   23,   16,   23,  139,  130,  151,
-
 /*  1930 */   153,  154,  155,  156,  157,  142,   25,   24,   20,  144,
-
 /*  1940 */    16,    1,  142,   61,  130,  130,  303,  151,   53,   37,
-
 /*  1950 */   303,   53,  300,   53,  130,   53,  116,   34,    1,  141,
-
 /*  1960 */     5,   22,  115,   25,  161,   75,   41,   68,  141,  115,
-
 /*  1970 */    24,   20,   68,   19,  131,  125,   23,   96,   22,   59,
-
 /*  1980 */    22,   67,   22,   22,   67,   22,   24,   28,   67,   37,
-
 /*  1990 */    23,  149,   22,   25,   23,   23,   23,   23,   22,  141,
-
 /*  2000 */    23,   23,   97,  116,   22,  143,   25,   88,   75,   34,
-
 /*  2010 */    44,   75,   86,   34,   34,   93,   23,   34,   34,   34,
-
 /*  2020 */    22,   24,   34,   22,   25,   25,   23,   23,   23,   23,
-
 /*  2030 */    23,   11,   23,   25,   22,   22,   25,   23,   23,   22,
-
 /*  2040 */    22,  135,   15,    1,   25,   23,    1,  319,  141,  319,
-
 /*  2050 */   319,  319,  319,  319,  319,  319,  319,  141,  319,  319,
-
 /*  2060 */   319,  319,  319,  319,  319,  319,  319,  319,  141,  141,
-
 /*  2070 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
+
 /*  1330 */    42,  150,  291,  216,  217,  116,  117,  118,   19,   20,
+
 /*  1340 */   193,   22,   70,  260,  116,  193,   24,  264,  193,  263,
+
 /*  1350 */    78,   63,   61,   81,  116,   36,  193,  260,  193,   29,
+
 /*  1360 */   193,  264,  193,   33,  145,  193,   59,   48,  216,  217,
+
 /*  1370 */    98,  216,  217,  193,  115,  193,  115,  193,   59,  216,
+
 /*  1380 */   217,  216,  217,  216,  217,  216,  217,  255,  216,  217,
+
 /*  1390 */    71,  193,  131,  193,   25,   65,  216,  217,  216,  217,
+
 /*  1400 */   216,  217,  208,  209,   85,  133,  193,  100,  193,   90,
+
 /*  1410 */   138,  139,  138,  139,  216,  217,  216,  217,  193,  100,
+
 /*  1420 */   193,  108,  135,  116,  117,  106,  107,  140,  121,  216,
+
 /*  1430 */   217,  216,  217,  114,  162,  116,  117,  118,  299,  300,
+
 /*  1440 */   121,  216,  217,  216,  217,  193,  244,  193,  135,  244,
+
 /*  1450 */   193,  256,  257,  140,  244,  193,  254,  193,  193,  254,
+
 /*  1460 */   153,  154,  155,  141,  254,  149,  150,  258,  216,  217,
+
 /*  1470 */   216,  217,  153,  154,  155,  156,  157,    0,    1,    2,
+
 /*  1480 */   216,  217,    5,  115,  158,  193,  160,   10,   11,   12,
+
 /*  1490 */    13,   14,  193,   59,   17,  126,  193,   19,   20,  129,
+
 /*  1500 */    22,  193,   22,   22,   24,  193,   23,   30,   25,   32,
+
 /*  1510 */    19,   20,  144,   22,   36,  216,  217,   40,  193,  216,
+
 /*  1520 */   217,  193,  152,  129,  216,  217,  193,   36,  216,  217,
+
 /*  1530 */   193,   99,  193,  193,   53,  193,  193,   59,   23,  193,
+
 /*  1540 */    25,  216,  217,  193,  216,  217,  152,   70,   59,   71,
+
 /*  1550 */    59,  117,  193,  216,  217,   78,  216,  217,   81,  216,
+
 /*  1560 */   217,  318,   71,   85,  193,  133,  193,  193,   90,   23,
+
 /*  1570 */    23,   25,   25,  120,  121,   98,   85,  193,  100,  193,
+
 /*  1580 */    23,   90,   25,  121,  106,  107,   19,  216,  217,  216,
+
 /*  1590 */   217,  100,  114,  131,  116,  117,  118,  106,  107,  121,
+
 /*  1600 */   216,  217,  216,  217,  193,  114,  117,  116,  117,  118,
+
 /*  1610 */   133,  193,  121,  193,  193,  138,  139,  193,   23,  193,
+
 /*  1620 */    25,   23,   23,   25,   25,    7,    8,  216,  217,  193,
+
 /*  1630 */   193,  153,  154,  155,  156,  157,  216,  217,  193,  162,
+
 /*  1640 */   216,  217,  216,  217,  153,  154,  155,  156,  157,    1,
+
 /*  1650 */     2,  193,  193,    5,   19,   20,   59,   22,   10,   11,
+
 /*  1660 */    12,   13,   14,  193,   97,   17,  193,   23,  193,   25,
+
 /*  1670 */   288,   36,  193,  242,  216,  217,  236,   23,   30,   25,
+
 /*  1680 */    32,   19,   20,   23,   22,   25,  216,  217,   40,  216,
+
 /*  1690 */   217,  216,  217,  193,   59,  216,  217,  193,   36,   83,
+
 /*  1700 */    84,  153,  153,  155,  155,   23,   71,   25,   23,  193,
+
 /*  1710 */    25,  193,  193,  193,  117,  193,  193,  193,   70,  193,
+
 /*  1720 */   193,   59,  193,  255,  255,  287,   78,  255,  243,   81,
+
 /*  1730 */   191,  255,  297,   71,  271,  100,  293,  245,  267,  214,
+
 /*  1740 */   246,  106,  107,  108,  246,  271,   98,  245,  293,  114,
+
 /*  1750 */   220,  116,  117,  118,  267,  271,  121,  271,  225,  219,
+
 /*  1760 */   229,  219,  100,  219,  259,  259,  259,  259,  106,  107,
+
 /*  1770 */   249,  196,   60,  280,  141,  243,  114,  249,  116,  117,
+
 /*  1780 */   118,  133,  245,  121,  200,  297,  138,  139,  153,  154,
+
 /*  1790 */   155,  156,  157,  297,  200,   38,   19,   20,  151,   22,
+
 /*  1800 */   200,  150,  140,  294,  294,   22,  272,   43,  234,   18,
+
 /*  1810 */   162,  270,  200,   36,  237,  153,  154,  155,  156,  157,
+
 /*  1820 */   237,  283,  237,  237,   18,  199,  149,  246,  272,  270,
+
 /*  1830 */   272,  200,  158,  246,  246,  234,   59,  234,  246,  199,
+
 /*  1840 */   290,   62,  289,  200,  199,   22,  221,  115,   71,  200,
+
 /*  1850 */   200,  199,  199,  221,  218,  218,   19,   20,   64,   22,
+
 /*  1860 */   218,  227,   22,  224,  126,  224,  165,  221,   24,  305,
+
 /*  1870 */   200,  113,  312,   36,  218,  220,  218,  100,  282,  218,
+
 /*  1880 */    91,  218,  317,  106,  107,  221,  227,  282,  317,   82,
+
 /*  1890 */   148,  114,  265,  116,  117,  118,   59,  145,  121,   22,
+
 /*  1900 */   277,  158,  200,  265,   25,  202,  147,  250,   71,  279,
+
 /*  1910 */    13,  146,  194,  194,  249,  248,  250,  140,  247,  246,
+
 /*  1920 */     6,  192,  192,  192,  303,  303,  213,  207,  300,  213,
+
 /*  1930 */   153,  154,  155,  156,  157,  213,  213,  100,  213,  222,
+
 /*  1940 */   207,  214,  214,  106,  107,    4,  222,  207,    3,   22,
+
 /*  1950 */   163,  114,   15,  116,  117,  118,   16,   23,  121,   23,
+
 /*  1960 */   139,  151,  130,   25,  142,   16,   24,   20,  144,    1,
+
 /*  1970 */   142,  130,  130,   61,   53,   53,   37,  151,   53,   53,
+
 /*  1980 */   130,  116,   34,    1,  141,    5,   22,  115,  161,  141,
+
 /*  1990 */   153,  154,  155,  156,  157,   25,   68,   68,   75,   41,
+
 /*  2000 */   115,   24,  131,   20,   19,  125,   22,   96,   22,   22,
+
 /*  2010 */    67,   23,   22,   67,   59,   24,   22,   28,   67,   23,
+
 /*  2020 */    22,   22,  149,   23,   23,   23,  116,   23,   25,   37,
+
 /*  2030 */    97,  141,   23,   23,   22,  143,   25,   75,   88,   34,
+
 /*  2040 */    34,   34,   34,   86,   75,   93,   23,   34,   22,   34,
+
 /*  2050 */    25,   24,   34,   25,   23,  142,   23,  142,   44,   23,
+
 /*  2060 */    23,   23,   11,   23,   25,   22,   22,   22,   15,   23,
+
 /*  2070 */    23,   22,   22,   25,    1,    1,  141,   25,   23,  135,
 /*  2080 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
-
 /*  2090 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
+
 /*  2090 */   319,  319,  319,  319,  141,  141,  319,  141,  319,  319,
 /*  2100 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
 /*  2110 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
 /*  2120 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
@@ -162426,176 +166265,179 @@ static const YYCODETYPE yy_lookahead[] = {
 /*  2220 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
 /*  2230 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
 /*  2240 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
-
 /*  2250 */   319,  319,  319,  319,  319,
+
 /*  2250 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
+
 /*  2260 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
+
 /*  2270 */   319,  319,  319,  319,  319,  319,  319,  319,  319,  319,
+
 /*  2280 */   319,  319,  319,
};
-
#define YY_SHIFT_COUNT    (573)
+
#define YY_SHIFT_COUNT    (575)
#define YY_SHIFT_MIN      (0)
-
#define YY_SHIFT_MAX      (2045)
+
#define YY_SHIFT_MAX      (2074)
static const unsigned short int yy_shift_ofst[] = {
-
 /*     0 */  1648, 1477, 1272,  322,  322,  262, 1319, 1478, 1491, 1662,
-
 /*    10 */  1662, 1662,  317,    0,    0,  214, 1093, 1662, 1662, 1662,
-
 /*    20 */  1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662,
-
 /*    30 */   271,  271, 1219, 1219,  216,   88,  262,  262,  262,  262,
-
 /*    40 */   262,   40,  111,  258,  361,  469,  512,  583,  622,  693,
+
 /*     0 */  1648, 1477, 1272,  322,  322,    1, 1319, 1478, 1491, 1837,
+
 /*    10 */  1837, 1837,  471,    0,    0,  214, 1093, 1837, 1837, 1837,
+
 /*    20 */  1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837,
+
 /*    30 */   271,  271, 1219, 1219,  216,   88,    1,    1,    1,    1,
+
 /*    40 */     1,   40,  111,  258,  361,  469,  512,  583,  622,  693,
 /*    50 */   732,  803,  842,  913, 1073, 1093, 1093, 1093, 1093, 1093,
 /*    60 */  1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093,
 /*    70 */  1093, 1093, 1093, 1113, 1093, 1216,  957,  957, 1635, 1662,
-
 /*    80 */  1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662,
-
 /*    90 */  1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662,
-
 /*   100 */  1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662,
-
 /*   110 */  1662, 1662, 1662, 1662, 1777, 1662, 1662, 1662, 1662, 1662,
-
 /*   120 */  1662, 1662, 1662, 1662, 1662, 1662, 1662, 1662,  137,  181,
-
 /*   130 */   181,  181,  181,  181,   94,  430,   66,   65,  112,  366,
-
 /*   140 */   475,  475,  629, 1058,  475,  475,  125,  125,  475,  686,
-
 /*   150 */   686,  686,  660,  686,   57,  184,  184,   77,   77, 2070,
-
 /*   160 */  2070,  328,  328,  328,  493,  373,  373,  373,  373, 1015,
-
 /*   170 */  1015,  409,  366, 1129, 1149,  475,  475,  475,  475,  475,
-
 /*   180 */   475,  475,  475,  475,  475,  475,  475,  475,  475,  475,
-
 /*   190 */   475,  475,  475,  475,  475,  621,  621,  475,  852,  899,
-
 /*   200 */   899, 1295, 1295,  406,  851, 2070, 2070, 2070, 2070, 2070,
-
 /*   210 */  2070, 2070, 1307,  954,  954,  640,  464,  695,  238,  700,
-
 /*   220 */   538,  541,  748,  475,  475,  475,  475,  475,  475,  475,
-
 /*   230 */   475,  475,  475,  634,  475,  475,  475,  475,  475,  475,
-
 /*   240 */   475,  475,  475,  475,  475,  475, 1175, 1175, 1175,  475,
-
 /*   250 */   475,  475,  580,  475,  475,  475, 1074, 1142,  475,  475,
-
 /*   260 */  1072,  475,  475,  475,  475,  475,  475,  475,  475,  797,
-
 /*   270 */  1330,  740, 1131, 1131, 1131, 1131, 1069,  740,  740, 1209,
-
 /*   280 */   167,  926, 1391, 1038, 1314,  187, 1408, 1314, 1408, 1435,
-
 /*   290 */  1109, 1038, 1038, 1109, 1038,  187, 1435,  227, 1090,  941,
-
 /*   300 */  1270, 1270, 1270, 1408, 1256, 1256, 1326, 1440,  513, 1461,
-
 /*   310 */  1685, 1685, 1613, 1613, 1722, 1722, 1613, 1612, 1614, 1745,
-
 /*   320 */  1728, 1755, 1755, 1755, 1755, 1613, 1766, 1651, 1614, 1614,
-
 /*   330 */  1651, 1745, 1728, 1651, 1728, 1651, 1613, 1766, 1653, 1758,
-
 /*   340 */  1613, 1766, 1802, 1613, 1766, 1613, 1766, 1802, 1716, 1716,
-
 /*   350 */  1716, 1768, 1813, 1813, 1802, 1716, 1713, 1716, 1768, 1716,
-
 /*   360 */  1716, 1675, 1817, 1729, 1729, 1802, 1706, 1742, 1706, 1742,
-
 /*   370 */  1706, 1742, 1706, 1742, 1613, 1774, 1774, 1786, 1786, 1723,
-
 /*   380 */  1730, 1851, 1613, 1720, 1723, 1733, 1735, 1651, 1857, 1873,
-
 /*   390 */  1873, 1896, 1896, 1896, 2070, 2070, 2070, 2070, 2070, 2070,
-
 /*   400 */  2070, 2070, 2070, 2070, 2070, 2070, 2070, 2070, 2070,  207,
-
 /*   410 */   915, 1010, 1030, 1217,  910, 1170, 1470, 1368, 1481, 1442,
-
 /*   420 */  1318, 1383, 1515, 1482, 1523, 1542, 1546, 1547, 1588, 1595,
-
 /*   430 */  1502, 1338, 1566, 1493, 1520, 1521, 1598, 1617, 1568, 1618,
-
 /*   440 */  1511, 1514, 1645, 1649, 1570, 1484, 1910, 1915, 1897, 1757,
-
 /*   450 */  1908, 1909, 1901, 1903, 1788, 1778, 1798, 1911, 1911, 1913,
-
 /*   460 */  1793, 1918, 1795, 1924, 1940, 1800, 1814, 1911, 1815, 1882,
-
 /*   470 */  1912, 1911, 1796, 1895, 1898, 1900, 1902, 1824, 1840, 1923,
-
 /*   480 */  1818, 1957, 1955, 1939, 1847, 1803, 1899, 1938, 1904, 1890,
-
 /*   490 */  1925, 1827, 1854, 1946, 1951, 1954, 1843, 1850, 1956, 1914,
-
 /*   500 */  1958, 1960, 1953, 1961, 1917, 1920, 1962, 1881, 1959, 1963,
-
 /*   510 */  1921, 1952, 1967, 1842, 1970, 1971, 1972, 1973, 1968, 1974,
-
 /*   520 */  1976, 1905, 1858, 1977, 1978, 1887, 1975, 1982, 1862, 1981,
-
 /*   530 */  1979, 1980, 1983, 1984, 1919, 1933, 1926, 1966, 1936, 1922,
-
 /*   540 */  1985, 1993, 1998, 1997, 1999, 2000, 1988, 2003, 1981, 2004,
-
 /*   550 */  2005, 2006, 2007, 2008, 2009, 2001, 2020, 2012, 2013, 2014,
-
 /*   560 */  2015, 2017, 2018, 2011, 1906, 1907, 1916, 1927, 1928, 2019,
-
 /*   570 */  2022, 2027, 2042, 2045,
+
 /*    80 */  1777, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837,
+
 /*    90 */  1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837,
+
 /*   100 */  1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837,
+
 /*   110 */  1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837,
+
 /*   120 */  1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837, 1837,
+
 /*   130 */   137,  181,  181,  181,  181,  181,  181,  181,   94,  430,
+
 /*   140 */    66,   65,  112,  366,  533,  533,  740, 1261,  533,  533,
+
 /*   150 */    79,   79,  533,  412,  412,  412,   77,  412,  123,  113,
+
 /*   160 */   113,   22,   22, 2098, 2098,  328,  328,  328,  239,  468,
+
 /*   170 */   468,  468,  468, 1015, 1015,  409,  366, 1129, 1186,  533,
+
 /*   180 */   533,  533,  533,  533,  533,  533,  533,  533,  533,  533,
+
 /*   190 */   533,  533,  533,  533,  533,  533,  533,  533,  533,  969,
+
 /*   200 */   621,  621,  533,  642,  788,  788, 1228, 1228,  822,  822,
+
 /*   210 */    67, 1274, 2098, 2098, 2098, 2098, 2098, 2098, 2098, 1307,
+
 /*   220 */   954,  954,  585,  472,  640,  387,  695,  538,  541,  700,
+
 /*   230 */   533,  533,  533,  533,  533,  533,  533,  533,  533,  533,
+
 /*   240 */   222,  533,  533,  533,  533,  533,  533,  533,  533,  533,
+
 /*   250 */   533,  533,  533, 1179, 1179, 1179,  533,  533,  533,  565,
+
 /*   260 */   533,  533,  533,  916, 1144,  533,  533, 1288,  533,  533,
+
 /*   270 */   533,  533,  533,  533,  533,  533,  639, 1330,  209, 1076,
+
 /*   280 */  1076, 1076, 1076,  580,  209,  209, 1313,  768,  917,  649,
+
 /*   290 */  1181, 1316,  405, 1316, 1238,  249, 1181, 1181,  249, 1181,
+
 /*   300 */   405, 1238, 1369,  464, 1259, 1012, 1012, 1012, 1368, 1368,
+
 /*   310 */  1368, 1368,  184,  184, 1326,  904, 1287, 1480, 1712, 1712,
+
 /*   320 */  1633, 1633, 1757, 1757, 1633, 1647, 1651, 1783, 1764, 1791,
+
 /*   330 */  1791, 1791, 1791, 1633, 1806, 1677, 1651, 1651, 1677, 1783,
+
 /*   340 */  1764, 1677, 1764, 1677, 1633, 1806, 1674, 1779, 1633, 1806,
+
 /*   350 */  1823, 1633, 1806, 1633, 1806, 1823, 1732, 1732, 1732, 1794,
+
 /*   360 */  1840, 1840, 1823, 1732, 1738, 1732, 1794, 1732, 1732, 1701,
+
 /*   370 */  1844, 1758, 1758, 1823, 1633, 1789, 1789, 1807, 1807, 1742,
+
 /*   380 */  1752, 1877, 1633, 1743, 1742, 1759, 1765, 1677, 1879, 1897,
+
 /*   390 */  1897, 1914, 1914, 1914, 2098, 2098, 2098, 2098, 2098, 2098,
+
 /*   400 */  2098, 2098, 2098, 2098, 2098, 2098, 2098, 2098, 2098,  207,
+
 /*   410 */  1095,  331,  620,  903,  806, 1074, 1483, 1432, 1481, 1322,
+
 /*   420 */  1370, 1394, 1515, 1291, 1546, 1547, 1557, 1595, 1598, 1599,
+
 /*   430 */  1434, 1453, 1618, 1462, 1567, 1489, 1644, 1654, 1616, 1660,
+
 /*   440 */  1548, 1549, 1682, 1685, 1597,  742, 1941, 1945, 1927, 1787,
+
 /*   450 */  1937, 1940, 1934, 1936, 1821, 1810, 1832, 1938, 1938, 1942,
+
 /*   460 */  1822, 1947, 1824, 1949, 1968, 1828, 1841, 1938, 1842, 1912,
+
 /*   470 */  1939, 1938, 1826, 1921, 1922, 1925, 1926, 1850, 1865, 1948,
+
 /*   480 */  1843, 1982, 1980, 1964, 1872, 1827, 1928, 1970, 1929, 1923,
+
 /*   490 */  1958, 1848, 1885, 1977, 1983, 1985, 1871, 1880, 1984, 1943,
+
 /*   500 */  1986, 1987, 1988, 1990, 1946, 1955, 1991, 1911, 1989, 1994,
+
 /*   510 */  1951, 1992, 1996, 1873, 1998, 2000, 2001, 2002, 2003, 2004,
+
 /*   520 */  1999, 1933, 1890, 2009, 2010, 1910, 2005, 2012, 1892, 2011,
+
 /*   530 */  2006, 2007, 2008, 2013, 1950, 1962, 1957, 2014, 1969, 1952,
+
 /*   540 */  2015, 2023, 2026, 2027, 2025, 2028, 2018, 1913, 1915, 2031,
+
 /*   550 */  2011, 2033, 2036, 2037, 2038, 2039, 2040, 2043, 2051, 2044,
+
 /*   560 */  2045, 2046, 2047, 2049, 2050, 2048, 1944, 1935, 1953, 1954,
+
 /*   570 */  1956, 2052, 2055, 2053, 2073, 2074,
};
#define YY_REDUCE_COUNT (408)
-
#define YY_REDUCE_MIN   (-267)
-
#define YY_REDUCE_MAX   (1715)
+
#define YY_REDUCE_MIN   (-271)
+
#define YY_REDUCE_MAX   (1740)
static const short yy_reduce_ofst[] = {
 /*     0 */  -125,  733,  789,  241,  293, -123, -193, -191, -183, -187,
-
 /*    10 */  -180,   83,  133, -207, -198, -267, -175,   -6,  166,  313,
-
 /*    20 */   487,  396,  489,  598,  615,  685,  687,   79,  781,  857,
-
 /*    30 */   490,  616,  240,  334, -188,  796,  841,  843, 1003, 1005,
-
 /*    40 */  1007, -260, -260, -260, -260, -260, -260, -260, -260, -260,
-
 /*    50 */  -260, -260, -260, -260, -260, -260, -260, -260, -260, -260,
-
 /*    60 */  -260, -260, -260, -260, -260, -260, -260, -260, -260, -260,
-
 /*    70 */  -260, -260, -260, -260, -260, -260, -260, -260,  158,  203,
-
 /*    80 */   391,  576,  724,  726,  886, 1021, 1035, 1063, 1081, 1083,
-
 /*    90 */  1097, 1099, 1117, 1152, 1155, 1158, 1163, 1165, 1167, 1169,
-
 /*   100 */  1172, 1180, 1183, 1198, 1200, 1205, 1215, 1225, 1227, 1236,
-
 /*   110 */  1252, 1264, 1299, 1303, 1306, 1309, 1312, 1315, 1325, 1328,
-
 /*   120 */  1337, 1340, 1343, 1371, 1373, 1384, 1386, 1411, -260, -260,
-
 /*   130 */  -260, -260, -260, -260, -260, -260, -260,  -53,  138,  302,
-
 /*   140 */  -158,  357,  223, -222,  411,  458,  -92,  556,  669,  581,
-
 /*   150 */   632,  581, -260,  632,  758,  778,  920, -260, -260, -260,
-
 /*   160 */  -260,  161,  161,  161,  307,  234,  392,  526,  790,  195,
-
 /*   170 */   359, -174, -173,  362,  362, -189,   16,  560,  567,  261,
-
 /*   180 */   689,  802,  853, -122, -166,  408,  335,  617,  690,  837,
-
 /*   190 */  1001,  746, 1061,  515, 1082,  994, 1034, -135, 1000, 1048,
-
 /*   200 */  1137,  877,  897,  186,  627, 1031, 1133, 1148, 1159, 1194,
-
 /*   210 */  1199, 1195, -194, -142,   18, -152,   68,  201,  253,  269,
-
 /*   220 */   294,  354,  521,  528,  676,  680,  736,  743,  850,  907,
-
 /*   230 */  1041, 1047, 1060,  727, 1139, 1147, 1201, 1237, 1278, 1359,
-
 /*   240 */  1393, 1400, 1413, 1429, 1433, 1437, 1126, 1410, 1430, 1444,
-
 /*   250 */  1480, 1483, 1405, 1486, 1490, 1492, 1420, 1372, 1496, 1498,
-
 /*   260 */  1441, 1499,  253, 1500, 1503, 1504, 1506, 1507, 1508, 1398,
-
 /*   270 */  1415, 1453, 1448, 1449, 1450, 1452, 1405, 1453, 1453, 1465,
-
 /*   280 */  1495, 1519, 1414, 1443, 1445, 1468, 1456, 1455, 1457, 1424,
-
 /*   290 */  1473, 1454, 1459, 1474, 1460, 1479, 1434, 1512, 1494, 1509,
-
 /*   300 */  1517, 1518, 1525, 1469, 1489, 1501, 1467, 1510, 1497, 1543,
-
 /*   310 */  1451, 1462, 1557, 1558, 1471, 1472, 1561, 1487, 1505, 1524,
-
 /*   320 */  1538, 1537, 1545, 1548, 1556, 1575, 1596, 1552, 1529, 1530,
-
 /*   330 */  1559, 1533, 1572, 1562, 1573, 1563, 1604, 1615, 1522, 1532,
-
 /*   340 */  1622, 1624, 1605, 1625, 1628, 1629, 1631, 1607, 1616, 1619,
-
 /*   350 */  1620, 1606, 1621, 1623, 1630, 1626, 1632, 1636, 1633, 1637,
-
 /*   360 */  1638, 1531, 1541, 1567, 1571, 1640, 1597, 1599, 1601, 1603,
-
 /*   370 */  1608, 1610, 1611, 1627, 1664, 1549, 1550, 1609, 1634, 1639,
-
 /*   380 */  1641, 1602, 1676, 1642, 1646, 1644, 1650, 1654, 1683, 1694,
-
 /*   390 */  1707, 1711, 1712, 1714, 1643, 1647, 1652, 1698, 1695, 1696,
-
 /*   400 */  1697, 1699, 1700, 1689, 1691, 1701, 1702, 1704, 1715,
+
 /*    10 */   166,  238,  133, -207, -199, -267, -176,   -6,  204,  489,
+
 /*    20 */   576, -175,  598,  686,  615,  725,  860,  778,  781,  857,
+
 /*    30 */   616,  887,   87,  240, -192,  408,  626,  796,  843,  854,
+
 /*    40 */  1003, -271, -271, -271, -271, -271, -271, -271, -271, -271,
+
 /*    50 */  -271, -271, -271, -271, -271, -271, -271, -271, -271, -271,
+
 /*    60 */  -271, -271, -271, -271, -271, -271, -271, -271, -271, -271,
+
 /*    70 */  -271, -271, -271, -271, -271, -271, -271, -271,   80,   83,
+
 /*    80 */   313,  886,  888,  996, 1034, 1059, 1081, 1100, 1117, 1152,
+
 /*    90 */  1155, 1163, 1165, 1167, 1169, 1172, 1180, 1182, 1184, 1198,
+
 /*   100 */  1200, 1213, 1215, 1225, 1227, 1252, 1254, 1264, 1299, 1303,
+
 /*   110 */  1308, 1312, 1325, 1328, 1337, 1340, 1343, 1371, 1373, 1384,
+
 /*   120 */  1386, 1411, 1420, 1424, 1426, 1458, 1470, 1473, 1475, 1479,
+
 /*   130 */  -271, -271, -271, -271, -271, -271, -271, -271, -271, -271,
+
 /*   140 */  -271,  138,  459,  396, -158,  470,  302, -212,  521,  201,
+
 /*   150 */  -195,  -92,  559,  630,  632,  630, -271,  632,  901,   63,
+
 /*   160 */   407, -271, -271, -271, -271,  161,  161,  161,  251,  335,
+
 /*   170 */   847,  960,  980,  537,  588,  618,  628,  688,  688, -166,
+
 /*   180 */  -161,  674,  790,  794,  799,  851,  852, -122,  680, -120,
+
 /*   190 */   995, 1038,  415, 1051,  893,  798,  962,  400, 1086,  779,
+
 /*   200 */   923,  924,  263, 1041,  979,  990, 1083, 1097, 1031, 1194,
+
 /*   210 */   362,  994, 1139, 1005, 1037, 1202, 1205, 1195, 1210, -194,
+
 /*   220 */    56,  185, -135,  232,  522,  560,  601,  617,  669,  683,
+
 /*   230 */   711,  856,  908,  941, 1048, 1101, 1147, 1257, 1262, 1265,
+
 /*   240 */   392, 1292, 1333, 1339, 1342, 1346, 1350, 1359, 1374, 1418,
+
 /*   250 */  1421, 1436, 1437,  593,  755,  770,  997, 1445, 1459, 1209,
+
 /*   260 */  1500, 1504, 1516, 1132, 1243, 1518, 1519, 1440, 1520,  560,
+
 /*   270 */  1522, 1523, 1524, 1526, 1527, 1529, 1382, 1438, 1431, 1468,
+
 /*   280 */  1469, 1472, 1476, 1209, 1431, 1431, 1485, 1525, 1539, 1435,
+
 /*   290 */  1463, 1471, 1492, 1487, 1443, 1494, 1474, 1484, 1498, 1486,
+
 /*   300 */  1502, 1455, 1530, 1531, 1533, 1540, 1542, 1544, 1505, 1506,
+
 /*   310 */  1507, 1508, 1521, 1528, 1493, 1537, 1532, 1575, 1488, 1496,
+
 /*   320 */  1584, 1594, 1509, 1510, 1600, 1538, 1534, 1541, 1574, 1577,
+
 /*   330 */  1583, 1585, 1586, 1612, 1626, 1581, 1556, 1558, 1587, 1559,
+
 /*   340 */  1601, 1588, 1603, 1592, 1631, 1640, 1550, 1553, 1643, 1645,
+
 /*   350 */  1625, 1649, 1652, 1650, 1653, 1632, 1636, 1637, 1642, 1634,
+
 /*   360 */  1639, 1641, 1646, 1656, 1655, 1658, 1659, 1661, 1663, 1560,
+
 /*   370 */  1564, 1596, 1605, 1664, 1670, 1565, 1571, 1627, 1638, 1657,
+
 /*   380 */  1665, 1623, 1702, 1630, 1666, 1667, 1671, 1673, 1703, 1718,
+
 /*   390 */  1719, 1729, 1730, 1731, 1621, 1622, 1628, 1720, 1713, 1716,
+
 /*   400 */  1722, 1723, 1733, 1717, 1724, 1727, 1728, 1725, 1740,
};
static const YYACTIONTYPE yy_default[] = {
-
 /*     0 */  1637, 1637, 1637, 1466, 1233, 1344, 1233, 1233, 1233, 1466,
-
 /*    10 */  1466, 1466, 1233, 1374, 1374, 1519, 1266, 1233, 1233, 1233,
-
 /*    20 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1465, 1233, 1233,
-
 /*    30 */  1233, 1233, 1554, 1554, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*    40 */  1233, 1233, 1383, 1233, 1390, 1233, 1233, 1233, 1233, 1233,
-
 /*    50 */  1467, 1468, 1233, 1233, 1233, 1518, 1520, 1483, 1397, 1396,
-
 /*    60 */  1395, 1394, 1501, 1361, 1388, 1381, 1385, 1461, 1462, 1460,
-
 /*    70 */  1464, 1468, 1467, 1233, 1384, 1431, 1445, 1430, 1233, 1233,
-
 /*    80 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*    90 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   100 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   110 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   120 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1439, 1444,
-
 /*   130 */  1451, 1443, 1440, 1433, 1432, 1434, 1435, 1233, 1233, 1257,
-
 /*   140 */  1233, 1233, 1254, 1308, 1233, 1233, 1233, 1233, 1233, 1538,
-
 /*   150 */  1537, 1233, 1436, 1233, 1266, 1425, 1424, 1448, 1437, 1447,
-
 /*   160 */  1446, 1526, 1590, 1589, 1484, 1233, 1233, 1233, 1233, 1233,
-
 /*   170 */  1233, 1554, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   180 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   190 */  1233, 1233, 1233, 1233, 1233, 1554, 1554, 1233, 1266, 1554,
-
 /*   200 */  1554, 1262, 1262, 1368, 1233, 1533, 1335, 1335, 1335, 1335,
-
 /*   210 */  1344, 1335, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   220 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1523, 1521, 1233,
-
 /*   230 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   240 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   250 */  1233, 1233, 1233, 1233, 1233, 1233, 1340, 1233, 1233, 1233,
-
 /*   260 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1583, 1233,
-
 /*   270 */  1496, 1322, 1340, 1340, 1340, 1340, 1342, 1323, 1321, 1334,
-
 /*   280 */  1267, 1240, 1629, 1400, 1389, 1341, 1363, 1389, 1363, 1626,
-
 /*   290 */  1387, 1400, 1400, 1387, 1400, 1341, 1626, 1283, 1606, 1278,
-
 /*   300 */  1374, 1374, 1374, 1363, 1368, 1368, 1463, 1341, 1334, 1233,
-
 /*   310 */  1629, 1629, 1349, 1349, 1628, 1628, 1349, 1484, 1613, 1409,
-
 /*   320 */  1311, 1317, 1317, 1317, 1317, 1349, 1251, 1387, 1613, 1613,
-
 /*   330 */  1387, 1409, 1311, 1387, 1311, 1387, 1349, 1251, 1500, 1623,
-
 /*   340 */  1349, 1251, 1474, 1349, 1251, 1349, 1251, 1474, 1309, 1309,
-
 /*   350 */  1309, 1298, 1233, 1233, 1474, 1309, 1283, 1309, 1298, 1309,
-
 /*   360 */  1309, 1572, 1233, 1478, 1478, 1474, 1367, 1362, 1367, 1362,
-
 /*   370 */  1367, 1362, 1367, 1362, 1349, 1564, 1564, 1377, 1377, 1382,
-
 /*   380 */  1368, 1469, 1349, 1233, 1382, 1380, 1378, 1387, 1301, 1586,
-
 /*   390 */  1586, 1582, 1582, 1582, 1634, 1634, 1533, 1599, 1266, 1266,
-
 /*   400 */  1266, 1266, 1599, 1285, 1285, 1267, 1267, 1266, 1599, 1233,
-
 /*   410 */  1233, 1233, 1233, 1233, 1233, 1594, 1233, 1528, 1485, 1353,
-
 /*   420 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   430 */  1233, 1233, 1233, 1233, 1539, 1233, 1233, 1233, 1233, 1233,
-
 /*   440 */  1233, 1233, 1233, 1233, 1233, 1414, 1233, 1236, 1530, 1233,
-
 /*   450 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1391, 1392, 1354,
-
 /*   460 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1406, 1233, 1233,
-
 /*   470 */  1233, 1401, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   480 */  1625, 1233, 1233, 1233, 1233, 1233, 1233, 1499, 1498, 1233,
-
 /*   490 */  1233, 1351, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   500 */  1233, 1233, 1233, 1233, 1233, 1281, 1233, 1233, 1233, 1233,
-
 /*   510 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   520 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1379,
-
 /*   530 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   540 */  1233, 1233, 1233, 1233, 1569, 1369, 1233, 1233, 1616, 1233,
-
 /*   550 */  1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233, 1233,
-
 /*   560 */  1233, 1233, 1233, 1610, 1325, 1416, 1233, 1415, 1419, 1255,
-
 /*   570 */  1233, 1245, 1233, 1233,
+
 /*     0 */  1647, 1647, 1647, 1475, 1240, 1351, 1240, 1240, 1240, 1475,
+
 /*    10 */  1475, 1475, 1240, 1381, 1381, 1528, 1273, 1240, 1240, 1240,
+
 /*    20 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1474, 1240, 1240,
+
 /*    30 */  1240, 1240, 1563, 1563, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*    40 */  1240, 1240, 1390, 1240, 1397, 1240, 1240, 1240, 1240, 1240,
+
 /*    50 */  1476, 1477, 1240, 1240, 1240, 1527, 1529, 1492, 1404, 1403,
+
 /*    60 */  1402, 1401, 1510, 1369, 1395, 1388, 1392, 1470, 1471, 1469,
+
 /*    70 */  1473, 1477, 1476, 1240, 1391, 1438, 1454, 1437, 1240, 1240,
+
 /*    80 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*    90 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   100 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   110 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   120 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   130 */  1446, 1453, 1452, 1451, 1460, 1450, 1447, 1440, 1439, 1441,
+
 /*   140 */  1442, 1240, 1240, 1264, 1240, 1240, 1261, 1315, 1240, 1240,
+
 /*   150 */  1240, 1240, 1240, 1547, 1546, 1240, 1443, 1240, 1273, 1432,
+
 /*   160 */  1431, 1457, 1444, 1456, 1455, 1535, 1599, 1598, 1493, 1240,
+
 /*   170 */  1240, 1240, 1240, 1240, 1240, 1563, 1240, 1240, 1240, 1240,
+
 /*   180 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   190 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1371,
+
 /*   200 */  1563, 1563, 1240, 1273, 1563, 1563, 1372, 1372, 1269, 1269,
+
 /*   210 */  1375, 1240, 1542, 1342, 1342, 1342, 1342, 1351, 1342, 1240,
+
 /*   220 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   230 */  1240, 1240, 1240, 1240, 1532, 1530, 1240, 1240, 1240, 1240,
+
 /*   240 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   250 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   260 */  1240, 1240, 1240, 1347, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   270 */  1240, 1240, 1240, 1240, 1240, 1592, 1240, 1505, 1329, 1347,
+
 /*   280 */  1347, 1347, 1347, 1349, 1330, 1328, 1341, 1274, 1247, 1639,
+
 /*   290 */  1407, 1396, 1348, 1396, 1636, 1394, 1407, 1407, 1394, 1407,
+
 /*   300 */  1348, 1636, 1290, 1615, 1285, 1381, 1381, 1381, 1371, 1371,
+
 /*   310 */  1371, 1371, 1375, 1375, 1472, 1348, 1341, 1240, 1639, 1639,
+
 /*   320 */  1357, 1357, 1638, 1638, 1357, 1493, 1623, 1416, 1318, 1324,
+
 /*   330 */  1324, 1324, 1324, 1357, 1258, 1394, 1623, 1623, 1394, 1416,
+
 /*   340 */  1318, 1394, 1318, 1394, 1357, 1258, 1509, 1633, 1357, 1258,
+
 /*   350 */  1483, 1357, 1258, 1357, 1258, 1483, 1316, 1316, 1316, 1305,
+
 /*   360 */  1240, 1240, 1483, 1316, 1290, 1316, 1305, 1316, 1316, 1581,
+
 /*   370 */  1240, 1487, 1487, 1483, 1357, 1573, 1573, 1384, 1384, 1389,
+
 /*   380 */  1375, 1478, 1357, 1240, 1389, 1387, 1385, 1394, 1308, 1595,
+
 /*   390 */  1595, 1591, 1591, 1591, 1644, 1644, 1542, 1608, 1273, 1273,
+
 /*   400 */  1273, 1273, 1608, 1292, 1292, 1274, 1274, 1273, 1608, 1240,
+
 /*   410 */  1240, 1240, 1240, 1240, 1240, 1603, 1240, 1537, 1494, 1361,
+
 /*   420 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   430 */  1240, 1240, 1240, 1240, 1548, 1240, 1240, 1240, 1240, 1240,
+
 /*   440 */  1240, 1240, 1240, 1240, 1240, 1421, 1240, 1243, 1539, 1240,
+
 /*   450 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1398, 1399, 1362,
+
 /*   460 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1413, 1240, 1240,
+
 /*   470 */  1240, 1408, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   480 */  1635, 1240, 1240, 1240, 1240, 1240, 1240, 1508, 1507, 1240,
+
 /*   490 */  1240, 1359, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   500 */  1240, 1240, 1240, 1240, 1240, 1288, 1240, 1240, 1240, 1240,
+
 /*   510 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   520 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1386,
+
 /*   530 */  1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   540 */  1240, 1240, 1240, 1240, 1578, 1376, 1240, 1240, 1240, 1240,
+
 /*   550 */  1626, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240, 1240,
+
 /*   560 */  1240, 1240, 1240, 1240, 1240, 1619, 1332, 1423, 1240, 1422,
+
 /*   570 */  1426, 1262, 1240, 1252, 1240, 1240,
};
/********** End of lemon-generated parsing tables *****************************/

@@ -163147,12 +166989,12 @@ static const char *const yyTokenName[] = {
  /*  256 */ "seltablist",
  /*  257 */ "stl_prefix",
  /*  258 */ "joinop",
-
  /*  259 */ "indexed_opt",
-
  /*  260 */ "on_opt",
-
  /*  261 */ "using_opt",
-
  /*  262 */ "exprlist",
-
  /*  263 */ "xfullname",
-
  /*  264 */ "idlist",
+
  /*  259 */ "on_using",
+
  /*  260 */ "indexed_by",
+
  /*  261 */ "exprlist",
+
  /*  262 */ "xfullname",
+
  /*  263 */ "idlist",
+
  /*  264 */ "indexed_opt",
  /*  265 */ "nulls",
  /*  266 */ "with",
  /*  267 */ "where_opt_ret",
@@ -163323,29 +167165,29 @@ static const char *const yyRuleName[] = {
 /* 106 */ "from ::= FROM seltablist",
 /* 107 */ "stl_prefix ::= seltablist joinop",
 /* 108 */ "stl_prefix ::=",
-
 /* 109 */ "seltablist ::= stl_prefix nm dbnm as indexed_opt on_opt using_opt",
-
 /* 110 */ "seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_opt using_opt",
-
 /* 111 */ "seltablist ::= stl_prefix LP select RP as on_opt using_opt",
-
 /* 112 */ "seltablist ::= stl_prefix LP seltablist RP as on_opt using_opt",
-
 /* 113 */ "dbnm ::=",
-
 /* 114 */ "dbnm ::= DOT nm",
-
 /* 115 */ "fullname ::= nm",
-
 /* 116 */ "fullname ::= nm DOT nm",
-
 /* 117 */ "xfullname ::= nm",
-
 /* 118 */ "xfullname ::= nm DOT nm",
-
 /* 119 */ "xfullname ::= nm DOT nm AS nm",
-
 /* 120 */ "xfullname ::= nm AS nm",
-
 /* 121 */ "joinop ::= COMMA|JOIN",
-
 /* 122 */ "joinop ::= JOIN_KW JOIN",
-
 /* 123 */ "joinop ::= JOIN_KW nm JOIN",
-
 /* 124 */ "joinop ::= JOIN_KW nm nm JOIN",
-
 /* 125 */ "on_opt ::= ON expr",
-
 /* 126 */ "on_opt ::=",
-
 /* 127 */ "indexed_opt ::=",
-
 /* 128 */ "indexed_opt ::= INDEXED BY nm",
-
 /* 129 */ "indexed_opt ::= NOT INDEXED",
-
 /* 130 */ "using_opt ::= USING LP idlist RP",
-
 /* 131 */ "using_opt ::=",
+
 /* 109 */ "seltablist ::= stl_prefix nm dbnm as on_using",
+
 /* 110 */ "seltablist ::= stl_prefix nm dbnm as indexed_by on_using",
+
 /* 111 */ "seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_using",
+
 /* 112 */ "seltablist ::= stl_prefix LP select RP as on_using",
+
 /* 113 */ "seltablist ::= stl_prefix LP seltablist RP as on_using",
+
 /* 114 */ "dbnm ::=",
+
 /* 115 */ "dbnm ::= DOT nm",
+
 /* 116 */ "fullname ::= nm",
+
 /* 117 */ "fullname ::= nm DOT nm",
+
 /* 118 */ "xfullname ::= nm",
+
 /* 119 */ "xfullname ::= nm DOT nm",
+
 /* 120 */ "xfullname ::= nm DOT nm AS nm",
+
 /* 121 */ "xfullname ::= nm AS nm",
+
 /* 122 */ "joinop ::= COMMA|JOIN",
+
 /* 123 */ "joinop ::= JOIN_KW JOIN",
+
 /* 124 */ "joinop ::= JOIN_KW nm JOIN",
+
 /* 125 */ "joinop ::= JOIN_KW nm nm JOIN",
+
 /* 126 */ "on_using ::= ON expr",
+
 /* 127 */ "on_using ::= USING LP idlist RP",
+
 /* 128 */ "on_using ::=",
+
 /* 129 */ "indexed_opt ::=",
+
 /* 130 */ "indexed_by ::= INDEXED BY nm",
+
 /* 131 */ "indexed_by ::= NOT INDEXED",
 /* 132 */ "orderby_opt ::=",
 /* 133 */ "orderby_opt ::= ORDER BY sortlist",
 /* 134 */ "sortlist ::= sortlist COMMA expr sortorder nulls",
@@ -163423,199 +167265,202 @@ static const char *const yyRuleName[] = {
 /* 206 */ "expr ::= expr NOT NULL",
 /* 207 */ "expr ::= expr IS expr",
 /* 208 */ "expr ::= expr IS NOT expr",
-
 /* 209 */ "expr ::= NOT expr",
-
 /* 210 */ "expr ::= BITNOT expr",
-
 /* 211 */ "expr ::= PLUS|MINUS expr",
-
 /* 212 */ "expr ::= expr PTR expr",
-
 /* 213 */ "between_op ::= BETWEEN",
-
 /* 214 */ "between_op ::= NOT BETWEEN",
-
 /* 215 */ "expr ::= expr between_op expr AND expr",
-
 /* 216 */ "in_op ::= IN",
-
 /* 217 */ "in_op ::= NOT IN",
-
 /* 218 */ "expr ::= expr in_op LP exprlist RP",
-
 /* 219 */ "expr ::= LP select RP",
-
 /* 220 */ "expr ::= expr in_op LP select RP",
-
 /* 221 */ "expr ::= expr in_op nm dbnm paren_exprlist",
-
 /* 222 */ "expr ::= EXISTS LP select RP",
-
 /* 223 */ "expr ::= CASE case_operand case_exprlist case_else END",
-
 /* 224 */ "case_exprlist ::= case_exprlist WHEN expr THEN expr",
-
 /* 225 */ "case_exprlist ::= WHEN expr THEN expr",
-
 /* 226 */ "case_else ::= ELSE expr",
-
 /* 227 */ "case_else ::=",
-
 /* 228 */ "case_operand ::= expr",
-
 /* 229 */ "case_operand ::=",
-
 /* 230 */ "exprlist ::=",
-
 /* 231 */ "nexprlist ::= nexprlist COMMA expr",
-
 /* 232 */ "nexprlist ::= expr",
-
 /* 233 */ "paren_exprlist ::=",
-
 /* 234 */ "paren_exprlist ::= LP exprlist RP",
-
 /* 235 */ "cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt",
-
 /* 236 */ "uniqueflag ::= UNIQUE",
-
 /* 237 */ "uniqueflag ::=",
-
 /* 238 */ "eidlist_opt ::=",
-
 /* 239 */ "eidlist_opt ::= LP eidlist RP",
-
 /* 240 */ "eidlist ::= eidlist COMMA nm collate sortorder",
-
 /* 241 */ "eidlist ::= nm collate sortorder",
-
 /* 242 */ "collate ::=",
-
 /* 243 */ "collate ::= COLLATE ID|STRING",
-
 /* 244 */ "cmd ::= DROP INDEX ifexists fullname",
-
 /* 245 */ "cmd ::= VACUUM vinto",
-
 /* 246 */ "cmd ::= VACUUM nm vinto",
-
 /* 247 */ "vinto ::= INTO expr",
-
 /* 248 */ "vinto ::=",
-
 /* 249 */ "cmd ::= PRAGMA nm dbnm",
-
 /* 250 */ "cmd ::= PRAGMA nm dbnm EQ nmnum",
-
 /* 251 */ "cmd ::= PRAGMA nm dbnm LP nmnum RP",
-
 /* 252 */ "cmd ::= PRAGMA nm dbnm EQ minus_num",
-
 /* 253 */ "cmd ::= PRAGMA nm dbnm LP minus_num RP",
-
 /* 254 */ "plus_num ::= PLUS INTEGER|FLOAT",
-
 /* 255 */ "minus_num ::= MINUS INTEGER|FLOAT",
-
 /* 256 */ "cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END",
-
 /* 257 */ "trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause",
-
 /* 258 */ "trigger_time ::= BEFORE|AFTER",
-
 /* 259 */ "trigger_time ::= INSTEAD OF",
-
 /* 260 */ "trigger_time ::=",
-
 /* 261 */ "trigger_event ::= DELETE|INSERT",
-
 /* 262 */ "trigger_event ::= UPDATE",
-
 /* 263 */ "trigger_event ::= UPDATE OF idlist",
-
 /* 264 */ "when_clause ::=",
-
 /* 265 */ "when_clause ::= WHEN expr",
-
 /* 266 */ "trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI",
-
 /* 267 */ "trigger_cmd_list ::= trigger_cmd SEMI",
-
 /* 268 */ "trnm ::= nm DOT nm",
-
 /* 269 */ "tridxby ::= INDEXED BY nm",
-
 /* 270 */ "tridxby ::= NOT INDEXED",
-
 /* 271 */ "trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt",
-
 /* 272 */ "trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt",
-
 /* 273 */ "trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt",
-
 /* 274 */ "trigger_cmd ::= scanpt select scanpt",
-
 /* 275 */ "expr ::= RAISE LP IGNORE RP",
-
 /* 276 */ "expr ::= RAISE LP raisetype COMMA nm RP",
-
 /* 277 */ "raisetype ::= ROLLBACK",
-
 /* 278 */ "raisetype ::= ABORT",
-
 /* 279 */ "raisetype ::= FAIL",
-
 /* 280 */ "cmd ::= DROP TRIGGER ifexists fullname",
-
 /* 281 */ "cmd ::= ATTACH database_kw_opt expr AS expr key_opt",
-
 /* 282 */ "cmd ::= DETACH database_kw_opt expr",
-
 /* 283 */ "key_opt ::=",
-
 /* 284 */ "key_opt ::= KEY expr",
-
 /* 285 */ "cmd ::= REINDEX",
-
 /* 286 */ "cmd ::= REINDEX nm dbnm",
-
 /* 287 */ "cmd ::= ANALYZE",
-
 /* 288 */ "cmd ::= ANALYZE nm dbnm",
-
 /* 289 */ "cmd ::= ALTER TABLE fullname RENAME TO nm",
-
 /* 290 */ "cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist",
-
 /* 291 */ "cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm",
-
 /* 292 */ "add_column_fullname ::= fullname",
-
 /* 293 */ "cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm",
-
 /* 294 */ "cmd ::= create_vtab",
-
 /* 295 */ "cmd ::= create_vtab LP vtabarglist RP",
-
 /* 296 */ "create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm",
-
 /* 297 */ "vtabarg ::=",
-
 /* 298 */ "vtabargtoken ::= ANY",
-
 /* 299 */ "vtabargtoken ::= lp anylist RP",
-
 /* 300 */ "lp ::= LP",
-
 /* 301 */ "with ::= WITH wqlist",
-
 /* 302 */ "with ::= WITH RECURSIVE wqlist",
-
 /* 303 */ "wqas ::= AS",
-
 /* 304 */ "wqas ::= AS MATERIALIZED",
-
 /* 305 */ "wqas ::= AS NOT MATERIALIZED",
-
 /* 306 */ "wqitem ::= nm eidlist_opt wqas LP select RP",
-
 /* 307 */ "wqlist ::= wqitem",
-
 /* 308 */ "wqlist ::= wqlist COMMA wqitem",
-
 /* 309 */ "windowdefn_list ::= windowdefn",
-
 /* 310 */ "windowdefn_list ::= windowdefn_list COMMA windowdefn",
-
 /* 311 */ "windowdefn ::= nm AS LP window RP",
-
 /* 312 */ "window ::= PARTITION BY nexprlist orderby_opt frame_opt",
-
 /* 313 */ "window ::= nm PARTITION BY nexprlist orderby_opt frame_opt",
-
 /* 314 */ "window ::= ORDER BY sortlist frame_opt",
-
 /* 315 */ "window ::= nm ORDER BY sortlist frame_opt",
-
 /* 316 */ "window ::= frame_opt",
-
 /* 317 */ "window ::= nm frame_opt",
-
 /* 318 */ "frame_opt ::=",
-
 /* 319 */ "frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt",
-
 /* 320 */ "frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt",
-
 /* 321 */ "range_or_rows ::= RANGE|ROWS|GROUPS",
-
 /* 322 */ "frame_bound_s ::= frame_bound",
-
 /* 323 */ "frame_bound_s ::= UNBOUNDED PRECEDING",
-
 /* 324 */ "frame_bound_e ::= frame_bound",
-
 /* 325 */ "frame_bound_e ::= UNBOUNDED FOLLOWING",
-
 /* 326 */ "frame_bound ::= expr PRECEDING|FOLLOWING",
-
 /* 327 */ "frame_bound ::= CURRENT ROW",
-
 /* 328 */ "frame_exclude_opt ::=",
-
 /* 329 */ "frame_exclude_opt ::= EXCLUDE frame_exclude",
-
 /* 330 */ "frame_exclude ::= NO OTHERS",
-
 /* 331 */ "frame_exclude ::= CURRENT ROW",
-
 /* 332 */ "frame_exclude ::= GROUP|TIES",
-
 /* 333 */ "window_clause ::= WINDOW windowdefn_list",
-
 /* 334 */ "filter_over ::= filter_clause over_clause",
-
 /* 335 */ "filter_over ::= over_clause",
-
 /* 336 */ "filter_over ::= filter_clause",
-
 /* 337 */ "over_clause ::= OVER LP window RP",
-
 /* 338 */ "over_clause ::= OVER nm",
-
 /* 339 */ "filter_clause ::= FILTER LP WHERE expr RP",
-
 /* 340 */ "input ::= cmdlist",
-
 /* 341 */ "cmdlist ::= cmdlist ecmd",
-
 /* 342 */ "cmdlist ::= ecmd",
-
 /* 343 */ "ecmd ::= SEMI",
-
 /* 344 */ "ecmd ::= cmdx SEMI",
-
 /* 345 */ "ecmd ::= explain cmdx SEMI",
-
 /* 346 */ "trans_opt ::=",
-
 /* 347 */ "trans_opt ::= TRANSACTION",
-
 /* 348 */ "trans_opt ::= TRANSACTION nm",
-
 /* 349 */ "savepoint_opt ::= SAVEPOINT",
-
 /* 350 */ "savepoint_opt ::=",
-
 /* 351 */ "cmd ::= create_table create_table_args",
-
 /* 352 */ "table_option_set ::= table_option",
-
 /* 353 */ "columnlist ::= columnlist COMMA columnname carglist",
-
 /* 354 */ "columnlist ::= columnname carglist",
-
 /* 355 */ "nm ::= ID|INDEXED",
-
 /* 356 */ "nm ::= STRING",
-
 /* 357 */ "nm ::= JOIN_KW",
-
 /* 358 */ "typetoken ::= typename",
-
 /* 359 */ "typename ::= ID|STRING",
-
 /* 360 */ "signed ::= plus_num",
-
 /* 361 */ "signed ::= minus_num",
-
 /* 362 */ "carglist ::= carglist ccons",
-
 /* 363 */ "carglist ::=",
-
 /* 364 */ "ccons ::= NULL onconf",
-
 /* 365 */ "ccons ::= GENERATED ALWAYS AS generated",
-
 /* 366 */ "ccons ::= AS generated",
-
 /* 367 */ "conslist_opt ::= COMMA conslist",
-
 /* 368 */ "conslist ::= conslist tconscomma tcons",
-
 /* 369 */ "conslist ::= tcons",
-
 /* 370 */ "tconscomma ::=",
-
 /* 371 */ "defer_subclause_opt ::= defer_subclause",
-
 /* 372 */ "resolvetype ::= raisetype",
-
 /* 373 */ "selectnowith ::= oneselect",
-
 /* 374 */ "oneselect ::= values",
-
 /* 375 */ "sclp ::= selcollist COMMA",
-
 /* 376 */ "as ::= ID|STRING",
-
 /* 377 */ "returning ::=",
-
 /* 378 */ "expr ::= term",
-
 /* 379 */ "likeop ::= LIKE_KW|MATCH",
-
 /* 380 */ "exprlist ::= nexprlist",
-
 /* 381 */ "nmnum ::= plus_num",
-
 /* 382 */ "nmnum ::= nm",
-
 /* 383 */ "nmnum ::= ON",
-
 /* 384 */ "nmnum ::= DELETE",
-
 /* 385 */ "nmnum ::= DEFAULT",
-
 /* 386 */ "plus_num ::= INTEGER|FLOAT",
-
 /* 387 */ "foreach_clause ::=",
-
 /* 388 */ "foreach_clause ::= FOR EACH ROW",
-
 /* 389 */ "trnm ::= nm",
-
 /* 390 */ "tridxby ::=",
-
 /* 391 */ "database_kw_opt ::= DATABASE",
-
 /* 392 */ "database_kw_opt ::=",
-
 /* 393 */ "kwcolumn_opt ::=",
-
 /* 394 */ "kwcolumn_opt ::= COLUMNKW",
-
 /* 395 */ "vtabarglist ::= vtabarg",
-
 /* 396 */ "vtabarglist ::= vtabarglist COMMA vtabarg",
-
 /* 397 */ "vtabarg ::= vtabarg vtabargtoken",
-
 /* 398 */ "anylist ::=",
-
 /* 399 */ "anylist ::= anylist LP anylist RP",
-
 /* 400 */ "anylist ::= anylist ANY",
-
 /* 401 */ "with ::=",
+
 /* 209 */ "expr ::= expr IS NOT DISTINCT FROM expr",
+
 /* 210 */ "expr ::= expr IS DISTINCT FROM expr",
+
 /* 211 */ "expr ::= NOT expr",
+
 /* 212 */ "expr ::= BITNOT expr",
+
 /* 213 */ "expr ::= PLUS|MINUS expr",
+
 /* 214 */ "expr ::= expr PTR expr",
+
 /* 215 */ "between_op ::= BETWEEN",
+
 /* 216 */ "between_op ::= NOT BETWEEN",
+
 /* 217 */ "expr ::= expr between_op expr AND expr",
+
 /* 218 */ "in_op ::= IN",
+
 /* 219 */ "in_op ::= NOT IN",
+
 /* 220 */ "expr ::= expr in_op LP exprlist RP",
+
 /* 221 */ "expr ::= LP select RP",
+
 /* 222 */ "expr ::= expr in_op LP select RP",
+
 /* 223 */ "expr ::= expr in_op nm dbnm paren_exprlist",
+
 /* 224 */ "expr ::= EXISTS LP select RP",
+
 /* 225 */ "expr ::= CASE case_operand case_exprlist case_else END",
+
 /* 226 */ "case_exprlist ::= case_exprlist WHEN expr THEN expr",
+
 /* 227 */ "case_exprlist ::= WHEN expr THEN expr",
+
 /* 228 */ "case_else ::= ELSE expr",
+
 /* 229 */ "case_else ::=",
+
 /* 230 */ "case_operand ::= expr",
+
 /* 231 */ "case_operand ::=",
+
 /* 232 */ "exprlist ::=",
+
 /* 233 */ "nexprlist ::= nexprlist COMMA expr",
+
 /* 234 */ "nexprlist ::= expr",
+
 /* 235 */ "paren_exprlist ::=",
+
 /* 236 */ "paren_exprlist ::= LP exprlist RP",
+
 /* 237 */ "cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt",
+
 /* 238 */ "uniqueflag ::= UNIQUE",
+
 /* 239 */ "uniqueflag ::=",
+
 /* 240 */ "eidlist_opt ::=",
+
 /* 241 */ "eidlist_opt ::= LP eidlist RP",
+
 /* 242 */ "eidlist ::= eidlist COMMA nm collate sortorder",
+
 /* 243 */ "eidlist ::= nm collate sortorder",
+
 /* 244 */ "collate ::=",
+
 /* 245 */ "collate ::= COLLATE ID|STRING",
+
 /* 246 */ "cmd ::= DROP INDEX ifexists fullname",
+
 /* 247 */ "cmd ::= VACUUM vinto",
+
 /* 248 */ "cmd ::= VACUUM nm vinto",
+
 /* 249 */ "vinto ::= INTO expr",
+
 /* 250 */ "vinto ::=",
+
 /* 251 */ "cmd ::= PRAGMA nm dbnm",
+
 /* 252 */ "cmd ::= PRAGMA nm dbnm EQ nmnum",
+
 /* 253 */ "cmd ::= PRAGMA nm dbnm LP nmnum RP",
+
 /* 254 */ "cmd ::= PRAGMA nm dbnm EQ minus_num",
+
 /* 255 */ "cmd ::= PRAGMA nm dbnm LP minus_num RP",
+
 /* 256 */ "plus_num ::= PLUS INTEGER|FLOAT",
+
 /* 257 */ "minus_num ::= MINUS INTEGER|FLOAT",
+
 /* 258 */ "cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END",
+
 /* 259 */ "trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause",
+
 /* 260 */ "trigger_time ::= BEFORE|AFTER",
+
 /* 261 */ "trigger_time ::= INSTEAD OF",
+
 /* 262 */ "trigger_time ::=",
+
 /* 263 */ "trigger_event ::= DELETE|INSERT",
+
 /* 264 */ "trigger_event ::= UPDATE",
+
 /* 265 */ "trigger_event ::= UPDATE OF idlist",
+
 /* 266 */ "when_clause ::=",
+
 /* 267 */ "when_clause ::= WHEN expr",
+
 /* 268 */ "trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI",
+
 /* 269 */ "trigger_cmd_list ::= trigger_cmd SEMI",
+
 /* 270 */ "trnm ::= nm DOT nm",
+
 /* 271 */ "tridxby ::= INDEXED BY nm",
+
 /* 272 */ "tridxby ::= NOT INDEXED",
+
 /* 273 */ "trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt",
+
 /* 274 */ "trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt",
+
 /* 275 */ "trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt",
+
 /* 276 */ "trigger_cmd ::= scanpt select scanpt",
+
 /* 277 */ "expr ::= RAISE LP IGNORE RP",
+
 /* 278 */ "expr ::= RAISE LP raisetype COMMA nm RP",
+
 /* 279 */ "raisetype ::= ROLLBACK",
+
 /* 280 */ "raisetype ::= ABORT",
+
 /* 281 */ "raisetype ::= FAIL",
+
 /* 282 */ "cmd ::= DROP TRIGGER ifexists fullname",
+
 /* 283 */ "cmd ::= ATTACH database_kw_opt expr AS expr key_opt",
+
 /* 284 */ "cmd ::= DETACH database_kw_opt expr",
+
 /* 285 */ "key_opt ::=",
+
 /* 286 */ "key_opt ::= KEY expr",
+
 /* 287 */ "cmd ::= REINDEX",
+
 /* 288 */ "cmd ::= REINDEX nm dbnm",
+
 /* 289 */ "cmd ::= ANALYZE",
+
 /* 290 */ "cmd ::= ANALYZE nm dbnm",
+
 /* 291 */ "cmd ::= ALTER TABLE fullname RENAME TO nm",
+
 /* 292 */ "cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist",
+
 /* 293 */ "cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm",
+
 /* 294 */ "add_column_fullname ::= fullname",
+
 /* 295 */ "cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm",
+
 /* 296 */ "cmd ::= create_vtab",
+
 /* 297 */ "cmd ::= create_vtab LP vtabarglist RP",
+
 /* 298 */ "create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm",
+
 /* 299 */ "vtabarg ::=",
+
 /* 300 */ "vtabargtoken ::= ANY",
+
 /* 301 */ "vtabargtoken ::= lp anylist RP",
+
 /* 302 */ "lp ::= LP",
+
 /* 303 */ "with ::= WITH wqlist",
+
 /* 304 */ "with ::= WITH RECURSIVE wqlist",
+
 /* 305 */ "wqas ::= AS",
+
 /* 306 */ "wqas ::= AS MATERIALIZED",
+
 /* 307 */ "wqas ::= AS NOT MATERIALIZED",
+
 /* 308 */ "wqitem ::= nm eidlist_opt wqas LP select RP",
+
 /* 309 */ "wqlist ::= wqitem",
+
 /* 310 */ "wqlist ::= wqlist COMMA wqitem",
+
 /* 311 */ "windowdefn_list ::= windowdefn",
+
 /* 312 */ "windowdefn_list ::= windowdefn_list COMMA windowdefn",
+
 /* 313 */ "windowdefn ::= nm AS LP window RP",
+
 /* 314 */ "window ::= PARTITION BY nexprlist orderby_opt frame_opt",
+
 /* 315 */ "window ::= nm PARTITION BY nexprlist orderby_opt frame_opt",
+
 /* 316 */ "window ::= ORDER BY sortlist frame_opt",
+
 /* 317 */ "window ::= nm ORDER BY sortlist frame_opt",
+
 /* 318 */ "window ::= frame_opt",
+
 /* 319 */ "window ::= nm frame_opt",
+
 /* 320 */ "frame_opt ::=",
+
 /* 321 */ "frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt",
+
 /* 322 */ "frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt",
+
 /* 323 */ "range_or_rows ::= RANGE|ROWS|GROUPS",
+
 /* 324 */ "frame_bound_s ::= frame_bound",
+
 /* 325 */ "frame_bound_s ::= UNBOUNDED PRECEDING",
+
 /* 326 */ "frame_bound_e ::= frame_bound",
+
 /* 327 */ "frame_bound_e ::= UNBOUNDED FOLLOWING",
+
 /* 328 */ "frame_bound ::= expr PRECEDING|FOLLOWING",
+
 /* 329 */ "frame_bound ::= CURRENT ROW",
+
 /* 330 */ "frame_exclude_opt ::=",
+
 /* 331 */ "frame_exclude_opt ::= EXCLUDE frame_exclude",
+
 /* 332 */ "frame_exclude ::= NO OTHERS",
+
 /* 333 */ "frame_exclude ::= CURRENT ROW",
+
 /* 334 */ "frame_exclude ::= GROUP|TIES",
+
 /* 335 */ "window_clause ::= WINDOW windowdefn_list",
+
 /* 336 */ "filter_over ::= filter_clause over_clause",
+
 /* 337 */ "filter_over ::= over_clause",
+
 /* 338 */ "filter_over ::= filter_clause",
+
 /* 339 */ "over_clause ::= OVER LP window RP",
+
 /* 340 */ "over_clause ::= OVER nm",
+
 /* 341 */ "filter_clause ::= FILTER LP WHERE expr RP",
+
 /* 342 */ "input ::= cmdlist",
+
 /* 343 */ "cmdlist ::= cmdlist ecmd",
+
 /* 344 */ "cmdlist ::= ecmd",
+
 /* 345 */ "ecmd ::= SEMI",
+
 /* 346 */ "ecmd ::= cmdx SEMI",
+
 /* 347 */ "ecmd ::= explain cmdx SEMI",
+
 /* 348 */ "trans_opt ::=",
+
 /* 349 */ "trans_opt ::= TRANSACTION",
+
 /* 350 */ "trans_opt ::= TRANSACTION nm",
+
 /* 351 */ "savepoint_opt ::= SAVEPOINT",
+
 /* 352 */ "savepoint_opt ::=",
+
 /* 353 */ "cmd ::= create_table create_table_args",
+
 /* 354 */ "table_option_set ::= table_option",
+
 /* 355 */ "columnlist ::= columnlist COMMA columnname carglist",
+
 /* 356 */ "columnlist ::= columnname carglist",
+
 /* 357 */ "nm ::= ID|INDEXED",
+
 /* 358 */ "nm ::= STRING",
+
 /* 359 */ "nm ::= JOIN_KW",
+
 /* 360 */ "typetoken ::= typename",
+
 /* 361 */ "typename ::= ID|STRING",
+
 /* 362 */ "signed ::= plus_num",
+
 /* 363 */ "signed ::= minus_num",
+
 /* 364 */ "carglist ::= carglist ccons",
+
 /* 365 */ "carglist ::=",
+
 /* 366 */ "ccons ::= NULL onconf",
+
 /* 367 */ "ccons ::= GENERATED ALWAYS AS generated",
+
 /* 368 */ "ccons ::= AS generated",
+
 /* 369 */ "conslist_opt ::= COMMA conslist",
+
 /* 370 */ "conslist ::= conslist tconscomma tcons",
+
 /* 371 */ "conslist ::= tcons",
+
 /* 372 */ "tconscomma ::=",
+
 /* 373 */ "defer_subclause_opt ::= defer_subclause",
+
 /* 374 */ "resolvetype ::= raisetype",
+
 /* 375 */ "selectnowith ::= oneselect",
+
 /* 376 */ "oneselect ::= values",
+
 /* 377 */ "sclp ::= selcollist COMMA",
+
 /* 378 */ "as ::= ID|STRING",
+
 /* 379 */ "indexed_opt ::= indexed_by",
+
 /* 380 */ "returning ::=",
+
 /* 381 */ "expr ::= term",
+
 /* 382 */ "likeop ::= LIKE_KW|MATCH",
+
 /* 383 */ "exprlist ::= nexprlist",
+
 /* 384 */ "nmnum ::= plus_num",
+
 /* 385 */ "nmnum ::= nm",
+
 /* 386 */ "nmnum ::= ON",
+
 /* 387 */ "nmnum ::= DELETE",
+
 /* 388 */ "nmnum ::= DEFAULT",
+
 /* 389 */ "plus_num ::= INTEGER|FLOAT",
+
 /* 390 */ "foreach_clause ::=",
+
 /* 391 */ "foreach_clause ::= FOR EACH ROW",
+
 /* 392 */ "trnm ::= nm",
+
 /* 393 */ "tridxby ::=",
+
 /* 394 */ "database_kw_opt ::= DATABASE",
+
 /* 395 */ "database_kw_opt ::=",
+
 /* 396 */ "kwcolumn_opt ::=",
+
 /* 397 */ "kwcolumn_opt ::= COLUMNKW",
+
 /* 398 */ "vtabarglist ::= vtabarg",
+
 /* 399 */ "vtabarglist ::= vtabarglist COMMA vtabarg",
+
 /* 400 */ "vtabarg ::= vtabarg vtabargtoken",
+
 /* 401 */ "anylist ::=",
+
 /* 402 */ "anylist ::= anylist LP anylist RP",
+
 /* 403 */ "anylist ::= anylist ANY",
+
 /* 404 */ "with ::=",
};
#endif /* NDEBUG */

@@ -163753,7 +167598,6 @@ sqlite3SelectDelete(pParse->db, (yypminor->yy47));
    case 217: /* expr */
    case 246: /* where_opt */
    case 248: /* having_opt */
-
    case 260: /* on_opt */
    case 267: /* where_opt_ret */
    case 278: /* case_operand */
    case 280: /* case_else */
@@ -163773,7 +167617,7 @@ sqlite3ExprDelete(pParse->db, (yypminor->yy528));
    case 249: /* orderby_opt */
    case 253: /* nexprlist */
    case 254: /* sclp */
-
    case 262: /* exprlist */
+
    case 261: /* exprlist */
    case 268: /* setlist */
    case 277: /* paren_exprlist */
    case 279: /* case_exprlist */
@@ -163786,7 +167630,7 @@ sqlite3ExprListDelete(pParse->db, (yypminor->yy322));
    case 245: /* from */
    case 256: /* seltablist */
    case 257: /* stl_prefix */
-
    case 263: /* xfullname */
+
    case 262: /* xfullname */
{
sqlite3SrcListDelete(pParse->db, (yypminor->yy131));
}
@@ -163802,8 +167646,7 @@ sqlite3WithDelete(pParse->db, (yypminor->yy521));
sqlite3WindowListDelete(pParse->db, (yypminor->yy41));
}
      break;
-
    case 261: /* using_opt */
-
    case 264: /* idlist */
+
    case 263: /* idlist */
    case 270: /* idlist_opt */
{
sqlite3IdListDelete(pParse->db, (yypminor->yy254));
@@ -164233,29 +168076,29 @@ static const YYCODETYPE yyRuleInfoLhs[] = {
   245,  /* (106) from ::= FROM seltablist */
   257,  /* (107) stl_prefix ::= seltablist joinop */
   257,  /* (108) stl_prefix ::= */
-
   256,  /* (109) seltablist ::= stl_prefix nm dbnm as indexed_opt on_opt using_opt */
-
   256,  /* (110) seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_opt using_opt */
-
   256,  /* (111) seltablist ::= stl_prefix LP select RP as on_opt using_opt */
-
   256,  /* (112) seltablist ::= stl_prefix LP seltablist RP as on_opt using_opt */
-
   200,  /* (113) dbnm ::= */
-
   200,  /* (114) dbnm ::= DOT nm */
-
   238,  /* (115) fullname ::= nm */
-
   238,  /* (116) fullname ::= nm DOT nm */
-
   263,  /* (117) xfullname ::= nm */
-
   263,  /* (118) xfullname ::= nm DOT nm */
-
   263,  /* (119) xfullname ::= nm DOT nm AS nm */
-
   263,  /* (120) xfullname ::= nm AS nm */
-
   258,  /* (121) joinop ::= COMMA|JOIN */
-
   258,  /* (122) joinop ::= JOIN_KW JOIN */
-
   258,  /* (123) joinop ::= JOIN_KW nm JOIN */
-
   258,  /* (124) joinop ::= JOIN_KW nm nm JOIN */
-
   260,  /* (125) on_opt ::= ON expr */
-
   260,  /* (126) on_opt ::= */
-
   259,  /* (127) indexed_opt ::= */
-
   259,  /* (128) indexed_opt ::= INDEXED BY nm */
-
   259,  /* (129) indexed_opt ::= NOT INDEXED */
-
   261,  /* (130) using_opt ::= USING LP idlist RP */
-
   261,  /* (131) using_opt ::= */
+
   256,  /* (109) seltablist ::= stl_prefix nm dbnm as on_using */
+
   256,  /* (110) seltablist ::= stl_prefix nm dbnm as indexed_by on_using */
+
   256,  /* (111) seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_using */
+
   256,  /* (112) seltablist ::= stl_prefix LP select RP as on_using */
+
   256,  /* (113) seltablist ::= stl_prefix LP seltablist RP as on_using */
+
   200,  /* (114) dbnm ::= */
+
   200,  /* (115) dbnm ::= DOT nm */
+
   238,  /* (116) fullname ::= nm */
+
   238,  /* (117) fullname ::= nm DOT nm */
+
   262,  /* (118) xfullname ::= nm */
+
   262,  /* (119) xfullname ::= nm DOT nm */
+
   262,  /* (120) xfullname ::= nm DOT nm AS nm */
+
   262,  /* (121) xfullname ::= nm AS nm */
+
   258,  /* (122) joinop ::= COMMA|JOIN */
+
   258,  /* (123) joinop ::= JOIN_KW JOIN */
+
   258,  /* (124) joinop ::= JOIN_KW nm JOIN */
+
   258,  /* (125) joinop ::= JOIN_KW nm nm JOIN */
+
   259,  /* (126) on_using ::= ON expr */
+
   259,  /* (127) on_using ::= USING LP idlist RP */
+
   259,  /* (128) on_using ::= */
+
   264,  /* (129) indexed_opt ::= */
+
   260,  /* (130) indexed_by ::= INDEXED BY nm */
+
   260,  /* (131) indexed_by ::= NOT INDEXED */
   249,  /* (132) orderby_opt ::= */
   249,  /* (133) orderby_opt ::= ORDER BY sortlist */
   231,  /* (134) sortlist ::= sortlist COMMA expr sortorder nulls */
@@ -164299,8 +168142,8 @@ static const YYCODETYPE yyRuleInfoLhs[] = {
   269,  /* (172) insert_cmd ::= REPLACE */
   270,  /* (173) idlist_opt ::= */
   270,  /* (174) idlist_opt ::= LP idlist RP */
-
   264,  /* (175) idlist ::= idlist COMMA nm */
-
   264,  /* (176) idlist ::= nm */
+
   263,  /* (175) idlist ::= idlist COMMA nm */
+
   263,  /* (176) idlist ::= nm */
   217,  /* (177) expr ::= LP expr RP */
   217,  /* (178) expr ::= ID|INDEXED */
   217,  /* (179) expr ::= JOIN_KW */
@@ -164333,199 +168176,202 @@ static const YYCODETYPE yyRuleInfoLhs[] = {
   217,  /* (206) expr ::= expr NOT NULL */
   217,  /* (207) expr ::= expr IS expr */
   217,  /* (208) expr ::= expr IS NOT expr */
-
   217,  /* (209) expr ::= NOT expr */
-
   217,  /* (210) expr ::= BITNOT expr */
-
   217,  /* (211) expr ::= PLUS|MINUS expr */
-
   217,  /* (212) expr ::= expr PTR expr */
-
   275,  /* (213) between_op ::= BETWEEN */
-
   275,  /* (214) between_op ::= NOT BETWEEN */
-
   217,  /* (215) expr ::= expr between_op expr AND expr */
-
   276,  /* (216) in_op ::= IN */
-
   276,  /* (217) in_op ::= NOT IN */
-
   217,  /* (218) expr ::= expr in_op LP exprlist RP */
-
   217,  /* (219) expr ::= LP select RP */
-
   217,  /* (220) expr ::= expr in_op LP select RP */
-
   217,  /* (221) expr ::= expr in_op nm dbnm paren_exprlist */
-
   217,  /* (222) expr ::= EXISTS LP select RP */
-
   217,  /* (223) expr ::= CASE case_operand case_exprlist case_else END */
-
   279,  /* (224) case_exprlist ::= case_exprlist WHEN expr THEN expr */
-
   279,  /* (225) case_exprlist ::= WHEN expr THEN expr */
-
   280,  /* (226) case_else ::= ELSE expr */
-
   280,  /* (227) case_else ::= */
-
   278,  /* (228) case_operand ::= expr */
-
   278,  /* (229) case_operand ::= */
-
   262,  /* (230) exprlist ::= */
-
   253,  /* (231) nexprlist ::= nexprlist COMMA expr */
-
   253,  /* (232) nexprlist ::= expr */
-
   277,  /* (233) paren_exprlist ::= */
-
   277,  /* (234) paren_exprlist ::= LP exprlist RP */
-
   190,  /* (235) cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt */
-
   281,  /* (236) uniqueflag ::= UNIQUE */
-
   281,  /* (237) uniqueflag ::= */
-
   221,  /* (238) eidlist_opt ::= */
-
   221,  /* (239) eidlist_opt ::= LP eidlist RP */
-
   232,  /* (240) eidlist ::= eidlist COMMA nm collate sortorder */
-
   232,  /* (241) eidlist ::= nm collate sortorder */
-
   282,  /* (242) collate ::= */
-
   282,  /* (243) collate ::= COLLATE ID|STRING */
-
   190,  /* (244) cmd ::= DROP INDEX ifexists fullname */
-
   190,  /* (245) cmd ::= VACUUM vinto */
-
   190,  /* (246) cmd ::= VACUUM nm vinto */
-
   283,  /* (247) vinto ::= INTO expr */
-
   283,  /* (248) vinto ::= */
-
   190,  /* (249) cmd ::= PRAGMA nm dbnm */
-
   190,  /* (250) cmd ::= PRAGMA nm dbnm EQ nmnum */
-
   190,  /* (251) cmd ::= PRAGMA nm dbnm LP nmnum RP */
-
   190,  /* (252) cmd ::= PRAGMA nm dbnm EQ minus_num */
-
   190,  /* (253) cmd ::= PRAGMA nm dbnm LP minus_num RP */
-
   211,  /* (254) plus_num ::= PLUS INTEGER|FLOAT */
-
   212,  /* (255) minus_num ::= MINUS INTEGER|FLOAT */
-
   190,  /* (256) cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END */
-
   285,  /* (257) trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause */
-
   287,  /* (258) trigger_time ::= BEFORE|AFTER */
-
   287,  /* (259) trigger_time ::= INSTEAD OF */
-
   287,  /* (260) trigger_time ::= */
-
   288,  /* (261) trigger_event ::= DELETE|INSERT */
-
   288,  /* (262) trigger_event ::= UPDATE */
-
   288,  /* (263) trigger_event ::= UPDATE OF idlist */
-
   290,  /* (264) when_clause ::= */
-
   290,  /* (265) when_clause ::= WHEN expr */
-
   286,  /* (266) trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI */
-
   286,  /* (267) trigger_cmd_list ::= trigger_cmd SEMI */
-
   292,  /* (268) trnm ::= nm DOT nm */
-
   293,  /* (269) tridxby ::= INDEXED BY nm */
-
   293,  /* (270) tridxby ::= NOT INDEXED */
-
   291,  /* (271) trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt */
-
   291,  /* (272) trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt */
-
   291,  /* (273) trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt */
-
   291,  /* (274) trigger_cmd ::= scanpt select scanpt */
-
   217,  /* (275) expr ::= RAISE LP IGNORE RP */
-
   217,  /* (276) expr ::= RAISE LP raisetype COMMA nm RP */
-
   236,  /* (277) raisetype ::= ROLLBACK */
-
   236,  /* (278) raisetype ::= ABORT */
-
   236,  /* (279) raisetype ::= FAIL */
-
   190,  /* (280) cmd ::= DROP TRIGGER ifexists fullname */
-
   190,  /* (281) cmd ::= ATTACH database_kw_opt expr AS expr key_opt */
-
   190,  /* (282) cmd ::= DETACH database_kw_opt expr */
-
   295,  /* (283) key_opt ::= */
-
   295,  /* (284) key_opt ::= KEY expr */
-
   190,  /* (285) cmd ::= REINDEX */
-
   190,  /* (286) cmd ::= REINDEX nm dbnm */
-
   190,  /* (287) cmd ::= ANALYZE */
-
   190,  /* (288) cmd ::= ANALYZE nm dbnm */
-
   190,  /* (289) cmd ::= ALTER TABLE fullname RENAME TO nm */
-
   190,  /* (290) cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist */
-
   190,  /* (291) cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm */
-
   296,  /* (292) add_column_fullname ::= fullname */
-
   190,  /* (293) cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm */
-
   190,  /* (294) cmd ::= create_vtab */
-
   190,  /* (295) cmd ::= create_vtab LP vtabarglist RP */
-
   298,  /* (296) create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm */
-
   300,  /* (297) vtabarg ::= */
-
   301,  /* (298) vtabargtoken ::= ANY */
-
   301,  /* (299) vtabargtoken ::= lp anylist RP */
-
   302,  /* (300) lp ::= LP */
-
   266,  /* (301) with ::= WITH wqlist */
-
   266,  /* (302) with ::= WITH RECURSIVE wqlist */
-
   305,  /* (303) wqas ::= AS */
-
   305,  /* (304) wqas ::= AS MATERIALIZED */
-
   305,  /* (305) wqas ::= AS NOT MATERIALIZED */
-
   304,  /* (306) wqitem ::= nm eidlist_opt wqas LP select RP */
-
   241,  /* (307) wqlist ::= wqitem */
-
   241,  /* (308) wqlist ::= wqlist COMMA wqitem */
-
   306,  /* (309) windowdefn_list ::= windowdefn */
-
   306,  /* (310) windowdefn_list ::= windowdefn_list COMMA windowdefn */
-
   307,  /* (311) windowdefn ::= nm AS LP window RP */
-
   308,  /* (312) window ::= PARTITION BY nexprlist orderby_opt frame_opt */
-
   308,  /* (313) window ::= nm PARTITION BY nexprlist orderby_opt frame_opt */
-
   308,  /* (314) window ::= ORDER BY sortlist frame_opt */
-
   308,  /* (315) window ::= nm ORDER BY sortlist frame_opt */
-
   308,  /* (316) window ::= frame_opt */
-
   308,  /* (317) window ::= nm frame_opt */
-
   309,  /* (318) frame_opt ::= */
-
   309,  /* (319) frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt */
-
   309,  /* (320) frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt */
-
   313,  /* (321) range_or_rows ::= RANGE|ROWS|GROUPS */
-
   315,  /* (322) frame_bound_s ::= frame_bound */
-
   315,  /* (323) frame_bound_s ::= UNBOUNDED PRECEDING */
-
   316,  /* (324) frame_bound_e ::= frame_bound */
-
   316,  /* (325) frame_bound_e ::= UNBOUNDED FOLLOWING */
-
   314,  /* (326) frame_bound ::= expr PRECEDING|FOLLOWING */
-
   314,  /* (327) frame_bound ::= CURRENT ROW */
-
   317,  /* (328) frame_exclude_opt ::= */
-
   317,  /* (329) frame_exclude_opt ::= EXCLUDE frame_exclude */
-
   318,  /* (330) frame_exclude ::= NO OTHERS */
-
   318,  /* (331) frame_exclude ::= CURRENT ROW */
-
   318,  /* (332) frame_exclude ::= GROUP|TIES */
-
   251,  /* (333) window_clause ::= WINDOW windowdefn_list */
-
   273,  /* (334) filter_over ::= filter_clause over_clause */
-
   273,  /* (335) filter_over ::= over_clause */
-
   273,  /* (336) filter_over ::= filter_clause */
-
   312,  /* (337) over_clause ::= OVER LP window RP */
-
   312,  /* (338) over_clause ::= OVER nm */
-
   311,  /* (339) filter_clause ::= FILTER LP WHERE expr RP */
-
   185,  /* (340) input ::= cmdlist */
-
   186,  /* (341) cmdlist ::= cmdlist ecmd */
-
   186,  /* (342) cmdlist ::= ecmd */
-
   187,  /* (343) ecmd ::= SEMI */
-
   187,  /* (344) ecmd ::= cmdx SEMI */
-
   187,  /* (345) ecmd ::= explain cmdx SEMI */
-
   192,  /* (346) trans_opt ::= */
-
   192,  /* (347) trans_opt ::= TRANSACTION */
-
   192,  /* (348) trans_opt ::= TRANSACTION nm */
-
   194,  /* (349) savepoint_opt ::= SAVEPOINT */
-
   194,  /* (350) savepoint_opt ::= */
-
   190,  /* (351) cmd ::= create_table create_table_args */
-
   203,  /* (352) table_option_set ::= table_option */
-
   201,  /* (353) columnlist ::= columnlist COMMA columnname carglist */
-
   201,  /* (354) columnlist ::= columnname carglist */
-
   193,  /* (355) nm ::= ID|INDEXED */
-
   193,  /* (356) nm ::= STRING */
-
   193,  /* (357) nm ::= JOIN_KW */
-
   208,  /* (358) typetoken ::= typename */
-
   209,  /* (359) typename ::= ID|STRING */
-
   210,  /* (360) signed ::= plus_num */
-
   210,  /* (361) signed ::= minus_num */
-
   207,  /* (362) carglist ::= carglist ccons */
-
   207,  /* (363) carglist ::= */
-
   215,  /* (364) ccons ::= NULL onconf */
-
   215,  /* (365) ccons ::= GENERATED ALWAYS AS generated */
-
   215,  /* (366) ccons ::= AS generated */
-
   202,  /* (367) conslist_opt ::= COMMA conslist */
-
   228,  /* (368) conslist ::= conslist tconscomma tcons */
-
   228,  /* (369) conslist ::= tcons */
-
   229,  /* (370) tconscomma ::= */
-
   233,  /* (371) defer_subclause_opt ::= defer_subclause */
-
   235,  /* (372) resolvetype ::= raisetype */
-
   239,  /* (373) selectnowith ::= oneselect */
-
   240,  /* (374) oneselect ::= values */
-
   254,  /* (375) sclp ::= selcollist COMMA */
-
   255,  /* (376) as ::= ID|STRING */
-
   272,  /* (377) returning ::= */
-
   217,  /* (378) expr ::= term */
-
   274,  /* (379) likeop ::= LIKE_KW|MATCH */
-
   262,  /* (380) exprlist ::= nexprlist */
-
   284,  /* (381) nmnum ::= plus_num */
-
   284,  /* (382) nmnum ::= nm */
-
   284,  /* (383) nmnum ::= ON */
-
   284,  /* (384) nmnum ::= DELETE */
-
   284,  /* (385) nmnum ::= DEFAULT */
-
   211,  /* (386) plus_num ::= INTEGER|FLOAT */
-
   289,  /* (387) foreach_clause ::= */
-
   289,  /* (388) foreach_clause ::= FOR EACH ROW */
-
   292,  /* (389) trnm ::= nm */
-
   293,  /* (390) tridxby ::= */
-
   294,  /* (391) database_kw_opt ::= DATABASE */
-
   294,  /* (392) database_kw_opt ::= */
-
   297,  /* (393) kwcolumn_opt ::= */
-
   297,  /* (394) kwcolumn_opt ::= COLUMNKW */
-
   299,  /* (395) vtabarglist ::= vtabarg */
-
   299,  /* (396) vtabarglist ::= vtabarglist COMMA vtabarg */
-
   300,  /* (397) vtabarg ::= vtabarg vtabargtoken */
-
   303,  /* (398) anylist ::= */
-
   303,  /* (399) anylist ::= anylist LP anylist RP */
-
   303,  /* (400) anylist ::= anylist ANY */
-
   266,  /* (401) with ::= */
+
   217,  /* (209) expr ::= expr IS NOT DISTINCT FROM expr */
+
   217,  /* (210) expr ::= expr IS DISTINCT FROM expr */
+
   217,  /* (211) expr ::= NOT expr */
+
   217,  /* (212) expr ::= BITNOT expr */
+
   217,  /* (213) expr ::= PLUS|MINUS expr */
+
   217,  /* (214) expr ::= expr PTR expr */
+
   275,  /* (215) between_op ::= BETWEEN */
+
   275,  /* (216) between_op ::= NOT BETWEEN */
+
   217,  /* (217) expr ::= expr between_op expr AND expr */
+
   276,  /* (218) in_op ::= IN */
+
   276,  /* (219) in_op ::= NOT IN */
+
   217,  /* (220) expr ::= expr in_op LP exprlist RP */
+
   217,  /* (221) expr ::= LP select RP */
+
   217,  /* (222) expr ::= expr in_op LP select RP */
+
   217,  /* (223) expr ::= expr in_op nm dbnm paren_exprlist */
+
   217,  /* (224) expr ::= EXISTS LP select RP */
+
   217,  /* (225) expr ::= CASE case_operand case_exprlist case_else END */
+
   279,  /* (226) case_exprlist ::= case_exprlist WHEN expr THEN expr */
+
   279,  /* (227) case_exprlist ::= WHEN expr THEN expr */
+
   280,  /* (228) case_else ::= ELSE expr */
+
   280,  /* (229) case_else ::= */
+
   278,  /* (230) case_operand ::= expr */
+
   278,  /* (231) case_operand ::= */
+
   261,  /* (232) exprlist ::= */
+
   253,  /* (233) nexprlist ::= nexprlist COMMA expr */
+
   253,  /* (234) nexprlist ::= expr */
+
   277,  /* (235) paren_exprlist ::= */
+
   277,  /* (236) paren_exprlist ::= LP exprlist RP */
+
   190,  /* (237) cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt */
+
   281,  /* (238) uniqueflag ::= UNIQUE */
+
   281,  /* (239) uniqueflag ::= */
+
   221,  /* (240) eidlist_opt ::= */
+
   221,  /* (241) eidlist_opt ::= LP eidlist RP */
+
   232,  /* (242) eidlist ::= eidlist COMMA nm collate sortorder */
+
   232,  /* (243) eidlist ::= nm collate sortorder */
+
   282,  /* (244) collate ::= */
+
   282,  /* (245) collate ::= COLLATE ID|STRING */
+
   190,  /* (246) cmd ::= DROP INDEX ifexists fullname */
+
   190,  /* (247) cmd ::= VACUUM vinto */
+
   190,  /* (248) cmd ::= VACUUM nm vinto */
+
   283,  /* (249) vinto ::= INTO expr */
+
   283,  /* (250) vinto ::= */
+
   190,  /* (251) cmd ::= PRAGMA nm dbnm */
+
   190,  /* (252) cmd ::= PRAGMA nm dbnm EQ nmnum */
+
   190,  /* (253) cmd ::= PRAGMA nm dbnm LP nmnum RP */
+
   190,  /* (254) cmd ::= PRAGMA nm dbnm EQ minus_num */
+
   190,  /* (255) cmd ::= PRAGMA nm dbnm LP minus_num RP */
+
   211,  /* (256) plus_num ::= PLUS INTEGER|FLOAT */
+
   212,  /* (257) minus_num ::= MINUS INTEGER|FLOAT */
+
   190,  /* (258) cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END */
+
   285,  /* (259) trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause */
+
   287,  /* (260) trigger_time ::= BEFORE|AFTER */
+
   287,  /* (261) trigger_time ::= INSTEAD OF */
+
   287,  /* (262) trigger_time ::= */
+
   288,  /* (263) trigger_event ::= DELETE|INSERT */
+
   288,  /* (264) trigger_event ::= UPDATE */
+
   288,  /* (265) trigger_event ::= UPDATE OF idlist */
+
   290,  /* (266) when_clause ::= */
+
   290,  /* (267) when_clause ::= WHEN expr */
+
   286,  /* (268) trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI */
+
   286,  /* (269) trigger_cmd_list ::= trigger_cmd SEMI */
+
   292,  /* (270) trnm ::= nm DOT nm */
+
   293,  /* (271) tridxby ::= INDEXED BY nm */
+
   293,  /* (272) tridxby ::= NOT INDEXED */
+
   291,  /* (273) trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt */
+
   291,  /* (274) trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt */
+
   291,  /* (275) trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt */
+
   291,  /* (276) trigger_cmd ::= scanpt select scanpt */
+
   217,  /* (277) expr ::= RAISE LP IGNORE RP */
+
   217,  /* (278) expr ::= RAISE LP raisetype COMMA nm RP */
+
   236,  /* (279) raisetype ::= ROLLBACK */
+
   236,  /* (280) raisetype ::= ABORT */
+
   236,  /* (281) raisetype ::= FAIL */
+
   190,  /* (282) cmd ::= DROP TRIGGER ifexists fullname */
+
   190,  /* (283) cmd ::= ATTACH database_kw_opt expr AS expr key_opt */
+
   190,  /* (284) cmd ::= DETACH database_kw_opt expr */
+
   295,  /* (285) key_opt ::= */
+
   295,  /* (286) key_opt ::= KEY expr */
+
   190,  /* (287) cmd ::= REINDEX */
+
   190,  /* (288) cmd ::= REINDEX nm dbnm */
+
   190,  /* (289) cmd ::= ANALYZE */
+
   190,  /* (290) cmd ::= ANALYZE nm dbnm */
+
   190,  /* (291) cmd ::= ALTER TABLE fullname RENAME TO nm */
+
   190,  /* (292) cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist */
+
   190,  /* (293) cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm */
+
   296,  /* (294) add_column_fullname ::= fullname */
+
   190,  /* (295) cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm */
+
   190,  /* (296) cmd ::= create_vtab */
+
   190,  /* (297) cmd ::= create_vtab LP vtabarglist RP */
+
   298,  /* (298) create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm */
+
   300,  /* (299) vtabarg ::= */
+
   301,  /* (300) vtabargtoken ::= ANY */
+
   301,  /* (301) vtabargtoken ::= lp anylist RP */
+
   302,  /* (302) lp ::= LP */
+
   266,  /* (303) with ::= WITH wqlist */
+
   266,  /* (304) with ::= WITH RECURSIVE wqlist */
+
   305,  /* (305) wqas ::= AS */
+
   305,  /* (306) wqas ::= AS MATERIALIZED */
+
   305,  /* (307) wqas ::= AS NOT MATERIALIZED */
+
   304,  /* (308) wqitem ::= nm eidlist_opt wqas LP select RP */
+
   241,  /* (309) wqlist ::= wqitem */
+
   241,  /* (310) wqlist ::= wqlist COMMA wqitem */
+
   306,  /* (311) windowdefn_list ::= windowdefn */
+
   306,  /* (312) windowdefn_list ::= windowdefn_list COMMA windowdefn */
+
   307,  /* (313) windowdefn ::= nm AS LP window RP */
+
   308,  /* (314) window ::= PARTITION BY nexprlist orderby_opt frame_opt */
+
   308,  /* (315) window ::= nm PARTITION BY nexprlist orderby_opt frame_opt */
+
   308,  /* (316) window ::= ORDER BY sortlist frame_opt */
+
   308,  /* (317) window ::= nm ORDER BY sortlist frame_opt */
+
   308,  /* (318) window ::= frame_opt */
+
   308,  /* (319) window ::= nm frame_opt */
+
   309,  /* (320) frame_opt ::= */
+
   309,  /* (321) frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt */
+
   309,  /* (322) frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt */
+
   313,  /* (323) range_or_rows ::= RANGE|ROWS|GROUPS */
+
   315,  /* (324) frame_bound_s ::= frame_bound */
+
   315,  /* (325) frame_bound_s ::= UNBOUNDED PRECEDING */
+
   316,  /* (326) frame_bound_e ::= frame_bound */
+
   316,  /* (327) frame_bound_e ::= UNBOUNDED FOLLOWING */
+
   314,  /* (328) frame_bound ::= expr PRECEDING|FOLLOWING */
+
   314,  /* (329) frame_bound ::= CURRENT ROW */
+
   317,  /* (330) frame_exclude_opt ::= */
+
   317,  /* (331) frame_exclude_opt ::= EXCLUDE frame_exclude */
+
   318,  /* (332) frame_exclude ::= NO OTHERS */
+
   318,  /* (333) frame_exclude ::= CURRENT ROW */
+
   318,  /* (334) frame_exclude ::= GROUP|TIES */
+
   251,  /* (335) window_clause ::= WINDOW windowdefn_list */
+
   273,  /* (336) filter_over ::= filter_clause over_clause */
+
   273,  /* (337) filter_over ::= over_clause */
+
   273,  /* (338) filter_over ::= filter_clause */
+
   312,  /* (339) over_clause ::= OVER LP window RP */
+
   312,  /* (340) over_clause ::= OVER nm */
+
   311,  /* (341) filter_clause ::= FILTER LP WHERE expr RP */
+
   185,  /* (342) input ::= cmdlist */
+
   186,  /* (343) cmdlist ::= cmdlist ecmd */
+
   186,  /* (344) cmdlist ::= ecmd */
+
   187,  /* (345) ecmd ::= SEMI */
+
   187,  /* (346) ecmd ::= cmdx SEMI */
+
   187,  /* (347) ecmd ::= explain cmdx SEMI */
+
   192,  /* (348) trans_opt ::= */
+
   192,  /* (349) trans_opt ::= TRANSACTION */
+
   192,  /* (350) trans_opt ::= TRANSACTION nm */
+
   194,  /* (351) savepoint_opt ::= SAVEPOINT */
+
   194,  /* (352) savepoint_opt ::= */
+
   190,  /* (353) cmd ::= create_table create_table_args */
+
   203,  /* (354) table_option_set ::= table_option */
+
   201,  /* (355) columnlist ::= columnlist COMMA columnname carglist */
+
   201,  /* (356) columnlist ::= columnname carglist */
+
   193,  /* (357) nm ::= ID|INDEXED */
+
   193,  /* (358) nm ::= STRING */
+
   193,  /* (359) nm ::= JOIN_KW */
+
   208,  /* (360) typetoken ::= typename */
+
   209,  /* (361) typename ::= ID|STRING */
+
   210,  /* (362) signed ::= plus_num */
+
   210,  /* (363) signed ::= minus_num */
+
   207,  /* (364) carglist ::= carglist ccons */
+
   207,  /* (365) carglist ::= */
+
   215,  /* (366) ccons ::= NULL onconf */
+
   215,  /* (367) ccons ::= GENERATED ALWAYS AS generated */
+
   215,  /* (368) ccons ::= AS generated */
+
   202,  /* (369) conslist_opt ::= COMMA conslist */
+
   228,  /* (370) conslist ::= conslist tconscomma tcons */
+
   228,  /* (371) conslist ::= tcons */
+
   229,  /* (372) tconscomma ::= */
+
   233,  /* (373) defer_subclause_opt ::= defer_subclause */
+
   235,  /* (374) resolvetype ::= raisetype */
+
   239,  /* (375) selectnowith ::= oneselect */
+
   240,  /* (376) oneselect ::= values */
+
   254,  /* (377) sclp ::= selcollist COMMA */
+
   255,  /* (378) as ::= ID|STRING */
+
   264,  /* (379) indexed_opt ::= indexed_by */
+
   272,  /* (380) returning ::= */
+
   217,  /* (381) expr ::= term */
+
   274,  /* (382) likeop ::= LIKE_KW|MATCH */
+
   261,  /* (383) exprlist ::= nexprlist */
+
   284,  /* (384) nmnum ::= plus_num */
+
   284,  /* (385) nmnum ::= nm */
+
   284,  /* (386) nmnum ::= ON */
+
   284,  /* (387) nmnum ::= DELETE */
+
   284,  /* (388) nmnum ::= DEFAULT */
+
   211,  /* (389) plus_num ::= INTEGER|FLOAT */
+
   289,  /* (390) foreach_clause ::= */
+
   289,  /* (391) foreach_clause ::= FOR EACH ROW */
+
   292,  /* (392) trnm ::= nm */
+
   293,  /* (393) tridxby ::= */
+
   294,  /* (394) database_kw_opt ::= DATABASE */
+
   294,  /* (395) database_kw_opt ::= */
+
   297,  /* (396) kwcolumn_opt ::= */
+
   297,  /* (397) kwcolumn_opt ::= COLUMNKW */
+
   299,  /* (398) vtabarglist ::= vtabarg */
+
   299,  /* (399) vtabarglist ::= vtabarglist COMMA vtabarg */
+
   300,  /* (400) vtabarg ::= vtabarg vtabargtoken */
+
   303,  /* (401) anylist ::= */
+
   303,  /* (402) anylist ::= anylist LP anylist RP */
+
   303,  /* (403) anylist ::= anylist ANY */
+
   266,  /* (404) with ::= */
};

/* For rule J, yyRuleInfoNRhs[J] contains the negative of the number
@@ -164640,29 +168486,29 @@ static const signed char yyRuleInfoNRhs[] = {
   -2,  /* (106) from ::= FROM seltablist */
   -2,  /* (107) stl_prefix ::= seltablist joinop */
    0,  /* (108) stl_prefix ::= */
-
   -7,  /* (109) seltablist ::= stl_prefix nm dbnm as indexed_opt on_opt using_opt */
-
   -9,  /* (110) seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_opt using_opt */
-
   -7,  /* (111) seltablist ::= stl_prefix LP select RP as on_opt using_opt */
-
   -7,  /* (112) seltablist ::= stl_prefix LP seltablist RP as on_opt using_opt */
-
    0,  /* (113) dbnm ::= */
-
   -2,  /* (114) dbnm ::= DOT nm */
-
   -1,  /* (115) fullname ::= nm */
-
   -3,  /* (116) fullname ::= nm DOT nm */
-
   -1,  /* (117) xfullname ::= nm */
-
   -3,  /* (118) xfullname ::= nm DOT nm */
-
   -5,  /* (119) xfullname ::= nm DOT nm AS nm */
-
   -3,  /* (120) xfullname ::= nm AS nm */
-
   -1,  /* (121) joinop ::= COMMA|JOIN */
-
   -2,  /* (122) joinop ::= JOIN_KW JOIN */
-
   -3,  /* (123) joinop ::= JOIN_KW nm JOIN */
-
   -4,  /* (124) joinop ::= JOIN_KW nm nm JOIN */
-
   -2,  /* (125) on_opt ::= ON expr */
-
    0,  /* (126) on_opt ::= */
-
    0,  /* (127) indexed_opt ::= */
-
   -3,  /* (128) indexed_opt ::= INDEXED BY nm */
-
   -2,  /* (129) indexed_opt ::= NOT INDEXED */
-
   -4,  /* (130) using_opt ::= USING LP idlist RP */
-
    0,  /* (131) using_opt ::= */
+
   -5,  /* (109) seltablist ::= stl_prefix nm dbnm as on_using */
+
   -6,  /* (110) seltablist ::= stl_prefix nm dbnm as indexed_by on_using */
+
   -8,  /* (111) seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_using */
+
   -6,  /* (112) seltablist ::= stl_prefix LP select RP as on_using */
+
   -6,  /* (113) seltablist ::= stl_prefix LP seltablist RP as on_using */
+
    0,  /* (114) dbnm ::= */
+
   -2,  /* (115) dbnm ::= DOT nm */
+
   -1,  /* (116) fullname ::= nm */
+
   -3,  /* (117) fullname ::= nm DOT nm */
+
   -1,  /* (118) xfullname ::= nm */
+
   -3,  /* (119) xfullname ::= nm DOT nm */
+
   -5,  /* (120) xfullname ::= nm DOT nm AS nm */
+
   -3,  /* (121) xfullname ::= nm AS nm */
+
   -1,  /* (122) joinop ::= COMMA|JOIN */
+
   -2,  /* (123) joinop ::= JOIN_KW JOIN */
+
   -3,  /* (124) joinop ::= JOIN_KW nm JOIN */
+
   -4,  /* (125) joinop ::= JOIN_KW nm nm JOIN */
+
   -2,  /* (126) on_using ::= ON expr */
+
   -4,  /* (127) on_using ::= USING LP idlist RP */
+
    0,  /* (128) on_using ::= */
+
    0,  /* (129) indexed_opt ::= */
+
   -3,  /* (130) indexed_by ::= INDEXED BY nm */
+
   -2,  /* (131) indexed_by ::= NOT INDEXED */
    0,  /* (132) orderby_opt ::= */
   -3,  /* (133) orderby_opt ::= ORDER BY sortlist */
   -5,  /* (134) sortlist ::= sortlist COMMA expr sortorder nulls */
@@ -164740,199 +168586,202 @@ static const signed char yyRuleInfoNRhs[] = {
   -3,  /* (206) expr ::= expr NOT NULL */
   -3,  /* (207) expr ::= expr IS expr */
   -4,  /* (208) expr ::= expr IS NOT expr */
-
   -2,  /* (209) expr ::= NOT expr */
-
   -2,  /* (210) expr ::= BITNOT expr */
-
   -2,  /* (211) expr ::= PLUS|MINUS expr */
-
   -3,  /* (212) expr ::= expr PTR expr */
-
   -1,  /* (213) between_op ::= BETWEEN */
-
   -2,  /* (214) between_op ::= NOT BETWEEN */
-
   -5,  /* (215) expr ::= expr between_op expr AND expr */
-
   -1,  /* (216) in_op ::= IN */
-
   -2,  /* (217) in_op ::= NOT IN */
-
   -5,  /* (218) expr ::= expr in_op LP exprlist RP */
-
   -3,  /* (219) expr ::= LP select RP */
-
   -5,  /* (220) expr ::= expr in_op LP select RP */
-
   -5,  /* (221) expr ::= expr in_op nm dbnm paren_exprlist */
-
   -4,  /* (222) expr ::= EXISTS LP select RP */
-
   -5,  /* (223) expr ::= CASE case_operand case_exprlist case_else END */
-
   -5,  /* (224) case_exprlist ::= case_exprlist WHEN expr THEN expr */
-
   -4,  /* (225) case_exprlist ::= WHEN expr THEN expr */
-
   -2,  /* (226) case_else ::= ELSE expr */
-
    0,  /* (227) case_else ::= */
-
   -1,  /* (228) case_operand ::= expr */
-
    0,  /* (229) case_operand ::= */
-
    0,  /* (230) exprlist ::= */
-
   -3,  /* (231) nexprlist ::= nexprlist COMMA expr */
-
   -1,  /* (232) nexprlist ::= expr */
-
    0,  /* (233) paren_exprlist ::= */
-
   -3,  /* (234) paren_exprlist ::= LP exprlist RP */
-
  -12,  /* (235) cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt */
-
   -1,  /* (236) uniqueflag ::= UNIQUE */
-
    0,  /* (237) uniqueflag ::= */
-
    0,  /* (238) eidlist_opt ::= */
-
   -3,  /* (239) eidlist_opt ::= LP eidlist RP */
-
   -5,  /* (240) eidlist ::= eidlist COMMA nm collate sortorder */
-
   -3,  /* (241) eidlist ::= nm collate sortorder */
-
    0,  /* (242) collate ::= */
-
   -2,  /* (243) collate ::= COLLATE ID|STRING */
-
   -4,  /* (244) cmd ::= DROP INDEX ifexists fullname */
-
   -2,  /* (245) cmd ::= VACUUM vinto */
-
   -3,  /* (246) cmd ::= VACUUM nm vinto */
-
   -2,  /* (247) vinto ::= INTO expr */
-
    0,  /* (248) vinto ::= */
-
   -3,  /* (249) cmd ::= PRAGMA nm dbnm */
-
   -5,  /* (250) cmd ::= PRAGMA nm dbnm EQ nmnum */
-
   -6,  /* (251) cmd ::= PRAGMA nm dbnm LP nmnum RP */
-
   -5,  /* (252) cmd ::= PRAGMA nm dbnm EQ minus_num */
-
   -6,  /* (253) cmd ::= PRAGMA nm dbnm LP minus_num RP */
-
   -2,  /* (254) plus_num ::= PLUS INTEGER|FLOAT */
-
   -2,  /* (255) minus_num ::= MINUS INTEGER|FLOAT */
-
   -5,  /* (256) cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END */
-
  -11,  /* (257) trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause */
-
   -1,  /* (258) trigger_time ::= BEFORE|AFTER */
-
   -2,  /* (259) trigger_time ::= INSTEAD OF */
-
    0,  /* (260) trigger_time ::= */
-
   -1,  /* (261) trigger_event ::= DELETE|INSERT */
-
   -1,  /* (262) trigger_event ::= UPDATE */
-
   -3,  /* (263) trigger_event ::= UPDATE OF idlist */
-
    0,  /* (264) when_clause ::= */
-
   -2,  /* (265) when_clause ::= WHEN expr */
-
   -3,  /* (266) trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI */
-
   -2,  /* (267) trigger_cmd_list ::= trigger_cmd SEMI */
-
   -3,  /* (268) trnm ::= nm DOT nm */
-
   -3,  /* (269) tridxby ::= INDEXED BY nm */
-
   -2,  /* (270) tridxby ::= NOT INDEXED */
-
   -9,  /* (271) trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt */
-
   -8,  /* (272) trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt */
-
   -6,  /* (273) trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt */
-
   -3,  /* (274) trigger_cmd ::= scanpt select scanpt */
-
   -4,  /* (275) expr ::= RAISE LP IGNORE RP */
-
   -6,  /* (276) expr ::= RAISE LP raisetype COMMA nm RP */
-
   -1,  /* (277) raisetype ::= ROLLBACK */
-
   -1,  /* (278) raisetype ::= ABORT */
-
   -1,  /* (279) raisetype ::= FAIL */
-
   -4,  /* (280) cmd ::= DROP TRIGGER ifexists fullname */
-
   -6,  /* (281) cmd ::= ATTACH database_kw_opt expr AS expr key_opt */
-
   -3,  /* (282) cmd ::= DETACH database_kw_opt expr */
-
    0,  /* (283) key_opt ::= */
-
   -2,  /* (284) key_opt ::= KEY expr */
-
   -1,  /* (285) cmd ::= REINDEX */
-
   -3,  /* (286) cmd ::= REINDEX nm dbnm */
-
   -1,  /* (287) cmd ::= ANALYZE */
-
   -3,  /* (288) cmd ::= ANALYZE nm dbnm */
-
   -6,  /* (289) cmd ::= ALTER TABLE fullname RENAME TO nm */
-
   -7,  /* (290) cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist */
-
   -6,  /* (291) cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm */
-
   -1,  /* (292) add_column_fullname ::= fullname */
-
   -8,  /* (293) cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm */
-
   -1,  /* (294) cmd ::= create_vtab */
-
   -4,  /* (295) cmd ::= create_vtab LP vtabarglist RP */
-
   -8,  /* (296) create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm */
-
    0,  /* (297) vtabarg ::= */
-
   -1,  /* (298) vtabargtoken ::= ANY */
-
   -3,  /* (299) vtabargtoken ::= lp anylist RP */
-
   -1,  /* (300) lp ::= LP */
-
   -2,  /* (301) with ::= WITH wqlist */
-
   -3,  /* (302) with ::= WITH RECURSIVE wqlist */
-
   -1,  /* (303) wqas ::= AS */
-
   -2,  /* (304) wqas ::= AS MATERIALIZED */
-
   -3,  /* (305) wqas ::= AS NOT MATERIALIZED */
-
   -6,  /* (306) wqitem ::= nm eidlist_opt wqas LP select RP */
-
   -1,  /* (307) wqlist ::= wqitem */
-
   -3,  /* (308) wqlist ::= wqlist COMMA wqitem */
-
   -1,  /* (309) windowdefn_list ::= windowdefn */
-
   -3,  /* (310) windowdefn_list ::= windowdefn_list COMMA windowdefn */
-
   -5,  /* (311) windowdefn ::= nm AS LP window RP */
-
   -5,  /* (312) window ::= PARTITION BY nexprlist orderby_opt frame_opt */
-
   -6,  /* (313) window ::= nm PARTITION BY nexprlist orderby_opt frame_opt */
-
   -4,  /* (314) window ::= ORDER BY sortlist frame_opt */
-
   -5,  /* (315) window ::= nm ORDER BY sortlist frame_opt */
-
   -1,  /* (316) window ::= frame_opt */
-
   -2,  /* (317) window ::= nm frame_opt */
-
    0,  /* (318) frame_opt ::= */
-
   -3,  /* (319) frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt */
-
   -6,  /* (320) frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt */
-
   -1,  /* (321) range_or_rows ::= RANGE|ROWS|GROUPS */
-
   -1,  /* (322) frame_bound_s ::= frame_bound */
-
   -2,  /* (323) frame_bound_s ::= UNBOUNDED PRECEDING */
-
   -1,  /* (324) frame_bound_e ::= frame_bound */
-
   -2,  /* (325) frame_bound_e ::= UNBOUNDED FOLLOWING */
-
   -2,  /* (326) frame_bound ::= expr PRECEDING|FOLLOWING */
-
   -2,  /* (327) frame_bound ::= CURRENT ROW */
-
    0,  /* (328) frame_exclude_opt ::= */
-
   -2,  /* (329) frame_exclude_opt ::= EXCLUDE frame_exclude */
-
   -2,  /* (330) frame_exclude ::= NO OTHERS */
-
   -2,  /* (331) frame_exclude ::= CURRENT ROW */
-
   -1,  /* (332) frame_exclude ::= GROUP|TIES */
-
   -2,  /* (333) window_clause ::= WINDOW windowdefn_list */
-
   -2,  /* (334) filter_over ::= filter_clause over_clause */
-
   -1,  /* (335) filter_over ::= over_clause */
-
   -1,  /* (336) filter_over ::= filter_clause */
-
   -4,  /* (337) over_clause ::= OVER LP window RP */
-
   -2,  /* (338) over_clause ::= OVER nm */
-
   -5,  /* (339) filter_clause ::= FILTER LP WHERE expr RP */
-
   -1,  /* (340) input ::= cmdlist */
-
   -2,  /* (341) cmdlist ::= cmdlist ecmd */
-
   -1,  /* (342) cmdlist ::= ecmd */
-
   -1,  /* (343) ecmd ::= SEMI */
-
   -2,  /* (344) ecmd ::= cmdx SEMI */
-
   -3,  /* (345) ecmd ::= explain cmdx SEMI */
-
    0,  /* (346) trans_opt ::= */
-
   -1,  /* (347) trans_opt ::= TRANSACTION */
-
   -2,  /* (348) trans_opt ::= TRANSACTION nm */
-
   -1,  /* (349) savepoint_opt ::= SAVEPOINT */
-
    0,  /* (350) savepoint_opt ::= */
-
   -2,  /* (351) cmd ::= create_table create_table_args */
-
   -1,  /* (352) table_option_set ::= table_option */
-
   -4,  /* (353) columnlist ::= columnlist COMMA columnname carglist */
-
   -2,  /* (354) columnlist ::= columnname carglist */
-
   -1,  /* (355) nm ::= ID|INDEXED */
-
   -1,  /* (356) nm ::= STRING */
-
   -1,  /* (357) nm ::= JOIN_KW */
-
   -1,  /* (358) typetoken ::= typename */
-
   -1,  /* (359) typename ::= ID|STRING */
-
   -1,  /* (360) signed ::= plus_num */
-
   -1,  /* (361) signed ::= minus_num */
-
   -2,  /* (362) carglist ::= carglist ccons */
-
    0,  /* (363) carglist ::= */
-
   -2,  /* (364) ccons ::= NULL onconf */
-
   -4,  /* (365) ccons ::= GENERATED ALWAYS AS generated */
-
   -2,  /* (366) ccons ::= AS generated */
-
   -2,  /* (367) conslist_opt ::= COMMA conslist */
-
   -3,  /* (368) conslist ::= conslist tconscomma tcons */
-
   -1,  /* (369) conslist ::= tcons */
-
    0,  /* (370) tconscomma ::= */
-
   -1,  /* (371) defer_subclause_opt ::= defer_subclause */
-
   -1,  /* (372) resolvetype ::= raisetype */
-
   -1,  /* (373) selectnowith ::= oneselect */
-
   -1,  /* (374) oneselect ::= values */
-
   -2,  /* (375) sclp ::= selcollist COMMA */
-
   -1,  /* (376) as ::= ID|STRING */
-
    0,  /* (377) returning ::= */
-
   -1,  /* (378) expr ::= term */
-
   -1,  /* (379) likeop ::= LIKE_KW|MATCH */
-
   -1,  /* (380) exprlist ::= nexprlist */
-
   -1,  /* (381) nmnum ::= plus_num */
-
   -1,  /* (382) nmnum ::= nm */
-
   -1,  /* (383) nmnum ::= ON */
-
   -1,  /* (384) nmnum ::= DELETE */
-
   -1,  /* (385) nmnum ::= DEFAULT */
-
   -1,  /* (386) plus_num ::= INTEGER|FLOAT */
-
    0,  /* (387) foreach_clause ::= */
-
   -3,  /* (388) foreach_clause ::= FOR EACH ROW */
-
   -1,  /* (389) trnm ::= nm */
-
    0,  /* (390) tridxby ::= */
-
   -1,  /* (391) database_kw_opt ::= DATABASE */
-
    0,  /* (392) database_kw_opt ::= */
-
    0,  /* (393) kwcolumn_opt ::= */
-
   -1,  /* (394) kwcolumn_opt ::= COLUMNKW */
-
   -1,  /* (395) vtabarglist ::= vtabarg */
-
   -3,  /* (396) vtabarglist ::= vtabarglist COMMA vtabarg */
-
   -2,  /* (397) vtabarg ::= vtabarg vtabargtoken */
-
    0,  /* (398) anylist ::= */
-
   -4,  /* (399) anylist ::= anylist LP anylist RP */
-
   -2,  /* (400) anylist ::= anylist ANY */
-
    0,  /* (401) with ::= */
+
   -6,  /* (209) expr ::= expr IS NOT DISTINCT FROM expr */
+
   -5,  /* (210) expr ::= expr IS DISTINCT FROM expr */
+
   -2,  /* (211) expr ::= NOT expr */
+
   -2,  /* (212) expr ::= BITNOT expr */
+
   -2,  /* (213) expr ::= PLUS|MINUS expr */
+
   -3,  /* (214) expr ::= expr PTR expr */
+
   -1,  /* (215) between_op ::= BETWEEN */
+
   -2,  /* (216) between_op ::= NOT BETWEEN */
+
   -5,  /* (217) expr ::= expr between_op expr AND expr */
+
   -1,  /* (218) in_op ::= IN */
+
   -2,  /* (219) in_op ::= NOT IN */
+
   -5,  /* (220) expr ::= expr in_op LP exprlist RP */
+
   -3,  /* (221) expr ::= LP select RP */
+
   -5,  /* (222) expr ::= expr in_op LP select RP */
+
   -5,  /* (223) expr ::= expr in_op nm dbnm paren_exprlist */
+
   -4,  /* (224) expr ::= EXISTS LP select RP */
+
   -5,  /* (225) expr ::= CASE case_operand case_exprlist case_else END */
+
   -5,  /* (226) case_exprlist ::= case_exprlist WHEN expr THEN expr */
+
   -4,  /* (227) case_exprlist ::= WHEN expr THEN expr */
+
   -2,  /* (228) case_else ::= ELSE expr */
+
    0,  /* (229) case_else ::= */
+
   -1,  /* (230) case_operand ::= expr */
+
    0,  /* (231) case_operand ::= */
+
    0,  /* (232) exprlist ::= */
+
   -3,  /* (233) nexprlist ::= nexprlist COMMA expr */
+
   -1,  /* (234) nexprlist ::= expr */
+
    0,  /* (235) paren_exprlist ::= */
+
   -3,  /* (236) paren_exprlist ::= LP exprlist RP */
+
  -12,  /* (237) cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt */
+
   -1,  /* (238) uniqueflag ::= UNIQUE */
+
    0,  /* (239) uniqueflag ::= */
+
    0,  /* (240) eidlist_opt ::= */
+
   -3,  /* (241) eidlist_opt ::= LP eidlist RP */
+
   -5,  /* (242) eidlist ::= eidlist COMMA nm collate sortorder */
+
   -3,  /* (243) eidlist ::= nm collate sortorder */
+
    0,  /* (244) collate ::= */
+
   -2,  /* (245) collate ::= COLLATE ID|STRING */
+
   -4,  /* (246) cmd ::= DROP INDEX ifexists fullname */
+
   -2,  /* (247) cmd ::= VACUUM vinto */
+
   -3,  /* (248) cmd ::= VACUUM nm vinto */
+
   -2,  /* (249) vinto ::= INTO expr */
+
    0,  /* (250) vinto ::= */
+
   -3,  /* (251) cmd ::= PRAGMA nm dbnm */
+
   -5,  /* (252) cmd ::= PRAGMA nm dbnm EQ nmnum */
+
   -6,  /* (253) cmd ::= PRAGMA nm dbnm LP nmnum RP */
+
   -5,  /* (254) cmd ::= PRAGMA nm dbnm EQ minus_num */
+
   -6,  /* (255) cmd ::= PRAGMA nm dbnm LP minus_num RP */
+
   -2,  /* (256) plus_num ::= PLUS INTEGER|FLOAT */
+
   -2,  /* (257) minus_num ::= MINUS INTEGER|FLOAT */
+
   -5,  /* (258) cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END */
+
  -11,  /* (259) trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause */
+
   -1,  /* (260) trigger_time ::= BEFORE|AFTER */
+
   -2,  /* (261) trigger_time ::= INSTEAD OF */
+
    0,  /* (262) trigger_time ::= */
+
   -1,  /* (263) trigger_event ::= DELETE|INSERT */
+
   -1,  /* (264) trigger_event ::= UPDATE */
+
   -3,  /* (265) trigger_event ::= UPDATE OF idlist */
+
    0,  /* (266) when_clause ::= */
+
   -2,  /* (267) when_clause ::= WHEN expr */
+
   -3,  /* (268) trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI */
+
   -2,  /* (269) trigger_cmd_list ::= trigger_cmd SEMI */
+
   -3,  /* (270) trnm ::= nm DOT nm */
+
   -3,  /* (271) tridxby ::= INDEXED BY nm */
+
   -2,  /* (272) tridxby ::= NOT INDEXED */
+
   -9,  /* (273) trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt */
+
   -8,  /* (274) trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt */
+
   -6,  /* (275) trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt */
+
   -3,  /* (276) trigger_cmd ::= scanpt select scanpt */
+
   -4,  /* (277) expr ::= RAISE LP IGNORE RP */
+
   -6,  /* (278) expr ::= RAISE LP raisetype COMMA nm RP */
+
   -1,  /* (279) raisetype ::= ROLLBACK */
+
   -1,  /* (280) raisetype ::= ABORT */
+
   -1,  /* (281) raisetype ::= FAIL */
+
   -4,  /* (282) cmd ::= DROP TRIGGER ifexists fullname */
+
   -6,  /* (283) cmd ::= ATTACH database_kw_opt expr AS expr key_opt */
+
   -3,  /* (284) cmd ::= DETACH database_kw_opt expr */
+
    0,  /* (285) key_opt ::= */
+
   -2,  /* (286) key_opt ::= KEY expr */
+
   -1,  /* (287) cmd ::= REINDEX */
+
   -3,  /* (288) cmd ::= REINDEX nm dbnm */
+
   -1,  /* (289) cmd ::= ANALYZE */
+
   -3,  /* (290) cmd ::= ANALYZE nm dbnm */
+
   -6,  /* (291) cmd ::= ALTER TABLE fullname RENAME TO nm */
+
   -7,  /* (292) cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist */
+
   -6,  /* (293) cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm */
+
   -1,  /* (294) add_column_fullname ::= fullname */
+
   -8,  /* (295) cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm */
+
   -1,  /* (296) cmd ::= create_vtab */
+
   -4,  /* (297) cmd ::= create_vtab LP vtabarglist RP */
+
   -8,  /* (298) create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm */
+
    0,  /* (299) vtabarg ::= */
+
   -1,  /* (300) vtabargtoken ::= ANY */
+
   -3,  /* (301) vtabargtoken ::= lp anylist RP */
+
   -1,  /* (302) lp ::= LP */
+
   -2,  /* (303) with ::= WITH wqlist */
+
   -3,  /* (304) with ::= WITH RECURSIVE wqlist */
+
   -1,  /* (305) wqas ::= AS */
+
   -2,  /* (306) wqas ::= AS MATERIALIZED */
+
   -3,  /* (307) wqas ::= AS NOT MATERIALIZED */
+
   -6,  /* (308) wqitem ::= nm eidlist_opt wqas LP select RP */
+
   -1,  /* (309) wqlist ::= wqitem */
+
   -3,  /* (310) wqlist ::= wqlist COMMA wqitem */
+
   -1,  /* (311) windowdefn_list ::= windowdefn */
+
   -3,  /* (312) windowdefn_list ::= windowdefn_list COMMA windowdefn */
+
   -5,  /* (313) windowdefn ::= nm AS LP window RP */
+
   -5,  /* (314) window ::= PARTITION BY nexprlist orderby_opt frame_opt */
+
   -6,  /* (315) window ::= nm PARTITION BY nexprlist orderby_opt frame_opt */
+
   -4,  /* (316) window ::= ORDER BY sortlist frame_opt */
+
   -5,  /* (317) window ::= nm ORDER BY sortlist frame_opt */
+
   -1,  /* (318) window ::= frame_opt */
+
   -2,  /* (319) window ::= nm frame_opt */
+
    0,  /* (320) frame_opt ::= */
+
   -3,  /* (321) frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt */
+
   -6,  /* (322) frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt */
+
   -1,  /* (323) range_or_rows ::= RANGE|ROWS|GROUPS */
+
   -1,  /* (324) frame_bound_s ::= frame_bound */
+
   -2,  /* (325) frame_bound_s ::= UNBOUNDED PRECEDING */
+
   -1,  /* (326) frame_bound_e ::= frame_bound */
+
   -2,  /* (327) frame_bound_e ::= UNBOUNDED FOLLOWING */
+
   -2,  /* (328) frame_bound ::= expr PRECEDING|FOLLOWING */
+
   -2,  /* (329) frame_bound ::= CURRENT ROW */
+
    0,  /* (330) frame_exclude_opt ::= */
+
   -2,  /* (331) frame_exclude_opt ::= EXCLUDE frame_exclude */
+
   -2,  /* (332) frame_exclude ::= NO OTHERS */
+
   -2,  /* (333) frame_exclude ::= CURRENT ROW */
+
   -1,  /* (334) frame_exclude ::= GROUP|TIES */
+
   -2,  /* (335) window_clause ::= WINDOW windowdefn_list */
+
   -2,  /* (336) filter_over ::= filter_clause over_clause */
+
   -1,  /* (337) filter_over ::= over_clause */
+
   -1,  /* (338) filter_over ::= filter_clause */
+
   -4,  /* (339) over_clause ::= OVER LP window RP */
+
   -2,  /* (340) over_clause ::= OVER nm */
+
   -5,  /* (341) filter_clause ::= FILTER LP WHERE expr RP */
+
   -1,  /* (342) input ::= cmdlist */
+
   -2,  /* (343) cmdlist ::= cmdlist ecmd */
+
   -1,  /* (344) cmdlist ::= ecmd */
+
   -1,  /* (345) ecmd ::= SEMI */
+
   -2,  /* (346) ecmd ::= cmdx SEMI */
+
   -3,  /* (347) ecmd ::= explain cmdx SEMI */
+
    0,  /* (348) trans_opt ::= */
+
   -1,  /* (349) trans_opt ::= TRANSACTION */
+
   -2,  /* (350) trans_opt ::= TRANSACTION nm */
+
   -1,  /* (351) savepoint_opt ::= SAVEPOINT */
+
    0,  /* (352) savepoint_opt ::= */
+
   -2,  /* (353) cmd ::= create_table create_table_args */
+
   -1,  /* (354) table_option_set ::= table_option */
+
   -4,  /* (355) columnlist ::= columnlist COMMA columnname carglist */
+
   -2,  /* (356) columnlist ::= columnname carglist */
+
   -1,  /* (357) nm ::= ID|INDEXED */
+
   -1,  /* (358) nm ::= STRING */
+
   -1,  /* (359) nm ::= JOIN_KW */
+
   -1,  /* (360) typetoken ::= typename */
+
   -1,  /* (361) typename ::= ID|STRING */
+
   -1,  /* (362) signed ::= plus_num */
+
   -1,  /* (363) signed ::= minus_num */
+
   -2,  /* (364) carglist ::= carglist ccons */
+
    0,  /* (365) carglist ::= */
+
   -2,  /* (366) ccons ::= NULL onconf */
+
   -4,  /* (367) ccons ::= GENERATED ALWAYS AS generated */
+
   -2,  /* (368) ccons ::= AS generated */
+
   -2,  /* (369) conslist_opt ::= COMMA conslist */
+
   -3,  /* (370) conslist ::= conslist tconscomma tcons */
+
   -1,  /* (371) conslist ::= tcons */
+
    0,  /* (372) tconscomma ::= */
+
   -1,  /* (373) defer_subclause_opt ::= defer_subclause */
+
   -1,  /* (374) resolvetype ::= raisetype */
+
   -1,  /* (375) selectnowith ::= oneselect */
+
   -1,  /* (376) oneselect ::= values */
+
   -2,  /* (377) sclp ::= selcollist COMMA */
+
   -1,  /* (378) as ::= ID|STRING */
+
   -1,  /* (379) indexed_opt ::= indexed_by */
+
    0,  /* (380) returning ::= */
+
   -1,  /* (381) expr ::= term */
+
   -1,  /* (382) likeop ::= LIKE_KW|MATCH */
+
   -1,  /* (383) exprlist ::= nexprlist */
+
   -1,  /* (384) nmnum ::= plus_num */
+
   -1,  /* (385) nmnum ::= nm */
+
   -1,  /* (386) nmnum ::= ON */
+
   -1,  /* (387) nmnum ::= DELETE */
+
   -1,  /* (388) nmnum ::= DEFAULT */
+
   -1,  /* (389) plus_num ::= INTEGER|FLOAT */
+
    0,  /* (390) foreach_clause ::= */
+
   -3,  /* (391) foreach_clause ::= FOR EACH ROW */
+
   -1,  /* (392) trnm ::= nm */
+
    0,  /* (393) tridxby ::= */
+
   -1,  /* (394) database_kw_opt ::= DATABASE */
+
    0,  /* (395) database_kw_opt ::= */
+
    0,  /* (396) kwcolumn_opt ::= */
+
   -1,  /* (397) kwcolumn_opt ::= COLUMNKW */
+
   -1,  /* (398) vtabarglist ::= vtabarg */
+
   -3,  /* (399) vtabarglist ::= vtabarglist COMMA vtabarg */
+
   -2,  /* (400) vtabarg ::= vtabarg vtabargtoken */
+
    0,  /* (401) anylist ::= */
+
   -4,  /* (402) anylist ::= anylist LP anylist RP */
+
   -2,  /* (403) anylist ::= anylist ANY */
+
    0,  /* (404) with ::= */
};

static void yy_accept(yyParser*);  /* Forward Declaration */
@@ -164992,7 +168841,7 @@ static YYACTIONTYPE yy_reduce(
      case 5: /* transtype ::= DEFERRED */
      case 6: /* transtype ::= IMMEDIATE */ yytestcase(yyruleno==6);
      case 7: /* transtype ::= EXCLUSIVE */ yytestcase(yyruleno==7);
-
      case 321: /* range_or_rows ::= RANGE|ROWS|GROUPS */ yytestcase(yyruleno==321);
+
      case 323: /* range_or_rows ::= RANGE|ROWS|GROUPS */ yytestcase(yyruleno==323);
{yymsp[0].minor.yy394 = yymsp[0].major; /*A-overwrites-X*/}
        break;
      case 8: /* cmd ::= COMMIT|END trans_opt */
@@ -165029,7 +168878,7 @@ static YYACTIONTYPE yy_reduce(
      case 72: /* defer_subclause_opt ::= */ yytestcase(yyruleno==72);
      case 81: /* ifexists ::= */ yytestcase(yyruleno==81);
      case 98: /* distinct ::= */ yytestcase(yyruleno==98);
-
      case 242: /* collate ::= */ yytestcase(yyruleno==242);
+
      case 244: /* collate ::= */ yytestcase(yyruleno==244);
{yymsp[1].minor.yy394 = 0;}
        break;
      case 16: /* ifnotexists ::= IF NOT EXISTS */
@@ -165213,9 +169062,9 @@ static YYACTIONTYPE yy_reduce(
        break;
      case 63: /* init_deferred_pred_opt ::= INITIALLY DEFERRED */
      case 80: /* ifexists ::= IF EXISTS */ yytestcase(yyruleno==80);
-
      case 214: /* between_op ::= NOT BETWEEN */ yytestcase(yyruleno==214);
-
      case 217: /* in_op ::= NOT IN */ yytestcase(yyruleno==217);
-
      case 243: /* collate ::= COLLATE ID|STRING */ yytestcase(yyruleno==243);
+
      case 216: /* between_op ::= NOT BETWEEN */ yytestcase(yyruleno==216);
+
      case 219: /* in_op ::= NOT IN */ yytestcase(yyruleno==219);
+
      case 245: /* collate ::= COLLATE ID|STRING */ yytestcase(yyruleno==245);
{yymsp[-1].minor.yy394 = 1;}
        break;
      case 64: /* init_deferred_pred_opt ::= INITIALLY IMMEDIATE */
@@ -165300,7 +169149,7 @@ static YYACTIONTYPE yy_reduce(
    Token x;
    x.n = 0;
    parserDoubleLinkSelect(pParse, pRhs);
-
    pFrom = sqlite3SrcListAppendFromTerm(pParse,0,0,0,&x,pRhs,0,0);
+
    pFrom = sqlite3SrcListAppendFromTerm(pParse,0,0,0,&x,pRhs,0);
    pRhs = sqlite3SelectNew(pParse,0,pFrom,0,0,0,0,0,0);
  }
  if( pRhs ){
@@ -165365,9 +169214,9 @@ static YYACTIONTYPE yy_reduce(
      case 99: /* sclp ::= */
      case 132: /* orderby_opt ::= */ yytestcase(yyruleno==132);
      case 142: /* groupby_opt ::= */ yytestcase(yyruleno==142);
-
      case 230: /* exprlist ::= */ yytestcase(yyruleno==230);
-
      case 233: /* paren_exprlist ::= */ yytestcase(yyruleno==233);
-
      case 238: /* eidlist_opt ::= */ yytestcase(yyruleno==238);
+
      case 232: /* exprlist ::= */ yytestcase(yyruleno==232);
+
      case 235: /* paren_exprlist ::= */ yytestcase(yyruleno==235);
+
      case 240: /* eidlist_opt ::= */ yytestcase(yyruleno==240);
{yymsp[1].minor.yy322 = 0;}
        break;
      case 100: /* selcollist ::= sclp scanpt expr scanpt as */
@@ -165392,9 +169241,9 @@ static YYACTIONTYPE yy_reduce(
}
        break;
      case 103: /* as ::= AS nm */
-
      case 114: /* dbnm ::= DOT nm */ yytestcase(yyruleno==114);
-
      case 254: /* plus_num ::= PLUS INTEGER|FLOAT */ yytestcase(yyruleno==254);
-
      case 255: /* minus_num ::= MINUS INTEGER|FLOAT */ yytestcase(yyruleno==255);
+
      case 115: /* dbnm ::= DOT nm */ yytestcase(yyruleno==115);
+
      case 256: /* plus_num ::= PLUS INTEGER|FLOAT */ yytestcase(yyruleno==256);
+
      case 257: /* minus_num ::= MINUS INTEGER|FLOAT */ yytestcase(yyruleno==257);
{yymsp[-1].minor.yy0 = yymsp[0].minor.yy0;}
        break;
      case 105: /* from ::= */
@@ -165404,7 +169253,7 @@ static YYACTIONTYPE yy_reduce(
      case 106: /* from ::= FROM seltablist */
{
  yymsp[-1].minor.yy131 = yymsp[0].minor.yy131;
-
  sqlite3SrcListShiftJoinType(yymsp[-1].minor.yy131);
+
  sqlite3SrcListShiftJoinType(pParse,yymsp[-1].minor.yy131);
}
        break;
      case 107: /* stl_prefix ::= seltablist joinop */
@@ -165412,35 +169261,43 @@ static YYACTIONTYPE yy_reduce(
   if( ALWAYS(yymsp[-1].minor.yy131 && yymsp[-1].minor.yy131->nSrc>0) ) yymsp[-1].minor.yy131->a[yymsp[-1].minor.yy131->nSrc-1].fg.jointype = (u8)yymsp[0].minor.yy394;
}
        break;
-
      case 109: /* seltablist ::= stl_prefix nm dbnm as indexed_opt on_opt using_opt */
+
      case 109: /* seltablist ::= stl_prefix nm dbnm as on_using */
+
{
+
  yymsp[-4].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-4].minor.yy131,&yymsp[-3].minor.yy0,&yymsp[-2].minor.yy0,&yymsp[-1].minor.yy0,0,&yymsp[0].minor.yy561);
+
}
+
        break;
+
      case 110: /* seltablist ::= stl_prefix nm dbnm as indexed_by on_using */
{
-
  yymsp[-6].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-6].minor.yy131,&yymsp[-5].minor.yy0,&yymsp[-4].minor.yy0,&yymsp[-3].minor.yy0,0,yymsp[-1].minor.yy528,yymsp[0].minor.yy254);
-
  sqlite3SrcListIndexedBy(pParse, yymsp[-6].minor.yy131, &yymsp[-2].minor.yy0);
+
  yymsp[-5].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-5].minor.yy131,&yymsp[-4].minor.yy0,&yymsp[-3].minor.yy0,&yymsp[-2].minor.yy0,0,&yymsp[0].minor.yy561);
+
  sqlite3SrcListIndexedBy(pParse, yymsp[-5].minor.yy131, &yymsp[-1].minor.yy0);
}
        break;
-
      case 110: /* seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_opt using_opt */
+
      case 111: /* seltablist ::= stl_prefix nm dbnm LP exprlist RP as on_using */
{
-
  yymsp[-8].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-8].minor.yy131,&yymsp[-7].minor.yy0,&yymsp[-6].minor.yy0,&yymsp[-2].minor.yy0,0,yymsp[-1].minor.yy528,yymsp[0].minor.yy254);
-
  sqlite3SrcListFuncArgs(pParse, yymsp[-8].minor.yy131, yymsp[-4].minor.yy322);
+
  yymsp[-7].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-7].minor.yy131,&yymsp[-6].minor.yy0,&yymsp[-5].minor.yy0,&yymsp[-1].minor.yy0,0,&yymsp[0].minor.yy561);
+
  sqlite3SrcListFuncArgs(pParse, yymsp[-7].minor.yy131, yymsp[-3].minor.yy322);
}
        break;
-
      case 111: /* seltablist ::= stl_prefix LP select RP as on_opt using_opt */
+
      case 112: /* seltablist ::= stl_prefix LP select RP as on_using */
{
-
    yymsp[-6].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-6].minor.yy131,0,0,&yymsp[-2].minor.yy0,yymsp[-4].minor.yy47,yymsp[-1].minor.yy528,yymsp[0].minor.yy254);
+
    yymsp[-5].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-5].minor.yy131,0,0,&yymsp[-1].minor.yy0,yymsp[-3].minor.yy47,&yymsp[0].minor.yy561);
  }
        break;
-
      case 112: /* seltablist ::= stl_prefix LP seltablist RP as on_opt using_opt */
+
      case 113: /* seltablist ::= stl_prefix LP seltablist RP as on_using */
{
-
    if( yymsp[-6].minor.yy131==0 && yymsp[-2].minor.yy0.n==0 && yymsp[-1].minor.yy528==0 && yymsp[0].minor.yy254==0 ){
-
      yymsp[-6].minor.yy131 = yymsp[-4].minor.yy131;
-
    }else if( yymsp[-4].minor.yy131->nSrc==1 ){
-
      yymsp[-6].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-6].minor.yy131,0,0,&yymsp[-2].minor.yy0,0,yymsp[-1].minor.yy528,yymsp[0].minor.yy254);
-
      if( yymsp[-6].minor.yy131 ){
-
        SrcItem *pNew = &yymsp[-6].minor.yy131->a[yymsp[-6].minor.yy131->nSrc-1];
-
        SrcItem *pOld = yymsp[-4].minor.yy131->a;
+
    if( yymsp[-5].minor.yy131==0 && yymsp[-1].minor.yy0.n==0 && yymsp[0].minor.yy561.pOn==0 && yymsp[0].minor.yy561.pUsing==0 ){
+
      yymsp[-5].minor.yy131 = yymsp[-3].minor.yy131;
+
    }else if( yymsp[-3].minor.yy131->nSrc==1 ){
+
      yymsp[-5].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-5].minor.yy131,0,0,&yymsp[-1].minor.yy0,0,&yymsp[0].minor.yy561);
+
      if( yymsp[-5].minor.yy131 ){
+
        SrcItem *pNew = &yymsp[-5].minor.yy131->a[yymsp[-5].minor.yy131->nSrc-1];
+
        SrcItem *pOld = yymsp[-3].minor.yy131->a;
        pNew->zName = pOld->zName;
        pNew->zDatabase = pOld->zDatabase;
        pNew->pSelect = pOld->pSelect;
+
        if( pNew->pSelect && (pNew->pSelect->selFlags & SF_NestedFrom)!=0 ){
+
          pNew->fg.isNestedFrom = 1;
+
        }
        if( pOld->fg.isTabFunc ){
          pNew->u1.pFuncArg = pOld->u1.pFuncArg;
          pOld->u1.pFuncArg = 0;
@@ -165450,94 +169307,78 @@ static YYACTIONTYPE yy_reduce(
        pOld->zName = pOld->zDatabase = 0;
        pOld->pSelect = 0;
      }
-
      sqlite3SrcListDelete(pParse->db, yymsp[-4].minor.yy131);
+
      sqlite3SrcListDelete(pParse->db, yymsp[-3].minor.yy131);
    }else{
      Select *pSubquery;
-
      sqlite3SrcListShiftJoinType(yymsp[-4].minor.yy131);
-
      pSubquery = sqlite3SelectNew(pParse,0,yymsp[-4].minor.yy131,0,0,0,0,SF_NestedFrom,0);
-
      yymsp[-6].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-6].minor.yy131,0,0,&yymsp[-2].minor.yy0,pSubquery,yymsp[-1].minor.yy528,yymsp[0].minor.yy254);
+
      sqlite3SrcListShiftJoinType(pParse,yymsp[-3].minor.yy131);
+
      pSubquery = sqlite3SelectNew(pParse,0,yymsp[-3].minor.yy131,0,0,0,0,SF_NestedFrom,0);
+
      yymsp[-5].minor.yy131 = sqlite3SrcListAppendFromTerm(pParse,yymsp[-5].minor.yy131,0,0,&yymsp[-1].minor.yy0,pSubquery,&yymsp[0].minor.yy561);
    }
  }
        break;
-
      case 113: /* dbnm ::= */
-
      case 127: /* indexed_opt ::= */ yytestcase(yyruleno==127);
+
      case 114: /* dbnm ::= */
+
      case 129: /* indexed_opt ::= */ yytestcase(yyruleno==129);
{yymsp[1].minor.yy0.z=0; yymsp[1].minor.yy0.n=0;}
        break;
-
      case 115: /* fullname ::= nm */
+
      case 116: /* fullname ::= nm */
{
  yylhsminor.yy131 = sqlite3SrcListAppend(pParse,0,&yymsp[0].minor.yy0,0);
  if( IN_RENAME_OBJECT && yylhsminor.yy131 ) sqlite3RenameTokenMap(pParse, yylhsminor.yy131->a[0].zName, &yymsp[0].minor.yy0);
}
  yymsp[0].minor.yy131 = yylhsminor.yy131;
        break;
-
      case 116: /* fullname ::= nm DOT nm */
+
      case 117: /* fullname ::= nm DOT nm */
{
  yylhsminor.yy131 = sqlite3SrcListAppend(pParse,0,&yymsp[-2].minor.yy0,&yymsp[0].minor.yy0);
  if( IN_RENAME_OBJECT && yylhsminor.yy131 ) sqlite3RenameTokenMap(pParse, yylhsminor.yy131->a[0].zName, &yymsp[0].minor.yy0);
}
  yymsp[-2].minor.yy131 = yylhsminor.yy131;
        break;
-
      case 117: /* xfullname ::= nm */
+
      case 118: /* xfullname ::= nm */
{yymsp[0].minor.yy131 = sqlite3SrcListAppend(pParse,0,&yymsp[0].minor.yy0,0); /*A-overwrites-X*/}
        break;
-
      case 118: /* xfullname ::= nm DOT nm */
+
      case 119: /* xfullname ::= nm DOT nm */
{yymsp[-2].minor.yy131 = sqlite3SrcListAppend(pParse,0,&yymsp[-2].minor.yy0,&yymsp[0].minor.yy0); /*A-overwrites-X*/}
        break;
-
      case 119: /* xfullname ::= nm DOT nm AS nm */
+
      case 120: /* xfullname ::= nm DOT nm AS nm */
{
   yymsp[-4].minor.yy131 = sqlite3SrcListAppend(pParse,0,&yymsp[-4].minor.yy0,&yymsp[-2].minor.yy0); /*A-overwrites-X*/
   if( yymsp[-4].minor.yy131 ) yymsp[-4].minor.yy131->a[0].zAlias = sqlite3NameFromToken(pParse->db, &yymsp[0].minor.yy0);
}
        break;
-
      case 120: /* xfullname ::= nm AS nm */
+
      case 121: /* xfullname ::= nm AS nm */
{
   yymsp[-2].minor.yy131 = sqlite3SrcListAppend(pParse,0,&yymsp[-2].minor.yy0,0); /*A-overwrites-X*/
   if( yymsp[-2].minor.yy131 ) yymsp[-2].minor.yy131->a[0].zAlias = sqlite3NameFromToken(pParse->db, &yymsp[0].minor.yy0);
}
        break;
-
      case 121: /* joinop ::= COMMA|JOIN */
+
      case 122: /* joinop ::= COMMA|JOIN */
{ yymsp[0].minor.yy394 = JT_INNER; }
        break;
-
      case 122: /* joinop ::= JOIN_KW JOIN */
+
      case 123: /* joinop ::= JOIN_KW JOIN */
{yymsp[-1].minor.yy394 = sqlite3JoinType(pParse,&yymsp[-1].minor.yy0,0,0);  /*X-overwrites-A*/}
        break;
-
      case 123: /* joinop ::= JOIN_KW nm JOIN */
+
      case 124: /* joinop ::= JOIN_KW nm JOIN */
{yymsp[-2].minor.yy394 = sqlite3JoinType(pParse,&yymsp[-2].minor.yy0,&yymsp[-1].minor.yy0,0); /*X-overwrites-A*/}
        break;
-
      case 124: /* joinop ::= JOIN_KW nm nm JOIN */
+
      case 125: /* joinop ::= JOIN_KW nm nm JOIN */
{yymsp[-3].minor.yy394 = sqlite3JoinType(pParse,&yymsp[-3].minor.yy0,&yymsp[-2].minor.yy0,&yymsp[-1].minor.yy0);/*X-overwrites-A*/}
        break;
-
      case 125: /* on_opt ::= ON expr */
-
      case 145: /* having_opt ::= HAVING expr */ yytestcase(yyruleno==145);
-
      case 152: /* where_opt ::= WHERE expr */ yytestcase(yyruleno==152);
-
      case 154: /* where_opt_ret ::= WHERE expr */ yytestcase(yyruleno==154);
-
      case 226: /* case_else ::= ELSE expr */ yytestcase(yyruleno==226);
-
      case 247: /* vinto ::= INTO expr */ yytestcase(yyruleno==247);
-
{yymsp[-1].minor.yy528 = yymsp[0].minor.yy528;}
+
      case 126: /* on_using ::= ON expr */
+
{yymsp[-1].minor.yy561.pOn = yymsp[0].minor.yy528; yymsp[-1].minor.yy561.pUsing = 0;}
        break;
-
      case 126: /* on_opt ::= */
-
      case 144: /* having_opt ::= */ yytestcase(yyruleno==144);
-
      case 146: /* limit_opt ::= */ yytestcase(yyruleno==146);
-
      case 151: /* where_opt ::= */ yytestcase(yyruleno==151);
-
      case 153: /* where_opt_ret ::= */ yytestcase(yyruleno==153);
-
      case 227: /* case_else ::= */ yytestcase(yyruleno==227);
-
      case 229: /* case_operand ::= */ yytestcase(yyruleno==229);
-
      case 248: /* vinto ::= */ yytestcase(yyruleno==248);
-
{yymsp[1].minor.yy528 = 0;}
+
      case 127: /* on_using ::= USING LP idlist RP */
+
{yymsp[-3].minor.yy561.pOn = 0; yymsp[-3].minor.yy561.pUsing = yymsp[-1].minor.yy254;}
        break;
-
      case 128: /* indexed_opt ::= INDEXED BY nm */
+
      case 128: /* on_using ::= */
+
{yymsp[1].minor.yy561.pOn = 0; yymsp[1].minor.yy561.pUsing = 0;}
+
        break;
+
      case 130: /* indexed_by ::= INDEXED BY nm */
{yymsp[-2].minor.yy0 = yymsp[0].minor.yy0;}
        break;
-
      case 129: /* indexed_opt ::= NOT INDEXED */
+
      case 131: /* indexed_by ::= NOT INDEXED */
{yymsp[-1].minor.yy0.z=0; yymsp[-1].minor.yy0.n=1;}
        break;
-
      case 130: /* using_opt ::= USING LP idlist RP */
-
{yymsp[-3].minor.yy254 = yymsp[-1].minor.yy254;}
-
        break;
-
      case 131: /* using_opt ::= */
-
      case 173: /* idlist_opt ::= */ yytestcase(yyruleno==173);
-
{yymsp[1].minor.yy254 = 0;}
-
        break;
      case 133: /* orderby_opt ::= ORDER BY sortlist */
      case 143: /* groupby_opt ::= GROUP BY nexprlist */ yytestcase(yyruleno==143);
{yymsp[-2].minor.yy322 = yymsp[0].minor.yy322;}
@@ -165570,6 +169411,22 @@ static YYACTIONTYPE yy_reduce(
      case 140: /* nulls ::= NULLS LAST */
{yymsp[-1].minor.yy394 = SQLITE_SO_DESC;}
        break;
+
      case 144: /* having_opt ::= */
+
      case 146: /* limit_opt ::= */ yytestcase(yyruleno==146);
+
      case 151: /* where_opt ::= */ yytestcase(yyruleno==151);
+
      case 153: /* where_opt_ret ::= */ yytestcase(yyruleno==153);
+
      case 229: /* case_else ::= */ yytestcase(yyruleno==229);
+
      case 231: /* case_operand ::= */ yytestcase(yyruleno==231);
+
      case 250: /* vinto ::= */ yytestcase(yyruleno==250);
+
{yymsp[1].minor.yy528 = 0;}
+
        break;
+
      case 145: /* having_opt ::= HAVING expr */
+
      case 152: /* where_opt ::= WHERE expr */ yytestcase(yyruleno==152);
+
      case 154: /* where_opt_ret ::= WHERE expr */ yytestcase(yyruleno==154);
+
      case 228: /* case_else ::= ELSE expr */ yytestcase(yyruleno==228);
+
      case 249: /* vinto ::= INTO expr */ yytestcase(yyruleno==249);
+
{yymsp[-1].minor.yy528 = yymsp[0].minor.yy528;}
+
        break;
      case 147: /* limit_opt ::= LIMIT expr */
{yymsp[-1].minor.yy528 = sqlite3PExpr(pParse,TK_LIMIT,yymsp[0].minor.yy528,0);}
        break;
@@ -165595,7 +169452,18 @@ static YYACTIONTYPE yy_reduce(
{
  sqlite3SrcListIndexedBy(pParse, yymsp[-5].minor.yy131, &yymsp[-4].minor.yy0);
  sqlite3ExprListCheckLength(pParse,yymsp[-2].minor.yy322,"set list");
-
  yymsp[-5].minor.yy131 = sqlite3SrcListAppendList(pParse, yymsp[-5].minor.yy131, yymsp[-1].minor.yy131);
+
  if( yymsp[-1].minor.yy131 ){
+
    SrcList *pFromClause = yymsp[-1].minor.yy131;
+
    if( pFromClause->nSrc>1 ){
+
      Select *pSubquery;
+
      Token as;
+
      pSubquery = sqlite3SelectNew(pParse,0,pFromClause,0,0,0,0,SF_NestedFrom,0);
+
      as.n = 0;
+
      as.z = 0;
+
      pFromClause = sqlite3SrcListAppendFromTerm(pParse,0,0,0,&as,pSubquery,0);
+
    }
+
    yymsp[-5].minor.yy131 = sqlite3SrcListAppendList(pParse, yymsp[-5].minor.yy131, pFromClause);
+
  }
  sqlite3Update(pParse,yymsp[-5].minor.yy131,yymsp[-2].minor.yy322,yymsp[0].minor.yy528,yymsp[-6].minor.yy394,0,0,0);
}
        break;
@@ -165653,6 +169521,9 @@ static YYACTIONTYPE yy_reduce(
      case 170: /* returning ::= RETURNING selcollist */
{sqlite3AddReturning(pParse,yymsp[0].minor.yy322);}
        break;
+
      case 173: /* idlist_opt ::= */
+
{yymsp[1].minor.yy254 = 0;}
+
        break;
      case 174: /* idlist_opt ::= LP idlist RP */
{yymsp[-2].minor.yy254 = yymsp[-1].minor.yy254;}
        break;
@@ -165838,17 +169709,29 @@ static YYACTIONTYPE yy_reduce(
  binaryToUnaryIfNull(pParse, yymsp[0].minor.yy528, yymsp[-3].minor.yy528, TK_NOTNULL);
}
        break;
-
      case 209: /* expr ::= NOT expr */
-
      case 210: /* expr ::= BITNOT expr */ yytestcase(yyruleno==210);
+
      case 209: /* expr ::= expr IS NOT DISTINCT FROM expr */
+
{
+
  yymsp[-5].minor.yy528 = sqlite3PExpr(pParse,TK_IS,yymsp[-5].minor.yy528,yymsp[0].minor.yy528);
+
  binaryToUnaryIfNull(pParse, yymsp[0].minor.yy528, yymsp[-5].minor.yy528, TK_ISNULL);
+
}
+
        break;
+
      case 210: /* expr ::= expr IS DISTINCT FROM expr */
+
{
+
  yymsp[-4].minor.yy528 = sqlite3PExpr(pParse,TK_ISNOT,yymsp[-4].minor.yy528,yymsp[0].minor.yy528);
+
  binaryToUnaryIfNull(pParse, yymsp[0].minor.yy528, yymsp[-4].minor.yy528, TK_NOTNULL);
+
}
+
        break;
+
      case 211: /* expr ::= NOT expr */
+
      case 212: /* expr ::= BITNOT expr */ yytestcase(yyruleno==212);
{yymsp[-1].minor.yy528 = sqlite3PExpr(pParse, yymsp[-1].major, yymsp[0].minor.yy528, 0);/*A-overwrites-B*/}
        break;
-
      case 211: /* expr ::= PLUS|MINUS expr */
+
      case 213: /* expr ::= PLUS|MINUS expr */
{
  yymsp[-1].minor.yy528 = sqlite3PExpr(pParse, yymsp[-1].major==TK_PLUS ? TK_UPLUS : TK_UMINUS, yymsp[0].minor.yy528, 0);
  /*A-overwrites-B*/
}
        break;
-
      case 212: /* expr ::= expr PTR expr */
+
      case 214: /* expr ::= expr PTR expr */
{
  ExprList *pList = sqlite3ExprListAppend(pParse, 0, yymsp[-2].minor.yy528);
  pList = sqlite3ExprListAppend(pParse, pList, yymsp[0].minor.yy528);
@@ -165856,11 +169739,11 @@ static YYACTIONTYPE yy_reduce(
}
  yymsp[-2].minor.yy528 = yylhsminor.yy528;
        break;
-
      case 213: /* between_op ::= BETWEEN */
-
      case 216: /* in_op ::= IN */ yytestcase(yyruleno==216);
+
      case 215: /* between_op ::= BETWEEN */
+
      case 218: /* in_op ::= IN */ yytestcase(yyruleno==218);
{yymsp[0].minor.yy394 = 0;}
        break;
-
      case 215: /* expr ::= expr between_op expr AND expr */
+
      case 217: /* expr ::= expr between_op expr AND expr */
{
  ExprList *pList = sqlite3ExprListAppend(pParse,0, yymsp[-2].minor.yy528);
  pList = sqlite3ExprListAppend(pParse,pList, yymsp[0].minor.yy528);
@@ -165873,7 +169756,7 @@ static YYACTIONTYPE yy_reduce(
  if( yymsp[-3].minor.yy394 ) yymsp[-4].minor.yy528 = sqlite3PExpr(pParse, TK_NOT, yymsp[-4].minor.yy528, 0);
}
        break;
-
      case 218: /* expr ::= expr in_op LP exprlist RP */
+
      case 220: /* expr ::= expr in_op LP exprlist RP */
{
    if( yymsp[-1].minor.yy322==0 ){
      /* Expressions of the form
@@ -165885,7 +169768,8 @@ static YYACTIONTYPE yy_reduce(
      ** regardless of the value of expr1.
      */
      sqlite3ExprUnmapAndDelete(pParse, yymsp[-4].minor.yy528);
-
      yymsp[-4].minor.yy528 = sqlite3Expr(pParse->db, TK_INTEGER, yymsp[-3].minor.yy394 ? "1" : "0");
+
      yymsp[-4].minor.yy528 = sqlite3Expr(pParse->db, TK_STRING, yymsp[-3].minor.yy394 ? "true" : "false");
+
      if( yymsp[-4].minor.yy528 ) sqlite3ExprIdToTrueFalse(yymsp[-4].minor.yy528);
    }else{
      Expr *pRHS = yymsp[-1].minor.yy322->a[0].pExpr;
      if( yymsp[-1].minor.yy322->nExpr==1 && sqlite3ExprIsConstant(pRHS) && yymsp[-4].minor.yy528->op!=TK_VECTOR ){
@@ -165913,20 +169797,20 @@ static YYACTIONTYPE yy_reduce(
    }
  }
        break;
-
      case 219: /* expr ::= LP select RP */
+
      case 221: /* expr ::= LP select RP */
{
    yymsp[-2].minor.yy528 = sqlite3PExpr(pParse, TK_SELECT, 0, 0);
    sqlite3PExprAddSelect(pParse, yymsp[-2].minor.yy528, yymsp[-1].minor.yy47);
  }
        break;
-
      case 220: /* expr ::= expr in_op LP select RP */
+
      case 222: /* expr ::= expr in_op LP select RP */
{
    yymsp[-4].minor.yy528 = sqlite3PExpr(pParse, TK_IN, yymsp[-4].minor.yy528, 0);
    sqlite3PExprAddSelect(pParse, yymsp[-4].minor.yy528, yymsp[-1].minor.yy47);
    if( yymsp[-3].minor.yy394 ) yymsp[-4].minor.yy528 = sqlite3PExpr(pParse, TK_NOT, yymsp[-4].minor.yy528, 0);
  }
        break;
-
      case 221: /* expr ::= expr in_op nm dbnm paren_exprlist */
+
      case 223: /* expr ::= expr in_op nm dbnm paren_exprlist */
{
    SrcList *pSrc = sqlite3SrcListAppend(pParse, 0,&yymsp[-2].minor.yy0,&yymsp[-1].minor.yy0);
    Select *pSelect = sqlite3SelectNew(pParse, 0,pSrc,0,0,0,0,0,0);
@@ -165936,14 +169820,14 @@ static YYACTIONTYPE yy_reduce(
    if( yymsp[-3].minor.yy394 ) yymsp[-4].minor.yy528 = sqlite3PExpr(pParse, TK_NOT, yymsp[-4].minor.yy528, 0);
  }
        break;
-
      case 222: /* expr ::= EXISTS LP select RP */
+
      case 224: /* expr ::= EXISTS LP select RP */
{
    Expr *p;
    p = yymsp[-3].minor.yy528 = sqlite3PExpr(pParse, TK_EXISTS, 0, 0);
    sqlite3PExprAddSelect(pParse, p, yymsp[-1].minor.yy47);
  }
        break;
-
      case 223: /* expr ::= CASE case_operand case_exprlist case_else END */
+
      case 225: /* expr ::= CASE case_operand case_exprlist case_else END */
{
  yymsp[-4].minor.yy528 = sqlite3PExpr(pParse, TK_CASE, yymsp[-3].minor.yy528, 0);
  if( yymsp[-4].minor.yy528 ){
@@ -165955,32 +169839,32 @@ static YYACTIONTYPE yy_reduce(
  }
}
        break;
-
      case 224: /* case_exprlist ::= case_exprlist WHEN expr THEN expr */
+
      case 226: /* case_exprlist ::= case_exprlist WHEN expr THEN expr */
{
  yymsp[-4].minor.yy322 = sqlite3ExprListAppend(pParse,yymsp[-4].minor.yy322, yymsp[-2].minor.yy528);
  yymsp[-4].minor.yy322 = sqlite3ExprListAppend(pParse,yymsp[-4].minor.yy322, yymsp[0].minor.yy528);
}
        break;
-
      case 225: /* case_exprlist ::= WHEN expr THEN expr */
+
      case 227: /* case_exprlist ::= WHEN expr THEN expr */
{
  yymsp[-3].minor.yy322 = sqlite3ExprListAppend(pParse,0, yymsp[-2].minor.yy528);
  yymsp[-3].minor.yy322 = sqlite3ExprListAppend(pParse,yymsp[-3].minor.yy322, yymsp[0].minor.yy528);
}
        break;
-
      case 228: /* case_operand ::= expr */
+
      case 230: /* case_operand ::= expr */
{yymsp[0].minor.yy528 = yymsp[0].minor.yy528; /*A-overwrites-X*/}
        break;
-
      case 231: /* nexprlist ::= nexprlist COMMA expr */
+
      case 233: /* nexprlist ::= nexprlist COMMA expr */
{yymsp[-2].minor.yy322 = sqlite3ExprListAppend(pParse,yymsp[-2].minor.yy322,yymsp[0].minor.yy528);}
        break;
-
      case 232: /* nexprlist ::= expr */
+
      case 234: /* nexprlist ::= expr */
{yymsp[0].minor.yy322 = sqlite3ExprListAppend(pParse,0,yymsp[0].minor.yy528); /*A-overwrites-Y*/}
        break;
-
      case 234: /* paren_exprlist ::= LP exprlist RP */
-
      case 239: /* eidlist_opt ::= LP eidlist RP */ yytestcase(yyruleno==239);
+
      case 236: /* paren_exprlist ::= LP exprlist RP */
+
      case 241: /* eidlist_opt ::= LP eidlist RP */ yytestcase(yyruleno==241);
{yymsp[-2].minor.yy322 = yymsp[-1].minor.yy322;}
        break;
-
      case 235: /* cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt */
+
      case 237: /* cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP sortlist RP where_opt */
{
  sqlite3CreateIndex(pParse, &yymsp[-7].minor.yy0, &yymsp[-6].minor.yy0,
                     sqlite3SrcListAppend(pParse,0,&yymsp[-4].minor.yy0,0), yymsp[-2].minor.yy322, yymsp[-10].minor.yy394,
@@ -165990,48 +169874,48 @@ static YYACTIONTYPE yy_reduce(
  }
}
        break;
-
      case 236: /* uniqueflag ::= UNIQUE */
-
      case 278: /* raisetype ::= ABORT */ yytestcase(yyruleno==278);
+
      case 238: /* uniqueflag ::= UNIQUE */
+
      case 280: /* raisetype ::= ABORT */ yytestcase(yyruleno==280);
{yymsp[0].minor.yy394 = OE_Abort;}
        break;
-
      case 237: /* uniqueflag ::= */
+
      case 239: /* uniqueflag ::= */
{yymsp[1].minor.yy394 = OE_None;}
        break;
-
      case 240: /* eidlist ::= eidlist COMMA nm collate sortorder */
+
      case 242: /* eidlist ::= eidlist COMMA nm collate sortorder */
{
  yymsp[-4].minor.yy322 = parserAddExprIdListTerm(pParse, yymsp[-4].minor.yy322, &yymsp[-2].minor.yy0, yymsp[-1].minor.yy394, yymsp[0].minor.yy394);
}
        break;
-
      case 241: /* eidlist ::= nm collate sortorder */
+
      case 243: /* eidlist ::= nm collate sortorder */
{
  yymsp[-2].minor.yy322 = parserAddExprIdListTerm(pParse, 0, &yymsp[-2].minor.yy0, yymsp[-1].minor.yy394, yymsp[0].minor.yy394); /*A-overwrites-Y*/
}
        break;
-
      case 244: /* cmd ::= DROP INDEX ifexists fullname */
+
      case 246: /* cmd ::= DROP INDEX ifexists fullname */
{sqlite3DropIndex(pParse, yymsp[0].minor.yy131, yymsp[-1].minor.yy394);}
        break;
-
      case 245: /* cmd ::= VACUUM vinto */
+
      case 247: /* cmd ::= VACUUM vinto */
{sqlite3Vacuum(pParse,0,yymsp[0].minor.yy528);}
        break;
-
      case 246: /* cmd ::= VACUUM nm vinto */
+
      case 248: /* cmd ::= VACUUM nm vinto */
{sqlite3Vacuum(pParse,&yymsp[-1].minor.yy0,yymsp[0].minor.yy528);}
        break;
-
      case 249: /* cmd ::= PRAGMA nm dbnm */
+
      case 251: /* cmd ::= PRAGMA nm dbnm */
{sqlite3Pragma(pParse,&yymsp[-1].minor.yy0,&yymsp[0].minor.yy0,0,0);}
        break;
-
      case 250: /* cmd ::= PRAGMA nm dbnm EQ nmnum */
+
      case 252: /* cmd ::= PRAGMA nm dbnm EQ nmnum */
{sqlite3Pragma(pParse,&yymsp[-3].minor.yy0,&yymsp[-2].minor.yy0,&yymsp[0].minor.yy0,0);}
        break;
-
      case 251: /* cmd ::= PRAGMA nm dbnm LP nmnum RP */
+
      case 253: /* cmd ::= PRAGMA nm dbnm LP nmnum RP */
{sqlite3Pragma(pParse,&yymsp[-4].minor.yy0,&yymsp[-3].minor.yy0,&yymsp[-1].minor.yy0,0);}
        break;
-
      case 252: /* cmd ::= PRAGMA nm dbnm EQ minus_num */
+
      case 254: /* cmd ::= PRAGMA nm dbnm EQ minus_num */
{sqlite3Pragma(pParse,&yymsp[-3].minor.yy0,&yymsp[-2].minor.yy0,&yymsp[0].minor.yy0,1);}
        break;
-
      case 253: /* cmd ::= PRAGMA nm dbnm LP minus_num RP */
+
      case 255: /* cmd ::= PRAGMA nm dbnm LP minus_num RP */
{sqlite3Pragma(pParse,&yymsp[-4].minor.yy0,&yymsp[-3].minor.yy0,&yymsp[-1].minor.yy0,1);}
        break;
-
      case 256: /* cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END */
+
      case 258: /* cmd ::= createkw trigger_decl BEGIN trigger_cmd_list END */
{
  Token all;
  all.z = yymsp[-3].minor.yy0.z;
@@ -166039,50 +169923,50 @@ static YYACTIONTYPE yy_reduce(
  sqlite3FinishTrigger(pParse, yymsp[-1].minor.yy33, &all);
}
        break;
-
      case 257: /* trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause */
+
      case 259: /* trigger_decl ::= temp TRIGGER ifnotexists nm dbnm trigger_time trigger_event ON fullname foreach_clause when_clause */
{
  sqlite3BeginTrigger(pParse, &yymsp[-7].minor.yy0, &yymsp[-6].minor.yy0, yymsp[-5].minor.yy394, yymsp[-4].minor.yy180.a, yymsp[-4].minor.yy180.b, yymsp[-2].minor.yy131, yymsp[0].minor.yy528, yymsp[-10].minor.yy394, yymsp[-8].minor.yy394);
  yymsp[-10].minor.yy0 = (yymsp[-6].minor.yy0.n==0?yymsp[-7].minor.yy0:yymsp[-6].minor.yy0); /*A-overwrites-T*/
}
        break;
-
      case 258: /* trigger_time ::= BEFORE|AFTER */
+
      case 260: /* trigger_time ::= BEFORE|AFTER */
{ yymsp[0].minor.yy394 = yymsp[0].major; /*A-overwrites-X*/ }
        break;
-
      case 259: /* trigger_time ::= INSTEAD OF */
+
      case 261: /* trigger_time ::= INSTEAD OF */
{ yymsp[-1].minor.yy394 = TK_INSTEAD;}
        break;
-
      case 260: /* trigger_time ::= */
+
      case 262: /* trigger_time ::= */
{ yymsp[1].minor.yy394 = TK_BEFORE; }
        break;
-
      case 261: /* trigger_event ::= DELETE|INSERT */
-
      case 262: /* trigger_event ::= UPDATE */ yytestcase(yyruleno==262);
+
      case 263: /* trigger_event ::= DELETE|INSERT */
+
      case 264: /* trigger_event ::= UPDATE */ yytestcase(yyruleno==264);
{yymsp[0].minor.yy180.a = yymsp[0].major; /*A-overwrites-X*/ yymsp[0].minor.yy180.b = 0;}
        break;
-
      case 263: /* trigger_event ::= UPDATE OF idlist */
+
      case 265: /* trigger_event ::= UPDATE OF idlist */
{yymsp[-2].minor.yy180.a = TK_UPDATE; yymsp[-2].minor.yy180.b = yymsp[0].minor.yy254;}
        break;
-
      case 264: /* when_clause ::= */
-
      case 283: /* key_opt ::= */ yytestcase(yyruleno==283);
+
      case 266: /* when_clause ::= */
+
      case 285: /* key_opt ::= */ yytestcase(yyruleno==285);
{ yymsp[1].minor.yy528 = 0; }
        break;
-
      case 265: /* when_clause ::= WHEN expr */
-
      case 284: /* key_opt ::= KEY expr */ yytestcase(yyruleno==284);
+
      case 267: /* when_clause ::= WHEN expr */
+
      case 286: /* key_opt ::= KEY expr */ yytestcase(yyruleno==286);
{ yymsp[-1].minor.yy528 = yymsp[0].minor.yy528; }
        break;
-
      case 266: /* trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI */
+
      case 268: /* trigger_cmd_list ::= trigger_cmd_list trigger_cmd SEMI */
{
  assert( yymsp[-2].minor.yy33!=0 );
  yymsp[-2].minor.yy33->pLast->pNext = yymsp[-1].minor.yy33;
  yymsp[-2].minor.yy33->pLast = yymsp[-1].minor.yy33;
}
        break;
-
      case 267: /* trigger_cmd_list ::= trigger_cmd SEMI */
+
      case 269: /* trigger_cmd_list ::= trigger_cmd SEMI */
{
  assert( yymsp[-1].minor.yy33!=0 );
  yymsp[-1].minor.yy33->pLast = yymsp[-1].minor.yy33;
}
        break;
-
      case 268: /* trnm ::= nm DOT nm */
+
      case 270: /* trnm ::= nm DOT nm */
{
  yymsp[-2].minor.yy0 = yymsp[0].minor.yy0;
  sqlite3ErrorMsg(pParse,
@@ -166090,39 +169974,39 @@ static YYACTIONTYPE yy_reduce(
        "statements within triggers");
}
        break;
-
      case 269: /* tridxby ::= INDEXED BY nm */
+
      case 271: /* tridxby ::= INDEXED BY nm */
{
  sqlite3ErrorMsg(pParse,
        "the INDEXED BY clause is not allowed on UPDATE or DELETE statements "
        "within triggers");
}
        break;
-
      case 270: /* tridxby ::= NOT INDEXED */
+
      case 272: /* tridxby ::= NOT INDEXED */
{
  sqlite3ErrorMsg(pParse,
        "the NOT INDEXED clause is not allowed on UPDATE or DELETE statements "
        "within triggers");
}
        break;
-
      case 271: /* trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt */
+
      case 273: /* trigger_cmd ::= UPDATE orconf trnm tridxby SET setlist from where_opt scanpt */
{yylhsminor.yy33 = sqlite3TriggerUpdateStep(pParse, &yymsp[-6].minor.yy0, yymsp[-2].minor.yy131, yymsp[-3].minor.yy322, yymsp[-1].minor.yy528, yymsp[-7].minor.yy394, yymsp[-8].minor.yy0.z, yymsp[0].minor.yy522);}
  yymsp[-8].minor.yy33 = yylhsminor.yy33;
        break;
-
      case 272: /* trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt */
+
      case 274: /* trigger_cmd ::= scanpt insert_cmd INTO trnm idlist_opt select upsert scanpt */
{
   yylhsminor.yy33 = sqlite3TriggerInsertStep(pParse,&yymsp[-4].minor.yy0,yymsp[-3].minor.yy254,yymsp[-2].minor.yy47,yymsp[-6].minor.yy394,yymsp[-1].minor.yy444,yymsp[-7].minor.yy522,yymsp[0].minor.yy522);/*yylhsminor.yy33-overwrites-yymsp[-6].minor.yy394*/
}
  yymsp[-7].minor.yy33 = yylhsminor.yy33;
        break;
-
      case 273: /* trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt */
+
      case 275: /* trigger_cmd ::= DELETE FROM trnm tridxby where_opt scanpt */
{yylhsminor.yy33 = sqlite3TriggerDeleteStep(pParse, &yymsp[-3].minor.yy0, yymsp[-1].minor.yy528, yymsp[-5].minor.yy0.z, yymsp[0].minor.yy522);}
  yymsp[-5].minor.yy33 = yylhsminor.yy33;
        break;
-
      case 274: /* trigger_cmd ::= scanpt select scanpt */
+
      case 276: /* trigger_cmd ::= scanpt select scanpt */
{yylhsminor.yy33 = sqlite3TriggerSelectStep(pParse->db, yymsp[-1].minor.yy47, yymsp[-2].minor.yy522, yymsp[0].minor.yy522); /*yylhsminor.yy33-overwrites-yymsp[-1].minor.yy47*/}
  yymsp[-2].minor.yy33 = yylhsminor.yy33;
        break;
-
      case 275: /* expr ::= RAISE LP IGNORE RP */
+
      case 277: /* expr ::= RAISE LP IGNORE RP */
{
  yymsp[-3].minor.yy528 = sqlite3PExpr(pParse, TK_RAISE, 0, 0);
  if( yymsp[-3].minor.yy528 ){
@@ -166130,7 +170014,7 @@ static YYACTIONTYPE yy_reduce(
  }
}
        break;
-
      case 276: /* expr ::= RAISE LP raisetype COMMA nm RP */
+
      case 278: /* expr ::= RAISE LP raisetype COMMA nm RP */
{
  yymsp[-5].minor.yy528 = sqlite3ExprAlloc(pParse->db, TK_RAISE, &yymsp[-1].minor.yy0, 1);
  if( yymsp[-5].minor.yy528 ) {
@@ -166138,118 +170022,118 @@ static YYACTIONTYPE yy_reduce(
  }
}
        break;
-
      case 277: /* raisetype ::= ROLLBACK */
+
      case 279: /* raisetype ::= ROLLBACK */
{yymsp[0].minor.yy394 = OE_Rollback;}
        break;
-
      case 279: /* raisetype ::= FAIL */
+
      case 281: /* raisetype ::= FAIL */
{yymsp[0].minor.yy394 = OE_Fail;}
        break;
-
      case 280: /* cmd ::= DROP TRIGGER ifexists fullname */
+
      case 282: /* cmd ::= DROP TRIGGER ifexists fullname */
{
  sqlite3DropTrigger(pParse,yymsp[0].minor.yy131,yymsp[-1].minor.yy394);
}
        break;
-
      case 281: /* cmd ::= ATTACH database_kw_opt expr AS expr key_opt */
+
      case 283: /* cmd ::= ATTACH database_kw_opt expr AS expr key_opt */
{
  sqlite3Attach(pParse, yymsp[-3].minor.yy528, yymsp[-1].minor.yy528, yymsp[0].minor.yy528);
}
        break;
-
      case 282: /* cmd ::= DETACH database_kw_opt expr */
+
      case 284: /* cmd ::= DETACH database_kw_opt expr */
{
  sqlite3Detach(pParse, yymsp[0].minor.yy528);
}
        break;
-
      case 285: /* cmd ::= REINDEX */
+
      case 287: /* cmd ::= REINDEX */
{sqlite3Reindex(pParse, 0, 0);}
        break;
-
      case 286: /* cmd ::= REINDEX nm dbnm */
+
      case 288: /* cmd ::= REINDEX nm dbnm */
{sqlite3Reindex(pParse, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0);}
        break;
-
      case 287: /* cmd ::= ANALYZE */
+
      case 289: /* cmd ::= ANALYZE */
{sqlite3Analyze(pParse, 0, 0);}
        break;
-
      case 288: /* cmd ::= ANALYZE nm dbnm */
+
      case 290: /* cmd ::= ANALYZE nm dbnm */
{sqlite3Analyze(pParse, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0);}
        break;
-
      case 289: /* cmd ::= ALTER TABLE fullname RENAME TO nm */
+
      case 291: /* cmd ::= ALTER TABLE fullname RENAME TO nm */
{
  sqlite3AlterRenameTable(pParse,yymsp[-3].minor.yy131,&yymsp[0].minor.yy0);
}
        break;
-
      case 290: /* cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist */
+
      case 292: /* cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist */
{
  yymsp[-1].minor.yy0.n = (int)(pParse->sLastToken.z-yymsp[-1].minor.yy0.z) + pParse->sLastToken.n;
  sqlite3AlterFinishAddColumn(pParse, &yymsp[-1].minor.yy0);
}
        break;
-
      case 291: /* cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm */
+
      case 293: /* cmd ::= ALTER TABLE fullname DROP kwcolumn_opt nm */
{
  sqlite3AlterDropColumn(pParse, yymsp[-3].minor.yy131, &yymsp[0].minor.yy0);
}
        break;
-
      case 292: /* add_column_fullname ::= fullname */
+
      case 294: /* add_column_fullname ::= fullname */
{
  disableLookaside(pParse);
  sqlite3AlterBeginAddColumn(pParse, yymsp[0].minor.yy131);
}
        break;
-
      case 293: /* cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm */
+
      case 295: /* cmd ::= ALTER TABLE fullname RENAME kwcolumn_opt nm TO nm */
{
  sqlite3AlterRenameColumn(pParse, yymsp[-5].minor.yy131, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0);
}
        break;
-
      case 294: /* cmd ::= create_vtab */
+
      case 296: /* cmd ::= create_vtab */
{sqlite3VtabFinishParse(pParse,0);}
        break;
-
      case 295: /* cmd ::= create_vtab LP vtabarglist RP */
+
      case 297: /* cmd ::= create_vtab LP vtabarglist RP */
{sqlite3VtabFinishParse(pParse,&yymsp[0].minor.yy0);}
        break;
-
      case 296: /* create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm */
+
      case 298: /* create_vtab ::= createkw VIRTUAL TABLE ifnotexists nm dbnm USING nm */
{
    sqlite3VtabBeginParse(pParse, &yymsp[-3].minor.yy0, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, yymsp[-4].minor.yy394);
}
        break;
-
      case 297: /* vtabarg ::= */
+
      case 299: /* vtabarg ::= */
{sqlite3VtabArgInit(pParse);}
        break;
-
      case 298: /* vtabargtoken ::= ANY */
-
      case 299: /* vtabargtoken ::= lp anylist RP */ yytestcase(yyruleno==299);
-
      case 300: /* lp ::= LP */ yytestcase(yyruleno==300);
+
      case 300: /* vtabargtoken ::= ANY */
+
      case 301: /* vtabargtoken ::= lp anylist RP */ yytestcase(yyruleno==301);
+
      case 302: /* lp ::= LP */ yytestcase(yyruleno==302);
{sqlite3VtabArgExtend(pParse,&yymsp[0].minor.yy0);}
        break;
-
      case 301: /* with ::= WITH wqlist */
-
      case 302: /* with ::= WITH RECURSIVE wqlist */ yytestcase(yyruleno==302);
+
      case 303: /* with ::= WITH wqlist */
+
      case 304: /* with ::= WITH RECURSIVE wqlist */ yytestcase(yyruleno==304);
{ sqlite3WithPush(pParse, yymsp[0].minor.yy521, 1); }
        break;
-
      case 303: /* wqas ::= AS */
+
      case 305: /* wqas ::= AS */
{yymsp[0].minor.yy516 = M10d_Any;}
        break;
-
      case 304: /* wqas ::= AS MATERIALIZED */
+
      case 306: /* wqas ::= AS MATERIALIZED */
{yymsp[-1].minor.yy516 = M10d_Yes;}
        break;
-
      case 305: /* wqas ::= AS NOT MATERIALIZED */
+
      case 307: /* wqas ::= AS NOT MATERIALIZED */
{yymsp[-2].minor.yy516 = M10d_No;}
        break;
-
      case 306: /* wqitem ::= nm eidlist_opt wqas LP select RP */
+
      case 308: /* wqitem ::= nm eidlist_opt wqas LP select RP */
{
  yymsp[-5].minor.yy385 = sqlite3CteNew(pParse, &yymsp[-5].minor.yy0, yymsp[-4].minor.yy322, yymsp[-1].minor.yy47, yymsp[-3].minor.yy516); /*A-overwrites-X*/
}
        break;
-
      case 307: /* wqlist ::= wqitem */
+
      case 309: /* wqlist ::= wqitem */
{
  yymsp[0].minor.yy521 = sqlite3WithAdd(pParse, 0, yymsp[0].minor.yy385); /*A-overwrites-X*/
}
        break;
-
      case 308: /* wqlist ::= wqlist COMMA wqitem */
+
      case 310: /* wqlist ::= wqlist COMMA wqitem */
{
  yymsp[-2].minor.yy521 = sqlite3WithAdd(pParse, yymsp[-2].minor.yy521, yymsp[0].minor.yy385);
}
        break;
-
      case 309: /* windowdefn_list ::= windowdefn */
+
      case 311: /* windowdefn_list ::= windowdefn */
{ yylhsminor.yy41 = yymsp[0].minor.yy41; }
  yymsp[0].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 310: /* windowdefn_list ::= windowdefn_list COMMA windowdefn */
+
      case 312: /* windowdefn_list ::= windowdefn_list COMMA windowdefn */
{
  assert( yymsp[0].minor.yy41!=0 );
  sqlite3WindowChain(pParse, yymsp[0].minor.yy41, yymsp[-2].minor.yy41);
@@ -166258,7 +170142,7 @@ static YYACTIONTYPE yy_reduce(
}
  yymsp[-2].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 311: /* windowdefn ::= nm AS LP window RP */
+
      case 313: /* windowdefn ::= nm AS LP window RP */
{
  if( ALWAYS(yymsp[-1].minor.yy41) ){
    yymsp[-1].minor.yy41->zName = sqlite3DbStrNDup(pParse->db, yymsp[-4].minor.yy0.z, yymsp[-4].minor.yy0.n);
@@ -166267,90 +170151,90 @@ static YYACTIONTYPE yy_reduce(
}
  yymsp[-4].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 312: /* window ::= PARTITION BY nexprlist orderby_opt frame_opt */
+
      case 314: /* window ::= PARTITION BY nexprlist orderby_opt frame_opt */
{
  yymsp[-4].minor.yy41 = sqlite3WindowAssemble(pParse, yymsp[0].minor.yy41, yymsp[-2].minor.yy322, yymsp[-1].minor.yy322, 0);
}
        break;
-
      case 313: /* window ::= nm PARTITION BY nexprlist orderby_opt frame_opt */
+
      case 315: /* window ::= nm PARTITION BY nexprlist orderby_opt frame_opt */
{
  yylhsminor.yy41 = sqlite3WindowAssemble(pParse, yymsp[0].minor.yy41, yymsp[-2].minor.yy322, yymsp[-1].minor.yy322, &yymsp[-5].minor.yy0);
}
  yymsp[-5].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 314: /* window ::= ORDER BY sortlist frame_opt */
+
      case 316: /* window ::= ORDER BY sortlist frame_opt */
{
  yymsp[-3].minor.yy41 = sqlite3WindowAssemble(pParse, yymsp[0].minor.yy41, 0, yymsp[-1].minor.yy322, 0);
}
        break;
-
      case 315: /* window ::= nm ORDER BY sortlist frame_opt */
+
      case 317: /* window ::= nm ORDER BY sortlist frame_opt */
{
  yylhsminor.yy41 = sqlite3WindowAssemble(pParse, yymsp[0].minor.yy41, 0, yymsp[-1].minor.yy322, &yymsp[-4].minor.yy0);
}
  yymsp[-4].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 316: /* window ::= frame_opt */
-
      case 335: /* filter_over ::= over_clause */ yytestcase(yyruleno==335);
+
      case 318: /* window ::= frame_opt */
+
      case 337: /* filter_over ::= over_clause */ yytestcase(yyruleno==337);
{
  yylhsminor.yy41 = yymsp[0].minor.yy41;
}
  yymsp[0].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 317: /* window ::= nm frame_opt */
+
      case 319: /* window ::= nm frame_opt */
{
  yylhsminor.yy41 = sqlite3WindowAssemble(pParse, yymsp[0].minor.yy41, 0, 0, &yymsp[-1].minor.yy0);
}
  yymsp[-1].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 318: /* frame_opt ::= */
+
      case 320: /* frame_opt ::= */
{
  yymsp[1].minor.yy41 = sqlite3WindowAlloc(pParse, 0, TK_UNBOUNDED, 0, TK_CURRENT, 0, 0);
}
        break;
-
      case 319: /* frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt */
+
      case 321: /* frame_opt ::= range_or_rows frame_bound_s frame_exclude_opt */
{
  yylhsminor.yy41 = sqlite3WindowAlloc(pParse, yymsp[-2].minor.yy394, yymsp[-1].minor.yy595.eType, yymsp[-1].minor.yy595.pExpr, TK_CURRENT, 0, yymsp[0].minor.yy516);
}
  yymsp[-2].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 320: /* frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt */
+
      case 322: /* frame_opt ::= range_or_rows BETWEEN frame_bound_s AND frame_bound_e frame_exclude_opt */
{
  yylhsminor.yy41 = sqlite3WindowAlloc(pParse, yymsp[-5].minor.yy394, yymsp[-3].minor.yy595.eType, yymsp[-3].minor.yy595.pExpr, yymsp[-1].minor.yy595.eType, yymsp[-1].minor.yy595.pExpr, yymsp[0].minor.yy516);
}
  yymsp[-5].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 322: /* frame_bound_s ::= frame_bound */
-
      case 324: /* frame_bound_e ::= frame_bound */ yytestcase(yyruleno==324);
+
      case 324: /* frame_bound_s ::= frame_bound */
+
      case 326: /* frame_bound_e ::= frame_bound */ yytestcase(yyruleno==326);
{yylhsminor.yy595 = yymsp[0].minor.yy595;}
  yymsp[0].minor.yy595 = yylhsminor.yy595;
        break;
-
      case 323: /* frame_bound_s ::= UNBOUNDED PRECEDING */
-
      case 325: /* frame_bound_e ::= UNBOUNDED FOLLOWING */ yytestcase(yyruleno==325);
-
      case 327: /* frame_bound ::= CURRENT ROW */ yytestcase(yyruleno==327);
+
      case 325: /* frame_bound_s ::= UNBOUNDED PRECEDING */
+
      case 327: /* frame_bound_e ::= UNBOUNDED FOLLOWING */ yytestcase(yyruleno==327);
+
      case 329: /* frame_bound ::= CURRENT ROW */ yytestcase(yyruleno==329);
{yylhsminor.yy595.eType = yymsp[-1].major; yylhsminor.yy595.pExpr = 0;}
  yymsp[-1].minor.yy595 = yylhsminor.yy595;
        break;
-
      case 326: /* frame_bound ::= expr PRECEDING|FOLLOWING */
+
      case 328: /* frame_bound ::= expr PRECEDING|FOLLOWING */
{yylhsminor.yy595.eType = yymsp[0].major; yylhsminor.yy595.pExpr = yymsp[-1].minor.yy528;}
  yymsp[-1].minor.yy595 = yylhsminor.yy595;
        break;
-
      case 328: /* frame_exclude_opt ::= */
+
      case 330: /* frame_exclude_opt ::= */
{yymsp[1].minor.yy516 = 0;}
        break;
-
      case 329: /* frame_exclude_opt ::= EXCLUDE frame_exclude */
+
      case 331: /* frame_exclude_opt ::= EXCLUDE frame_exclude */
{yymsp[-1].minor.yy516 = yymsp[0].minor.yy516;}
        break;
-
      case 330: /* frame_exclude ::= NO OTHERS */
-
      case 331: /* frame_exclude ::= CURRENT ROW */ yytestcase(yyruleno==331);
+
      case 332: /* frame_exclude ::= NO OTHERS */
+
      case 333: /* frame_exclude ::= CURRENT ROW */ yytestcase(yyruleno==333);
{yymsp[-1].minor.yy516 = yymsp[-1].major; /*A-overwrites-X*/}
        break;
-
      case 332: /* frame_exclude ::= GROUP|TIES */
+
      case 334: /* frame_exclude ::= GROUP|TIES */
{yymsp[0].minor.yy516 = yymsp[0].major; /*A-overwrites-X*/}
        break;
-
      case 333: /* window_clause ::= WINDOW windowdefn_list */
+
      case 335: /* window_clause ::= WINDOW windowdefn_list */
{ yymsp[-1].minor.yy41 = yymsp[0].minor.yy41; }
        break;
-
      case 334: /* filter_over ::= filter_clause over_clause */
+
      case 336: /* filter_over ::= filter_clause over_clause */
{
  if( yymsp[0].minor.yy41 ){
    yymsp[0].minor.yy41->pFilter = yymsp[-1].minor.yy528;
@@ -166361,7 +170245,7 @@ static YYACTIONTYPE yy_reduce(
}
  yymsp[-1].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 336: /* filter_over ::= filter_clause */
+
      case 338: /* filter_over ::= filter_clause */
{
  yylhsminor.yy41 = (Window*)sqlite3DbMallocZero(pParse->db, sizeof(Window));
  if( yylhsminor.yy41 ){
@@ -166373,13 +170257,13 @@ static YYACTIONTYPE yy_reduce(
}
  yymsp[0].minor.yy41 = yylhsminor.yy41;
        break;
-
      case 337: /* over_clause ::= OVER LP window RP */
+
      case 339: /* over_clause ::= OVER LP window RP */
{
  yymsp[-3].minor.yy41 = yymsp[-1].minor.yy41;
  assert( yymsp[-3].minor.yy41!=0 );
}
        break;
-
      case 338: /* over_clause ::= OVER nm */
+
      case 340: /* over_clause ::= OVER nm */
{
  yymsp[-1].minor.yy41 = (Window*)sqlite3DbMallocZero(pParse->db, sizeof(Window));
  if( yymsp[-1].minor.yy41 ){
@@ -166387,72 +170271,73 @@ static YYACTIONTYPE yy_reduce(
  }
}
        break;
-
      case 339: /* filter_clause ::= FILTER LP WHERE expr RP */
+
      case 341: /* filter_clause ::= FILTER LP WHERE expr RP */
{ yymsp[-4].minor.yy528 = yymsp[-1].minor.yy528; }
        break;
      default:
-
      /* (340) input ::= cmdlist */ yytestcase(yyruleno==340);
-
      /* (341) cmdlist ::= cmdlist ecmd */ yytestcase(yyruleno==341);
-
      /* (342) cmdlist ::= ecmd (OPTIMIZED OUT) */ assert(yyruleno!=342);
-
      /* (343) ecmd ::= SEMI */ yytestcase(yyruleno==343);
-
      /* (344) ecmd ::= cmdx SEMI */ yytestcase(yyruleno==344);
-
      /* (345) ecmd ::= explain cmdx SEMI (NEVER REDUCES) */ assert(yyruleno!=345);
-
      /* (346) trans_opt ::= */ yytestcase(yyruleno==346);
-
      /* (347) trans_opt ::= TRANSACTION */ yytestcase(yyruleno==347);
-
      /* (348) trans_opt ::= TRANSACTION nm */ yytestcase(yyruleno==348);
-
      /* (349) savepoint_opt ::= SAVEPOINT */ yytestcase(yyruleno==349);
-
      /* (350) savepoint_opt ::= */ yytestcase(yyruleno==350);
-
      /* (351) cmd ::= create_table create_table_args */ yytestcase(yyruleno==351);
-
      /* (352) table_option_set ::= table_option (OPTIMIZED OUT) */ assert(yyruleno!=352);
-
      /* (353) columnlist ::= columnlist COMMA columnname carglist */ yytestcase(yyruleno==353);
-
      /* (354) columnlist ::= columnname carglist */ yytestcase(yyruleno==354);
-
      /* (355) nm ::= ID|INDEXED */ yytestcase(yyruleno==355);
-
      /* (356) nm ::= STRING */ yytestcase(yyruleno==356);
-
      /* (357) nm ::= JOIN_KW */ yytestcase(yyruleno==357);
-
      /* (358) typetoken ::= typename */ yytestcase(yyruleno==358);
-
      /* (359) typename ::= ID|STRING */ yytestcase(yyruleno==359);
-
      /* (360) signed ::= plus_num (OPTIMIZED OUT) */ assert(yyruleno!=360);
-
      /* (361) signed ::= minus_num (OPTIMIZED OUT) */ assert(yyruleno!=361);
-
      /* (362) carglist ::= carglist ccons */ yytestcase(yyruleno==362);
-
      /* (363) carglist ::= */ yytestcase(yyruleno==363);
-
      /* (364) ccons ::= NULL onconf */ yytestcase(yyruleno==364);
-
      /* (365) ccons ::= GENERATED ALWAYS AS generated */ yytestcase(yyruleno==365);
-
      /* (366) ccons ::= AS generated */ yytestcase(yyruleno==366);
-
      /* (367) conslist_opt ::= COMMA conslist */ yytestcase(yyruleno==367);
-
      /* (368) conslist ::= conslist tconscomma tcons */ yytestcase(yyruleno==368);
-
      /* (369) conslist ::= tcons (OPTIMIZED OUT) */ assert(yyruleno!=369);
-
      /* (370) tconscomma ::= */ yytestcase(yyruleno==370);
-
      /* (371) defer_subclause_opt ::= defer_subclause (OPTIMIZED OUT) */ assert(yyruleno!=371);
-
      /* (372) resolvetype ::= raisetype (OPTIMIZED OUT) */ assert(yyruleno!=372);
-
      /* (373) selectnowith ::= oneselect (OPTIMIZED OUT) */ assert(yyruleno!=373);
-
      /* (374) oneselect ::= values */ yytestcase(yyruleno==374);
-
      /* (375) sclp ::= selcollist COMMA */ yytestcase(yyruleno==375);
-
      /* (376) as ::= ID|STRING */ yytestcase(yyruleno==376);
-
      /* (377) returning ::= */ yytestcase(yyruleno==377);
-
      /* (378) expr ::= term (OPTIMIZED OUT) */ assert(yyruleno!=378);
-
      /* (379) likeop ::= LIKE_KW|MATCH */ yytestcase(yyruleno==379);
-
      /* (380) exprlist ::= nexprlist */ yytestcase(yyruleno==380);
-
      /* (381) nmnum ::= plus_num (OPTIMIZED OUT) */ assert(yyruleno!=381);
-
      /* (382) nmnum ::= nm (OPTIMIZED OUT) */ assert(yyruleno!=382);
-
      /* (383) nmnum ::= ON */ yytestcase(yyruleno==383);
-
      /* (384) nmnum ::= DELETE */ yytestcase(yyruleno==384);
-
      /* (385) nmnum ::= DEFAULT */ yytestcase(yyruleno==385);
-
      /* (386) plus_num ::= INTEGER|FLOAT */ yytestcase(yyruleno==386);
-
      /* (387) foreach_clause ::= */ yytestcase(yyruleno==387);
-
      /* (388) foreach_clause ::= FOR EACH ROW */ yytestcase(yyruleno==388);
-
      /* (389) trnm ::= nm */ yytestcase(yyruleno==389);
-
      /* (390) tridxby ::= */ yytestcase(yyruleno==390);
-
      /* (391) database_kw_opt ::= DATABASE */ yytestcase(yyruleno==391);
-
      /* (392) database_kw_opt ::= */ yytestcase(yyruleno==392);
-
      /* (393) kwcolumn_opt ::= */ yytestcase(yyruleno==393);
-
      /* (394) kwcolumn_opt ::= COLUMNKW */ yytestcase(yyruleno==394);
-
      /* (395) vtabarglist ::= vtabarg */ yytestcase(yyruleno==395);
-
      /* (396) vtabarglist ::= vtabarglist COMMA vtabarg */ yytestcase(yyruleno==396);
-
      /* (397) vtabarg ::= vtabarg vtabargtoken */ yytestcase(yyruleno==397);
-
      /* (398) anylist ::= */ yytestcase(yyruleno==398);
-
      /* (399) anylist ::= anylist LP anylist RP */ yytestcase(yyruleno==399);
-
      /* (400) anylist ::= anylist ANY */ yytestcase(yyruleno==400);
-
      /* (401) with ::= */ yytestcase(yyruleno==401);
+
      /* (342) input ::= cmdlist */ yytestcase(yyruleno==342);
+
      /* (343) cmdlist ::= cmdlist ecmd */ yytestcase(yyruleno==343);
+
      /* (344) cmdlist ::= ecmd (OPTIMIZED OUT) */ assert(yyruleno!=344);
+
      /* (345) ecmd ::= SEMI */ yytestcase(yyruleno==345);
+
      /* (346) ecmd ::= cmdx SEMI */ yytestcase(yyruleno==346);
+
      /* (347) ecmd ::= explain cmdx SEMI (NEVER REDUCES) */ assert(yyruleno!=347);
+
      /* (348) trans_opt ::= */ yytestcase(yyruleno==348);
+
      /* (349) trans_opt ::= TRANSACTION */ yytestcase(yyruleno==349);
+
      /* (350) trans_opt ::= TRANSACTION nm */ yytestcase(yyruleno==350);
+
      /* (351) savepoint_opt ::= SAVEPOINT */ yytestcase(yyruleno==351);
+
      /* (352) savepoint_opt ::= */ yytestcase(yyruleno==352);
+
      /* (353) cmd ::= create_table create_table_args */ yytestcase(yyruleno==353);
+
      /* (354) table_option_set ::= table_option (OPTIMIZED OUT) */ assert(yyruleno!=354);
+
      /* (355) columnlist ::= columnlist COMMA columnname carglist */ yytestcase(yyruleno==355);
+
      /* (356) columnlist ::= columnname carglist */ yytestcase(yyruleno==356);
+
      /* (357) nm ::= ID|INDEXED */ yytestcase(yyruleno==357);
+
      /* (358) nm ::= STRING */ yytestcase(yyruleno==358);
+
      /* (359) nm ::= JOIN_KW */ yytestcase(yyruleno==359);
+
      /* (360) typetoken ::= typename */ yytestcase(yyruleno==360);
+
      /* (361) typename ::= ID|STRING */ yytestcase(yyruleno==361);
+
      /* (362) signed ::= plus_num (OPTIMIZED OUT) */ assert(yyruleno!=362);
+
      /* (363) signed ::= minus_num (OPTIMIZED OUT) */ assert(yyruleno!=363);
+
      /* (364) carglist ::= carglist ccons */ yytestcase(yyruleno==364);
+
      /* (365) carglist ::= */ yytestcase(yyruleno==365);
+
      /* (366) ccons ::= NULL onconf */ yytestcase(yyruleno==366);
+
      /* (367) ccons ::= GENERATED ALWAYS AS generated */ yytestcase(yyruleno==367);
+
      /* (368) ccons ::= AS generated */ yytestcase(yyruleno==368);
+
      /* (369) conslist_opt ::= COMMA conslist */ yytestcase(yyruleno==369);
+
      /* (370) conslist ::= conslist tconscomma tcons */ yytestcase(yyruleno==370);
+
      /* (371) conslist ::= tcons (OPTIMIZED OUT) */ assert(yyruleno!=371);
+
      /* (372) tconscomma ::= */ yytestcase(yyruleno==372);
+
      /* (373) defer_subclause_opt ::= defer_subclause (OPTIMIZED OUT) */ assert(yyruleno!=373);
+
      /* (374) resolvetype ::= raisetype (OPTIMIZED OUT) */ assert(yyruleno!=374);
+
      /* (375) selectnowith ::= oneselect (OPTIMIZED OUT) */ assert(yyruleno!=375);
+
      /* (376) oneselect ::= values */ yytestcase(yyruleno==376);
+
      /* (377) sclp ::= selcollist COMMA */ yytestcase(yyruleno==377);
+
      /* (378) as ::= ID|STRING */ yytestcase(yyruleno==378);
+
      /* (379) indexed_opt ::= indexed_by (OPTIMIZED OUT) */ assert(yyruleno!=379);
+
      /* (380) returning ::= */ yytestcase(yyruleno==380);
+
      /* (381) expr ::= term (OPTIMIZED OUT) */ assert(yyruleno!=381);
+
      /* (382) likeop ::= LIKE_KW|MATCH */ yytestcase(yyruleno==382);
+
      /* (383) exprlist ::= nexprlist */ yytestcase(yyruleno==383);
+
      /* (384) nmnum ::= plus_num (OPTIMIZED OUT) */ assert(yyruleno!=384);
+
      /* (385) nmnum ::= nm (OPTIMIZED OUT) */ assert(yyruleno!=385);
+
      /* (386) nmnum ::= ON */ yytestcase(yyruleno==386);
+
      /* (387) nmnum ::= DELETE */ yytestcase(yyruleno==387);
+
      /* (388) nmnum ::= DEFAULT */ yytestcase(yyruleno==388);
+
      /* (389) plus_num ::= INTEGER|FLOAT */ yytestcase(yyruleno==389);
+
      /* (390) foreach_clause ::= */ yytestcase(yyruleno==390);
+
      /* (391) foreach_clause ::= FOR EACH ROW */ yytestcase(yyruleno==391);
+
      /* (392) trnm ::= nm */ yytestcase(yyruleno==392);
+
      /* (393) tridxby ::= */ yytestcase(yyruleno==393);
+
      /* (394) database_kw_opt ::= DATABASE */ yytestcase(yyruleno==394);
+
      /* (395) database_kw_opt ::= */ yytestcase(yyruleno==395);
+
      /* (396) kwcolumn_opt ::= */ yytestcase(yyruleno==396);
+
      /* (397) kwcolumn_opt ::= COLUMNKW */ yytestcase(yyruleno==397);
+
      /* (398) vtabarglist ::= vtabarg */ yytestcase(yyruleno==398);
+
      /* (399) vtabarglist ::= vtabarglist COMMA vtabarg */ yytestcase(yyruleno==399);
+
      /* (400) vtabarg ::= vtabarg vtabargtoken */ yytestcase(yyruleno==400);
+
      /* (401) anylist ::= */ yytestcase(yyruleno==401);
+
      /* (402) anylist ::= anylist LP anylist RP */ yytestcase(yyruleno==402);
+
      /* (403) anylist ::= anylist ANY */ yytestcase(yyruleno==403);
+
      /* (404) with ::= */ yytestcase(yyruleno==404);
        break;
/********** End reduce actions ************************************************/
  };
@@ -167899,6 +171784,7 @@ SQLITE_PRIVATE int sqlite3RunParser(Parse *pParse, const char *zSql){
    mxSqlLen -= n;
    if( mxSqlLen<0 ){
      pParse->rc = SQLITE_TOOBIG;
+
      pParse->nErr++;
      break;
    }
#ifndef SQLITE_OMIT_WINDOWFUNC
@@ -167995,7 +171881,7 @@ SQLITE_PRIVATE int sqlite3RunParser(Parse *pParse, const char *zSql){
  if( pParse->pNewTrigger && !IN_RENAME_OBJECT ){
    sqlite3DeleteTrigger(db, pParse->pNewTrigger);
  }
-
  sqlite3DbFree(db, pParse->pVList);
+
  if( pParse->pVList ) sqlite3DbNNFreeNN(db, pParse->pVList);
  db->pParse = pParentParse;
  assert( nErr==0 || pParse->rc!=SQLITE_OK );
  return nErr;
@@ -169351,18 +173237,19 @@ static int setupLookaside(sqlite3 *db, void *pBuf, int sz, int cnt){
    db->lookaside.bMalloced = pBuf==0 ?1:0;
    db->lookaside.nSlot = nBig+nSm;
  }else{
-
    db->lookaside.pStart = db;
+
    db->lookaside.pStart = 0;
#ifndef SQLITE_OMIT_TWOSIZE_LOOKASIDE
    db->lookaside.pSmallInit = 0;
    db->lookaside.pSmallFree = 0;
-
    db->lookaside.pMiddle = db;
+
    db->lookaside.pMiddle = 0;
#endif /* SQLITE_OMIT_TWOSIZE_LOOKASIDE */
-
    db->lookaside.pEnd = db;
+
    db->lookaside.pEnd = 0;
    db->lookaside.bDisable = 1;
    db->lookaside.sz = 0;
    db->lookaside.bMalloced = 0;
    db->lookaside.nSlot = 0;
  }
+
  db->lookaside.pTrueEnd = db->lookaside.pEnd;
  assert( sqlite3LookasideUsed(db,0)==0 );
#endif /* SQLITE_OMIT_LOOKASIDE */
  return SQLITE_OK;
@@ -169441,6 +173328,7 @@ SQLITE_API int sqlite3_db_cacheflush(sqlite3 *db){
SQLITE_API int sqlite3_db_config(sqlite3 *db, int op, ...){
  va_list ap;
  int rc;
+
  sqlite3_mutex_enter(db->mutex);
  va_start(ap, op);
  switch( op ){
    case SQLITE_DBCONFIG_MAINDBNAME: {
@@ -169506,6 +173394,7 @@ SQLITE_API int sqlite3_db_config(sqlite3 *db, int op, ...){
    }
  }
  va_end(ap);
+
  sqlite3_mutex_leave(db->mutex);
  return rc;
}

@@ -171876,6 +175765,19 @@ static int openDatabase(
    goto opendb_out;
  }

+
#if SQLITE_OS_UNIX && defined(SQLITE_OS_KV_OPTIONAL)
+
  /* Process magic filenames ":localStorage:" and ":sessionStorage:" */
+
  if( zFilename && zFilename[0]==':' ){
+
    if( strcmp(zFilename, ":localStorage:")==0 ){
+
      zFilename = "file:local?vfs=kvvfs";
+
      flags |= SQLITE_OPEN_URI;
+
    }else if( strcmp(zFilename, ":sessionStorage:")==0 ){
+
      zFilename = "file:session?vfs=kvvfs";
+
      flags |= SQLITE_OPEN_URI;
+
    }
+
  }
+
#endif /* SQLITE_OS_UNIX && defined(SQLITE_OS_KV_OPTIONAL) */
+

  /* Parse the filename/URI argument
  **
  ** Only allow sensible combinations of bits in the flags argument.
@@ -171906,6 +175808,12 @@ static int openDatabase(
    sqlite3_free(zErrMsg);
    goto opendb_out;
  }
+
  assert( db->pVfs!=0 );
+
#if SQLITE_OS_KV || defined(SQLITE_OS_KV_OPTIONAL)
+
  if( sqlite3_stricmp(db->pVfs->zName, "kvvfs")==0 ){
+
    db->temp_store = 2;
+
  }
+
#endif

  /* Open the backend database driver */
  rc = sqlite3BtreeOpen(db->pVfs, zOpen, db, &db->aDb[0].pBt, 0,
@@ -172624,6 +176532,28 @@ SQLITE_API int sqlite3_test_control(int op, ...){
      volatile int x = 0;
      assert( /*side-effects-ok*/ (x = va_arg(ap,int))!=0 );
      rc = x;
+
#if defined(SQLITE_DEBUG)
+
      /* Invoke these debugging routines so that the compiler does not
+
      ** issue "defined but not used" warnings. */
+
      if( x==9999 ){
+
        sqlite3ShowExpr(0);
+
        sqlite3ShowExpr(0);
+
        sqlite3ShowExprList(0);
+
        sqlite3ShowIdList(0);
+
        sqlite3ShowSrcList(0);
+
        sqlite3ShowWith(0);
+
        sqlite3ShowUpsert(0);
+
        sqlite3ShowTriggerStep(0);
+
        sqlite3ShowTriggerStepList(0);
+
        sqlite3ShowTrigger(0);
+
        sqlite3ShowTriggerList(0);
+
#ifndef SQLITE_OMIT_WINDOWFUNC
+
        sqlite3ShowWindow(0);
+
        sqlite3ShowWinFunc(0);
+
#endif
+
        sqlite3ShowSelect(0);
+
      }
+
#endif
      break;
    }

@@ -172885,8 +176815,8 @@ SQLITE_API int sqlite3_test_control(int op, ...){
    **
    **  "ptr" is a pointer to a u32.
    **
-
    **   op==0       Store the current sqlite3SelectTrace in *ptr
-
    **   op==1       Set sqlite3SelectTrace to the value *ptr
+
    **   op==0       Store the current sqlite3TreeTrace in *ptr
+
    **   op==1       Set sqlite3TreeTrace to the value *ptr
    **   op==3       Store the current sqlite3WhereTrace in *ptr
    **   op==3       Set sqlite3WhereTrace to the value *ptr
    */
@@ -172894,10 +176824,10 @@ SQLITE_API int sqlite3_test_control(int op, ...){
       int opTrace = va_arg(ap, int);
       u32 *ptr = va_arg(ap, u32*);
       switch( opTrace ){
-
         case 0:   *ptr = sqlite3SelectTrace;      break;
-
         case 1:   sqlite3SelectTrace = *ptr;      break;
-
         case 2:   *ptr = sqlite3WhereTrace;       break;
-
         case 3:   sqlite3WhereTrace = *ptr;       break;
+
         case 0:   *ptr = sqlite3TreeTrace;      break;
+
         case 1:   sqlite3TreeTrace = *ptr;      break;
+
         case 2:   *ptr = sqlite3WhereTrace;     break;
+
         case 3:   sqlite3WhereTrace = *ptr;     break;
       }
       break;
    }
@@ -172914,10 +176844,12 @@ SQLITE_API int sqlite3_test_control(int op, ...){
    case SQLITE_TESTCTRL_LOGEST: {
      double rIn = va_arg(ap, double);
      LogEst rLogEst = sqlite3LogEstFromDouble(rIn);
-
      u64 iInt = sqlite3LogEstToInt(rLogEst);
-
      va_arg(ap, int*)[0] = rLogEst;
-
      va_arg(ap, u64*)[0] = iInt;
-
      va_arg(ap, int*)[0] = sqlite3LogEst(iInt);
+
      int *pI1 = va_arg(ap,int*);
+
      u64 *pU64 = va_arg(ap,u64*);
+
      int *pI2 = va_arg(ap,int*);
+
      *pI1 = rLogEst;
+
      *pU64 = sqlite3LogEstToInt(rLogEst);
+
      *pI2 = sqlite3LogEst(*pU64);
      break;
    }

@@ -172991,7 +176923,7 @@ static char *appendText(char *p, const char *z){
** Memory layout must be compatible with that generated by the pager
** and expected by sqlite3_uri_parameter() and databaseName().
*/
-
SQLITE_API char *sqlite3_create_filename(
+
SQLITE_API const char *sqlite3_create_filename(
  const char *zDatabase,
  const char *zJournal,
  const char *zWal,
@@ -173027,10 +176959,10 @@ SQLITE_API char *sqlite3_create_filename(
** error to call this routine with any parameter other than a pointer
** previously obtained from sqlite3_create_filename() or a NULL pointer.
*/
-
SQLITE_API void sqlite3_free_filename(char *p){
+
SQLITE_API void sqlite3_free_filename(const char *p){
  if( p==0 ) return;
-
  p = (char*)databaseName(p);
-
  sqlite3_free(p - 4);
+
  p = databaseName(p);
+
  sqlite3_free((char*)p - 4);
}


@@ -173133,6 +177065,24 @@ SQLITE_PRIVATE Btree *sqlite3DbNameToBtree(sqlite3 *db, const char *zDbName){
}

/*
+
** Return the name of the N-th database schema.  Return NULL if N is out
+
** of range.
+
*/
+
SQLITE_API const char *sqlite3_db_name(sqlite3 *db, int N){
+
#ifdef SQLITE_ENABLE_API_ARMOR
+
  if( !sqlite3SafetyCheckOk(db) ){
+
    (void)SQLITE_MISUSE_BKPT;
+
    return 0;
+
  }
+
#endif
+
  if( N<0 || N>=db->nDb ){
+
    return 0;
+
  }else{
+
    return db->aDb[N].zDbSName;
+
  }
+
}
+

+
/*
** Return the filename of the database associated with a database
** connection.
*/
@@ -173263,8 +177213,8 @@ SQLITE_API int sqlite3_snapshot_open(
*/
SQLITE_API int sqlite3_snapshot_recover(sqlite3 *db, const char *zDb){
  int rc = SQLITE_ERROR;
-
  int iDb;
#ifndef SQLITE_OMIT_WAL
+
  int iDb;

#ifdef SQLITE_ENABLE_API_ARMOR
  if( !sqlite3SafetyCheckOk(db) ){
@@ -174819,7 +178769,7 @@ struct Fts3MultiSegReader {
  int nAdvance;                   /* How many seg-readers to advance */
  Fts3SegFilter *pFilter;         /* Pointer to filter object */
  char *aBuffer;                  /* Buffer to merge doclists in */
-
  int nBuffer;                    /* Allocated size of aBuffer[] in bytes */
+
  i64 nBuffer;                    /* Allocated size of aBuffer[] in bytes */

  int iColFilter;                 /* If >=0, filter for this column */
  int bRestart;
@@ -177515,7 +181465,7 @@ static int fts3TermSelectMerge(
    **
    ** Similar padding is added in the fts3DoclistOrMerge() function.
    */
-
    pTS->aaOutput[0] = sqlite3_malloc(nDoclist + FTS3_VARINT_MAX + 1);
+
    pTS->aaOutput[0] = sqlite3_malloc64((i64)nDoclist + FTS3_VARINT_MAX + 1);
    pTS->anOutput[0] = nDoclist;
    if( pTS->aaOutput[0] ){
      memcpy(pTS->aaOutput[0], aDoclist, nDoclist);
@@ -178935,8 +182885,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
  char *aPoslist = 0;             /* Position list for deferred tokens */
  int nPoslist = 0;               /* Number of bytes in aPoslist */
  int iPrev = -1;                 /* Token number of previous deferred token */
-

-
  assert( pPhrase->doclist.bFreeList==0 );
+
  char *aFree = (pPhrase->doclist.bFreeList ? pPhrase->doclist.pList : 0);

  for(iToken=0; iToken<pPhrase->nToken; iToken++){
    Fts3PhraseToken *pToken = &pPhrase->aToken[iToken];
@@ -178950,6 +182899,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){

      if( pList==0 ){
        sqlite3_free(aPoslist);
+
        sqlite3_free(aFree);
        pPhrase->doclist.pList = 0;
        pPhrase->doclist.nList = 0;
        return SQLITE_OK;
@@ -178970,6 +182920,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
        nPoslist = (int)(aOut - aPoslist);
        if( nPoslist==0 ){
          sqlite3_free(aPoslist);
+
          sqlite3_free(aFree);
          pPhrase->doclist.pList = 0;
          pPhrase->doclist.nList = 0;
          return SQLITE_OK;
@@ -179002,13 +182953,14 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
        nDistance = iPrev - nMaxUndeferred;
      }

-
      aOut = (char *)sqlite3_malloc(nPoslist+8);
+
      aOut = (char *)sqlite3Fts3MallocZero(nPoslist+FTS3_BUFFER_PADDING);
      if( !aOut ){
        sqlite3_free(aPoslist);
        return SQLITE_NOMEM;
      }

      pPhrase->doclist.pList = aOut;
+
      assert( p1 && p2 );
      if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){
        pPhrase->doclist.bFreeList = 1;
        pPhrase->doclist.nList = (int)(aOut - pPhrase->doclist.pList);
@@ -179021,6 +182973,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
    }
  }

+
  if( pPhrase->doclist.pList!=aFree ) sqlite3_free(aFree);
  return SQLITE_OK;
}
#endif /* SQLITE_DISABLE_FTS4_DEFERRED */
@@ -179369,7 +183322,7 @@ static int fts3EvalIncrPhraseNext(
      if( bEof==0 ){
        int nList = 0;
        int nByte = a[p->nToken-1].nList;
-
        char *aDoclist = sqlite3_malloc(nByte+FTS3_BUFFER_PADDING);
+
        char *aDoclist = sqlite3_malloc64((i64)nByte+FTS3_BUFFER_PADDING);
        if( !aDoclist ) return SQLITE_NOMEM;
        memcpy(aDoclist, a[p->nToken-1].pList, nByte+1);
        memset(&aDoclist[nByte], 0, FTS3_BUFFER_PADDING);
@@ -180195,11 +184148,10 @@ static int fts3EvalTestExpr(

      default: {
#ifndef SQLITE_DISABLE_FTS4_DEFERRED
-
        if( pCsr->pDeferred
-
         && (pExpr->iDocid==pCsr->iPrevId || pExpr->bDeferred)
-
        ){
+
        if( pCsr->pDeferred && (pExpr->bDeferred || (
+
            pExpr->iDocid==pCsr->iPrevId && pExpr->pPhrase->doclist.pList
+
        ))){
          Fts3Phrase *pPhrase = pExpr->pPhrase;
-
          assert( pExpr->bDeferred || pPhrase->doclist.bFreeList==0 );
          if( pExpr->bDeferred ){
            fts3EvalInvalidatePoslist(pPhrase);
          }
@@ -183606,7 +187558,7 @@ static int porterNext(
      if( n>c->nAllocated ){
        char *pNew;
        c->nAllocated = n+20;
-
        pNew = sqlite3_realloc(c->zToken, c->nAllocated);
+
        pNew = sqlite3_realloc64(c->zToken, c->nAllocated);
        if( !pNew ) return SQLITE_NOMEM;
        c->zToken = pNew;
      }
@@ -184358,7 +188310,7 @@ static int simpleNext(
      if( n>c->nTokenAllocated ){
        char *pNew;
        c->nTokenAllocated = n+20;
-
        pNew = sqlite3_realloc(c->pToken, c->nTokenAllocated);
+
        pNew = sqlite3_realloc64(c->pToken, c->nTokenAllocated);
        if( !pNew ) return SQLITE_NOMEM;
        c->pToken = pNew;
      }
@@ -185520,7 +189472,7 @@ static int fts3PendingListAppendVarint(

  /* Allocate or grow the PendingList as required. */
  if( !p ){
-
    p = sqlite3_malloc(sizeof(*p) + 100);
+
    p = sqlite3_malloc64(sizeof(*p) + 100);
    if( !p ){
      return SQLITE_NOMEM;
    }
@@ -185529,14 +189481,14 @@ static int fts3PendingListAppendVarint(
    p->nData = 0;
  }
  else if( p->nData+FTS3_VARINT_MAX+1>p->nSpace ){
-
    int nNew = p->nSpace * 2;
-
    p = sqlite3_realloc(p, sizeof(*p) + nNew);
+
    i64 nNew = p->nSpace * 2;
+
    p = sqlite3_realloc64(p, sizeof(*p) + nNew);
    if( !p ){
      sqlite3_free(*pp);
      *pp = 0;
      return SQLITE_NOMEM;
    }
-
    p->nSpace = nNew;
+
    p->nSpace = (int)nNew;
    p->aData = (char *)&p[1];
  }

@@ -186093,7 +190045,7 @@ SQLITE_PRIVATE int sqlite3Fts3ReadBlock(
    int nByte = sqlite3_blob_bytes(p->pSegments);
    *pnBlob = nByte;
    if( paBlob ){
-
      char *aByte = sqlite3_malloc(nByte + FTS3_NODE_PADDING);
+
      char *aByte = sqlite3_malloc64((i64)nByte + FTS3_NODE_PADDING);
      if( !aByte ){
        rc = SQLITE_NOMEM;
      }else{
@@ -186210,7 +190162,7 @@ static int fts3SegReaderNext(
        int nTerm = fts3HashKeysize(pElem);
        if( (nTerm+1)>pReader->nTermAlloc ){
          sqlite3_free(pReader->zTerm);
-
          pReader->zTerm = (char*)sqlite3_malloc((nTerm+1)*2);
+
          pReader->zTerm = (char*)sqlite3_malloc64(((i64)nTerm+1)*2);
          if( !pReader->zTerm ) return SQLITE_NOMEM;
          pReader->nTermAlloc = (nTerm+1)*2;
        }
@@ -186218,7 +190170,7 @@ static int fts3SegReaderNext(
        pReader->zTerm[nTerm] = '\0';
        pReader->nTerm = nTerm;

-
        aCopy = (char*)sqlite3_malloc(nCopy);
+
        aCopy = (char*)sqlite3_malloc64(nCopy);
        if( !aCopy ) return SQLITE_NOMEM;
        memcpy(aCopy, pList->aData, nCopy);
        pReader->nNode = pReader->nDoclist = nCopy;
@@ -186505,7 +190457,7 @@ SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(
    nExtra = nRoot + FTS3_NODE_PADDING;
  }

-
  pReader = (Fts3SegReader *)sqlite3_malloc(sizeof(Fts3SegReader) + nExtra);
+
  pReader = (Fts3SegReader *)sqlite3_malloc64(sizeof(Fts3SegReader) + nExtra);
  if( !pReader ){
    return SQLITE_NOMEM;
  }
@@ -186597,7 +190549,7 @@ SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(
        if( nElem==nAlloc ){
          Fts3HashElem **aElem2;
          nAlloc += 16;
-
          aElem2 = (Fts3HashElem **)sqlite3_realloc(
+
          aElem2 = (Fts3HashElem **)sqlite3_realloc64(
              aElem, nAlloc*sizeof(Fts3HashElem *)
          );
          if( !aElem2 ){
@@ -186931,7 +190883,7 @@ static int fts3NodeAddTerm(
        ** this is not expected to be a serious problem.
        */
        assert( pTree->aData==(char *)&pTree[1] );
-
        pTree->aData = (char *)sqlite3_malloc(nReq);
+
        pTree->aData = (char *)sqlite3_malloc64(nReq);
        if( !pTree->aData ){
          return SQLITE_NOMEM;
        }
@@ -186949,7 +190901,7 @@ static int fts3NodeAddTerm(

      if( isCopyTerm ){
        if( pTree->nMalloc<nTerm ){
-
          char *zNew = sqlite3_realloc(pTree->zMalloc, nTerm*2);
+
          char *zNew = sqlite3_realloc64(pTree->zMalloc, (i64)nTerm*2);
          if( !zNew ){
            return SQLITE_NOMEM;
          }
@@ -186975,7 +190927,7 @@ static int fts3NodeAddTerm(
  ** now. Instead, the term is inserted into the parent of pTree. If pTree
  ** has no parent, one is created here.
  */
-
  pNew = (SegmentNode *)sqlite3_malloc(sizeof(SegmentNode) + p->nNodeSize);
+
  pNew = (SegmentNode *)sqlite3_malloc64(sizeof(SegmentNode) + p->nNodeSize);
  if( !pNew ){
    return SQLITE_NOMEM;
  }
@@ -187113,7 +191065,7 @@ static int fts3SegWriterAdd(
){
  int nPrefix;                    /* Size of term prefix in bytes */
  int nSuffix;                    /* Size of term suffix in bytes */
-
  int nReq;                       /* Number of bytes required on leaf page */
+
  i64 nReq;                       /* Number of bytes required on leaf page */
  int nData;
  SegmentWriter *pWriter = *ppWriter;

@@ -187122,13 +191074,13 @@ static int fts3SegWriterAdd(
    sqlite3_stmt *pStmt;

    /* Allocate the SegmentWriter structure */
-
    pWriter = (SegmentWriter *)sqlite3_malloc(sizeof(SegmentWriter));
+
    pWriter = (SegmentWriter *)sqlite3_malloc64(sizeof(SegmentWriter));
    if( !pWriter ) return SQLITE_NOMEM;
    memset(pWriter, 0, sizeof(SegmentWriter));
    *ppWriter = pWriter;

    /* Allocate a buffer in which to accumulate data */
-
    pWriter->aData = (char *)sqlite3_malloc(p->nNodeSize);
+
    pWriter->aData = (char *)sqlite3_malloc64(p->nNodeSize);
    if( !pWriter->aData ) return SQLITE_NOMEM;
    pWriter->nSize = p->nNodeSize;

@@ -187203,7 +191155,7 @@ static int fts3SegWriterAdd(
  ** the buffer to make it large enough.
  */
  if( nReq>pWriter->nSize ){
-
    char *aNew = sqlite3_realloc(pWriter->aData, nReq);
+
    char *aNew = sqlite3_realloc64(pWriter->aData, nReq);
    if( !aNew ) return SQLITE_NOMEM;
    pWriter->aData = aNew;
    pWriter->nSize = nReq;
@@ -187228,7 +191180,7 @@ static int fts3SegWriterAdd(
  */
  if( isCopyTerm ){
    if( nTerm>pWriter->nMalloc ){
-
      char *zNew = sqlite3_realloc(pWriter->zMalloc, nTerm*2);
+
      char *zNew = sqlite3_realloc64(pWriter->zMalloc, (i64)nTerm*2);
      if( !zNew ){
        return SQLITE_NOMEM;
      }
@@ -187536,12 +191488,12 @@ static void fts3ColumnFilter(
static int fts3MsrBufferData(
  Fts3MultiSegReader *pMsr,       /* Multi-segment-reader handle */
  char *pList,
-
  int nList
+
  i64 nList
){
  if( nList>pMsr->nBuffer ){
    char *pNew;
    pMsr->nBuffer = nList*2;
-
    pNew = (char *)sqlite3_realloc(pMsr->aBuffer, pMsr->nBuffer);
+
    pNew = (char *)sqlite3_realloc64(pMsr->aBuffer, pMsr->nBuffer);
    if( !pNew ) return SQLITE_NOMEM;
    pMsr->aBuffer = pNew;
  }
@@ -187597,7 +191549,7 @@ SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext(
      fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp);

      if( nList>0 && fts3SegReaderIsPending(apSegment[0]) ){
-
        rc = fts3MsrBufferData(pMsr, pList, nList+1);
+
        rc = fts3MsrBufferData(pMsr, pList, (i64)nList+1);
        if( rc!=SQLITE_OK ) return rc;
        assert( (pMsr->aBuffer[nList] & 0xFE)==0x00 );
        pList = pMsr->aBuffer;
@@ -187734,11 +191686,11 @@ SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr){
  return SQLITE_OK;
}

-
static int fts3GrowSegReaderBuffer(Fts3MultiSegReader *pCsr, int nReq){
+
static int fts3GrowSegReaderBuffer(Fts3MultiSegReader *pCsr, i64 nReq){
  if( nReq>pCsr->nBuffer ){
    char *aNew;
    pCsr->nBuffer = nReq*2;
-
    aNew = sqlite3_realloc(pCsr->aBuffer, pCsr->nBuffer);
+
    aNew = sqlite3_realloc64(pCsr->aBuffer, pCsr->nBuffer);
    if( !aNew ){
      return SQLITE_NOMEM;
    }
@@ -187829,7 +191781,8 @@ SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(
    ){
      pCsr->nDoclist = apSegment[0]->nDoclist;
      if( fts3SegReaderIsPending(apSegment[0]) ){
-
        rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist, pCsr->nDoclist);
+
        rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist,
+
                               (i64)pCsr->nDoclist);
        pCsr->aDoclist = pCsr->aBuffer;
      }else{
        pCsr->aDoclist = apSegment[0]->aDoclist;
@@ -187882,7 +191835,8 @@ SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(

          nByte = sqlite3Fts3VarintLen(iDelta) + (isRequirePos?nList+1:0);

-
          rc = fts3GrowSegReaderBuffer(pCsr, nByte+nDoclist+FTS3_NODE_PADDING);
+
          rc = fts3GrowSegReaderBuffer(pCsr,
+
                                   (i64)nByte+nDoclist+FTS3_NODE_PADDING);
          if( rc ) return rc;

          if( isFirst ){
@@ -187908,7 +191862,7 @@ SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(
        fts3SegReaderSort(apSegment, nMerge, j, xCmp);
      }
      if( nDoclist>0 ){
-
        rc = fts3GrowSegReaderBuffer(pCsr, nDoclist+FTS3_NODE_PADDING);
+
        rc = fts3GrowSegReaderBuffer(pCsr, (i64)nDoclist+FTS3_NODE_PADDING);
        if( rc ) return rc;
        memset(&pCsr->aBuffer[nDoclist], 0, FTS3_NODE_PADDING);
        pCsr->aDoclist = pCsr->aBuffer;
@@ -188621,7 +192575,7 @@ struct NodeReader {
static void blobGrowBuffer(Blob *pBlob, int nMin, int *pRc){
  if( *pRc==SQLITE_OK && nMin>pBlob->nAlloc ){
    int nAlloc = nMin;
-
    char *a = (char *)sqlite3_realloc(pBlob->a, nAlloc);
+
    char *a = (char *)sqlite3_realloc64(pBlob->a, nAlloc);
    if( a ){
      pBlob->nAlloc = nAlloc;
      pBlob->a = a;
@@ -188770,6 +192724,8 @@ static int fts3IncrmergePush(
          pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nPrefix);
        }
        pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nSuffix);
+
        assert( nPrefix+nSuffix<=nTerm );
+
        assert( nPrefix>=0 );
        memcpy(&pBlk->a[pBlk->n], &zTerm[nPrefix], nSuffix);
        pBlk->n += nSuffix;

@@ -188892,6 +192848,7 @@ static int fts3IncrmergeAppend(
  pLeaf = &pWriter->aNodeWriter[0];
  nPrefix = fts3PrefixCompress(pLeaf->key.a, pLeaf->key.n, zTerm, nTerm);
  nSuffix = nTerm - nPrefix;
+
  if(nSuffix<=0 ) return FTS_CORRUPT_VTAB;

  nSpace  = sqlite3Fts3VarintLen(nPrefix);
  nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix;
@@ -189415,7 +193372,7 @@ static int fts3RepackSegdirLevel(
      if( nIdx>=nAlloc ){
        int *aNew;
        nAlloc += 16;
-
        aNew = sqlite3_realloc(aIdx, nAlloc*sizeof(int));
+
        aNew = sqlite3_realloc64(aIdx, nAlloc*sizeof(int));
        if( !aNew ){
          rc = SQLITE_NOMEM;
          break;
@@ -189789,7 +193746,7 @@ SQLITE_PRIVATE int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){

  /* Allocate space for the cursor, filter and writer objects */
  const int nAlloc = sizeof(*pCsr) + sizeof(*pFilter) + sizeof(*pWriter);
-
  pWriter = (IncrmergeWriter *)sqlite3_malloc(nAlloc);
+
  pWriter = (IncrmergeWriter *)sqlite3_malloc64(nAlloc);
  if( !pWriter ) return SQLITE_NOMEM;
  pFilter = (Fts3SegFilter *)&pWriter[1];
  pCsr = (Fts3MultiSegReader *)&pFilter[1];
@@ -190425,7 +194382,7 @@ SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList(
    return SQLITE_OK;
  }

-
  pRet = (char *)sqlite3_malloc(p->pList->nData);
+
  pRet = (char *)sqlite3_malloc64(p->pList->nData);
  if( !pRet ) return SQLITE_NOMEM;

  nSkip = sqlite3Fts3GetVarint(p->pList->aData, &dummy);
@@ -190445,7 +194402,7 @@ SQLITE_PRIVATE int sqlite3Fts3DeferToken(
  int iCol                        /* Column that token must appear in (or -1) */
){
  Fts3DeferredToken *pDeferred;
-
  pDeferred = sqlite3_malloc(sizeof(*pDeferred));
+
  pDeferred = sqlite3_malloc64(sizeof(*pDeferred));
  if( !pDeferred ){
    return SQLITE_NOMEM;
  }
@@ -194849,7 +198806,7 @@ static JsonNode *jsonMergePatch(
  if( pPatch->eType!=JSON_OBJECT ){
    return pPatch;
  }
-
  assert( iTarget>=0 && iTarget<pParse->nNode );
+
  assert( iTarget<pParse->nNode );
  pTarget = &pParse->aNode[iTarget];
  assert( (pPatch->jnFlags & JNODE_APPEND)==0 );
  if( pTarget->eType!=JSON_OBJECT ){
@@ -199160,7 +203117,7 @@ static int rtreeUpdate(
  rtreeReference(pRtree);
  assert(nData>=1);

-
  cell.iRowid = 0;  /* Used only to suppress a compiler warning */
+
  memset(&cell, 0, sizeof(cell));

  /* Constraint handling. A write operation on an r-tree table may return
  ** SQLITE_CONSTRAINT for two reasons:
@@ -202024,7 +205981,7 @@ static int geopolyUpdate(
      sqlite3_free(p);
      nChange = 1;
    }
-
    for(jj=1; jj<pRtree->nAux; jj++){
+
    for(jj=1; jj<nData-2; jj++){
      nChange++;
      sqlite3_bind_value(pUp, jj+2, aData[jj+2]);
    }
@@ -202627,8 +206584,9 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){

    if( U_SUCCESS(status) ){
      sqlite3_set_auxdata(p, 0, pExpr, icuRegexpDelete);
-
    }else{
-
      assert(!pExpr);
+
      pExpr = sqlite3_get_auxdata(p, 0);
+
    }
+
    if( !pExpr ){
      icuFunctionError(p, "uregex_open", status);
      return;
    }
@@ -203786,6 +207744,34 @@ SQLITE_API void sqlite3rbu_bp_progress(sqlite3rbu *pRbu, int *pnOne, int*pnTwo);
SQLITE_API int sqlite3rbu_state(sqlite3rbu *pRbu);

/*
+
** As part of applying an RBU update or performing an RBU vacuum operation,
+
** the system must at one point move the *-oal file to the equivalent *-wal
+
** path. Normally, it does this by invoking POSIX function rename(2) directly.
+
** Except on WINCE platforms, where it uses win32 API MoveFileW(). This
+
** function may be used to register a callback that the RBU module will invoke
+
** instead of one of these APIs.
+
**
+
** If a callback is registered with an RBU handle, it invokes it instead
+
** of rename(2) when it needs to move a file within the file-system. The
+
** first argument passed to the xRename() callback is a copy of the second
+
** argument (pArg) passed to this function. The second is the full path
+
** to the file to move and the third the full path to which it should be
+
** moved. The callback function should return SQLITE_OK to indicate
+
** success. If an error occurs, it should return an SQLite error code.
+
** In this case the RBU operation will be abandoned and the error returned
+
** to the RBU user.
+
**
+
** Passing a NULL pointer in place of the xRename argument to this function
+
** restores the default behaviour.
+
*/
+
SQLITE_API void sqlite3rbu_rename_handler(
+
  sqlite3rbu *pRbu,
+
  void *pArg,
+
  int (*xRename)(void *pArg, const char *zOld, const char *zNew)
+
);
+

+

+
/*
** Create an RBU VFS named zName that accesses the underlying file-system
** via existing VFS zParent. Or, if the zParent parameter is passed NULL,
** then the new RBU VFS uses the default system VFS to access the file-system.
@@ -204152,6 +208138,8 @@ struct sqlite3rbu {
  int nPagePerSector;             /* Pages per sector for pTargetFd */
  i64 iOalSz;
  i64 nPhaseOneStep;
+
  void *pRenameArg;
+
  int (*xRename)(void*, const char*, const char*);

  /* The following state variables are used as part of the incremental
  ** checkpoint stage (eStage==RBU_STAGE_CKPT). See comments surrounding
@@ -206540,7 +210528,7 @@ static void rbuOpenDatabase(sqlite3rbu *p, sqlite3 *dbMain, int *pbRetry){
    sqlite3_file_control(p->dbRbu, "main", SQLITE_FCNTL_RBUCNT, (void*)p);
    if( p->zState==0 ){
      const char *zFile = sqlite3_db_filename(p->dbRbu, "main");
-
      p->zState = rbuMPrintf(p, "file://%s-vacuum?modeof=%s", zFile, zFile);
+
      p->zState = rbuMPrintf(p, "file:///%s-vacuum?modeof=%s", zFile, zFile);
    }
  }

@@ -207000,32 +210988,7 @@ static void rbuMoveOalFile(sqlite3rbu *p){
    }

    if( p->rc==SQLITE_OK ){
-
#if defined(_WIN32_WCE)
-
      {
-
        LPWSTR zWideOal;
-
        LPWSTR zWideWal;
-

-
        zWideOal = rbuWinUtf8ToUnicode(zOal);
-
        if( zWideOal ){
-
          zWideWal = rbuWinUtf8ToUnicode(zWal);
-
          if( zWideWal ){
-
            if( MoveFileW(zWideOal, zWideWal) ){
-
              p->rc = SQLITE_OK;
-
            }else{
-
              p->rc = SQLITE_IOERR;
-
            }
-
            sqlite3_free(zWideWal);
-
          }else{
-
            p->rc = SQLITE_IOERR_NOMEM;
-
          }
-
          sqlite3_free(zWideOal);
-
        }else{
-
          p->rc = SQLITE_IOERR_NOMEM;
-
        }
-
      }
-
#else
-
      p->rc = rename(zOal, zWal) ? SQLITE_IOERR : SQLITE_OK;
-
#endif
+
      p->rc = p->xRename(p->pRenameArg, zOal, zWal);
    }

    if( p->rc!=SQLITE_OK
@@ -207764,6 +211727,7 @@ static sqlite3rbu *openRbuHandle(

    /* Create the custom VFS. */
    memset(p, 0, sizeof(sqlite3rbu));
+
    sqlite3rbu_rename_handler(p, 0, 0);
    rbuCreateVfs(p);

    /* Open the target, RBU and state databases */
@@ -208155,6 +212119,54 @@ SQLITE_API int sqlite3rbu_savestate(sqlite3rbu *p){
  return rc;
}

+
/*
+
** Default xRename callback for RBU.
+
*/
+
static int xDefaultRename(void *pArg, const char *zOld, const char *zNew){
+
  int rc = SQLITE_OK;
+
#if defined(_WIN32_WCE)
+
  {
+
    LPWSTR zWideOld;
+
    LPWSTR zWideNew;
+

+
    zWideOld = rbuWinUtf8ToUnicode(zOld);
+
    if( zWideOld ){
+
      zWideNew = rbuWinUtf8ToUnicode(zNew);
+
      if( zWideNew ){
+
        if( MoveFileW(zWideOld, zWideNew) ){
+
          rc = SQLITE_OK;
+
        }else{
+
          rc = SQLITE_IOERR;
+
        }
+
        sqlite3_free(zWideNew);
+
      }else{
+
        rc = SQLITE_IOERR_NOMEM;
+
      }
+
      sqlite3_free(zWideOld);
+
    }else{
+
      rc = SQLITE_IOERR_NOMEM;
+
    }
+
  }
+
#else
+
  rc = rename(zOld, zNew) ? SQLITE_IOERR : SQLITE_OK;
+
#endif
+
  return rc;
+
}
+

+
SQLITE_API void sqlite3rbu_rename_handler(
+
  sqlite3rbu *pRbu,
+
  void *pArg,
+
  int (*xRename)(void *pArg, const char *zOld, const char *zNew)
+
){
+
  if( xRename ){
+
    pRbu->xRename = xRename;
+
    pRbu->pRenameArg = pArg;
+
  }else{
+
    pRbu->xRename = xDefaultRename;
+
    pRbu->pRenameArg = 0;
+
  }
+
}
+

/**************************************************************************
** Beginning of RBU VFS shim methods. The VFS shim modifies the behaviour
** of a standard VFS in the following ways:
@@ -210167,7 +214179,7 @@ static int dbpageBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo){
  ){
    pIdxInfo->orderByConsumed = 1;
  }
-
  sqlite3VtabWriteAll(pIdxInfo);
+
  sqlite3VtabUsesAllSchemas(pIdxInfo);
  return SQLITE_OK;
}

@@ -210285,12 +214297,18 @@ static int dbpageColumn(
    }
    case 1: {           /* data */
      DbPage *pDbPage = 0;
-
      rc = sqlite3PagerGet(pCsr->pPager, pCsr->pgno, (DbPage**)&pDbPage, 0);
-
      if( rc==SQLITE_OK ){
-
        sqlite3_result_blob(ctx, sqlite3PagerGetData(pDbPage), pCsr->szPage,
-
                            SQLITE_TRANSIENT);
+
      if( pCsr->pgno==((PENDING_BYTE/pCsr->szPage)+1) ){
+
        /* The pending byte page. Assume it is zeroed out. Attempting to
+
        ** request this page from the page is an SQLITE_CORRUPT error. */
+
        sqlite3_result_zeroblob(ctx, pCsr->szPage);
+
      }else{
+
        rc = sqlite3PagerGet(pCsr->pPager, pCsr->pgno, (DbPage**)&pDbPage, 0);
+
        if( rc==SQLITE_OK ){
+
          sqlite3_result_blob(ctx, sqlite3PagerGetData(pDbPage), pCsr->szPage,
+
              SQLITE_TRANSIENT);
+
        }
+
        sqlite3PagerUnref(pDbPage);
      }
-
      sqlite3PagerUnref(pDbPage);
      break;
    }
    default: {          /* schema */
@@ -210299,7 +214317,7 @@ static int dbpageColumn(
      break;
    }
  }
-
  return SQLITE_OK;
+
  return rc;
}

static int dbpageRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
@@ -210345,7 +214363,7 @@ static int dbpageUpdate(
    goto update_fail;
  }
  pBt = pTab->db->aDb[iDb].pBt;
-
  if( pgno<1 || pBt==0 || pgno>(int)sqlite3BtreeLastPage(pBt) ){
+
  if( pgno<1 || pBt==0 || pgno>sqlite3BtreeLastPage(pBt) ){
    zErr = "bad page number";
    goto update_fail;
  }
@@ -210359,11 +214377,12 @@ static int dbpageUpdate(
  pPager = sqlite3BtreePager(pBt);
  rc = sqlite3PagerGet(pPager, pgno, (DbPage**)&pDbPage, 0);
  if( rc==SQLITE_OK ){
-
    rc = sqlite3PagerWrite(pDbPage);
-
    if( rc==SQLITE_OK ){
-
      memcpy(sqlite3PagerGetData(pDbPage),
-
             sqlite3_value_blob(argv[3]),
-
             szPage);
+
    const void *pData = sqlite3_value_blob(argv[3]);
+
    assert( pData!=0 || pTab->db->mallocFailed );
+
    if( pData
+
     && (rc = sqlite3PagerWrite(pDbPage))==SQLITE_OK
+
    ){
+
      memcpy(sqlite3PagerGetData(pDbPage), pData, szPage);
    }
  }
  sqlite3PagerUnref(pDbPage);
@@ -210383,11 +214402,12 @@ static int dbpageBegin(sqlite3_vtab *pVtab){
  DbpageTable *pTab = (DbpageTable *)pVtab;
  sqlite3 *db = pTab->db;
  int i;
-
  for(i=0; i<db->nDb; i++){
+
  int rc = SQLITE_OK;
+
  for(i=0; rc==SQLITE_OK && i<db->nDb; i++){
    Btree *pBt = db->aDb[i].pBt;
-
    if( pBt ) sqlite3BtreeBeginTrans(pBt, 1, 0);
+
    if( pBt ) rc = sqlite3BtreeBeginTrans(pBt, 1, 0);
  }
-
  return SQLITE_OK;
+
  return rc;
}


@@ -217111,7 +221131,7 @@ static void sqlite3Fts5BufferAppendPrintf(int *, Fts5Buffer*, char *zFmt, ...);
static char *sqlite3Fts5Mprintf(int *pRc, const char *zFmt, ...);

#define fts5BufferZero(x)             sqlite3Fts5BufferZero(x)
-
#define fts5BufferAppendVarint(a,b,c) sqlite3Fts5BufferAppendVarint(a,b,c)
+
#define fts5BufferAppendVarint(a,b,c) sqlite3Fts5BufferAppendVarint(a,b,(i64)c)
#define fts5BufferFree(a)             sqlite3Fts5BufferFree(a)
#define fts5BufferAppendBlob(a,b,c,d) sqlite3Fts5BufferAppendBlob(a,b,c,d)
#define fts5BufferSet(a,b,c,d)        sqlite3Fts5BufferSet(a,b,c,d)
@@ -222895,6 +226915,9 @@ static Fts5ExprNearset *sqlite3Fts5ParseNearset(
  }else{
    if( pRet->nPhrase>0 ){
      Fts5ExprPhrase *pLast = pRet->apPhrase[pRet->nPhrase-1];
+
      assert( pParse!=0 );
+
      assert( pParse->apPhrase!=0 );
+
      assert( pParse->nPhrase>=2 );
      assert( pLast==pParse->apPhrase[pParse->nPhrase-2] );
      if( pPhrase->nTerm==0 ){
        fts5ExprPhraseFree(pPhrase);
@@ -225194,7 +229217,7 @@ struct Fts5Index {
  sqlite3_stmt *pWriter;          /* "INSERT ... %_data VALUES(?,?)" */
  sqlite3_stmt *pDeleter;         /* "DELETE FROM %_data ... id>=? AND id<=?" */
  sqlite3_stmt *pIdxWriter;       /* "INSERT ... %_idx VALUES(?,?,?,?)" */
-
  sqlite3_stmt *pIdxDeleter;      /* "DELETE FROM %_idx WHERE segid=? */
+
  sqlite3_stmt *pIdxDeleter;      /* "DELETE FROM %_idx WHERE segid=?" */
  sqlite3_stmt *pIdxSelect;
  int nRead;                      /* Total number of blocks read */

@@ -228985,7 +233008,9 @@ static void fts5WriteAppendRowid(
      fts5BufferAppendVarint(&p->rc, &pPage->buf, iRowid);
    }else{
      assert_nc( p->rc || iRowid>pWriter->iPrevRowid );
-
      fts5BufferAppendVarint(&p->rc, &pPage->buf, iRowid - pWriter->iPrevRowid);
+
      fts5BufferAppendVarint(&p->rc, &pPage->buf,
+
          (u64)iRowid - (u64)pWriter->iPrevRowid
+
      );
    }
    pWriter->iPrevRowid = iRowid;
    pWriter->bFirstRowidInDoclist = 0;
@@ -229749,7 +233774,7 @@ static int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge){

static void fts5AppendRowid(
  Fts5Index *p,
-
  i64 iDelta,
+
  u64 iDelta,
  Fts5Iter *pUnused,
  Fts5Buffer *pBuf
){
@@ -229759,7 +233784,7 @@ static void fts5AppendRowid(

static void fts5AppendPoslist(
  Fts5Index *p,
-
  i64 iDelta,
+
  u64 iDelta,
  Fts5Iter *pMulti,
  Fts5Buffer *pBuf
){
@@ -229834,10 +233859,10 @@ static void fts5MergeAppendDocid(
}
#endif

-
#define fts5MergeAppendDocid(pBuf, iLastRowid, iRowid) {       \
-
  assert( (pBuf)->n!=0 || (iLastRowid)==0 );                   \
-
  fts5BufferSafeAppendVarint((pBuf), (iRowid) - (iLastRowid)); \
-
  (iLastRowid) = (iRowid);                                     \
+
#define fts5MergeAppendDocid(pBuf, iLastRowid, iRowid) {                 \
+
  assert( (pBuf)->n!=0 || (iLastRowid)==0 );                             \
+
  fts5BufferSafeAppendVarint((pBuf), (u64)(iRowid) - (u64)(iLastRowid)); \
+
  (iLastRowid) = (iRowid);                                               \
}

/*
@@ -230108,7 +234133,7 @@ static void fts5SetupPrefixIter(
  int nMerge = 1;

  void (*xMerge)(Fts5Index*, Fts5Buffer*, int, Fts5Buffer*);
-
  void (*xAppend)(Fts5Index*, i64, Fts5Iter*, Fts5Buffer*);
+
  void (*xAppend)(Fts5Index*, u64, Fts5Iter*, Fts5Buffer*);
  if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){
    xMerge = fts5MergeRowidLists;
    xAppend = fts5AppendRowid;
@@ -230147,7 +234172,7 @@ static void fts5SetupPrefixIter(
        Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ];
        p1->xSetOutputs(p1, pSeg);
        if( p1->base.nData ){
-
          xAppend(p, p1->base.iRowid-iLastRowid, p1, &doclist);
+
          xAppend(p, (u64)p1->base.iRowid-(u64)iLastRowid, p1, &doclist);
          iLastRowid = p1->base.iRowid;
        }
      }
@@ -230195,7 +234220,7 @@ static void fts5SetupPrefixIter(
        iLastRowid = 0;
      }

-
      xAppend(p, p1->base.iRowid-iLastRowid, p1, &doclist);
+
      xAppend(p, (u64)p1->base.iRowid-(u64)iLastRowid, p1, &doclist);
      iLastRowid = p1->base.iRowid;
    }

@@ -231174,6 +235199,7 @@ static int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum, int bUseCksum

    /* If this is a new term, query for it. Update cksum3 with the results. */
    fts5TestTerm(p, &term, z, n, cksum2, &cksum3);
+
    if( p->rc ) break;

    if( eDetail==FTS5_DETAIL_NONE ){
      if( 0==fts5MultiIterIsEmpty(p, pIter) ){
@@ -234511,7 +238537,7 @@ static void fts5SourceIdFunc(
){
  assert( nArg==0 );
  UNUSED_PARAM2(nArg, apUnused);
-
  sqlite3_result_text(pCtx, "fts5: 2022-05-06 15:25:27 78d9c993d404cdfaa7fdd2973fa1052e3da9f66215cff9c5540ebe55c407d9fe", -1, SQLITE_TRANSIENT);
+
  sqlite3_result_text(pCtx, "fts5: 2022-11-16 12:10:08 89c459e766ea7e9165d0beeb124708b955a4950d0f4792f457465d71b158d318", -1, SQLITE_TRANSIENT);
}

/*
@@ -239182,6 +243208,16 @@ SQLITE_EXTENSION_INIT1

#ifndef SQLITE_OMIT_VIRTUALTABLE

+

+
#define STMT_NUM_INTEGER_COLUMN 10
+
typedef struct StmtRow StmtRow;
+
struct StmtRow {
+
  sqlite3_int64 iRowid;                /* Rowid value */
+
  char *zSql;                          /* column "sql" */
+
  int aCol[STMT_NUM_INTEGER_COLUMN+1]; /* all other column values */
+
  StmtRow *pNext;                      /* Next row to return */
+
};
+

/* stmt_vtab is a subclass of sqlite3_vtab which will
** serve as the underlying representation of a stmt virtual table
*/
@@ -239199,8 +243235,7 @@ typedef struct stmt_cursor stmt_cursor;
struct stmt_cursor {
  sqlite3_vtab_cursor base;  /* Base class - must be first */
  sqlite3 *db;               /* Database connection for this cursor */
-
  sqlite3_stmt *pStmt;       /* Statement cursor is currently pointing at */
-
  sqlite3_int64 iRowid;      /* The rowid */
+
  StmtRow *pRow;             /* Current row */
};

/*
@@ -239244,7 +243279,7 @@ static int stmtConnect(
     "CREATE TABLE x(sql,ncol,ro,busy,nscan,nsort,naidx,nstep,"
                    "reprep,run,mem)");
  if( rc==SQLITE_OK ){
-
    pNew = sqlite3_malloc( sizeof(*pNew) );
+
    pNew = sqlite3_malloc64( sizeof(*pNew) );
    *ppVtab = (sqlite3_vtab*)pNew;
    if( pNew==0 ) return SQLITE_NOMEM;
    memset(pNew, 0, sizeof(*pNew));
@@ -239266,7 +243301,7 @@ static int stmtDisconnect(sqlite3_vtab *pVtab){
*/
static int stmtOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  stmt_cursor *pCur;
-
  pCur = sqlite3_malloc( sizeof(*pCur) );
+
  pCur = sqlite3_malloc64( sizeof(*pCur) );
  if( pCur==0 ) return SQLITE_NOMEM;
  memset(pCur, 0, sizeof(*pCur));
  pCur->db = ((stmt_vtab*)p)->db;
@@ -239274,10 +243309,21 @@ static int stmtOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
  return SQLITE_OK;
}

+
static void stmtCsrReset(stmt_cursor *pCur){
+
  StmtRow *pRow = 0;
+
  StmtRow *pNext = 0;
+
  for(pRow=pCur->pRow; pRow; pRow=pNext){
+
    pNext = pRow->pNext;
+
    sqlite3_free(pRow);
+
  }
+
  pCur->pRow = 0;
+
}
+

/*
** Destructor for a stmt_cursor.
*/
static int stmtClose(sqlite3_vtab_cursor *cur){
+
  stmtCsrReset((stmt_cursor*)cur);
  sqlite3_free(cur);
  return SQLITE_OK;
}
@@ -239288,8 +243334,9 @@ static int stmtClose(sqlite3_vtab_cursor *cur){
*/
static int stmtNext(sqlite3_vtab_cursor *cur){
  stmt_cursor *pCur = (stmt_cursor*)cur;
-
  pCur->iRowid++;
-
  pCur->pStmt = sqlite3_next_stmt(pCur->db, pCur->pStmt);
+
  StmtRow *pNext = pCur->pRow->pNext;
+
  sqlite3_free(pCur->pRow);
+
  pCur->pRow = pNext;
  return SQLITE_OK;
}

@@ -239303,39 +243350,11 @@ static int stmtColumn(
  int i                       /* Which column to return */
){
  stmt_cursor *pCur = (stmt_cursor*)cur;
-
  switch( i ){
-
    case STMT_COLUMN_SQL: {
-
      sqlite3_result_text(ctx, sqlite3_sql(pCur->pStmt), -1, SQLITE_TRANSIENT);
-
      break;
-
    }
-
    case STMT_COLUMN_NCOL: {
-
      sqlite3_result_int(ctx, sqlite3_column_count(pCur->pStmt));
-
      break;
-
    }
-
    case STMT_COLUMN_RO: {
-
      sqlite3_result_int(ctx, sqlite3_stmt_readonly(pCur->pStmt));
-
      break;
-
    }
-
    case STMT_COLUMN_BUSY: {
-
      sqlite3_result_int(ctx, sqlite3_stmt_busy(pCur->pStmt));
-
      break;
-
    }
-
    default: {
-
      assert( i==STMT_COLUMN_MEM );
-
      i = SQLITE_STMTSTATUS_MEMUSED +
-
            STMT_COLUMN_NSCAN - SQLITE_STMTSTATUS_FULLSCAN_STEP;
-
      /* Fall thru */
-
    }
-
    case STMT_COLUMN_NSCAN:
-
    case STMT_COLUMN_NSORT:
-
    case STMT_COLUMN_NAIDX:
-
    case STMT_COLUMN_NSTEP:
-
    case STMT_COLUMN_REPREP:
-
    case STMT_COLUMN_RUN: {
-
      sqlite3_result_int(ctx, sqlite3_stmt_status(pCur->pStmt,
-
                      i-STMT_COLUMN_NSCAN+SQLITE_STMTSTATUS_FULLSCAN_STEP, 0));
-
      break;
-
    }
+
  StmtRow *pRow = pCur->pRow;
+
  if( i==STMT_COLUMN_SQL ){
+
    sqlite3_result_text(ctx, pRow->zSql, -1, SQLITE_TRANSIENT);
+
  }else{
+
    sqlite3_result_int(ctx, pRow->aCol[i]);
  }
  return SQLITE_OK;
}
@@ -239346,7 +243365,7 @@ static int stmtColumn(
*/
static int stmtRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
  stmt_cursor *pCur = (stmt_cursor*)cur;
-
  *pRowid = pCur->iRowid;
+
  *pRowid = pCur->pRow->iRowid;
  return SQLITE_OK;
}

@@ -239356,7 +243375,7 @@ static int stmtRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
*/
static int stmtEof(sqlite3_vtab_cursor *cur){
  stmt_cursor *pCur = (stmt_cursor*)cur;
-
  return pCur->pStmt==0;
+
  return pCur->pRow==0;
}

/*
@@ -239371,9 +243390,53 @@ static int stmtFilter(
  int argc, sqlite3_value **argv
){
  stmt_cursor *pCur = (stmt_cursor *)pVtabCursor;
-
  pCur->pStmt = 0;
-
  pCur->iRowid = 0;
-
  return stmtNext(pVtabCursor);
+
  sqlite3_stmt *p = 0;
+
  sqlite3_int64 iRowid = 1;
+
  StmtRow **ppRow = 0;
+

+
  stmtCsrReset(pCur);
+
  ppRow = &pCur->pRow;
+
  for(p=sqlite3_next_stmt(pCur->db, 0); p; p=sqlite3_next_stmt(pCur->db, p)){
+
    const char *zSql = sqlite3_sql(p);
+
    sqlite3_int64 nSql = zSql ? strlen(zSql)+1 : 0;
+
    StmtRow *pNew = (StmtRow*)sqlite3_malloc64(sizeof(StmtRow) + nSql);
+

+
    if( pNew==0 ) return SQLITE_NOMEM;
+
    memset(pNew, 0, sizeof(StmtRow));
+
    if( zSql ){
+
      pNew->zSql = (char*)&pNew[1];
+
      memcpy(pNew->zSql, zSql, nSql);
+
    }
+
    pNew->aCol[STMT_COLUMN_NCOL] = sqlite3_column_count(p);
+
    pNew->aCol[STMT_COLUMN_RO] = sqlite3_stmt_readonly(p);
+
    pNew->aCol[STMT_COLUMN_BUSY] = sqlite3_stmt_busy(p);
+
    pNew->aCol[STMT_COLUMN_NSCAN] = sqlite3_stmt_status(
+
        p, SQLITE_STMTSTATUS_FULLSCAN_STEP, 0
+
    );
+
    pNew->aCol[STMT_COLUMN_NSORT] = sqlite3_stmt_status(
+
        p, SQLITE_STMTSTATUS_SORT, 0
+
    );
+
    pNew->aCol[STMT_COLUMN_NAIDX] = sqlite3_stmt_status(
+
        p, SQLITE_STMTSTATUS_AUTOINDEX, 0
+
    );
+
    pNew->aCol[STMT_COLUMN_NSTEP] = sqlite3_stmt_status(
+
        p, SQLITE_STMTSTATUS_VM_STEP, 0
+
    );
+
    pNew->aCol[STMT_COLUMN_REPREP] = sqlite3_stmt_status(
+
        p, SQLITE_STMTSTATUS_REPREPARE, 0
+
    );
+
    pNew->aCol[STMT_COLUMN_RUN] = sqlite3_stmt_status(
+
        p, SQLITE_STMTSTATUS_RUN, 0
+
    );
+
    pNew->aCol[STMT_COLUMN_MEM] = sqlite3_stmt_status(
+
        p, SQLITE_STMTSTATUS_MEMUSED, 0
+
    );
+
    pNew->iRowid = iRowid++;
+
    *ppRow = pNew;
+
    ppRow = &pNew->pNext;
+
  }
+

+
  return SQLITE_OK;
}

/*
modified external/sqlite/sqlite3.h
@@ -146,9 +146,9 @@ extern "C" {
** [sqlite3_libversion_number()], [sqlite3_sourceid()],
** [sqlite_version()] and [sqlite_source_id()].
*/
-
#define SQLITE_VERSION        "3.38.5"
-
#define SQLITE_VERSION_NUMBER 3038005
-
#define SQLITE_SOURCE_ID      "2022-05-06 15:25:27 78d9c993d404cdfaa7fdd2973fa1052e3da9f66215cff9c5540ebe55c407d9fe"
+
#define SQLITE_VERSION        "3.40.0"
+
#define SQLITE_VERSION_NUMBER 3040000
+
#define SQLITE_SOURCE_ID      "2022-11-16 12:10:08 89c459e766ea7e9165d0beeb124708b955a4950d0f4792f457465d71b158d318"

/*
** CAPI3REF: Run-Time Library Version Numbers
@@ -670,13 +670,17 @@ SQLITE_API int sqlite3_exec(
**
** SQLite uses one of these integer values as the second
** argument to calls it makes to the xLock() and xUnlock() methods
-
** of an [sqlite3_io_methods] object.
+
** of an [sqlite3_io_methods] object.  These values are ordered from
+
** lest restrictive to most restrictive.
+
**
+
** The argument to xLock() is always SHARED or higher.  The argument to
+
** xUnlock is either SHARED or NONE.
*/
-
#define SQLITE_LOCK_NONE          0
-
#define SQLITE_LOCK_SHARED        1
-
#define SQLITE_LOCK_RESERVED      2
-
#define SQLITE_LOCK_PENDING       3
-
#define SQLITE_LOCK_EXCLUSIVE     4
+
#define SQLITE_LOCK_NONE          0       /* xUnlock() only */
+
#define SQLITE_LOCK_SHARED        1       /* xLock() or xUnlock() */
+
#define SQLITE_LOCK_RESERVED      2       /* xLock() only */
+
#define SQLITE_LOCK_PENDING       3       /* xLock() only */
+
#define SQLITE_LOCK_EXCLUSIVE     4       /* xLock() only */

/*
** CAPI3REF: Synchronization Type Flags
@@ -754,7 +758,14 @@ struct sqlite3_file {
** <li> [SQLITE_LOCK_PENDING], or
** <li> [SQLITE_LOCK_EXCLUSIVE].
** </ul>
-
** xLock() increases the lock. xUnlock() decreases the lock.
+
** xLock() upgrades the database file lock.  In other words, xLock() moves the
+
** database file lock in the direction NONE toward EXCLUSIVE. The argument to
+
** xLock() is always on of SHARED, RESERVED, PENDING, or EXCLUSIVE, never
+
** SQLITE_LOCK_NONE.  If the database file lock is already at or above the
+
** requested lock, then the call to xLock() is a no-op.
+
** xUnlock() downgrades the database file lock to either SHARED or NONE.
+
*  If the lock is already at or below the requested lock state, then the call
+
** to xUnlock() is a no-op.
** The xCheckReservedLock() method checks whether any database connection,
** either in this process or in some other process, is holding a RESERVED,
** PENDING, or EXCLUSIVE lock on the file.  It returns true
@@ -859,9 +870,8 @@ struct sqlite3_io_methods {
** opcode causes the xFileControl method to write the current state of
** the lock (one of [SQLITE_LOCK_NONE], [SQLITE_LOCK_SHARED],
** [SQLITE_LOCK_RESERVED], [SQLITE_LOCK_PENDING], or [SQLITE_LOCK_EXCLUSIVE])
-
** into an integer that the pArg argument points to. This capability
-
** is used during testing and is only available when the SQLITE_TEST
-
** compile-time option is used.
+
** into an integer that the pArg argument points to.
+
** This capability is only available if SQLite is compiled with [SQLITE_DEBUG].
**
** <li>[[SQLITE_FCNTL_SIZE_HINT]]
** The [SQLITE_FCNTL_SIZE_HINT] opcode is used by SQLite to give the VFS
@@ -1254,6 +1264,26 @@ typedef struct sqlite3_mutex sqlite3_mutex;
typedef struct sqlite3_api_routines sqlite3_api_routines;

/*
+
** CAPI3REF: File Name
+
**
+
** Type [sqlite3_filename] is used by SQLite to pass filenames to the
+
** xOpen method of a [VFS]. It may be cast to (const char*) and treated
+
** as a normal, nul-terminated, UTF-8 buffer containing the filename, but
+
** may also be passed to special APIs such as:
+
**
+
** <ul>
+
** <li>  sqlite3_filename_database()
+
** <li>  sqlite3_filename_journal()
+
** <li>  sqlite3_filename_wal()
+
** <li>  sqlite3_uri_parameter()
+
** <li>  sqlite3_uri_boolean()
+
** <li>  sqlite3_uri_int64()
+
** <li>  sqlite3_uri_key()
+
** </ul>
+
*/
+
typedef const char *sqlite3_filename;
+

+
/*
** CAPI3REF: OS Interface Object
**
** An instance of the sqlite3_vfs object defines the interface between
@@ -1431,7 +1461,7 @@ struct sqlite3_vfs {
  sqlite3_vfs *pNext;      /* Next registered VFS */
  const char *zName;       /* Name of this virtual file system */
  void *pAppData;          /* Pointer to application-specific data */
-
  int (*xOpen)(sqlite3_vfs*, const char *zName, sqlite3_file*,
+
  int (*xOpen)(sqlite3_vfs*, sqlite3_filename zName, sqlite3_file*,
               int flags, int *pOutFlags);
  int (*xDelete)(sqlite3_vfs*, const char *zName, int syncDir);
  int (*xAccess)(sqlite3_vfs*, const char *zName, int flags, int *pResOut);
@@ -2309,6 +2339,7 @@ struct sqlite3_mem_methods {
** <ul>
** <li> The [PRAGMA writable_schema=ON] statement.
** <li> The [PRAGMA journal_mode=OFF] statement.
+
** <li> The [PRAGMA schema_version=N] statement.
** <li> Writes to the [sqlite_dbpage] virtual table.
** <li> Direct writes to [shadow tables].
** </ul>
@@ -3424,6 +3455,9 @@ SQLITE_API void sqlite3_progress_handler(sqlite3*, int, int(*)(void*), void*);
** <dd>The database is opened [shared cache] enabled, overriding
** the default shared cache setting provided by
** [sqlite3_enable_shared_cache()].)^
+
** The [use of shared cache mode is discouraged] and hence shared cache
+
** capabilities may be omitted from many builds of SQLite.  In such cases,
+
** this option is a no-op.
**
** ^(<dt>[SQLITE_OPEN_PRIVATECACHE]</dt>
** <dd>The database is opened [shared cache] disabled, overriding
@@ -3439,7 +3473,7 @@ SQLITE_API void sqlite3_progress_handler(sqlite3*, int, int(*)(void*), void*);
** to return an extended result code.</dd>
**
** [[OPEN_NOFOLLOW]] ^(<dt>[SQLITE_OPEN_NOFOLLOW]</dt>
-
** <dd>The database filename is not allowed to be a symbolic link</dd>
+
** <dd>The database filename is not allowed to contain a symbolic link</dd>
** </dl>)^
**
** If the 3rd parameter to sqlite3_open_v2() is not one of the
@@ -3698,10 +3732,10 @@ SQLITE_API int sqlite3_open_v2(
**
** See the [URI filename] documentation for additional information.
*/
-
SQLITE_API const char *sqlite3_uri_parameter(const char *zFilename, const char *zParam);
-
SQLITE_API int sqlite3_uri_boolean(const char *zFile, const char *zParam, int bDefault);
-
SQLITE_API sqlite3_int64 sqlite3_uri_int64(const char*, const char*, sqlite3_int64);
-
SQLITE_API const char *sqlite3_uri_key(const char *zFilename, int N);
+
SQLITE_API const char *sqlite3_uri_parameter(sqlite3_filename z, const char *zParam);
+
SQLITE_API int sqlite3_uri_boolean(sqlite3_filename z, const char *zParam, int bDefault);
+
SQLITE_API sqlite3_int64 sqlite3_uri_int64(sqlite3_filename, const char*, sqlite3_int64);
+
SQLITE_API const char *sqlite3_uri_key(sqlite3_filename z, int N);

/*
** CAPI3REF:  Translate filenames
@@ -3730,9 +3764,9 @@ SQLITE_API const char *sqlite3_uri_key(const char *zFilename, int N);
** return value from [sqlite3_db_filename()], then the result is
** undefined and is likely a memory access violation.
*/
-
SQLITE_API const char *sqlite3_filename_database(const char*);
-
SQLITE_API const char *sqlite3_filename_journal(const char*);
-
SQLITE_API const char *sqlite3_filename_wal(const char*);
+
SQLITE_API const char *sqlite3_filename_database(sqlite3_filename);
+
SQLITE_API const char *sqlite3_filename_journal(sqlite3_filename);
+
SQLITE_API const char *sqlite3_filename_wal(sqlite3_filename);

/*
** CAPI3REF:  Database File Corresponding To A Journal
@@ -3798,14 +3832,14 @@ SQLITE_API sqlite3_file *sqlite3_database_file_object(const char*);
** then the corresponding [sqlite3_module.xClose() method should also be
** invoked prior to calling sqlite3_free_filename(Y).
*/
-
SQLITE_API char *sqlite3_create_filename(
+
SQLITE_API sqlite3_filename sqlite3_create_filename(
  const char *zDatabase,
  const char *zJournal,
  const char *zWal,
  int nParam,
  const char **azParam
);
-
SQLITE_API void sqlite3_free_filename(char*);
+
SQLITE_API void sqlite3_free_filename(sqlite3_filename);

/*
** CAPI3REF: Error Codes And Messages
@@ -5508,6 +5542,16 @@ SQLITE_API SQLITE_DEPRECATED int sqlite3_memory_alarm(void(*)(void*,sqlite3_int6
** then the conversion is performed.  Otherwise no conversion occurs.
** The [SQLITE_INTEGER | datatype] after conversion is returned.)^
**
+
** ^(The sqlite3_value_encoding(X) interface returns one of [SQLITE_UTF8],
+
** [SQLITE_UTF16BE], or [SQLITE_UTF16LE] according to the current encoding
+
** of the value X, assuming that X has type TEXT.)^  If sqlite3_value_type(X)
+
** returns something other than SQLITE_TEXT, then the return value from
+
** sqlite3_value_encoding(X) is meaningless.  ^Calls to
+
** sqlite3_value_text(X), sqlite3_value_text16(X), sqlite3_value_text16be(X),
+
** sqlite3_value_text16le(X), sqlite3_value_bytes(X), or
+
** sqlite3_value_bytes16(X) might change the encoding of the value X and
+
** thus change the return from subsequent calls to sqlite3_value_encoding(X).
+
**
** ^Within the [xUpdate] method of a [virtual table], the
** sqlite3_value_nochange(X) interface returns true if and only if
** the column corresponding to X is unchanged by the UPDATE operation
@@ -5572,6 +5616,7 @@ SQLITE_API int sqlite3_value_type(sqlite3_value*);
SQLITE_API int sqlite3_value_numeric_type(sqlite3_value*);
SQLITE_API int sqlite3_value_nochange(sqlite3_value*);
SQLITE_API int sqlite3_value_frombind(sqlite3_value*);
+
SQLITE_API int sqlite3_value_encoding(sqlite3_value*);

/*
** CAPI3REF: Finding The Subtype Of SQL Values
@@ -5593,7 +5638,8 @@ SQLITE_API unsigned int sqlite3_value_subtype(sqlite3_value*);
** object D and returns a pointer to that copy.  ^The [sqlite3_value] returned
** is a [protected sqlite3_value] object even if the input is not.
** ^The sqlite3_value_dup(V) interface returns NULL if V is NULL or if a
-
** memory allocation fails.
+
** memory allocation fails. ^If V is a [pointer value], then the result
+
** of sqlite3_value_dup(V) is a NULL value.
**
** ^The sqlite3_value_free(V) interface frees an [sqlite3_value] object
** previously obtained from [sqlite3_value_dup()].  ^If V is a NULL pointer
@@ -5624,7 +5670,7 @@ SQLITE_API void sqlite3_value_free(sqlite3_value*);
**
** ^The sqlite3_aggregate_context(C,N) routine returns a NULL pointer
** when first called if N is less than or equal to zero or if a memory
-
** allocate error occurs.
+
** allocation error occurs.
**
** ^(The amount of space allocated by sqlite3_aggregate_context(C,N) is
** determined by the N parameter on first successful call.  Changing the
@@ -5829,9 +5875,10 @@ typedef void (*sqlite3_destructor_type)(void*);
** of [SQLITE_UTF8], [SQLITE_UTF16], [SQLITE_UTF16BE], or [SQLITE_UTF16LE].
** ^SQLite takes the text result from the application from
** the 2nd parameter of the sqlite3_result_text* interfaces.
-
** ^If the 3rd parameter to the sqlite3_result_text* interfaces
-
** is negative, then SQLite takes result text from the 2nd parameter
-
** through the first zero character.
+
** ^If the 3rd parameter to any of the sqlite3_result_text* interfaces
+
** other than sqlite3_result_text64() is negative, then SQLite computes
+
** the string length itself by searching the 2nd parameter for the first
+
** zero character.
** ^If the 3rd parameter to the sqlite3_result_text* interfaces
** is non-negative, then as many bytes (not characters) of the text
** pointed to by the 2nd parameter are taken as the application-defined
@@ -6276,6 +6323,28 @@ SQLITE_API int sqlite3_get_autocommit(sqlite3*);
SQLITE_API sqlite3 *sqlite3_db_handle(sqlite3_stmt*);

/*
+
** CAPI3REF: Return The Schema Name For A Database Connection
+
** METHOD: sqlite3
+
**
+
** ^The sqlite3_db_name(D,N) interface returns a pointer to the schema name
+
** for the N-th database on database connection D, or a NULL pointer of N is
+
** out of range.  An N value of 0 means the main database file.  An N of 1 is
+
** the "temp" schema.  Larger values of N correspond to various ATTACH-ed
+
** databases.
+
**
+
** Space to hold the string that is returned by sqlite3_db_name() is managed
+
** by SQLite itself.  The string might be deallocated by any operation that
+
** changes the schema, including [ATTACH] or [DETACH] or calls to
+
** [sqlite3_serialize()] or [sqlite3_deserialize()], even operations that
+
** occur on a different thread.  Applications that need to
+
** remember the string long-term should make their own copy.  Applications that
+
** are accessing the same database connection simultaneously on multiple
+
** threads should mutex-protect calls to this API and should make their own
+
** private copy of the result prior to releasing the mutex.
+
*/
+
SQLITE_API const char *sqlite3_db_name(sqlite3 *db, int N);
+

+
/*
** CAPI3REF: Return The Filename For A Database Connection
** METHOD: sqlite3
**
@@ -6305,7 +6374,7 @@ SQLITE_API sqlite3 *sqlite3_db_handle(sqlite3_stmt*);
** <li> [sqlite3_filename_wal()]
** </ul>
*/
-
SQLITE_API const char *sqlite3_db_filename(sqlite3 *db, const char *zDbName);
+
SQLITE_API sqlite3_filename sqlite3_db_filename(sqlite3 *db, const char *zDbName);

/*
** CAPI3REF: Determine if a database is read-only
@@ -6442,7 +6511,7 @@ SQLITE_API void *sqlite3_rollback_hook(sqlite3*, void(*)(void *), void*);
** function C that is invoked prior to each autovacuum of the database
** file.  ^The callback is passed a copy of the generic data pointer (P),
** the schema-name of the attached database that is being autovacuumed,
-
** the the size of the database file in pages, the number of free pages,
+
** the size of the database file in pages, the number of free pages,
** and the number of bytes per page, respectively.  The callback should
** return the number of free pages that should be removed by the
** autovacuum.  ^If the callback returns zero, then no autovacuum happens.
@@ -6563,6 +6632,11 @@ SQLITE_API void *sqlite3_update_hook(
** to the same database. Sharing is enabled if the argument is true
** and disabled if the argument is false.)^
**
+
** This interface is omitted if SQLite is compiled with
+
** [-DSQLITE_OMIT_SHARED_CACHE].  The [-DSQLITE_OMIT_SHARED_CACHE]
+
** compile-time option is recommended because the
+
** [use of shared cache mode is discouraged].
+
**
** ^Cache sharing is enabled and disabled for an entire process.
** This is a change as of SQLite [version 3.5.0] ([dateof:3.5.0]).
** In prior versions of SQLite,
@@ -6661,7 +6735,7 @@ SQLITE_API int sqlite3_db_release_memory(sqlite3*);
** ^The soft heap limit may not be greater than the hard heap limit.
** ^If the hard heap limit is enabled and if sqlite3_soft_heap_limit(N)
** is invoked with a value of N that is greater than the hard heap limit,
-
** the the soft heap limit is set to the value of the hard heap limit.
+
** the soft heap limit is set to the value of the hard heap limit.
** ^The soft heap limit is automatically enabled whenever the hard heap
** limit is enabled. ^When sqlite3_hard_heap_limit64(N) is invoked and
** the soft heap limit is outside the range of 1..N, then the soft heap
@@ -8956,7 +9030,7 @@ typedef struct sqlite3_backup sqlite3_backup;
** if the application incorrectly accesses the destination [database connection]
** and so no error code is reported, but the operations may malfunction
** nevertheless.  Use of the destination database connection while a
-
** backup is in progress might also also cause a mutex deadlock.
+
** backup is in progress might also cause a mutex deadlock.
**
** If running in [shared cache mode], the application must
** guarantee that the shared cache used by the destination database
@@ -9384,7 +9458,7 @@ SQLITE_API int sqlite3_wal_checkpoint_v2(
*/
#define SQLITE_CHECKPOINT_PASSIVE  0  /* Do as much as possible w/o blocking */
#define SQLITE_CHECKPOINT_FULL     1  /* Wait for writers, then checkpoint */
-
#define SQLITE_CHECKPOINT_RESTART  2  /* Like FULL but wait for for readers */
+
#define SQLITE_CHECKPOINT_RESTART  2  /* Like FULL but wait for readers */
#define SQLITE_CHECKPOINT_TRUNCATE 3  /* Like RESTART but also truncate WAL */

/*
@@ -9554,8 +9628,8 @@ SQLITE_API SQLITE_EXPERIMENTAL const char *sqlite3_vtab_collation(sqlite3_index_
** of a [virtual table] implementation. The result of calling this
** interface from outside of xBestIndex() is undefined and probably harmful.
**
-
** ^The sqlite3_vtab_distinct() interface returns an integer that is
-
** either 0, 1, or 2.  The integer returned by sqlite3_vtab_distinct()
+
** ^The sqlite3_vtab_distinct() interface returns an integer between 0 and
+
** 3.  The integer returned by sqlite3_vtab_distinct()
** gives the virtual table additional information about how the query
** planner wants the output to be ordered. As long as the virtual table
** can meet the ordering requirements of the query planner, it may set
@@ -9587,6 +9661,13 @@ SQLITE_API SQLITE_EXPERIMENTAL const char *sqlite3_vtab_collation(sqlite3_index_
** that have the same value for all columns identified by "aOrderBy".
** ^However omitting the extra rows is optional.
** This mode is used for a DISTINCT query.
+
** <li value="3"><p>
+
** ^(If the sqlite3_vtab_distinct() interface returns 3, that means
+
** that the query planner needs only distinct rows but it does need the
+
** rows to be sorted.)^ ^The virtual table implementation is free to omit
+
** rows that are identical in all aOrderBy columns, if it wants to, but
+
** it is not required to omit any rows.  This mode is used for queries
+
** that have both DISTINCT and ORDER BY clauses.
** </ol>
**
** ^For the purposes of comparing virtual table output values to see if the
modified libpkg/Makefile.autosetup
@@ -1,7 +1,6 @@
include @builddir@/mk/defs.mk
LIB=	pkg
-
SRCS=	backup.c \
-
	backup_lib.c \
+
SRCS=	backup_lib.c \
	merge3.c \
	pkg_audit.c \
	pkg_deps.c \
modified libpkg/fetch_libfetch.c
@@ -1,5 +1,5 @@
/*-
-
 * Copyright (c) 2020 Baptiste Daroussin <bapt@FreeBSD.org>
+
 * Copyright (c) 2020-2022 Baptiste Daroussin <bapt@FreeBSD.org>
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
@@ -105,7 +105,6 @@ fetch_connect(struct pkg_repo *repo, struct url *u)
	max_retry = pkg_object_int(pkg_config_get("FETCH_RETRY"));
	fetch_timeout = pkg_object_int(pkg_config_get("FETCH_TIMEOUT"));

-
	fetchConnectionCacheInit(-1, -1);
	fetchTimeout = (int)MIN(fetch_timeout, INT_MAX);

	repourl = fetchParseURL(repo->url);
@@ -209,8 +208,6 @@ fetch_connect(struct pkg_repo *repo, struct url *u)
				http_current = repo->http->next;
				if (http_current == NULL)
					http_current = repo->http;
-
			} else {
-
				sleep(1);
			}
		}
	}
modified libpkg/libpkg.ver
@@ -178,12 +178,10 @@ global:
	pkgdb_compact;
	pkgdb_delete_annotation;
	pkgdb_downgrade_lock;
-
	pkgdb_dump;
	pkgdb_it_count;
	pkgdb_it_free;
	pkgdb_it_next;
	pkgdb_it_reset;
-
	pkgdb_load;
	pkgdb_modify_annotation;
	pkgdb_obtain_lock;
	pkgdb_open;
modified libpkg/lua.c
@@ -1,5 +1,5 @@
/*-
-
 * Copyright (c) 2019-2021 Baptiste Daroussin <bapt@FreeBSD.org>
+
 * Copyright (c) 2019-2022 Baptiste Daroussin <bapt@FreeBSD.org>
 * All rights reserved.
 * 
 * Redistribution and use in source and binary forms, with or without
@@ -328,6 +328,21 @@ lua_pkg_filecmp(lua_State *L)
}

int
+
lua_pkg_symlink(lua_State *L)
+
{
+
	int n = lua_gettop(L);
+
	luaL_argcheck(L, n == 2, n > 2 ? 3 : n,
+
	    "pkg.symlink takes exactly two arguments");
+
	const char *from = luaL_checkstring(L, 1);
+
	const char *to = luaL_checkstring(L, 2);
+
	lua_getglobal(L, "rootfd");
+
	int rootfd = lua_tointeger(L, -1);
+
	if (symlinkat(from, rootfd, RELATIVE_PATH(to)) == -1)
+
		return (luaL_fileresult(L, 0, from));
+
	return (1);
+
}
+

+
int
lua_prefix_path(lua_State *L)
{
	int n = lua_gettop(L);
modified libpkg/lua_scripts.c
@@ -87,6 +87,7 @@ pkg_lua_script_run(struct pkg * const pkg, pkg_lua_script type, bool upgrade)
				{ "stat", lua_stat },
				{ "readdir", lua_readdir },
				{ "exec", lua_exec },
+
				{ "symlink", lua_pkg_symlink },
				{ NULL, NULL },
			};
			close(cur_pipe[0]);
modified libpkg/pkg.c
@@ -703,9 +703,9 @@ pkg_addluascript_fileat(int fd, struct pkg *pkg, const char *filename)
		type = PKG_LUA_PRE_INSTALL;
	} else if (strcmp(filename, "pkg-post-install.lua") == 0) {
		type = PKG_LUA_POST_INSTALL;
-
	} else if (strcmp(filename, "pkg-pre-deinstall") == 0) {
+
	} else if (strcmp(filename, "pkg-pre-deinstall.lua") == 0) {
		type = PKG_LUA_PRE_DEINSTALL;
-
	} else if (strcmp(filename, "pkg-post-deinstall") == 0) {
+
	} else if (strcmp(filename, "pkg-post-deinstall.lua") == 0) {
		type = PKG_LUA_POST_DEINSTALL;
	} else {
		pkg_emit_error("unknown lua script '%s'", filename);
modified libpkg/pkg.h.in
@@ -763,6 +763,9 @@ int pkgdb_set2(struct pkgdb *db, struct pkg *pkg, ...);
int64_t pkg_set_debug_level(int64_t debug_level);
int pkg_set_rootdir(const char *rootdir);

+
int pkg_open_devnull(void);
+
void pkg_close_devnull(void);
+

/**
 * Allocate a new struct pkg and add it to the deps of pkg.
 * @return An error code.
@@ -928,15 +931,6 @@ void pkgdb_close(struct pkgdb *db);
int pkgdb_remote_init(struct pkgdb *db, const char *reponame);

/**
-
 * Dump to or load from a backup copy of the main database file
-
 * (local.sqlite)
-
 */
-

-
int pkgdb_dump(struct pkgdb *db, const char *dest);
-
int pkgdb_load(struct pkgdb *db, const char *src);
-

-

-
/**
 * Set the case sensitivity flag on or off.  Defaults to
 * true (case_sensitive)
 */
modified libpkg/pkg_add.c
@@ -1,5 +1,5 @@
/*-
-
 * Copyright (c) 2011-2020 Baptiste Daroussin <bapt@FreeBSD.org>
+
 * Copyright (c) 2011-2022 Baptiste Daroussin <bapt@FreeBSD.org>
 * Copyright (c) 2011-2012 Julien Laffaye <jlaffaye@FreeBSD.org>
 * Copyright (c) 2016, Vsevolod Stakhov
 *
@@ -53,13 +53,6 @@
#include "private/pkg.h"
#include "private/pkgdb.h"

-
struct store_hardlinks {
-
	ino_t ino;
-
	dev_t dev;
-
	const char *path;
-
};
-
typedef tll(struct store_hardlinks *) hls;
-

#if defined(UF_NOUNLINK)
#define NOCHANGESFLAGS	(UF_IMMUTABLE | UF_APPEND | UF_NOUNLINK | SF_IMMUTABLE | SF_APPEND | SF_NOUNLINK)
#else
@@ -79,11 +72,6 @@ attempt_to_merge(int rootfd, struct pkg_config_file *rcf, struct pkg *local,
	off_t sz;
	char *localsum;

-
	if (rcf == NULL) {
-
		pkg_debug(3, "No remote config file");
-
		return;
-
	}
-

	if (local == NULL) {
		pkg_debug(3, "No local package");
		if (fstatat(rootfd, RELATIVE_PATH(rcf->path), &st, 0) == 0) {
@@ -191,6 +179,20 @@ out:
	return (grent.gr_gid);
}

+
static int
+
set_chflags(int fd, const char *path, u_long fflags)
+
{
+
#ifdef HAVE_CHFLAGSAT
+
	if (getenv("INSTALL_AS_USER"))
+
		return (EPKG_OK);
+
	if (fflags == 0)
+
		return (EPKG_OK);
+
	if (chflagsat(fd, RELATIVE_PATH(path), fflags, AT_SYMLINK_NOFOLLOW) == -1) {
+
		pkg_fatal_errno("Fail to chflags %s", path);
+
	}
+
#endif
+
	return (EPKG_OK);
+
}
int
set_attrsat(int fd, const char *path, mode_t perm, uid_t uid, gid_t gid,
    const struct timespec *ats, const struct timespec *mts)
@@ -311,23 +313,79 @@ fill_timespec_buf(const struct stat *aest, struct timespec tspec[2])
#endif
}

+
static void
+
reopen_tempdir(int rootfd, struct tempdir *t)
+
{
+
	if (t->fd != -1)
+
		return;
+
	t->fd = openat(rootfd, RELATIVE_PATH(t->temp), O_DIRECTORY|O_CLOEXEC);
+
}
+

+
static struct tempdir *
+
get_tempdir(int rootfd, const char *path, tempdirs_t *tempdirs)
+
{
+
	struct tempdir *tmpdir = NULL;
+

+
	tll_foreach(*tempdirs, t) {
+
		if (strncmp(t->item->name, path, t->item->len) == 0 && path[t->item->len] == '/') {
+
			reopen_tempdir(rootfd, t->item);
+
			return (t->item);
+
		}
+
	}
+

+
	tmpdir = open_tempdir(rootfd, path);
+
	if (tmpdir != NULL)
+
		tll_push_back(*tempdirs, tmpdir);
+

+
	return (tmpdir);
+
}
+

+
static void
+
close_tempdir(struct tempdir *t)
+
{
+
	if (t == NULL)
+
		return;
+
	if (t->fd != -1)
+
		close(t->fd);
+
	t->fd = -1;
+
}
+

static int
-
create_dir(struct pkg *pkg, struct pkg_dir *d)
+
create_dir(struct pkg *pkg, struct pkg_dir *d, tempdirs_t *tempdirs)
{
	struct stat st;
+
	struct tempdir *tmpdir = NULL;
+
	int fd;
+
	const char *path;

-
	if (mkdirat(pkg->rootfd, RELATIVE_PATH(d->path), 0755) == -1)
-
		if (!mkdirat_p(pkg->rootfd, RELATIVE_PATH(d->path)))
+
	tmpdir = get_tempdir(pkg->rootfd, d->path, tempdirs);
+
	if (tmpdir == NULL) {
+
		fd = pkg->rootfd;
+
		path = d->path;
+
	} else {
+
		fd = tmpdir->fd;
+
		path = d->path + tmpdir->len;
+
	}
+

+
	if (mkdirat(fd, RELATIVE_PATH(path), 0755) == -1)
+
		if (!mkdirat_p(fd, RELATIVE_PATH(path))) {
+
			close_tempdir(tmpdir);
			return (EPKG_FATAL);
-
	if (fstatat(pkg->rootfd, RELATIVE_PATH(d->path), &st, 0) == -1) {
+
		}
+
	if (fstatat(fd, RELATIVE_PATH(path), &st, 0) == -1) {
		if (errno != ENOENT) {
-
			pkg_fatal_errno("Fail to stat directory %s", d->path);
+
			close_tempdir(tmpdir);
+
			pkg_fatal_errno("Fail to stat directory %s", path);
		}
-
		if (fstatat(pkg->rootfd, RELATIVE_PATH(d->path), &st, AT_SYMLINK_NOFOLLOW) == 0) {
-
			unlinkat(pkg->rootfd, RELATIVE_PATH(d->path), 0);
+
		if (fstatat(fd, RELATIVE_PATH(path), &st, AT_SYMLINK_NOFOLLOW) == 0) {
+
			unlinkat(fd, RELATIVE_PATH(path), 0);
		}
-
		if (mkdirat(pkg->rootfd, RELATIVE_PATH(d->path), 0755) == -1) {
-
			pkg_fatal_errno("Fail to create directory %s", d->path);
+
		if (mkdirat(fd, RELATIVE_PATH(path), 0755) == -1) {
+
			if (tmpdir != NULL) {
+
				close_tempdir(tmpdir);
+
				pkg_fatal_errno("Fail to create directory '%s/%s'", tmpdir->temp, path);
+
			}
+
			pkg_fatal_errno("Fail to create directory %s", path);
		}
	}

@@ -336,13 +394,14 @@ create_dir(struct pkg *pkg, struct pkg_dir *d)
		d->noattrs = true;
	}

+
	close_tempdir(tmpdir);
	return (EPKG_OK);
}

/* In case of directories create the dir and extract the creds */
static int
do_extract_dir(struct pkg* pkg, struct archive *a __unused, struct archive_entry *ae,
-
    const char *path, struct pkg *local __unused)
+
    const char *path, struct pkg *local __unused, tempdirs_t *tempdirs)
{
	struct pkg_dir *d;
	const struct stat *aest;
@@ -361,7 +420,7 @@ do_extract_dir(struct pkg* pkg, struct archive *a __unused, struct archive_entry
	fill_timespec_buf(aest, d->time);
	archive_entry_fflags(ae, &d->fflags, &clear);

-
	if (create_dir(pkg, d) == EPKG_FATAL) {
+
	if (create_dir(pkg, d, tempdirs) == EPKG_FATAL) {
		return (EPKG_FATAL);
	}

@@ -387,34 +446,53 @@ try_mkdir(int fd, const char *path)
}

static int
-
create_symlinks(struct pkg *pkg, struct pkg_file *f, const char *target)
+
create_symlinks(struct pkg *pkg, struct pkg_file *f, const char *target, tempdirs_t *tempdirs)
{
+
	struct tempdir *tmpdir = NULL;
+
	int fd;
+
	const char *path;
	bool tried_mkdir = false;

-
	hidden_tempfile(f->temppath, sizeof(f->temppath), f->path);
+
	tmpdir = get_tempdir(pkg->rootfd, f->path, tempdirs);
+
	if (tmpdir == NULL && errno == 0)
+
		hidden_tempfile(f->temppath, sizeof(f->temppath), f->path);
+
	if (tmpdir == NULL) {
+
		fd = pkg->rootfd;
+
		path = f->temppath;
+
	} else {
+
		fd = tmpdir->fd;
+
		path = f->path + tmpdir->len;
+
	}
retry:
-
	if (symlinkat(target, pkg->rootfd, RELATIVE_PATH(f->temppath)) == -1) {
+
	if (symlinkat(target, fd, RELATIVE_PATH(path)) == -1) {
		if (!tried_mkdir) {
-
			if (!try_mkdir(pkg->rootfd, f->path))
+
			if (!try_mkdir(fd, path)) {
+
				close_tempdir(tmpdir);
				return (EPKG_FATAL);
+
			}
			tried_mkdir = true;
			goto retry;
		}

-
		pkg_fatal_errno("Fail to create symlink: %s", f->temppath);
+
		pkg_fatal_errno("Fail to create symlink: %s", path);
	}

-
	if (set_attrsat(pkg->rootfd, f->temppath, f->perm, f->uid, f->gid,
+
	if (set_attrsat(fd, path, f->perm, f->uid, f->gid,
	    &f->time[0], &f->time[1]) != EPKG_OK) {
+
		close_tempdir(tmpdir);
		return (EPKG_FATAL);
	}
+
	if (tmpdir != NULL)
+
		set_chflags(fd, path, f->fflags);
+
	close_tempdir(tmpdir);
+

	return (EPKG_OK);
}

/* In case of a symlink create it directly with a random name */
static int
do_extract_symlink(struct pkg *pkg, struct archive *a __unused, struct archive_entry *ae,
-
    const char *path, struct pkg *local __unused)
+
    const char *path, struct pkg *local __unused, tempdirs_t *tempdirs)
{
	struct pkg_file *f;
	const struct stat *aest;
@@ -434,7 +512,7 @@ do_extract_symlink(struct pkg *pkg, struct archive *a __unused, struct archive_e
	fill_timespec_buf(aest, f->time);
	archive_entry_fflags(ae, &f->fflags, &clear);

-
	if (create_symlinks(pkg, f, archive_entry_symlink(ae)) == EPKG_FATAL)
+
	if (create_symlinks(pkg, f, archive_entry_symlink(ae), tempdirs) == EPKG_FATAL)
		return (EPKG_FATAL);

	metalog_add(PKG_METALOG_LINK, RELATIVE_PATH(path),
@@ -445,39 +523,82 @@ do_extract_symlink(struct pkg *pkg, struct archive *a __unused, struct archive_e
}

static int
-
create_hardlink(struct pkg *pkg, struct pkg_file *f, const char *path)
+
create_hardlink(struct pkg *pkg, struct pkg_file *f, const char *path, tempdirs_t *tempdirs)
{
	bool tried_mkdir = false;
	struct pkg_file *fh;
-

-
	hidden_tempfile(f->temppath, sizeof(f->temppath), f->path);
+
	int fd, fdh;
+
	const char *pathfrom, *pathto;
+
	struct tempdir *tmpdir = NULL;
+
	struct tempdir *tmphdir = NULL;
+

+
	tmpdir = get_tempdir(pkg->rootfd, f->path, tempdirs);
+
	if (tmpdir == NULL && errno == 0)
+
		hidden_tempfile(f->temppath, sizeof(f->temppath), f->path);
+
	if (tmpdir != NULL) {
+
		fd = tmpdir->fd;
+
	} else {
+
		fd = pkg->rootfd;
+
	}
	fh = pkg_get_file(pkg, path);
	if (fh == NULL) {
+
		close_tempdir(tmpdir);
		pkg_emit_error("Can't find the file %s is supposed to be"
		    " hardlinked to %s", f->path, path);
		return (EPKG_FATAL);
	}
+
	if (fh->temppath[0] == '\0') {
+
		tll_foreach(*tempdirs, t) {
+
			if (strncmp(t->item->name, fh->path, t->item->len) == 0 &&
+
			    fh->path[t->item->len] == '/' ) {
+
				tmphdir = t->item;
+
				reopen_tempdir(pkg->rootfd, tmphdir);
+
				break;
+
			}
+
		}
+
	}
+
	if (tmpdir == NULL) {
+
		pathto = f->temppath;
+
		fd = pkg->rootfd;
+
	} else {
+
		pathto = f->path + tmpdir->len;
+
		fd = tmpdir->fd;
+
	}

+
	if (tmphdir == NULL) {
+
		pathfrom = fh->temppath;
+
		fdh = pkg->rootfd;
+
	} else {
+
		pathfrom = fh->path + tmphdir->len;
+
		fdh = tmphdir->fd;
+
	}

retry:
-
	if (linkat(pkg->rootfd, RELATIVE_PATH(fh->temppath),
-
	    pkg->rootfd, RELATIVE_PATH(f->temppath), 0) == -1) {
+
	if (linkat(fdh, RELATIVE_PATH(pathfrom),
+
	    fd, RELATIVE_PATH(pathto), 0) == -1) {
		if (!tried_mkdir) {
-
			if (!try_mkdir(pkg->rootfd, f->path))
+
			if (!try_mkdir(fd, pathto)) {
+
				close_tempdir(tmpdir);
+
				close_tempdir(tmphdir);
				return (EPKG_FATAL);
+
			}
			tried_mkdir = true;
			goto retry;
		}

-
		pkg_fatal_errno("Fail to create hardlink: %s", f->temppath);
+
		close_tempdir(tmpdir);
+
		close_tempdir(tmphdir);
+
		pkg_fatal_errno("Fail to create hardlink: %s <-> %s", pathfrom, pathto);
	}
+
	close_tempdir(tmpdir);
+
	close_tempdir(tmphdir);

	return (EPKG_OK);
}

static int
do_extract_hardlink(struct pkg *pkg, struct archive *a __unused, struct archive_entry *ae,
-
    const char *path, struct pkg *local __unused)
+
    const char *path, struct pkg *local __unused, tempdirs_t *tempdirs)
{
	struct pkg_file *f;
	const struct stat *aest;
@@ -491,7 +612,7 @@ do_extract_hardlink(struct pkg *pkg, struct archive *a __unused, struct archive_
	lp = archive_entry_hardlink(ae);
	aest = archive_entry_stat(ae);

-
	if (create_hardlink(pkg, f, lp) == EPKG_FATAL)
+
	if (create_hardlink(pkg, f, lp, tempdirs) == EPKG_FATAL)
		return (EPKG_FATAL);

	metalog_add(PKG_METALOG_FILE, RELATIVE_PATH(path),
@@ -502,11 +623,11 @@ do_extract_hardlink(struct pkg *pkg, struct archive *a __unused, struct archive_
}

static int
-
create_regfile(struct pkg *pkg, struct pkg_file *f, struct archive *a,
-
    struct archive_entry *ae, int fromfd, struct pkg *local)
+
open_tempfile(int rootfd, const char *path, int perm)
{
-
	int fd = -1;
+
	int fd;
	bool tried_mkdir = false;
+
<<<<<<< HEAD
	size_t len;
	char buf[32768];
	const char *attrname;
@@ -514,20 +635,53 @@ create_regfile(struct pkg *pkg, struct pkg_file *f, struct archive *a,
	size_t attrsz;

	hidden_tempfile(f->temppath, sizeof(f->temppath), f->path);
+
=======
+
>>>>>>> upstream-master

retry:
-
	/* Create the new temp file */
-
	fd = openat(pkg->rootfd, RELATIVE_PATH(f->temppath),
-
	    O_CREAT|O_WRONLY|O_EXCL, f->perm);
+
	fd = openat(rootfd, RELATIVE_PATH(path), O_CREAT|O_WRONLY|O_EXCL, perm);
	if (fd == -1) {
		if (!tried_mkdir) {
-
			if (!try_mkdir(pkg->rootfd, f->path))
-
				return (EPKG_FATAL);
+
			if (!try_mkdir(rootfd, path))
+
				return (-2);
			tried_mkdir = true;
			goto retry;
		}
-
		pkg_fatal_errno("Fail to create temporary file: %s",
-
		    f->temppath);
+
		return (-1);
+
	}
+
	return (fd);
+
}
+

+
static int
+
create_regfile(struct pkg *pkg, struct pkg_file *f, struct archive *a,
+
    struct archive_entry *ae, int fromfd, struct pkg *local, tempdirs_t *tempdirs)
+
{
+
	int fd = -1;
+
	size_t len;
+
	char buf[32768];
+
	char *path;
+
	struct tempdir *tmpdir = NULL;
+

+
	tmpdir = get_tempdir(pkg->rootfd, f->path, tempdirs);
+
	if (tmpdir == NULL && errno == 0)
+
		hidden_tempfile(f->temppath, sizeof(f->temppath), f->path);
+

+
	if (tmpdir != NULL) {
+
		fd = open_tempfile(tmpdir->fd, f->path + tmpdir->len, f->perm);
+
	} else {
+
		fd = open_tempfile(pkg->rootfd, f->temppath,  f->perm);
+
	}
+
	if (fd == -2) {
+
		close_tempdir(tmpdir);
+
		return (EPKG_FATAL);
+
	}
+

+
	if (fd == -1) {
+
		if (tmpdir != NULL) {
+
			close_tempdir(tmpdir);
+
			pkg_fatal_errno("Fail to create temporary file '%s/%s' for %s", tmpdir->name, f->path + tmpdir->len, f->path);
+
		}
+
		pkg_fatal_errno("Fail to create temporary file for %s", f->path);
	}

	if (fromfd == -1) {
@@ -551,6 +705,7 @@ retry:
				free(f->config->newcontent);
		} else {
			if (ftruncate(fd, archive_entry_size(ae)) == -1) {
+
				close_tempdir(tmpdir);
				pkg_errno("Fail to truncate file: %s", f->temppath);
			}
		}
@@ -575,6 +730,7 @@ retry:
		}

		if (!f->config && archive_read_data_into_fd(a, fd) != ARCHIVE_OK) {
+
			close_tempdir(tmpdir);
			pkg_emit_error("Fail to extract %s from package: %s",
			    f->path, archive_error_string(a));
			return (EPKG_FATAL);
@@ -585,20 +741,31 @@ retry:
				pkg_errno("Fail to write file: %s", f->temppath);
			}
	}
-
	if (fd != -1) {
+
	if (fd != -1)
		close(fd);
+
	if (tmpdir == NULL) {
+
		fd = pkg->rootfd;
+
		path = f->temppath;
+
	} else {
+
		fd = tmpdir->fd;
+
		path = f->path + tmpdir->len;
	}

-
	if (set_attrsat(pkg->rootfd, f->temppath, f->perm, f->uid, f->gid,
-
	    &f->time[0], &f->time[1]) != EPKG_OK)
-
			return (EPKG_FATAL);
+
	if (set_attrsat(fd, path, f->perm, f->uid, f->gid,
+
	    &f->time[0], &f->time[1]) != EPKG_OK) {
+
		close_tempdir(tmpdir);
+
		return (EPKG_FATAL);
+
	}
+
	if (tmpdir != NULL)
+
		set_chflags(fd, path, f->fflags);

+
	close_tempdir(tmpdir);
	return (EPKG_OK);
}

static int
do_extract_regfile(struct pkg *pkg, struct archive *a, struct archive_entry *ae,
-
    const char *path, struct pkg *local)
+
    const char *path, struct pkg *local, tempdirs_t *tempdirs)
{
	struct pkg_file *f;
	const struct stat *aest;
@@ -618,7 +785,7 @@ do_extract_regfile(struct pkg *pkg, struct archive *a, struct archive_entry *ae,
	fill_timespec_buf(aest, f->time);
	archive_entry_fflags(ae, &f->fflags, &clear);

-
	if (create_regfile(pkg, f, a, ae, -1, local) == EPKG_FATAL)
+
	if (create_regfile(pkg, f, a, ae, -1, local, tempdirs) == EPKG_FATAL)
		return (EPKG_FATAL);

	metalog_add(PKG_METALOG_FILE, RELATIVE_PATH(path),
@@ -630,13 +797,14 @@ do_extract_regfile(struct pkg *pkg, struct archive *a, struct archive_entry *ae,

static int
do_extract(struct archive *a, struct archive_entry *ae,
-
    int nfiles, struct pkg *pkg, struct pkg *local)
+
    int nfiles, struct pkg *pkg, struct pkg *local, tempdirs_t *tempdirs)
{
	int	retcode = EPKG_OK;
	int	ret = 0, cur_file = 0;
	char	path[MAXPATHLEN];
	int (*extract_cb)(struct pkg *pkg, struct archive *a,
-
	    struct archive_entry *ae, const char *path, struct pkg *local);
+
	    struct archive_entry *ae, const char *path, struct pkg *local,
+
	    tempdirs_t *tempdirs);

#ifndef HAVE_ARC4RANDOM
	srand(time(NULL));
@@ -699,7 +867,7 @@ do_extract(struct archive *a, struct archive_entry *ae,
			break;
		}

-
		if (extract_cb(pkg, a, ae, path, local) != EPKG_OK) {
+
		if (extract_cb(pkg, a, ae, path, local, tempdirs) != EPKG_OK) {
			retcode = EPKG_FATAL;
			goto cleanup;
		}
@@ -723,7 +891,7 @@ cleanup:
}

static int
-
pkg_extract_finalize(struct pkg *pkg)
+
pkg_extract_finalize(struct pkg *pkg, tempdirs_t *tempdirs)
{
	struct stat st;
	struct pkg_file *f = NULL;
@@ -736,6 +904,17 @@ pkg_extract_finalize(struct pkg *pkg)
	install_as_user = (getenv("INSTALL_AS_USER") != NULL);
#endif

+

+
	if (tempdirs != NULL) {
+
		tll_foreach(*tempdirs, t) {
+
			if (renameat(pkg->rootfd, RELATIVE_PATH(t->item->temp),
+
			    pkg->rootfd, RELATIVE_PATH(t->item->name)) != 0) {
+
				pkg_fatal_errno("Fail to rename %s -> %s",
+
				    t->item->temp, t->item->name);
+
			}
+
			free(t->item);
+
		}
+
	}
	while (pkg_files(pkg, &f) == EPKG_OK) {

		if (match_ucl_lists(f->path,
@@ -789,14 +968,8 @@ pkg_extract_finalize(struct pkg *pkg)
			    f->temppath, fto);
		}

-
#ifdef HAVE_CHFLAGSAT
-
		if (!install_as_user && f->fflags != 0) {
-
			if (chflagsat(pkg->rootfd, RELATIVE_PATH(fto),
-
			    f->fflags, AT_SYMLINK_NOFOLLOW) == -1) {
-
				pkg_fatal_errno("Fail to chflags %s", fto);
-
			}
-
		}
-
#endif
+
		if (set_chflags(pkg->rootfd, fto, f->fflags) != EPKG_OK)
+
			return (EPKG_FATAL);
	}

	while (pkg_dirs(pkg, &d) == EPKG_OK) {
@@ -806,7 +979,11 @@ pkg_extract_finalize(struct pkg *pkg)
		if (set_attrsat(pkg->rootfd, d->path, d->perm,
		    d->uid, d->gid, &d->time[0], &d->time[1]) != EPKG_OK)
			return (EPKG_FATAL);
+
		if (set_chflags(pkg->rootfd, d->path, d->fflags) != EPKG_OK)
+
			return (EPKG_FATAL);
	}
+
	if (tempdirs != NULL)
+
		tll_free(*tempdirs);

	return (EPKG_OK);
}
@@ -1015,7 +1192,7 @@ pkg_add_cleanup_old(struct pkgdb *db, struct pkg *old, struct pkg *new, struct t
				}

				trigger_is_it_a_cleanup(t, f->path);
-
				pkg_delete_file(old, f, flags & PKG_DELETE_FORCE ? 1 : 0);
+
				pkg_delete_file(old, f);
			}
		}

@@ -1069,12 +1246,10 @@ pkg_add_common(struct pkgdb *db, const char *path, unsigned flags,
	int			 retcode = EPKG_OK;
	int			 ret;
	int			 nfiles;
+
	tempdirs_t		 tempdirs = tll_init();

	assert(path != NULL);

-
	if (local != NULL)
-
		flags |= PKG_ADD_UPGRADE;
-

	/*
	 * Open the package archive file, read all the meta files and set the
	 * current archive_entry to the first non-meta file.
@@ -1087,16 +1262,13 @@ pkg_add_common(struct pkgdb *db, const char *path, unsigned flags,
		retcode = ret;
		goto cleanup;
	}
-
	if ((flags & PKG_ADD_SPLITTED_UPGRADE) != PKG_ADD_SPLITTED_UPGRADE)
+
	if ((flags & PKG_ADD_SPLITTED_UPGRADE) == 0)
		pkg_emit_new_action();
-
	if ((flags & PKG_ADD_UPGRADE) == 0)
+
	if ((flags & (PKG_ADD_UPGRADE | PKG_ADD_SPLITTED_UPGRADE)) !=
+
	    PKG_ADD_UPGRADE)
		pkg_emit_install_begin(pkg);
-
	else {
-
		if (local != NULL)
-
			pkg_emit_upgrade_begin(pkg, local);
-
		else
-
			pkg_emit_install_begin(pkg);
-
	}
+
	else
+
		pkg_emit_upgrade_begin(pkg, local);

	if (pkg_is_valid(pkg) != EPKG_OK) {
		pkg_emit_error("the package is not valid");
@@ -1178,7 +1350,7 @@ pkg_add_common(struct pkgdb *db, const char *path, unsigned flags,
	 */
	if (extract) {
		pkg_register_cleanup_callback(pkg_rollback_cb, pkg);
-
		retcode = do_extract(a, ae, nfiles, pkg, local);
+
		retcode = do_extract(a, ae, nfiles, pkg, local, &tempdirs);
		pkg_unregister_cleanup_callback(pkg_rollback_cb, pkg);
		if (retcode != EPKG_OK) {
			/* If the add failed, clean up (silently) */
@@ -1190,7 +1362,11 @@ pkg_add_common(struct pkgdb *db, const char *path, unsigned flags,
		}
	}

-
	if (local != NULL) {
+
	/*
+
	 * If this was a split upgrade, the old package has been entirely
+
	 * removed already.
+
	 */
+
	if (local != NULL && (flags & PKG_ADD_SPLITTED_UPGRADE) == 0) {
		pkg_open_root_fd(local);
		pkg_debug(1, "Cleaning up old version");
		if (pkg_add_cleanup_old(db, local, pkg, t, flags) != EPKG_OK) {
@@ -1203,7 +1379,7 @@ pkg_add_common(struct pkgdb *db, const char *path, unsigned flags,
	/* Update configuration file content with db with newer versions */
	pkgdb_update_config_file_content(pkg, db->sqlite);

-
	retcode = pkg_extract_finalize(pkg);
+
	retcode = pkg_extract_finalize(pkg, &tempdirs);

	pkgdb_register_finale(db, retcode, NULL);
	/*
@@ -1224,14 +1400,11 @@ pkg_add_common(struct pkgdb *db, const char *path, unsigned flags,

	pkg_start_stop_rc_scripts(pkg, PKG_RC_START);

-
	if ((flags & PKG_ADD_UPGRADE) == 0)
+
	if ((flags & (PKG_ADD_UPGRADE | PKG_ADD_SPLITTED_UPGRADE)) !=
+
	    PKG_ADD_UPGRADE)
		pkg_emit_install_finished(pkg, local);
-
	else {
-
		if (local != NULL)
-
			pkg_emit_upgrade_finished(pkg, local);
-
		else
-
			pkg_emit_install_finished(pkg, local);
-
	}
+
	else
+
		pkg_emit_upgrade_finished(pkg, local);

	tll_foreach(pkg->message, m) {
		msg = m->item;
@@ -1322,11 +1495,12 @@ pkg_add_fromdir(struct pkg *pkg, const char *src)
	struct group *gr, grent;
	int err, fd, fromfd;
	int retcode;
-
	hls hardlinks = tll_init();
+
	hardlinks_t hardlinks = tll_init();
	const char *path;
	char buffer[1024];
	size_t link_len;
	bool install_as_user;
+
	tempdirs_t tempdirs = tll_init();

	install_as_user = (getenv("INSTALL_AS_USER") != NULL);

@@ -1382,7 +1556,7 @@ pkg_add_fromdir(struct pkg *pkg, const char *src)
#endif
#endif

-
		if (create_dir(pkg, d) == EPKG_FATAL) {
+
		if (create_dir(pkg, d, &tempdirs) == EPKG_FATAL) {
			retcode = EPKG_FATAL;
			goto cleanup;
		}
@@ -1454,7 +1628,7 @@ pkg_add_fromdir(struct pkg *pkg, const char *src)
				    "'%s'", f->path);
			}
			target[link_len] = '\0';
-
			if (create_symlinks(pkg, f, target) == EPKG_FATAL) {
+
			if (create_symlinks(pkg, f, target, &tempdirs) == EPKG_FATAL) {
				retcode = EPKG_FATAL;
				goto cleanup;
			}
@@ -1475,18 +1649,18 @@ pkg_add_fromdir(struct pkg *pkg, const char *src)
				}
			}
			if (path != NULL) {
-
				if (create_hardlink(pkg, f, path) == EPKG_FATAL) {
+
				if (create_hardlink(pkg, f, path, &tempdirs) == EPKG_FATAL) {
					close(fd);
					retcode = EPKG_FATAL;
					goto cleanup;
				}
			} else {
-
				if (create_regfile(pkg, f, NULL, NULL, fd, NULL) == EPKG_FATAL) {
+
				if (create_regfile(pkg, f, NULL, NULL, fd, NULL, &tempdirs) == EPKG_FATAL) {
					close(fd);
					retcode = EPKG_FATAL;
					goto cleanup;
				}
-
				struct store_hardlinks *h = xcalloc(1, sizeof(*h));
+
				struct hardlink *h = xcalloc(1, sizeof(*h));
				h->ino = st.st_ino;
				h->dev = st.st_dev;
				h->path = f->path;
@@ -1500,7 +1674,7 @@ pkg_add_fromdir(struct pkg *pkg, const char *src)
		}
	}

-
	retcode = pkg_extract_finalize(pkg);
+
	retcode = pkg_extract_finalize(pkg, &tempdirs);

cleanup:
	tll_free_and_free(hardlinks, free);
modified libpkg/pkg_attributes.c
@@ -319,6 +319,11 @@ pkg_get_element(struct pkg *p, pkg_attr a)
		e->stringlist->list = &p->groups;
		e->type = PKG_STRINGLIST;
		break;
+
	case PKG_LICENSES:
+
		e->stringlist = xcalloc(1, sizeof(struct pkg_stringlist *));
+
		e->stringlist->list = &p->licenses;
+
		e->type = PKG_STRINGLIST;
+
		break;
	}

	return (e);
modified libpkg/pkg_checksum.c
@@ -34,12 +34,6 @@
#include "sha256.h"
#include "blake2.h"

-
struct pkg_checksum_entry {
-
	const char *field;
-
	char *value;
-
	struct pkg_checksum_entry *next, *prev;
-
};
-

/* Separate checksum parts */
#define PKG_CKSUM_SEPARATOR '$'

@@ -49,7 +43,7 @@ struct pkg_checksum_entry {
#define PKG_CHECKSUM_BLAKE2S_LEN (BLAKE2S_OUTBYTES * 8 / 5 + sizeof("100") * 2 + 2)
#define PKG_CHECKSUM_CUR_VERSION 2

-
typedef void (*pkg_checksum_hash_func)(struct pkg_checksum_entry *entries,
+
typedef void (*pkg_checksum_hash_func)(kvlist_t *entries,
				unsigned char **out, size_t *outlen);
typedef void (*pkg_checksum_hash_bulk_func)(const unsigned char *in, size_t inlen,
				unsigned char **out, size_t *outlen);
@@ -59,19 +53,19 @@ typedef void (*pkg_checksum_encode_func)(unsigned char *in, size_t inlen,
typedef void (*pkg_checksum_hash_file_func)(int fd, unsigned char **out,
    size_t *outlen);

-
static void pkg_checksum_hash_sha256(struct pkg_checksum_entry *entries,
+
static void pkg_checksum_hash_sha256(kvlist_t *entries,
				unsigned char **out, size_t *outlen);
static void pkg_checksum_hash_sha256_bulk(const unsigned char *in, size_t inlen,
				unsigned char **out, size_t *outlen);
static void pkg_checksum_hash_sha256_file(int fd, unsigned char **out,
    size_t *outlen);
-
static void pkg_checksum_hash_blake2(struct pkg_checksum_entry *entries,
+
static void pkg_checksum_hash_blake2(kvlist_t *entries,
				unsigned char **out, size_t *outlen);
static void pkg_checksum_hash_blake2_bulk(const unsigned char *in, size_t inlen,
				unsigned char **out, size_t *outlen);
static void pkg_checksum_hash_blake2_file(int fd, unsigned char **out,
    size_t *outlen);
-
static void pkg_checksum_hash_blake2s(struct pkg_checksum_entry *entries,
+
static void pkg_checksum_hash_blake2s(kvlist_t *entries,
				unsigned char **out, size_t *outlen);
static void pkg_checksum_hash_blake2s_bulk(const unsigned char *in, size_t inlen,
				unsigned char **out, size_t *outlen);
@@ -155,36 +149,14 @@ static const struct _pkg_cksum_type {
	}
};

-
static void
-
pkg_checksum_free_entry(struct pkg_checksum_entry *e)
-
{
-
	if (e == NULL)
-
		return;
-
	free(e->value);
-
	free(e);
-
}
-

-
static void
-
pkg_checksum_add_entry(const char *key,
-
	const char *value,
-
	struct pkg_checksum_entry **entries)
-
{
-
	struct pkg_checksum_entry *e;
-

-
	e = xmalloc(sizeof(*e));
-
	e->field = key;
-
	e->value = xstrdup(value);
-
	DL_APPEND(*entries, e);
-
}
-

static int
-
pkg_checksum_entry_cmp(struct pkg_checksum_entry *e1,
-
	struct pkg_checksum_entry *e2)
+
pkg_checksum_entry_cmp(struct pkg_kv *e1,
+
	struct pkg_kv *e2)
{
	int r;

	/* Compare field names first. */
-
	r = strcmp(e1->field, e2->field);
+
	r = strcmp(e1->key, e2->key);
	if (r != 0)
		return r;

@@ -214,7 +186,7 @@ pkg_checksum_generate(struct pkg *pkg, char *dest, size_t destlen,
	unsigned char *bdigest;
	char *olduid;
	size_t blen;
-
	struct pkg_checksum_entry *entries = NULL;
+
	kvlist_t entries = tll_init();
	struct pkg_option *option = NULL;
	struct pkg_dep *dep = NULL;
	struct pkg_file *f = NULL;
@@ -224,72 +196,69 @@ pkg_checksum_generate(struct pkg *pkg, char *dest, size_t destlen,
					destlen < checksum_types[type].hlen)
		return (EPKG_FATAL);

-
	pkg_checksum_add_entry("name", pkg->name, &entries);
-
	pkg_checksum_add_entry("origin", pkg->origin, &entries);
+
	tll_push_back(entries, pkg_kv_new("name", pkg->name));
+
	tll_push_back(entries, pkg_kv_new("origin", pkg->origin));
	if (inc_version)
-
		pkg_checksum_add_entry("version", pkg->version, &entries);
-
	pkg_checksum_add_entry("arch", pkg->arch, &entries);
+
		tll_push_back(entries, pkg_kv_new("version", pkg->version));
+
	tll_push_back(entries, pkg_kv_new("arch", pkg->arch));

	while (pkg_options(pkg, &option) == EPKG_OK) {
-
		pkg_checksum_add_entry(option->key, option->value, &entries);
+
		tll_push_back(entries, pkg_kv_new(option->key, option->value));
	}

	tll_foreach(pkg->shlibs_required, s) {
-
		pkg_checksum_add_entry("required_shlib", s->item, &entries);
+
		tll_push_back(entries, pkg_kv_new("required_shlib", s->item));
	}

	tll_foreach(pkg->shlibs_provided, s) {
-
		pkg_checksum_add_entry("provided_shlib", s->item, &entries);
+
		tll_push_back(entries, pkg_kv_new("provided_shlib", s->item));
	}

	tll_foreach(pkg->users, u) {
-
		pkg_checksum_add_entry("user", u->item, &entries);
+
		tll_push_back(entries, pkg_kv_new("user", u->item));
	}

	tll_foreach(pkg->groups, g) {
-
		pkg_checksum_add_entry("group", g->item, &entries);
+
		tll_push_back(entries, pkg_kv_new("group", g->item));
	}

	while (pkg_deps(pkg, &dep) == EPKG_OK) {
		xasprintf(&olduid, "%s~%s", dep->name, dep->origin);
-
		pkg_checksum_add_entry("depend", olduid, &entries);
+
		tll_push_back(entries, pkg_kv_new("depend", olduid));
		free(olduid);
	}

	tll_foreach(pkg->provides, p) {
-
		pkg_checksum_add_entry("provide", p->item, &entries);
+
		tll_push_back(entries, pkg_kv_new("provide", p->item));
	}

	tll_foreach(pkg->requires, r) {
-
		pkg_checksum_add_entry("require", r->item, &entries);
+
		tll_push_back(entries, pkg_kv_new("require", r->item));
	}

	if (inc_scripts) {
		for (int i = 0; i < PKG_NUM_SCRIPTS; i++) {
			if (pkg->scripts[i] != NULL) {
				fflush(pkg->scripts[i]->fp);
-
				pkg_checksum_add_entry("script",
-
				    pkg->scripts[i]->buf,
-
				    &entries);
+
				tll_push_back(entries, pkg_kv_new("script", pkg->scripts[i]->buf));
			}
		}
		for (int i = 0; i < PKG_NUM_LUA_SCRIPTS; i++) {
			tll_foreach(pkg->lua_scripts[i], s)
-
				pkg_checksum_add_entry("lua_script",
-
				    s->item, &entries);
+
				tll_push_back(entries, pkg_kv_new("lua_script", s->item));
		}
	}

	while (pkg_files(pkg, &f) == EPKG_OK) {
-
		pkg_checksum_add_entry(f->path, f->sum, &entries);
+
		tll_push_back(entries, pkg_kv_new(f->path, f->sum));
	}

	/* Sort before hashing */
-
	DL_SORT(entries, pkg_checksum_entry_cmp);
+
	tll_sort(entries, pkg_checksum_entry_cmp);

-
	checksum_types[type].hfunc(entries, &bdigest, &blen);
+
	checksum_types[type].hfunc(&entries, &bdigest, &blen);
	if (blen == 0 || bdigest == NULL) {
-
		LL_FREE(entries, pkg_checksum_free_entry);
+
		tll_free_and_free(entries, pkg_kv_free);
		return (EPKG_FATAL);
	}

@@ -306,7 +275,7 @@ pkg_checksum_generate(struct pkg *pkg, char *dest, size_t destlen,
	}

	free(bdigest);
-
	LL_FREE(entries, pkg_checksum_free_entry);
+
	tll_free_and_free(entries, pkg_kv_free);

	return (EPKG_OK);
}
@@ -376,17 +345,16 @@ pkg_checksum_get_type(const char *cksum, size_t clen __unused)
}

static void
-
pkg_checksum_hash_sha256(struct pkg_checksum_entry *entries,
+
pkg_checksum_hash_sha256(kvlist_t *entries,
		unsigned char **out, size_t *outlen)
{
	SHA256_CTX sign_ctx;

	sha256_init(&sign_ctx);

-
	while(entries) {
-
		sha256_update(&sign_ctx, entries->field, strlen(entries->field));
-
		sha256_update(&sign_ctx, entries->value, strlen(entries->value));
-
		entries = entries->next;
+
	tll_foreach(*entries, e) {
+
		sha256_update(&sign_ctx, e->item->key, strlen(e->item->key));
+
		sha256_update(&sign_ctx, e->item->value, strlen(e->item->value));
	}
	*out = xmalloc(SHA256_BLOCK_SIZE);
	sha256_final(&sign_ctx, *out);
@@ -428,17 +396,16 @@ pkg_checksum_hash_sha256_file(int fd, unsigned char **out, size_t *outlen)
}

static void
-
pkg_checksum_hash_blake2(struct pkg_checksum_entry *entries,
+
pkg_checksum_hash_blake2(kvlist_t *entries,
		unsigned char **out, size_t *outlen)
{
	blake2b_state st;

	blake2b_init (&st, BLAKE2B_OUTBYTES);

-
	while(entries) {
-
		blake2b_update (&st, entries->field, strlen(entries->field));
-
		blake2b_update (&st, entries->value, strlen(entries->value));
-
		entries = entries->next;
+
	tll_foreach(*entries, e) {
+
		blake2b_update (&st, e->item->key, strlen(e->item->key));
+
		blake2b_update (&st, e->item->value, strlen(e->item->value));
	}
	*out = xmalloc(BLAKE2B_OUTBYTES);
	blake2b_final (&st, *out, BLAKE2B_OUTBYTES);
@@ -477,17 +444,16 @@ pkg_checksum_hash_blake2_file(int fd, unsigned char **out, size_t *outlen)
}

static void
-
pkg_checksum_hash_blake2s(struct pkg_checksum_entry *entries,
+
pkg_checksum_hash_blake2s(kvlist_t *entries,
		unsigned char **out, size_t *outlen)
{
	blake2s_state st;

	blake2s_init (&st, BLAKE2S_OUTBYTES);

-
	while(entries) {
-
		blake2s_update (&st, entries->field, strlen(entries->field));
-
		blake2s_update (&st, entries->value, strlen(entries->value));
-
		entries = entries->next;
+
	tll_foreach(*entries, e) {
+
		blake2s_update (&st, e->item->key, strlen(e->item->key));
+
		blake2s_update (&st, e->item->value, strlen(e->item->value));
	}
	*out = xmalloc(BLAKE2S_OUTBYTES);
	blake2s_final (&st, *out, BLAKE2S_OUTBYTES);
modified libpkg/pkg_config.c
@@ -72,6 +72,7 @@ struct pkg_ctx ctx = {
	.rootfd = -1,
	.cachedirfd = -1,
	.pkg_dbdirfd = -1,
+
	.devnullfd = -1,
	.osversion = 0,
	.backup_libraries = false,
	.triggers = true,
@@ -1683,3 +1684,24 @@ pkg_get_dbdirfd(void)

	return (ctx.pkg_dbdirfd);
}
+

+
int
+
pkg_open_devnull(void) {
+
	pkg_close_devnull();
+

+
	if ((ctx.devnullfd = open("/dev/null", O_RDWR)) < 0) {
+
		pkg_emit_error("Cannot open /dev/null");
+
		return (EPKG_FATAL);
+
	}
+

+
	return (EPKG_OK);
+
}
+

+
void
+
pkg_close_devnull(void) {
+
	if (ctx.devnullfd != 1) {
+
		close(ctx.devnullfd);
+
	}
+

+
	return;
+
}
modified libpkg/pkg_cudf.c
@@ -336,7 +336,7 @@ cudf_strdup(const char *in)
}

static void
-
pkg_jobs_cudf_insert_res_job (struct pkg_solved **target,
+
pkg_jobs_cudf_insert_res_job (pkg_solved *target,
		struct pkg_job_universe_item *it_new,
		struct pkg_job_universe_item *it_old,
		int type)
@@ -350,7 +350,7 @@ pkg_jobs_cudf_insert_res_job (struct pkg_solved **target,
	if (it_old != NULL)
		res->items[1] = it_old;

-
	DL_APPEND(*target, res);
+
	tll_push_back(*target, res);
}

struct pkg_cudf_entry {
modified libpkg/pkg_delete.c
@@ -54,7 +54,8 @@
#endif

int
-
pkg_delete(struct pkg *pkg, struct pkgdb *db, unsigned flags, struct triggers *t)
+
pkg_delete(struct pkg *pkg, struct pkg *rpkg, struct pkgdb *db, int flags,
+
    struct triggers *t)
{
	xstring		*message = NULL;
	int		 ret;
@@ -67,11 +68,11 @@ pkg_delete(struct pkg *pkg, struct pkgdb *db, unsigned flags, struct triggers *t

	if (pkgdb_ensure_loaded(db, pkg, load_flags) != EPKG_OK)
		return (EPKG_FATAL);
+
	if (rpkg != NULL && pkgdb_ensure_loaded(db, rpkg, load_flags) != EPKG_OK)
+
		return (EPKG_FATAL);

-
	if ((flags & PKG_DELETE_UPGRADE) == 0) {
-
		pkg_emit_new_action();
-
		pkg_emit_deinstall_begin(pkg);
-
	}
+
	pkg_emit_new_action();
+
	pkg_emit_deinstall_begin(pkg);

	/* If the package is locked */
	if (pkg->locked) {
@@ -87,20 +88,17 @@ pkg_delete(struct pkg *pkg, struct pkgdb *db, unsigned flags, struct triggers *t
	if (handle_rc)
		pkg_start_stop_rc_scripts(pkg, PKG_RC_STOP);

-
	if ((flags & PKG_DELETE_NOSCRIPT) == 0) {
+
	if ((flags & (PKG_DELETE_NOSCRIPT | PKG_DELETE_UPGRADE)) == 0) {
		pkg_open_root_fd(pkg);
-
		if (!(flags & PKG_DELETE_UPGRADE)) {
-
			ret = pkg_lua_script_run(pkg, PKG_LUA_PRE_DEINSTALL, false);
-
			if (ret != EPKG_OK && ctx.developer_mode)
-
				return (ret);
-
			ret = pkg_script_run(pkg, PKG_SCRIPT_PRE_DEINSTALL, false);
-
			if (ret != EPKG_OK && ctx.developer_mode)
-
				return (ret);
-
		}
+
		ret = pkg_lua_script_run(pkg, PKG_LUA_PRE_DEINSTALL, false);
+
		if (ret != EPKG_OK && ctx.developer_mode)
+
			return (ret);
+
		ret = pkg_script_run(pkg, PKG_SCRIPT_PRE_DEINSTALL, false);
+
		if (ret != EPKG_OK && ctx.developer_mode)
+
			return (ret);
	}

-
	if ((ret = pkg_delete_files(pkg, flags & PKG_DELETE_FORCE ? 1 : 0, t))
-
            != EPKG_OK)
+
	if ((ret = pkg_delete_files(pkg, rpkg, flags, t)) != EPKG_OK)
		return (ret);

	if ((flags & (PKG_DELETE_NOSCRIPT | PKG_DELETE_UPGRADE)) == 0) {
@@ -129,7 +127,6 @@ pkg_delete(struct pkg *pkg, struct pkgdb *db, unsigned flags, struct triggers *t
			pkg_emit_message(message->buf);
			xstring_free(message);
		}
-

	}

	return (pkgdb_unregister_pkg(db, pkg->id));
@@ -268,7 +265,7 @@ pkg_effective_rmdir(struct pkgdb *db, struct pkg *pkg)
}

void
-
pkg_delete_file(struct pkg *pkg, struct pkg_file *file, unsigned force)
+
pkg_delete_file(struct pkg *pkg, struct pkg_file *file)
{
	const char *path;
	const char *prefix_rel;
@@ -309,12 +306,10 @@ pkg_delete_file(struct pkg *pkg, struct pkg_file *file, unsigned force)
#endif
	pkg_debug(1, "Deleting file: '%s'", path);
	if (unlinkat(pkg->rootfd, path, 0) == -1) {
-
		if (force < 2) {
-
			if (errno == ENOENT)
-
				pkg_emit_file_missing(pkg, file);
-
			else
-
				pkg_emit_errno("unlinkat", path);
-
		}
+
		if (errno == ENOENT)
+
			pkg_emit_file_missing(pkg, file);
+
		else
+
			pkg_emit_errno("unlinkat", path);
		return;
	}

@@ -323,19 +318,37 @@ pkg_delete_file(struct pkg *pkg, struct pkg_file *file, unsigned force)
		pkg_add_dir_to_del(pkg, path, NULL);
}

+
/*
+
 * Handle a special case: the package is to be upgraded but is being deleted
+
 * temporarily to handle a file path conflict.  In this situation we shouldn't
+
 * remove configuration files.  For now, keep them if the replacement package
+
 * contains a configuration file at the same path.
+
 *
+
 * Note, this currently doesn't handle the case where a configuration file
+
 * participates in the conflict, i.e., it moves from one package to another.
+
 */
+
static bool
+
pkg_delete_skip_config(struct pkg *pkg, struct pkg *rpkg, struct pkg_file *file,
+
    int flags)
+
{
+
	if ((flags & PKG_DELETE_UPGRADE) == 0)
+
		return (false);
+
	if (pkghash_get(pkg->config_files_hash, file->path) == NULL)
+
		return (false);
+
	if (pkghash_get(rpkg->config_files_hash, file->path) == NULL)
+
		return (false);
+
	return (true);
+
}
+

int
-
pkg_delete_files(struct pkg *pkg, unsigned force, struct triggers *t)
-
	/* force: 0 ... be careful and vocal about it.
-
	 *        1 ... remove files without bothering about checksums.
-
	 *        2 ... like 1, but remain silent if removal fails.
-
	 */
+
pkg_delete_files(struct pkg *pkg, struct pkg *rpkg, int flags,
+
    struct triggers *t)
{
	struct pkg_file	*file = NULL;

	int		nfiles, cur_file = 0;

	nfiles = pkghash_count(pkg->filehash);
-

	if (nfiles == 0)
		return (EPKG_OK);

@@ -343,10 +356,12 @@ pkg_delete_files(struct pkg *pkg, unsigned force, struct triggers *t)
	pkg_emit_progress_start(NULL);

	while (pkg_files(pkg, &file) == EPKG_OK) {
+
		if (pkg_delete_skip_config(pkg, rpkg, file, flags))
+
			continue;
		append_touched_file(file->path);
		pkg_emit_progress_tick(cur_file++, nfiles);
		trigger_is_it_a_cleanup(t, file->path);
-
		pkg_delete_file(pkg, file, force);
+
		pkg_delete_file(pkg, file);
	}

	pkg_emit_progress_tick(nfiles, nfiles);
modified libpkg/pkg_jobs.c
@@ -1,10 +1,9 @@
/*-
-
 * Copyright (c) 2011-2016 Baptiste Daroussin <bapt@FreeBSD.org>
+
 * Copyright (c) 2011-2022 Baptiste Daroussin <bapt@FreeBSD.org>
 * Copyright (c) 2011-2012 Julien Laffaye <jlaffaye@FreeBSD.org>
 * Copyright (c) 2011-2012 Marin Atanasov Nikolov <dnaeon@gmail.com>
 * Copyright (c) 2013 Matthew Seaman <matthew@FreeBSD.org>
 * Copyright (c) 2013-2016 Vsevolod Stakhov <vsevolod@FreeBSD.org>
-
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
@@ -186,7 +185,7 @@ pkg_jobs_free(struct pkg_jobs *j)
	j->request_delete = NULL;

	pkg_jobs_universe_free(j->universe);
-
	LL_FREE(j->jobs, free);
+
	tll_free_and_free(j->jobs, free);
	LL_FREE(j->patterns, pkg_jobs_pattern_free);
	if (j->triggers.cleanup != NULL) {
		tll_free_and_free(*j->triggers.cleanup, trigger_free);
@@ -279,28 +278,32 @@ pkg_jobs_add(struct pkg_jobs *j, match_t match, char **argv, int argc)
}

bool
-
pkg_jobs_iter(struct pkg_jobs *jobs, void **iter,
+
pkg_jobs_iter(struct pkg_jobs *j, void **iter,
				struct pkg **new, struct pkg **old,
				int *type)
{
	struct pkg_solved *s;
-
	assert(iter != NULL);
-
	if (jobs->jobs == NULL) {
-
		return (false);
-
	}
+
	struct {
+
		typeof(*(j->jobs.head)) *it;
+
	} *t;
+
	t = *iter;
	if (*iter == NULL) {
-
		s = jobs->jobs;
+
		t = xcalloc(1, sizeof(*t));
+
		*iter = t;
+
	} else if (t->it == NULL) {
+
			free(t);
+
			return (false);
	}
-
	else if (*iter == jobs->jobs) {
+

+
	if (tll_length(j->jobs) == 0)
		return (false);
-
	}
-
	else {
-
		s = *iter;
-
	}
+
	if (t->it == NULL)
+
		t->it = j->jobs.head;
+
	s = t->it->item;
	*new = s->items[0]->pkg;
	*old = s->items[1] ? s->items[1]->pkg : NULL;
	*type = s->type;
-
	*iter = s->next ? s->next : jobs->jobs;
+
	t->it = t->it->next;
	return (true);
}

@@ -428,87 +431,68 @@ pkg_jobs_add_req(struct pkg_jobs *j, struct pkg *pkg)
	return (nit);
}

-
/*
-
 * Post-process add request and handle flags:
-
 * upgrade - search for upgrades for dependencies and add them to the request
-
 * force - all upgrades are forced
-
 * reverse - try to upgrade reverse deps as well
-
 */
-
static void
-
pkg_jobs_process_add_request(struct pkg_jobs *j)
+
static bool
+
append_to_del_request(struct pkg_jobs *j, pkg_chain_t *to_process, const char *uid, const char *reqname)
{
-
	bool force = j->flags & PKG_FLAG_FORCE,
-
		 reverse = j->flags & PKG_FLAG_RECURSIVE,
-
		 upgrade = j->type == PKG_JOBS_UPGRADE;
-
	struct pkg_job_request *req;
-
	struct pkg_job_request_item *it;
-
	struct pkg_job_universe_item *un, *cur;
-
	struct pkg_dep *d;
-
	struct pkg *lp;
-
	int (*deps_func)(const struct pkg *pkg, struct pkg_dep **d);
-
	tll(struct pkg_job_universe_item *) to_process = tll_init();
-
	pkghash_it hit;
-

-
	if (!upgrade && !reverse)
-
		return;
+
	struct pkg *p;

-
	hit = pkghash_iterator(j->request_add);
-
	while (pkghash_next(&hit)) {
-
		req = hit.value;
-
		it = req->item;
+
	p = pkg_jobs_universe_get_local(j->universe, uid, 0);
+
	if (p == NULL)
+
		return (true);
+
	if (p->locked) {
+
		pkg_emit_error("%s is locked cannot delete %s", p->name,
+
		   reqname);
+
		return (false);
+
	}
+
	tll_push_back(*to_process, p);
+
	return (true);
+
}

-
		if (reverse)
-
			deps_func = pkg_rdeps;
-
		else
-
			deps_func = pkg_deps;
+
bool
+
delete_process_provides(struct pkg_jobs *j, struct pkg *lp, const char *provide,
+
    struct pkgdb_it *(*provideq)(struct pkgdb *db, const char *req),
+
    struct pkgdb_it *(*requireq)(struct pkgdb *db, const char *req),
+
    pkg_chain_t *to_process)
+
{
+
	struct pkgdb_it *lit, *rit;
+
	struct pkg *pkg;
+
	struct pkg_job_request *req;
+
	bool ret = true;

-
		d = NULL;
+
	/* if something else to provide the same thing we can safely delete */
+
	lit = provideq(j->db, provide);
+
	if (lit == NULL)
+
		return (ret);
+
	pkg = NULL;
+
	while (pkgdb_it_next(lit, &pkg, PKG_LOAD_BASIC) == EPKG_OK) {
+
		/* skip myself */
+
		if (strcmp(pkg->uid, lp->uid) == 0)
+
			continue;
+
		req = pkghash_get_value(j->request_delete, pkg->uid);
		/*
-
		 * Here we get deps of local packages only since we are pretty sure
-
		 * that they are completely expanded
+
		 * skip already processed provides
+
		 * if N packages providing the same "provide"
+
		 * are in the request delete they needs to be
+
		 * counted as to be removed and then if no
+
		 * packages are left providing the provide are
+
		 * left after the removal of those packages
+
		 * cascade.
		 */
-
		lp = pkg_jobs_universe_get_local(j->universe,
-
		    it->pkg->uid, 0);
-
		while (lp != NULL && deps_func(lp, &d) == EPKG_OK) {
-
			/*
-
			 * Do not add duplicated upgrade candidates
-
			 */
-
			if (pkghash_get(j->request_add, d->uid))
-
				continue;
-

-
			pkg_debug(4, "adding dependency %s to request", d->uid);
-
			lp = pkg_jobs_universe_get_local(j->universe,
-
				d->uid, 0);
-
			/*
-
			 * Here we need to check whether specific remote package
-
			 * is newer than a local one
-
			 */
-
			un = pkg_jobs_universe_get_upgrade_candidates(j->universe,
-
				d->uid, lp, force, NULL);
-
			if (un == NULL)
-
				continue;
-

-
			cur = un->prev;
-
			while (cur != un) {
-
				if (cur->pkg->type != PKG_INSTALLED) {
-
					tll_push_back(to_process, un);
-
					break;
-
				}
-
				cur = cur->prev;
-
			}
-
		}
+
		if (req != NULL && req->processed)
+
			continue;
+
		return (ret);
	}
+
	rit = requireq(j->db, provide);
+
	if (rit == NULL)
+
		return (ret);

-
	/* Add all items to the request */
-
	tll_foreach(to_process, pit) {
-
		un = pit->item;
-
		pkg_jobs_add_req_from_universe(&j->request_add, un, false, true);
+
	pkg = NULL;
+
	while (pkgdb_it_next(rit, &pkg, PKG_LOAD_BASIC) == EPKG_OK) {
+
		if (!append_to_del_request(j, to_process,
+
		    pkg->uid, lp->name))
+
			ret = false;
	}
-
	/* Now recursively process all items checked */
-
	if (tll_length(to_process) > 0)
-
		pkg_jobs_process_add_request(j);
-

-
	tll_free(to_process);
+
	return (ret);
}

/*
@@ -522,7 +506,7 @@ pkg_jobs_process_delete_request(struct pkg_jobs *j)
	struct pkg_dep *d = NULL;
	struct pkg *lp;
	int rc = EPKG_OK;
-
	tll(struct pkg *) to_process = tll_init();
+
	pkg_chain_t to_process = tll_init();
	pkghash_it it;

	if (force)
@@ -534,21 +518,29 @@ pkg_jobs_process_delete_request(struct pkg_jobs *j)
	it = pkghash_iterator(j->request_delete);
	while (pkghash_next(&it)) {
		req = it.value;
+
		if (req->processed)
+
			continue;
+
		req->processed = true;
+
		lp = req->item->pkg;
		d = NULL;
-
		while (pkg_rdeps(req->item->pkg, &d) == EPKG_OK) {
-
			if (pkghash_get(j->request_delete, d->uid))
-
				continue;
+
		while (pkg_rdeps(lp, &d) == EPKG_OK) {
+
			if (!append_to_del_request(j, &to_process, d->uid,
+
			    lp->name))
+
				rc = EPKG_FATAL;
+
		}

-
			lp = pkg_jobs_universe_get_local(j->universe, d->uid, 0);
-
			if (lp) {
-
				if (lp->locked) {
-
					pkg_emit_error("%s is locked, "
-
					    "cannot delete %s", lp->name,
-
					   req->item->pkg->name);
-
					rc = EPKG_FATAL;
-
				}
-
				tll_push_back(to_process, lp);
-
			}
+
		tll_foreach(lp->provides, i) {
+
			if (!delete_process_provides(j, lp, i->item,
+
			    pkgdb_query_provide, pkgdb_query_require,
+
			    &to_process))
+
				rc = EPKG_FATAL;
+
		}
+

+
		tll_foreach(lp->shlibs_provided, i) {
+
			if (!delete_process_provides(j, lp, i->item,
+
			    pkgdb_query_shlib_provide,
+
			    pkgdb_query_shlib_require, &to_process))
+
				rc = EPKG_FATAL;
		}
	}

@@ -562,6 +554,7 @@ pkg_jobs_process_delete_request(struct pkg_jobs *j)
			return (EPKG_FATAL);
		}
	}
+

	if (tll_length(to_process) > 0)
		rc = pkg_jobs_process_delete_request(j);
	tll_free(to_process);
@@ -589,9 +582,11 @@ pkg_jobs_set_execute_priority(struct pkg_jobs *j, struct pkg_solved *solved)
			ts = xcalloc(1, sizeof(struct pkg_solved));
			ts->type = PKG_SOLVED_UPGRADE_REMOVE;
			ts->items[0] = solved->items[1];
+
			ts->xlink = solved;
			solved->items[1] = NULL;
			solved->type = PKG_SOLVED_UPGRADE_INSTALL;
-
			DL_APPEND(j->jobs, ts);
+
			solved->xlink = ts;
+
			tll_push_back(j->jobs, ts);
			j->count++;
			pkg_debug(2, "split upgrade request for %s",
			   ts->items[0]->pkg->uid);
@@ -638,19 +633,52 @@ pkg_jobs_set_priorities(struct pkg_jobs *j)
	struct pkg_solved *req;

iter_again:
-
	LL_FOREACH(j->jobs, req) {
+
	tll_foreach(j->jobs, r) {
+
		req = r->item;
		req->items[0]->priority = 0;
		if (req->items[1] != NULL)
			req->items[1]->priority = 0;
	}
-
	LL_FOREACH(j->jobs, req) {
-
		if (pkg_jobs_set_execute_priority(j, req) == EPKG_CONFLICT)
+
	tll_foreach(j->jobs, r) {
+
		if (pkg_jobs_set_execute_priority(j, r->item) == EPKG_CONFLICT)
			goto iter_again;
	}

-
	DL_SORT(j->jobs, pkg_jobs_sort_priority);
+
	tll_sort(j->jobs, pkg_jobs_sort_priority);
}

+
static bool pkg_jobs_test_automatic(struct pkg_jobs *j, struct pkg *p);
+

+
static bool
+
is_orphaned(struct pkg_jobs *j, const char *uid)
+
{
+
	struct pkg_job_universe_item *unit;
+
	struct pkg *npkg;
+

+
	unit = pkg_jobs_universe_find(j->universe, uid);
+
	if (unit != NULL) {
+
		if (unit->pkg->automatic)
+
			return (false);
+
		npkg = unit->pkg;
+
	} else {
+
		npkg = pkg_jobs_universe_get_local(j->universe, uid,
+
		    PKG_LOAD_BASIC|PKG_LOAD_RDEPS|PKG_LOAD_ANNOTATIONS|
+
		    PKG_LOAD_SHLIBS_REQUIRED|PKG_LOAD_REQUIRES);
+
		if (npkg == NULL)
+
			return (false);
+
		if (!npkg->automatic) {
+
			pkg_free(npkg);
+
			return (false);
+
		}
+
		if (pkg_jobs_universe_process(j->universe, npkg) != EPKG_OK)
+
			return (false);
+
	}
+

+
	if (!pkg_jobs_test_automatic(j, npkg))
+
		return (false);
+

+
	return (true);
+
}

/**
 * Test whether package specified is automatic with all its rdeps
@@ -662,38 +690,38 @@ static bool
pkg_jobs_test_automatic(struct pkg_jobs *j, struct pkg *p)
{
	struct pkg_dep *d = NULL;
-
	struct pkg_job_universe_item *unit;
	struct pkg *npkg;
-
	bool ret = true;
+
	struct pkgdb_it *it;
+

+
	while (pkg_rdeps(p, &d) == EPKG_OK) {
+
		if (!is_orphaned(j, d->uid))
+
			return (false);
+
	}

-
	while (pkg_rdeps(p, &d) == EPKG_OK && ret) {
-
		unit = pkg_jobs_universe_find(j->universe, d->uid);
-
		if (unit != NULL) {
-
			if (!unit->pkg->automatic) {
+
	tll_foreach(p->provides, i) {
+
		it = pkgdb_query_require(j->db, i->item);
+
		if (it == NULL)
+
			continue;
+
		npkg = NULL;
+
		while (pkgdb_it_next(it, &npkg, PKG_LOAD_BASIC) == EPKG_OK) {
+
			if (!is_orphaned(j, npkg->uid))
				return (false);
-
			}
-
			npkg = unit->pkg;
		}
-
		else {
-
			npkg = pkg_jobs_universe_get_local(j->universe, d->uid,
-
					PKG_LOAD_BASIC|PKG_LOAD_RDEPS|PKG_LOAD_ANNOTATIONS);
-
			if (npkg == NULL)
-
				return (false);
-
			if (!npkg->automatic) {
-
				/*
-
				 * Safe to free, as d->uid is not in the universe
-
				 */
-
				pkg_free(npkg);
-
				return (false);
-
			}
-
			if (pkg_jobs_universe_process(j->universe, npkg) != EPKG_OK)
+
	}
+

+
	tll_foreach(p->shlibs_provided, i) {
+
		it = pkgdb_query_shlib_require(j->db, i->item);
+
		if (it == NULL)
+
			continue;
+
		npkg = NULL;
+
		while (pkgdb_it_next(it, &npkg, PKG_LOAD_BASIC) == EPKG_OK) {
+
			if (!is_orphaned(j, npkg->uid))
				return (false);
		}
-

-
		ret = pkg_jobs_test_automatic(j, npkg);
	}

-
	return (ret);
+

+
	return (true);
}


@@ -890,7 +918,7 @@ pkg_jobs_guess_upgrade_candidate(struct pkg_jobs *j, const char *pattern)
		strlcpy(cpy, pos, len + 1);
		if (pkg_jobs_try_remote_candidate(j, cpy, NULL, MATCH_INTERNAL) != EPKG_OK) {
			free(cpy);
-
			cpy = sqlite3_mprintf(" WHERE name REGEXP ('^' || %.*Q || '[0-9.]*$')",
+
			cpy = sqlite3_mprintf(" WHERE p.name REGEXP ('^' || %.*Q || '[0-9.]*$')",
					len, pos);

			if (pkg_jobs_try_remote_candidate(j, cpy, opattern, MATCH_ALL)
@@ -1403,7 +1431,8 @@ pkg_jobs_set_deinstall_reasons(struct pkg_jobs *j)
	struct pkg_job_request *jreq;
	struct pkg *req_pkg, *pkg;

-
	LL_FOREACH(j->jobs, sit) {
+
	tll_foreach(j->jobs, it) {
+
		sit = it->item;
		jreq = pkg_jobs_find_deinstall_request(sit->items[0], j, 0);
		if (jreq != NULL && jreq->item->unit != sit->items[0]) {
			req_pkg = jreq->item->pkg;
@@ -1438,8 +1467,9 @@ jobs_solve_deinstall(struct pkg_jobs *j)
			pkg_emit_notice("No packages matched for pattern '%s'\n", jp->pattern);
		}

-
		while (pkgdb_it_next(it, &pkg,
-
				PKG_LOAD_BASIC|PKG_LOAD_RDEPS|PKG_LOAD_DEPS|PKG_LOAD_ANNOTATIONS) == EPKG_OK) {
+
		while (pkgdb_it_next(it, &pkg, PKG_LOAD_BASIC|PKG_LOAD_RDEPS|
+
		    PKG_LOAD_DEPS|PKG_LOAD_ANNOTATIONS|PKG_LOAD_PROVIDES|
+
		    PKG_LOAD_SHLIBS_PROVIDED) == EPKG_OK) {
			if(pkg->locked) {
				if (tsearch(pkg, &j->lockedpkgs, comp) == NULL) {
					return (EPKG_FATAL);
@@ -1468,7 +1498,9 @@ jobs_solve_autoremove(struct pkg_jobs *j)
		return (EPKG_FATAL);

	while (pkgdb_it_next(it, &pkg,
-
			PKG_LOAD_BASIC|PKG_LOAD_RDEPS|PKG_LOAD_DEPS|PKG_LOAD_ANNOTATIONS)
+
			PKG_LOAD_BASIC|PKG_LOAD_RDEPS|PKG_LOAD_DEPS|
+
			PKG_LOAD_ANNOTATIONS|PKG_LOAD_PROVIDES|
+
			PKG_LOAD_SHLIBS_PROVIDED)
			== EPKG_OK) {
		if(pkg->locked) {
			pkg_emit_locked(pkg);
@@ -1701,10 +1733,6 @@ jobs_solve_install_upgrade(struct pkg_jobs *j)
		}
	}

-
#if 0
-
	/* XXX: check if we can safely remove this function */
-
	pkg_jobs_process_add_request(j);
-
#endif
	if (pkg_conflicts_request_resolve(j) != EPKG_OK) {
		pkg_emit_error("Cannot resolve conflicts in a request");
		return (EPKG_FATAL);
@@ -1905,7 +1933,6 @@ int
pkg_jobs_solve(struct pkg_jobs *j)
{
	int ret;
-
	struct pkg_solved *job;
	const char *cudf_solver;

	pkgdb_begin_solver(j->db);
@@ -1950,10 +1977,10 @@ pkg_jobs_solve(struct pkg_jobs *j)
	pkg_jobs_apply_replacements(j);

	/* Check if we need to fetch and re-run the solver */
-
	DL_FOREACH(j->jobs, job) {
+
	tll_foreach(j->jobs, job) {
		struct pkg *p;

-
		p = job->items[0]->pkg;
+
		p = ((struct pkg_solved *)job->item)->items[0]->pkg;
		if (p->type != PKG_REMOTE)
			continue;

@@ -1972,8 +1999,7 @@ pkg_jobs_solve(struct pkg_jobs *j)
			rc = pkg_jobs_check_conflicts(j);
			if (rc == EPKG_CONFLICT) {
				/* Cleanup results */
-
				LL_FREE(j->jobs, free);
-
				j->jobs = NULL;
+
				tll_free_and_free(j->jobs, free);
				j->count = 0;
				has_conflicts = true;
				pkg_jobs_solve(j);
@@ -2021,8 +2047,17 @@ pkg_jobs_handle_install(struct pkg_solved *ps, struct pkg_jobs *j,
	int flags = 0;
	int retcode = EPKG_FATAL;

-
	old = ps->items[1] ? ps->items[1]->pkg : NULL;
+
	/*
+
	 * For a split upgrade, pass along the old package even though it's
+
	 * already deleted, since we need it in order to merge configuration
+
	 * file changes.
+
	 */
	new = ps->items[0]->pkg;
+
	old = NULL;
+
	if (ps->items[1] != NULL)
+
		old = ps->items[1]->pkg;
+
	else if (ps->type == PKG_SOLVED_UPGRADE_INSTALL)
+
		old = ps->xlink->items[0]->pkg;

	req = pkghash_get_value(j->request_add, new->uid);
	if (req != NULL && req->item->jp != NULL &&
@@ -2050,9 +2085,11 @@ pkg_jobs_handle_install(struct pkg_solved *ps, struct pkg_jobs *j,
		flags |= PKG_ADD_NOSCRIPT;
	if ((j->flags & PKG_FLAG_FORCE_MISSING) == PKG_FLAG_FORCE_MISSING)
		flags |= PKG_ADD_FORCE_MISSING;
-
	flags |= PKG_ADD_UPGRADE;
-
	if (ps->type == PKG_SOLVED_UPGRADE_INSTALL)
-
		flags |= PKG_ADD_SPLITTED_UPGRADE;
+
	if (ps->type != PKG_SOLVED_INSTALL) {
+
		flags |= PKG_ADD_UPGRADE;
+
		if (ps->type == PKG_SOLVED_UPGRADE_INSTALL)
+
			flags |= PKG_ADD_SPLITTED_UPGRADE;
+
	}
	if (new->automatic || (j->flags & PKG_FLAG_AUTOMATIC) == PKG_FLAG_AUTOMATIC)
		flags |= PKG_ADD_AUTOMATIC;

@@ -2065,12 +2102,28 @@ pkg_jobs_handle_install(struct pkg_solved *ps, struct pkg_jobs *j,
}

static int
+
pkg_jobs_handle_delete(struct pkg_solved *ps, struct pkg_jobs *j)
+
{
+
	struct pkg *rpkg;
+
	int flags;
+

+
	rpkg = NULL;
+
	flags = 0;
+
	if ((j->flags & PKG_FLAG_NOSCRIPT) != 0)
+
		flags |= PKG_DELETE_NOSCRIPT;
+
	if (ps->type == PKG_SOLVED_UPGRADE_REMOVE) {
+
		flags |= PKG_DELETE_UPGRADE;
+
		rpkg = ps->xlink->items[0]->pkg;
+
	}
+
	return (pkg_delete(ps->items[0]->pkg, rpkg, j->db, flags,
+
	    &j->triggers));
+
}
+

+
static int
pkg_jobs_execute(struct pkg_jobs *j)
{
-
	struct pkg *p = NULL;
-
	struct pkg_solved *ps;
+
	struct pkg *p;
	struct pkg_manifest_key *keys = NULL;
-
	int flags = 0;
	int retcode = EPKG_FATAL;
	pkg_plugin_hook_t pre, post;

@@ -2095,12 +2148,6 @@ pkg_jobs_execute(struct pkg_jobs *j)
	if (j->flags & PKG_FLAG_SKIP_INSTALL)
		return (EPKG_OK);

-
	if ((j->flags & PKG_FLAG_FORCE) == PKG_FLAG_FORCE)
-
		flags |= PKG_DELETE_FORCE;
-

-
	if ((j->flags & PKG_FLAG_NOSCRIPT) == PKG_FLAG_NOSCRIPT)
-
		flags |= PKG_DELETE_NOSCRIPT;
-

	retcode = pkgdb_upgrade_lock(j->db, PKGDB_LOCK_ADVISORY,
			PKGDB_LOCK_EXCLUSIVE);
	if (retcode != EPKG_OK)
@@ -2108,52 +2155,45 @@ pkg_jobs_execute(struct pkg_jobs *j)

	pkg_plugins_hook_run(pre, j, j->db);

-
	p = NULL;
	pkg_manifest_keys_new(&keys);

	pkg_jobs_set_priorities(j);

-
	DL_FOREACH(j->jobs, ps) {
+
	tll_foreach(j->jobs, _p) {
+
		struct pkg_solved *ps = _p->item;
+

		switch (ps->type) {
		case PKG_SOLVED_DELETE:
-
		case PKG_SOLVED_UPGRADE_REMOVE:
-
			p = ps->items[0]->pkg;
-
			if (ps->type == PKG_SOLVED_DELETE && p->vital && ((flags & PKG_DELETE_FORCE) == 0)) {
-
				pkg_emit_error("Cannot delete vital package: %s!", p->name);
-
				pkg_emit_error("If you are sure you want to remove %s, ", p->name);
-
				pkg_emit_error("unset the 'vital' flag with: pkg set -v 0 %s", p->name);
-
				retcode = EPKG_FATAL;
-
				goto cleanup;
-
			}
-
			if (ps->type == PKG_SOLVED_DELETE &&
-
			    (strcmp(p->name, "pkg") == 0 ||
-
			    strcmp(p->name, "pkg-devel") == 0) &&
-
			    (flags & PKG_DELETE_FORCE) == 0) {
-
				if (j->patterns->match == MATCH_ALL) {
-
					continue;
-
				} else {
-
					pkg_emit_error("Cannot delete pkg itself without force flag");
+
			if ((j->flags & PKG_FLAG_FORCE) == 0) {
+
				p = ps->items[0]->pkg;
+
				if (p->vital) {
+
					pkg_emit_error(
+
					    "Cannot delete vital package: %s!", p->name);
+
					pkg_emit_error(
+
					    "If you are sure you want to remove %s, ", p->name);
+
					pkg_emit_error(
+
					    "unset the 'vital' flag with: pkg set -v 0 %s", p->name);
+
					retcode = EPKG_FATAL;
+
					goto cleanup;
+
				}
+
				if (strcmp(p->name, "pkg") == 0 ||
+
				    strcmp(p->name, "pkg-devel") == 0) {
+
					if (j->patterns->match == MATCH_ALL)
+
						continue;
+
					pkg_emit_error(
+
					    "Cannot delete pkg itself without force flag");
					retcode = EPKG_FATAL;
					goto cleanup;
				}
			}
-
			/*
-
			 * Assume that in upgrade we can remove packages with rdeps as
-
			 * in further they will be upgraded correctly.
-
			 */
-
			if (j->type == PKG_JOBS_UPGRADE)
-
				retcode = pkg_delete(p, j->db, flags | PKG_DELETE_CONFLICT, &j->triggers);
-
			else
-
				retcode = pkg_delete(p, j->db, flags, &j->triggers);
+
			/* FALLTHROUGH */
+
		case PKG_SOLVED_UPGRADE_REMOVE:
+
			retcode = pkg_jobs_handle_delete(ps, j);
			if (retcode != EPKG_OK)
				goto cleanup;
			break;
		case PKG_SOLVED_INSTALL:
		case PKG_SOLVED_UPGRADE_INSTALL:
-
			retcode = pkg_jobs_handle_install(ps, j, keys);
-
			if (retcode != EPKG_OK)
-
				goto cleanup;
-
			break;
		case PKG_SOLVED_UPGRADE:
			retcode = pkg_jobs_handle_install(ps, j, keys);
			if (retcode != EPKG_OK)
@@ -2163,7 +2203,6 @@ pkg_jobs_execute(struct pkg_jobs *j)
			retcode = EPKG_FATAL;
			pkg_emit_error("internal error: bad job type");
			goto cleanup;
-
			break;
		}

	}
@@ -2206,8 +2245,7 @@ pkg_jobs_apply(struct pkg_jobs *j)
						rc = pkg_jobs_check_conflicts(j);
						if (rc == EPKG_CONFLICT) {
							/* Cleanup results */
-
							LL_FREE(j->jobs, free);
-
							j->jobs = NULL;
+
							tll_free_and_free(j->jobs, free);
							j->count = 0;
							has_conflicts = true;
							rc = pkg_jobs_solve(j);
@@ -2255,7 +2293,6 @@ static int
pkg_jobs_fetch(struct pkg_jobs *j)
{
	struct pkg *p = NULL;
-
	struct pkg_solved *ps;
	struct stat st;
	int64_t dlsize = 0, fs_avail = -1;
	const char *cachedir = NULL;
@@ -2269,7 +2306,8 @@ pkg_jobs_fetch(struct pkg_jobs *j)
		cachedir = j->destdir;

	/* check for available size to fetch */
-
	DL_FOREACH(j->jobs, ps) {
+
	tll_foreach(j->jobs, _p) {
+
		struct pkg_solved *ps = _p->item;
		if (ps->type != PKG_SOLVED_DELETE && ps->type != PKG_SOLVED_UPGRADE_REMOVE) {
			p = ps->items[0]->pkg;
			if (p->type != PKG_REMOTE)
@@ -2334,7 +2372,8 @@ pkg_jobs_fetch(struct pkg_jobs *j)
		return (EPKG_OK); /* don't download anything */

	/* Fetch */
-
	DL_FOREACH(j->jobs, ps) {
+
	tll_foreach(j->jobs, _p) {
+
		struct pkg_solved *ps = _p->item;
		if (ps->type != PKG_SOLVED_DELETE && ps->type != PKG_SOLVED_UPGRADE_REMOVE) {
			p = ps->items[0]->pkg;
			if (p->type != PKG_REMOTE)
@@ -2357,14 +2396,14 @@ pkg_jobs_fetch(struct pkg_jobs *j)
static int
pkg_jobs_check_conflicts(struct pkg_jobs *j)
{
-
	struct pkg_solved *ps;
	struct pkg *p = NULL;
	int ret = EPKG_OK, res, added = 0;

	pkg_emit_integritycheck_begin();
	j->conflicts_registered = 0;

-
	DL_FOREACH(j->jobs, ps) {
+
	tll_foreach(j->jobs, _p) {
+
		struct pkg_solved *ps = _p->item;
		if (ps->type == PKG_SOLVED_DELETE || ps->type == PKG_SOLVED_UPGRADE_REMOVE) {
			continue;
		}
modified libpkg/pkg_jobs_conflicts.c
@@ -37,11 +37,7 @@
#include "private/pkg_jobs.h"
#include "siphash.h"

-
struct pkg_conflict_chain {
-
	struct pkg_job_request *req;
-
	struct pkg_conflict_chain *next;
-
};
-

+
typedef tll(struct pkg_job_request *) conflict_chain_t;

TREE_DEFINE(pkg_jobs_conflict_item, entry);

@@ -59,36 +55,36 @@ pkg_conflicts_sipkey_init(void)
}

static int
-
pkg_conflicts_chain_cmp_cb(struct pkg_conflict_chain *a, struct pkg_conflict_chain *b)
+
pkg_conflicts_chain_cmp_cb(struct pkg_job_request *a, struct pkg_job_request *b)
{
	const char *vera, *verb;

-
	if (a->req->skip || b->req->skip) {
-
		return (a->req->skip - b->req->skip);
+
	if (a->skip || b->skip) {
+
		return (a->skip - b->skip);
	}

-
	vera = a->req->item->pkg->version;
-
	verb = b->req->item->pkg->version;
+
	vera = a->item->pkg->version;
+
	verb = b->item->pkg->version;

	/* Inverse sort to get the maximum version as the first element */
	return (pkg_version_cmp(vera, verb));
}

static int
-
pkg_conflicts_request_resolve_chain(struct pkg *req, struct pkg_conflict_chain *chain)
+
pkg_conflicts_request_resolve_chain(struct pkg *req, conflict_chain_t *chain)
{
-
	struct pkg_conflict_chain *elt, *selected = NULL;
+
	struct pkg_job_request *selected = NULL;
	const char *slash_pos;

	/*
	 * First of all prefer pure origins, where the last element of
	 * an origin is pkg name
	 */
-
	LL_FOREACH(chain, elt) {
-
		slash_pos = strrchr(elt->req->item->pkg->origin, '/');
+
	tll_foreach(*chain, e) {
+
		slash_pos = strrchr(e->item->item->pkg->origin, '/');
		if (slash_pos != NULL) {
			if (strcmp(slash_pos + 1, req->name) == 0) {
-
				selected = elt;
+
				selected = e->item;
				break;
			}
		}
@@ -97,44 +93,33 @@ pkg_conflicts_request_resolve_chain(struct pkg *req, struct pkg_conflict_chain *
	if (selected == NULL) {
		/* XXX: add manual selection here */
		/* Sort list by version of package */
-
		LL_SORT(chain, pkg_conflicts_chain_cmp_cb);
-
		selected = chain;
+
		tll_sort(*chain, pkg_conflicts_chain_cmp_cb);
+
		selected = tll_front(*chain);
	}

	pkg_debug(2, "select %s in the chain of conflicts for %s",
-
	    selected->req->item->pkg->name, req->name);
+
	    selected->item->pkg->name, req->name);
	/* Disable conflicts from a request */
-
	LL_FOREACH(chain, elt) {
-
		if (elt != selected)
-
			elt->req->skip = true;
+
	tll_foreach(*chain, e) {
+
		if (e->item != selected)
+
			e->item->skip = true;
	}

	return (EPKG_OK);
}

-
static void
-
pkg_conflicts_request_add_chain(struct pkg_conflict_chain **chain, struct pkg_job_request *req)
-
{
-
	struct pkg_conflict_chain *elt;
-

-
	elt = xcalloc(1, sizeof(struct pkg_conflict_chain));
-
	elt->req = req;
-
	LL_PREPEND(*chain, elt);
-
}
-

int
pkg_conflicts_request_resolve(struct pkg_jobs *j)
{
	struct pkg_job_request *req, *found;
	struct pkg_conflict *c;
-
	struct pkg_conflict_chain *chain;
	struct pkg_job_universe_item *unit;
	pkghash_it it;

	it = pkghash_iterator(j->request_add);
	while (pkghash_next(&it)) {
		req = it.value;
-
		chain = NULL;
+
		conflict_chain_t  chain = tll_init();
		if (req->skip)
			continue;

@@ -143,59 +128,25 @@ pkg_conflicts_request_resolve(struct pkg_jobs *j)
			if (unit != NULL) {
				found = pkghash_get_value(j->request_add, unit->pkg->uid);
				if (found != NULL && !found->skip) {
-
					pkg_conflicts_request_add_chain(&chain, found);
+
					tll_push_front(chain, found);
				}
			}
		}
-
		if (chain != NULL) {
+
		if (tll_length(chain) > 0) {
			/* Add package itself */
-
			pkg_conflicts_request_add_chain(&chain, req);
+
			tll_push_front(chain, req);

-
			if (pkg_conflicts_request_resolve_chain(req->item->pkg, chain) != EPKG_OK) {
-
				LL_FREE(chain, free);
+
			if (pkg_conflicts_request_resolve_chain(req->item->pkg, &chain) != EPKG_OK) {
+
				tll_free(chain);
				return (EPKG_FATAL);
			}
-
			LL_FREE(chain, free);
		}
+
		tll_free(chain);
	}

	return (EPKG_OK);
}

-
void
-
pkg_conflicts_register(struct pkg *p1, struct pkg *p2, enum pkg_conflict_type type)
-
{
-
	struct pkg_conflict *c1, *c2;
-

-
	c1 = xcalloc(1, sizeof(*c1));
-
	c2 = xcalloc(1, sizeof(*c2));
-

-
	c1->type = c2->type = type;
-
	if (pkghash_get(p1->conflictshash, p2->uid) == NULL) {
-
		c1->uid = xstrdup(p2->uid);
-
		pkghash_safe_add(p1->conflictshash, c1->uid, c1, NULL);
-
		DL_APPEND(p1->conflicts, c1);
-
		pkg_debug(2, "registering conflict between %s(%s) and %s(%s)",
-
				p1->uid, p1->type == PKG_INSTALLED ? "l" : "r",
-
				p2->uid, p2->type == PKG_INSTALLED ? "l" : "r");
-
	} else {
-
		pkg_conflict_free(c1);
-
	}
-

-
	if (pkghash_get(p2->conflictshash, p1->uid) == NULL) {
-
		c2->uid = xstrdup(p1->uid);
-
		pkghash_safe_add(p2->conflictshash, c2->uid, c2, NULL);
-
		DL_APPEND(p2->conflicts, c2);
-
		pkg_debug(2, "registering conflict between %s(%s) and %s(%s)",
-
				p2->uid, p2->type == PKG_INSTALLED ? "l" : "r",
-
				p1->uid, p1->type == PKG_INSTALLED ? "l" : "r");
-
	} else {
-
		pkg_conflict_free(c2);
-
	}
-
}
-

-

-

static int
pkg_conflicts_item_cmp(struct pkg_jobs_conflict_item *a,
	struct pkg_jobs_conflict_item *b)
@@ -246,51 +197,38 @@ pkg_conflicts_need_conflict(struct pkg_jobs *j, struct pkg *p1, struct pkg *p2)
	return (false);
}

-
/*
-
 * Just insert new conflicts items to the packages
-
 */
static void
-
pkg_conflicts_register_unsafe(struct pkg *p1, struct pkg *p2,
-
	const char *path,
-
	enum pkg_conflict_type type,
-
	bool use_digest)
+
pkg_conflicts_register_one(struct pkg *p, struct pkg *op,
+
    enum pkg_conflict_type type)
{
-
	struct pkg_conflict *c1, *c2;
-

-
	c1 = pkghash_get_value(p1->conflictshash, p2->uid);
-
	c2 = pkghash_get_value(p2->conflictshash, p1->uid);
-
	if (c1 == NULL) {
-
		c1 = xcalloc(1, sizeof(*c1));
-
		c1->type = type;
-
		c1->uid = xstrdup(p2->uid);
-

-
		if (use_digest) {
-
			c1->digest = xstrdup(p2->digest);
-
		}
+
	struct pkg_conflict *conflict;

-
		pkghash_safe_add(p1->conflictshash, c1->uid, c1, NULL);
-
		DL_APPEND(p1->conflicts, c1);
-
	}
-

-
	if (c2 == NULL) {
-
		c2 = xcalloc(1, sizeof(*c2));
-
		c2->type = type;
-

-
		c2->uid = xstrdup(p1->uid);
+
	conflict = pkghash_get_value(p->conflictshash, op->uid);
+
	if (conflict != NULL)
+
		return;

-
		if (use_digest) {
-
			/* We also add digest information into account */
+
	conflict = xcalloc(1, sizeof(*conflict));
+
	conflict->type = type;
+
	conflict->uid = xstrdup(op->uid);
+
	conflict->digest = xstrdup(op->digest);

-
			c2->digest = xstrdup(p1->digest);
-
		}
+
	pkghash_safe_add(p->conflictshash, op->uid, conflict, NULL);
+
	DL_APPEND(p->conflicts, conflict);
+
}

-
		pkghash_safe_add(p2->conflictshash, c2->uid, c2, NULL);
-
		DL_APPEND(p2->conflicts, c2);
-
	}
+
/*
+
 * Record the existence of a file conflict between a pair of packages.
+
 */
+
static void
+
pkg_conflicts_register(struct pkg *p1, struct pkg *p2, const char *path,
+
    enum pkg_conflict_type type)
+
{
+
	pkg_conflicts_register_one(p1, p2, type);
+
	pkg_conflicts_register_one(p2, p1, type);

	pkg_debug(2, "registering conflict between %s(%s) and %s(%s) on path %s",
-
			p1->uid, p1->type == PKG_INSTALLED ? "l" : "r",
-
			p2->uid, p2->type == PKG_INSTALLED ? "l" : "r", path);
+
	    p1->uid, p1->type == PKG_INSTALLED ? "local" : "remote",
+
	    p2->uid, p2->type == PKG_INSTALLED ? "local" : "remote", path);
}

/*
@@ -301,44 +239,32 @@ pkg_conflicts_register_chain(struct pkg_jobs *j, struct pkg_job_universe_item *u
	struct pkg_job_universe_item *u2, const char *path)
{
	struct pkg_job_universe_item *cur1, *cur2;
+
	enum pkg_conflict_type type;
	bool ret = false;

	cur1 = u1;
-

	do {
-

		cur2 = u2;
		do {
			struct pkg *p1 = cur1->pkg, *p2 = cur2->pkg;

-
			if (p1->type == PKG_INSTALLED && p2->type == PKG_INSTALLED) {
-
				/* Local and local packages cannot conflict */
-
				cur2 = cur2->prev;
-
				continue;
-
			}
-
			else if (p1->type == PKG_INSTALLED || p2->type == PKG_INSTALLED) {
-
				/* local <-> remote conflict */
-
				if (pkg_conflicts_need_conflict(j, p1, p2)) {
-
					pkg_emit_conflicts(p1, p2, path);
-
					pkg_conflicts_register_unsafe(p1, p2, path,
-
						PKG_CONFLICT_REMOTE_LOCAL, true);
-
					j->conflicts_registered ++;
-
					ret = true;
-
				}
-
			}
-
			else {
-
				/* two remote packages */
-
				if (pkg_conflicts_need_conflict(j, p1, p2)) {
-
					pkg_emit_conflicts(p1, p2, path);
-
					pkg_conflicts_register_unsafe(p1, p2, path,
-
						PKG_CONFLICT_REMOTE_REMOTE, true);
-
					j->conflicts_registered ++;
-
					ret = true;
-
				}
+
			if (p1->type == PKG_INSTALLED && p2->type == PKG_INSTALLED)
+
				type = PKG_CONFLICT_LOCAL_LOCAL;
+
			else if (p1->type == PKG_INSTALLED || p2->type == PKG_INSTALLED)
+
				type = PKG_CONFLICT_REMOTE_LOCAL;
+
			else
+
				type = PKG_CONFLICT_REMOTE_REMOTE;
+

+
			/* A pair of installed packages cannot conflict. */
+
			if (type != PKG_CONFLICT_LOCAL_LOCAL &&
+
			    pkg_conflicts_need_conflict(j, p1, p2)) {
+
				pkg_emit_conflicts(p1, p2, path);
+
				pkg_conflicts_register(p1, p2, path, type);
+
				j->conflicts_registered++;
+
				ret = true;
			}
			cur2 = cur2->prev;
		} while (cur2 != u2);
-

		cur1 = cur1->prev;
	} while (cur1 != u1);

modified libpkg/pkg_jobs_universe.c
@@ -47,8 +47,6 @@

#define IS_DELETE(j) ((j)->type == PKG_JOBS_DEINSTALL || (j)->type == PKG_JOBS_AUTOREMOVE)

-
typedef tll(struct pkg *) pkg_chain_t;
-

struct pkg *
pkg_jobs_universe_get_local(struct pkg_jobs_universe *universe,
	const char *uid, unsigned flag)
@@ -58,13 +56,10 @@ pkg_jobs_universe_get_local(struct pkg_jobs_universe *universe,
	struct pkg_job_universe_item *unit, *cur, *found;

	if (flag == 0) {
-
		if (!IS_DELETE(universe->j))
-
			flag = PKG_LOAD_BASIC|PKG_LOAD_DEPS|PKG_LOAD_RDEPS|PKG_LOAD_OPTIONS|
-
				PKG_LOAD_REQUIRES|PKG_LOAD_PROVIDES|
-
				PKG_LOAD_SHLIBS_REQUIRED|PKG_LOAD_SHLIBS_PROVIDED|PKG_LOAD_ANNOTATIONS|
-
				PKG_LOAD_CONFLICTS;
-
		else
-
			flag = PKG_LOAD_BASIC|PKG_LOAD_RDEPS|PKG_LOAD_DEPS|PKG_LOAD_ANNOTATIONS;
+
		flag = PKG_LOAD_BASIC|PKG_LOAD_DEPS|PKG_LOAD_RDEPS|PKG_LOAD_OPTIONS|
+
			PKG_LOAD_REQUIRES|PKG_LOAD_PROVIDES|
+
			PKG_LOAD_SHLIBS_REQUIRED|PKG_LOAD_SHLIBS_PROVIDED|PKG_LOAD_ANNOTATIONS|
+
			PKG_LOAD_CONFLICTS;
	}

	unit = pkghash_get_value(universe->items, uid);
modified libpkg/pkg_repo_create.c
@@ -72,8 +72,8 @@ struct digest_list_entry {
	long files_pos;
	long manifest_length;
	char *checksum;
-
	struct digest_list_entry *prev, *next;
};
+
typedef tll(struct digest_list_entry *) digest_list_t;

struct pkg_conflict_bulk {
	struct pkg_conflict *conflicts;
@@ -524,7 +524,7 @@ cleanup:
}

static int
-
pkg_create_repo_read_pipe(int fd, struct digest_list_entry **dlist, struct pkg_fts_item **items)
+
pkg_create_repo_read_pipe(int fd, digest_list_t *dlist, struct pkg_fts_item **items)
{
	struct digest_list_entry *dig = NULL;
	char buf[1024];
@@ -576,7 +576,7 @@ pkg_create_repo_read_pipe(int fd, struct digest_list_entry **dlist, struct pkg_f
			dig->manifest_length = mp_decode_uint(&rbuf);
			c = mp_decode_str(&rbuf, &len);
			dig->checksum = xstrndup(c, len);
-
			DL_APPEND(*dlist, dig);
+
			tll_push_back(*dlist, dig);
		}

		if (msgtype == MSG_PKG_READY) {
@@ -626,7 +626,7 @@ pkg_create_repo(char *path, const char *output_dir, bool filelist,
	struct pkg_conflict_bulk *curcb;
	int num_workers, i, remaining_workers;
	size_t len, ntask;
-
	struct digest_list_entry *dlist = NULL, *cur_dig, *dtmp;
+
	digest_list_t dlist = tll_init();
	struct pollfd *pfd = NULL;
	int cur_pipe[2], fd, outputdir_fd, mfd, ffd;
	struct pkg_repo_meta *meta = NULL;
@@ -821,7 +821,7 @@ pkg_create_repo(char *path, const char *output_dir, bool filelist,

	/* Now sort all digests */
	if (meta->version == 1)
-
		DL_SORT(dlist, pkg_digest_sort_compare_func);
+
		tll_sort(dlist, pkg_digest_sort_compare_func);

	/* Write metafile */
	snprintf(repodb, sizeof(repodb), "%s/%s", output_dir,
@@ -868,22 +868,23 @@ cleanup:
	LL_FREE(fts_items, pkg_create_repo_fts_free);

	if (meta->version == 1) {
-
		LL_FOREACH_SAFE(dlist, cur_dig, dtmp) {
-
			if (cur_dig->checksum != NULL)
-
				fprintf(mandigests, "%s:%s:%ld:%ld:%ld:%s\n", cur_dig->origin,
-
				    cur_dig->digest, cur_dig->manifest_pos, cur_dig->files_pos,
-
				    cur_dig->manifest_length, cur_dig->checksum);
+
		tll_foreach(dlist, it) {
+
			if (it->item->checksum != NULL)
+
				fprintf(mandigests, "%s:%s:%ld:%ld:%ld:%s\n", it->item->origin,
+
				    it->item->digest, it->item->manifest_pos, it->item->files_pos,
+
				    it->item->manifest_length, it->item->checksum);
			else
-
				fprintf(mandigests, "%s:%s:%ld:%ld:%ld\n", cur_dig->origin,
-
				    cur_dig->digest, cur_dig->manifest_pos, cur_dig->files_pos,
-
				    cur_dig->manifest_length);
+
				fprintf(mandigests, "%s:%s:%ld:%ld:%ld\n", it->item->origin,
+
				    it->item->digest, it->item->manifest_pos, it->item->files_pos,
+
				    it->item->manifest_length);

-
			free(cur_dig->digest);
-
			free(cur_dig->origin);
-
			free(cur_dig);
+
			free(it->item->digest);
+
			free(it->item->origin);
+
			free(it->item);
		}
	}

+
	tll_free(dlist);
	if (meta->version == 1 && mandigests != NULL)
		fclose(mandigests);
	pkg_repo_meta_free(meta);
modified libpkg/pkg_solve.c
@@ -1264,7 +1264,7 @@ pkg_solve_insert_res_job (struct pkg_solve_variable *var,
				res->items[0] = add_var->unit;
				res->type = (j->type == PKG_JOBS_FETCH) ?
								PKG_SOLVED_FETCH : PKG_SOLVED_INSTALL;
-
				DL_APPEND(j->jobs, res);
+
				tll_push_back(j->jobs, res);
				pkg_debug(3, "pkg_solve: schedule installation of %s %s",
					add_var->uid, add_var->digest);
			}
@@ -1273,7 +1273,7 @@ pkg_solve_insert_res_job (struct pkg_solve_variable *var,
				res->items[0] = add_var->unit;
				res->items[1] = del_var->unit;
				res->type = PKG_SOLVED_UPGRADE;
-
				DL_APPEND(j->jobs, res);
+
				tll_push_back(j->jobs, res);
				pkg_debug(3, "pkg_solve: schedule upgrade of %s from %s to %s",
					del_var->uid, del_var->digest, add_var->digest);
			}
@@ -1294,7 +1294,7 @@ pkg_solve_insert_res_job (struct pkg_solve_variable *var,
				res = xcalloc(1, sizeof(struct pkg_solved));
				res->items[0] = cur_var->unit;
				res->type = PKG_SOLVED_DELETE;
-
				DL_APPEND(j->jobs, res);
+
				tll_push_back(j->jobs, res);
				pkg_debug(3, "pkg_solve: schedule deletion of %s %s",
					cur_var->uid, cur_var->digest);
				j->count ++;
modified libpkg/pkgdb.c
@@ -336,6 +336,8 @@ static int
pkgdb_init(sqlite3 *sdb)
{
	const char	sql[] = ""
+
	"PRAGMA journal_mode = TRUNCATE;"
+
	"PRAGMA synchronous = FULL;"
	"BEGIN;"
	"CREATE TABLE packages ("
		"id INTEGER PRIMARY KEY,"
@@ -1244,6 +1246,7 @@ typedef enum _sql_prstmt_index {
	ANNOTATE1,
	ANNOTATE2,
	ANNOTATE_ADD1,
+
	ANNOTATE_MOD1,
	ANNOTATE_DEL1,
	ANNOTATE_DEL2,
	CONFLICT,
@@ -1420,6 +1423,15 @@ static sql_prstmt sql_prepared_statements[PRSTMT_LAST] = {
		" (SELECT annotation_id FROM annotation WHERE annotation = ?3))",
		"TTTT", // "TTT"???
	},
+
	[ANNOTATE_MOD1] = {
+
		NULL,
+
		"INSERT OR REPLACE INTO pkg_annotation(package_id, tag_id, value_id) "
+
		"VALUES ("
+
		" (SELECT id FROM packages WHERE name = ?1 ),"
+
		" (SELECT annotation_id FROM annotation WHERE annotation = ?2),"
+
		" (SELECT annotation_id FROM annotation WHERE annotation = ?3))",
+
		"TTTT", // "TTT"???
+
	},
	[ANNOTATE_DEL1] = {
		NULL,
		"DELETE FROM pkg_annotation WHERE "
@@ -2148,29 +2160,30 @@ pkgdb_modify_annotation(struct pkgdb *db, struct pkg *pkg, const char *tag,
	if (pkgdb_transaction_begin_sqlite(db->sqlite, NULL) != EPKG_OK)
		return (EPKG_FATAL);

-
	if (run_prstmt(ANNOTATE_DEL1, pkg->uid, tag) != SQLITE_DONE
-
	    ||
-
	    run_prstmt(ANNOTATE1, tag) != SQLITE_DONE
+
	if (run_prstmt(ANNOTATE1, tag) != SQLITE_DONE
	    ||
	    run_prstmt(ANNOTATE1, value) != SQLITE_DONE
	    ||
-
	    run_prstmt(ANNOTATE_ADD1, pkg->uid, tag, value) !=
-
	        SQLITE_DONE
-
	    ||
-
	    run_prstmt(ANNOTATE_DEL2) != SQLITE_DONE) {
-
		ERROR_STMT_SQLITE(db->sqlite, STMT(ANNOTATE_DEL2));
+
	    run_prstmt(ANNOTATE_MOD1, pkg->uid, tag, value) !=
+
	        SQLITE_DONE) {
+
		ERROR_STMT_SQLITE(db->sqlite, STMT(ANNOTATE_MOD1));
		pkgdb_transaction_rollback_sqlite(db->sqlite, NULL);

		return (EPKG_FATAL);
	}
+
	rows_changed = sqlite3_changes(db->sqlite);

-
	/* Something has gone very wrong if rows_changed != 1 here */
+
	if (run_prstmt(ANNOTATE_DEL2) != SQLITE_DONE) {
+
		ERROR_STMT_SQLITE(db->sqlite, STMT(ANNOTATE_DEL2));
+
		pkgdb_transaction_rollback_sqlite(db->sqlite, NULL);

-
	rows_changed = sqlite3_changes(db->sqlite);
+
		return (EPKG_FATAL);
+
	}

	if (pkgdb_transaction_commit_sqlite(db->sqlite, NULL) != EPKG_OK)
		return (EPKG_FATAL);

+
	/* Something has gone very wrong if rows_changed != 1 here */
	return (rows_changed == 1 ? EPKG_OK : EPKG_WARN);
}

@@ -2914,8 +2927,6 @@ int
pkgdb_begin_solver(struct pkgdb *db)
{
	const char solver_sql[] = ""
-
		"PRAGMA synchronous = OFF;"
-
		"PRAGMA journal_mode = MEMORY;"
		"BEGIN TRANSACTION;";
	const char update_digests_sql[] = ""
		"DROP INDEX IF EXISTS pkg_digest_id;"
@@ -2981,9 +2992,7 @@ int
pkgdb_end_solver(struct pkgdb *db)
{
	const char solver_sql[] = ""
-
		"END TRANSACTION;"
-
		"PRAGMA synchronous = NORMAL;"
-
		"PRAGMA journal_mode = DELETE;";
+
		"END TRANSACTION;";

	return (sql_exec(db->sqlite, solver_sql));
}
modified libpkg/pkgdb_iterator.c
@@ -1302,19 +1302,14 @@ pkgdb_ensure_loaded_sqlite(sqlite3 *sqlite, struct pkg *pkg, unsigned flags)
int
pkgdb_ensure_loaded(struct pkgdb *db, struct pkg *pkg, unsigned flags)
{
-
	int ret;
-

-
	if (pkg->type == PKG_INSTALLED) {
+
	if (pkg->type == PKG_INSTALLED)
		return (pkgdb_ensure_loaded_sqlite(db->sqlite, pkg, flags));
-
	}
-
	else {
-
		/* Call repo functions */
-
		tll_foreach(db->repos, cur) {
-
			if (cur->item == pkg->repo) {
-
				if (cur->item->ops->ensure_loaded) {
-
					ret = cur->item->ops->ensure_loaded(cur->item, pkg, flags);
-
					return (ret);
-
				}
+

+
	tll_foreach(db->repos, cur) {
+
		if (cur->item == pkg->repo) {
+
			if (cur->item->ops->ensure_loaded) {
+
				return (cur->item->ops->ensure_loaded(cur->item,
+
				    pkg, flags));
			}
		}
	}
modified libpkg/pkgdb_query.c
@@ -62,14 +62,11 @@ const char *
pkgdb_get_pattern_query(const char *pattern, match_t match)
{
	char		*checkorigin = NULL;
-
	char		*checkuid = NULL;
	char		*checkflavor = NULL;
	const char	*comp = NULL;

	if (pattern != NULL) {
-
		checkuid = strchr(pattern, '~');
-
		if (checkuid == NULL)
-
			checkorigin = strchr(pattern, '/');
+
		checkorigin = strchr(pattern, '/');
		if (checkorigin != NULL)
			checkflavor = strchr(checkorigin, '@');
	}
@@ -83,32 +80,24 @@ pkgdb_get_pattern_query(const char *pattern, match_t match)
		break;
	case MATCH_EXACT:
		if (pkgdb_case_sensitive()) {
-
			if (checkuid == NULL) {
-
				if (checkorigin == NULL)
-
					comp = " WHERE (p.name = ?1 OR p.name || '-' || version = ?1)";
-
				else if (checkflavor == NULL)
-
					comp = " WHERE (origin = ?1 OR categories.name || substr(origin, instr(origin, '/')) = ?1)";
-
				else
-
					comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor = ?1)";
-
			} else {
-
				comp = " WHERE p.name = ?1";
-
			}
+
			if (checkorigin == NULL)
+
				comp = " WHERE (p.name = ?1 OR p.name || '-' || version = ?1)";
+
			else if (checkflavor == NULL)
+
				comp = " WHERE (origin = ?1 OR categories.name || substr(origin, instr(origin, '/')) = ?1)";
+
			else
+
				comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor = ?1)";
		} else {
-
			if (checkuid == NULL) {
-
				if (checkorigin == NULL)
-
					comp = " WHERE (p.name = ?1 COLLATE NOCASE OR "
-
					"p.name || '-' || version = ?1 COLLATE NOCASE)";
-
				else if (checkflavor == NULL)
-
					comp = " WHERE (origin = ?1 COLLATE NOCASE OR categories.name || substr(origin, instr(origin, '/'))  = ?1 COLLATE NOCASE)";
-
				else
-
					comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor = ?1 COLLATE NOCASE)";
-
			} else {
-
				comp = " WHERE p.name = ?1 COLLATE NOCASE";
-
			}
+
			if (checkorigin == NULL)
+
				comp = " WHERE (p.name = ?1 COLLATE NOCASE OR "
+
				"p.name || '-' || version = ?1 COLLATE NOCASE)";
+
			else if (checkflavor == NULL)
+
				comp = " WHERE (origin = ?1 COLLATE NOCASE OR categories.name || substr(origin, instr(origin, '/'))  = ?1 COLLATE NOCASE)";
+
			else
+
				comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor = ?1 COLLATE NOCASE)";
		}
		break;
	case MATCH_GLOB:
-
		if (checkuid == NULL) {
+
		if (pkgdb_case_sensitive()) {
			if (checkorigin == NULL)
				comp = " WHERE (p.name GLOB ?1 "
					"OR p.name || '-' || version GLOB ?1)";
@@ -116,23 +105,25 @@ pkgdb_get_pattern_query(const char *pattern, match_t match)
				comp = " WHERE (origin GLOB ?1 OR categories.name || substr(origin, instr(origin, '/')) GLOB ?1)";
			else
				comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor GLOB ?1)";
-
		} else {
-
			comp = " WHERE p.name GLOB ?1";
-
		}
-
		break;
-
	case MATCH_REGEX:
-
		if (checkuid == NULL) {
+
		} else  {
			if (checkorigin == NULL)
-
				comp = " WHERE (p.name REGEXP ?1 "
-
				    "OR p.name || '-' || version REGEXP ?1)";
+
				comp = " WHERE (p.name GLOB ?1 COLLATE NOCASE "
+
					"OR p.name || '-' || version GLOB ?1 COLLATE NOCASE)";
			else if (checkflavor == NULL)
-
				comp = " WHERE (origin REGEXP ?1 OR categories.name || substr(origin, instr(origin, '/')) REGEXP ?1)";
+
				comp = " WHERE (origin GLOB ?1 COLLATE NOCASE OR categories.name || substr(origin, instr(origin, '/')) GLOB ?1 COLLATE NOCASE)";
			else
-
				comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor REGEXP ?1)";
-
		} else {
-
			comp = " WHERE p.name REGEXP ?1";
+
				comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor GLOB ?1 COLLATE NOCASE)";
		}
		break;
+
	case MATCH_REGEX:
+
		if (checkorigin == NULL)
+
			comp = " WHERE (p.name REGEXP ?1 "
+
			    "OR p.name || '-' || version REGEXP ?1)";
+
		else if (checkflavor == NULL)
+
			comp = " WHERE (origin REGEXP ?1 OR categories.name || substr(origin, instr(origin, '/')) REGEXP ?1)";
+
		else
+
			comp = "WHERE (categories.name || substr(origin, instr(origin, '/')) || '@' || flavor REGEXP ?1)";
+
		break;
	}

	return (comp);
modified libpkg/pkghash.c
@@ -141,8 +141,8 @@ pkghash_expand(pkghash *table)
bool
pkghash_add(pkghash *table, const char *key, void *value, void (*free_func)(void *))
{
-
	if (table->count >= table->capacity -1 && !pkghash_expand(table))
-
		return (NULL);
+
	if (table->count * 2 >= table->capacity && !pkghash_expand(table))
+
		return (false);

	return (pkghash_set_entry(table->entries, table->capacity, key, value,
	    &table->count, free_func));
@@ -151,7 +151,7 @@ pkghash_add(pkghash *table, const char *key, void *value, void (*free_func)(void
size_t
pkghash_count(pkghash *table)
{
-
	if (table == 0)
+
	if (table == NULL)
		return (0);
	return (table->count);
}
modified libpkg/private/lua.h
@@ -1,5 +1,5 @@
/*-
-
 * Copyright (c) 2019 Baptiste Daroussin <bapt@FreeBSD.org>
+
 * Copyright (c) 2019-2022 Baptiste Daroussin <bapt@FreeBSD.org>
 * All rights reserved.
 * 
 * Redistribution and use in source and binary forms, with or without
@@ -31,6 +31,7 @@ lua_CFunction stack_dump(lua_State *L);
int lua_print_msg(lua_State *L);
int lua_pkg_copy(lua_State *L);
int lua_pkg_filecmp(lua_State *L);
+
int lua_pkg_symlink(lua_State *L);
int lua_prefix_path(lua_State *L);
int lua_exec(lua_State *L);
void lua_override_ios(lua_State *L, bool);
modified libpkg/private/pkg.h
@@ -158,6 +158,7 @@ struct pkg_ctx {
	int compression_level;
	int rootfd;
	int cachedirfd;
+
	int devnullfd;
	int dbdirfd;
	int pkg_dbdirfd;
	int osversion;
@@ -242,6 +243,7 @@ struct pkg {
	pkg_t		 type;
	struct pkg_repo		*repo;
};
+
typedef tll(struct pkg *) pkg_chain_t;

typedef enum {
	SCRIPT_UNKNOWN = 0,
@@ -608,16 +610,14 @@ int pkg_get_myarch_legacy(char *pkgarch, size_t sz);
/**
 * Remove and unregister the package.
 * @param pkg An installed package to delete
+
 * @param rpkg A package which will replace pkg, or NULL
 * @param db An opened pkgdb
-
 * @param force If set to one, the function will not fail if the package is
-
 * required by other packages.
 * @return An error code.
 */
-
int pkg_delete(struct pkg *pkg, struct pkgdb *db, unsigned flags, struct triggers *);
-
#define PKG_DELETE_FORCE (1<<0)
-
#define PKG_DELETE_UPGRADE (1<<1)
-
#define PKG_DELETE_NOSCRIPT (1<<2)
-
#define PKG_DELETE_CONFLICT (1<<3)
+
int pkg_delete(struct pkg *pkg, struct pkg *rpkg, struct pkgdb *db, int flags,
+
    struct triggers *);
+
#define PKG_DELETE_UPGRADE	(1 << 1)	/* delete as a split upgrade */
+
#define PKG_DELETE_NOSCRIPT	(1 << 2)	/* don't run delete scripts */

int pkg_fetch_file_to_fd(struct pkg_repo *repo, const char *url, int dest,
    time_t *t, ssize_t offset, int64_t size, bool silent);
@@ -671,8 +671,6 @@ void pkg_option_free(struct pkg_option *);
void pkg_conflict_free(struct pkg_conflict *);
void pkg_config_file_free(struct pkg_config_file *);

-
int pkg_jobs_resolv(struct pkg_jobs *jobs);
-

struct packing;

int packing_init(struct packing **pack, const char *path, pkg_formats format, int clevel, time_t timestamp, bool overwrite, bool archive_symlink);
@@ -687,7 +685,8 @@ pkg_formats packing_format_from_string(const char *str);
const char* packing_format_to_string(pkg_formats format);
bool packing_is_valid_format(const char *str);

-
int pkg_delete_files(struct pkg *pkg, unsigned force, struct triggers *t);
+
int pkg_delete_files(struct pkg *pkg, struct pkg *rpkg, int flags,
+
    struct triggers *t);
int pkg_delete_dirs(struct pkgdb *db, struct pkg *pkg, struct pkg *p);

/* pkgdb commands */
@@ -754,7 +753,7 @@ int pkg_add_from_remote(struct pkgdb *db, const char *path, unsigned flags,
    struct pkg_manifest_key *keys, const char *location, struct pkg *rp,
    struct triggers *);
void pkg_delete_dir(struct pkg *pkg, struct pkg_dir *dir);
-
void pkg_delete_file(struct pkg *pkg, struct pkg_file *file, unsigned force);
+
void pkg_delete_file(struct pkg *pkg, struct pkg_file *file);
int pkg_open_root_fd(struct pkg *pkg);
void pkg_add_dir_to_del(struct pkg *pkg, const char *file, const char *dir);
struct plist *plist_new(struct pkg *p, const char *stage);
modified libpkg/private/pkg_jobs.h
@@ -29,6 +29,7 @@
#include <stdbool.h>
#include <utlist.h>
#include <ucl.h>
+
#include <tllist.h>

#include "private/utils.h"
#include "private/pkg.h"
@@ -56,14 +57,16 @@ struct pkg_job_request_item {
struct pkg_job_request {
	struct pkg_job_request_item *item;
	bool skip;
+
	bool processed;
	bool automatic;
};

struct pkg_solved {
	struct pkg_job_universe_item *items[2]; /* to-add/to-delete */
+
	struct pkg_solved *xlink;	/* link split jobs together */
	pkg_solved_t type;
-
	struct pkg_solved *prev, *next;
};
+
typedef tll(struct pkg_solved *) pkg_solved;

struct pkg_job_provide {
	struct pkg_job_universe_item *un;
@@ -97,7 +100,7 @@ struct pkg_jobs {
	struct pkg_jobs_universe *universe;
	pkghash	*request_add;
	pkghash	*request_delete;
-
	struct pkg_solved *jobs;
+
	pkg_solved	 jobs;
	struct pkgdb	*db;
	pkg_jobs_t	 type;
	pkg_flags	 flags;
@@ -186,8 +189,6 @@ void pkg_jobs_universe_change_uid(struct pkg_jobs_universe *universe,
	struct pkg_job_universe_item *unit,
	const char *new_uid, bool update_rdeps);

-

-

/*
 * Find local package in db or universe
 */
@@ -204,15 +205,11 @@ int pkg_conflicts_request_resolve(struct pkg_jobs *j);
 */
int pkg_conflicts_append_chain(struct pkg_job_universe_item *it,
	struct pkg_jobs *j);
+

/*
 * Perform integrity check for the jobs specified
 */
int pkg_conflicts_integrity_check(struct pkg_jobs *j);
-
/*
-
 * Register a conflict between two packages
-
 */
-
void pkg_conflicts_register(struct pkg *p1, struct pkg *p2,
-
		enum pkg_conflict_type type);

/*
 * Check whether `rp` is an upgrade for `lp`
modified libpkg/private/utils.h
@@ -53,8 +53,17 @@ typedef tll(struct hardlink *) hardlinks_t;
struct hardlink {
	ino_t ino;
	dev_t dev;
+
	const char *path;
};

+
struct tempdir {
+
	char name[MAXHOSTNAMELEN];
+
	char temp[MAXHOSTNAMELEN];
+
	size_t len;
+
	int fd;
+
};
+
typedef tll(struct tempdir *) tempdirs_t;
+

struct dns_srvinfo {
	unsigned int type;
	unsigned int class;
@@ -117,5 +126,6 @@ bool copy_file(int from, int to);
void hidden_tempfile(char *buf, int buflen, const char *path);
void append_random_suffix(char *buf, int buflen, int suffixlen);
char *json_escape(const char *str);
+
struct tempdir *open_tempdir(int rootfd, const char *path);

#endif
modified libpkg/repo/binary/init.c
@@ -482,7 +482,12 @@ pkg_repo_binary_init(struct pkg_repo *repo)

	sqlite3_create_function(sqlite, "file_exists", 2, SQLITE_ANY, NULL,
		    sqlite_file_exists, NULL, NULL);
-
	retcode = sql_exec(sqlite, "PRAGMA synchronous=default");
+

+
	retcode = sql_exec(sqlite, "PRAGMA journal_mode=TRUNCATE;");
+
	if (retcode != EPKG_OK)
+
		return (retcode);
+

+
	retcode = sql_exec(sqlite, "PRAGMA synchronous=FULL");
	if (retcode != EPKG_OK)
		return (retcode);

modified libpkg/repo/binary/query.c
@@ -295,7 +295,10 @@ pkg_repo_binary_search_how(match_t match)
			how = "%s = ?1 COLLATE NOCASE";
		break;
	case MATCH_GLOB:
-
		how = "%s GLOB ?1";
+
		if (pkgdb_case_sensitive())
+
			how = "%s GLOB ?1";
+
		else
+
			how = "%s GLOB ?1 COLLATE NOCASE";
		break;
	case MATCH_REGEX:
		how = "%s REGEXP ?1";
@@ -468,13 +471,18 @@ pkg_repo_binary_ensure_loaded(struct pkg_repo *repo,

	/* Now move required elements to the provided package */
	pkg_list_free(pkg, PKG_FILES);
+
	pkg_list_free(pkg, PKG_CONFIG_FILES);
	pkg_list_free(pkg, PKG_DIRS);
	pkg->files = cached->files;
	pkg->filehash = cached->filehash;
+
	pkg->config_files = cached->config_files;
+
	pkg->config_files_hash = cached->config_files_hash;
	pkg->dirs = cached->dirs;
	pkg->dirhash = cached->dirhash;
	cached->files = NULL;
	cached->filehash = NULL;
+
	cached->config_files = NULL;
+
	cached->config_files_hash = NULL;
	cached->dirs = NULL;
	cached->dirhash = NULL;

@@ -484,7 +492,6 @@ pkg_repo_binary_ensure_loaded(struct pkg_repo *repo,
	return EPKG_OK;
}

-

int64_t
pkg_repo_binary_stat(struct pkg_repo *repo, pkg_stats_t type)
{
modified libpkg/repo/binary/update.c
@@ -519,7 +519,8 @@ pkg_repo_binary_update_proceed(const char *name, struct pkg_repo *repo,
	sql_exec(sqlite, "PRAGMA mmap_size = 209715200;");
	sql_exec(sqlite, "PRAGMA page_size = %d;", getpagesize());
	sql_exec(sqlite, "PRAGMA foreign_keys = OFF;");
-
	sql_exec(sqlite, "PRAGMA synchronous = OFF;");
+
	sql_exec(sqlite, "PRAGMA journal_mode = TRUNCATE;");
+
	sql_exec(sqlite, "PRAGMA synchronous = FULL;");

	rc = pkgdb_transaction_begin_sqlite(sqlite, "REPO");
	if (rc != EPKG_OK)
modified libpkg/scripts.c
@@ -61,7 +61,6 @@ pkg_script_run(struct pkg * const pkg, pkg_script type, bool upgrade)
	const char *argv[4];
	char **ep;
	int ret = EPKG_OK;
-
	int fd = -1;
	int stdin_pipe[2] = {-1, -1};
	posix_spawn_file_actions_t action;
	bool use_pipe = 0;
@@ -161,7 +160,7 @@ pkg_script_run(struct pkg * const pkg, pkg_script type, bool upgrade)
			 * opened fd close all unuseful fd up to there
			 */
			for (int i = 5; i <= cur_pipe[1]; i++) {
-
				if (i != cur_pipe[0])
+
				if (i != cur_pipe[0] && i != ctx.devnullfd)
					posix_spawn_file_actions_addclose(&action, i);
			}
			if (script_len > argmax) {
@@ -181,15 +180,8 @@ pkg_script_run(struct pkg * const pkg, pkg_script type, bool upgrade)

				use_pipe = 1;
			} else {
-
				fd = open("/dev/null", O_RDWR);
-
				if (fd < 0) {
-
					pkg_errno("Cannot open %s", "/dev/null");
-
					ret = EPKG_FATAL;
-
					posix_spawn_file_actions_destroy(&action);
-
					goto cleanup;
-
				}
				posix_spawn_file_actions_adddup2(&action,
-
				    fd, STDIN_FILENO);
+
				    ctx.devnullfd, STDIN_FILENO);

				argv[0] = _PATH_BSHELL;
				argv[1] = "-c";
@@ -209,8 +201,6 @@ pkg_script_run(struct pkg * const pkg, pkg_script type, bool upgrade)
			}
			posix_spawn_file_actions_destroy(&action);

-
			if (fd != -1)
-
				close(fd);
			if (use_pipe) {
				script_cmd_p = script_cmd->buf;
				while (script_len > 0) {
modified libpkg/triggers.c
@@ -443,6 +443,7 @@ trigger_execute_lua(const char *script, bool sandbox, pkghash *args)
			{ "stat", lua_stat },
			{ "readdir", lua_readdir },
			{ "exec", lua_exec },
+
			{ "symlink", lua_pkg_symlink },
			{ NULL, NULL },
		};
		luaL_newlib(L, pkg_lib);
modified libpkg/utils.c
@@ -1001,3 +1001,50 @@ json_escape(const char *str)
	return (xstring_get(buf));
}

+

+
struct tempdir *
+
open_tempdir(int rootfd, const char *path)
+
{
+
	struct stat st;
+
	char walk[MAXPATHLEN];
+
	char *dir;
+
	size_t cnt = 0;
+
	
+
	strlcpy(walk, path, sizeof(walk));
+
	while ((dir = strrchr(walk, '/')) != NULL) {
+
		struct tempdir *t;
+
		*dir = '\0';
+
		cnt++;
+
		/* accept symlinks pointing to directories */
+
		if (strlen(walk) == 0 && cnt == 1)
+
			break;
+
		if (strlen(walk) > 0) {
+
			if (fstatat(rootfd, RELATIVE_PATH(walk), &st, 0) == -1) 
+
				continue;
+
			if (S_ISDIR(st.st_mode) && cnt == 1)
+
				break;
+
			if (!S_ISDIR(st.st_mode))
+
				continue;
+
		}
+
		*dir = '/';
+
		t = xcalloc(1, sizeof(*t));
+
		hidden_tempfile(t->temp, sizeof(t->temp), walk);
+
		if (mkdirat(rootfd, RELATIVE_PATH(t->temp), 0755) == -1) {
+
			pkg_errno("Fail to create temporary directory: %s", t->temp);
+
			free(t);
+
			return (NULL);
+
		}
+

+
		strlcpy(t->name, walk, sizeof(t->name));
+
		t->len = strlen(t->name);
+
		t->fd = openat(rootfd, RELATIVE_PATH(t->temp), O_DIRECTORY);
+
		if (t->fd == -1) {
+
			pkg_errno("Fail to open directory %s", t->temp);
+
			free(t);
+
			return (NULL);
+
		}
+
		return (t);
+
	}
+
	errno = 0;
+
	return (NULL);
+
}
modified scripts/completion/_pkg.bash.in
@@ -93,18 +93,6 @@ _pkgng_autoremove () {
    large_info=""
}

-
_pkgng_backup () {
-
    local cur prev opts lopts
-
    COMPREPLY=()
-

-
    opts=('-r' '-d')
-
    lopts=('-r[Restore the local package database]' 
-
           '-d[Dumps the local package database]')
-

-
    small_info="Backs-up and restores the local package database"
-
    large_info=""
-
}
-

_pkgng_check () {
    local cur prev opts lopts
    COMPREPLY=()
@@ -373,7 +361,7 @@ _pkg () {
    prev="${COMP_WORDS[COMP_CWORD-1]}"
    
    # init opts for first completion
-
    opts='add audit autoremove backup check clean create delete
+
    opts='add audit autoremove check clean create delete
          fetch help info install query rquery search set shell
          shlib stats update updating upgrade version which'

@@ -381,7 +369,6 @@ _pkg () {
    lopts=( 'add[Registers a package and installs it on the system]'
            'audit[Reports vulnerable packages]'
            'autoremove[Removes orphan packages]'
-
            'backup[Backs-up and restores the local package database]'
            'check[Checks for missing dependencies and database consistency]'
            'clean[Cleans old packages from the cache]'
            'convert[Convert database from/to pkgng]'
@@ -428,14 +415,6 @@ _pkg () {
	    COMPREPLY=( $(compgen) ) && \
		return 0 ;;

-
        backup) 
-
	    COMPREPLY=(
-
		'(-r)-d[Dumps the local package database]'
-
		'(-d)-r[Restore the local package database]'
-
	    )
-
	    return 0 
-
	    ;;
-

        check) 
	    COMPREPLY=(
		'-B[reanalyse the shared libraries]' 
modified scripts/completion/_pkg.in
@@ -48,7 +48,6 @@ _pkg_subcommands() {
		'annotate:add, modify or delete tag-value style annotations on packages'
		'audit:report vulnerable packages'
		'autoremove:remove orphaned packages'
-
		'backup:back-up and restore the local package database'
		'check:check for missing dependencies and database consistency'
		'clean:clean old packages from the cache'
		'config:display value of a configuration option'
@@ -248,13 +247,6 @@ _pkg_args() {
				'(-y --yes -n --dry-run)'{-n,--dry-run}'[assume no (dry run) when asked for confirmation]'
			return
			;;
-
		(backup)
-
			_arguments -s \
-
				'(-q --quiet)'{-q,--quiet}'[force quiet output]' \
-
				'(-d --dump -r --restore)'{-d+,--dump=}'[dump the local package database]:destination:_files' \
-
				'(-d --dump -r --restore)'{-r+,--restore=}'[restore the local package database]:destination:_files'
-
			return
-
			;;
		(check)
			_arguments -A '-*' -s \
				'(-B --shlibs)'{-B,--shlibs}'[reanalyze the shared libraries]' \
modified src/Makefile.autosetup
@@ -5,7 +5,6 @@ SRCS= add.c \
	annotate.c \
	audit.c \
	autoremove.c \
-
	backup.c \
	check.c \
	clean.c \
	config.c \
modified src/audit.c
@@ -70,7 +70,7 @@ static const char* vop_names[] = {
void
usage_audit(void)
{
-
	fprintf(stderr, "Usage: pkg audit [-RFqr] [--raw[=format]|-R[format]| [-f file] <pattern>\n\n");
+
	fprintf(stderr, "Usage: pkg audit [-RFqr] [--raw[=format]|-R[format] [-f file] <pattern>\n\n");
	fprintf(stderr, "For more information see 'pkg help audit'.\n");
}

@@ -190,7 +190,7 @@ format_issue(struct pkg_audit_issue *issue, ucl_object_t *array)
		ucl_object_insert_key(o, ucl_object_fromstring(e->url), "url", 3, false);
	else if (e->id) {
		char *url;
-
		xasprintf(&url, "https://vuxml.FreeBSD.org/freebsd/%s.html\n\n", e->id);
+
		xasprintf(&url, "https://vuxml.FreeBSD.org/freebsd/%s.html", e->id);
		ucl_object_insert_key(o, ucl_object_fromstring(url), "url", 3, false);
		free(url);
	}
@@ -455,7 +455,10 @@ exec_audit(int argc, char **argv)
				}
			}
			pkg_audit_issues_free(issues);
-
			pkg_free(pkg);
+
		}
+
		hit = pkghash_iterator(check);
+
		while (pkghash_next(&hit)) {
+
			pkg_free(hit.value);
		}
		pkghash_destroy(check);

deleted src/backup.c
@@ -1,103 +0,0 @@
-
/*-
-
 * Copyright (c) 2011-2012 Baptiste Daroussin <bapt@FreeBSD.org>
-
 * Copyright (c) 2011-2012 Marin Atanasov Nikolov <dnaeon@gmail.com>
-
 * Copyright (c) 2014 Matthew Seaman <matthew@FreeBSD.org>
-
 * All rights reserved.
-
 * 
-
 * Redistribution and use in source and binary forms, with or without
-
 * modification, are permitted provided that the following conditions
-
 * are met:
-
 * 1. Redistributions of source code must retain the above copyright
-
 *    notice, this list of conditions and the following disclaimer
-
 *    in this position and unchanged.
-
 * 2. Redistributions in binary form must reproduce the above copyright
-
 *    notice, this list of conditions and the following disclaimer in the
-
 *    documentation and/or other materials provided with the distribution.
-
 * 
-
 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR
-
 * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
-
 * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
-
 * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT,
-
 * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
-
 * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
-
 * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
 */
-

-
#include <pkg.h>
-
#include <getopt.h>
-
#include <unistd.h>
-

-
#include "pkgcli.h"
-

-
void
-
usage_backup(void)
-
{
-
	fprintf(stderr, "Usage: pkg backup [-q] -d <dest_file>\n");
-
	fprintf(stderr, "       pkg backup [-q] -r <src_file>\n\n");
-
	fprintf(stderr, "For more information see 'pkg help backup'.\n");
-
}
-

-
int
-
exec_backup(int argc, char **argv)
-
{
-
	struct pkgdb	*db = NULL;
-
	char		*backup_file = NULL;
-
	bool		 dump = false;
-
	bool		 restore = false;
-
	int		 ch;
-

-
	struct option longopts[] = {
-
		{ "dump",	required_argument,	NULL,	'd' },
-
		{ "quiet",	no_argument,		NULL,	'q' },
-
		{ "restore",	required_argument,	NULL,	'r' },
-
		{ NULL,		0,			NULL,	0   },
-
	};
-

-
	while ((ch = getopt_long(argc, argv, "+d:qr:", longopts, NULL)) != -1) {
-
		switch (ch) {
-
		case 'd':
-
			dump = true;
-
			backup_file = optarg;
-
			break;
-
		case 'q':
-
			quiet = true;
-
			break;
-
		case 'r':
-
			restore = true;
-
			backup_file = optarg;
-
			break;
-
		default:
-
			usage_backup();
-
			return (EXIT_FAILURE);
-
		}
-
	}
-

-
	if ( dump == restore ) {
-
		usage_backup();
-
		return (EXIT_FAILURE);
-
	}
-

-
	if (pkgdb_open(&db, PKGDB_DEFAULT) != EPKG_OK)
-
		return (EXIT_FAILURE);
-

-
	if (dump) {
-
		if (!quiet)
-
			printf("Dumping database:\n");
-
		if (pkgdb_dump(db, backup_file) == EPKG_FATAL)
-
			return (EXIT_FAILURE);
-
	}
-

-
	if (restore) {
-
		if (!quiet)
-
			printf("Restoring database:\n");
-
		if (pkgdb_load(db, backup_file) == EPKG_FATAL)
-
			return (EXIT_FAILURE);
-
	}
-

-
	pkgdb_close(db);
-

-
	return (EXIT_SUCCESS);
-
}
modified src/lock.c
@@ -32,6 +32,7 @@
#include <unistd.h>

#include <pkg.h>
+
#include <tllist.h>

#include "pkgcli.h"

@@ -101,6 +102,7 @@ do_lock_unlock(struct pkgdb *db, int match, const char *pkgname,
	struct pkg	*pkg = NULL;
	int		 retcode;
	int		 exitcode = EXIT_SUCCESS;
+
	tll(struct pkg *)pkgs = tll_init();

	if (pkgdb_obtain_lock(db, PKGDB_LOCK_EXCLUSIVE) != EPKG_OK) {
		pkgdb_close(db);
@@ -115,10 +117,14 @@ do_lock_unlock(struct pkgdb *db, int match, const char *pkgname,
	}

	while (pkgdb_it_next(it, &pkg, 0) == EPKG_OK) {
+
		tll_push_back(pkgs, pkg);
+
		pkg = NULL;
+
	}
+
	tll_foreach(pkgs, p) {
		if (action == LOCK)
-
			retcode = do_lock(db, pkg);
+
			retcode = do_lock(db, p->item);
		else
-
			retcode = do_unlock(db, pkg);
+
			retcode = do_unlock(db, p->item);

		if (retcode != EPKG_OK) {
			exitcode = EXIT_FAILURE;
@@ -127,7 +133,7 @@ do_lock_unlock(struct pkgdb *db, int match, const char *pkgname,
	}

cleanup:
-
	pkg_free(pkg);
+
	tll_free_and_free(pkgs, pkg_free);
	pkgdb_it_free(it);

	pkgdb_release_lock(db, PKGDB_LOCK_EXCLUSIVE);
modified src/main.c
@@ -38,11 +38,6 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
-
#ifdef __FreeBSD__
-
#include <sys/sysctl.h>
-
#include <sys/user.h>
-
#include <sys/proc.h>
-
#endif

#include <assert.h>
#include <ctype.h>
@@ -89,7 +84,6 @@ static struct commands {
	{ "annotate", "Add, modify or delete tag-value style annotations on packages", exec_annotate, usage_annotate},
	{ "audit", "Reports vulnerable packages", exec_audit, usage_audit},
	{ "autoremove", "Removes orphan packages", exec_autoremove, usage_autoremove},
-
	{ "backup", "Backs-up and restores the local package database", exec_backup, usage_backup},
	{ "check", "Checks for missing dependencies and database consistency", exec_check, usage_check},
	{ "clean", "Cleans old packages from the cache", exec_clean, usage_clean},
	{ "config", "Display the value of the configuration options", exec_config, usage_config},
@@ -554,30 +548,6 @@ expand_aliases(int argc, char ***argv)
	return (newargc);
}

-
static
-
bool ptraced(void)
-
{
-
#if defined(__FreeBSD__)
-
	int                 mib[4];
-
	struct kinfo_proc   info;
-
	size_t              size;
-

-
	info.ki_flag = 0;
-

-
	mib[0] = CTL_KERN;
-
	mib[1] = KERN_PROC;
-
	mib[2] = KERN_PROC_PID;
-
	mib[3] = getpid();
-

-
	size = sizeof(info);
-
	sysctl(mib, sizeof(mib) / sizeof(*mib), &info, &size, NULL, 0);
-

-
	return ((info.ki_flag & P_TRACED) != 0 );
-
#else
-
	return (false);
-
#endif
-
}
-

int
main(int argc, char **argv)
{
@@ -698,6 +668,8 @@ main(int argc, char **argv)
	argv += optind;

	pkg_set_debug_level(debug);
+
	if (pkg_open_devnull() != EPKG_OK)
+
		errx(EXIT_FAILURE, "Cannot open dev/null");

	if (version == 1)
		show_version_info(version);
@@ -717,7 +689,7 @@ main(int argc, char **argv)
	optreset = 1;
	optind = 1;

-
	if (debug == 0 && version == 0 && !ptraced())
+
	if (debug == 0 && version == 0)
		start_process_worker(save_argv);

#ifdef HAVE_ARC4RANDOM_STIR
@@ -890,6 +862,8 @@ main(int argc, char **argv)
	if (save_argv != argv)
		free(argv);

+
	pkg_close_devnull();
+

	if (ret == EXIT_SUCCESS && newpkgversion)
		return (EX_NEEDRESTART);

modified src/pkgcli.h
@@ -64,10 +64,6 @@ void usage_audit(void);
int exec_autoremove(int, char **);
void usage_autoremove(void);

-
/* pkg backup */
-
int exec_backup(int, char **);
-
void usage_backup(void);
-

/* pkg check */
int exec_check(int, char **);
void usage_check(void);
modified src/query.c
@@ -475,6 +475,7 @@ format_sql_condition(const char *str, xstring *sqlcond, bool for_remote)
	state_t state = NONE;
	unsigned int bracket_level = 0;
	const char *sqlop;
+
	bool collate_nocase = false;

	fprintf(sqlcond->fp, " WHERE ");
	while (str[0] != '\0') {
@@ -557,7 +558,7 @@ format_sql_condition(const char *str, xstring *sqlcond, bool for_remote)
							fprintf(sqlcond->fp, "(SELECT %s FROM deps AS d WHERE d.package_id=p.id)", sqlop);
							break;
						case 'r':
-
							fprintf(sqlcond->fp, "(SELECT %s FROM deps AS d WHERE d.origin=p.origin)", sqlop);
+
							fprintf(sqlcond->fp, "(SELECT %s FROM deps AS d WHERE d.name=p.name)", sqlop);
							break;
						case 'C':
							fprintf(sqlcond->fp, "(SELECT %s FROM pkg_categories AS d WHERE d.package_id=p.id)", sqlop);
@@ -689,7 +690,9 @@ bad_option:
				fprintf(sqlcond->fp, "%c", str[0]);
				if (str[1] == '=') {
					str++;
-
					fprintf(sqlcond->fp, "%c", str[0]);
+
				} else if (str[1] == '~' && state == NEXT_IS_STRING) {
+
					str++;
+
					collate_nocase = true;
				}
			} else if (str[0] == '!') {
				if (str[1] == '=') {
@@ -707,6 +710,10 @@ bad_option:
				} else {
					state = NEXT_IS_INT;
				}
+
				if (str[0] == '~' && state == NEXT_IS_STRING) {
+
					str++;
+
					collate_nocase = true;
+
				}
			} else {
				fprintf(stderr, "an operator is expected, got %c\n", str[0]);
				return (EPKG_FATAL);
@@ -747,6 +754,10 @@ bad_option:
			    (state == SQUOTEDSTRING && str[0] == '\'')) {
				fprintf(sqlcond->fp, "%c", '\'');
				state = POST_EXPR;
+
				if (collate_nocase) {
+
					fprintf(sqlcond->fp, " COLLATE NOCASE ");
+
					collate_nocase = false;
+
				}
			} else {
				fprintf(sqlcond->fp, "%c", str[0]);
				if (str[0] == '\'')
@@ -760,6 +771,10 @@ bad_option:
	if (state == STRING) {
		fprintf(sqlcond->fp, "%c", '\'');
		state = POST_EXPR;
+
		if (collate_nocase) {
+
			fprintf(sqlcond->fp, " COLLATE NOCASE ");
+
			collate_nocase = false;
+
		}
	}

	if (state != POST_EXPR && state != INT) {
@@ -890,6 +905,7 @@ exec_query(int argc, char **argv)
	const char 		*condition_sql = NULL;
	xstring			*sqlcond = NULL;
	const unsigned int	 q_flags_len = NELEM(accepted_query_flags);
+
	size_t			optscount = 0;

	struct option longopts[] = {
		{ "all",		no_argument,		NULL,	'a' },
@@ -903,6 +919,7 @@ exec_query(int argc, char **argv)
	};

	while ((ch = getopt_long(argc, argv, "+aCe:F:gix", longopts, NULL)) != -1) {
+
		optscount++;
		switch (ch) {
		case 'a':
			match = MATCH_ALL;
@@ -934,6 +951,18 @@ exec_query(int argc, char **argv)
	argc -= optind;
	argv += optind;

+
	if (optscount > 1 && condition != NULL) {
+
		fprintf(stderr, "Usage: pkg query -e <evaluation> <query-format>\n"
+
		    "'-e' flag do not accept any other argument\n");
+
		return (EXIT_FAILURE);
+
	}
+

+
	if ((match == MATCH_ALL || condition != NULL || pkgname != NULL)
+
	    && argc > 1) {
+
		usage_query();
+
		return (EXIT_FAILURE);
+
	}
+

	if (argc == 0) {
		usage_query();
		return (EXIT_FAILURE);
modified src/rquery.c
@@ -122,6 +122,7 @@ exec_rquery(int argc, char **argv)
	bool			 onematched = false;
	bool			 old_quiet;
	bool			 index_output = false;
+
	size_t			 optscount = 0;

	struct option longopts[] = {
		{ "all",		no_argument,		NULL,	'a' },
@@ -139,6 +140,7 @@ exec_rquery(int argc, char **argv)
	portsdir = pkg_object_string(pkg_config_get("PORTSDIR"));

	while ((ch = getopt_long(argc, argv, "+aCgiIxe:r:U", longopts, NULL)) != -1) {
+
		optscount++;
		switch (ch) {
		case 'a':
			match = MATCH_ALL;
@@ -176,6 +178,12 @@ exec_rquery(int argc, char **argv)
	argc -= optind;
	argv += optind;

+
	if (optscount > 1 && condition != NULL) {
+
		fprintf(stderr, "Usage: pkg rquery -e <evaluation> [-r reponame] <query-format>\n"
+
		    "'-e' flag do not accept any other argument\n");
+
		exit(EXIT_FAILURE);
+
	}
+

	if (argc == 0 && !index_output) {
		usage_rquery();
		return (EXIT_FAILURE);
modified tests/frontend/annotate.sh
@@ -3,7 +3,8 @@
. $(atf_get_srcdir)/test_environment.sh

tests_init \
-
	annotate
+
	annotate \
+
	annotate_multiple

annotate_body() {
	for pkg in 'png' 'sqlite3' ; do
@@ -64,6 +65,18 @@ annotate_body() {
	    pkg info -A png

	atf_check \
+
	    -o match:"Modified annotation tagged: TEST1" \
+
	    -e empty \
+
	    -s exit:0 \
+
	    pkg annotate -My sqlite3 TEST1 test1-modified
+

+
	atf_check \
+
	    -o match:"TEST1 +: test1-modified" \
+
	    -e empty \
+
	    -s exit:0 \
+
	    pkg info -A sqlite3
+

+
	atf_check \
	    -o match:"^png-1.5.18: Tag: TEST1 Value: test1-modified$" \
	    -e empty \
	    -s exit:0 \
@@ -95,6 +108,32 @@ annotate_body() {
	    -e empty \
	    pkg info -A png

+
	atf_check \
+
	    -o match:"Deleted annotation tagged: TEST1" \
+
	    -s exit:0 \
+
	    -e empty \
+
	    pkg annotate -Dy sqlite3 TEST1
+

+
	atf_check \
+
	    -o not-match:"TEST1" \
+
	    -o not-match:"TEST2" \
+
	    -s exit:0 \
+
	    -e empty \
+
	    pkg info -A sqlite3
+
}
+

+
annotate_multiple_body() {
+
	for pkg in 'png' 'sqlite3' ; do
+
	    atf_check \
+
		-o match:".*Installing.*\.\.\.$" \
+
		-e empty \
+
		-s exit:0 \
+
		pkg register -t -M ${RESOURCEDIR}/$pkg.ucl
+
	done
+

+
	[ -f "./local.sqlite" ] || \
+
	    atf_fail "Can't populate $PKG_DBDIR/local.sqlite"
+

	# Check multiple annotations
	atf_check \
	    -o match:"^png-1.5.18: added annotation tagged: TEST1$" \
@@ -111,6 +150,20 @@ annotate_body() {
	    pkg annotate --all --show TEST1

	atf_check \
+
	    -o match:"^png-1.5.18: Modified annotation tagged: TEST1$" \
+
	    -o match:"^sqlite3-3.8.6: Modified annotation tagged: TEST1$" \
+
	    -e empty \
+
	    -s exit:0 \
+
	    pkg annotate -aMy TEST1 test1-changed
+

+
	atf_check \
+
	    -o match:"^png-1.5.18: Tag: TEST1 Value: test1-changed$" \
+
	    -o match:"^sqlite3-3.8.6: Tag: TEST1 Value: test1-changed$" \
+
	    -e empty \
+
	    -s exit:0 \
+
	    pkg annotate --all --show TEST1
+

+
	atf_check \
	    -o match:"^png-1.5.18: Deleted annotation tagged: TEST1$" \
	    -o match:"^sqlite3-3.8.6: Deleted annotation tagged: TEST1$" \
	    -e empty \
modified tests/frontend/configmerge.sh
@@ -3,6 +3,7 @@
. $(atf_get_srcdir)/test_environment.sh
tests_init \
	config \
+
	config_conflict \
	config_duplicate \
	config_duplicate_keyword \
	config_fileexist \
@@ -52,6 +53,77 @@ config_body()
		cat ${TMPDIR}/target/${TMPDIR}/a
}

+
# Make sure that config file modifications are preserved across split upgrades.
+
# Here, package "test" contains a config file, and the "other" package introduces a
+
# conflict on upgrade when file "b" moves from "test" to "other".  To resolve the
+
# conflict, pkg must temporarily uninstall "test", and we check that modifications
+
# to the config file are not lost.
+
config_conflict_body()
+
{
+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "test" "test" "1"
+
	echo "@config ${TMPDIR}/a" > plist
+
	echo "${TMPDIR}/b" >> plist
+

+
	echo "be provocative" > a
+
	touch b
+

+
	atf_check \
+
		pkg create -M test.ucl -p plist
+

+
	atf_check \
+
		-o match:"^config" \
+
		pkg info -R --raw-format ucl -F ${TMPDIR}/test-1.pkg
+

+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "test" "test" "2"
+
	echo "@config ${TMPDIR}/a" > plist
+

+
	atf_check \
+
		pkg create -M test.ucl -p plist
+

+
	atf_check \
+
		-o match:"^config" \
+
		pkg info -R --raw-format ucl -F ${TMPDIR}/test-2.pkg
+

+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "other" "other" "1"
+
	echo "${TMPDIR}/c" > plist
+

+
	touch c
+

+
	atf_check \
+
		pkg create -M other.ucl -p plist
+

+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "other" "other" "2"
+
	echo "${TMPDIR}/b" > plist
+
	echo "${TMPDIR}/c" >> plist
+

+
	touch c
+

+
	atf_check \
+
		pkg create -M other.ucl -p plist
+

+
	mkdir ${TMPDIR}/target
+
	unset PKG_DBDIR
+
	atf_check \
+
		pkg -o REPOS_DIR=/dev/null -r ${TMPDIR}/target install -qy ${TMPDIR}/test-1.pkg
+
	atf_check \
+
		pkg -o REPOS_DIR=/dev/null -r ${TMPDIR}/target install -qy ${TMPDIR}/other-1.pkg
+
	test -f ${TMPDIR}/target/${TMPDIR}/a || atf_fail "file absent"
+

+
	echo "be organized" >> ${TMPDIR}/target/${TMPDIR}/a
+

+
	pkg repo .
+
	mkdir reposconf
+
	echo "local: { url: file://${TMPDIR} }" > reposconf/local.conf
+
	atf_check \
+
		pkg -o REPOS_DIR=${TMPDIR}/reposconf -r ${TMPDIR}/target upgrade -qy
+

+
	test -f ${TMPDIR}/target/${TMPDIR}/a || atf_fail "file absent"
+

+
	atf_check \
+
		-o inline:"be provocative\nbe organized\n" \
+
		cat ${TMPDIR}/target/${TMPDIR}/a
+
}
+

config_duplicate_body()
{
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "test" "test" "1"
modified tests/frontend/fingerprint.sh
@@ -8,8 +8,10 @@ tests_init \

setup() {
	local _root=$1
-
        atf_skip_on Darwin Test fails on Darwin
-
        atf_skip_on Linux Test fails on Linux
+
	local _fingerprint
+

+
	atf_skip_on Darwin Test fails on Darwin
+
	atf_skip_on Linux Test fails on Linux

	atf_check -o ignore -e ignore \
		openssl genrsa -out repo.key 2048
@@ -19,10 +21,9 @@ setup() {
	chmod 0400 repo.key
	atf_check -o ignore -e ignore \
		openssl rsa -in repo.key -out repo.pub -pubout
+
	_fingerprint=$(openssl dgst -sha256 -hex repo.pub | sed 's/^.* //')
	echo "function: sha256" > ${_root}/${TMPDIR}/keys/trusted/key
-
	echo -n "fingerprint: " >> ${_root}/${TMPDIR}/keys/trusted/key
-
	openssl dgst -sha256 -hex repo.pub | sed 's/^.* //' >> ${_root}/${TMPDIR}/keys/trusted/key
-
	echo "" >> ${_root}/${TMPDIR}/keys/trusted/key
+
	echo "fingerprint: \"${_fingerprint}\"" >> ${_root}/${TMPDIR}/keys/trusted/key
	mkdir fakerepo

	cat >> sign.sh << EOF
modified tests/frontend/query.sh
@@ -73,33 +73,33 @@ EOF
		pkg query -e "%#r>0" "%n: %rn %rv %ro"

	atf_check \
-
		-o inline:"test: plop 1 plop\n" \
-
		-e empty \
-
		-s exit:0 \
+
		-o empty \
+
		-e match:"Usage:.*" \
+
		-s exit:1 \
		pkg query -e "%#r>0" "%n: %rn %rv %ro" test

	atf_check \
		-o empty \
-
		-e empty \
+
		-e match:"Usage:.*" \
		-s exit:1 \
		pkg query -e "%#r>0" "%n: %rn %rv %ro" plop

	atf_check \
-
		-o inline:"test: plop 1 plop\n" \
-
		-e empty \
-
		-s exit:0 \
+
		-o empty \
+
		-e match:"Usage:.*" \
+
		-s exit:1 \
		pkg query -e "%#r>0" "%n: %rn %rv %ro" plop test

	atf_check \
-
		-o inline:"test: plop 1 plop\n" \
-
		-e empty \
-
		-s exit:0 \
+
		-o empty \
+
		-e inline:"Usage: pkg query -e <evaluation> <query-format>\n'-e' flag do not accept any other argument\n" \
+
		-s exit:1 \
		pkg query -ge "%#r>0" "%n: %rn %rv %ro" "p*p" "t*t"

	atf_check \
-
		-o inline:"test: plop 1 plop\n" \
-
		-e empty \
-
		-s exit:0 \
+
		-o empty \
+
		-e inline:"Usage: pkg query -e <evaluation> <query-format>\n'-e' flag do not accept any other argument\n" \
+
		-s exit:1 \
		pkg query -xe "%#r>0" "%n: %rn %rv %ro" "p.*p" "t.*t"

	atf_check \
@@ -109,9 +109,9 @@ EOF
		pkg query -e "%#d>0" "%n: %dn %dv %do"

	atf_check \
-
		-o inline:"plop: test 1 test\n" \
-
		-e empty \
-
		-s exit:0 \
+
		-o empty \
+
		-e match:"Usage:.*" \
+
		-s exit:1 \
		pkg query -e "%#d>0" "%n: %dn %dv %do" plop test

	atf_check \
modified tests/frontend/requires.sh
@@ -3,7 +3,8 @@
. $(atf_get_srcdir)/test_environment.sh

tests_init \
-
	requires
+
	requires \
+
	requires_del

requires_body() {
	mkdir reposconf
@@ -57,4 +58,87 @@ Number of packages to be installed: 2
	    -o inline:"${OUTPUT}" \
	    -s exit:1 \
	    pkg -o REPOS_DIR="${TMPDIR}/reposconf" install -n b
+

+
	atf_check \
+
	    -o ignore \
+
	    -s exit:0 \
+
	    pkg -o REPOS_DIR="${TMPDIR}/reposconf" install -y b
+

+
	atf_check \
+
		-o match:".*Nothing to do.*" \
+
		-s exit:0 \
+
		pkg autoremove -n
+
}
+

+
requires_del_body() {
+
	mkdir reposconf
+
	cat << EOF >> reposconf/repo.conf
+
local1: {
+
	url: file://${TMPDIR},
+
	enabled: true
+
}
+
EOF
+

+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg a a 1.0
+
	cat << EOF >> a.ucl
+
provides: [a-1]
+
EOF
+

+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg b b 1.0
+
	cat << EOF >> b.ucl
+
requires: [a-1]
+
EOF
+

+
	for p in a b; do
+
		atf_check \
+
		    -o ignore \
+
		    -e empty \
+
		    -s exit:0 \
+
		    pkg create -M ./${p}.ucl
+
	done
+

+
	atf_check \
+
	    -o ignore \
+
	    -e empty \
+
	    -s exit:0 \
+
	    pkg repo .
+

+
	OUTPUT="Updating local1 repository catalogue...
+
${JAILED}Fetching meta.conf:  done
+
${JAILED}Fetching packagesite.pkg:  done
+
Processing entries:  done
+
local1 repository update completed. 2 packages processed.
+
All repositories are up to date.
+
Checking integrity... done (0 conflicting)
+
The following 2 package(s) will be affected (of 0 checked):
+

+
New packages to be INSTALLED:
+
	a: 1.0
+
	b: 1.0
+

+
Number of packages to be installed: 2
+
"
+
	atf_check \
+
	    -o inline:"${OUTPUT}" \
+
	    -s exit:1 \
+
	    pkg -o REPOS_DIR="${TMPDIR}/reposconf" install -n b
+

+
	atf_check \
+
	    -o ignore \
+
	    -s exit:0 \
+
	    pkg -o REPOS_DIR="${TMPDIR}/reposconf" install -y b
+

+
	OUTPUT2="Checking integrity... done (0 conflicting)
+
Deinstallation has been requested for the following 2 packages (of 0 packages in the universe):
+

+
Installed packages to be REMOVED:
+
	a: 1.0
+
	b: 1.0
+

+
Number of packages to be removed: 2
+
"
+
	atf_check \
+
		-o inline:"${OUTPUT2}" \
+
		-s exit:0 \
+
		pkg delete -n a
}
modified tests/frontend/upgrade.sh
@@ -6,7 +6,9 @@ tests_init \
	issue1881 \
	issue1881_newdep \
	three_digit_revision \
-
	dual_conflict
+
	dual_conflict \
+
	file_become_dir \
+
	dir_become_file

issue1881_body() {
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg pkg1 pkg_a 1
@@ -227,3 +229,36 @@ EOF
		-o inline:'pkg-1-2\n' \
		pkg -r ${TMPDIR}/target which -q ${TMPDIR}/file-pkg-2
}
+

+
file_become_dir_body() {
+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "pkg" "pkg" "1"
+
	echo "${TMPDIR}/file-pkg-1" > plist-1
+
	echo "entry" > file-pkg-1
+
	atf_check pkg create -M pkg.ucl -p plist-1
+
	mkdir target
+
	atf_check -o ignore pkg -o REPOS_DIR="${TMPDIR}" -r ${TMPDIR}/target install -Uy ${TMPDIR}/pkg-1.pkg
+
	atf_check test -f target/${TMPDIR}/file-pkg-1
+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "pkg" "pkg" "2"
+
	rm file-pkg-1
+
	mkdir file-pkg-1
+
	echo entry > file-pkg-1/file
+
	echo "${TMPDIR}/file-pkg-1/file" > plist-2
+
	atf_check pkg create -M pkg.ucl -p plist-2
+
	atf_check -o ignore pkg -o REPOS_DIR="${TMPDIR}" -r ${TMPDIR}/target install -Uy ${TMPDIR}/pkg-2.pkg
+
}
+

+
dir_become_file_body() {
+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "pkg" "pkg" "1"
+
	mkdir file-pkg-1
+
	echo entry > file-pkg-1/file
+
	echo "${TMPDIR}/file-pkg-1/file" > plist-1
+
	atf_check pkg create -M pkg.ucl -p plist-1
+
	mkdir target
+
	atf_check -o ignore pkg -o REPOS_DIR="${TMPDIR}" -r ${TMPDIR}/target install -Uy ${TMPDIR}/pkg-1.pkg
+
	atf_check -s exit:0 sh ${RESOURCEDIR}/test_subr.sh new_pkg "pkg" "pkg" "2"
+
	rm -rf file-pkg-1
+
	echo entry > file-pkg-1
+
	echo "${TMPDIR}/file-pkg-1" > plist-2
+
	atf_check pkg create -M pkg.ucl -p plist-2
+
	atf_check -o ignore pkg -o REPOS_DIR="${TMPDIR}" -r ${TMPDIR}/target install -Uy ${TMPDIR}/pkg-2.pkg
+
}
modified tests/lib/checksum.c
@@ -29,18 +29,10 @@
#include <pkg.h>
#include <private/pkg.h>

-
ATF_TC(check_symlinks);
-
ATF_TC(check_pkg);
-
ATF_TC(check_types);
-
ATF_TC(check_validity);
-

-
ATF_TC_HEAD(check_symlinks, tc)
-
{
-
	atf_tc_set_md_var(tc, "descr", "testing checksums on symlinks");
-
}
-
ATF_TC_HEAD(check_pkg, tc) {}
-
ATF_TC_HEAD(check_types, tc) {}
-
ATF_TC_HEAD(check_validity, tc) {}
+
ATF_TC_WITHOUT_HEAD(check_symlinks);
+
ATF_TC_WITHOUT_HEAD(check_pkg);
+
ATF_TC_WITHOUT_HEAD(check_types);
+
ATF_TC_WITHOUT_HEAD(check_validity);

ATF_TC_BODY(check_validity, tc)
{
modified tests/lib/deps_formula.c
@@ -30,14 +30,9 @@
#include <private/pkg.h>
#include <private/pkg_deps.h>

-
ATF_TC(check_parsing);
-
ATF_TC(check_sql);
-
ATF_TC(check_op_parsing);
-

-
ATF_TC_HEAD(check_parsing, tc)
-
{
-
	atf_tc_set_md_var(tc, "descr", "testing parsing of deps formula");
-
}
+
ATF_TC_WITHOUT_HEAD(check_parsing);
+
ATF_TC_WITHOUT_HEAD(check_sql);
+
ATF_TC_WITHOUT_HEAD(check_op_parsing);

ATF_TC_BODY(check_parsing, tc)
{
modified tests/lib/lua.c
@@ -23,6 +23,8 @@
 * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

+
#include <sys/stat.h>
+

#include <atf-c.h>
#include <err.h>
#include <unistd.h>
@@ -31,21 +33,13 @@
#include <fcntl.h>
#include <stdlib.h>

-
ATF_TC(readdir);
-
ATF_TC(stat);
-
ATF_TC(print_msg);
-
ATF_TC(execute);
-
ATF_TC(override);
-
ATF_TC(fileops);
-
ATF_TC(prefix_path);
-

-
ATF_TC_HEAD(readdir, tc) { }
-
ATF_TC_HEAD(stat, tc) {}
-
ATF_TC_HEAD(print_msg, tc) {}
-
ATF_TC_HEAD(execute, tc) {}
-
ATF_TC_HEAD(override, tc) {}
-
ATF_TC_HEAD(fileops, tc) {}
-
ATF_TC_HEAD(prefix_path, tc) {}
+
ATF_TC_WITHOUT_HEAD(readdir);
+
ATF_TC_WITHOUT_HEAD(stat);
+
ATF_TC_WITHOUT_HEAD(print_msg);
+
ATF_TC_WITHOUT_HEAD(execute);
+
ATF_TC_WITHOUT_HEAD(override);
+
ATF_TC_WITHOUT_HEAD(fileops);
+
ATF_TC_WITHOUT_HEAD(prefix_path);

ATF_TC_BODY(readdir, tc)
{
@@ -353,6 +347,7 @@ ATF_TC_BODY(override, tc)

ATF_TC_BODY(fileops, tc)
{
+
	char b[1024];
	int rootfd = open(getcwd(NULL, 0), O_DIRECTORY);
	lua_State *L = luaL_newstate();
	luaL_openlibs(L);
@@ -362,6 +357,7 @@ ATF_TC_BODY(fileops, tc)
	static const luaL_Reg test_lib[] = {
		{ "copy", lua_pkg_copy },
		{ "cmp", lua_pkg_filecmp},
+
		{ "symlink", lua_pkg_symlink},
		{ NULL, NULL },
	};
	luaL_newlib(L, test_lib);
@@ -459,6 +455,42 @@ ATF_TC_BODY(fileops, tc)
		exit(lua_tonumber(L, -1));
	}
	atf_utils_wait(p, 0, "", "");
+

+
	p = atf_utils_fork();
+
	if (p == 0) {
+
		if (luaL_dostring(L, "test.symlink(\"a\", \"b\", \"meh\")\n")) {
+
			printf("%s\n", lua_tostring(L, -1));
+
		}
+
		exit(lua_tonumber(L, -1));
+
	}
+
	atf_utils_wait(p, 0, "[string \"test.symlink(\"a\", \"b\", \"meh\")...\"]:1: bad argument #3 to 'symlink' (pkg.symlink takes exactly two arguments)\n", "");
+

+
	p = atf_utils_fork();
+
	if (p == 0) {
+
		if (luaL_dostring(L, "test.symlink(\"a\")\n")) {
+
			printf("%s\n", lua_tostring(L, -1));
+
		}
+
		exit(lua_tonumber(L, -1));
+
	}
+
	atf_utils_wait(p, 0, "[string \"test.symlink(\"a\")...\"]:1: bad argument #1 to 'symlink' (pkg.symlink takes exactly two arguments)\n", "");
+

+
	p = atf_utils_fork();
+
	if (p == 0) {
+
		if (luaL_dostring(L, "test.symlink(\"a\", \"b\")\n")) {
+
			printf("%s\n", lua_tostring(L, -1));
+
		}
+
		exit(lua_tonumber(L, -1));
+
	}
+
	atf_utils_wait(p, 0, "", "");
+
	struct stat st;
+
	if (lstat("b", &st) != 0)
+
		atf_tc_fail("File 'b' not created");
+
	if (!S_ISLNK(st.st_mode))
+
		atf_tc_fail("File 'b' is not a symlink");
+
	memset(b, 0, sizeof(b));
+
	readlink("b", b, sizeof(b));
+
	ATF_REQUIRE_STREQ(b, "a");
+

}

ATF_TC_BODY(prefix_path, tc)
modified tests/lib/merge.c
@@ -28,13 +28,7 @@
#include <sys/types.h>
#include <private/utils.h>

-
ATF_TC(merge);
-

-
ATF_TC_HEAD(merge, tc)
-
{
-
	atf_tc_set_md_var(tc, "descr",
-
	    "3 way merge");
-
}
+
ATF_TC_WITHOUT_HEAD(merge);

ATF_TC_BODY(merge, tc)
{
modified tests/lib/metalog.c
@@ -28,9 +28,7 @@
#include <private/pkg.h>
#include <pkg_config.h>

-
ATF_TC(basics);
-

-
ATF_TC_HEAD(basics, tc) {}
+
ATF_TC_WITHOUT_HEAD(basics);

ATF_TC_BODY(basics, tc) {
#ifdef HAVE_FFLAGSTOSTR
modified tests/lib/packing.c
@@ -30,15 +30,10 @@
#include <private/packing.h>
#include <pkg_config.h>

-
ATF_TC(packing_format_from_string);
-
ATF_TC(packing_format_to_string);
-
ATF_TC(packing_format_is_valid);
-
ATF_TC(packing_set_format);
-

-
ATF_TC_HEAD(packing_format_from_string, tc) { }
-
ATF_TC_HEAD(packing_format_to_string, tc) { }
-
ATF_TC_HEAD(packing_format_is_valid, tc) { }
-
ATF_TC_HEAD(packing_set_format, tc) {}
+
ATF_TC_WITHOUT_HEAD(packing_format_from_string);
+
ATF_TC_WITHOUT_HEAD(packing_format_to_string);
+
ATF_TC_WITHOUT_HEAD(packing_format_is_valid);
+
ATF_TC_WITHOUT_HEAD(packing_set_format);

ATF_TC_BODY(packing_format_from_string, tc)
{
modified tests/lib/pkg_add_dir_to_del.c
@@ -29,13 +29,7 @@
#include <private/pkg.h>
#include <tllist.h>

-
ATF_TC(pkg_add_dir_to_del);
-

-
ATF_TC_HEAD(pkg_add_dir_to_del, tc)
-
{
-
	atf_tc_set_md_var(tc, "descr",
-
	    "pkg_add_dir_to_del()");
-
}
+
ATF_TC_WITHOUT_HEAD(pkg_add_dir_to_del);

ATF_TC_BODY(pkg_add_dir_to_del, tc)
{
modified tests/lib/pkg_printf_test.c
@@ -31,12 +31,8 @@
#include <pkg.h>
#include <private/pkg_printf.h>

-
ATF_TC(gen_format);
-
ATF_TC_HEAD(gen_format, tc)
-
{
-
	atf_tc_set_md_var(tc, "descr",
-
			  "Generate printf format code for final output");
-
}
+
ATF_TC_WITHOUT_HEAD(gen_format);
+

ATF_TC_BODY(gen_format, tc)
{
	char		 buf[32];
modified tests/lib/pkg_validation.c
@@ -42,13 +42,7 @@

xstring *msg;

-
ATF_TC(valid_installed);
-

-
ATF_TC_HEAD(valid_installed, tc)
-
{
-
	atf_tc_set_md_var(tc, "descr",
-
	    "pkg_valid() tests installed packages");
-
}
+
ATF_TC_WITHOUT_HEAD(valid_installed);

int
event_callback(void *data __unused, struct pkg_event *ev)
modified tests/lib/plist.c
@@ -28,13 +28,7 @@
#include <pkg.h>
#include <private/pkg.h>

-
ATF_TC(parse_mode);
-

-
ATF_TC_HEAD(parse_mode, tc)
-
{
-
	atf_tc_set_md_var(tc, "descr",
-
	    "parse_mode()");
-
}
+
ATF_TC_WITHOUT_HEAD(parse_mode);

ATF_TC_BODY(parse_mode, tc)
{
modified tests/lib/ssh.c
@@ -32,17 +32,11 @@
#include <stdlib.h>
#include <xmalloc.h>

-
ATF_TC(badcommand);
-
ATF_TC(badrestrict);
-
ATF_TC(getfile);
-
ATF_TC(notrestricted);
-
ATF_TC(restricted);
-

-
ATF_TC_HEAD(badcommand, tc) { }
-
ATF_TC_HEAD(getfile, tc) { }
-
ATF_TC_HEAD(badrestrict, tc) { }
-
ATF_TC_HEAD(notrestricted, tc) { }
-
ATF_TC_HEAD(restricted, tc) { }
+
ATF_TC_WITHOUT_HEAD(badcommand);
+
ATF_TC_WITHOUT_HEAD(badrestrict);
+
ATF_TC_WITHOUT_HEAD(getfile);
+
ATF_TC_WITHOUT_HEAD(notrestricted);
+
ATF_TC_WITHOUT_HEAD(restricted);

ATF_TC_BODY(badcommand, tc)
{
modified tests/lib/test.c
@@ -1,22 +1,14 @@
#include <atf-c.h>
#include "tests.h"

-
ATF_TC(manifest);
-
ATF_TC_HEAD(manifest, tc)
-
{
-
    atf_tc_set_md_var(tc, "descr", "Testing manifest loading...");
-
}
+
ATF_TC_WITHOUT_HEAD(manifest);
+
ATF_TC_WITHOUT_HEAD(pkg);
+

ATF_TC_BODY(manifest, tc)
{
    test_manifest();
}

-
ATF_TC(pkg);
-
ATF_TC_HEAD(pkg, tc)
-
{
-
    atf_tc_set_md_var(tc, "descr", "Testing pkg interface...");
-
}
-

ATF_TC_BODY(pkg, tc)
{
    test_pkg();
modified tests/lib/utils.c
@@ -25,15 +25,13 @@

#include <atf-c.h>
#include <err.h>
+
#include <fcntl.h>
#include <private/utils.h>

-
ATF_TC(hidden_tempfile);
-
ATF_TC(random_suffix);
-
ATF_TC(json_escape);
-

-
ATF_TC_HEAD(hidden_tempfile, tc) {}
-
ATF_TC_HEAD(random_suffix, tc) {}
-
ATF_TC_HEAD(json_escape, tc) {}
+
ATF_TC_WITHOUT_HEAD(hidden_tempfile);
+
ATF_TC_WITHOUT_HEAD(random_suffix);
+
ATF_TC_WITHOUT_HEAD(json_escape);
+
ATF_TC_WITHOUT_HEAD(open_tempdir);

ATF_TC_BODY(hidden_tempfile, tc) {
	const char *filename = "plop";
@@ -71,11 +69,48 @@ ATF_TC_BODY(json_escape, tc) {
	ATF_REQUIRE_STREQ_MSG(m, "entry1\\\"\\\"\\\\ ", "Invalid escaping");
}

+
ATF_TC_BODY(open_tempdir, tc) {
+
	struct tempdir *t;
+
	int rootfd = open(getenv("TMPDIR"), O_DIRECTORY);
+
	ATF_REQUIRE_MSG(rootfd  != -1, "impossible to open TMPDIR");
+
	t = open_tempdir(rootfd, "/plop");
+
	ATF_REQUIRE(t == NULL);
+
	mkdirat(rootfd, "usr", 0755);
+
	t = open_tempdir(rootfd, "/usr/local/directory");
+
	ATF_REQUIRE(t != NULL);
+
	ATF_REQUIRE_STREQ(t->name, "/usr/local");
+
	ATF_REQUIRE_EQ(t->len, strlen("/usr/local"));
+
	ATF_REQUIRE(strncmp(t->temp, "/usr/.pkgtemp.", 14) == 0);
+
	ATF_REQUIRE(t->fd != -1);
+
	close(t->fd);
+
	free(t);
+
	t = open_tempdir(rootfd, "/nousr/local/directory");
+
	ATF_REQUIRE(t != NULL);
+
	ATF_REQUIRE_STREQ(t->name, "/nousr");
+
	ATF_REQUIRE_EQ(t->len, strlen("/nousr"));
+
	ATF_REQUIRE(strncmp(t->temp, "/.pkgtemp.", 10) == 0);
+
	ATF_REQUIRE(t->fd != -1);
+
	close(t->fd);
+
	free(t);
+
	mkdirat(rootfd, "dir", 0755);
+
	/* a file in the path */
+
	close(openat(rootfd, "dir/file1", O_CREAT|O_WRONLY, 0644));
+
	t = open_tempdir(rootfd, "/dir/file1/test");
+
	ATF_REQUIRE(t != NULL);
+
	ATF_REQUIRE_STREQ(t->name, "/dir/file1");
+
	ATF_REQUIRE_EQ(t->len, strlen("/dir/file1"));
+
	ATF_REQUIRE(strncmp(t->temp, "/dir/.pkgtemp.", 14) == 0);
+
	ATF_REQUIRE(t->fd != -1);
+
	close(t->fd);
+
	free(t);
+
}
+

ATF_TP_ADD_TCS(tp)
{
	ATF_TP_ADD_TC(tp, hidden_tempfile);
	ATF_TP_ADD_TC(tp, random_suffix);
	ATF_TP_ADD_TC(tp, json_escape);
+
	ATF_TP_ADD_TC(tp, open_tempdir);

	return (atf_no_error());
}