Repo sync

content/code-security/codeql-cli/codeql-cli-manual/database-analyze.md

@@ -256,7 +256,7 @@ Download any missing queries before analyzing.
 
 A list of threat models to enable or disable.
 
-The argument is the name of a threat model, optionally preceeded by a
+The argument is the name of a threat model, optionally preceded by a
 '!'. If no '!' is present, the named threat model and all of its
 descendants are enabled. If a '!' is present, the named threat model
 and all of its descendants are disabled.
@@ -400,7 +400,13 @@ expense of making it much less human readable.
 
 #### `-M, --ram=<MB>`
 
-Set total amount of RAM the query evaluator should be allowed to use.
+The query evaluator will try hard to keep its total memory footprint
+below this value. (However, for large databases it is possible that the
+threshold may be broken by file-backed memory maps, which can be swapped
+to disk in case of memory pressure).
+
+The value should be at least 2048 MB; smaller values will be
+transparently rounded up.
 
 ### Options to control QL compilation
 

content/code-security/codeql-cli/codeql-cli-manual/database-bundle.md

@@ -48,6 +48,12 @@ that results, logs, TRAP, or similar should be included.
 
 \[Mandatory] The output file, typically with the extension ".zip".
 
+#### `--include-diagnostics`
+
+Include diagnostics in the bundle.
+
+Available since `v2.16.0`.
+
 #### `--include-results`
 
 Include any precomputed query results in the bundle.

content/code-security/codeql-cli/codeql-cli-manual/database-run-queries.md

@@ -105,7 +105,7 @@ during database creation from a Code Scanning configuration file.
 
 A list of threat models to enable or disable.
 
-The argument is the name of a threat model, optionally preceeded by a
+The argument is the name of a threat model, optionally preceded by a
 '!'. If no '!' is present, the named threat model and all of its
 descendants are enabled. If a '!' is present, the named threat model
 and all of its descendants are disabled.
@@ -249,7 +249,13 @@ expense of making it much less human readable.
 
 #### `-M, --ram=<MB>`
 
-Set total amount of RAM the query evaluator should be allowed to use.
+The query evaluator will try hard to keep its total memory footprint
+below this value. (However, for large databases it is possible that the
+threshold may be broken by file-backed memory maps, which can be swapped
+to disk in case of memory pressure).
+
+The value should be at least 2048 MB; smaller values will be
+transparently rounded up.
 
 ### Options to control QL compilation
 

content/code-security/codeql-cli/codeql-cli-manual/database-upgrade.md

@@ -216,7 +216,13 @@ expense of making it much less human readable.
 
 #### `-M, --ram=<MB>`
 
-Set total amount of RAM the query evaluator should be allowed to use.
+The query evaluator will try hard to keep its total memory footprint
+below this value. (However, for large databases it is possible that the
+threshold may be broken by file-backed memory maps, which can be swapped
+to disk in case of memory pressure).
+
+The value should be at least 2048 MB; smaller values will be
+transparently rounded up.
 
 ### Common options
 

content/code-security/codeql-cli/codeql-cli-manual/dataset-upgrade.md

@@ -212,7 +212,13 @@ expense of making it much less human readable.
 
 #### `-M, --ram=<MB>`
 
-Set total amount of RAM the query evaluator should be allowed to use.
+The query evaluator will try hard to keep its total memory footprint
+below this value. (However, for large databases it is possible that the
+threshold may be broken by file-backed memory maps, which can be swapped
+to disk in case of memory pressure).
+
+The value should be at least 2048 MB; smaller values will be
+transparently rounded up.
 
 ### Common options
 

content/code-security/codeql-cli/codeql-cli-manual/execute-queries.md

@@ -88,7 +88,7 @@ stored in the output location.
 
 A list of threat models to enable or disable.
 
-The argument is the name of a threat model, optionally preceeded by a
+The argument is the name of a threat model, optionally preceded by a
 '!'. If no '!' is present, the named threat model and all of its
 descendants are enabled. If a '!' is present, the named threat model
 and all of its descendants are disabled.

content/code-security/codeql-cli/codeql-cli-manual/generate-extensible-predicate-metadata.md

@@ -29,8 +29,8 @@ codeql generate extensible-predicate-metadata <options>... -- <pack-root-dir>
 
 ## Description
 
-\[Experimental] \[Deep plumbing] Report the extensible predicates
-found in the given pack.
+\[Deep plumbing] Report the extensible predicates found in the given
+pack.
 
 ## Options
 

content/code-security/codeql-cli/codeql-cli-manual/github-upload-results.md

@@ -49,9 +49,9 @@ This token must have the `security_events` scope.
 
 \[Mandatory] Path to the SARIF file to upload. This should be the
 output of [codeql database analyze](/code-security/codeql-cli/codeql-cli-manual/database-analyze) (or [codeql database interpret-results](/code-security/codeql-cli/codeql-cli-manual/database-interpret-results)) with `--format sarif-latest` for upload to github.com or
-GitHub AE, or the appropriate supported format tag for GitHub Enterprise
-Server instances (see [AUTOTITLE](/enterprise-server@latest/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning)
-for SARIF versions supported by your release).
+the appropriate supported format tag for GitHub Enterprise Server
+instances (see [AUTOTITLE](/enterprise-server@latest/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning) for
+SARIF versions supported by your release).
 
 #### `-r, --repository=<repository-name>`
 

content/code-security/codeql-cli/codeql-cli-manual/pack-add.md

@@ -32,8 +32,8 @@ codeql pack add <options>... -- <scope/name[@range]>...
 
 ## Description
 
-\[Experimental] Adds a list of QL library packs with optional version
-ranges as dependencies of the current package, and then installs them.
+Adds a list of QL library packs with optional version ranges as
+dependencies of the current package, and then installs them.
 
 This command modifies the qlpack.yml file of the current package.
 Formatting and comments will be removed.

content/code-security/codeql-cli/codeql-cli-manual/pack-bundle.md

@@ -30,7 +30,7 @@ codeql pack bundle [--output=<file.tgz>] [--threads=<num>] [--ram=<MB>] <options
 
 ## Description
 
-\[Experimental] \[Plumbing] Bundle a QL library pack.
+\[Plumbing] Bundle a QL library pack.
 
 This command bundles a QL library pack from the contents of the current
 directory.

content/code-security/codeql-cli/codeql-cli-manual/pack-ci.md

@@ -32,8 +32,8 @@ codeql pack ci [--force] <options>... -- <dir>
 
 ## Description
 
-\[Experimental] Clean install dependencies for this pack, verifying
-that the existing lock file is up to date.
+Clean install dependencies for this pack, verifying that the existing
+lock file is up to date.
 
 This command installs the dependencies of the pack, using the versions
 specified in the codeql-pack.lock.yml file. If any of the versions

content/code-security/codeql-cli/codeql-cli-manual/pack-create.md

@@ -32,8 +32,7 @@ codeql pack create [--output=<output>] [--threads=<num>] [--ram=<MB>] <options>.
 
 ## Description
 
-\[Experimental] \[Plumbing] Builds the contents of a QL package from
-source code.
+\[Plumbing] Builds the contents of a QL package from source code.
 
 This command builds the complete contents of a QL package, including the
 original source code, library dependencies, compiled queries, and

content/code-security/codeql-cli/codeql-cli-manual/pack-download.md

@@ -33,9 +33,9 @@ codeql pack download [--dir=<dir>] [--force] <options>... -- <scope/name[@versio
 
 ## Description
 
-\[Experimental] Download the set of qlpacks referenced by the query
-spec of the command line from the registry. Packs can be provided by
-name or implicitly inside of a query suite (.qls) file.
+Download the set of qlpacks referenced by the query spec of the command
+line from the registry. Packs can be provided by name or implicitly
+inside of a query suite (.qls) file.
 
 The packs will be downloaded and unzipped into the package cache by
 default.

content/code-security/codeql-cli/codeql-cli-manual/pack-init.md

@@ -30,7 +30,7 @@ codeql pack init [--dir=<dir>] [--extractor=<extractor>] <options>... -- <packag
 
 ## Description
 
-\[Experimental] Initializes a qlpack in the specified directory.
+Initializes a qlpack in the specified directory.
 
 The pack will be created in a child directory of the specified
 directory.

content/code-security/codeql-cli/codeql-cli-manual/pack-install.md

@@ -30,7 +30,7 @@ codeql pack install [--force] <options>... -- <dir>
 
 ## Description
 
-\[Experimental] Install dependencies for this pack.
+Install dependencies for this pack.
 
 This command installs the dependencies of the pack. If a
 codeql-pack.lock.yml exists, the versions specified in that lock file

content/code-security/codeql-cli/codeql-cli-manual/pack-ls.md

@@ -33,9 +33,8 @@ codeql pack ls <options>... -- <dir>
 
 ## Description
 
-\[Experimental] \[Deep plumbing] List the CodeQL packages rooted at
-this directory. This directory must contain a qlpack.yml or
-.codeqlmanifest.json file.
+\[Deep plumbing] List the CodeQL packages rooted at this directory.
+This directory must contain a qlpack.yml or .codeqlmanifest.json file.
 
 Available since `v2.7.1`.
 

content/code-security/codeql-cli/codeql-cli-manual/pack-packlist.md

@@ -32,8 +32,8 @@ codeql pack packlist <options>... -- <dir>
 
 ## Description
 
-\[Experimental] \[Plumbing] Compute the set of files to be included in
-a QL query pack or library pack.
+\[Plumbing] Compute the set of files to be included in a QL query pack
+or library pack.
 
 This command determines the set of files to be included in the pack
 based on the patterns specified in any `.gitignore` files present in the

content/code-security/codeql-cli/codeql-cli-manual/pack-publish.md

@@ -30,7 +30,7 @@ codeql pack publish [--dry-run] [--threads=<num>] [--ram=<MB>] [--pack=<folder>
 
 ## Description
 
-\[Experimental] Publishes a QL library pack to a package registry.
+Publishes a QL library pack to a package registry.
 
 This command publishes a pack to a package registry. Before publishing,
 the pack is first compiled (if necessary) and bundled.

content/code-security/codeql-cli/codeql-cli-manual/pack-resolve-dependencies.md

@@ -32,8 +32,7 @@ codeql pack resolve-dependencies <options>... -- <dir>
 
 ## Description
 
-\[Experimental] \[Plumbing] Compute the set of required dependencies
-for this QL pack.
+\[Plumbing] Compute the set of required dependencies for this QL pack.
 
 This command searches the configured registries for required
 dependencies and returns the list of resolved dependencies.

content/code-security/codeql-cli/codeql-cli-manual/pack-upgrade.md

@@ -32,8 +32,7 @@ codeql pack upgrade [--force] <options>... -- <dir>
 
 ## Description
 
-\[Experimental] Update the dependencies for this pack to the latest
-available versions.
+Update the dependencies for this pack to the latest available versions.
 
 This command installs the latest compatible version of each dependency
 of the pack, updating the lock file with the newly acquired versions.

content/code-security/codeql-cli/codeql-cli-manual/query-run.md

@@ -195,7 +195,13 @@ expense of making it much less human readable.
 
 #### `-M, --ram=<MB>`
 
-Set total amount of RAM the query evaluator should be allowed to use.
+The query evaluator will try hard to keep its total memory footprint
+below this value. (However, for large databases it is possible that the
+threshold may be broken by file-backed memory maps, which can be swapped
+to disk in case of memory pressure).
+
+The value should be at least 2048 MB; smaller values will be
+transparently rounded up.
 
 ### Options to control QL compilation
 

content/code-security/codeql-cli/codeql-cli-manual/resolve-extensions-by-pack.md

@@ -30,9 +30,8 @@ codeql resolve extensions-by-pack <options>... -- <pack>...
 
 ## Description
 
-\[Experimental] \[Deep plumbing] Determine accessible extensions for
-the given paths to pack roots. This includes machine learning models and
-data extensions.
+\[Deep plumbing] Determine accessible extensions for the given paths to
+pack roots. This includes machine learning models and data extensions.
 
 This plumbing command resolves the set of data extensions and
 GitHub-created machine learning models that are available to the paths
@@ -91,7 +90,7 @@ value.
 
 A list of threat models to enable or disable.
 
-The argument is the name of a threat model, optionally preceeded by a
+The argument is the name of a threat model, optionally preceded by a
 '!'. If no '!' is present, the named threat model and all of its
 descendants are enabled. If a '!' is present, the named threat model
 and all of its descendants are disabled.

content/code-security/codeql-cli/codeql-cli-manual/resolve-extensions.md

@@ -32,8 +32,8 @@ codeql resolve extensions <options>... -- <query|dir|suite|pack>...
 
 ## Description
 
-\[Experimental] \[Deep plumbing] Determine accessible extensions. This
-includes machine learning models and data extensions.
+\[Deep plumbing] Determine accessible extensions. This includes machine
+learning models and data extensions.
 
 This plumbing command resolves the set of data extensions and
 GitHub-created machine learning models that are available to the query
@@ -110,7 +110,7 @@ value.
 
 A list of threat models to enable or disable.
 
-The argument is the name of a threat model, optionally preceeded by a
+The argument is the name of a threat model, optionally preceded by a
 '!'. If no '!' is present, the named threat model and all of its
 descendants are enabled. If a '!' is present, the named threat model
 and all of its descendants are disabled.

content/code-security/codeql-cli/codeql-cli-manual/resolve-ml-models.md

@@ -32,8 +32,8 @@ codeql resolve ml-models <options>... -- <query|dir|suite|pack>...
 
 ## Description
 
-\[Deprecated] \[Experimental] \[Deep plumbing] Determine accessible
-machine learning models.
+\[Deprecated] \[Deep plumbing] Determine accessible machine learning
+models.
 
 This plumbing command is deprecated. Previously it resolved the set of
 GitHub-created machine learning models that were available to the query

content/code-security/codeql-cli/codeql-cli-manual/resolve-ram.md

@@ -57,7 +57,20 @@ Select output format. Choices include:
 
 #### `-M, --ram=<MB>`
 
-Set total amount of RAM the query evaluator should be allowed to use.
+The query evaluator will try hard to keep its total memory footprint
+below this value. (However, for large databases it is possible that the
+threshold may be broken by file-backed memory maps, which can be swapped
+to disk in case of memory pressure).
+
+The value should be at least 2048 MB; smaller values will be
+transparently rounded up.
+
+#### `--dataset=<directory>`
+
+\[Advanced] Tune the RAM settings appropriately for querying the given
+dataset, taking into account components of RAM usage that scale with the
+size of the database. If this is not given, a generic default size will
+be assumed.
 
 ### Common options