From b1f56bd6740ed952b7fc8aa24dcf95a3cdd27000 Mon Sep 17 00:00:00 2001 From: Techassi Date: Wed, 31 May 2023 13:40:59 +0200 Subject: [PATCH 1/6] Update main README --- README.md | 71 +++++++++++++++++++++++++++++++++++++++++--------- src/README.md | 1 + web/Cargo.toml | 2 -- web/README.md | 2 +- 4 files changed, 61 insertions(+), 15 deletions(-) create mode 100644 src/README.md diff --git a/README.md b/README.md index 17318699..42ddb7e0 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,71 @@ -# (WIP) Stackable +# Stackable Cockpit -This repository contains the Stackable library, the Stackable CLI tool, and the Stackable server. +⚠️ Notice: This repository and all its components are currently WIP. At this point in time, no stable version is +released. Users of `stackablectl` should continue to use the current latest version. A switch can be made when the time +is appropriate. -## Components +This repository contains the Stackable library `stackable`, the Stackable CLI tool `stackablectl`, the Stackable server +`stackabled`, and the web-based admin user interface. -- `stackable-lib`: The underlying library for all actions related to the Stackable Data Platform -- `stackablectl`: CLI tool to interact with local and remote deployments of the data platform. [Link][ctl-readme] -- `stackabled`: API server used by frontends to interact with the data platform. [Link][daemon-readme] +## Components -[daemon-readme]: ./bins/stackabled/README.md -[ctl-readme]: ./bins/stackablectl/README.md +- [`stackable`][lib-readme]: The underlying library for all actions related to the Stackable Data Platform +- [`stackablectl`][ctl-readme]: CLI tool to interact with local and remote deployments of the data platform +- [`stackabled`][server-readme]: API server used by frontends to interact with the data platform +- [`stackabled-web`][web-readme]: The web-based admin UI powered by SolidJS, TypeScript and Vite ## Developer Setup -This repository ships custom Git hooks. They are located in the `.githooks` directory. Before you enable them, make sure -to check out the code they execute. Currently, the `pre-commit` hook runs two tasks: `gen-man` and `gen-comp`. See -[here][xtasks] what these tasks do. To enable all hooks inside the directory, use the following command: +### Prerequisites + +- A working (and up2date) NodeJS installation, with pnpm as the preferred package manager +- A working (and up2date) Rust installation including rustc, clippy, and cargo +- Optional, but strongly advised: a working [pre-commit][pre-commit] installation + +### Getting started ```shell -git config --local core.hooksPath .githooks/ +git clone git@github.com:stackabletech/stackable-cockpit.git +cd stackable-cockpit ``` +The admin UI is registered as a crate and is part of the build process, as the HTML/CSS/JS bundle is included in the +final `stackabled` binary. To get the build process running, first execute `pnpm i` to install all required NodeJS +dependencies in the `node_modules` folder. + +--- + +Each component can be build separately like this: + +```shell +cargo build --release -p stackablectl # Builds the stackablectl +cargo build --release -p stackabled # Builds the Stackable API server +cargo build --release -p stackabled --features ui # Builds the Stackable API server bundled with the admin UI +cd web && pnpm build && cd - # Builds the admin UI +``` + +### Pre-commit hooks and xtasks + +This repository uses multiple pre-commit hooks to run checks, formatting and code-generation on different files. The +hooks are: + +- [`trailing-whitespace`](https://github.com/pre-commit/pre-commit-hooks#trailing-whitespace): Trims trailing whitespace + in all files +- [`end-of-file-fixer`](https://github.com/pre-commit/pre-commit-hooks#end-of-file-fixer): Files need to end with + newlines +- [`detect-aws-credentials`](https://github.com/pre-commit/pre-commit-hooks#detect-aws-credentials): Detect AWS secrets +- [`detect-private-key`](https://github.com/pre-commit/pre-commit-hooks#detect-private-key): Detect private keys +- [`yamllint`](https://github.com/adrienverge/yamllint): Runs linting on all YAML files +- [`markdownlint`](https://github.com/igorshubovych/markdownlint-cli): Runs linting on all Markdown files +- [`prettier`](https://github.com/pre-commit/mirrors-prettier): Runs prettier on files located in `web` +- `cargo clippy -- -D warnings`: Runs Clippy on all files and errors on warnings +- `cargo fmt -- --check`: Checks if Rust code needs formatting +- `cargo xtask gen-comp`: Runs shell completions generation for `stackablectl` +- `cargo xtask gen-mam`: Runs man page generation for `stackablectl` + +[server-readme]: ./bins/stackabled/README.md +[ctl-readme]: ./bins/stackablectl/README.md +[pre-commit]: https://pre-commit.com/ +[web-readme]: ./web/README.md +[lib-readme]: ./src/README.md [xtasks]: ./xtask/src/main.rs diff --git a/src/README.md b/src/README.md new file mode 100644 index 00000000..5c17069c --- /dev/null +++ b/src/README.md @@ -0,0 +1 @@ +# stackable diff --git a/web/Cargo.toml b/web/Cargo.toml index 1f5b5af3..9712afeb 100644 --- a/web/Cargo.toml +++ b/web/Cargo.toml @@ -3,8 +3,6 @@ name = "stackabled-web" version = "0.1.0" edition = "2021" -# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html - [dependencies] phf = { workspace = true } diff --git a/web/README.md b/web/README.md index 7d1429b5..4f0c6faa 100644 --- a/web/README.md +++ b/web/README.md @@ -1,4 +1,4 @@ -# stackable-ui +# stackabled-web ## The stack From cbe4cee184be9bf1ba73d93c2b2eed1858b38ec3 Mon Sep 17 00:00:00 2001 From: Techassi Date: Wed, 31 May 2023 17:06:19 +0200 Subject: [PATCH 2/6] Update stackable-lib README --- src/README.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/src/README.md b/src/README.md index 5c17069c..83af651d 100644 --- a/src/README.md +++ b/src/README.md @@ -1 +1,27 @@ # stackable + +Stackable library powering the `stackablectl` CLI tool and `stackabled` API server. The components are: + +- `cluster`: Cluster-related functions and data structures. Currently two kind of cluster orchestration tools are + supported: + - kind + - minikube +- `common`: Some commonly used types, like `List` and `ManifestSpec` +- `platform`: Stackable Data Platform related code: + - `demo`: Demo related functions and types + - `operator`: Module containing operator related types and functions to install these components using Helm + - `release`: Module containing the `ReleaseSpec` and installation methods which install individual operators + - `stack`: Stacks describe commonly used collections of operators, provides methods to install complete stacks +- `utils`: Various utility functions and helper types like `PathOrUrl` or `read_yaml_data_from_remote` +- `constants`: Constants used across the codebase +- `helm`: A wrapper around the Go Helm library. Provides all Helm-related functions for example for installation of + charts +- `kube`: Functions which make use of direct Kubernetes API access using the `kube_rs` crate + +## Developer setup + +### Testing + +```shell +cargo test -p stackable +``` From 776dc455476953a28a089c8cdd001b4153de13f8 Mon Sep 17 00:00:00 2001 From: Techassi Date: Tue, 6 Jun 2023 16:53:01 +0200 Subject: [PATCH 3/6] Update stackablectl README --- bins/stackablectl/README.md | 104 ++++++++++++++++++++++++++++++++---- 1 file changed, 94 insertions(+), 10 deletions(-) diff --git a/bins/stackablectl/README.md b/bins/stackablectl/README.md index 9a04b996..6864a097 100644 --- a/bins/stackablectl/README.md +++ b/bins/stackablectl/README.md @@ -6,28 +6,112 @@ The documentation of `stackablectl` can be found in the [documentation of the St ## Usage -TODO +The `stackablectl` binary provides extensive help and usage information. This help can be displayed for the root as well +as any subcommands using the `--help` flag. -## Building +``` +Usage: stackablectl [OPTIONS] -The CLI tool `stackablectl` can be build using Cargo: +Commands: + operator Interact with single operator instead of the full platform + release Interact with all operators of the platform which are released together + stack Interact with stacks, which are ready-to-use product combinations + services Interact with deployed services of products + demo Interact with demos, which are end-to-end usage demonstrations of the Stackable data platform + completions Generate shell completions for this tool + cache Interact with locally cached files + help Print this message or the help of the given subcommand(s) -```shell -cargo build --release -p stackablectl +Options: + -l, --log-level + Log level this application uses + + --no-cache + Do not cache the remote (default) demo, stack and release files + + Cached files are saved at '$XDG_CACHE_HOME/stackablectl', which is usually + '$HOME/.cache/stackablectl' when not explicitly set. + + --offline + Do not request any remote files via the network + + -n, --operator-namespace + Namespace in the cluster used to deploy the products and operators + + [default: default] + + -d, --demo-file + Provide one or more additional (custom) demo file(s) + + Demos are loaded in the following order: Remote (default) demo file, custom + demo files provided via the 'STACKABLE_DEMO_FILES' environment variable, and + lastly demo files provided via the '-d/--demo-file' argument(s). If there are + demos with the same name, the last demo definition will be used. + + Use "stackablectl -d path/to/demos1.yaml -d path/to/demos2.yaml [OPTIONS] " + to provide multiple additional demo files. + + -s, --stack-file + Provide one or more additional (custom) stack file(s) + + Stacks are loaded in the following order: Remote (default) stack file, custom + stack files provided via the 'STACKABLE_STACK_FILES' environment variable, and + lastly demo files provided via the '-s/--stack-file' argument(s). If there are + stacks with the same name, the last stack definition will be used. + + Use "stackablectl -s path/to/stacks1.yaml -s path/to/stacks2.yaml [OPTIONS] " + to provide multiple additional stack files. + + -r, --release-file + Provide one or more additional (custom) release file(s) + + Releases are loaded in the following order: Remote (default) release file, + custom release files provided via the 'STACKABLE_RELEASE_FILES' environment + variable, and lastly release files provided via the '-r/--release-file' + argument(s). If there are releases with the same name, the last stack definition + will be used. + + Use "stackablectl -r path/to/releases1.yaml -r path/to/releases2.yaml [OPTIONS] " + to provide multiple additional stack files. + + --helm-repo-stable + Provide a custom Helm stable repository URL + + [default: https://repo.stackable.tech/repository/helm-stable/] + + --helm-repo-test + Provide a custom Helm test repository URL + + [default: https://repo.stackable.tech/repository/helm-test/] + + --helm-repo-dev + Provide a custom Helm dev repository URL + + [default: https://repo.stackable.tech/repository/helm-dev/] + + -h, --help + Print help (see a summary with '-h') + + -V, --version + Print version ``` -### Man Pages +## Dev Setup -This generates man pages in `extra/man`: +### Building + +The CLI tool `stackablectl` can be build using Cargo: ```shell -cargo xtask gen-man +cargo build --release -p stackablectl ``` -### Shell Completions +### Generating man pages and shell completions -This generates shell completions in `extra/completions`. Currently, we generate completions for Bash, Fish and ZSH. +The generation of the man pages and the shell completions is part of pre-commit hooks, but can however be run manually +using the following commands: ```shell cargo xtask gen-comp +cargo xtask gen-man ``` From 022377cf1a5e2e52142df93350ca40557aa0afd2 Mon Sep 17 00:00:00 2001 From: Techassi Date: Tue, 6 Jun 2023 16:55:12 +0200 Subject: [PATCH 4/6] Fix markdownlint error --- bins/stackablectl/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bins/stackablectl/README.md b/bins/stackablectl/README.md index 6864a097..97cabd77 100644 --- a/bins/stackablectl/README.md +++ b/bins/stackablectl/README.md @@ -9,7 +9,7 @@ The documentation of `stackablectl` can be found in the [documentation of the St The `stackablectl` binary provides extensive help and usage information. This help can be displayed for the root as well as any subcommands using the `--help` flag. -``` +```plain Usage: stackablectl [OPTIONS] Commands: From 07059a016d543229f3abb3645d8a914629058907 Mon Sep 17 00:00:00 2001 From: Techassi Date: Wed, 7 Jun 2023 08:51:45 +0200 Subject: [PATCH 5/6] Update README.md Co-authored-by: Sebastian Bernauer --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 42ddb7e0..c1f6a4a9 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ dependencies in the `node_modules` folder. Each component can be build separately like this: ```shell -cargo build --release -p stackablectl # Builds the stackablectl +cargo build --release -p stackablectl # Builds stackablectl cargo build --release -p stackabled # Builds the Stackable API server cargo build --release -p stackabled --features ui # Builds the Stackable API server bundled with the admin UI cd web && pnpm build && cd - # Builds the admin UI From d5e9eefb692b57d5b3f7dfbc90fc217e442d0782 Mon Sep 17 00:00:00 2001 From: Techassi Date: Wed, 7 Jun 2023 08:51:55 +0200 Subject: [PATCH 6/6] Update src/README.md Co-authored-by: Sebastian Bernauer --- src/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/README.md b/src/README.md index 83af651d..73e7ddd8 100644 --- a/src/README.md +++ b/src/README.md @@ -11,7 +11,7 @@ Stackable library powering the `stackablectl` CLI tool and `stackabled` API serv - `demo`: Demo related functions and types - `operator`: Module containing operator related types and functions to install these components using Helm - `release`: Module containing the `ReleaseSpec` and installation methods which install individual operators - - `stack`: Stacks describe commonly used collections of operators, provides methods to install complete stacks + - `stack`: Stacks describe commonly used collections of data tools, provides methods to install complete stacks - `utils`: Various utility functions and helper types like `PathOrUrl` or `read_yaml_data_from_remote` - `constants`: Constants used across the codebase - `helm`: A wrapper around the Go Helm library. Provides all Helm-related functions for example for installation of