Autogenerated documentation for bundle config by ilyakuz-db · Pull Request #2033 · databricks/cli

ilyakuz-db · 2024-12-18T18:24:19Z

Changes

Documentation autogeneration tool. This tool uses same annotations_*.yml files as in json-schema

Result will go there and there

Tests

Manually

eng-dev-ecosystem-bot · 2024-12-18T20:50:54Z

Test Details: go/deco-tests/12401106836

…erence-doc-autogen

github-actions · 2025-01-03T12:53:29Z

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

PR number: 2033
Commit SHA: c6703c13637c0d85e6754e4c802f38052304a5d6

Checks will be approved automatically on success.

…erence-doc-autogen

denik · 2025-01-16T08:56:05Z

 schema:
 	go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

+docs: vendor


docs: vendor

We don't add this implicitly anymore to other targets. If you're on CI, you can always specify both with "make vendor docs".

BTW, do you intended to run it on CI?

BTW, do you intended to run it on CI?

Currently the only use case of using that tool is to checkout CLI repo and run locally

We don't add this implicitly anymore to other targets

By implicitly you mean using make deps or in general?

I mean the make entry should be:

docs: ...

no dependency on vendor

Could you elaborate little bit more on the reasoning behind "We don't add this implicitly anymore to other targets" so I can understand how to better fix it?

Is it just that we don't want to rely on make deps syntax and this is still allowed?

docs: make vendor go run ....

Removed and update the readme

denik · 2025-01-16T08:59:23Z

 	go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

+docs: vendor
+	go run ./bundle/internal/docs ./bundle/internal/schema ./bundle/internal/docs


Why is this part if /bundle/internal/ ?

internal is meant for code that is not meant to be used outside of the package, but this is clearly intended to be used outside the package, that's why it's in Makefile.

How about placing this at top level? /docsgen/ ?

I see you're following existing structure, so it's a more of a question for @pietern

I see you're following existing structure,

Yes I reused same approach as with json schema generation which is used in .codegen.json along with other internal scripts

This is bundle specific, so bundle/docsgen would be more appropriate.

It doesn't need to be internal.

…erence-doc-autogen

ilyakuz-db · 2025-01-20T16:18:25Z

@pietern @denik could you take another look? Want to merge it so docs team could start using this tool

denik · 2025-01-21T13:52:08Z

+)
+
+// Parsed file with annotations, expected format:
+// github.com/databricks/cli/bundle/config.Bundle:


This is copied, right?

It would be much easier to review if PRs was split into

refactoring: copy / renames

new functionality

Yes, this was moved as is from schema package into the new package

It would be much easier to review if PRs was split into

refactoring: copy / renames
new functionality

Agree but it feels too painful to split now. For next PRs I'll try to follow more convenient approach

pietern · 2025-01-27T11:27:00Z

+
+	// Examples of the value for properties in the schema.
+	// https://json-schema.org/understanding-json-schema/reference/annotations
+	Examples []any `json:"examples,omitempty"`


Do you know if the schema check we perform also validates that entries here match the containing schema?

I think it only tests yamls against the schema but not the schema consistency itsef

I.e. this is allowed

In general spec says that example values should validate against schema but it isn't used and not required
https://json-schema.org/understanding-json-schema/reference/annotations see draft 6

Worth to mention that this field currently is not used in schema generation

pietern · 2025-01-27T11:33:23Z

 	go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

+docs: vendor
+	go run ./bundle/internal/docs ./bundle/internal/schema ./bundle/internal/docs


This is bundle specific, so bundle/docsgen would be more appropriate.

It doesn't need to be internal.

…erence-doc-autogen

## Changes Documentation autogeneration tool. This tool uses same annotations_*.yml files as in json-schema Result will go [there](https://docs.databricks.com/en/dev-tools/bundles/reference.html) and [there](https://docs.databricks.com/en/dev-tools/bundles/resources.html#cluster) ## Tests Manually

…5568) `bundle/docsgen` (added in databricks#2033) generated the DABs configuration reference markdown. It has been superseded by the docs-only JSON schema added in databricks#4414 (`bundle/schema/jsonschema_for_docs.json`), which is now the source of truth for the published documentation; docsgen's output is no longer consumed anywhere. This removes the package along with its call sites and references across the repository. This pull request and its description were written by Isaac.

ilyakuz-db added 2 commits December 18, 2024 17:07

tmp: Documentation autogeneration script

8c58d48

fix: Custom table markup changes

e27cc9a

ilyakuz-db temporarily deployed to test-trigger-is December 18, 2024 18:24 — with GitHub Actions Inactive

ilyakuz-db added 2 commits December 18, 2024 19:59

fix: Styling issues

fe4c6b8

fix: Remove openAPI docs

cfa2be3

ilyakuz-db temporarily deployed to test-trigger-is December 18, 2024 19:53 — with GitHub Actions Inactive

ilyakuz-db temporarily deployed to test-trigger-is December 18, 2024 19:54 — with GitHub Actions Inactive

ilyakuz-db added 2 commits December 18, 2024 21:12

fix: Remove extra headings

10e0d27

fix: Small readme

64fa2bf

ilyakuz-db temporarily deployed to test-trigger-is December 18, 2024 20:50 — with GitHub Actions Inactive

ilyakuz-db added 5 commits December 23, 2024 12:25

Merge branch 'main' of github.com:databricks/cli into feat/config-ref…

40c4b3a

…erence-doc-autogen

feat: Add examples to the docs

820dd5f

chore: Extract annotation package

460a455

feat: More explicit type for arrays

541c3e3

feat: Support for resources

2aadfcb

ilyakuz-db requested review from andrewnester, denik, pietern and shreyas-goenka as code owners January 2, 2025 17:24

fix: Updated styles

c6703c1

ilyakuz-db added 6 commits January 7, 2025 16:52

fix: Styling

fe6ba76

fix: Description of root types with additional properties

c355fbf

docs: Add override for volume spec

f9278c2

fix: Missing array types

6c5268a

fix: Sync annotations

bad77bd

Merge branch 'main' of github.com:databricks/cli into feat/config-ref…

954ef76

…erence-doc-autogen

ilyakuz-db added 3 commits January 15, 2025 19:47

fix: Remove type inconsistency in maps

26466d2

fix: Rename fields to make it more clear

d27282c

fix: More ducmentation + tests

13181fc

denik reviewed Jan 16, 2025

View reviewed changes

ilyakuz-db requested a review from pietern January 16, 2025 11:28

ilyakuz-db added 2 commits January 20, 2025 12:26

Merge branch 'main' of github.com:databricks/cli into feat/config-ref…

f04a841

…erence-doc-autogen

fix: Update artifacts

e40b53f

ilyakuz-db temporarily deployed to test-trigger-is January 20, 2025 11:27 — with GitHub Actions Inactive

ilyakuz-db requested a review from denik January 20, 2025 16:17

denik reviewed Jan 21, 2025

View reviewed changes

fix: Remove unnecessary vendor make dependency

2154f2a

ilyakuz-db temporarily deployed to test-trigger-is January 21, 2025 15:05 — with GitHub Actions Inactive

ilyakuz-db changed the title ~~feat: Config reference documentation autogeneration~~ Config reference documentation autogeneration Jan 21, 2025

pietern approved these changes Jan 27, 2025

View reviewed changes

ilyakuz-db added 3 commits January 27, 2025 16:42

Move to bundle/docsgen

e5e47f4

Merge branch 'main' of github.com:databricks/cli into feat/config-ref…

21bd217

…erence-doc-autogen

Regenerate docs with latest SDK changes

b5fda64

ilyakuz-db temporarily deployed to test-trigger-is January 27, 2025 15:47 — with GitHub Actions Inactive

Update schema

bc88fe3

ilyakuz-db temporarily deployed to test-trigger-is January 27, 2025 15:49 — with GitHub Actions Inactive

ilyakuz-db changed the title ~~Config reference documentation autogeneration~~ Autogenerated documentation for config reference Jan 29, 2025

ilyakuz-db changed the title ~~Autogenerated documentation for config reference~~ Autogenerated documentation for bundle config Jan 29, 2025

ilyakuz-db added this pull request to the merge queue Jan 29, 2025

Merged via the queue into main with commit 708c4fb Jan 29, 2025

ilyakuz-db deleted the feat/config-reference-doc-autogen branch January 29, 2025 12:18

pietern mentioned this pull request Jun 12, 2026

Remove the unused bundle/docsgen documentation generator #5568

Merged

Conversation

ilyakuz-db commented Dec 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

Uh oh!

eng-dev-ecosystem-bot commented Dec 18, 2024

Uh oh!

github-actions Bot commented Jan 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilyakuz-db commented Jan 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ilyakuz-db commented Dec 18, 2024 •

edited

Loading