diff --git a/docs/AGENTS.md b/docs/AGENTS.md new file mode 100644 index 00000000000..a449ac52550 --- /dev/null +++ b/docs/AGENTS.md @@ -0,0 +1,187 @@ + + +# AGENTS.md + +> Scoped guidance for work under `docs/`. This file complements the +> repository-root `AGENTS.md`. + +## Scope And Ownership + +- `docs/` is the source for Apache Zeppelin's versioned product documentation. +- The main `zeppelin.apache.org` website is maintained in + `apache/zeppelin-site`; its homepage does not need to use the same generator + as these versioned docs. +- Markdown, layouts, includes, and assets in this directory are built here. + The generated site is written to `docs/_site/`. +- `docs/_site/` is generated and gitignored. Never edit or commit it. + +## Build Model + +The current build is: + +```text +docs sources + docs/_config.yml + -> Jekyll from docs/Gemfile.lock + -> docs/_site/ + -> zeppelin-site/docs// during a separate publication step +``` + +- `Gemfile` declares Jekyll and its documentation build dependencies. +- `Gemfile.lock` pins the actual Ruby dependency versions. The Docker commands + use `bundle exec` so the pinned Jekyll version is used. +- `_config.yml` supplies `ZEPPELIN_VERSION` and `JB.BASE_PATH`. +- `_includes/JB/setup` applies `JB.BASE_PATH` only for a safe build. Therefore + a publication build must include `--safe`. +- `Rakefile` contains legacy Jekyll-Bootstrap helpers. It is not the primary + build entry point; use the Docker commands below. +- The Maven build does not generate this site. +- Docker is the supported build environment. Do not install or run Ruby, + Bundler, or Jekyll directly on the host. + +## Preview And Build + +Preview with Docker: + +```bash +cd docs +docker run --rm -it \ + -v "$PWD:/docs" \ + -w /docs \ + -p '4000:4000' \ + ruby:4.0.6 \ + bash -lc "bundle install && bundle exec jekyll serve --watch --host 0.0.0.0" +``` + +Open `http://localhost:4000`. The preview intentionally runs without +`--safe`, so links are rooted at `/` instead of the production version path. + +Build the publication artifact with Docker: + +```bash +cd docs +docker run --rm \ + -v "$PWD:/docs" \ + -w /docs \ + ruby:4.0.6 \ + bash -lc "bundle install && bundle exec jekyll build --safe" +``` + +The output must be under `_site/`, and generated links and assets must use the +`JB.BASE_PATH` configured in `_config.yml`. + +When `Gemfile` changes, update `Gemfile.lock` inside Docker: + +```bash +cd docs +docker run --rm \ + -v "$PWD:/docs" \ + -w /docs \ + ruby:4.0.6 \ + bundle lock --update +``` + +Run the publication build after updating the lockfile. + +## Authoring Conventions + +- Preserve the ASF license header in every new source file. +- Follow the front matter used by nearby pages: + + ```yaml + --- + layout: page + title: "Page title" + description: "Short description" + group: section/subsection + --- + ``` + +- Include `{% include JB/setup %}` before page content when following the + existing page layout. +- Prefix internal site links and assets with `{{BASE_PATH}}` when an absolute + site path is needed. Production docs are hosted below `/docs//`, + not at the domain root. +- Update `_includes/themes/zeppelin/_navigation.html` when a page must appear + in the global documentation navigation. +- Keep filenames, headings, and link targets stable unless the task explicitly + includes redirects or link migration. +- Check the corresponding source code or configuration template when + documenting runtime behavior. Do not infer current behavior from an older + documentation page. + +## Version Handling + +- `ZEPPELIN_VERSION` and `JB.BASE_PATH` in `_config.yml` must identify the same + version. +- `dev/change_zeppelin_version.sh` updates both values as part of a repository + version change. Do not change them for an ordinary documentation edit. +- Before producing release docs, verify that `JB.BASE_PATH` is exactly + `/docs/`. + +## Publication Boundary + +- Building this directory does not publish the website. +- The generated `_site/` tree is copied into + `apache/zeppelin-site/docs//` by separate release/site work. +- The `zeppelin-site` repository owns the homepage, ASF staging/publishing, + and the mapping or redirect for `/docs/latest/`. +- Do not modify `zeppelin-site`, historical documentation snapshots, or + publication branches unless the user explicitly includes that work. + +## ASF Website Policy + +- Follow the ASF project website policy at + `https://privacy.apache.org/policies/website-policy.html` and the Infra CSP + guidance at `https://infra.apache.org/csp.html`. +- Do not add Google Analytics or any other third-party analytics, tracker, + tracking pixel, advertising tag, or external monitoring script. +- Do not load JavaScript, CSS, fonts, images, or other assets from non-ASF + domains. Host an asset in this repository when its license permits, or use a + normal external link instead of embedding it. +- Third-party embeds require the consent and DPA handling described by the ASF + policy. Prefer a direct link unless the task explicitly includes an approved + consent flow. +- The production layout uses the ASF-hosted Matomo instance provisioned for + Apache Zeppelin as site ID `69`. Do not replace it with another analytics + service or change its endpoint without Privacy team approval. + +## Verification + +For every documentation change: + +1. Run the Docker publication build above from `docs/`. +2. Confirm `_site/index.html` and the generated file for each changed page + exist. +3. Check generated navigation, links, images, and code blocks for the affected + pages. +4. Confirm generated URLs use the configured `/docs//` prefix. +5. Check the generated site for external trackers and embedded resources: + + ```bash + docker run --rm \ + -v "$PWD:/docs:ro" \ + -w /docs \ + ruby:4.0.6 \ + ruby check_external_resources.rb _site + ``` + +6. Run `git status --short` and keep `_site/` and incidental dependency changes + out of the commit. + +For navigation, layout, CSS, or JavaScript changes, also run the preview server +and inspect the affected pages at desktop and narrow viewport widths. diff --git a/docs/Gemfile b/docs/Gemfile index 9cc8cfef180..160ec9c8fef 100644 --- a/docs/Gemfile +++ b/docs/Gemfile @@ -14,9 +14,7 @@ # limitations under the License. # source 'https://rubygems.org' -gem 'github-pages' -gem 'redcarpet' -gem 'jekyll-twitter-plugin' +gem 'jekyll', '4.4.1' gem 'nokogiri', '1.19.3' gem 'mini_portile2', '2.8.4' diff --git a/docs/Gemfile.lock b/docs/Gemfile.lock index e27c883f6f5..7ac28335bcb 100644 --- a/docs/Gemfile.lock +++ b/docs/Gemfile.lock @@ -1,237 +1,69 @@ GEM remote: https://rubygems.org/ specs: - activesupport (7.2.3.1) - base64 - benchmark (>= 0.3) - bigdecimal - concurrent-ruby (~> 1.0, >= 1.3.1) - connection_pool (>= 2.2.5) - drb - i18n (>= 1.6, < 2) - logger (>= 1.4.2) - minitest (>= 5.1, < 6) - securerandom (>= 0.3) - tzinfo (~> 2.0, >= 2.0.5) addressable (2.9.0) public_suffix (>= 2.0.2, < 8.0) base64 (0.3.0) - benchmark (0.5.0) bigdecimal (4.1.2) - coffee-script (2.4.1) - coffee-script-source - execjs - coffee-script-source (1.12.2) colorator (1.1.0) - commonmarker (0.23.10) - concurrent-ruby (1.3.6) - connection_pool (3.0.2) - dnsruby (1.72.2) - simpleidn (~> 0.2.1) - drb (2.2.3) + concurrent-ruby (1.3.8) + csv (3.3.5) em-websocket (0.5.3) eventmachine (>= 0.12.9) http_parser.rb (~> 0) - ethon (0.16.0) - ffi (>= 1.15.0) eventmachine (1.2.7) - execjs (2.9.1) - faraday (2.14.1) - faraday-net_http (>= 2.0, < 3.5) - json - logger - faraday-net_http (3.4.2) - net-http (~> 0.5) - ffi (1.17.0) - ffi (1.17.0-arm64-darwin) - ffi (1.17.0-x86_64-linux-gnu) + ffi (1.17.4) + ffi (1.17.4-arm64-darwin) + ffi (1.17.4-x86_64-linux-gnu) forwardable-extended (2.6.0) - gemoji (4.1.0) - github-pages (231) - github-pages-health-check (= 1.18.2) - jekyll (= 3.9.5) - jekyll-avatar (= 0.8.0) - jekyll-coffeescript (= 1.2.2) - jekyll-commonmark-ghpages (= 0.4.0) - jekyll-default-layout (= 0.1.5) - jekyll-feed (= 0.17.0) - jekyll-gist (= 1.5.0) - jekyll-github-metadata (= 2.16.1) - jekyll-include-cache (= 0.2.1) - jekyll-mentions (= 1.6.0) - jekyll-optional-front-matter (= 0.3.2) - jekyll-paginate (= 1.1.0) - jekyll-readme-index (= 0.3.0) - jekyll-redirect-from (= 0.16.0) - jekyll-relative-links (= 0.6.1) - jekyll-remote-theme (= 0.4.3) - jekyll-sass-converter (= 1.5.2) - jekyll-seo-tag (= 2.8.0) - jekyll-sitemap (= 1.4.0) - jekyll-swiss (= 1.0.0) - jekyll-theme-architect (= 0.2.0) - jekyll-theme-cayman (= 0.2.0) - jekyll-theme-dinky (= 0.2.0) - jekyll-theme-hacker (= 0.2.0) - jekyll-theme-leap-day (= 0.2.0) - jekyll-theme-merlot (= 0.2.0) - jekyll-theme-midnight (= 0.2.0) - jekyll-theme-minimal (= 0.2.0) - jekyll-theme-modernist (= 0.2.0) - jekyll-theme-primer (= 0.6.0) - jekyll-theme-slate (= 0.2.0) - jekyll-theme-tactile (= 0.2.0) - jekyll-theme-time-machine (= 0.2.0) - jekyll-titles-from-headings (= 0.5.3) - jemoji (= 0.13.0) - kramdown (= 2.4.0) - kramdown-parser-gfm (= 1.1.0) - liquid (= 4.0.4) - mercenary (~> 0.3) - minima (= 2.5.1) - nokogiri (>= 1.13.6, < 2.0) - rouge (= 3.30.0) - terminal-table (~> 1.4) - github-pages-health-check (1.18.2) - addressable (~> 2.3) - dnsruby (~> 1.60) - octokit (>= 4, < 8) - public_suffix (>= 3.0, < 6.0) - typhoeus (~> 1.3) - html-pipeline (2.14.3) - activesupport (>= 2) - nokogiri (>= 1.4) - http_parser.rb (0.8.0) - i18n (1.14.8) + google-protobuf (4.35.1) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-arm64-darwin) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-x86_64-linux-gnu) + bigdecimal + rake (~> 13.3) + http_parser.rb (0.8.1) + i18n (1.15.2) concurrent-ruby (~> 1.0) - jekyll (3.9.5) + jekyll (4.4.1) addressable (~> 2.4) + base64 (~> 0.2) colorator (~> 1.0) + csv (~> 3.0) em-websocket (~> 0.5) - i18n (>= 0.7, < 2) - jekyll-sass-converter (~> 1.0) + i18n (~> 1.0) + jekyll-sass-converter (>= 2.0, < 4.0) jekyll-watch (~> 2.0) - kramdown (>= 1.17, < 3) + json (~> 2.6) + kramdown (~> 2.3, >= 2.3.1) + kramdown-parser-gfm (~> 1.0) liquid (~> 4.0) - mercenary (~> 0.3.3) + mercenary (~> 0.3, >= 0.3.6) pathutil (~> 0.9) - rouge (>= 1.7, < 4) + rouge (>= 3.0, < 5.0) safe_yaml (~> 1.0) - jekyll-avatar (0.8.0) - jekyll (>= 3.0, < 5.0) - jekyll-coffeescript (1.2.2) - coffee-script (~> 2.2) - coffee-script-source (~> 1.12) - jekyll-commonmark (1.4.0) - commonmarker (~> 0.22) - jekyll-commonmark-ghpages (0.4.0) - commonmarker (~> 0.23.7) - jekyll (~> 3.9.0) - jekyll-commonmark (~> 1.4.0) - rouge (>= 2.0, < 5.0) - jekyll-default-layout (0.1.5) - jekyll (>= 3.0, < 5.0) - jekyll-feed (0.17.0) - jekyll (>= 3.7, < 5.0) - jekyll-gist (1.5.0) - octokit (~> 4.2) - jekyll-github-metadata (2.16.1) - jekyll (>= 3.4, < 5.0) - octokit (>= 4, < 7, != 4.4.0) - jekyll-include-cache (0.2.1) - jekyll (>= 3.7, < 5.0) - jekyll-mentions (1.6.0) - html-pipeline (~> 2.3) - jekyll (>= 3.7, < 5.0) - jekyll-optional-front-matter (0.3.2) - jekyll (>= 3.0, < 5.0) - jekyll-paginate (1.1.0) - jekyll-readme-index (0.3.0) - jekyll (>= 3.0, < 5.0) - jekyll-redirect-from (0.16.0) - jekyll (>= 3.3, < 5.0) - jekyll-relative-links (0.6.1) - jekyll (>= 3.3, < 5.0) - jekyll-remote-theme (0.4.3) - addressable (~> 2.0) - jekyll (>= 3.5, < 5.0) - jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0) - rubyzip (>= 1.3.0, < 3.0) - jekyll-sass-converter (1.5.2) - sass (~> 3.4) - jekyll-seo-tag (2.8.0) - jekyll (>= 3.8, < 5.0) - jekyll-sitemap (1.4.0) - jekyll (>= 3.7, < 5.0) - jekyll-swiss (1.0.0) - jekyll-theme-architect (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-cayman (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-dinky (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-hacker (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-leap-day (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-merlot (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-midnight (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-minimal (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-modernist (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-primer (0.6.0) - jekyll (> 3.5, < 5.0) - jekyll-github-metadata (~> 2.9) - jekyll-seo-tag (~> 2.0) - jekyll-theme-slate (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-tactile (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-theme-time-machine (0.2.0) - jekyll (> 3.5, < 5.0) - jekyll-seo-tag (~> 2.0) - jekyll-titles-from-headings (0.5.3) - jekyll (>= 3.3, < 5.0) - jekyll-twitter-plugin (2.1.0) + terminal-table (>= 1.8, < 4.0) + webrick (~> 1.7) + jekyll-sass-converter (3.1.0) + sass-embedded (~> 1.75) jekyll-watch (2.2.1) listen (~> 3.0) - jemoji (0.13.0) - gemoji (>= 3, < 5) - html-pipeline (~> 2.2) - jekyll (>= 3.0, < 5.0) - json (2.19.5) - kramdown (2.4.0) - rexml + json (2.21.1) + kramdown (2.5.2) + rexml (>= 3.4.4) kramdown-parser-gfm (1.1.0) kramdown (~> 2.0) liquid (4.0.4) - listen (3.9.0) + listen (3.10.0) + logger rb-fsevent (~> 0.10, >= 0.10.3) rb-inotify (~> 0.9, >= 0.9.10) logger (1.7.0) - mercenary (0.3.6) + mercenary (0.4.0) mini_portile2 (2.8.4) - minima (2.5.1) - jekyll (>= 3.5, < 5.0) - jekyll-feed (~> 0.9) - jekyll-seo-tag (~> 2.1) - minitest (5.27.0) - net-http (0.9.1) - uri (>= 0.11.1) nokogiri (1.19.3) mini_portile2 (~> 2.8.2) racc (~> 1.4) @@ -239,40 +71,28 @@ GEM racc (~> 1.4) nokogiri (1.19.3-x86_64-linux-gnu) racc (~> 1.4) - octokit (4.25.1) - faraday (>= 1, < 3) - sawyer (~> 0.9) pathutil (0.16.2) forwardable-extended (~> 2.6) - public_suffix (5.1.1) + public_suffix (7.0.5) racc (1.8.1) + rake (13.4.2) rb-fsevent (0.11.2) rb-inotify (0.11.1) ffi (~> 1.0) - redcarpet (3.6.0) - rexml (3.4.2) - rouge (3.30.0) - rubyzip (2.3.2) + rexml (3.4.4) + rouge (4.7.0) safe_yaml (1.0.5) - sass (3.7.4) - sass-listen (~> 4.0.0) - sass-listen (4.0.0) - rb-fsevent (~> 0.9, >= 0.9.4) - rb-inotify (~> 0.9, >= 0.9.7) - sawyer (0.9.2) - addressable (>= 2.3.5) - faraday (>= 0.17.3, < 3) - securerandom (0.4.1) - simpleidn (0.2.3) - terminal-table (1.8.0) - unicode-display_width (~> 1.1, >= 1.1.1) - typhoeus (1.4.1) - ethon (>= 0.9.0) - tzinfo (2.0.6) - concurrent-ruby (~> 1.0) - unicode-display_width (1.8.0) - uri (1.1.1) - webrick (1.8.2) + sass-embedded (1.102.0) + google-protobuf (~> 4.31) + rake (>= 13) + sass-embedded (1.102.0-arm64-darwin) + google-protobuf (~> 4.31) + sass-embedded (1.102.0-x86_64-linux-gnu) + google-protobuf (~> 4.31) + terminal-table (3.0.2) + unicode-display_width (>= 1.1.1, < 3) + unicode-display_width (2.6.0) + webrick (1.9.2) PLATFORMS arm64-darwin @@ -280,12 +100,10 @@ PLATFORMS x86_64-linux DEPENDENCIES - github-pages - jekyll-twitter-plugin + jekyll (= 4.4.1) mini_portile2 (= 2.8.4) nokogiri (= 1.19.3) - redcarpet webrick (~> 1.8) BUNDLED WITH - 2.5.16 + 4.0.16 diff --git a/docs/README.md b/docs/README.md index 736b58fa41c..5b8762a73bc 100644 --- a/docs/README.md +++ b/docs/README.md @@ -3,94 +3,61 @@ This README will walk you through building the documentation of Apache Zeppelin. The documentation is included here with Apache Zeppelin source code. The online documentation at [https://zeppelin.apache.org/docs/](https://zeppelin.apache.org/docs/latest/) is also generated from the files found in here. ## Build documentation -Zeppelin is using [Jekyll](https://jekyllrb.com/) which is a static site generator and [Github Pages](https://pages.github.com/) as a site publisher. For the more details, see [help.github.com/articles/about-github-pages-and-jekyll/](https://help.github.com/articles/about-github-pages-and-jekyll/). +Zeppelin uses [Jekyll](https://jekyllrb.com/) to generate the static versioned documentation published on the Apache Zeppelin website. **Requirements** -``` -# ruby --version >= 2.0.0 -# Install Bundler using gem -gem install bundler - -cd $ZEPPELIN_HOME/docs -# Install all dependencies declared in the Gemfile -bundle install -``` - -For the further information about requirements, please see [here](https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/#requirements). - -On OS X 10.9, you may need to do +- [Docker](https://docs.docker.com/get-docker/) -``` -xcode-select --install -``` - -**Docker** +Ruby, Bundler, and Jekyll run only inside the Docker container. No host Ruby +installation is required. -Local docker environments are also supported and have been tested using: -* [Docker version 20.10.2](https://docs.docker.com/get-docker/) +## Preview documentation -## Run website locally -If you don't want to encounter ugly rendered pages, run the documentation site in your local environment first. +From `$ZEPPELIN_HOME/docs`, run: -In `$ZEPPELIN_HOME/docs`, run one of the desired commands: - -**Run locally** -``` -bundle exec jekyll serve --watch -``` - -**Run locally using docker** -``` +```bash docker run --rm -it \ - -v $PWD:/docs \ - -w /docs \ - -p '4000:4000' \ - ruby:3.3.5 \ - bash -c "bundle install && bundle exec jekyll serve --watch --host 0.0.0.0" + -v "$PWD:/docs" \ + -w /docs \ + -p '4000:4000' \ + ruby:4.0.6 \ + bash -lc "bundle install && bundle exec jekyll serve --watch --host 0.0.0.0" ``` -Using the above command, Jekyll will start a web server at `http://localhost:4000` and watch the `/docs` directory for updates. - - +Jekyll starts at `http://localhost:4000` and watches the `docs/` sources for +updates. ## Contribute to Zeppelin documentation If you wish to help us and contribute to Zeppelin Documentation, please look at [Zeppelin Documentation's contribution guideline](https://zeppelin.apache.org/contribution/contributions.html). - ## For committers only ### Bumping up version in a new release - * `ZEPPELIN_VERSION` and `BASE_PATH` property in _config.yml - -### Deploy to ASF svnpubsub infra - 1. generate static website in `./_site` +- Update `ZEPPELIN_VERSION` and `JB.BASE_PATH` in `_config.yml`. - ``` - # go to /docs under Zeppelin source - bundle exec jekyll build --safe - ``` +### Build versioned documentation - 2. verify no analytics scripts in the generated output +From `$ZEPPELIN_HOME/docs`, run: - ``` - ( - grep -rnE --include='*.html' \ - "google-analytics\.com|googletagmanager\.com|analytics\.js|ga\.js|UA-[0-9]" \ - _site/ - case $? in - 0) echo "FAIL: analytics found"; exit 1 ;; - 1) ;; - *) echo "ERROR: scan failed"; exit 2 ;; - esac - ) - ``` +```bash +docker run --rm \ + -v "$PWD:/docs" \ + -w /docs \ + ruby:4.0.6 \ + bash -lc "bundle install && bundle exec jekyll build --safe" +``` - 3. checkout ASF repo +Check the generated site for external resources and trackers: - ``` - svn co https://svn.apache.org/repos/asf/zeppelin asf-zeppelin - ``` +```bash +docker run --rm \ + -v "$PWD:/docs:ro" \ + -w /docs \ + ruby:4.0.6 \ + ruby check_external_resources.rb _site +``` - 4. copy `zeppelin/docs/_site` to `asf-zeppelin/site/docs/[VERSION]` - 5. `svn commit` +The generated site is written to `_site/`. Copy it to +`zeppelin-site/docs//` as part of the separate website publication +workflow. diff --git a/docs/_config.yml b/docs/_config.yml index cf009ae6ec8..eb2b9317025 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -2,11 +2,12 @@ # For more see: http://jekyllrb.com/docs/permalinks/ permalink: /:categories/:year/:month/:day/:title -exclude: [".rvmrc", ".rbenv-version", "README.md", "Rakefile", "changelog.md", "vendor", "node_modules", "scss"] +exclude: [".rvmrc", ".rbenv-version", "AGENTS.md", "Gemfile", "Gemfile.lock", "README.md", "Rakefile", "changelog.md", "check_external_resources.rb", "vendor", "node_modules", "scss"] highlighter: rouge -markdown: redcarpet -redcarpet: - extensions: ["tables"] +markdown: kramdown +kramdown: + input: GFM + show_warnings: true encoding: utf-8 # Themes are encouraged to use these universal variables @@ -18,8 +19,6 @@ author : name : The Apache Software Foundation email : dev@zeppelin.apache.org github : apache - twitter : ASF - feedburner : feedname ZEPPELIN_VERSION : 0.13.0-SNAPSHOT @@ -36,6 +35,7 @@ production_url : http://zeppelin.apache.org # JB : version : 0.3.0 + matomo_site_id : 69 # All links will be namespaced by BASE_PATH if defined. # Links in your website should always be prefixed with {{BASE_PATH}} @@ -77,48 +77,6 @@ JB : archive_path: /archive.html categories_path : /categories.html tags_path : /tags.html - atom_path : /atom.xml - rss_path : /rss.xml - - # Settings for comments helper - # Set 'provider' to the comment provider you want to use. - # Set 'provider' to false to turn commenting off globally. - # - comments : - provider : disqus - disqus : - short_name : jekyllbootstrap - livefyre : - site_id : 123 - intensedebate : - account : 123abc - facebook : - appid : 123 - num_posts: 5 - width: 580 - colorscheme: light - - # Settings for analytics helper - # Set 'provider' to the analytics provider you want to use. - # Set 'provider' to false to turn analytics off globally. - # - analytics : - provider : false - getclicky : - site_id : - mixpanel : - token : '_MIXPANEL_TOKEN_' - piwik : - baseURL : 'myserver.tld/piwik' # Piwik installation address (without protocol) - idsite : '1' # the id of the site on Piwik - - # Settings for sharing helper. - # Sharing is for things like tweet, plusone, like, reddit buttons etc. - # Set 'provider' to the sharing provider you want to use. - # Set 'provider' to false to turn sharing off globally. - # - sharing : - provider : false # Settings for all other include helpers can be defined by creating # a hash with key named for the given helper. ex: diff --git a/docs/_includes/JB/analytics b/docs/_includes/JB/analytics deleted file mode 100644 index 48d87c25fa0..00000000000 --- a/docs/_includes/JB/analytics +++ /dev/null @@ -1,18 +0,0 @@ -{% if site.safe and site.JB.analytics.provider and page.JB.analytics != false %} - -{% case site.JB.analytics.provider %} -{% when "google_classic" %} - {% include JB/analytics-providers/google-classic %} -{% when "google_universal" %} - {% include JB/analytics-providers/google-universal %} -{% when "getclicky" %} - {% include JB/analytics-providers/getclicky %} -{% when "mixpanel" %} - {% include JB/analytics-providers/mixpanel %} -{% when "piwik" %} - {% include JB/analytics-providers/piwik %} -{% when "custom" %} - {% include custom/analytics %} -{% endcase %} - -{% endif %} \ No newline at end of file diff --git a/docs/_includes/JB/analytics-providers/getclicky b/docs/_includes/JB/analytics-providers/getclicky deleted file mode 100644 index e9462f4f67f..00000000000 --- a/docs/_includes/JB/analytics-providers/getclicky +++ /dev/null @@ -1,12 +0,0 @@ - - diff --git a/docs/_includes/JB/analytics-providers/google-classic b/docs/_includes/JB/analytics-providers/google-classic deleted file mode 100644 index af099078a58..00000000000 --- a/docs/_includes/JB/analytics-providers/google-classic +++ /dev/null @@ -1,11 +0,0 @@ - \ No newline at end of file diff --git a/docs/_includes/JB/analytics-providers/google-universal b/docs/_includes/JB/analytics-providers/google-universal deleted file mode 100644 index dae744b994a..00000000000 --- a/docs/_includes/JB/analytics-providers/google-universal +++ /dev/null @@ -1,11 +0,0 @@ - \ No newline at end of file diff --git a/docs/_includes/JB/analytics-providers/mixpanel b/docs/_includes/JB/analytics-providers/mixpanel deleted file mode 100644 index 4406eb048d2..00000000000 --- a/docs/_includes/JB/analytics-providers/mixpanel +++ /dev/null @@ -1,11 +0,0 @@ - \ No newline at end of file diff --git a/docs/_includes/JB/analytics-providers/piwik b/docs/_includes/JB/analytics-providers/piwik deleted file mode 100755 index f016ed7ca4f..00000000000 --- a/docs/_includes/JB/analytics-providers/piwik +++ /dev/null @@ -1,10 +0,0 @@ - \ No newline at end of file diff --git a/docs/_includes/JB/comments b/docs/_includes/JB/comments deleted file mode 100644 index 4e9e600f6c0..00000000000 --- a/docs/_includes/JB/comments +++ /dev/null @@ -1,16 +0,0 @@ -{% if site.JB.comments.provider and page.comments != false %} - -{% case site.JB.comments.provider %} -{% when "disqus" %} - {% include JB/comments-providers/disqus %} -{% when "livefyre" %} - {% include JB/comments-providers/livefyre %} -{% when "intensedebate" %} - {% include JB/comments-providers/intensedebate %} -{% when "facebook" %} - {% include JB/comments-providers/facebook %} -{% when "custom" %} - {% include custom/comments %} -{% endcase %} - -{% endif %} \ No newline at end of file diff --git a/docs/_includes/JB/comments-providers/disqus b/docs/_includes/JB/comments-providers/disqus deleted file mode 100644 index 618a7b737bd..00000000000 --- a/docs/_includes/JB/comments-providers/disqus +++ /dev/null @@ -1,14 +0,0 @@ -
- - -blog comments powered by Disqus diff --git a/docs/_includes/JB/comments-providers/facebook b/docs/_includes/JB/comments-providers/facebook deleted file mode 100644 index 6b3e5e06921..00000000000 --- a/docs/_includes/JB/comments-providers/facebook +++ /dev/null @@ -1,9 +0,0 @@ -
- -
\ No newline at end of file diff --git a/docs/_includes/JB/comments-providers/intensedebate b/docs/_includes/JB/comments-providers/intensedebate deleted file mode 100644 index ab0c3c9769c..00000000000 --- a/docs/_includes/JB/comments-providers/intensedebate +++ /dev/null @@ -1,6 +0,0 @@ - - diff --git a/docs/_includes/JB/comments-providers/livefyre b/docs/_includes/JB/comments-providers/livefyre deleted file mode 100644 index 704b80392b1..00000000000 --- a/docs/_includes/JB/comments-providers/livefyre +++ /dev/null @@ -1,6 +0,0 @@ - - \ No newline at end of file diff --git a/docs/_includes/JB/matomo b/docs/_includes/JB/matomo new file mode 100644 index 00000000000..136bbb1b773 --- /dev/null +++ b/docs/_includes/JB/matomo @@ -0,0 +1,33 @@ + + + + + diff --git a/docs/_includes/JB/sharing b/docs/_includes/JB/sharing deleted file mode 100644 index f5b11518bd2..00000000000 --- a/docs/_includes/JB/sharing +++ /dev/null @@ -1,8 +0,0 @@ -{% if site.safe and site.JB.sharing.provider and page.JB.sharing != false %} - -{% case site.JB.sharing.provider %} -{% when "custom" %} - {% include custom/sharing %} -{% endcase %} - -{% endif %} \ No newline at end of file diff --git a/docs/_includes/themes/zeppelin/default.html b/docs/_includes/themes/zeppelin/default.html index d2cd0719da3..36d88a36031 100644 --- a/docs/_includes/themes/zeppelin/default.html +++ b/docs/_includes/themes/zeppelin/default.html @@ -9,11 +9,6 @@ - - - @@ -37,9 +32,9 @@ - - - + {% if site.safe %} + {% include JB/matomo %} + {% endif %} @@ -54,6 +49,5 @@ - {% include JB/analytics %} diff --git a/docs/_includes/themes/zeppelin/post.html b/docs/_includes/themes/zeppelin/post.html index 2774711aecb..14934d35b5b 100644 --- a/docs/_includes/themes/zeppelin/post.html +++ b/docs/_includes/themes/zeppelin/post.html @@ -43,7 +43,5 @@

{{ page.title }} {% if page.tagline %}{{page.tagline}}{% endi {% endif %} -
- {% include JB/comments %} diff --git a/docs/assets/themes/zeppelin/img/docs-img/labeled-property-graph-model.svg b/docs/assets/themes/zeppelin/img/docs-img/labeled-property-graph-model.svg new file mode 100644 index 00000000000..0d2a53fa7f0 --- /dev/null +++ b/docs/assets/themes/zeppelin/img/docs-img/labeled-property-graph-model.svg @@ -0,0 +1,86 @@ + + + + Labeled property graph data model + People, notebooks, and a dataset represented as labeled nodes connected by typed relationships with properties. + + + + + + + + + + + + AUTHORED + since: 2024 + + + + VIEWED + date: Jul 18 + + + + READS + format: parquet + + + + READS + format: csv + + + + :Person:Author + name: Mina + team: Analytics + + + + :Person + name: Yun + team: Operations + + + + :Notebook + title: Flight Analysis + version: 3 + + + + :Notebook + title: Revenue Forecast + version: 7 + + + + :Dataset + name: Flight Records + owner: Data Platform + diff --git a/docs/assets/themes/zeppelin/img/docs-img/property-graph-example.svg b/docs/assets/themes/zeppelin/img/docs-img/property-graph-example.svg new file mode 100644 index 00000000000..7ae1a683dca --- /dev/null +++ b/docs/assets/themes/zeppelin/img/docs-img/property-graph-example.svg @@ -0,0 +1,87 @@ + + + + Property graph example + Four nodes connected by labeled, directed edges, with properties attached to both nodes and edges. + + + + + + + + + + + + KNOWS + + since: 2022 + + + + CREATED + + role: owner + + + + CONTRIBUTED + + commits: 18 + + + + READS + + format: parquet + + + name: Mina + age: 34 + + 1 + person + + + 2 + person + + name: Yun + + + title: Flight Analysis + language: Python + + 3 + notebook + + + 4 + dataset + + name: Flight Records + rows: 2.1M + diff --git a/docs/atom.xml b/docs/atom.xml deleted file mode 100644 index 7ec29339dd6..00000000000 --- a/docs/atom.xml +++ /dev/null @@ -1,28 +0,0 @@ ---- -layout: nil -title : ---- - - - - {{ site.title }} - - - {{ site.time | date_to_xmlschema }} - {{ site.production_url }} - - {{ site.author.name }} - {{ site.author.email }} - - - {% for post in site.posts %} - - {{ post.title }} - - {{ post.date | date_to_xmlschema }} - {{ site.production_url }}{{ post.id }} - {{ post.content | xml_escape }} - - {% endfor %} - - diff --git a/docs/check_external_resources.rb b/docs/check_external_resources.rb new file mode 100644 index 00000000000..b4d40628461 --- /dev/null +++ b/docs/check_external_resources.rb @@ -0,0 +1,79 @@ +#!/usr/bin/env ruby +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require "uri" + +SITE_DIR = ARGV.fetch(0, "_site") +RESOURCE_TAGS = %w[script img iframe link source video audio embed object].freeze +RESOURCE_PATTERN = + /<(#{RESOURCE_TAGS.join("|")})\b[^>]*\b(?:src|href|data)\s*=\s*["']([^"']+)["']/i +CSS_URL_PATTERN = /url\(\s*["']?([^"')]+)["']?\s*\)/i +TRACKER_PATTERN = + /google-analytics|googletag|gtag\s*\(|mixpanel|getclicky|piwik|disqus|connect\.facebook|(?:platform\.)?twitter\.com\/widgets/i +MATOMO_CODE_PATTERN = /matomo\.(?:js|php)|setTrackerUrl|\b_paq\b/i +ASF_MATOMO_URL = "https://analytics.apache.org/" + +def remote_host(value) + return nil unless value.match?(%r{\A(?:https?:)?//}i) + + normalized = value.start_with?("//") ? "https:#{value}" : value + URI.parse(normalized).host || :invalid +rescue URI::InvalidURIError + :invalid +end + +def asf_host?(host) + host != :invalid && (host == "apache.org" || host.end_with?(".apache.org")) +end + +violations = [] + +Dir.glob(File.join(SITE_DIR, "**", "*.html")).sort.each do |file| + File.read(file).scan(RESOURCE_PATTERN) do |tag, value| + host = remote_host(value) + next if host.nil? || asf_host?(host) + + violations << "#{file}: external #{tag} resource #{value}" + end +end + +Dir.glob(File.join(SITE_DIR, "**", "*.css")).sort.each do |file| + File.read(file).scan(CSS_URL_PATTERN) do |match| + value = match.first + host = remote_host(value) + next if host.nil? || asf_host?(host) + + violations << "#{file}: external CSS resource #{value}" + end +end + +Dir.glob(File.join(SITE_DIR, "**", "*.{html,js,css}")).sort.each do |file| + content = File.read(file) + violations << "#{file}: tracker or external embed code" if content.match?(TRACKER_PATTERN) + if content.match?(MATOMO_CODE_PATTERN) && !content.include?(ASF_MATOMO_URL) + violations << "#{file}: Matomo must use #{ASF_MATOMO_URL}" + end +end + +if violations.empty? + puts "No disallowed external resources or trackers found in #{SITE_DIR}" + exit 0 +end + +warn violations.join("\n") +exit 1 diff --git a/docs/interpreter/spark.md b/docs/interpreter/spark.md index f7311a2133e..be1098ea65d 100644 --- a/docs/interpreter/spark.md +++ b/docs/interpreter/spark.md @@ -132,17 +132,17 @@ You can also set other Spark properties which are not listed in the table. For a `SPARK_HOME` Location of spark distribution - + spark.master local[*] Spark master uri.
e.g. spark://master_host:7077 - + spark.submit.deployMode The deploy mode of Spark driver program, either "client" or "cluster", Which means to launch driver program locally ("client") or remotely ("cluster") on one of the nodes inside the cluster. - + spark.app.name Zeppelin The name of spark application. diff --git a/docs/rss.xml b/docs/rss.xml deleted file mode 100644 index 8c2a9dd9a8c..00000000000 --- a/docs/rss.xml +++ /dev/null @@ -1,28 +0,0 @@ ---- -layout: nil -title : ---- - - - - - {{ site.title }} - {{ site.title }} - {{ site.author.name }} - {{ site.production_url }}{{ site.rss_path }} - {{ site.production_url }} - {{ site.time | date_to_xmlschema }} - {{ site.time | date_to_xmlschema }} - 1800 - -{% for post in site.posts %} - - {{ post.title }} - {{ post.content | xml_escape }} - {{ site.production_url }}{{ post.url }} - {{ site.production_url }}{{ post.id }} - {{ post.date | date_to_xmlschema }} - -{% endfor %} - - - diff --git a/docs/usage/display_system/basic.md b/docs/usage/display_system/basic.md index 01d46297fcf..0f0faebeb8c 100644 --- a/docs/usage/display_system/basic.md +++ b/docs/usage/display_system/basic.md @@ -86,11 +86,11 @@ A [Property Graph](https://github.com/tinkerpop/gremlin/wiki/Defining-a-Property * each edge has a label that denotes the type of relationship between its two vertices. * each edge has a collection of properties defined by a map from key to value. - +![Property graph example]({{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/property-graph-example.svg) A [Labelled Property Graph](https://neo4j.com/developer/graph-database/#property-graph) is a Property Graph where the nodes can be tagged with **labels** representing their different roles in the graph model - +![Labeled property graph data model]({{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/labeled-property-graph-model.svg) ### What are the APIs? diff --git a/docs/usage/rest_api/interpreter.md b/docs/usage/rest_api/interpreter.md index 9d81dd60e06..747427b3485 100644 --- a/docs/usage/rest_api/interpreter.md +++ b/docs/usage/rest_api/interpreter.md @@ -620,7 +620,7 @@ The role of registered interpreters, settings and interpreters group are describ } ``` - +