[deploy] 0.7 alpha 2#258
Merged
Merged
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Add Superset data source plugin with support for browsing and loading datasets via password or SSO login. Integrate OIDC authentication system with frontend login button and callback handling. Extend authentication regex to support OIDC's sub claim format. Add GitHub OAuth as an alternative authentication option. Introduce plugin system framework supporting automatic discovery and registration of frontend and backend plugins. Optimize internationalization files with Superset-related translations. Add test cases covering authentication and plugin functionality. New files include: - Superset plugin frontend components and API modules - OIDC authentication configuration and callback pages - GitHub OAuth gateway and authentication provider - Plugin system base classes and registration logic - Related test cases and internationalization resources Modified files involve: - Enhanced authentication logic - Extended application configuration interfaces - Dependency updates (pyproject.toml, package.json) - Security-related regex adjustments
…plugin interface Refactor Superset plugin interface with dashboard search and filtering capabilities Migrate translation files from public directory to plugin local directory Add plugin translation registration mechanism with support for dynamically loading plugin localization resources Optimize table name generation logic with automatic suffix functionality Improve filter dialog interactions with support for more operators and type checking
…plugin interface Refactor Superset plugin interface with dashboard search and filtering capabilities Migrate translation files from public directory to plugin local directory Add plugin translation registration mechanism with support for dynamically loading plugin localization resources Optimize table name generation logic with automatic suffix functionality Improve filter dialog interactions with support for more operators and type checking
在数据源插件架构文档中新增第12节，详细说明插件如何自带翻译文件并通过框架自动合并的方案。该方案解决插件翻译与宿主项目混杂的问题，保持插件自包含性，同时支持多语言自动切换。
…structions Add detailed documentation about plugin self-contained translations, including directory structure, JSON format, export mechanism, and merge rules Update section numbering to reflect new content and provide a quick reference for plugin translations
Add oauth2 as an alias for oidc to avoid confusion, and update authentication configuration documentation in .env.template with options for OIDC, GitHub, and Azure authentication
添加对接OIDC/OAuth2身份提供者的详细文档说明，包括环境变量配置、IdP必须满足的条件、Discovery端点最小JSON格式以及JWT Access Token的claim要求。同时更新配置说明，明确oidc和oauth2是等价别名。
… Superset plugin - Add password show/hide toggle button in login form - Add Chinese and English description text for Superset plugin - Update data upload dialog to use plugin description text
… UserInfo validation fallback Support two OIDC/OAuth2 configuration modes: 1. Auto-discovery mode (standard OIDC IdP) 2. Manual endpoint mode (OAuth2 servers without Discovery) New token validation strategies:
前端检测到匿名用户首次登录时，自动触发迁移流程并提供数据导入选项。后端实现安全的数据复制机制，确保迁移过程幂等且不删除源数据。同时添加必要的安全约束，防止非法迁移请求。
Implement workspace data migration from anonymous browser identity to authenticated user - Add migration dialog component and internationalization text - Extend workspace manager to support copying workspaces - Add /sessions/migrate API endpoint - Detect identity change after login and prompt for migration
…ndpoint Add local logout handling logic to perform local cleanup and redirect when IdP does not provide end_session_endpoint
… authenticated user migration Add local storage flag to track migration status, ensuring migration is only performed once per user
…ge management - Change anonymous workspace migration from copy to move operation, with merge functionality added - Add cleanup anonymous workspace API endpoint - Persist identity type and browser ID in local storage - Modify identity migration dialog logic to use the new cleanup API - Fix local storage state inconsistency after migration
…and cleanup fix(superset): Enhance SSO login popup handling and add documentation refactor(workspace): Change migration operation to copy-then-delete pattern docs: Add Superset SSO bridge configuration guide documentation test: Add test cases for workspace locking scenarios
Add new internationalization text and logic handling to support IdP-initiated SSO login flow. When users directly redirect from the SSO system, automatically re-initiate the standard SP flow and display corresponding waiting prompt messages.
Feature/plugin architecture
Contributor
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
…ty configuration Add silent refresh capability for expired tokens in OIDC authentication, along with enhanced Flask session security configuration Add related test cases to verify token refresh and session configuration Add offline_access to OIDC scopes to support refresh tokens
…a during anonymous user migration test(IdentityMigrationDialog): Add unit tests for migration dialog refactor(DataFormulator): Refactor workspace list display logic
…oader Merge verbose_name, description, and expression into a single description field for improved catalog search, while preserving individual fields for consumers that need them separately.
- Enhanced the oauth_config.py example with detailed comments and a new staging handler for SSO integration. - Updated the superset_config.py to include a TokenExchangeView for silent token exchange and improved security checks. - Revised the SSO configuration guide to clarify the structure and requirements for SSO user information parsing and role mapping.
- Refactor data loading to utilize Superset's Chart Data API instead of SQL Lab API, simplifying permission requirements to only `datasource access`. - Enhance documentation to clarify permission needs and security advantages of the new approach. - Update related tests to validate the new data retrieval method and ensure compatibility with the Chart Data API. - Remove obsolete SQL Lab related code and helper functions.
…ized errors - Introduce `extractErrorMessage` function to handle various error types, including RTK serialized errors, ensuring proper message extraction. - Update error handling in `DataSourceSidebar`, `DataThread`, and `DBTableManager` components to utilize the new extraction method, preventing `[object Object]` outputs. - Add tests for `extractErrorMessage` to validate behavior with different error shapes, including ApiRequestError and plain Error instances. - Refactor existing error handling logic to improve clarity and maintainability.
Feature/plugin architecture
…ersetLoader - Introduce functionality to detect and convert epoch-ms temporal columns to appropriate Arrow date/timestamp types.
…ia parallel fetch
dataloader 没有统一每个实现类最大可加载的行
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Summary
Agents & AI Pipeline
agent_py_data_rec,agent_sql_data_rec,agent_py_data_transform,agent_sql_data_transform,agent_concept_derive,agent_py_concept_derive,agent_data_clean, andagent_explorationinto three unified agents:data_agent.py,agent_data_rec.py, andagent_data_transform.pysemantic_types.pybackend module and full frontend type registry (src/lib/agents-chart/core/type-registry.ts,field-semantics.ts,semantic-types.ts) with domain shape inference, tick constraints, zero-baseline classification, and snap-to-bound heuristicsagent_chart_insight.pyfor AI-generated chart takeawaysagent_language.pyfor i18n-aware promptsagent_diagnostics.pywith unified diagnostic information builder for better error reportingVisualization
src/lib/agents-chart/, 120 files, ~44K lines) with multi-backend support for Vega-Lite, ECharts, Chart.js, and GoFish — includes template system, semantic-aware axis/domain/tick handling, color decisions, layout computation, faceting, and overflow filteringChartGallery.tsxwith expanded chart type support including pie, US map, world map, bump, candlestick, density, lollipop, pyramid, radar, rose, streamgraph, strip plot, waterfall, and moreChartRenderService.tsxreplacing static SVG rendering withvega-embedfor interactive chartsSimpleChartRecBox.tsxandchartRecommendation.tsfor improved chart suggestion workflowScoretype with small domain spans (e.g., [0,1]) no longer forces integer-only ticks, preserving intermediate decimal ticksData Thread & Workflow
DataThread.tsxrewrite, newDataThreadCards.tsx,InteractionEntryCard.tsx)useFormulateData.tsconsolidating data derivation logicTiptapReportEditor.tsx) with richer editing supportData Loading & Management
UnifiedDataUploadDialog.tsxreplacing the old table selection view — supports file upload, URL, paste, database, and sample datasets in a single dialog with loading state indicatorsMultiTablePreview.tsxfor previewing multiple tables before loadingtableThunks.tshandling all data source types with server-side workspace storageuseDataRefresh.tsxwith auto-refresh, stream data sources, andRefreshDataDialog.tsx#rowId) viaROW_NUMBER()in DuckDB and pandas paths, preserving original row positions after sortData Loaders (Database Plugins)
Datalake / Workspace Backend
datalake/package withworkspace.py,azure_blob_workspace.py,cached_azure_blob_workspace.py,file_manager.py,metadata.py,cache_manager.py,parquet_utils.py, andtable_names.pyworkspace_factory.pyfor configuration-driven workspace initializationsession_routes.pyfor session-level API endpointsSecurity
code_signing.pyfor generated code integrity verificationauth.pyfor authentication handlingurl_allowlist.pyfor URL validationsanitize.pyto prevent leaking sensitive info in error messagessandbox/package withlocal_sandbox.py,docker_sandbox.py,not_a_sandbox.py, andDockerfile.sandboxreplacing the oldpy_sandbox.pyidentity.tswith browser-based identity for multi-user supportInternationalization (i18n)
react-i18nextwith English and Chinese locale files across 7 namespaces (common, chart, encoding, messages, model, navigation, upload)TRANSLATION_GUIDE.mdfor contributorsUI & Design System
tokens.tswith centralized color, spacing, shadow, transition, and radius tokensDataFormulator.tsxandApp.tsxwithTopNavButton,AppShellnavigation, and model management UIEncodingShelfCard.tsxandEncodingShelfThread.tsxConceptCard.tsx,ConceptShelf.tsx,DerivedDataDialog.tsxModel Management
model_registry.pyfor managing model configurations server-sideModelSelectionDialog.tsxwith multi-model supportInfrastructure & DevOps
Dockerfile,docker-compose.yml,docker-compose.test.ymlwith volume permissions and sandbox user handling.devcontainer/devcontainer.jsonuv.lock, updatedpyproject.tomlandrequirements.txtTesting
vitest.config.ts,pytest.ini,conftest.py, frontend setup, andtest_plan.md