Skip to content

feat(frontend): add task-aware field visibility and preview to HuggingFace property editor#5568

Open
ELin2025 wants to merge 12 commits into
apache:mainfrom
ELin2025:hf/07-property-editor
Open

feat(frontend): add task-aware field visibility and preview to HuggingFace property editor#5568
ELin2025 wants to merge 12 commits into
apache:mainfrom
ELin2025:hf/07-property-editor

Conversation

@ELin2025

@ELin2025 ELin2025 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

⚠️ This PR is stacked on #5567. Until that lands, the diff below may also include PR 6a/6b's task selector, model browser, and audio upload changes depending on which base GitHub is showing. The new code in this PR is the task-aware field visibility rules, task preview cards, and custom validators in operator-property-edit-frame.component.ts/.html/.scss, plus the test infrastructure (mockHuggingFaceSchema, mockHuggingFacePredicate) and 7 new spec tests. Once PR #5567 merges and this PR is retargeted to main, the diff should auto-clean to the PR 7 property-editor changes only.

What changes were proposed in this PR?

Wire up the HuggingFace operator's property editor so that selecting a task dynamically controls which fields are visible and shows a media preview card. This is the PR that makes the formly components from PRs #5566/ #5567 user-visible by mapping operator fields to custom field types in jsonSchemaMapIntercept.

Changes to operator-property-edit-frame.component.ts:

  • Map modelIdhuggingface formly type, imageInputhuggingface-image-upload, audioInputhuggingface-audio-upload
  • Hide the task field (it is controlled by the HuggingFaceComponent's task dropdown instead)
  • ~13 field visibility rules via formly expressions that show/hide fields based on the selected task (e.g., imageInput only for image tasks, contextColumn only for question-answering, systemPrompt/maxNewTokens/temperature only for text-generation)
  • 3 custom validators: requiredImageInput, requiredAudioInput, requiredPromptColumn — each checks whether the direct input or the corresponding column selector is filled
  • Removed validation: { show: true } from imageInput, audioInput, and promptColumn so errors only appear after the field is touched, not on a freshly-added operator
  • Added image-to-video to imageInputTasks so the image input field appears for that task
  • Task preview cards with sample media for 22 task types across 4 kinds (image, video, audio, text), plus a fallback for unknown tasks
  • Operator description truncation — clamps to 3 lines with CSS and adds a tooltip for the full text on hover

Changes to test infrastructure:

  • mockHuggingFaceSchema in mock-operator-metadata.data.ts (added to mockOperatorSchemaList)
  • mockHuggingFacePredicate in mock-workflow-data.ts

Any related issues, documentation, discussions?

How was this PR tested?

69 unit tests (24 new) in operator-property-edit-frame.component.spec.ts covering:

  • Task preview: text-generation, image-classification, text-to-speech, text-to-video, automatic-speech-recognition, image-to-image, visual-question-answering, question-answering, unknown task fallback, empty task, non-HF operator, deleted operator guard
  • Field type assignments: modelIdhuggingface, imageInputhuggingface-image-upload, audioInputhuggingface-audio-upload
  • Field visibility: imageInput, audioInput, inputImageColumn, inputAudioColumn, promptColumn, contextColumn, candidateLabels, sentencesColumn, systemPrompt, maxNewTokens, temperature — tested across text-generation, image-classification, automatic-speech-recognition, audio-classification, image-to-video, image-to-image, document-question-answering, image-text-to-text, visual-question-answering, zero-shot-classification, summarization, translation, sentence-similarity, text-ranking, zero-shot-image-classification
  • Undefined task visibility: all fields hide when no task is selected
  • Validators: requiredPromptColumn, requiredImageInput, requiredAudioInput — pass/fail cases for empty values, model values, formControl values, and inputColumn alternatives
  • Operator description truncation: tooltip rendering, hidden when unset

Run with ng test.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Opus 4.6

@github-actions github-actions Bot added frontend Changes related to the frontend GUI common labels Jun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 61.84211% with 58 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.35%. Comparing base (878eb8a) to head (4aba551).

Files with missing lines Patch % Lines
...-frame/operator-property-edit-frame.component.html 3.84% 50 Missing ⚠️
...it-frame/operator-property-edit-frame.component.ts 91.83% 5 Missing and 3 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5568      +/-   ##
============================================
- Coverage     56.37%   56.35%   -0.03%     
+ Complexity     2991     2986       -5     
============================================
  Files          1129     1129              
  Lines         43802    43924     +122     
  Branches       4743     4786      +43     
============================================
+ Hits          24694    24753      +59     
- Misses        17658    17715      +57     
- Partials       1450     1456       +6     
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø)
agent-service 44.59% <ø> (ø) Carriedforward from 322babf
amber 57.33% <ø> (-0.04%) ⬇️
computing-unit-managing-service 0.00% <ø> (ø)
config-service 52.30% <ø> (ø)
file-service 62.81% <ø> (ø)
frontend 50.17% <61.84%> (+0.10%) ⬆️
notebook-migration-service 78.57% <ø> (ø)
pyamber 90.51% <ø> (-0.64%) ⬇️ Carriedforward from 322babf
python 90.69% <ø> (ø) Carriedforward from 322babf
workflow-compiling-service 55.14% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ELin2025

ELin2025 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

/request-review @Ma77Ball

@Ma77Ball Ma77Ball left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left comments for review below.

@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch from 553d90d to d1dc52a Compare June 11, 2026 23:38
@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 4 better · 🔴 3 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 878eb8a benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 392 0.239 25,475/36,012/36,012 us 🔴 +7.9% / 🔴 +143.6%
🟢 bs=100 sw=10 sl=64 828 0.506 118,141/142,507/142,507 us 🟢 -23.1% / 🔴 +33.3%
bs=1000 sw=10 sl=64 938 0.573 1,067,081/1,115,481/1,115,481 us ⚪ within ±5% / 🔴 +9.0%
Baseline details

Latest main 878eb8a from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 392 tuples/sec 418 tuples/sec 786.27 tuples/sec -6.2% -50.1%
bs=10 sw=10 sl=64 MB/s 0.239 MB/s 0.255 MB/s 0.48 MB/s -6.3% -50.2%
bs=10 sw=10 sl=64 p50 25,475 us 23,614 us 12,495 us +7.9% +103.9%
bs=10 sw=10 sl=64 p95 36,012 us 35,094 us 14,784 us +2.6% +143.6%
bs=10 sw=10 sl=64 p99 36,012 us 35,094 us 18,468 us +2.6% +95.0%
bs=100 sw=10 sl=64 throughput 828 tuples/sec 782 tuples/sec 991.49 tuples/sec +5.9% -16.5%
bs=100 sw=10 sl=64 MB/s 0.506 MB/s 0.478 MB/s 0.605 MB/s +5.9% -16.4%
bs=100 sw=10 sl=64 p50 118,141 us 118,813 us 100,929 us -0.6% +17.1%
bs=100 sw=10 sl=64 p95 142,507 us 185,337 us 106,894 us -23.1% +33.3%
bs=100 sw=10 sl=64 p99 142,507 us 185,337 us 114,085 us -23.1% +24.9%
bs=1000 sw=10 sl=64 throughput 938 tuples/sec 926 tuples/sec 1,023 tuples/sec +1.3% -8.3%
bs=1000 sw=10 sl=64 MB/s 0.573 MB/s 0.565 MB/s 0.624 MB/s +1.4% -8.2%
bs=1000 sw=10 sl=64 p50 1,067,081 us 1,085,941 us 983,835 us -1.7% +8.5%
bs=1000 sw=10 sl=64 p95 1,115,481 us 1,129,206 us 1,023,777 us -1.2% +9.0%
bs=1000 sw=10 sl=64 p99 1,115,481 us 1,129,206 us 1,053,883 us -1.2% +5.8%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,510.49,200,128000,392,0.239,25474.83,36011.51,36011.51
1,100,10,64,20,2414.33,2000,1280000,828,0.506,118140.76,142506.85,142506.85
2,1000,10,64,20,21321.19,20000,12800000,938,0.573,1067081.03,1115480.66,1115480.66

@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch 3 times, most recently from eafce6d to c009c34 Compare June 12, 2026 00:09
@ELin2025 ELin2025 requested a review from Ma77Ball June 15, 2026 00:52
@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch 2 times, most recently from 7d88b62 to 1bf97cb Compare June 16, 2026 18:08
@github-actions github-actions Bot added engine dependencies Pull requests that update a dependency file pyamber ci changes related to CI docs Changes related to documentations infra platform Non-amber Scala service paths agent-service labels Jun 16, 2026
@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch from 1bf97cb to 5f3afac Compare June 16, 2026 18:14
@github-actions github-actions Bot removed engine dependencies Pull requests that update a dependency file pyamber ci changes related to CI docs Changes related to documentations infra labels Jun 16, 2026
@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch 3 times, most recently from 57f7306 to bdcd1c8 Compare June 23, 2026 16:33
@github-actions github-actions Bot removed the engine label Jun 23, 2026
@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @rbelavadi, @aglinxinyuan, @PG1204
    You can notify them by mentioning @rbelavadi, @aglinxinyuan, @PG1204 in a comment.

@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch 2 times, most recently from fcb2185 to 9c67820 Compare June 23, 2026 23:56

@xuang7 xuang7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks good to me. Please update the target branch to the latest main and rebase your branch, as the current diff includes some unrelated changes. Thanks.

@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch 7 times, most recently from 748349b to 94bc7b2 Compare June 29, 2026 21:45
ELin2025 and others added 11 commits June 30, 2026 10:10
…gFace property editor

Show/hide operator fields based on the selected HuggingFace task (e.g.,
imageInput only for image tasks, contextColumn only for question-answering).
Adds task preview cards with media samples per task kind (image/video/audio/text),
custom validators for required inputs, and ~13 field visibility rules inside
the formly jsonSchemaMapIntercept.

Co-Authored-By: Anish Shivamurthy <anish@uci.edu>
Add mockHuggingFaceSchema and mockHuggingFacePredicate to test infrastructure.
Add 7 spec tests covering huggingFaceTaskPreview for known tasks (text, image,
audio, video), unknown tasks (fallback), empty tasks, and non-HF operators.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ew sample

Fix inverted fallback for systemPrompt/maxNewTokens/temperature: these
fields now correctly hide when no task is selected, matching the behavior
of all other HuggingFace fields. Add missing image-text-to-text entry to
huggingFaceTaskPreviewSamples so it no longer falls through to the
generic text fallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add sample-video.mp4 and sample-audio.wav for task preview cards
- Check hasOperator() before getOperator() in isHuggingFaceOperator()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lassification

Prefer self.CANDIDATE_LABELS over prompt column for candidate labels,
with fallback to prompt_value for backward compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cover hide expressions for imageInput, audioInput, promptColumn,
systemPrompt, contextColumn, and candidateLabels across task types.
Also test requiredPromptColumn, requiredImageInput, and
requiredAudioInput validators pass/fail for relevant tasks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…iew tests

Add tests for sentencesColumn, inputImageColumn, inputAudioColumn
visibility, validator edge cases with column fallbacks, and additional
task preview kinds.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ation show:true

Add missing image-to-video task to imageInputTasks so the image input
field appears for that task. Remove validation: { show: true } from
imageInput, audioInput, and promptColumn so errors only appear after
the field is touched, not on a freshly-added operator.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…fields

Add tests for field type assignments, undefined task visibility,
image-to-video/image-to-image/document-question-answering tasks,
audio-classification, prompt column visibility for mixed tasks,
validator formControl value paths, additional task previews, and
deleted operator guard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ELin2025 ELin2025 force-pushed the hf/07-property-editor branch from 600ba81 to 319fbe4 Compare June 30, 2026 17:10
@ELin2025

Copy link
Copy Markdown
Contributor Author

@xuang7 branch has been rebased

@xuang7 xuang7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good!
sample-audio.wav (170 KB) and sample-video.mp4 (1.25 MB) are new binary files. Could you confirm whether they are self-created or under an ASF-compatible license, and note the source in the PR? Also, the .mp4 is relatively large for a preview thumbnail, so compressing or shortening it would be nice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add task-aware field visibility and preview to HuggingFace property editor

5 participants