Spaces:
Running
feat: Improve New Evaluation screen hardware selection
Browse filesEnhancements to provider and hardware configuration:
Provider Changes:
- Removed 'ollama' from provider dropdown (not supported in HF Jobs/Modal)
- Updated provider choices to: litellm, inference, transformers
- Added clarification info text: (litellm/inference=API, transformers=local)
Hardware Changes:
- Simplified GPU options: removed gpu_a10 and gpu_h200
- Added single GPU option: gpu_a100 (widely available on HF/Modal)
- Updated hardware choices to: auto, cpu, gpu_a100
- Changed info text to reflect auto-selection
Auto-Selection Logic:
- Added on_provider_change() function
- Wired up eval_provider.change event
- Smart hardware selection:
* litellm/inference → cpu (API models don't need GPU)
* transformers → gpu_a100 (local models need GPU)
* default → auto
Benefits:
- Prevents invalid configurations (e.g., ollama without server)
- Automatically selects appropriate hardware for provider type
- Simplified GPU options (single A100 choice vs multiple)
- Better UX with clear provider/hardware relationship
|
@@ -2220,10 +2220,10 @@ with gr.Blocks(title="TraceMind-AI", theme=theme) as app:
|
|
| 2220 |
)
|
| 2221 |
|
| 2222 |
eval_hardware = gr.Radio(
|
| 2223 |
-
choices=["auto", "cpu", "
|
| 2224 |
value="auto",
|
| 2225 |
label="Hardware",
|
| 2226 |
-
info="CPU for API
|
| 2227 |
)
|
| 2228 |
|
| 2229 |
# Section 2: Model Configuration
|
|
@@ -2239,10 +2239,10 @@ with gr.Blocks(title="TraceMind-AI", theme=theme) as app:
|
|
| 2239 |
)
|
| 2240 |
|
| 2241 |
eval_provider = gr.Dropdown(
|
| 2242 |
-
choices=["litellm", "inference", "transformers"
|
| 2243 |
value="litellm",
|
| 2244 |
label="Provider",
|
| 2245 |
-
info="Model inference provider"
|
| 2246 |
)
|
| 2247 |
|
| 2248 |
with gr.Row():
|
|
@@ -2678,6 +2678,17 @@ with gr.Blocks(title="TraceMind-AI", theme=theme) as app:
|
|
| 2678 |
|
| 2679 |
return gr.update(value=success_html, visible=True)
|
| 2680 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2681 |
# Navigation handlers (define before use)
|
| 2682 |
def navigate_to_dashboard():
|
| 2683 |
"""Navigate to dashboard screen and load dashboard data"""
|
|
@@ -3233,6 +3244,13 @@ Result: {result}
|
|
| 3233 |
outputs=[eval_cost_estimate]
|
| 3234 |
)
|
| 3235 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3236 |
eval_submit_btn.click(
|
| 3237 |
fn=on_submit_evaluation_comprehensive,
|
| 3238 |
inputs=[
|
|
|
|
| 2220 |
)
|
| 2221 |
|
| 2222 |
eval_hardware = gr.Radio(
|
| 2223 |
+
choices=["auto", "cpu", "gpu_a100"],
|
| 2224 |
value="auto",
|
| 2225 |
label="Hardware",
|
| 2226 |
+
info="Auto-selected based on provider (CPU for API, GPU for local models)"
|
| 2227 |
)
|
| 2228 |
|
| 2229 |
# Section 2: Model Configuration
|
|
|
|
| 2239 |
)
|
| 2240 |
|
| 2241 |
eval_provider = gr.Dropdown(
|
| 2242 |
+
choices=["litellm", "inference", "transformers"],
|
| 2243 |
value="litellm",
|
| 2244 |
label="Provider",
|
| 2245 |
+
info="Model inference provider (litellm/inference=API, transformers=local)"
|
| 2246 |
)
|
| 2247 |
|
| 2248 |
with gr.Row():
|
|
|
|
| 2678 |
|
| 2679 |
return gr.update(value=success_html, visible=True)
|
| 2680 |
|
| 2681 |
+
def on_provider_change(provider):
|
| 2682 |
+
"""Auto-select hardware based on provider type"""
|
| 2683 |
+
# litellm and inference are for API models → CPU
|
| 2684 |
+
# transformers is for local models → GPU
|
| 2685 |
+
if provider in ["litellm", "inference"]:
|
| 2686 |
+
return gr.update(value="cpu")
|
| 2687 |
+
elif provider == "transformers":
|
| 2688 |
+
return gr.update(value="gpu_a100")
|
| 2689 |
+
else:
|
| 2690 |
+
return gr.update(value="auto")
|
| 2691 |
+
|
| 2692 |
# Navigation handlers (define before use)
|
| 2693 |
def navigate_to_dashboard():
|
| 2694 |
"""Navigate to dashboard screen and load dashboard data"""
|
|
|
|
| 3244 |
outputs=[eval_cost_estimate]
|
| 3245 |
)
|
| 3246 |
|
| 3247 |
+
# Auto-select hardware when provider changes
|
| 3248 |
+
eval_provider.change(
|
| 3249 |
+
fn=on_provider_change,
|
| 3250 |
+
inputs=[eval_provider],
|
| 3251 |
+
outputs=[eval_hardware]
|
| 3252 |
+
)
|
| 3253 |
+
|
| 3254 |
eval_submit_btn.click(
|
| 3255 |
fn=on_submit_evaluation_comprehensive,
|
| 3256 |
inputs=[
|