[Feature]: speedup autoscheme with multi-process and cache files

[xin3he]

Description

Feature Description

Save and use cached per-op score files to skip time-consuming sensitivity evaluation.
Enable multi-process to speedup the evaluation. It will fallback if multi-process fails, like OOM.
export AR_ENABLE_AUTO_SCHEME_PARALLEL=1

Motivation and Use Case

For LLM, like GLM5.2, the AutoScheme usually takes 1h.
For different bits and shared_layers, we usually need to re-run the entire AutoScheme process, which is not necessary if we cache the per-op score.

Real model test

1. auto-round /models/Qwen3.6-35B-A3B/ --options "MXFP4,MXFP8" --target_bits 4.3
2. auto-round /models/Qwen3.6-35B-A3B/ --options "MXFP4,MXFP8" --target_bits 4.5 --shared_layers q_proj k_proj v_proj
3. auto-round /models/Qwen3-32B/ --options "MXFP4,MXFP8" --target_bits 4.5
4. auto-round /models/Qwen3-32B/ --options "MXFP4,MXFP8" --target_bits 4.6 --shared_layers q_proj k_proj v_proj

IDX	Main 耗时	This Branch 耗时	耗时变化	Main RAM	This Branch RAM	RAM 变化	Main VRAM	This Branch VRAM	VRAM 变化
1	29m 53s	18m 30s	↓ 11m 23s（38.1%）	70.82 GB	140.45 GB	↑ 69.63 GB（98.3%）	10.23 GB	20.45 GB	↑ 10.22 GB（99.9%）
2	23m 27s	38s	↓ 22m 49s（97.3%）	70.93 GB	4.31 GB	↓ 66.62 GB（93.9%）	10.23 GB	2.76 GB	↓ 7.47 GB（73.0%）
3	8m 01s	6m 49s	↓ 1m 12s（15.0%）	67.23 GB	132.68 GB	↑ 65.45 GB（97.4%）	14.36 GB	28.97 GB	↑ 14.61 GB（101.7%）
4	7m 02s	1s	↓ 7m 01s（99.8%）	67.27 GB	6.92 GB	↓ 60.35 GB（89.7%）	14.36 GB	2.90 GB	↓ 11.46 GB（79.8%）

Type of Change

New feature

Related Issues

Fixes or relates to #2080 #1988

Checklist Before Submitting

• My code has been tested locally.
• Documentation has been updated as needed.
• New or updated tests are included where applicable.
• The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.

[Copilot]

Pull request overview

This PR speeds up AutoScheme by adding per-scheme, per-op score caching and optional multi-process scheme scoring, reducing repeated sensitivity evaluation cost across runs. It also broadens related CLI parsing and model-free export support to better interoperate with AutoScheme outputs.

Changes:

• Add per-scheme JSON caching (per-op layer scores) and multiprocessing-based scoring with serial fallback in AutoScheme.
• Improve CLI ergonomics for AutoScheme flags (--options, --shared_layers) and consolidate/expand CPU tests around the new behaviors.
• Extend model-free pathways with BF16 handling and MXFP “auto_round” metadata support; add unified load_model helper and adjust device/memory utilities.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/test_cpu/utils/test_cli_usage.py	Adds tests for comma/space normalization of --options and --shared_layers.
test/test_cpu/schemes/test_auto_scheme.py	Adds tests for env overrides, cache files, cache key behavior, multiprocessing helpers, and expert grouping; consolidates prior internal tests.
test/test_cpu/schemes/test_auto_scheme_internal.py	Removes now-duplicated internal tests (moved into test_auto_scheme.py).
test/test_cpu/quantization/test_model_free.py	Expands model-free tests for BF16 and MXFP auto_round-format metadata output.
test/helpers.py	Updates the MoE reference model ID used by fixtures.
test/fixtures.py	Updates tiny MoE fixture generation (from-config) and workspace cleanup paths.
auto_round/utils/model.py	Introduces unified load_model() dispatching across LLM/MLLM/diffusion.
auto_round/utils/device.py	Replaces expensive partitioning with greedy balancing; adds process-tree RSS accounting.
auto_round/utils/device_manager.py	Hardens total-memory querying across backends and improves error handling.
auto_round/modeling/fused_moe/replace_modules.py	Adjusts safe_to_cpu_ to avoid .to("cpu") when meta tensors exist.
auto_round/compressors/model_free.py	Adds BF16 support, MXFP auto_round-style quantization_config generation, and better AutoScheme option validation/conversion.
auto_round/cli/parser.py	Makes --options accept multiple args; clarifies --shared_layers help text.
auto_round/cli/main.py	Normalizes new --options/--shared_layers argument shapes before constructing AutoScheme.
auto_round/auto_scheme/utils.py	Improves scheme-loss matrix headers (supports expert column).
auto_round/auto_scheme/delta_loss.py	Implements per-scheme cache, multiprocessing scoring workers, safer grad/progress handling, and uses unified load_model().

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

auto_round/auto_scheme/delta_loss.py:2177

• Per-scheme cache keys are computed even when _model_id_for_cache is None (e.g., when AutoScheme is run on an in-memory model without config._name_or_path). That yields the same cache key across unrelated models (model_name=null), which can silently reuse incorrect per-op losses. Cache read/write should be disabled when no stable model identifier is available.

        scheme_cache_meta = []
        for index, scheme in enumerate(schemes):
            if check_bf16_scheme(scheme):
                scheme_cache_meta.append((None, None, None))
                logger.info(

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[n1ck-guo]

LGTM

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).