Improving Hybrid Quantum Models through Efficient Model Selection: A pathway for scalable quantum machine learning

Hello, I am Rahul Bhowmick from the Quantum team at Fujitsu Research India Private Limited (FRIPL).

With advancements in both quantum hardware and software, quantum computing is steadily moving from theory toward practical applications. Fields such as chemistry, materials science, optimization, and machine learning are all expected to benefit from this progress. At FRIPL, we are actively contributing to this effort through both theory and algorithmic research. While fault-tolerant quantum computers are still some years away, there is growing interest in identifying useful methods for today’s hybrid quantum-classical setting.

In this article, I will discuss our recent work on improving the development of hybrid quantum machine learning models by making model selection more efficient. Instead of training every possible candidate architecture which can run into 100s of variants, our approach aims to identify promising models in advance using lightweight evaluation metrics.

Introduction

Quantum machine learning models offer exciting possibilities, but they also introduce a difficult design problem. When building a hybrid quantum neural network, we need to decide several things:

• how to encode the input data,
• what kind of quantum circuit to use,
• how to initialize the trainable parameters, and
• how to combine the quantum and classical parts of the model.

In practice, evaluating these choices usually requires full training of each candidate model. This is expensive even in classical machine learning, and becomes even more challenging when quantum resources are involved. As a result, model development can quickly become a bottleneck.

In our recent work, we address this issue by developing a framework that allows us to estimate the quality of candidate models before carrying out full training.

Current Status and Challenges

Model selection is a standard part of machine learning. In classical deep learning, practitioners often compare multiple architectures, hyperparameters, and initialization strategies before deciding on a final model. However, in quantum machine learning, this process is much less mature.

One major challenge is that quantum models generally do not yet have strong design rules that tell us in advance which architecture is likely to perform well for a given task. As a result, researchers often rely on repeated training across many candidate models. This brute-force approach is resource intensive and does not scale well.

Our Technical Contribution

To address this, we introduce the Quantum Bias-Expressivity Toolbox (QBET), a framework for efficient model selection in quantum, classical, and hybrid neural architectures. The main idea is simple: instead of fully training every candidate model, we first evaluate it using lightweight metrics that capture important properties of the model class.

Design and algorithmic workflow of QBET: The process consists of three stages: (i) Input, where a model is selected. This stage is data-agnostic and focuses solely on architectural specification rather than dataset-dependent characteristics; (ii) Execution, which involves converting the model into a Boolean classifier, generating Boolean functions, computing LZ complexity, and producing probability distributions over multiple trials; and (iii) Evaluation, where cumulative distribution functions are constructed, Area Under Curve (AUC) and EXP values are derived to quantify the bias–expressivity trade-off, and top-k configurations are selected for downstream performance metric computation based on AUC.

In particular, QBET uses two key quantities:

Simplicity Bias

Simplicity bias measures how strongly a model tends to favor simpler functions, and connects input and output relations for tasks like classification, generation, etc. A model with stronger simplicity bias may generalize better and avoid overfitting, while a model with weaker simplicity bias may be able to represent more complicated patterns but may also become harder to train reliably.

Expressivity

Expressivity measures how rich or flexible the model is. A highly expressive model can represent a wider range of functions, but high expressivity alone does not guarantee strong performance. In fact, overly expressive models can sometimes become difficult to optimize or may generalize poorly.

The key point is that neither simplicity bias nor expressivity should be considered in isolation. Instead, good model design often depends on a balance between the two.

Application to Hybrid Quantum Transformers

We apply QBET to model selection in hybrid quantum neural networks, with a particular focus on quantum transformer architectures. Transformers have become central to many modern machine learning applications, so they provide a natural testbed for studying how quantum components may be incorporated into practical model pipelines. In our work, we evaluate both classification and generative settings, allowing us to test whether the proposed bias–expressivity analysis remains useful across different tasks.

Overview of QBET: input data types (molecular graphs, text sequences, images), tasks (generation and classification), and candidate models (Transformer Encoder, Self-Attention Mechanism-Generative Adversarial Network [SAM-GAN], Transformer Decoder). The model selection workflow involves three steps: (1) choosing the data, task, and model; (2) listing model variants such as classical, quantum, and hybrid approaches, initialization strategies, circuit or network types, and depth/activation choices; and (3) running the toolbox to identify the variant that achieves optimal performance using minimal resources

More specifically, we study image classification on CIFAR-10 and molecular generation on QM9. Across the transformer-based experiments, we use a total of 18 qubits for embeddings, allocated as 6 qubits each for the query, key, and value representations. We then compare classical, quantum, and hybrid architectural variants using QBET before full training, so that the ranking is guided by intrinsic properties of the models rather than downstream optimization alone.

The architectural search space includes multiple choices such as the encoding scheme, the quantum measurement strategy, the attention-score computation method, and the depth or layer configuration. Rather than trying all such combinations exhaustively through full training, QBET first computes lightweight metrics such as Simplicity Bias (via Area Under Curve or AUC) and Expressivity, and then uses these quantities to prioritize the most promising candidates.

Our Key Findings

Our study leads to several important observations.

QBET can identify promising transformer variants before full training

Across the evaluated transformer-based architectures, QBET ranks candidate models using lightweight metrics derived from simplicity bias and expressivity, without requiring full end-to-end optimization. This makes model exploration substantially more efficient, especially when the design space includes multiple choices for encoding, attention construction, initialization, and hybridization strategy.

High-ranked configurations show competitive downstream performance

The CIFAR-10 results provide a concrete illustration of this point. The top-ranked TF-Encoder configurations consistently achieve higher test accuracies compared to the rest of the configurations, as shown below by the Pearson-correlation factor between AUC and performance metrics: test and train accuracy. The p-value of the correlation is also provided in the upper triangle.

This heatmap illustrates the strength of correlations between performance metrics and bias-expressivity measures for TF-Encoder model for image classification task. The upper triangle displays the statistical significance of the observed relationships, highlighting the reliability of the experimental findings.

These results show that QBET is able to narrow the search toward configurations that remain competitive after training.

Expressivity alone is not enough

One might expect that the most expressive model would always perform best. However, the results support a more nuanced picture: performance depends not only on the ability to represent many functions, but also on the model’s inductive tendency toward simpler ones. This is precisely why the joint use of Simplicity Bias and Expressivity is useful for ranking candidate architectures.

The framework extends beyond classification

An important strength of QBET is that it is not tied to a single task type. In the paper, the same methodology is extended from standard classification to generative modeling and multiclass settings, and is evaluated on both CIFAR-10 and QM9. For these architectures, we simply replace the output layers with a fully connected layer with two output nodes to replicate a binary classifier, keeping the core blocks intact. In the below table, we show that QBET can identify hybrid quantum-classical variants which can outperform the classical baselines in performance metrics while using similar number of parameters:

Performance comparison of classical and top-10 AUC value SAM-GAN model configurations for molecular graph generation. Performance metrics are averaged over 10 initializations.

This suggests that the approach can serve as a more general toolbox for architecture selection in quantum, classical, and hybrid neural networks.

Hybrid quantum variants can be worthwhile in selected regimes

The paper identifies scenarios in which quantum self-attention variants surpass their classical counterparts when the architectures are ranked using the proposed simplicity-bias metric and then evaluated downstream. The identified quantum variants show improvement in metrics like test accuracy, uniqueness in generation or reduction in number of parameters or both, compared with a classical baseline of similar architecture. This does not by itself establish quantum advantage in the strict complexity-theoretic sense, but it does show that principled model selection can reveal practically useful quantum or hybrid designs.

Conclusion

In this article, I discussed our recent work on model selection in hybrid quantum neural networks.

Instead of training every possible model candidate, we propose using the Quantum Bias-Expressivity Toolbox (QBET) to evaluate models in advance through lightweight metrics such as simplicity bias and expressivity. This enables efficient pre-screening of promising architectures and can significantly reduce the burden of model development.

We believe that such model selection tools will become increasingly important as quantum machine learning continues to grow and hybrid architectures become more sophisticated.

For more details, please check our paper: https://arxiv.org/abs/2603.21749