Score a dummy baseline on the same split and metric as your model

scikit-learn's docs describe comparing an estimator against simple rules of thumb as a basic sanity check, and provide DummyClassifier and DummyRegressor for it. They ignore the input features and predict, for example, the most frequent class or the mean target.

If your model barely beats the dummy, the docs note that something has probably gone wrong: the features are not helpful, a hyperparameter is poorly tuned or class imbalance is distorting the result.

In the docs' own example on an imbalanced subset of the Iris data, a linear support vector classifier scores 0.63 accuracy against 0.579 for the most-frequent dummy, while an RBF-kernel classifier reaches 0.94. The dummy makes it obvious which model is actually learning.

from sklearn.dummy import DummyClassifier
from sklearn.metrics import balanced_accuracy_score

dummy = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print("dummy:", balanced_accuracy_score(y_val, dummy.predict(X_val)))
print("model:", balanced_accuracy_score(y_val, model.predict(X_val)))