10 Prompts for Machine Learning with Scikit-learn, XGBoost, and CatBoost

10 Prompts for Machine Learning with Scikit-learn, XGBoost, and CatBoost

Machine learning workflows often involve repetitive tasks: cleaning data, tuning hyperparameters, handling categorical features, or training boosted trees. Having a set of reusable, battle-tested prompts — code snippets or task descriptions — can save hours and reduce errors. This collection covers three popular libraries: scikit-learn (for classical ML and preprocessing), XGBoost (gradient boosting), and CatBoost (specialized in categorical features). Each prompt is a self-contained task you can adapt to your own dataset.

All examples use Python 3.9+ and the latest versions of the libraries (scikit-learn 1.6, XGBoost 2.1, CatBoost 1.2 as of mid-2026). Official documentation references are provided for each library.

Basic Level

1. Clean and Scale Numerical Features

Task: Impute missing values with median and standardize features to zero mean and unit variance.

Prompt: Use SimpleImputer and StandardScaler from sklearn to create a preprocessing pipeline.

Example Result:

from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
import pandas as pd

# Sample data
df = pd.DataFrame({'age': [25, 30, None, 40], 'income': [50000, 60000, 55000, None]})

pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

cleaned = pipeline.fit_transform(df)
print(cleaned)
# Output (approximate): [[-1.34 -1.41] [0.   0.  ] [0.   0.  ] [1.34 1.41]]

Source: scikit-learn SimpleImputer docs

2. Train-Test Split and Baseline Model

Task: Split data into train/test sets and train a logistic regression classifier.

Prompt: Use train_test_split and LogisticRegression with default parameters.

Example Result:

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)
print('Baseline accuracy:', model.score(X_test, y_test))
# Output: Baseline accuracy: 1.0 (on iris dataset)

Source: scikit-learn train_test_split

Advanced Level

3. Hyperparameter Tuning with GridSearchCV for RandomForest

Task: Find the best n_estimators and max_depth for a Random Forest using 5-fold cross-validation.

Prompt: Use GridSearchCV with RandomForestClassifier and a parameter grid.

Example Result:

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import make_classification

X, y = make_classification(n_samples=1000, n_features=10, random_state=42)

param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [5, 10, None]
}

grid = GridSearchCV(RandomForestClassifier(random_state=42), param_grid, cv=5, scoring='accuracy')
grid.fit(X, y)

print('Best params:', grid.best_params_)
print('Best CV score:', grid.best_score_)
# Output: Best params: {'max_depth': 10, 'n_estimators': 200}
# (results may vary)

Source: scikit-learn GridSearchCV

4. Feature Importance with XGBoost and SHAP

Task: Train an XGBoost classifier and explain predictions using SHAP values.

Prompt: Fit XGBClassifier, then compute SHAP values with TreeExplainer.

Example Result:

import xgboost as xgb
import shap
from sklearn.datasets import load_diabetes

X, y = load_diabetes(return_X_y=True)
model = xgb.XGBRegressor(n_estimators=100, random_state=42)
model.fit(X, y)

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X)

# Summary plot (in notebook):
shap.summary_plot(shap_values, X, feature_names=load_diabetes().feature_names)

Source: XGBoost Python docs and SHAP docs

5. Handling Imbalanced Data with XGBoost scale_pos_weight

Task: Improve recall on rare positive class by adjusting scale_pos_weight.

Prompt: Set scale_pos_weight = sum(negative) / sum(positive) in XGBClassifier.

Example Result:

import numpy as np
from sklearn.datasets import make_classification

X, y = make_classification(n_samples=10000, weights=[0.95, 0.05], random_state=42)
ratio = np.sum(y == 0) / np.sum(y == 1)

model = xgb.XGBClassifier(scale_pos_weight=ratio, random_state=42)
model.fit(X, y)

from sklearn.metrics import classification_report
print(classification_report(y, model.predict(X)))
# Precision/recall for class 1 will improve significantly.

Source: XGBoost Handling Imbalanced Data

Expert Level

6. Custom Objective Function in XGBoost

Task: Implement a custom squared log error objective for regression with XGBoost.

Prompt: Define a function grad_hess(preds, dtrain) that returns gradient and hessian.

Example Result:

import xgboost as xgb
from sklearn.datasets import make_regression

X, y = make_regression(n_samples=1000, random_state=42)
dtrain = xgb.DMatrix(X, label=y)

def squared_log_error(preds, dtrain):
    y = dtrain.get_label()
    preds = np.clip(preds, 1e-7, None)
    res = (np.log1p(y) - np.log1p(preds))
    grad = -2 * res / (1 + preds)
    hess = 2 * (res + 1) / ((1 + preds)**2)
    return grad, hess

params = {'objective': squared_log_error, 'eval_metric': 'rmse'}
model = xgb.train(params, dtrain, num_boost_round=50)

Source: XGBoost Custom Objective

7. Automatic Categorical Features in CatBoost

Task: Train a CatBoost classifier without manual encoding of categorical variables.

Prompt: Pass categorical feature indices via cat_features parameter in CatBoostClassifier.

Example Result:

from catboost import CatBoostClassifier, Pool
import pandas as pd

df = pd.DataFrame({
    'color': ['red', 'blue', 'green', 'red'],
    'size': ['S', 'M', 'L', 'S'],
    'target': [0, 1, 1, 0]
})
X = df[['color', 'size']]
y = df['target']

# Indices of categorical columns (0 and 1)
model = CatBoostClassifier(cat_features=[0, 1], iterations=10, verbose=0)
model.fit(X, y)
print('Predictions:', model.predict([['blue', 'L']]))
# Output: Predictions: [1.]

Source: CatBoost Python Tutorial

8. Early Stopping and Cross-Validation with CatBoost

Task: Use cv function to find optimal number of iterations with early stopping.

Prompt: Call catboost.cv() with early_stopping_rounds=10.

Example Result:

from catboost import Pool, cv
import numpy as np

X, y = make_classification(n_samples=500, random_state=42)
pool = Pool(X, y)

params = {
    'iterations': 1000,
    'learning_rate': 0.03,
    'loss_function': 'Logloss',
    'verbose': 0
}

cv_results = cv(pool, params, fold_count=5, early_stopping_rounds=10, verbose_eval=False)
best_iter = np.argmin(cv_results['test-Logloss-mean'])
print(f'Best iteration: {best_iter}')

Source: CatBoost Cross-Validation

9. Pipeline with ColumnTransformer for Mixed Data

Task: Combine numeric and categorical pipelines using ColumnTransformer.

Prompt: Use ColumnTransformer to apply StandardScaler to numeric columns and OneHotEncoder to categorical columns.

Example Result:

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.ensemble import RandomForestClassifier

numeric_features = ['age', 'income']
categorical_features = ['education', 'city']

numeric_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

categorical_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('onehot', OneHotEncoder(drop='first'))
])

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numeric_pipeline, numeric_features),
        ('cat', categorical_pipeline, categorical_features)
    ]
)

pipeline = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('classifier', RandomForestClassifier())
])

# Now pipeline.fit(X_train, y_train) handles mixed data automatically.

Source: scikit-learn ColumnTransformer

10. Interpretability with Permutation Importance and Partial Dependence

Task: Measure feature importance using permutation method and visualize partial dependence.

Prompt: Use permutation_importance from sklearn and PartialDependenceDisplay.

Example Result:

from sklearn.inspection import permutation_importance, PartialDependenceDisplay
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import load_diabetes
import matplotlib.pyplot as plt

X, y = load_diabetes(return_X_y=True)
model = RandomForestRegressor(random_state=42)
model.fit(X, y)

result = permutation_importance(model, X, y, n_repeats=10, random_state=42)
print('Mean importance:', result.importances_mean)

# Partial dependence for top feature
PartialDependenceDisplay.from_estimator(model, X, [0, 1])
plt.show()

Source: scikit-learn Inspection

Conclusion

These 10 prompts cover the most common ML tasks using scikit-learn, XGBoost, and CatBoost. They provide a starting point for data cleaning, model training, tuning, and interpretation. Adapt them to your own data, adjust parameters, and consult the official documentation for deeper customization. The key is to build reusable patterns — once you have a working pipeline, you can iterate faster and focus on solving the real problem.

If you found a particular prompt useful or have your own favorite, share it with the community. Happy modelling!