15 Prompts for Machine Learning: From Preprocessing to Model Training with Scikit-learn, XGBoost, and CatBoost

Introduction

Machine learning (ML) is transforming industries, but building effective models requires more than just knowing algorithms—it demands systematic workflows. Whether you are a data scientist fine-tuning a gradient booster or a ML engineer debugging a pipeline, the right prompts can guide your thinking and code. This article collects 15 practical prompts for ML tasks using three of the most popular libraries: Scikit-learn, XGBoost, and CatBoost. Each prompt is a complete, actionable template you can adapt to your own data.

These prompts cover the full ML lifecycle: data preprocessing, feature engineering, model training, hyperparameter tuning, and evaluation. They are designed for both beginners—who need clear explanations—and experts looking for concise references. By the end, you will have a reusable toolkit for faster, more reliable ML workflows.

Why These Libraries?

• Scikit-learn (Pedregosa et al., 2011) is the industry standard for classical ML algorithms, preprocessing, and pipelines. Its API is consistent and well-documented at scikit-learn.org.
• XGBoost (Chen & Guestrin, 2016) is a gradient boosting framework optimized for speed and performance, widely used in Kaggle competitions. Official docs: xgboost.readthedocs.io.
• CatBoost (Dorogush et al., 2018) handles categorical features natively and reduces overfitting with ordered boosting. Learn more at catboost.ai.

Prompt 1: Automated Missing Value Imputation

Task: Write a function that detects missing values in a DataFrame and imputes them using Scikit-learn's SimpleImputer with a strategy based on column data type.

Prompt:

"Create a Python function smart_impute(df) using Scikit-learn's SimpleImputer. For numeric columns, use median imputation; for categorical columns, use most_frequent. Return the imputed DataFrame and a dictionary of imputers per column."

Example Result:

import pandas as pd
from sklearn.impute import SimpleImputer

def smart_impute(df):
    imputers = {}
    df_imp = df.copy()
    for col in df.columns:
        if df[col].dtype in ['int64', 'float64']:
            imputer = SimpleImputer(strategy='median')
        else:
            imputer = SimpleImputer(strategy='most_frequent')
        df_imp[col] = imputer.fit_transform(df[[col]]).ravel()
        imputers[col] = imputer
    return df_imp, imputers

Explanation: This prompt ensures you handle missing data robustly. Using median for numerics avoids outlier sensitivity, while most_frequent preserves categorical distributions. Always fit imputers on training data only, then transform test data—this function can be extended with a transform_only mode.

Prompt 2: Custom One-Hot Encoding with Fallback

Task: Encode categorical features with OneHotEncoder, but automatically handle unseen categories during inference by ignoring them.

Prompt:

"Using Scikit-learn's OneHotEncoder, configure it to handle unknown categories by ignoring them. Show code that fits on training data and transforms test data with a new category."

Example Result:

from sklearn.preprocessing import OneHotEncoder
import pandas as pd

train = pd.DataFrame({'color': ['red', 'blue', 'green']})
test = pd.DataFrame({'color': ['red', 'yellow']})  # 'yellow' unseen

encoder = OneHotEncoder(handle_unknown='ignore', sparse_output=False)
train_encoded = encoder.fit_transform(train)
test_encoded = encoder.transform(test)
print(test_encoded.shape)  # (1, 3) because 'yellow' ignored

Explanation: The handle_unknown='ignore' parameter creates a zero vector for unseen categories, preventing pipeline crashes. This is critical for production systems where new categories appear over time.

Prompt 3: Feature Scaling Pipeline

Task: Build a Scikit-learn Pipeline that standardizes numeric features and leaves others unchanged.

Prompt:

"Construct a pipeline using ColumnTransformer and StandardScaler. Numeric columns should be scaled, categorical columns should be passed through unchanged. Use the iris dataset as an example."

Example Result:

from sklearn.datasets import load_iris
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
import pandas as pd

iris = load_iris()
df = pd.DataFrame(iris.data, columns=iris.feature_names)
df['species'] = iris.target_names[iris.target]

numeric_features = iris.feature_names
preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numeric_features)
    ],
    remainder='passthrough'  # keep species column as-is
)

pipeline = Pipeline(steps=[('preprocessor', preprocessor)])
df_scaled = pipeline.fit_transform(df)
print(df_scaled.shape)  # (150, 5)

Explanation: ColumnTransformer lets you apply different preprocessing steps to different columns. remainder='passthrough' ensures non-numeric columns (like species) are kept, avoiding data loss.

Prompt 4: Train-Test Split with Stratification

Task: Split a dataset into training and test sets while preserving the class distribution.

Prompt:

"Write code using train_test_split from Scikit-learn to perform a stratified split on a classification dataset. Use 80% training, 20% test, and set a random seed for reproducibility."

Example Result:

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)
print(f"Train size: {X_train.shape[0]}, Test size: {X_test.shape[0]}")

Explanation: Stratification ensures each class is proportionally represented in both splits. This prevents biased evaluation when classes are imbalanced. Always set random_state for reproducibility.

Prompt 5: XGBoost Model with Early Stopping

Task: Train an XGBoost classifier using early stopping to avoid overfitting, with a validation set.

Prompt:

"Using XGBoost's XGBClassifier, train a model with eval_set, early_stopping_rounds=10, and eval_metric='logloss'. Show how to retrieve the best iteration."

Example Result:

import xgboost as xgb
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

model = xgb.XGBClassifier(
    n_estimators=1000,
    learning_rate=0.1,
    early_stopping_rounds=10,
    eval_metric='logloss',
    random_state=42
)
model.fit(
    X_train, y_train,
    eval_set=[(X_val, y_val)],
    verbose=False
)
print(f"Best iteration: {model.best_iteration}")

Explanation: Early stopping monitors validation loss and halts training when no improvement occurs for early_stopping_rounds consecutive rounds. This saves time and reduces overfitting. The best_iteration attribute shows the optimal number of trees.

Prompt 6: CatBoost with Categorical Features

Task: Train a CatBoost classifier that automatically handles categorical features without manual encoding.

Prompt:

"Using CatBoost's CatBoostClassifier, specify which columns are categorical using the cat_features parameter. Train on a dataset with mixed types and evaluate accuracy."

Example Result:

from catboost import CatBoostClassifier
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

df = pd.DataFrame({
    'age': [25, 30, 35],
    'city': ['NYC', 'LA', 'NYC'],
    'income': [50000, 60000, 70000],
    'target': [0, 1, 0]
})
X = df.drop('target', axis=1)
y = df['target']
cat_features = ['city']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

model = CatBoostClassifier(iterations=10, verbose=0, random_seed=42)
model.fit(X_train, y_train, cat_features=cat_features)
y_pred = model.predict(X_test)
print(f"Accuracy: {accuracy_score(y_test, y_pred):.2f}")

Explanation: CatBoost's native categorical support uses ordered target encoding to reduce overfitting. You only need to pass column indices or names—no OneHotEncoder required. This simplifies pipelines and often improves performance.

Prompt 7: Hyperparameter Tuning with GridSearchCV

Task: Perform grid search over XGBoost hyperparameters using Scikit-learn's GridSearchCV.

Prompt:

"Set up a GridSearchCV for XGBClassifier with parameter grid: max_depth [3,5,7], learning_rate [0.01, 0.1], and n_estimators [50, 100]. Use 5-fold cross-validation and return the best parameters."

Example Result:

from sklearn.model_selection import GridSearchCV
import xgboost as xgb
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
model = xgb.XGBClassifier(random_state=42, use_label_encoder=False, eval_metric='logloss')
param_grid = {
    'max_depth': [3, 5, 7],
    'learning_rate': [0.01, 0.1],
    'n_estimators': [50, 100]
}
grid = GridSearchCV(model, param_grid, cv=5, scoring='accuracy', n_jobs=-1)
grid.fit(X, y)
print(f"Best params: {grid.best_params_}")

Explanation: GridSearchCV exhaustively searches all combinations—here 3×2×2=12 fits, each with 5-fold CV, totaling 60 model trainings. Use n_jobs=-1 to parallelize. For larger grids, consider RandomizedSearchCV.

Prompt 8: Randomized Search for CatBoost

Task: Use RandomizedSearchCV to tune CatBoost parameters more efficiently than grid search.

Prompt:

"Implement a randomized search for CatBoostClassifier with iterations (10, 50), depth (4, 6, 8), and learning_rate (0.01, 0.1, 0.3). Sample 10 random combinations with 3-fold CV."

Example Result:

from sklearn.model_selection import RandomizedSearchCV
from catboost import CatBoostClassifier
from sklearn.datasets import load_breast_cancer
import numpy as np

X, y = load_breast_cancer(return_X_y=True)
model = CatBoostClassifier(verbose=0, random_seed=42)
param_dist = {
    'iterations': [10, 50],
    'depth': [4, 6, 8],
    'learning_rate': [0.01, 0.1, 0.3]
}
random_search = RandomizedSearchCV(
    model, param_dist, n_iter=10, cv=3, scoring='accuracy', random_state=42, n_jobs=-1
)
random_search.fit(X, y)
print(f"Best params: {random_search.best_params_}")

Explanation: Randomized search samples a fixed number of combinations (n_iter), making it faster for large spaces. It's often more effective than grid search because it explores more values per hyperparameter.

Prompt 9: Feature Importance Visualization

Task: Plot feature importance from a trained XGBoost or CatBoost model.

Prompt:

"After training an XGBoost classifier, extract feature importances using model.feature_importances_ and plot a horizontal bar chart of the top 10 features using Matplotlib."

Example Result:

import matplotlib.pyplot as plt
import pandas as pd
import xgboost as xgb
from sklearn.datasets import load_iris

iris = load_iris()
X, y = iris.data, iris.target
model = xgb.XGBClassifier(random_state=42, eval_metric='mlogloss')
model.fit(X, y)
importances = model.feature_importances_
features = iris.feature_names

imp_df = pd.DataFrame({'feature': features, 'importance': importances})
imp_df = imp_df.sort_values('importance', ascending=False).head(10)

plt.figure(figsize=(8, 5))
plt.barh(imp_df['feature'], imp_df['importance'])
plt.xlabel('Importance')
plt.title('XGBoost Feature Importances')
plt.gca().invert_yaxis()
plt.show()

Explanation: Feature importance helps interpret which variables drive predictions. XGBoost's importance is based on how often a feature is used for splitting. For CatBoost, use get_feature_importance().

Prompt 10: Cross-Validation with Pipeline

Task: Integrate preprocessing and model training in a single pipeline, then cross-validate.

Prompt:

"Build a pipeline with StandardScaler and LogisticRegression, then evaluate it using cross_val_score with 5-fold CV on the wine dataset."

Example Result:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_wine

X, y = load_wine(return_X_y=True)
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression(max_iter=1000, random_state=42))
])
scores = cross_val_score(pipeline, X, y, cv=5, scoring='accuracy')
print(f"Accuracies: {scores}")
print(f"Mean: {scores.mean():.3f} (+/- {scores.std():.3f})")

Explanation: Pipelines ensure that preprocessing (scaling) is applied inside each fold, preventing data leakage. This is a best practice for honest evaluation.

Prompt 11: Handling Imbalanced Data with Class Weights

Task: Train a Scikit-learn classifier with class weights to handle imbalanced datasets.

Prompt:

"Use RandomForestClassifier with class_weight='balanced' on the breast cancer dataset. Compare accuracy and recall with a model without class weights."

Example Result:

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Without class weights
rf_default = RandomForestClassifier(random_state=42)
rf_default.fit(X_train, y_train)
y_pred_default = rf_default.predict(X_test)
print("Default:")
print(classification_report(y_test, y_pred_default))

# With balanced class weights
rf_balanced = RandomForestClassifier(class_weight='balanced', random_state=42)
rf_balanced.fit(X_train, y_train)
y_pred_balanced = rf_balanced.predict(X_test)
print("Balanced:")
print(classification_report(y_test, y_pred_balanced))

Explanation: class_weight='balanced' automatically adjusts weights inversely proportional to class frequencies. This improves recall for minority classes, often at a slight cost to precision.

Prompt 12: Saving and Loading Models

Task: Save a trained XGBoost model to a file and load it later for inference.

Prompt:

"Using joblib, save a trained XGBoost model to 'model.pkl', then load it and make predictions on new data."

Example Result:

import joblib
import xgboost as xgb
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, _, y_train, _ = train_test_split(X, y, test_size=0.2, random_state=42)
model = xgb.XGBClassifier(random_state=42, eval_metric='logloss')
model.fit(X_train, y_train)

# Save
joblib.dump(model, 'model.pkl')

# Load
loaded_model = joblib.load('model.pkl')
preds = loaded_model.predict(X[:5])
print(preds)

Explanation: joblib is efficient for large NumPy arrays. For XGBoost, you can also use model.save_model('model.json') and xgb.Booster(model_file='model.json') for cross-language compatibility.

Prompt 13: Model Evaluation with Multiple Metrics

Task: Compute precision, recall, F1-score, and ROC-AUC for a binary classifier.

Prompt:

"After training a CatBoost classifier, evaluate it on test data using precision_score, recall_score, f1_score, and roc_auc_score. Print all metrics."

Example Result:

from catboost import CatBoostClassifier
from sklearn.metrics import precision_score, recall_score, f1_score, roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = CatBoostClassifier(iterations=50, verbose=0, random_seed=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
y_proba = model.predict_proba(X_test)[:, 1]

print(f"Precision: {precision_score(y_test, y_pred):.3f}")
print(f"Recall: {recall_score(y_test, y_pred):.3f}")
print(f"F1: {f1_score(y_test, y_pred):.3f}")
print(f"ROC-AUC: {roc_auc_score(y_test, y_proba):.3f}")

Explanation: Relying solely on accuracy can be misleading for imbalanced data. A combination of precision, recall, F1, and AUC provides a fuller picture of model performance.

Prompt 14: Gradient Boosting with Custom Objective

Task: Implement a custom objective function for XGBoost (e.g., weighted squared error).

Prompt:

"Write a custom squared error objective with sample weights for XGBoost. The objective should return gradient and hessian. Train the model using xgb.train."

Example Result:

import xgboost as xgb
import numpy as np

def weighted_squared_error(predt, dtrain):
    y = dtrain.get_label()
    w = dtrain.get_weight() if dtrain.get_weight() is not None else np.ones_like(y)
    grad = 2 * (predt - y) * w
    hess = 2 * w
    return grad, hess

# Dummy data
X = np.random.rand(100, 5)
y = np.random.rand(100)
weights = np.random.rand(100)
dtrain = xgb.DMatrix(X, label=y, weight=weights)

params = {'objective': weighted_squared_error, 'eval_metric': 'rmse'}
model = xgb.train(params, dtrain, num_boost_round=10)

Explanation: Custom objectives give you full control over the loss function. The gradient (first derivative) and hessian (second derivative) are required for XGBoost's Newton boosting.

Prompt 15: CatBoost Multiclass with Custom Loss

Task: Train a CatBoost multiclass classifier and evaluate per-class metrics.

Prompt:

"Using the iris dataset, train a CatBoost classifier for multiclass classification. After training, compute the confusion matrix and per-class F1-score."

Example Result:

from catboost import CatBoostClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, f1_score
import seaborn as sns
import matplotlib.pyplot as plt

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = CatBoostClassifier(iterations=50, verbose=0, random_seed=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

cm = confusion_matrix(y_test, y_pred)
print("Confusion matrix:")
print(cm)

f1 = f1_score(y_test, y_pred, average=None)
print(f"Per-class F1: {f1}")

# Plot
sns.heatmap(cm, annot=True, fmt='d')
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.show()

Explanation: Multiclass evaluation requires per-class metrics. The confusion matrix reveals which classes are confused; low F1 for a class suggests the model struggles to distinguish it.

Conclusion

These 15 prompts cover the essential tasks in any ML project: data cleaning, feature engineering, model training, tuning, and evaluation. By mastering them with Scikit-learn, XGBoost, and CatBoost, you can build robust pipelines that handle real-world data challenges. Start by adapting the code snippets to your own datasets—each prompt is a reusable template. For further learning, consult the official documentation of each library. What prompt would you add to this list? Share your thoughts in the comments.