10 Essential Prompts for Machine Learning: Scikit-learn, XGBoost, and CatBoost

Introduction

Machine learning has become a cornerstone of modern data science, but building effective models requires more than just importing libraries. Whether you are a beginner exploring classification or a seasoned practitioner tuning hyperparameters, having a set of reusable prompts—structured instructions for your ML pipeline—can dramatically speed up experimentation and reduce errors.

This article provides ten ready-to-use prompts covering the full lifecycle of a supervised ML project: from data preprocessing and feature engineering to training with Scikit-learn, XGBoost, and CatBoost, and finally evaluating and interpreting models. Each prompt is accompanied by a practical example and explanation, so you can copy, adapt, and apply them directly to your own datasets. By the end, you will have a cheat-sheet that you can reference for common tasks, saving hours of search and trial-and-error.

Why Prompts for Machine Learning?

A prompt in this context is a structured template or code snippet that encapsulates best practices for a specific ML task. Instead of rewriting boilerplate code every time, you can use prompts as starting points. They help ensure consistency, avoid common pitfalls (like data leakage), and enforce good habits such as scaling features or using cross-validation. This approach is especially valuable when you work with multiple libraries that have different APIs—Scikit-learn for traditional algorithms, XGBoost and CatBoost for gradient boosting—each with its own quirks.

1. Data Preprocessing with Pipelines

Prompt:

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer

numeric_features = ['age', 'income', 'score']
categorical_features = ['gender', 'city']

numeric_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())])

categorical_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('encoder', OneHotEncoder(handle_unknown='ignore'))])

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numeric_transformer, numeric_features),
        ('cat', categorical_transformer, categorical_features)])

Explanation: This prompt builds a reusable preprocessing pipeline that handles missing values, scales numeric features, and encodes categorical variables. It prevents data leakage by fitting transformations only on training data. Use it as the first step in any ML project.

Example: For a customer churn dataset with columns like age, income, gender, and city, this pipeline will automatically clean and prepare the data for modeling.

2. Train-Test Split with Stratification

Prompt:

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

Explanation: Stratified splitting ensures that the class distribution in the training and test sets mirrors the original dataset. This is critical for imbalanced classification problems. The random_state guarantees reproducibility.

Example: In a fraud detection dataset where only 2% of transactions are fraudulent, stratification preserves that ratio in both splits.

3. Training a Scikit-learn Model with Cross-Validation

Prompt:

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

model = RandomForestClassifier(n_estimators=100, random_state=42)
scores = cross_val_score(model, X_train, y_train, cv=5, scoring='f1_macro')
print(f'F1 scores: {scores.mean():.3f} +/- {scores.std():.3f}')

Explanation: Cross-validation gives a more reliable estimate of model performance than a single train-test split. The scoring parameter can be changed to accuracy, roc_auc, or any other metric. This prompt uses F1 macro, suitable for multi-class problems.

Example: For a product categorization task with 10 categories, this prompt returns the average F1 score across 5 folds, indicating how well the model generalizes.

4. Hyperparameter Tuning with GridSearchCV

Prompt:

from sklearn.model_selection import GridSearchCV

param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [None, 10, 20],
    'min_samples_split': [2, 5]}

grid_search = GridSearchCV(
    RandomForestClassifier(random_state=42),
    param_grid, cv=5, scoring='f1_macro', n_jobs=-1)
grid_search.fit(X_train, y_train)
print(f'Best params: {grid_search.best_params_}')

Explanation: GridSearch exhaustively searches over a predefined hyperparameter grid. Use n_jobs=-1 to parallelize across all CPU cores. For larger grids, consider RandomizedSearchCV.

Example: Tuning a Random Forest for a loan default prediction task might find that n_estimators=200 and max_depth=10 yield the best F1 score.

5. Training an XGBoost Model with Early Stopping

Prompt:

import xgboost as xgb

model = xgb.XGBClassifier(
    n_estimators=1000,
    learning_rate=0.1,
    max_depth=6,
    early_stopping_rounds=50,
    eval_metric='logloss')

model.fit(
    X_train, y_train,
    eval_set=[(X_test, y_test)],
    verbose=False)
print(f'Best iteration: {model.best_iteration}')

Explanation: Early stopping prevents overfitting by stopping training when validation performance stops improving. The eval_set is used for monitoring. XGBoost also supports GPU acceleration with tree_method='gpu_hist' if you have a compatible GPU.

Example: In a click-through rate prediction task, this prompt trains an XGBoost model that automatically stops after 50 rounds without improvement, saving time and reducing overfitting.

6. Feature Importance with XGBoost

Prompt:

import matplotlib.pyplot as plt

importance = model.feature_importances_
for name, imp in sorted(zip(feature_names, importance), key=lambda x: x[1], reverse=True)[:10]:
    print(f'{name}: {imp:.4f}')

xgb.plot_importance(model, max_num_features=10)
plt.show()

Explanation: XGBoost provides built-in feature importance based on how often a feature is used for splitting. This prompt prints the top 10 features and plots them for quick visualization.

Example: For a house price prediction model, you might discover that square_footage and location_score are the most influential features.

7. Training a CatBoost Model with Categorical Features

Prompt:

from catboost import CatBoostClassifier

cat_features = ['gender', 'city', 'education_level']

model = CatBoostClassifier(
    iterations=500,
    learning_rate=0.1,
    depth=6,
    cat_features=cat_features,
    verbose=100)

model.fit(X_train, y_train, eval_set=(X_test, y_test))

Explanation: CatBoost handles categorical features natively without one-hot encoding. Just pass the column indices or names as cat_features. It also supports ordered boosting to reduce overfitting.

Example: For a customer segmentation dataset with many categorical columns, this prompt trains a model that automatically handles them, often outperforming one-hot encoding.

8. Model Evaluation with Multiple Metrics

Prompt:

from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score

y_pred = model.predict(X_test)
y_proba = model.predict_proba(X_test)[:, 1]

print(classification_report(y_test, y_pred))
print(f'ROC AUC: {roc_auc_score(y_test, y_proba):.3f}')
print(confusion_matrix(y_test, y_pred))

Explanation: This prompt provides a comprehensive evaluation: precision, recall, F1-score for each class, the confusion matrix, and the ROC AUC score for binary classification. Adjust y_proba for multi-class by selecting the appropriate column.

Example: In a medical diagnosis model, the classification report quickly reveals if the model has low recall for the positive class (disease present).

9. Saving and Loading Models

Prompt:

import joblib

# Save
joblib.dump(model, 'model.pkl')
joblib.dump(preprocessor, 'preprocessor.pkl')

# Load
loaded_model = joblib.load('model.pkl')
loaded_preprocessor = joblib.load('preprocessor.pkl')

Explanation: joblib is the recommended way to save Scikit-learn, XGBoost, and CatBoost models. It is efficient for large numpy arrays. Always save the preprocessor together with the model so that new data is transformed consistently.

Example: After training a churn prediction model, you save both the model and the pipeline, then load them in a production API to serve predictions.

10. Interpreting Models with SHAP

Prompt:

import shap

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

shap.summary_plot(shap_values, X_test, feature_names=feature_names)

Explanation: SHAP (SHapley Additive exPlanations) provides unified feature importance and explains individual predictions. This prompt generates a summary plot showing how each feature affects the model output across all samples.

Example: In a credit scoring model, a SHAP plot might reveal that having a low income increases the probability of default, while a high credit score decreases it.

Putting It All Together: A Complete Workflow

Here is a minimal end-to-end workflow combining the above prompts:

1. Load and split data (Prompt 2).
2. Build preprocessor (Prompt 1) and fit on training data.
3. Transform both train and test sets.
4. Train an XGBoost model with early stopping (Prompt 5).
5. Evaluate using multiple metrics (Prompt 8).
6. Save model and preprocessor (Prompt 9).
7. Interpret with SHAP (Prompt 10).

This workflow works for most tabular regression or classification tasks. For large datasets, consider using RandomizedSearchCV instead of GridSearch, and for deep learning tasks, you would replace the boosting libraries with frameworks like PyTorch or TensorFlow.

Conclusion

These ten prompts cover the essential steps of a machine learning project using Scikit-learn, XGBoost, and CatBoost. By turning common tasks into reusable templates, you reduce cognitive load and ensure consistent quality across experiments. Start with the preprocessing pipeline, then move to model training, evaluation, and interpretation. Adapt the parameters to your specific data and problem—there is no one-size-fits-all configuration.

Remember that the best model is not just the one with the highest test score, but the one that is robust, interpretable, and deployable. Use these prompts as a foundation, and refine them with domain knowledge and iterative experimentation. Happy modeling!