Introduction
Machine learning (ML) is no longer just a buzzword — it's a core skill for data scientists and developers. Whether you're preprocessing data with Scikit-learn, boosting tree models with XGBoost, or handling categorical features with CatBoost, the right prompt can save hours of trial and error. This article collects 15 battle-tested prompts I use daily in my ML workflow. Each prompt includes a real usage example and code snippet, so you can copy-paste and adapt. Let's dive in.
1. Data Preprocessing with Scikit-learn
Prompt: "Use sklearn.preprocessing to handle missing values and scale features for a dataset with 10 columns and 1000 rows. Apply SimpleImputer with median strategy, then StandardScaler."
Why it works: Scikit-learn's preprocessing module is the standard for cleaning data before modeling. The prompt forces you to specify strategy (median is robust to outliers) and scaling method.
Example:
import pandas as pd
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
df = pd.DataFrame({'age': [25, 30, None, 40], 'income': [50000, 60000, 55000, None]})
imputer = SimpleImputer(strategy='median')
scaler = StandardScaler()
df_imputed = pd.DataFrame(imputer.fit_transform(df), columns=df.columns)
df_scaled = pd.DataFrame(scaler.fit_transform(df_imputed), columns=df.columns)
print(df_scaled)
2. Train-Test Split with Stratification
Prompt: "Split dataset df into train and test sets with 80-20 ratio, stratified by target column 'label'. Use sklearn.model_selection.train_test_split."
Why it works: Stratification preserves class distribution, critical for imbalanced datasets.
Example:
from sklearn.model_selection import train_test_split
X = df.drop('label', axis=1)
y = df['label']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
print(f'Train size: {len(X_train)}, Test size: {len(X_test)}')
3. Random Forest Classifier with Hyperparameter Tuning
Prompt: "Train a Random Forest classifier on X_train, y_train using sklearn.ensemble.RandomForestClassifier. Tune n_estimators (50-200) and max_depth (5-15) with GridSearchCV and 5-fold CV."
Why it works: GridSearchCV automates hyperparameter search, preventing overfitting.
Example:
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
rf = RandomForestClassifier(random_state=42)
param_grid = {'n_estimators': [50, 100, 200], 'max_depth': [5, 10, 15]}
grid = GridSearchCV(rf, param_grid, cv=5, scoring='accuracy')
grid.fit(X_train, y_train)
print(f'Best params: {grid.best_params_}, Best score: {grid.best_score_:.3f}')
4. XGBoost Model with Early Stopping
Prompt: "Implement XGBoost classifier with xgboost.XGBClassifier. Use early stopping with 10 rounds of patience and validation set."
Why it works: Early stopping prevents overfitting by halting training when validation error stops improving.
Example:
import xgboost as xgb
model = xgb.XGBClassifier(n_estimators=1000, random_state=42, early_stopping_rounds=10, eval_metric='logloss')
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
print(f'Best iteration: {model.best_iteration}')
5. Feature Importance from XGBoost
Prompt: "Extract feature importance from trained XGBoost model and plot top 10 features using plot_importance."
Why it works: Feature importance helps interpret which variables drive predictions.
Example:
import matplotlib.pyplot as plt
from xgboost import plot_importance
plot_importance(model, max_num_features=10)
plt.show()
6. CatBoost with Categorical Features
Prompt: "Train CatBoost classifier on data with mixed numeric and categorical columns. Use cat_features parameter to specify categorical indices."
Why it works: CatBoost natively handles categoricals without one-hot encoding.
Example:
from catboost import CatBoostClassifier
cat_cols = [0, 2] # indices of categorical columns
model = CatBoostClassifier(iterations=100, random_seed=42, verbose=False)
model.fit(X_train, y_train, cat_features=cat_cols)
print(f'Accuracy: {model.score(X_test, y_test):.3f}')
7. CatBoost with Cross-Validation
Prompt: "Use CatBoost's cv function for 5-fold cross-validation and return mean logloss."
Why it works: Built-in CV saves code and ensures consistency.
Example:
from catboost import cv, Pool
pool = Pool(X_train, y_train, cat_features=cat_cols)
cv_results = cv(pool, params={'iterations': 100, 'loss_function': 'Logloss', 'verbose': False}, fold_count=5)
print(f'Mean logloss: {cv_results["test-Logloss-mean"].values[-1]:.4f}')
8. Pipeline with Preprocessing and Model
Prompt: "Create a Scikit-learn Pipeline that imputes missing values, scales features, and trains a Random Forest."
Why it works: Pipelines streamline workflow and prevent data leakage.
Example:
from sklearn.pipeline import Pipeline
pipeline = Pipeline([
('imputer', SimpleImputer(strategy='median')),
('scaler', StandardScaler()),
('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
])
pipeline.fit(X_train, y_train)
print(f'Pipeline accuracy: {pipeline.score(X_test, y_test):.3f}')
9. Model Comparison with Multiple Metrics
Prompt: "Compare XGBoost, CatBoost, and Random Forest using accuracy, precision, recall, and F1-score on the same test set."
Why it works: Multiple metrics reveal trade-offs (e.g., precision vs recall).
Example:
from sklearn.metrics import classification_report
models = {
'XGBoost': xgb.XGBClassifier(random_state=42).fit(X_train, y_train),
'CatBoost': CatBoostClassifier(verbose=False, random_seed=42).fit(X_train, y_train, cat_features=cat_cols),
'RF': RandomForestClassifier(random_state=42).fit(X_train, y_train)
}
for name, model in models.items():
print(f'\n{name}:')
print(classification_report(y_test, model.predict(X_test)))
10. Hyperparameter Tuning with XGBoost and Optuna
Prompt: "Use Optuna to tune XGBoost hyperparameters: learning_rate, max_depth, subsample, colsample_bytree."
Why it works: Optuna finds better params faster than grid search.
Example:
import optuna
def objective(trial):
params = {
'learning_rate': trial.suggest_float('learning_rate', 0.01, 0.3),
'max_depth': trial.suggest_int('max_depth', 3, 10),
'subsample': trial.suggest_float('subsample', 0.5, 1.0),
'colsample_bytree': trial.suggest_float('colsample_bytree', 0.5, 1.0),
}
model = xgb.XGBClassifier(**params, n_estimators=100, random_state=42)
return -model.fit(X_train, y_train).score(X_test, y_test)
study = optuna.create_study()
study.optimize(objective, n_trials=20)
print(f'Best params: {study.best_params}')
11. Handling Imbalanced Data with SMOTE and XGBoost
Prompt: "Apply SMOTE from imbalanced-learn to oversample minority class before training XGBoost."
Why it works: SMOTE generates synthetic samples, improving recall for rare classes.
Example:
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import make_pipeline
pipeline = make_pipeline(SMOTE(random_state=42), xgb.XGBClassifier(random_state=42))
pipeline.fit(X_train, y_train)
print(f'Accuracy with SMOTE: {pipeline.score(X_test, y_test):.3f}')
12. Feature Selection with Recursive Feature Elimination (RFE)
Prompt: "Use sklearn.feature_selection.RFE with a Random Forest estimator to select top 5 features."
Why it works: RFE iteratively removes least important features, reducing overfitting.
Example:
from sklearn.feature_selection import RFE
estimator = RandomForestClassifier(random_state=42)
selector = RFE(estimator, n_features_to_select=5, step=1)
selector.fit(X_train, y_train)
print(f'Selected features: {X_train.columns[selector.support_]}')
13. Calibration of Probabilities with Scikit-learn
Prompt: "Calibrate XGBoost probability outputs using CalibratedClassifierCV with isotonic regression."
Why it works: Calibration improves probability estimates, crucial for risk scoring.
Example:
from sklearn.calibration import CalibratedClassifierCV
base_model = xgb.XGBClassifier(random_state=42)
calibrated = CalibratedClassifierCV(base_model, method='isotonic', cv=5)
calibrated.fit(X_train, y_train)
print(f'Calibrated accuracy: {calibrated.score(X_test, y_test):.3f}')
14. Model Interpretation with SHAP
Prompt: "Use SHAP to explain predictions of an XGBoost model and create a summary plot."
Why it works: SHAP provides consistent, game-theoretic feature attributions.
Example:
import shap
model = xgb.XGBClassifier(random_state=42).fit(X_train, y_train)
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test)
15. Saving and Loading Models with Joblib
Prompt: "Save a trained CatBoost model using joblib.dump and load it later for inference."
Why it works: Joblib is optimized for large numpy arrays, faster than pickle.
Example:
import joblib
model = CatBoostClassifier(verbose=False, random_seed=42).fit(X_train, y_train)
joblib.dump(model, 'catboost_model.joblib')
loaded_model = joblib.load('catboost_model.joblib')
print(f'Loaded model accuracy: {loaded_model.score(X_test, y_test):.3f}')
Conclusion
These 15 prompts cover the full ML pipeline — from preprocessing to deployment. They work with Scikit-learn, XGBoost, and CatBoost, and are designed to be copy-pasted and adapted. Try them on your own datasets, and you'll see productivity gains immediately. For more advanced workflows, check the official docs:
- Scikit-learn: https://scikit-learn.org/stable/
- XGBoost: https://xgboost.readthedocs.io/en/stable/
- CatBoost: https://catboost.ai/en/docs/
Start using these prompts today, and share your results with the community!
Comments