12 Prompts for Machine Learning: Scikit-learn, XGBoost, and CatBoost

Why Prompts Matter for ML

Working with machine learning libraries like Scikit-learn, XGBoost, and CatBoost can be tricky—each has its own API, parameter names, and best practices. A well-crafted prompt for an AI assistant can save hours of documentation digging and debugging. This collection gives you 12 ready-to-use prompts that take you from data preprocessing to model evaluation and interpretation. You can copy them, adapt them to your dataset, and learn from the generated code.

Quick Overview of Libraries

Sources: Scikit-learn User Guide, XGBoost Documentation, CatBoost Documentation.

Prompts for Preprocessing and EDA

1. End-to-End Preprocessing Pipeline

Prompt:

Act as an ML engineer. Write Python code using Scikit-learn's ColumnTransformer and Pipeline to preprocess a CSV with numeric and categorical columns. Include mean imputation, StandardScaler, and OneHotEncoder, and output a transformed dataset ready for modeling.

Usage example: Paste the prompt into your AI assistant along with a short description of your columns. You'll get code that you can run directly in a notebook. This is ideal for standardizing all your preprocessing steps in one repeatable object.

2. Automated EDA

Prompt:

Generate Python code using ydata-profiling to create a full EDA report for a dataframe. Based on the report, suggest three candidate models and explain why.

Usage example: Run the produced script to get missing values, correlations, and distributions. The model suggestions help you decide e.g., whether to start with tree-based models or linear ones.

3. Feature Selection with Mutual Information

Prompt:

Write a function that combines mutual information and recursive feature elimination (RFE) to select important features. Show how to use it with an XGBoost classifier and return the selected feature names.

Usage example: Use this function to cut dozens of irrelevant columns before training. It works especially well when you have non-linear relationships.

Prompts for Feature Engineering and Model Building

4. Ensemble Comparison

Prompt:

Explain bagging, boosting, and stacking. Then write code to build a VotingClassifier with RandomForest, XGBoost, and CatBoost using Scikit-learn.

Usage example: The assistant will return both a short theory explanation and a working ensemble. You can see how different models complement each other and improve overall accuracy.

5. Handling Imbalanced Data

Prompt:

Given a classification problem with severe class imbalance, write code using imbalanced-learn to apply SMOTE and random undersampling. Compare Logistic Regression, XGBoost, and CatBoost with and without resampling.

Usage example: This prompt is perfect for fraud detection or churn prediction tasks. It produces a comparison table of metrics like F1-score and ROC-AUC so you can see the effect of each sampling method.

6. XGBoost with Early Stopping

Prompt:

Write a Python script to train XGBoost with early stopping, GridSearchCV, log-loss metric, and feature importance visualization.

Usage example: Copy the script and adjust the parameter grid for your dataset. The early stopping callback prevents overfitting and saves training time.

7. CatBoost Without Manual Encoding

Prompt:

Show how to train CatBoost on data with categorical features without manual one-hot encoding. Explain the cat_features parameter and the overfitting_detector feature.

Usage example: This prompt is valuable if you are new to CatBoost. You learn how to pass column indices directly to the model and how to use built-in overfitting protection.

Prompts for Model Evaluation and Interpretation

8. Comprehensive Evaluation

Prompt:

Create a Python function to evaluate a binary classifier using confusion matrix, precision/recall, ROC-AUC, and a calibration plot. Then generate a SHAP summary plot from the model.

Usage example: The function works for both XGBoost and CatBoost and returns all core metrics at once. SHAP plots show which features drive predictions—great for stakeholder presentations.

9. Nested Cross-Validation

Prompt:

Write code for nested cross-validation to compare XGBoost and CatBoost. Report mean and standard deviation of accuracy and F1-score. Use a fixed random seed for reproducibility.

Usage example: Nested CV gives you an unbiased estimate of model performance. The generated code is ready to run and returns a clean comparison table.

10. Model Persistence

Prompt:

Write code to save and load trained Scikit-learn, XGBoost, and CatBoost models using joblib. Compare file sizes and loading time, and discuss when to use pickle vs joblib.

Usage example: This prompt helps you package models for deployment. The benchmark code will show you which serialization method is fastest for your case.

11. Debugging Overfitting

Prompt:

Give me a checklist to debug a model that works well on training data but fails on validation. Provide code snippets for Scikit-learn, XGBoost, and CatBoost to plot learning curves.

Usage example: Follow the checklist to diagnose high variance. The code for learning curves shows whether more data or lower model complexity will help.

12. End-to-End Pipeline for Regression

Prompt:

Build a complete ML pipeline for regression using Scikit-learn, XGBoost, and CatBoost, including custom transformers and a voting regressor. Use a real dataset like California housing and show final RMSE and R².

Usage example: This is a final project-style prompt: it combines preprocessing, multiple models, and final evaluation. You get a reusable template for any tabular regression task.

Conclusion

These 12 prompts cover the most common pain points in machine learning with Scikit-learn, XGBoost, and CatBoost. Instead of memorizing every API, you can rely on a good prompt to guide you through data cleaning, model building, tuning, and interpretation. Bookmark this list, open your favorite AI assistant, and start with prompt #1 today.