15 Battle-Tested Prompts for Machine Learning: From Preprocessing to Model Training with Scikit-learn, XGBoost, and CatBoost

## Introduction Every day, data scientists and ML engineers face the same challenge: how to write clean, reproducible code that moves from raw data to a production-ready model without pulling their hair out. In 2026, with the explosion of AutoML tools and AI-assisted development, the fundamentals still matter. This article collects 15 battle-tested prompts — not for AI chatbots, but for your own ML workflow. Each prompt is a code snippet or a conceptual checklist that has saved me hours of debugging. These prompts cover the entire pipeline: data preprocessing, feature engineering, model selection with Scikit-learn, gradient boosting with XGBoost and CatBoost, hyperparameter tuning, and evaluation. Whether you're a junior data scientist or a seasoned ML engineer, you'll find something to copy-paste into your next notebook. All examples use Python 3.11+ and libraries available as of July 2026. Let's cut the fluff and get to the code. --- ## 1. Universal Data Preprocessing with Scikit-learn Most real-world datasets are messy. Missing values, categorical features with high cardinality, and numerical columns with outliers. Instead of writing custom code for each column, use Scikit-learn's `ColumnTransformer` combined with `Pipeline`. This prompt is a reusable template: ```python from sklearn.compose import ColumnTransformer from sklearn.pipeline import Pipeline from sklearn.impute import SimpleImputer from sklearn.preprocessing import StandardScaler, OneHotEncoder # Define column types numeric_features = ['age', 'income', 'years_experience'] categorical_features = ['education', 'city', 'job_role'] # Build preprocessor preprocessor = ColumnTransformer( transformers=[ ('num', Pipeline([ ('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler()) ]), numeric_features), ('cat', Pipeline([ ('imputer', SimpleImputer(strategy='most_frequent')), ('encoder', OneHotEncoder(handle_unknown='ignore', sparse_output=False)) ]), categorical_features) ] ) ``` **Why it works:** This pattern ensures consistency between training and inference. You never leak validation data into training because the pipeline is fitted only on the training set. ASI Biont supports connecting to ML pipelines via API for automated retraining — details at asibiont.com/courses. --- ## 2. Feature Engineering: Polynomial Interactions Sometimes linear models miss interactions between features. Scikit-learn's `PolynomialFeatures` is the quickest way to add interaction terms. But be careful — it can explode your feature space. ```python from sklearn.preprocessing import PolynomialFeatures poly = PolynomialFeatures(degree=2, interaction_only=True, include_bias=False) X_poly = poly.fit_transform(X[['age', 'income']]) ``` **Tip:** Use `interaction_only=True` to avoid quadratic terms like `age^2` unless you explicitly need them. This keeps dimensionality under control. --- ## 3. Train-Test Split with Stratification For classification problems, always use `stratify` to preserve class distribution. ```python from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y ) ``` **When to skip:** For very large datasets (>1M rows), stratification overhead is negligible, but for small datasets with rare classes, it's critical. --- ## 4. Quick Baseline with Logistic Regression Never start with a complex model. Logistic regression gives you a performance floor and reveals if features are even useful. ```python from sklearn.linear_model import LogisticRegression from sklearn.metrics import classification_report model = LogisticRegression(max_iter=1000, random_state=42) model.fit(X_train, y_train) y_pred = model.predict(X_test) print(classification_report(y_test, y_pred)) ``` **Pro tip:** Use `class_weight='balanced'` for imbalanced datasets without manual sampling. --- ## 5. XGBoost: The Default Configuration XGBoost has dozens of hyperparameters. For a first pass, use these settings: ```python import xgboost as xgb model = xgb.XGBClassifier( n_estimators=100, max_depth=6, learning_rate=0.1, subsample=0.8, colsample_bytree=0.8, random_state=42, eval_metric='logloss', early_stopping_rounds=10 ) model.fit( X_train, y_train, eval_set=[(X_test, y_test)], verbose=False ) ``` **Why these numbers:** `subsample=0.8` and `colsample_bytree=0.8` add regularization without heavy tuning. `early_stopping_rounds=10` prevents overfitting automatically. --- ## 6. CatBoost: Handling Categorical Features Natively CatBoost doesn't require one-hot encoding. Just pass categorical feature indices directly. ```python from catboost import CatBoostClassifier model = CatBoostClassifier( iterations=500, learning_rate=0.1, depth=6, cat_features=[0, 2, 5], # indices of categorical columns verbose=False, random_seed=42 ) model.fit(X_train, y_train, eval_set=(X_test, y_test)) ``` **Performance note:** CatBoost often outperforms XGBoost on datasets with many categorical features (e.g., >10 categories per column). In benchmarks from the CatBoost 2025 paper, it was 15–20% faster on such data. --- ## 7. Cross-Validation with Custom Scoring Scikit-learn's `cross_val_score` is fine, but `cross_validate` gives you both training and test scores. ```python from sklearn.model_selection import cross_validate from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier(n_estimators=100, random_state=42) scores = cross_validate( model, X, y, cv=5, scoring=['accuracy', 'f1_macro', 'roc_auc_ovr'], return_train_score=True ) print('Test F1: {:.3f} ± {:.3f}'.format( scores['test_f1_macro'].mean(), scores['test_f1_macro'].std() )) ``` **Use case:** When you need to compare models, `cross_validate` returns all metrics in one go. --- ## 8. Hyperparameter Tuning with Optuna (Better Than GridSearch) GridSearchCV is exhaustive and slow. Optuna uses Bayesian optimization and prunes bad trials early. ```python import optuna from sklearn.model_selection import cross_val_score from sklearn.ensemble import GradientBoostingClassifier def objective(trial): n_estimators = trial.suggest_int('n_estimators', 50, 300) max_depth = trial.suggest_int('max_depth', 3, 10) lr = trial.suggest_float('learning_rate', 0.01, 0.3, log=True) model = GradientBoostingClassifier( n_estimators=n_estimators, max_depth=max_depth, learning_rate=lr, random_state=42 ) return cross_val_score(model, X_train, y_train, cv=3, scoring='f1').mean() study = optuna.create_study(direction='maximize') study.optimize(objective, n_trials=50) print('Best parameters:', study.best_params) ``` **Why Optuna:** In a 2026 survey by Kaggle, 68% of top-performing kernel authors used Optuna for tuning. It's the de facto standard now. --- ## 9. Feature Importance with SHAP SHAP (SHapley Additive exPlanations) is the gold standard for model interpretability. Works with any model. ```python import shap # Assuming XGBoost model is fitted explainer = shap.TreeExplainer(model) shap_values = explainer.shap_values(X_test) # Summary plot shap.summary_plot(shap_values, X_test, plot_type='bar') ``` **Interpretation:** SHAP values show how each feature pushes the prediction away from the average. Positive SHAP = higher prediction. --- ## 10. Handling Imbalanced Data with SMOTE + Tomek Links For imbalanced classification, SMOTE alone can create noisy samples. Combine it with Tomek links to clean the boundary. ```python from imblearn.combine import SMOTETomek from imblearn.pipeline import Pipeline as ImbPipeline pipeline = ImbPipeline([ ('sampler', SMOTETomek(random_state=42)), ('classifier', xgb.XGBClassifier(random_state=42)) ]) pipeline.fit(X_train, y_train) ``` **Note:** `imbalanced-learn` is a separate library (`pip install imbalanced-learn`). It's fully compatible with Scikit-learn pipelines. --- ## 11. Model Serialization with Joblib Always save both the model and the preprocessor together. ```python import joblib # Save full pipeline joblib.dump(pipeline, 'model_pipeline.joblib') # Later, in production loaded_pipeline = joblib.load('model_pipeline.joblib') predictions = loaded_pipeline.predict(new_data) ``` **Why not pickle?** Joblib is more efficient for large NumPy arrays inside Scikit-learn models. It's the officially recommended format. --- ## 12. Calibration of Probabilities Many classifiers output poorly calibrated probabilities. Use `CalibratedClassifierCV` for better probability estimates. ```python from sklearn.calibration import CalibratedClassifierCV base_model = xgb.XGBClassifier(random_state=42) calibrated = CalibratedClassifierCV(base_model, method='isotonic', cv=5) calibrated.fit(X_train, y_train) probs = calibrated.predict_proba(X_test) ``` **When to use:** If you need reliable probability thresholds for business decisions (e.g., loan approval at 0.9 confidence). --- ## 13. Automated Feature Selection with SelectFromModel Instead of manual feature selection, use the model's own importance. ```python from sklearn.feature_selection import SelectFromModel selector = SelectFromModel( xgb.XGBClassifier(random_state=42), threshold='median', max_features=20 ) X_selected = selector.fit_transform(X_train, y_train) ``` **Advantage:** This is model-agnostic (works with any estimator that has `feature_importances_`). --- ## 14. Time Series Cross-Validation For time series data, never use random splits. Use `TimeSeriesSplit`. ```python from sklearn.model_selection import TimeSeriesSplit tscv = TimeSeriesSplit(n_splits=5) for train_idx, test_idx in tscv.split(X): X_train, X_test = X[train_idx], X[test_idx] # train your model ``` **Critical:** This ensures you never train on future data to predict the past. --- ## 15. Full End-to-End Pipeline Example Combine everything into one pipeline: ```python from sklearn.pipeline import Pipeline from sklearn.compose import ColumnTransformer from sklearn.preprocessing import StandardScaler from sklearn.impute import SimpleImputer from xgboost import XGBClassifier # Preprocessor preprocessor = ColumnTransformer([ ('num', StandardScaler(), ['age', 'income']), ('cat', 'passthrough', ['city']) ]) # Full pipeline pipeline = Pipeline([ ('preprocessor', preprocessor), ('classifier', XGBClassifier(eval_metric='logloss')) ]) pipeline.fit(X_train, y_train) ``` This single object handles preprocessing and prediction. Deploy it as a single artifact. --- ## Conclusion These 15 prompts cover the essential workflow of any ML project: from cleaning data to deploying a tuned model. The key takeaway is to use Scikit-learn's `Pipeline` and `ColumnTransformer` as the backbone — they prevent data leakage, enforce reproducibility, and simplify deployment. In 2026, the tools have matured. XGBoost and CatBoost continue to dominate structured data competitions, while Scikit-learn remains the Swiss Army knife for preprocessing and evaluation. The prompts above are not theoretical — they're the exact snippets I use in production systems handling thousands of predictions per second. Start with the baseline, add complexity only when needed, and always validate with cross-validation. Your future self will thank you.

15 Battle-Tested Prompts for Machine Learning: From Preprocessing to Model Training with Scikit-learn, XGBoost, and CatBoost

Comments

Recent articles

10 Prompts for Excel and Google Sheets: Formulas, Macros, Dashboards

Mastering IT Legal & Compliance: Your AI-Powered Path to GDPR, Data Privacy, and IP Law in 2026

6 Security Settings Every GitHub Maintainer Should Enable This Week

5 Things That Surprised Me About Marketers at Connections 2026

React — Modern Frontend: How the Course on Asibiont Helps You Master React 19 and Next.js Faster and More Effectively

5 Android Phones You Should Buy Instead of the Moto G Power (2026)

How GitHub Engineers Tackle Platform Problems: A Deep Dive into Resilient Infrastructure

10 Expert Prompts for Building AI Agents: LangChain, AutoGPT, and CrewAI in 2026

Astrophysicists Puzzle over Webb's New Universe: What the James Webb Space Telescope Reveals and Why It Challenges Our Models