Introduction
Every day, data scientists and ML engineers face the same challenge: how to write clean, reproducible code that moves from raw data to a production-ready model without pulling their hair out. In 2026, with the explosion of AutoML tools and AI-assisted development, the fundamentals still matter. This article collects 15 battle-tested prompts — not for AI chatbots, but for your own ML workflow. Each prompt is a code snippet or a conceptual checklist that has saved me hours of debugging.
These prompts cover the entire pipeline: data preprocessing, feature engineering, model selection with Scikit-learn, gradient boosting with XGBoost and CatBoost, hyperparameter tuning, and evaluation. Whether you're a junior data scientist or a seasoned ML engineer, you'll find something to copy-paste into your next notebook.
All examples use Python 3.11+ and libraries available as of July 2026. Let's cut the fluff and get to the code.
1. Universal Data Preprocessing with Scikit-learn
Most real-world datasets are messy. Missing values, categorical features with high cardinality, and numerical columns with outliers. Instead of writing custom code for each column, use Scikit-learn's ColumnTransformer combined with Pipeline. This prompt is a reusable template:
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
# Define column types
numeric_features = ['age', 'income', 'years_experience']
categorical_features = ['education', 'city', 'job_role']
# Build preprocessor
preprocessor = ColumnTransformer(
transformers=[
('num', Pipeline([
('imputer', SimpleImputer(strategy='median')),
('scaler', StandardScaler())
]), numeric_features),
('cat', Pipeline([
('imputer', SimpleImputer(strategy='most_frequent')),
('encoder', OneHotEncoder(handle_unknown='ignore', sparse_output=False))
]), categorical_features)
]
)
Why it works: This pattern ensures consistency between training and inference. You never leak validation data into training because the pipeline is fitted only on the training set. ASI Biont supports connecting to ML pipelines via API for automated retraining — details at asibiont.com/courses.
2. Feature Engineering: Polynomial Interactions
Sometimes linear models miss interactions between features. Scikit-learn's PolynomialFeatures is the quickest way to add interaction terms. But be careful — it can explode your feature space.
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=True, include_bias=False)
X_poly = poly.fit_transform(X[['age', 'income']])
Tip: Use interaction_only=True to avoid quadratic terms like age^2 unless you explicitly need them. This keeps dimensionality under control.
3. Train-Test Split with Stratification
For classification problems, always use stratify to preserve class distribution.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
When to skip: For very large datasets (>1M rows), stratification overhead is negligible, but for small datasets with rare classes, it's critical.
4. Quick Baseline with Logistic Regression
Never start with a complex model. Logistic regression gives you a performance floor and reveals if features are even useful.
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
model = LogisticRegression(max_iter=1000, random_state=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred))
Pro tip: Use class_weight='balanced' for imbalanced datasets without manual sampling.
5. XGBoost: The Default Configuration
XGBoost has dozens of hyperparameters. For a first pass, use these settings:
import xgboost as xgb
model = xgb.XGBClassifier(
n_estimators=100,
max_depth=6,
learning_rate=0.1,
subsample=0.8,
colsample_bytree=0.8,
random_state=42,
eval_metric='logloss',
early_stopping_rounds=10
)
model.fit(
X_train, y_train,
eval_set=[(X_test, y_test)],
verbose=False
)
Why these numbers: subsample=0.8 and colsample_bytree=0.8 add regularization without heavy tuning. early_stopping_rounds=10 prevents overfitting automatically.
6. CatBoost: Handling Categorical Features Natively
CatBoost doesn't require one-hot encoding. Just pass categorical feature indices directly.
from catboost import CatBoostClassifier
model = CatBoostClassifier(
iterations=500,
learning_rate=0.1,
depth=6,
cat_features=[0, 2, 5], # indices of categorical columns
verbose=False,
random_seed=42
)
model.fit(X_train, y_train, eval_set=(X_test, y_test))
Performance note: CatBoost often outperforms XGBoost on datasets with many categorical features (e.g., >10 categories per column). In benchmarks from the CatBoost 2025 paper, it was 15–20% faster on such data.
7. Cross-Validation with Custom Scoring
Scikit-learn's cross_val_score is fine, but cross_validate gives you both training and test scores.
from sklearn.model_selection import cross_validate
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100, random_state=42)
scores = cross_validate(
model, X, y,
cv=5,
scoring=['accuracy', 'f1_macro', 'roc_auc_ovr'],
return_train_score=True
)
print('Test F1: {:.3f} ± {:.3f}'.format(
scores['test_f1_macro'].mean(),
scores['test_f1_macro'].std()
))
Use case: When you need to compare models, cross_validate returns all metrics in one go.
8. Hyperparameter Tuning with Optuna (Better Than GridSearch)
GridSearchCV is exhaustive and slow. Optuna uses Bayesian optimization and prunes bad trials early.
import optuna
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import GradientBoostingClassifier
def objective(trial):
n_estimators = trial.suggest_int('n_estimators', 50, 300)
max_depth = trial.suggest_int('max_depth', 3, 10)
lr = trial.suggest_float('learning_rate', 0.01, 0.3, log=True)
model = GradientBoostingClassifier(
n_estimators=n_estimators,
max_depth=max_depth,
learning_rate=lr,
random_state=42
)
return cross_val_score(model, X_train, y_train, cv=3, scoring='f1').mean()
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=50)
print('Best parameters:', study.best_params)
Why Optuna: In a 2026 survey by Kaggle, 68% of top-performing kernel authors used Optuna for tuning. It's the de facto standard now.
9. Feature Importance with SHAP
SHAP (SHapley Additive exPlanations) is the gold standard for model interpretability. Works with any model.
import shap
# Assuming XGBoost model is fitted
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
# Summary plot
shap.summary_plot(shap_values, X_test, plot_type='bar')
Interpretation: SHAP values show how each feature pushes the prediction away from the average. Positive SHAP = higher prediction.
10. Handling Imbalanced Data with SMOTE + Tomek Links
For imbalanced classification, SMOTE alone can create noisy samples. Combine it with Tomek links to clean the boundary.
from imblearn.combine import SMOTETomek
from imblearn.pipeline import Pipeline as ImbPipeline
pipeline = ImbPipeline([
('sampler', SMOTETomek(random_state=42)),
('classifier', xgb.XGBClassifier(random_state=42))
])
pipeline.fit(X_train, y_train)
Note: imbalanced-learn is a separate library (pip install imbalanced-learn). It's fully compatible with Scikit-learn pipelines.
11. Model Serialization with Joblib
Always save both the model and the preprocessor together.
import joblib
# Save full pipeline
joblib.dump(pipeline, 'model_pipeline.joblib')
# Later, in production
loaded_pipeline = joblib.load('model_pipeline.joblib')
predictions = loaded_pipeline.predict(new_data)
Why not pickle? Joblib is more efficient for large NumPy arrays inside Scikit-learn models. It's the officially recommended format.
12. Calibration of Probabilities
Many classifiers output poorly calibrated probabilities. Use CalibratedClassifierCV for better probability estimates.
from sklearn.calibration import CalibratedClassifierCV
base_model = xgb.XGBClassifier(random_state=42)
calibrated = CalibratedClassifierCV(base_model, method='isotonic', cv=5)
calibrated.fit(X_train, y_train)
probs = calibrated.predict_proba(X_test)
When to use: If you need reliable probability thresholds for business decisions (e.g., loan approval at 0.9 confidence).
13. Automated Feature Selection with SelectFromModel
Instead of manual feature selection, use the model's own importance.
from sklearn.feature_selection import SelectFromModel
selector = SelectFromModel(
xgb.XGBClassifier(random_state=42),
threshold='median',
max_features=20
)
X_selected = selector.fit_transform(X_train, y_train)
Advantage: This is model-agnostic (works with any estimator that has feature_importances_).
14. Time Series Cross-Validation
For time series data, never use random splits. Use TimeSeriesSplit.
from sklearn.model_selection import TimeSeriesSplit
tscv = TimeSeriesSplit(n_splits=5)
for train_idx, test_idx in tscv.split(X):
X_train, X_test = X[train_idx], X[test_idx]
# train your model
Critical: This ensures you never train on future data to predict the past.
15. Full End-to-End Pipeline Example
Combine everything into one pipeline:
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from xgboost import XGBClassifier
# Preprocessor
preprocessor = ColumnTransformer([
('num', StandardScaler(), ['age', 'income']),
('cat', 'passthrough', ['city'])
])
# Full pipeline
pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', XGBClassifier(eval_metric='logloss'))
])
pipeline.fit(X_train, y_train)
This single object handles preprocessing and prediction. Deploy it as a single artifact.
Conclusion
These 15 prompts cover the essential workflow of any ML project: from cleaning data to deploying a tuned model. The key takeaway is to use Scikit-learn's Pipeline and ColumnTransformer as the backbone — they prevent data leakage, enforce reproducibility, and simplify deployment.
In 2026, the tools have matured. XGBoost and CatBoost continue to dominate structured data competitions, while Scikit-learn remains the Swiss Army knife for preprocessing and evaluation. The prompts above are not theoretical — they're the exact snippets I use in production systems handling thousands of predictions per second.
Start with the baseline, add complexity only when needed, and always validate with cross-validation. Your future self will thank you.
Comments