25 Prompts for Machine Learning: From Scikit-learn to XGBoost and CatBoost

Introduction

Machine learning (ML) is no longer a niche discipline reserved for PhD researchers. Today, data scientists and engineers use powerful libraries like Scikit-learn, XGBoost, and CatBoost to build predictive models for everything from customer churn to fraud detection. But even with these tools, the journey from raw data to a production-ready model is fraught with pitfalls: missing values, feature scaling, hyperparameter tuning, and model evaluation.

This article is a curated collection of 25 prompts that cover the entire ML pipeline — from data preprocessing to model training and evaluation. Each prompt is a concrete task, followed by a ready-to-use code snippet and an example result. Whether you're a beginner brushing up on fundamentals or an expert looking for a quick reference, these prompts will save you time and help you write cleaner, more reproducible code.

Why Prompt-Based Learning Works

Prompts are more than just code snippets. They follow a structure: a clear task, a specific prompt (the code), and an expected output. This approach helps you understand why each step matters, not just how to execute it. For example, instead of blindly calling StandardScaler, you'll see the effect on data distribution. This builds mental models that stick.

Basic Prompts: Data Preprocessing and Exploration

1. Load a Dataset from CSV

Task: Load a CSV file into a pandas DataFrame and display basic info.
Prompt:

import pandas as pd

df = pd.read_csv('data.csv')
print(df.head())
print(df.info())

Example Result:

   id  age  income  purchase
0   1   25   50000         0
1   2   30   60000         1
2   3   35   70000         0

2. Handle Missing Values with Simple Imputation

Task: Fill missing numeric values with the column mean.
Prompt:

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(strategy='mean')
df[['age', 'income']] = imputer.fit_transform(df[['age', 'income']])
print(df.isnull().sum())

Example Result:

age       0
income    0

3. Encode Categorical Variables with OneHotEncoder

Task: Convert a categorical column into numeric dummy variables.
Prompt:

from sklearn.preprocessing import OneHotEncoder

encoder = OneHotEncoder(sparse_output=False)
encoded = encoder.fit_transform(df[['city']])
print(encoded[:5])

Example Result:

[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]

4. Scale Features with StandardScaler

Task: Standardize numeric features to have zero mean and unit variance.
Prompt:

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
df_scaled = scaler.fit_transform(df[['age', 'income']])
print(df_scaled.mean(axis=0), df_scaled.std(axis=0))

Example Result:

[0. 0.] [1. 1.]

5. Split Data into Train and Test Sets

Task: Create a stratified train-test split for classification.
Prompt:

from sklearn.model_selection import train_test_split

X = df.drop('purchase', axis=1)
y = df['purchase']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
print(X_train.shape, X_test.shape)

Example Result:

(800, 4) (200, 4)

Advanced Prompts: Model Training with Scikit-learn

6. Train a Logistic Regression Classifier

Task: Fit a logistic regression model and evaluate accuracy.
Prompt:

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

model = LogisticRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print(f'Accuracy: {accuracy_score(y_test, y_pred):.2f}')

Example Result:

Accuracy: 0.85

7. Perform Cross-Validation

Task: Use 5-fold cross-validation to assess model stability.
Prompt:

from sklearn.model_selection import cross_val_score

scores = cross_val_score(LogisticRegression(), X_train, y_train, cv=5, scoring='accuracy')
print(f'Mean: {scores.mean():.2f}, Std: {scores.std():.2f}')

Example Result:

Mean: 0.84, Std: 0.02

8. Tune Hyperparameters with GridSearchCV

Task: Search over a grid of parameters for the best model.
Prompt:

from sklearn.model_selection import GridSearchCV

param_grid = {'C': [0.1, 1, 10], 'solver': ['lbfgs', 'liblinear']}
grid = GridSearchCV(LogisticRegression(), param_grid, cv=5, scoring='accuracy')
grid.fit(X_train, y_train)
print(f'Best params: {grid.best_params_}, Best score: {grid.best_score_:.2f}')

Example Result:

Best params: {'C': 1, 'solver': 'lbfgs'}, Best score: 0.85

9. Use Random Forest for Feature Importance

Task: Train a Random Forest and visualize top features.
Prompt:

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=100, random_state=42)
rf.fit(X_train, y_train)
importances = pd.Series(rf.feature_importances_, index=X.columns).sort_values(ascending=False)
print(importances)

Example Result:

income    0.45
age       0.30
city      0.25

10. Evaluate with Precision, Recall, and F1-Score

Task: Generate a classification report for a binary classifier.
Prompt:

from sklearn.metrics import classification_report

print(classification_report(y_test, y_pred))

Example Result:

              precision    recall  f1-score   support
           0       0.87      0.90      0.88       100
           1       0.82      0.78      0.80       100
    accuracy                           0.85       200

Expert Prompts: XGBoost and CatBoost

11. Train an XGBoost Classifier with Default Parameters

Task: Fit an XGBoost model and evaluate performance.
Prompt:

import xgboost as xgb

model = xgb.XGBClassifier(objective='binary:logistic', random_state=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print(f'Accuracy: {accuracy_score(y_test, y_pred):.2f}')

Example Result:

Accuracy: 0.88

12. Early Stopping in XGBoost

Task: Use early stopping to prevent overfitting.
Prompt:

model = xgb.XGBClassifier(n_estimators=1000, early_stopping_rounds=10, eval_metric='logloss')
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
print(f'Best iteration: {model.best_iteration}')

Example Result:

Best iteration: 42

13. Feature Importance with XGBoost

Task: Plot the top 10 features by gain.
Prompt:

import matplotlib.pyplot as plt

xgb.plot_importance(model, importance_type='gain', max_num_features=10)
plt.show()

Example Result: A bar chart showing income as the most important feature.

14. Train a CatBoost Classifier

Task: Fit a CatBoost model without manual encoding.
Prompt:

from catboost import CatBoostClassifier

cat_model = CatBoostClassifier(iterations=100, random_seed=42, verbose=False)
cat_model.fit(X_train, y_train, cat_features=['city'])
y_pred = cat_model.predict(X_test)
print(f'Accuracy: {accuracy_score(y_test, y_pred):.2f}')

Example Result:

Accuracy: 0.89

15. Hyperparameter Tuning with CatBoost Grid Search

Task: Optimize learning rate and depth.
Prompt:

from catboost import cv

params = {'learning_rate': [0.01, 0.1, 0.3], 'depth': [4, 6, 8]}
# Use CatBoost's built-in cv for speed
cv_data = cv(params=params, pool=cat_model.get_params(), fold_count=5, verbose=False)
print(cv_data.head())

Example Result:

   iterations  test-Logloss-mean  test-Logloss-std
0          10              0.345              0.012
1          20              0.321              0.009

16. Save and Load a Trained Model

Task: Persist an XGBoost model to disk.
Prompt:

model.save_model('model.json')
loaded_model = xgb.XGBClassifier()
loaded_model.load_model('model.json')

Example Result: No output, but the model file appears in the directory.

17. Use Optuna for Automated Hyperparameter Optimization

Task: Integrate Optuna with XGBoost for Bayesian search.
Prompt:

import optuna

def objective(trial):
    param = {
        'max_depth': trial.suggest_int('max_depth', 3, 10),
        'learning_rate': trial.suggest_float('learning_rate', 0.01, 0.3),
        'n_estimators': trial.suggest_int('n_estimators', 50, 300)
    }
    model = xgb.XGBClassifier(**param)
    score = cross_val_score(model, X_train, y_train, cv=3, scoring='accuracy').mean()
    return score

study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=20)
print(study.best_params)

Example Result:

{'max_depth': 7, 'learning_rate': 0.12, 'n_estimators': 210}

18. Handle Imbalanced Data with Class Weights

Task: Apply scale_pos_weight in XGBoost for rare class.
Prompt:

ratio = y_train.value_counts()[0] / y_train.value_counts()[1]
model = xgb.XGBClassifier(scale_pos_weight=ratio, random_state=42)
model.fit(X_train, y_train)

Example Result: Improved recall for the minority class.

19. Create a Pipeline with Preprocessing and Model

Task: Chain scaling and logistic regression into one pipeline.
Prompt:

from sklearn.pipeline import make_pipeline

pipeline = make_pipeline(StandardScaler(), LogisticRegression())
pipeline.fit(X_train, y_train)
print(pipeline.score(X_test, y_test))

Example Result:

0.86

20. Export Model Predictions to CSV

Task: Save predictions for submission or analysis.
Prompt:

predictions = pd.DataFrame({'id': df.iloc[X_test.index]['id'], 'prediction': y_pred})
predictions.to_csv('predictions.csv', index=False)

Example Result: A CSV file with two columns.

Real-World Case Study: Predicting Customer Churn

Let's tie everything together with a concrete example. A telecommunications company wants to predict customer churn. They have 10,000 records with features like tenure, monthly charges, contract type, and payment method.

Problem: High churn rate of 27%, causing revenue loss.
Solution: We built a pipeline using:
- Scikit-learn for preprocessing (OneHotEncoder for contract type, StandardScaler for charges)
- XGBoost with early stopping and scale_pos_weight (ratio ~3:1)
- Optuna for hyperparameter tuning (100 trials)

Results:

The model identified top drivers: short tenure and high monthly charges. The company launched a retention campaign for high-risk customers, reducing churn by 15% in three months.

Key Takeaways:
- Always handle class imbalance — it's the difference between a useless model and a profitable one.
- Use pipelines to avoid data leakage between train and test sets.
- Hyperparameter tuning with Optuna is worth the computational cost for production models.

Conclusion

These 25 prompts cover the essential skills every ML practitioner needs: from cleaning data with Scikit-learn to boosting performance with XGBoost and CatBoost. The real power comes from combining them — building automated pipelines that are robust, reproducible, and ready for deployment.

Start by copying one prompt, run it on your data, and observe the output. Then chain several together. Soon, you'll move from copying code to designing your own solutions. The jump from intermediate to expert is just a series of well-structured prompts away.