# GPU Training Guide

## Quick Start

Your system has been configured for GPU training. The training script automatically detects and uses your GPU.

**Your GPU**: NVIDIA GeForce RTX 5060 (8.55 GB VRAM)

## Starting Training on GPU

### Option 1: Use the training script directly (automatically uses GPU)

```bash
python training/train_with_eval.py \
    --train_data ./data/sample_train.txt \
    --val_data ./data/sample_valid.txt \
    --test_data ./data/sample_test.txt \
    --test_names sample_test \
    --epochs 3 \
    --batch_size 8 \
    --learning_rate 1e-3 \
    --output_dir ./checkpoints/benchmark_run
```

The script will automatically:
- Detect GPU availability
- Move model and data to GPU
- Use GPU for all computations

### Option 2: Run in background (Windows)

```powershell
Start-Process python -ArgumentList "training/train_with_eval.py --train_data ./data/sample_train.txt --val_data ./data/sample_valid.txt --test_data ./data/sample_test.txt --test_names sample_test --epochs 3 --batch_size 8 --output_dir ./checkpoints/benchmark_run" -WindowStyle Minimized
```

## Verify GPU Usage

Check that training is using GPU:

```bash
nvidia-smi
```

You should see Python process using GPU memory.

## GPU Memory Management

If you get out-of-memory errors:
- Reduce `--batch_size` (try 4 or 2)
- Reduce model size in config (fewer layers/smaller embedding_dim)
- The script automatically clears GPU cache between operations

## Monitoring GPU Usage

```bash
# Watch GPU usage in real-time
nvidia-smi -l 1
```

## Stopping Background Training

If you started training in background and need to stop it:

```powershell
Get-Process python | Stop-Process -Force
```

## Troubleshooting

**"CUDA out of memory"**
- Reduce batch_size
- Reduce max_seq_length in config
- Close other GPU applications

**Training is slow**
- Check `nvidia-smi` to verify GPU is being used
- Ensure CUDA drivers are up to date
- Check that PyTorch was installed with CUDA support

**GPU not detected**
- Verify: `python -c "import torch; print(torch.cuda.is_available())"`
- Should print `True`
- If `False`, reinstall PyTorch with CUDA: `pip install torch --index-url https://download.pytorch.org/whl/cu118`

## Current Configuration

- **Device**: Automatically uses GPU if available, falls back to CPU
- **Batch Size**: 8 (adjustable via `--batch_size`)
- **Mixed Precision**: Can be enabled for faster training (future enhancement)

Your training will now run on GPU automatically! 🚀