---
language:
- en
license: mit
base_model: microsoft/Phi-4-mini-instruct
tags:
- finance
- financial-analysis
- qlora
- peft
- lora
- fine-tuned
- sec-filings
- 10-k
datasets:
- virattt/financial-qa-10K
pipeline_tag: text-generation
---

# Phi-4 Mini — Financial Analyst (Fine-Tuned)

A **Phi-4 Mini 3.8B** model fine-tuned on SEC 10-K financial Q&A data using **QLoRA**, 
transforming a general-purpose LLM into a focused financial analyst capable of answering 
questions about earnings reports, balance sheets, risk disclosures, and business performance.

## Evaluation Results

Fine-tuned vs base Phi-4 Mini on 100 held-out test samples from `virattt/financial-qa-10K`:

| Metric | Base Model | Fine-Tuned | Improvement |
|--------|-----------|------------|-------------|
| ROUGE-1 | 0.4657 | 0.7523 | **+61.6%** |
| ROUGE-2 | 0.3560 | 0.6106 | **+71.5%** |
| ROUGE-L | 0.4242 | 0.7168 | **+69.0%** |

## Training Details

| Setting | Value |
|---------|-------|
| Base model | `microsoft/Phi-4-mini-instruct` |
| Dataset | `virattt/financial-qa-10K` |
| Training samples | ~6,300 |
| Method | QLoRA (4-bit quantization + LoRA adapters) |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| Epochs | 3 |
| Learning rate | 2e-4 |
| Optimizer | paged_adamw_8bit |
| Hardware | 2x NVIDIA Tesla T4 (Kaggle) |
| Training time | ~4 hours |

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_id = "microsoft/Phi-4-mini-instruct"
adapter_id    = "Emar7/phi4-finance-finetuned"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

def ask(question, context=""):
    user_content = f"{question}\n\nContext: {context}" if context else question
    prompt = f"""<|system|>
You are an expert financial analyst. Answer questions about company financials, 
earnings reports, and business performance accurately and concisely.<|end|>
<|user|>
{user_content}<|end|>
<|assistant|>
"""
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=256,
            temperature=0.1,
            do_sample=True,
            pad_token_id=tokenizer.eos_token_id,
        )
    return tokenizer.decode(
        outputs[0][inputs["input_ids"].shape[1]:],
        skip_special_tokens=True
    ).strip()

print(ask("What does a high debt-to-equity ratio indicate?"))
```

## Example Output

**Question:** What does it mean when revenue grows but net income declines?

**Base model:** *"When a company's revenue grows but its net income declines, it could mean..."* (generic, unfocused)

**Fine-tuned model:** *"This pattern typically indicates rising operating costs outpacing revenue growth. 
Key areas to investigate in the 10-K include: cost of goods sold trends, SG&A expense increases, 
higher interest expense from increased leverage, or one-time charges impacting the bottom line. 
Analysts refer to this as 'margin compression' and it warrants scrutiny of the company's pricing 
power and cost management."* (structured, analyst-quality)

## Project Links

- **Training notebook:** [Kaggle](https://www.kaggle.com/code/nwangumaemmanuel/phi4-finance-finetuning)
- **GitHub repo:** *(coming soon)*

## Intended Use & Limitations

- **Intended for:** Financial Q&A, document analysis, earnings report interpretation
- **Not intended for:** Real investment advice or trading decisions
- **Limitations:** Trained on 10-K data only; performance on other financial document 
  types (10-Q, 8-K) may vary
- **Base model cutoff:** June 2024

## Author

**Emmanuel Nwanguma** — ML Engineer  
Fine-tuned as a portfolio project demonstrating the full MLOps lifecycle:
data → QLoRA training → evaluation → deployment.