Why
A label is easy to display. Building the full path from an image to that label is the real engineering task.
BSc PROJECT / 2025 · UNIVERSITY OF SHEFFIELD
From an image
to an expression.
Fine-tuning an existing vision model — and connecting it to a working web application.
A VISUAL SYSTEM
01 / THE OVERVIEW
The project, at a glance.
Question
Can a pretrained model classify expressions, and does more training data help across datasets?
Approach
Fine-tune Swin Tiny, compare two training stages, then connect Python inference to a Java web app.
Evidence
Dataset-level accuracy and confusion matrices reveal gains in some settings and losses in others.
Contribution
Data preparation, training control, evaluation and system integration — not a new Transformer.
Lesson
Making a model run is a starting point. Understanding its errors is the next step.
02 / FOLLOW THE IMAGE
A model is one part of the system.
Explore the idea without uploading a face.
CONCEPTUAL DEMO · NO INFERENCE
A single captured image is resized and converted to a three-channel grayscale tensor.
The model divides the input into small patches. Local attention connects nearby patches.
Shifted windows and multiple stages build increasingly contextual image features.
The final layer produces seven scores. The prototype returns the highest-scoring label.
The illustration explains the architecture. It does not recognise emotion, produce a real prediction or request camera access.
- 01CaptureBrowser camera & canvas
- 02UploadJava Servlet receives a file
- 03PredictPython loads the checkpoint
- 04ReturnJSON label; remove temporary file
03 / REPORTED EVIDENCE
More data. Uneven gains.
The same architecture, before and after further training.
THE IMPORTANT PART
A higher combined score can hide a decline.
RAF-DB improved, while FER2013 was nearly unchanged and the synthetic set declined. The mixed-set result is not a real-world accuracy estimate.
RAF-DB was included in the second training stage. This is not evidence of generalisation to a completely unseen data source.
Exact values & comparison limits
| Test set | Epoch 50 | Epoch 62 | Δ percentage points |
|---|---|---|---|
| FER2013 | 71.66% | 71.40% | -0.26 |
| RAF-DB | 60.76% | 85.23% | +24.47 |
| Synthetic | 47.71% | 39.69% | -8.02 |
| Mixed | 69.57% | 76.25% | +6.68 |
The mixed set combines the three sources; it is not a fourth independent population. Duration and data composition changed together. These observations do not isolate the benefit of synthetic data.
04 / TECHNICAL DEEP DIVE
The full workflow, without the wall of text.
01Question & contributionWhat I built — and what I reused.
This was an application and evaluation project. I used an existing pretrained model rather than designing a new attention mechanism.
My work
Prepare inputs, configure training, analyse errors and connect the model to the web interface.
Existing foundations
Swin Transformer and timm provide the network. PyTorch provides automatic differentiation and optimisation.
02Data & preparationBenchmark images, synthetic examples and consistent inputs.
FER2013 and RAF-DB supplied benchmark images; an SDXL script generated auxiliary synthetic images. The classifier used seven labels: angry, disgust, fear, happy, neutral, sad and surprise.
- 01CheckOpen files; check image integrity
- 02StandardiseGrayscale × 3; 224 × 224
- 03AugmentTraining only: flip and ±15° rotation
- 04NormaliseTensor; mean 0.5, std 0.5
x′ = (x − 0.5) / 0.5
For 8-bit inputs converted to [0, 1], this maps values to [−1, 1]. Testing and inference omit random augmentation.
Synthetic-image prompts varied lighting, pose, background, occlusion and expression strength. The archived generator used SDXL Base 1.0, FP16, guidance 9.5 and 50 steps, targeting 200 images per active category. Only “disgust” was active in the saved script; that snapshot does not establish the complete historical generation run.
03Architecture & methodA pretrained Swin Tiny, not a newly invented network.
The timm model identifier was swin_tiny_patch4_window7_224. Pretrained weights were adapted to seven output classes. No custom Swin block was implemented.
Stage 1
56 × 56 × 96 · 2 blocks
Stage 2
28 × 28 × 192 · 2 blocks
Stage 3
14 × 14 × 384 · 6 blocks
Stage 4
7 × 7 × 768 · 2 blocks
(224 / 4)2 = 3,136
A 224 × 224 input is split into 4 × 4 patches. Window attention operates locally; shifted windows exchange information across window boundaries.
pc = exp(zc) / Σj exp(zj)
z is a raw class score. Softmax explains the probability mapping, but the deployed prototype simply selected the highest logit; calibrated confidence was not evaluated.
A theoretical correction I now make: global self-attention scales quadratically, O(N²), in token count, not exponentially as described in the original report.
04Training & configurationTwo reported stages; standard PyTorch optimisation.
0150 epochs
→FER2013 → first checkpoint
02+12 epochs
FER2013 + RAF-DB + synthetic → epoch 62
| Setting | Archived value |
|---|---|
| Batch size | 128 |
| Initial learning rate | 0.0001 |
| Optimiser | AdamW; all model parameters supplied |
| Loss | Cross entropy; label smoothing 0.1 |
| Schedule | CosineAnnealingLR |
L = −Σc yc log(pc)
Cross entropy measures disagreement with the target distribution. Label smoothing softens the targets; it does not establish trustworthy uncertainty.
- 01Predictmodel(images)
- 02Comparecriterion(outputs, labels)
- 03Differentiateloss.backward()
- 04Updateoptimizer.step()
The report describes 50 + 12 epochs. The saved training script instead contains a later 100-epoch configuration and loads an existing checkpoint. I do not treat that snapshot as proof of the original run, or use changed file paths to infer experimental mistakes.
05Evaluation & validationWhat the measurements support, and what they do not.
Overall accuracy answers “how many labels matched?” Class-wise accuracy and confusion matrices reveal which labels were confused. The archived code does not establish macro-F1, a confidence-calibration study or an independent robustness benchmark.
Separate development from testing
A clean final test should not guide checkpoint or hyperparameter selection. Historical split membership needs records, not guesses from folder names.
Explain the comparison
The second stage changed both training duration and data sources. Without controls, their individual effects cannot be isolated.
06System design & implementationHow the browser, Java and Python communicated.
The original prototype was a capture-and-submit web application, not continuous frame-by-frame emotion recognition. The current portfolio does not run that classifier.
| Module / function | Responsibility |
|---|---|
| openCamera() / captureImage() | Browser camera permission and canvas capture |
| submitImage() | PNG upload with FormData and fetch |
| doPost() | Accept upload, create temporary file, return JSON |
| runPythonScript() | Start Python with ProcessBuilder; read output |
| predict_image() | Preprocess → checkpoint inference → highest-score label |
| finally / deleteIfExists() | Attempt temporary image cleanup |
The interface included home, detection, about, contact and privacy pages. HTML, CSS, JavaScript and Bootstrap formed the client; Java Servlet / Tomcat invoked Python. Training used a cloud GPU; the web demonstration used a local server with an HTTPS tunnel. No production load test or comprehensive security audit is claimed.
07Testing, deployment & scopeAn academic prototype, with clear operating limits.
| Stage | What was checked | Evidence boundary |
|---|---|---|
| Functional | Camera, upload, inference and response | Report and code; not a new execution |
| Compatibility | Desktop and mobile interfaces | Reported manual testing |
| Privacy | HTTPS and temporary-file cleanup | Basic controls; not a compliance certification |
| Performance | A response-time design target | No complete benchmark log available |
05 / CRITICAL REFLECTION
From “it runs” to “why should I trust it?”
I learned to connect a complete machine-learning application. Looking back, I would spend more time understanding errors and testing alternatives — not only making the pipeline run.
THEN
Make one model work
WHAT I NOW SEE
A working pipeline is not a fair comparison.
NEXT TIME
Add a matched ResNet baseline and controlled ablations.
THEN
Focus on overall accuracy
WHAT I NOW SEE
Class imbalance and domain differences can hide errors.
NEXT TIME
Report macro-F1, class errors and independent shift tests.
THEN
Use synthetic data as extra examples
WHAT I NOW SEE
A prompt is not a verified label.
NEXT TIME
Review labels and separate the effects of source and duration.
THEN
Keep an executable project
WHAT I NOW SEE
A code snapshot does not reconstruct every experiment.
NEXT TIME
Record splits, seeds, environment and run-specific settings.
These are current reflections and proposed improvements, not claims that the additional experiments have already been completed.
↗Sources, credits & publication boundariesWhat was reviewed, what is public, and why some materials stay private.
Checked against the 2025 BSc report (especially Chapters 5–6) and the supplied model, training, inference, data and web files. Reported numbers were not reproduced by retraining. Archived configurations are distinguished from the reported experiment.
Method credits
Policy review: 4 November 2026. Assessed student work may be shareable, subject to ownership, collaboration, module, third-party and data conditions. The general policy is not project-specific permission. This site distributes no dissertation PDF, original research source, model weights or patient images.
Diagrams are original, illustrative geometry. They do not reconstruct patient images or show new model experiments.