Research
What I work on across NLP and speech, what is current in those areas, and the studies I have released so far.
01 Interests
I work across NLP and speech in roughly equal measure. In language: large language models and how they adapt to new domains and languages, multilingual representation, and clinical and biomedical text. In speech: representation learning, prosody, recognition, and synthetic speech and its detection. Multimodal models sit between the two and have taken more of my attention lately.
A question that runs through most of it is what a model has actually learned, and whether that survives a change of domain, language or recording conditions. It is also why I end up doing a fair amount of evaluation work.
The studies below are the experimental groundwork. They use a manually segmented Persian read-speech corpus and a channel-paired telephone recording set as controlled test beds; the findings are methodological. Code and aggregate results for each are public.
02 Background
The areas I follow, and what is current in each.
Speech. Self-supervised speech representations (the wav2vec 2.0 / HuBERT line) and the speech language models built on them, in particular the shift to discrete speech tokens and the design of the tokenizers themselves — semantic versus acoustic tokens, prosody-aware tokenization, and the trade-off where continuous features still win for ASR and robustness while discrete tokens win on bitrate and phone-level tasks. Audio deepfake and spoofing detection around ASVspoof 5: self-supervised front-ends fine-tuned for spoof detection, frame-level supervision, and the open problem of generalising to generators unseen in training. Speech for health and for child and low-resource language communities, which are the themes Interspeech is foregrounding this year.
NLP. Evaluation of large language models, which has moved from a methodological footnote to a research area of its own: construct validity — whether a benchmark measures the thing it names — benchmark saturation, training-data contamination — including translated-benchmark leakage that surface-form checks miss — and the limits of LLM-as-judge, which recent work finds can be reliable without being valid. The regulatory side matters here too: with the AI Act, demonstrating that an evaluation is sound is becoming a requirement rather than good practice, and the European Commission’s own review of benchmarking practice reads as a research agenda. Multilingual and low-resource modelling, where the gap is now as much about evaluation coverage as about model quality: low-resource languages are scored on one to three task types against fourteen for high-resource ones, and non-Latin scripts remain the hardest case. Small-data and cognitively plausible training (the BabyLM line, now multilingual), and language models for low-resource languages as a research community of its own.
Machine learning. Test-time compute and reasoning models: spending inference compute on intermediate steps instead of scaling pre-training, and how that changes what an evaluation measures. Efficiency in architecture — hybrid attention / state-space models, sparse mixture-of-experts, linear-time attention — as pure scaling meets data exhaustion. Interpretability, and the statistics of evaluation: paired comparisons, confidence intervals and controls, which is where my own work sits.
03 Evaluation methodology
The PARTAGES consortium released PARHAF — an openly licensed corpus of human-authored French clinical reports for fictitious patients — with annotated subsets for seven clinical use cases and no published baselines. This provides the first open baselines for CU3, conclusion extraction, with the controls needed to interpret them.
| System | char-F1 | 95% CI | Median |
|---|---|---|---|
| whole_document (control) | 0.143 | 0.128–0.159 | 0.130 |
| BioMistral-7B (best prompt) | 0.457 | 0.376–0.537 | 0.273 |
| last400chars (positional control) | 0.572 | 0.520–0.623 | 0.628 |
| Rule: first cue, no heading | 0.647 | 0.574–0.714 | 0.817 |
| Rule: last cue, no heading | 0.789 | 0.735–0.837 | 0.904 |
| Qwen2.5-7B-Instruct (best prompt) | 0.812 | 0.754–0.864 | 0.946 |
| Rule: last cue + heading | 0.832 | 0.784–0.874 | 0.920 |
108 manually curated evaluation documents. Bootstrap confidence intervals, 20,000 resamples.
A few things came out of it. Returning the last 400 characters of a document, with no knowledge of French, of clinical text, or of the task, scores 0.572; conclusions occupy a narrow positional band, with mean relative start position 0.887 (s.d. 0.063). Qwen2.5-7B-Instruct reaches 0.812 against the rule baseline's 0.832, a paired difference of −0.020 (95% CI −0.067, +0.027). Prompt variation moves Qwen across 0.651–0.812 and BioMistral across 0.001–0.457, so for BioMistral the prompt range exceeds the gap between models; an explicit abstention option induces spurious abstention on 48% of documents for BioMistral and 22% for Qwen, though every evaluation document contains a conclusion. BioMistral's deficit is largely instruction-following rather than domain knowledge — its token-F1 exceeds its character-F1 by 0.087–0.116, against −0.005 to +0.015 for Qwen.
I also audited the gold annotations, and found that five of the 108 spans were exactly 11 characters long, with the conclusion text sitting beneath them. A 400-character prediction against an 11-character gold span gives F1 = 0.054 no matter how correct it is.
Two implementation details mattered more than they look. Predictions are aligned back onto the source through an explicit normalised-to-original character index map; without it, whitespace collapsing shifts recovered offsets and systematically penalises every generative system. Generations falling below an overlap threshold score zero rather than being force-matched to the nearest passage, so paraphrase and fabrication are penalised rather than absorbed. Two protocol decisions left unspecified in the dataset documentation move scores materially: heading inclusion is worth +0.043 F1 (+0.033, +0.055), and taking the first rather than the last content-bearing cue costs 0.142.
The limitations are worth stating plainly: n = 108, and 81% of the evaluation set is hospitalisation reports. I developed the rule baselines against the same 108 documents I evaluate them on, so 0.832 is an optimistic estimate of held-out performance — the embargoed CU3 test set is the right venue for an unbiased figure. And two models, zero-shot and 4-bit quantised, don’t support any general conclusion about biomedical adaptation.
04 Speech & prosody
How predictable is pronunciation variation in read Persian, and does modelling it help forced alignment? Six pronunciation lexicons — canonical, oracle, and model-generated from phoneme-to-phone transducers (FST, logistic regression, BiLSTM) — evaluated with Montreal Forced Aligner against manual boundaries on a 24-speaker corpus with parallel phoneme and phone tiers.
A channel-paired design: every utterance was captured simultaneously by a studio microphone and through a landline, so any difference between the two is the channel alone. Measures per-segment cue loss and phone identification across four channel conditions, and shows that a simulated narrowband channel does not stand in for a real line.
Uses a corpus in which every speaker reads every sentence twice to test, rather than assume, which timing level is controlled. Cross-repetition variance decomposition over segment, C/V/CV/VV intervals, syllable, word and utterance, each compared against an aggregation baseline.
Rhythm and voice style in Persian read speech, from duration, intensity and MFCC feature tables. Utterance-level stratified cross-validation by talker, a machine-learning benchmark and a DNN comparison alongside the descriptive analysis. The released code is seeded across Python, NumPy, scikit-learn, t-SNE, UMAP, PCA and PyTorch, carries golden-corpus checksums, and regenerates the full manuscript figure set and results summary from one command.
A complete audio analysis stack in pure Rust, no system dependencies — roughly the ground librosa covers. Time–frequency transforms (STFT, constant-Q, variable-Q, reassigned spectrograms), spectral and MIR features, formant estimation and VAD, YIN and pYIN pitch tracking, DTW and Viterbi decoding, NMF, Griffin-Lim reconstruction, loudness weighting, harmonic–percussive separation, and extensive music and phonetics notation utilities. Documentation.
05 Low-resource & Persian NLP
Normalisation, informal-to-formal conversion with multi-candidate output, sentence and word tokenisation, lexicon-aware stemming, dictionary and conjugation-table lemmatisation, full verb conjugation, compound token splitting, and frequency-ranked spell correction. Readers for Bijankhan, Peykare, Hamshahri, Arman and NAAB; pluggable word and sentence embedding traits. POS tagging, chunking, NER and dependency parsing attach as feature-gated components.
A generic structured-perceptron sequence labeler with Viterbi decoding, used as the engine under the taggers below.
06 Learning & models
Feed-forward neural networks and a training loop in pure Rust, dependency-light and embedded-capable.
07 Directions
Where I would like to take this next, stated broadly — I am looking to join a funded project rather than pursue an agenda of my own.
What representations encode. Speech and language models are trained on objectives that never mention most of the properties we then use them for. Whether those properties are encoded anyway, whether the encoding is stable across speakers, domains and languages, and whether it corresponds to anything perceptually or linguistically real, is testable and largely untested outside English.
Robustness under domain shift. A change of channel, recording condition, generator, language or clinical site is where most systems quietly fail. Much of my experimental work has been building designs in which a confound can be isolated and measured rather than argued about, and that carries over to spoofing detection, multilingual adaptation and clinical deployment alike.
Evaluation that holds up. Construct validity, contamination, degenerate baselines and controls, and the reporting conventions around them. This is now a research area in its own right, and in Europe it carries regulatory weight as well.
Language technology for medicine is the application area I would most like to move further into. The PARHAF work above was a first step rather than a specialism.
I am applying for funded PhD positions in NLP and speech. There is more on what I am looking for in my research statement, or write to me at amirhosseinghanipour@proton.me.