LING 322: Speech Technology

Information · Coursework · Policies · Schedule

Information

Course description

How do computers turn speech into text, recognize who is speaking, or generate a human-sounding voice? Why do these systems work well for some speakers and languages but poorly for others? This course introduces the technical knowledge to understand the current practice in speech technology and the relavant linguistic concepts behind it.

We begin with the study of speech as a physical and linguistic signal, and then examine how speech is represented computationally and how modern systems perform tasks such as automatic speech recognition, speaker and language recognition, speech synthesis, and speech-to-speech interaction. We also discuss multilingual and low-resource speech technology, speech data and consent, evaluation, and bias.

The course is open to students with little or no programming background. Supplementary materials in programming and linguistics are provided to help students from a range of backgrounds build the skills they need to participate fully. We will use structured coding notebooks to explore speech data and models. The emphasis is on understanding what a system does, designing informative experiments, interpreting outputs, and connecting computational behavior to concepts from linguistics such as phonetics, phonology, sociolinguistics, and language variation.

Learning outcomes

By the end of the course, you will be able to:

• Explain how speech is represented as a waveform, spectrogram, and sequence of acoustic features.
• Identify and interpret basic acoustic properties including duration, intensity, fundamental frequency, formants, and spectral structure.
• Use Praat and Python notebooks to inspect, visualize, and manipulate speech recordings.
• Explain at a conceptual level how major speech technologies work, including automatic speech recognition, forced alignment, speaker and language recognition, speech synthesis, and modern speech foundation models.
• Run pretrained speech models and use their outputs to investigate linguistic questions.
• Evaluate automatic speech recognition using word error rate and systematic linguistic error analysis.
• Design simple experiments testing how factors such as accent, speaking rate, noise, language, and recording conditions affect speech systems.
• Read a speech corpus critically: how it was collected, who is in it, and what was annotated.
• Critically assess variation, bias, privacy, accessibility, and resource inequality in speech technology.
• Communicate the results of a small empirical speech-technology study to technical and nontechnical audiences.

Prerequisites

No prerequisites. ANTH 200/LING 200 or LING 430 is helpful but not required.

Meeting times and format

This course is taught in person. Class meetings will combine short lectures, demonstrations, activities, and discussion. Please bring a laptop to every class.

Instructor

Texts and readings

There is no required textbook. All required readings are available at no cost through open-access sources or Canvas. The main texts are:

• Jurafsky & Martin. Speech and Language Processing (3rd ed. draft)
• Hugging Face. Audio Course
• Styler. Using Praat for Linguistic Research
• Chodroff. Corpus Phonetics Tutorial

Many research papers in speech technology contain equations or implementation details beyond what is expected for this course. Unless otherwise indicated, read for the research question, central idea, experimental design, major findings, and linguistic implications rather than every technical detail in a research paper.

Additional materials

You may find the following materials useful for background knowledge or additional information not covered in the class.

Phonetics

• Crash Course Linguistics (#8 Consonants, #9 Vowels, #10 Phonology)
• Seeing Speech (MRI and ultrasound video for each IPA symbol)
• Interactive IPA chart

Praat

• Gouskova, “Praat tutorial”
• Praat Tutorials for Speech & Voice Analysis
• Russell, “Using Praat to find vowel formants”

Coding

• Kaggle Python Course

Speech technology

• Hugging Face Audio Course
• Simon King, Edinburgh Speech Processing

Acknowledgments

This course was influenced by several excellent courses and open resources during its development, with gratitude: CS 224S Spoken Language Processing (Stanford, Andrew Maas and Tolúlọpẹ́ Ogunremi), Speech Processing (Edinburgh, Simon King), 11-751/18-781 Speech Recognition and Understanding (CMU, Shinji Watanabe), and the Hugging Face Audio Course.

Coursework

Assignments

There will be six assignments in the class. Most computational work will be completed in notebooks containing starter code, explanations, and partially completed examples. You will modify small pieces of code, inspect outputs, make plots, and write short interpretations. The goal is not to become an expert programmer but to become comfortable with the applicable knowledge to write code to investigate speech.

Homework is graded primarily on completion, interpretation, and evidence of genuine engagement rather than technical correctness. A thoughtful explanation of an unexpected or imperfect result is more valuable than unexplained output. Detailed specifications will be provided on Canvas. The lowest grade from the assignments will be dropped. All homework is due at 11:59 PM on the listed date.

Final project

The final project is a small empirical study using speech data or an existing speech model. Students may work individually or in pairs. Projects should pose a focused question, make a prediction, analyze a manageable dataset or set of recordings, and interpret the results.

Projects are evaluated on the quality of the question, experimental design, analysis, interpretation, and communication rather than on model performance or programming sophistication.

Grading

Letter grades will be assigned using the following scale.

Policies

Participation and keeping up

This course combines concepts from linguistics with unfamiliar computational tools. You are expected to participate actively in code walkthroughs and to ask questions when something is unclear. Computational activities build cumulatively, so if you miss a class, review the posted notebook and materials before the next class.

Collaboration

Discussion and collaborative problem solving are encouraged. You may talk through concepts, compare interpretations, and help classmates diagnose technical problems. Unless an assignment says otherwise, submitted analyses and written responses should be your own, and you should be able to explain the code and results in your submission. List collaborators and cite external code, data, or ideas where appropriate.

Recording and using other people’s voices

Several assignments involve recording speech. A voice is identifying information. Unless an assignment says otherwise, record only yourself, or record another person only with their informed spoken consent captured at the start of the recording. Do not upload recordings of other people to external services, do not submit recordings made without the speaker’s knowledge, and do not use a classmate’s voice to test voice-cloning or speaker-verification tools. If your final project involves recording people, discuss the plan with the instructor first.

Late work and extensions

If unexpected circumstances make it difficult to complete work on time, email the instructor as soon as possible so that we can agree on a reasonable timeline. Without an approved extension or a different policy stated in the assignment, late homework loses 10% of the earned score for each 24-hour period or part thereof, up to 48 hours.

Use of large language models

AI assistants may be useful for explaining unfamiliar code, interpreting error messages, or reviewing concepts, but they should not substitute for understanding the experiment you are conducting. Unless an assignment explicitly permits broader use, an AI assistant may be used as a tutor but should not generate substantial portions of submitted analysis. Any permitted use should be disclosed. You must be able to explain your workflow and the meaning of the results you submit.

To make these expectations concrete, this course uses the AI Usage Scale (Parsons et al. 2024) as the shared vocabulary for describing AI assistance. For the written portions of your work: Levels 0–2 (grammar and sentence-level polish) are always acceptable; Levels 3–5 (brainstorming, feedback on drafts, paraphrasing for your own understanding) are acceptable with disclosure; Levels 6–8 (structural assistance and substantial generation) are not permitted unless an assignment explicitly says otherwise. For code, apply the same scale: having AI explain a concept or an error message sits in the acceptable range, while having it produce the analysis you submit does not. When you disclose AI use, name the level.

Communication

• Use the relevant Canvas discussion board for technical and logistical questions so that everyone can benefit from the answer.
• Use email for grades, accommodations, or other private matters.
• On weekdays, the instructor will normally respond to email within 24 hours; responses may take longer on weekends or university holidays.

Academic integrity

All work in this course is governed by the Rice Honor System. Academic misconduct includes presenting another person’s work as your own, falsifying data or results, using unauthorized assistance, or violating an assignment’s collaboration rules. When in doubt about collaboration, reuse of code, citation, or AI assistance, ask before submitting the work.

Accessibility and accommodations

If a documented disability or other condition may affect your academic work, first contact Rice’s Student Disability Resource Center to determine appropriate accommodations, and then meet with the instructor to discuss how they will be implemented in this course. Please do this as early as possible. You are also encouraged to tell the instructor about temporary conditions or other access barriers; if there is something reasonable I can do to make the course more accessible, please let me know.

Several activities involve listening; each has a visual equivalent. Please tell the instructor early if listening tasks are difficult for you.

Title IX, support, and mandatory reporting

Rice prohibits harassment, sexual harassment, sexual assault, dating violence, and stalking. Under Rice policy, university employees, including the instructor, are Mandatory Reporters and must promptly share information they receive about such conduct involving a Rice student or employee with a Title IX Coordinator. Students seeking support or information about reporting and supportive measures can contact The SAFE Office or review Rice’s reporting options. For an emergency, call 911 or the appropriate campus emergency service.

Religious observances

Please notify the instructor as early as possible if a religious observance conflicts with a class meeting, assignment, presentation, or project deadline so that a reasonable accommodation can be arranged.

Schedule

Class schedule

The schedule is a working plan and may change in response to class progress. Readings are to be completed before the associated meeting. Unless otherwise stated, assignments are due at 11:59 PM on the listed date.