Introduction Methods Ulnar Flexion Conclusions
HANDSHAPE ASSIMILATION IN ASL FINGERSPELLING ULNAR DIGIT FLEXION AND SELECTED FINGERS
Jonathan Keane, Diane Brentari, Jason Riggle University of Chicago
Societas Linguistica Europa , August – September Introduction Methods Ulnar Flexion Conclusions
Broad Questions
How do handshapes in fingerspelling, and broadly, vary, and what causes this variation? ▸ environment? ▸ signer? ▸ phonological properties?
Objectives of this research
▸ Analyze a large corpus of fingerspelling data using a variety of manual, automated, and statistical techniques. ▸ Quantify and investigate the sources of variation within fingerspelling. ▸ Develop an articulatory model of handshape. Introduction Methods Ulnar Flexion Conclusions
A basic description of fingerspelling
▸ Fingerspelling is a type of loanword system that makes up anywhere from – of discourse (Padden, ; Padden and Gunsauls, ). ▸ Simplistically, fingerspelling is a set of static (except for -- and --) handshape-orientation combinations strung together sequentially, where each maps to one letter in an English word. ▸ Many note that this description is not quite accurate (Wilcox (); Akamatsu () &c.).
a b c d e f j z Introduction Methods Ulnar Flexion Conclusions
What fingerspelling looks like; full speed
data.mp4 Introduction Methods Ulnar Flexion Conclusions
Fingerspelling in borrowings
ere has been some work on the process of borrowing from neutral fingerspelling to core lexical items. Battison () looked at loan signs, noting changes to handshape, among other phonological parameters.
Phonological paramaters for signed languages ▸ Handshape ▸ Location ▸ Movement ▸ Orientation ▸ Non-manuals neutral fingerspelling fully assimilated loan signs native foreign core spatial fingerspelling
s foreign other partiallyfullyneutraltwo assimilatedinitialized assimilated fingerspelling-letter signs loan loans loansigns signs egegegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon
native
adapted from Brentari and Padden (); Battison () neutral fingerspelling fully assimilated loan signs
foreign core fingerspelling
s foreign other partiallyfullyneutraltwo assimilatedinitialized assimilated fingerspelling-letter signs loan loans loansigns signs egegegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon
native spatial
adapted from Brentari and Padden (); Battison () neutral fingerspelling fully assimilated loan signs
foreign fingerspelling
s foreign other partiallyfullyneutraltwo assimilatedinitialized assimilated fingerspelling-letter signs loan loans loansigns signs egegegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon
native core spatial
adapted from Brentari and Padden (); Battison () neutral fingerspelling fully assimilated loan signs
foreign fingerspelling
partiallyfullyneutraltwo assimilatedinitialized assimilated fingerspelling-letter signs loan loans loansigns signs egegegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon
native core spatial
s foreign other
adapted from Brentari and Padden (); Battison () neutral fingerspelling fully assimilated loan signs
partiallyfullyneutraltwo assimilatedinitialized assimilated fingerspelling-letter signs loan loans loansigns signs egegegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon
native foreign core spatial fingerspelling
s foreign other
adapted from Brentari and Padden (); Battison () partiallyfullyneutraltwo assimilatedinitialized assimilated fingerspelling-letter signs loan loans loansigns signs egegegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon neutral fingerspelling fully assimilated loan signs native foreign core spatial fingerspelling
s foreign other
adapted from Brentari and Padden (); Battison () partiallyneutraltwoinitialized assimilated fingerspelling-letter signs loans loansigns egegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon neutral fingerspelling fully assimilated loan signs native foreign core spatial fingerspelling
s foreign other fully assimilated loan signs eg
adapted from Brentari and Padden (); Battison () partiallyfullyneutraltwo assimilated assimilated fingerspelling-letter loan loans loansigns signs egegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon neutral fingerspelling fully assimilated loan signs native foreign core spatial fingerspelling
s foreign other initialized signs eg
adapted from Brentari and Padden (); Battison () partiallyfullyneutral assimilatedinitialized assimilated fingerspelling signs loan loansigns signs egegegk---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon neutral fingerspelling fully assimilated loan signs native foreign core spatial fingerspelling
s foreign other two -letter loans eg
adapted from Brentari and Padden (); Battison () fullyneutraltwo assimilatedinitialized fingerspelling-letter signs loan loans signs egegegeg---
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon neutral fingerspelling fully assimilated loan signs native foreign core spatial fingerspelling
s foreign other partially assimilated loansigns eg k
adapted from Brentari and Padden (); Battison () partiallyfullytwo assimilatedinitialized assimilated-letter signs loan loans loansigns signs egegegegk
Introduction Methods Ulnar Flexion Conclusions
Structure of the lexicon neutral fingerspelling fully assimilated loan signs native foreign core spatial fingerspelling
s foreign other neutral fingerspelling eg ---
adapted from Brentari and Padden (); Battison () Introduction Methods Ulnar Flexion Conclusions
Why neutral fingerspelling?
Fingerspelling involves quick and sequential handshape changes, unlike signing. Which results in an ideal data set to look at variation in handshape because there are ▸ a large number of individual tokens ▸ in a huge variety of contexts ▸ using (most of) the handshapes in Introduction Methods Ulnar Flexion Conclusions
Ulnar flexion
We defined ulnar flexion variants as apogees where either the proximal interphalangeal or the metacarpophalangeal joint was more flexed in ulnar digits than radial digits.
--[+flex] --[−flex] --[+flex] --[−flex] --[+flex] --[−flex]
Apogees from --------, -----, -------, -------, -----, and ---, Introduction Methods Ulnar Flexion Conclusions
Ulnar flexion
We defined ulnar flexion variants as apogees where either the proximal interphalangeal or the metacarpophalangeal joint was more flexed in ulnar digits than radial digits.
--[+flex] --[−flex] --[+flex] --[−flex] --[+flex] --[−flex]
Apogees from --------, -----, -------, -------, -----, and ---, Introduction Methods Ulnar Flexion Conclusions
Ulnar flexion
We defined ulnar flexion variants as apogees where either the proximal interphalangeal or the metacarpophalangeal joint was more flexed in ulnar digits than radial digits.
--[+flex] --[−flex] --[+flex] --[−flex] --[+flex] --[−flex]
Apogees from --------, -----, -------, -------, -----, and ---, Introduction Methods Ulnar Flexion Conclusions
Data collection Recording specifications
▸ native signers, early leaner ( (native) coded so far) produced ▸ words ▸ names ▸ nouns ▸ non-English words ▸ repeating each word twice ▸ being recorded by or video cameras ▸ recording at ▸ for a total of , apogees Introduction Methods Ulnar Flexion Conclusions
Coding method Apogee detection
We used a combination of human coders, algorithmic averaging, forced alignment, and verification to code timing data.
Apogees
▸ are the point where the hand reached a target handshape and orientation, or ▸ the point of minimum instantaneous velocity of all of the articulators, but ▸ crucially are not defined as the canonical form.
(Keane et al., ) Introduction Methods Ulnar Flexion Conclusions
Coding method Feature annotation
Feature annotation ▸ We extracted still images from each apogee that has been coded. ▸ We human coded features of hand configuration: ▸ all --, --, --, and -- apogees for ulnar flexion. ▸ ere are , word medial apogees annotated. Two goals ▸ A simple task with only a minimal amount of training necessary ▸ A metric that would apply regardless of how canonical a given handshape was Introduction Methods Ulnar Flexion Conclusions
Description Handshape variation
c d e o
400
300
200 count
100
0
canonical +flexion canonical +flexion canonical +flexion canonical +flexion handshape variant Introduction Methods Ulnar Flexion Conclusions
Description Handshape variation
c d e o
400
300
200 count
100
0
canonical +flexion canonical +flexion canonical +flexion canonical +flexion handshape variant ⎫ ⎪ ⎪ − ⎪ radial: [ flex] ⎬ conditioning ⎪ (>extension) ⎪ ulnar: [+flex] ⎪ ⎭⎪ ⎪⎫ ⎪ all: [±flex] ⎪ non- ⎬ or ⎪ radial:conditioning [+flex] ⎪ ⎪ ulnar: [−flex] ⎭⎪
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What features encourage ulnar digit flexion? Selected Fingers ▸ are described as the most salient fingers for a given handshape, ▸ are oen (but not always!) extended, with other fingers (more) flexed, ▸ are used by many models of sign language phonology.
one finger two fingers three fingers --, --, --, --, --, --, --, --, and --, --, and --, --, --, -- -- and -- all fingers others (ulnar) --, --, --, --, --, --, --, --, and and -- -- ⎪⎫ ⎪ ⎪ ⎬ conditioning ⎪ ⎪ ⎭⎪ ⎪⎫ ⎪ all: [±flex] ⎪ non- ⎬ or ⎪ radial:conditioning [+flex] ⎪ ⎪ ulnar: [−flex] ⎭⎪
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What features encourage ulnar digit flexion? Selected Fingers ▸ are described as the most salient fingers for a given handshape, ▸ are oen (but not always!) extended, with other fingers (more) flexed, ▸ are used by many models of sign language phonology. ⎫ one finger two fingers three fingers ⎪ ⎪ − ⎪ radial: [ flex] --, --, --, --, --, --, --, --, and ⎬ ⎪ (>extension) --, --, and --, --, --, -- ⎪ ulnar: [+flex] ⎪ -- and -- ⎭⎪ all fingers others (ulnar) --, --, --, --, --, --, --, --, and and -- -- ⎪⎫ ⎪ ⎪ ⎬ conditioning ⎪ ⎪ ⎭⎪ ⎪⎫ ⎪ ⎪ non- ⎬ conditioning ⎪ ⎪ ⎭⎪
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What features encourage ulnar digit flexion? Selected Fingers ▸ are described as the most salient fingers for a given handshape, ▸ are oen (but not always!) extended, with other fingers (more) flexed, ▸ are used by many models of sign language phonology. ⎫ one finger two fingers three fingers ⎪ ⎪ − ⎪ radial: [ flex] --, --, --, --, --, --, --, --, and ⎬ ⎪ (>extension) --, --, and --, --, --, -- ⎪ ulnar: [+flex] ⎪ -- and -- ⎭⎪ all fingers others (ulnar) ⎫ ⎪ all: [±flex] ⎪ --, --, --, --, --, --, ⎪ ⎬ or --, --, and and -- ⎪ radial: [+flex] ⎪ -- ⎪ ulnar: [−flex] ⎭⎪ ⎫ ⎪ ⎪ − ⎪ radial: [ flex] ⎬ ⎪ (>extension) ⎪ ulnar: [+flex] ⎪ ⎭⎪ ⎪⎫ ⎪ all: [±flex] ⎪ ⎬ or ⎪ radial: [+flex] ⎪ ⎪ ulnar: [−flex] ⎭⎪
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What features encourage ulnar digit flexion? Selected Fingers ▸ are described as the most salient fingers for a given handshape, ▸ are oen (but not always!) extended, with other fingers (more) flexed, ▸ are used by many models of sign language phonology.
one finger two fingers three fingers ⎪⎫ ⎪ ⎪ --, --, --, --, --, --, --, --, and ⎬ conditioning --, --, and --, --, --, -- ⎪ ⎪ -- and -- ⎭⎪ ⎫ all fingers others (ulnar) ⎪ ⎪ --, --, --, --, --, --, ⎪ non- ⎬ --, --, and and -- ⎪ conditioning ⎪ -- ⎪ ⎭⎪ Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables Flexion based on surrounding handshapes
conditioning percent +flex ● 0.00 ● 0.05 ● 0.10 ● 0.15 ● 0.20
● nonconditioning 0.25 handshape − immediately following
nonconditioning conditioning handshape − immediately previous word type signer name, noun, foreign s, s
previous handshape following handshape
previous transition time following transition time
apogee handshape
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What effects the -- handshape?
-- -- -- -- -- -- -- -- word type signer name, noun, foreign s, s
previous handshape following handshape
previous transition time following transition time
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What effects the -- handshape?
-- -- -- -- -- -- -- --
apogee handshape previous handshape following handshape
previous transition time following transition time
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What effects the -- handshape?
-- -- -- -- -- -- -- -- word type signer name, noun, foreign s, s
apogee handshape following handshape
previous transition time following transition time
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What effects the -- handshape?
-- -- -- -- -- -- -- -- word type signer name, noun, foreign s, s
previous handshape
apogee handshape following handshape
following transition time
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What effects the -- handshape?
-- -- -- -- -- -- -- -- word type signer name, noun, foreign s, s
previous handshape
previous transition time
apogee handshape following transition time
Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What effects the -- handshape?
-- -- -- -- -- -- -- -- word type signer name, noun, foreign s, s
previous handshape following handshape
previous transition time
apogee handshape Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables What effects the -- handshape?
-- -- -- -- -- -- -- -- word type signer name, noun, foreign s, s
previous handshape following handshape
previous transition time following transition time
apogee handshape Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables Significant effects for ulnar flexion word type signer name, noun, non-English s, s
previous handshape following handshape interaction interaction previous transition time following transition time
apogee handshape
(Keane et al., ) Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables Significant effects for ulnar flexion word type signer name, noun, non-English s, s
previous handshape following handshape interaction interaction previous transition time following transition time
apogee handshape
(Keane et al., ) Introduction Methods Ulnar Flexion Conclusions
Conditioning Variables
Model predictions for ulnar flexion (mean transition times)
e o 1.0
0.8 number of conditioning 0.6 handshapes ● none ● 0.4 ● one ● two
0.2 ●
probability of ulnar digit flexion ● ● ● 0.0 ● ● ●
neither previousfollowing both neither previousfollowing both conditioning handshape position Introduction Methods Ulnar Flexion Conclusions
Explanation Conditioning features
▸ Increased ulnar digit flexion occurs in the context of surrounding apogees with handshapes that have extended radial fingers and flexed ulnar fingers. ▸ It seems that flexed, nonselected ulnar fingers are spreading from surrounding apogees to -- and --.
-- -- -- -- -- -- Apogees from ------- and ------ Introduction Methods Ulnar Flexion Conclusions
Explanation Conditioning features
▸ Increased ulnar digit flexion occurs in the context of surrounding apogees with handshapes that have extended radial fingers and flexed ulnar fingers. ▸ It seems that flexed, nonselected ulnar fingers are spreading from surrounding apogees to -- and --.
-- -- -- -- -- -- Apogees from ------- and ------ Introduction Methods Ulnar Flexion Conclusions
Selected finger quantity assimilation Conditioning nonselected fingers? Really?
Alternatively, the selected fingers specification assimilates from surrounding apogees, to apogees that are underspecified for selected fingers (ie [all]). ▸ Many phonological theories of handshape use underspecification, especially for [all] selected fingers. ▸ Underspecified features are traditionally assumed to be the most susceptible to assimilation (cf alveolar nasals). ▸ A similar phenomenon has been observed in sign errors (eg (Klima and Bellugi, )). Introduction Methods Ulnar Flexion Conclusions
Selected finger quantity assimilation Pushing the boundaries
Because of underspecification, handshapes that have [all] fingers selected should be more susceptible to assimilation.
Of the [all] fingers selected handshapes: ▸ -- and -- show variation. ▸ -- and -- show no variation, but all fingers completely flexed. ▸ -- shows little variation, but this could be physiological, ( lexical handshape contours). ▸ -- (seemingly) shows no variation, has all fingers extended, this might be physiological or phonological blocking ( --). Introduction Methods Ulnar Flexion Conclusions
Selected finger quantity assimilation Predictions
If this phenomenon is the result of the assimilation of a phonological feature, and not the result of overlapping gestural constraints more like coarticulation, we expect: ▸ no temporal variation ▸ no effect of word category Introduction Methods Ulnar Flexion Conclusions
Case study: ------- -------; half speed
Clipsinterest.mp4
-- -- -- -- -- -- -- -- Introduction Methods Ulnar Flexion Conclusions
Case study: ------- Articulator trajectories
-- -- -- -- -- -- -- -- 0 500 1000 1500 1 I N T E R E S T 0 thumb 1
0 index 1
0 middle
extension 1
0 ring 1
0 pinky
0 500 1000 1500 time (msec) Introduction Methods Ulnar Flexion Conclusions
Conclusions
For ulnar digit flexion, handshapes vary based on phonological properties of surrounding apogees.
is variation is similar to processes that are used to nativize fingerspelling loan signs.
is process reduces the number of changes to which fingers are selected, bringing the neutral fingerspelling more phonologically toward , without other large scale reductions or deletions.
Fingerspelling is important to study to develop theories about coarticulation and assimilation of handshape, because it pushes the boundaries of what is allowed by phonology. Introduction Methods Ulnar Flexion Conclusions
Future Directions
▸ We need feature annotation between the apogees for a more concrete sense of temporal variation. ▸ We need more precise timing measurements to look at the systematicity of holds and transitions. ▸ We need more precise articulator movement measurements. Introduction Methods Ulnar Flexion Conclusions
I must also acknowledge the contributions of many who contributed in ways big and small: Fingerspelling data Andy Gabel, Rita Mowl, Drucilla Ronchen, and Robin Shay Main advisors Jason Riggle and Diane Brentari Other researchers Susan Rizzo, Karen Livescu, Greg Shakhnarovich, Raquel Urtasun, Erin Dahlgren, and Katie Henry. References Additional plots
ReferencesI Akamatsu, Carol Tane. . e acquisition of fingerspelling in pre-school children. University of Rochester dissertation. Battison, Robbin. . Lexical borrowing in american sign language. Silver Spring, Maryland: Linstock Press, Inc. Brentari, Diane, and Carol Padden. . Foreign vocabulary in sign languages: A cross-linguistic investigation of word formation, chap. Native and foreign vocabulary in American Sign Language: A lexicon with multiple origins, –. Mahwah, NJ: Lawrence Erlbaum. Keane, Jonathan, Diane Brentari, and Jason Riggle. . Handshape and coarticulation in fingerspelling. conference presentation. Linguistic Society of America Annual Meeting. Keane, Jonathan, Susan Rizzo, Diane Brentari, and Jason Riggle. . Phonetic coding of fingerspelling. conference presentation. Building sign language corpora in North America. Klima, Edward, and Ursula Bellugi. . e signs of language. Harvard University Press. References Additional plots
ReferencesII
Padden, Carol. . eoretical issues in sign language research, chap. e Acquisition of Fingerspelling by Deaf Children, –. e University of Chicago press. Padden, Carol, and Darline Clark Gunsauls. . How the alphabet came to be used in a sign language. Sign Language Studies .–. Wilcox, Sherman. . e phonetics of fingerspelling. John Benjamins Publishing Company. References Additional plots
Model predictions based on transition time
e o 1.0
0.8 both 0.6 0.4 0.2 0.0
1.0 following 0.8 number of 0.6 0.4 conditioning 0.2 handshapes 0.0 1.0 none 0.8 neither 0.6 one 0.4 two 0.2 0.0
1.0 previous probability of ulnar flexion 0.8 0.6 0.4 0.2 0.0 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 zscore(log(time)) References Additional plots
Canonical -- (no increased ulnar digit flexion)
hand
nonselected fingers selected fingers [flexed]
joints fingers [flexed]
nonbase thumb fingers
quantity point of ref. [all] [radial] References Additional plots
Canonical -- (no increased ulnar digit flexion)
hand
nonselected fingers selected fingers [flexed]
joints fingers [flexed]
nonbase thumb fingers
quantity point of ref. [all] [radial] References Additional plots
Canonical -- (no increased ulnar digit flexion)
hand
nonselected fingers selected fingers [flexed]
joints fingers [flexed]
nonbase thumb fingers
quantity point of ref. [all] [radial] References Additional plots
-- with increased ulnar digit flexion
hand
nonselected fingers selected fingers [flexed]
joints fingers [flexed]
nonbase thumb fingers
quantity point of ref. [one]>[all] [radial] References Additional plots
-- with increased ulnar digit flexion
hand
nonselected fingers selected fingers [flexed]
joints fingers [flexed]
nonbase thumb fingers
quantity point of ref. [one]>[all] [radial] References Additional plots
-- with increased ulnar digit flexion
hand
nonselected fingers selected fingers [flexed]
joints fingers [flexed]
nonbase thumb fingers
quantity point of ref. [one]>[all] [radial]