International Journal of Big Data Intelligent Technology, 2024, 5(1); doi: 10.38007/IJBDIT.2024.050110.
Qingbei Kong
The School of International Education of Chinese Language, Beijing International Studies University, Beijing, China
Multimodal oral history discourse encompasses auditory, visual, and other modes, primarily consisting of language supplemented by imagery. The interplay between sound and picture comprises two categories: synchronization and counterpoint. Synchronization predominates in audiovisual materials and interview dialogues within short videos, with the framing, shooting angle, and color mode closely aligned with the video's theme and communication objectives. Sound-picture counterpoint includes correspondence, alignment, and vacant relationships, with correspondence being most prevalent.
Multimodal Oral History Short Video; Auditory Modes; Visual Modes; Sound-Picture Relationship
Qingbei Kong. The Relationship between Sound and Picture in Multimodal Oral History Discourse. International Journal of Big Data Intelligent Technology (2024), Vol. 5, Issue 1: 82-95. https://doi.org/10.38007/IJBDIT.2024.050110.
[1] Bateman,J.Multimodality and Genre:A Foundation for the Systematic Analysis of Multimodal Documents[M].New York:Palgrave-Macmillan,2008.
[2] Flewitt,R.Bringing ethnography to a multimodal investigation of early literacy in a digital age[J].Qualitative Research,2011(3):293-310.
[3] Gunther Kress and Theo van Leeuwen,Reading Images: The Grammar of Visual Design(2nd ed)[M]London:Routledge, 2006.
[4] Halliday M.A.K. & Hasan. Language, Context, and Text: Aspects of Language in a Social-semiotic Perspective [M]. Deakin: Deakin University Press, 1985b.
[5] Holsanova,J.Reception of multimodality:Applying eye tracking methodology in multimodal research[A].In Jewitt,C.(ed).The Routledge Handbook of Multimodal Analysis[C].London:Routledge,2014:287-298.
[6] Jewitt,C.The Routledge Handbook of Multimodal Analysis[M].London:Routledge,2014.
[7] Jewitt , Jeff Bezemer , Kay O'Halloran.Introducing Multimodality[M].Routledge.2016.
[8] John Flowerdew and John E.Richardson.Routledge Handbook of Critical Discourse Studies[M].Routledge,2022.
[9] Kress, G. & T. Van Leeuwen. Multimodal Discourse: The Modes and Media of ContemporaryCommunication[M]. London: Arnold, 2001.
[10] Kress,G.Multimodality:A Social Semiotic Approach to Contemporary Communication[M].London:Routledge,2010.
[11] Longacre, Robert E..The grammar of discourse[M].Plenum Press.1983.
[12] M .O’Toole,The Language of Displayed Art[M].London:Routledge,1994/2011.
[13] Norris,S.Analyzing Multimodal Interaction:A Methodological Framework[M].London:Routledge,2004.
[14] Norris Sigrid.Multimodal Theory and Methodology: For the Analysis of (Inter)action and Identity[M].Routledge.2020.
[15] Scollon,R.& S.Scollon.Discourses in Place:Language in the Material World[M].London:Routledge,2003.
[16] ]Tuire Oittinen11.Multimodal and collaborative practices in the organization of word searches in lingua franca military meetings[J].Journal of Pragmatics.2022:41-55.
[17] Wendy Sandler.Redefining Multimodality[J].Frontiers in Communication.2022.