Media Theory, Digital Culture & Cognitive Technologies
Professor Martin Irvine
Communication, Culture & Technology Program
[Archived version of course, 2012-2014
Current updated version of this course is here.]
This course will introduce the central ideas in media and communication theory, semiotics and meaning systems, the study of the digital media environment, and nature of "cognitive technologies". The guiding question of the course: how can we use all our interdisciplinary resources to develop a media and communication theory for all the forms of media, software-produced media artefacts, and computational environments that we experience today?
The course will provide a foundation in the major traditions of media, communication, and information theory; semiotics and language; mediology and network theory; and an introduction to computation, technology theory, and cognitive science approaches to meaning and communication. The course will have methodological emphasis in applying interdisciplinary models for investigating media and mediation in their technical and social dimensions.
Central topics for study will include the continuum of human symbolic systems, the implications of the expanding pan-digital platform for all media, the analog-digital continuum, the question of "big data" and the collective memory of computational networks, and the ongoing renewal of media content through new technologies of digitization.
Students will be expected to create the seminar in real time through readings, discussion, and proposal of cases and examples for study. Grading and weekly seminar assignments will be based on student presentations, short applied theory essays in a weekly Wiki-style environment (using a Wordpress site), and a final project involving the application of a theoretical model to a contemporary or historical case.
- Floridi, Luciano. Information: A Very Short Introduction. Oxford, UK: Oxford University Press, 2010. ISBN: 0745645720
- William Gibson, Pattern Recognition. New York: Putnam, 2003; Berkeley Publishing, Reprint edition, 2004. ISBN: 0425192938
- Lev Manovich, Software Takes Command: Extending the Language of New Media (London; New York: Bloomsbury Academic, 2013).
- W. Brian Arthur, The Nature of Technology: What It Is and How It Evolves. New York, NY: Free Press, 2009. [ISBN 1416544062]
- Regis Debray, Media Manifestos. Trans. Eric Rauth. London and NY: Verso, 1996. ISBN: 1859840876
- George Dyson, Turing’s Cathedral: The Origins of the Digital Universe. New York, NY: Pantheon/Vintage, 2012. ISBN: 1400075998
- Bruno Latour, Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford University Press, USA, 2005. ISBN: 0199256055
- Lev Manovich, The Language of New Media. Cambridge, MA: MIT Press, 2002. ISBN: 0262632551
- Neal Stephenson, The Diamond Age, or, A Young Lady’s Illustrated Primer. New York, NY: Bantam Spectra, 1995; reprinted 2002. ISBN: 0553380966
- Neal Stephenson, Cryptonomicon. New York, NY: William Morrow / Avon Books, 1999. ISBN: 0380788624
- Noah Wardruip-Fruin and Nick Montfort, eds. The New Media Reader. Cambridge, MA: MIT Press, 2003. ISBN: 0262232278
|1 Introduction to Media Theory: Communication, Mediation, Information, Symbolic Systems||[−]|
Introduction to Doing Media, Communication, and Information Theory
How can we approach the subject matter and domains of knowledge constructed through this concept cluster:
- Communication, information
- Language, natural language, mathematics
- Signs, Symbols, Symbolic Systems and Symbolic Processes
- Symbolic cognition, cultural meaning, collective cognition, memory
- Medium, mediation, media, mass media
- Technology, “Information and Communication Technologies” (ICTs)
- Computation, Software Code, Digital Media
- Interfaces, Human-Computer Interaction, Digital and Analogue Continuum
These concepts and terms are part of long histories of discourse, disciplines, research agendas, and knowledge paradigms. The terms are often used to name areas of concern for separate disciplinary domains, but are now inseparable in our current media and communications environment.
We use many of these terms every day without thinking through what they entail. We need to investigate the consequences of holding various embedded assumptions in these discursive fields for real-world thought and action. In this course, we will consider the big picture of theory and philosophy and provide ways to critique and work out new interdisciplinary directions.
These combined and overlapping areas of research and theory now constitute one large interdisciplinary field. Academic departments and professional organizations sustain the demarcations of practice, disciplinary identity, and knowledge legitimization. Media and technology theory opens up the understanding all the codes we live by: text, images, photography, film/video, TV, advertising, Internet and Web media, and the whole mediasphere in the digital environment.
Introduction to Theory and Conceptual Models (review first):
- Media Theory: An Introduction (Introduction, working draft of book chapter)
- Seminar Introduction: Media Theory and Cognitive Technologies (presentation) (Irvine)
- Main Theory Cluster for media interpretation: Presentation slide (Irvine)
- A Map of Received Theory: Traditions, Philosophies, Methodologies (Irvine)
Introductory Case Studies: art works, film, television, Web, i-device apps
|2 Communication & Information Theory: Foundational Concepts||[−]|
Learning Objectives and Discussion Questions
Communication theory from the 1960s-80s provides some major common assumptions that are assumed or critiqued in current media and information theory. As models of the transmission of meaning, these theories also inform how we think about visual culture as a language and set of communications media. Consider the main assumptions, then ask what is left out of the models? For example, transmission through time, the limits of linear one-way models, larger questions about production and receptions contexts that are more like networks than point-to-point connections. Models must explain all the communication "modalities": one to one, one-many, many-one, many-many, and dialog (one-other, mutually assumed); synchronous (at the same time) and asynchronous (delays in reception-response, short time span or very long).
Models of Communication and Information and Their Consequences
- Floridi, Information, Chapters 1-4.
- Day, Ronald E. "The ‘conduit Metaphor’ and the Nature and Politics of Information Studies." Journal of the American Society for Information Science 51, no. 9 (2000): 805-811.
Models and metaphors for "communication" have long been constrained by "transport", "conduit," and "container/content" metaphors that provide only one map of a larger process. How can we describe "communication" and "meaning" in better ways that account for all the conditions, contexts, and environments of "meaning making"?
- Irvine, Conceptual Models in Communication Theory: Communication Models and Methods (presentation)
- Models of the Communication Process and An Ecological Model of Communication
(Davis Foulger, Brooklyn College/CUNY)
Good overviews of the history of approaches to communication and a more inclusive ecological systems model. Note the context of the original transmission models and further complex models. Note that some models come from rhetorical thought, engineering and and information sciences, and sociology and semiotics.
Communication and Culture: Beginnings of a New Paradigm
- James Carey, "Communication
and Culture" (from Communication as Culture: Essays on Media and Society, 1992) [pdf]. Summary
of Carey's Views (excerpts)
[An influential essay by a leader in the modern field of "Communication" that repositions the study of communication and mediating technologies in the cultural, ideological, and economic context.]
- Stuart Hall, "Encoding/Decoding" (first published, 1973). An important revision of communication theory with the semiotics of cultural codes and meaning systems. Influential in many fields.
Supplementary Background, Bibliography, and Sources
- Primary sources:
- Claude Shannon, The Mathematical Theory of Communication (1948). The Bell System Technical Journal,
Vol. 27, pp. 379–423, 623–656, July, October, 1948.
- Warren Weaver, "Recent Contributions to the Mathematical Theory of Communication." An introduction to Shannon's theory and article. From: Shannon, Claude E., and Warren Weaver. The Mathematical Theory of Communication. Champaign, IL: University of Illinois Press, 1949.
- Craig, Robert T. "Communication Theory as a Field." Communication Theory 9, no. 2 (1999): 119–161.
- Jerry Norman's History of Information Site (wealth of primary source backgrounds)
See especially: Timeline of artefacts in the history of Information and Communication.
Using the concepts in the readings, think through questions these questions:
Are message "contents" what are "communicated" in a communication act or event?
Is human natural language a sufficient model of--and modeling system for--communication?
Where and when are "meanings" in communication and information? What are the conceptual consequences of the content - container - transport/conduit models of communication? Can we model the contexts and environments of meaning not explicitly stated in the "information" of a "transmitted message"?
Examples and cases:
Communication theory and media artefacts: reverse engineer (decompose and segment components) an iPhone for the history and dependencies of component technologies (convergence of media and technologies). Are symbolic artefacts and expressions (music, artworks, photographs, film/video) "communications"? what is missing in the "information theory" model in accounting for messages and meanings. Where are the meanings in our understanding of artefacts?
|3 Introduction to the Study of Language and Symbolic Cognition: Key Issues in Linguistics||[−]|
The Nature of Human Language and Human Symbolic Systems
The fact of the human capacity for language is the starting point of many disciplines and research programs in all aspects of communication, symbolic culture, and media. Much of the research questions and the terminology for the study of language and human symbol systems ("vocabularies" of description, as Rorty would say) has been set by the various specialities of modern linguistics. The terms and categories for analysis in linguistics have also been widely used heuristically by other disciplines, and all students studying media and communication need to be familiar with the basic agenda and research programs in the major branches of linguistics.
Familiarity with the concepts, terms, and assumptions of contemporary linguistics is thus essential for discussing and describing all other symbolic combinatorial systems that use language or function as language-like systems. An important open question for interdisciplinary research in all related sciences is whether we can accept the faculty of language (FOL), and the cognitive “triggers” that happen in language acquisition, as the foundation of all human symbolic processes (all those based on combinatoriality, recursion, and intersubjective material-conceptual symbols) from writing to mathematics and multimedia, or, rather, should we research further the evidence for a more generalized symbolic faculty of which language is one major (or the major) implementation. Either model leads us into the central questions about communication, culture, and technology.
We can only do a top-level overview here, but with some familiarity in the problems and core concepts, you can advance to topics of interest in your own research.
Key concepts to be used in this course: generativity, combinatoriality/compositionality, intersubjectivity, pragmatics (contextual and situational analysis, shared assumptions, speech and discourse genres and speech acts), sociolinguistics (language in everyday use and language in social group formation and identities).
Language as a Symbolic Cognitive System: The Linguistics Model
Read the "Introductions" and "Backgrounds" sections, and others as time permits, and then experiment with the online syntax tool below
- Introductions (read in this order)
- Steven Pinker, Video presentation on Language and the Human Brain (start here) (Floating University)
- Steven Pinker, "How Language Works." Excerpt from: Pinker, The Language Instinct: How the Mind Creates Language. New York, NY: William Morrow & Company, 1994: 83-123.
[Accessible introduction to central issues by a leading cognitive linguist.]
- Excerpt from Andrew Radford, et al. Linguistics: An Introduction. 2nd ed. Cambridge, UK: Cambridge University Press, 2009. Read and scan enough in each section to gain familiarity with the main concepts.
Includes the Table of Contents for the whole book so that you can see the contents of a typical recent textbook (following a post-Chomsky approach). A good starting reference.
Excerpts on Introduction to Linguistics as a field, and sections on Words (lexicon) and Sentences (grammatical functions and syntax).
- Backgrounds to Main Concepts
- Martin Irvine, "Linguistics: Key Concepts" (Comparing structuralist and Chomsky's linguistic concepts)
- John Searle, "Chomsky's Revolution in Linguistics," The New York Review of Books, June 29, 1972.
[If you have no prior experience in linguistics, this article is a good introduction to the backgrounds in philosophy of language and the problems that Chomsky took on in the 1960s. Many of the issues have developed in other ways since the 1970s, but the fundamental research questions remain.]
- More advanced
- Morten H. Christiansen and Simon Kirby, “Language Evolution: The Hardest Problem in Science?,” Introductory chapter in Language Evolution, ed. Morten H. Christiansen and Simon Kirby (Oxford; New York: Oxford University Press, 2003), 1-15. Excerpt.
- Ray Jackendoff, “Précis of Foundations of Language: Brain, Meaning, Grammar, Evolution,” Behavioral and Brain Sciences 26, no. 06 (2003): 651-665.
[Scan for the basic issues and key arguments. An introduction to the main argument and research backgrounds in Jackendoff's major work, Foundations of Language (2003).]
- Advanced: Primary Sources
- Noam Chomsky, "Form and meaning in natural languages." Excerpt from Language and Mind, 3rd. Edition. Cambridge University Press, 2006.
[This exposition is from the middle period of Chomsky's thought, and provides the "standard theory" of generative linguistics that now has many branches, modifications, and elaborations. If you work through this, you'll have a sense of what motivates much linguistic research on the central question of "what is language" right up today. Note: Chomsky focuses on syntax, and does not include important questions in semantics and pragmatics.]
- Noam Chomsky, Aspects of the Theory of Syntax , Chapter 1 (excerpt). Cambridge, MA: MIT Press, 1965. Excerpt in pdf.
[This was the book that launched a new way to do linguistics with a syntax-centered generative model.]
Syntax Tool: Visualization of Sentence Structure and Computational Parsing
- XLE-Web: parsing tool and tree generator for syntax based on the "Lexical Functional Grammar" model of generative grammar. Choose "English" and map any sentence for its constituent (c- ) and functional (f- ) structure! (Uses linguistic notation from two formal systems.) Syntax parsers are used in computational linguistics and all complex text analysis in software and network applications. Web search algorithms and Siri voice recognition and interpretation have to use parsers for generating the probable grammatical structure of natural language phrases
- Try the test sentence "I like dark beer but dark beer doesn't like me." You will see how the parser generates several grammatical structure options because of the ambiguities in switching subject and object with the verb [like]. Then try any sentence; try some very complex ones (compound subjects, many conjunctions and clauses, etc.).
Weekly Seminar Discussion (Wordpress site)
Drawing from the readings, how would you answer these questions for someone who has little or no knowledge of linguistics as a science or field of research: What is language? What is "a language," what is required for any language to be a language?
From these foundations, we will go on to ask other important questions that follow: what are the implications of using language as a system as the model for other symbolic systems (visual, audio, combinations) and most forms of communication and media? Do other symbolic systems that we use for expression (visual, music, multimedia combinations) work like a language (combinations by rules that precede individual expressions)? Try working with one or two of the main linguistic concepts as models for new ways of investigating language use and other media in the forms of communication we use every day.
|4 Symbolic Cognition and Human Meaning Making: Intro to Cognitive Science Approaches||[−]|
From Linguistics to larger questions of the Human Symbolic Faculty in all media
The readings and concepts for weeks 3 and week 4 are intended to provide "building blocks" of conceptual resources for modeling the ways we organize and structure meaning systems in all our media. Within a broad cluster of fields--ranging from neuroscience to cognitive linguistics, cognitive anthropology, and computational models of cognition and artificial intelligence research--there has been a major convergence on questions and interdisciplinary methods for studying cognition, human meaning-making and the "symbolic faculty" generally, including all our cumulative mediations and externalizations in "cognitive technologies."
Many disciplines now converge on how our meaning systems all work with analogous and parallel "architectures" that must include (1) rules for combinatoriality of components (an underlying syntax for forming complex and recursive expressions of meaning units), (2) intersubjective preconditions “built-in” to the meaning system for collective and shared cognition, and (3) material symbolic-cognitive externalizations (e.g, writing, images, artefacts) transmitted by means of "cognitive technologies" (everything from writing to digital media and computer code) which enable human cultures and cultural memory. This recent interdisciplinary research is a "game changer" for the way we think about human communication and media technologies.
The human symbolic faculty has generated a continuum of functions from language and abstract symbolic thought to machines, media technologies, and computation:
Language > Symbol Combinatoriality > Abstraction > Mathematics > Machines > Computation
The mainstream disciplines in communication and media studies are very conservative (remaining within a demarcated field in the humanities and social sciences) and have not yet incorporated recent advances in cognitive science fields that are directly relevant to core assumptions and research questions on language, symbolic culture, and media technologies. We therefore have an open opportunity to learn from cognitive science fields and reconfigure the inter- and transdisciplinary field in promising ways for both theoretical and applied work.
Related Principles in all Symbolic Systems and Technologies: Combinatoriality, Compositionality, Componentiality, Recursion, Externalized Memory, Intersubjective and Collective Culture.
Main arguments to know in the various cognitive science disciplines:
Terrence Deacon: human evolution to the symbolic species, and possible model for the "symbolic threshold" in the transition to symbolic cognitive abilities.
Lakoff and Johnson: the centrality of metaphor for semantics and human thought in general
Edwin Hutchins and Andy Clark on the model of "distributed cognition" and "extended mind"
Neuro-evolutionary biology of cognition: Merlin Donald on being human as being symbolic
Introductions and Orientations
Approaches to Symbolic Cognition and Cognitive Bases for Human Meaning Systems
- Andy Clark, Mindware: An Introduction to the Philosophy of Cognitive Science. New York: Oxford University Press, 2001. (Excerpts from Introduction, Chapters1-2)
- Terrence W. Deacon, The Symbolic Species: The Co-evolution of Language and the Brain. New York, NY: W. W. Norton & Company, 1998. Excerpts from chapters 1 and 3.
- Kate Wong, “The Morning of the Modern Mind: Symbolic Culture.” Scientific American 292, no. 6 (June 2005): 86-95.
- George Lakoff, "Conceptual Metaphor." Excerpt from Geeraerts, Dirk, ed. Cognitive Linguistics: Basic Readings. Berlin: Mouton de Gruyter, 2006.
- Hollan, James, Edwin Hutchins, and David Kirsh. “Distributed Cognition: Toward a New Foundation for Human-computer Interaction Research.” ACM Transactions, Computer-Human Interaction 7, no. 2 (June 2000): 174-196.
[Edwin Hutchins has pioneered research into "distributed cognition," and this work has important implications for the our concept of technology and the interface.]
- Zhang, Jiajie, and Vimla L. Patel. “Distributed Cognition, Representation, and Affordance.” Pragmatics & Cognition 14, no. 2 (July 2006): 333-341.
[This is a useful, short summary of central issues in this field of research, accessible to non-specialists.]
- Background Readings (for reference and as time allows for this week):
- Merlin Donald, "Evolutionary Origins of the Social Brain," from Social Brain Matters: Stances on the Neurobiology of Social Cognition, ed. Oscar Vilarroya, et al. Amsterdam: Rodophi, 2007.
- Colin Renfrew, “Mind and Matter: Cognitive Archaeology and External Symbolic Storage.” In Cognition and Material Culture: The Archaeology of Symbolic Storage, edited by Colin Renfrew, 1-6. Cambridge, UK: McDonald Institute for Archaeological Research, 1999.
[Important argument to supplement Merlin Donald's view about the evolutionary origins of the symbolic brain: material culture is part of the externalizing cognitive process.]
- Irvine, "Introduction to Cognition, Symbol, Meaning, and Mediation" (presentation)
Weekly Seminar Discussion (Wordpress site)
This week is an introduction to some very big, macro-level questions about human cognition, the symbolic and language faculties, and possible explanations for how and why humans are the "symbolic species." This involves difficult, often (initially) counter-intuitive ideas at a high level of abstraction and interpretation of research evidence. For your writing this week, choose one of these two approaches: (1) consider a media artefact as an instance of a combinatorial meaning system (an artwork, a Website, a movie, music composition) with some features investigated in readings . We will investigate the structure of meaning systems more fully later, but this week you can begin with developing intuitive conceptual insights into the "grammar" of a media form: example: how does a genre (the type, the "cliche" inventory) provide the grammar of possibilities for a form, and what happens when we combine symbolic resources (language, visual form, music/sound, video/cinema, etc.) in a media form. (2) How would key concepts in this week's readings enable a thorough revision of the traditional information and communication models (from Week 2), and how could these concepts change the way you think about meaning systems and media more generally?
|5 Introduction to Media Theory: Medium, Media, Mediations||[−]|
Understanding major schools of thought in the history of media theory, the conceptual metaphors and discourse for describing and analyzing medium, media, mediation, and mediality, and the related concept of interface.
This week, we begin developing tools for analysis, critique, and research that we will expand for the later focus on digital media and culture specifically. How do the concepts of "medium," "mediation," and "interface" connect with the conceptual models for information/communication and for symbolic cognition that we studied?
- Irvine, "Media Theory: An Introduction" (overview of issues and approaches; read through section 7)
Marshall McLuhan and the Influence of His Theory
- Marshall McLuhan, "The Medium is the Message," (Excerpts from Understanding Media, The Extensions of Man, Part I, 2nd Edition; originally published, 1964). (See especially Part 1, "The Medium is the Message," sections 1-3.).
- Daniel Czitrom, "Metahistory, Mythology, and the Media: The American Thought of Marshall McLuhan," excerpt from Daniel J. Czitrom, Media and the American Mind: From Morse to McLuhan. Chapel Hill, NC: University of North Carolina Press, 1982; read pp.172-182.
- Lisa Gitelman, Always Already New: Media, History, and the Data of Culture. Cambridge, MA: The MIT Press, 2008. Excerpt from Introduction. [Includes excellent bibliography of references.]
- Daniel Chandler, "Processes of Mediation." Background to the issues that Debray is thinking beyond.
New Media before "New Media": Print Culture and the Telegraph as Paradigm Cases
- Elizabeth L. Eisenstein, "Some Features of Book Culture," from Elizabeth L. Eisenstein, The Printing Revolution in Early Modern Europe. Cambridge, UK: Cambridge University Press, 1983. Rev. ed. 2005.
[This chapter summarizes the highly influential work of Eisenstein on the features of print culture after the development of the printing press in the Renaissance. Compare with McLuhan. Consider how the theorizing proceeds by extrapolating from material-functional properties of the medium and historical conditions.]
- James W. Carey, "Technology and Ideology: The Case of the Telegraph," excerpt from Carey, Communication as Culture: Essays on Media and Society. Revised edition. New York, NY and London, UK: Routledge, 1989.
[The telegraph has been called "The Victorian Internet." This essay is a classic in the field. An important analysis of the telegraph as allowing, for the first time, the separation of messages from a physical medium requiring transportation through space. Important consequences leading up to the Internet.]
More than Messages in the Medium: Theory of Mediation and Media Systems
- Regis Debray, "What
is Mediology?" Le Monde Diplomatique, Aug., 1999. (Trans.
[This essay is a brief introduction to the approach of mediology, which we will explore more fully later.]
Background Sources and Supplementary Readings: Media Theory History (optional, as time allows)
- "The Long History of New Media," Introduction to Park, David W., Steve Jones, and Nicholas W. Jankowski, eds. The Long History of New Media. New York, NY: Peter Lang, 2011.
[An interesting book that opens up the larger history of "new" media. Note chapter topics and approaches.]
- Specters of McLuhan: Research Site for the History of McLuhan's influences
- Marshall McLuhan, Official Site (McLuhan Estate).
and Technological Determinism:
Paul Jones, "The Technology is not the Cultural Form?: Raymond Williams's Sociological Critique of Marshall McLuhan," Canadian Journal of Communication, Vol 23, No 4 (1998).
- James C. Morrison, "The Place of Marshall McLuhan in the Learning of His Time," Counterblast, 1 (2001), NYU.
- James Carey, "Marshall McLuhan: Genealogy and Legacy," Canadian Journal of Communication, 23, Mar. 1998.
- Friedrich Kittler, "The History of Communication Media," C-Theory, 1996.
[This essay is in the spirit of McLuhan, and still perplexed by material-deterministic views. Important for Kitler's kind of argument, and an overview of media history as commonly accepted in university disciplines.]
- Specters of McLuhan: Research Site for the History of McLuhan's influences
Weekly Seminar Discussion (Wordpress site)
In studying media today, we often get caught up in the discourse of "new media" without understanding longer histories and dependencies, and social conditions in place before the arrival of the "new." Consider "text" in today's media system and the various interfaces and representations of text in media today. What is the medium, and what is being mediated? Try to begin uncovering the system of dependencies and mediations at work for a medium to function. Concepts to work with: media as orchestrated combinations of technologies and social conditions; media and the conditions of space and time; the social-ideological value, power, and authority of a medium (example: books, print, and publishing retain high value and prestige at a time of major transition to digital media). Your reflections on these key questions are a starting point for the analysis of digital media culture specifically in the weeks ahead.
|6 The Grammar of Meaning Making: Introduction to Semiotics and Symbolic Systems||[−]|
Understanding the key concepts in semiotics as a major conceptual framework for studying cognitive technologies. Building on prior weeks, in this unit we will investigate models for symbolic cognition and meaning in collective, intersubjective, rule-governed systems.
Semiotics and related fields are notoriously difficult to understand at first, and it's also too common to get started in a confusing way and then never get to the pay off conceptually. So many of the textbooks are boring and merely academic; I couldn't bring myself to assign them to you.
So, in our interdisciplinary adventures, we find semiotics now being integrated like a common software routine across many other sciences and disciplines (media and cultural studies fields in the humanities, anthropology and many social science fields, distributed and symbolic cognition research in the cognitive sciences, information and computation theory). Some may even hold out a utopian dream of a way to "unify" humanistic and scientific fields in a generalizable semiotics that accounts for language, concepts, cognition, meaning, knowledge, symbolic processing in mathematics and computation, and the ongoing dialogic development of all cultures.
Be that as it may, we need to work through the foundational concepts and problems, many of which are counter-intuitive (not common sense), and see what we see by extending semiotic and symbolic cognition hypotheses into our worlds of meaning.
- Martin Irvine, "Introduction to Meaning Making, Symbolic Cognition, and Semiotics" (begin here)
Read sections 1-4.
- Floyd Merrell, "Peirce's Concept of the Sign," excerpt from Paul Cobley, ed., The Routledge Companion to Semiotics and Linguistics. New York, NY: Routledge, 2001.
- Graham Allen, Roland Barthes. London; New York: Routledge, 2003. Excerpt on Barthes' semiology.
[Provides background on the use of de Saussure's model. Barthes' work has become canonical in literary and cultural studies, but the de Saussurian foundation prevents the model from wider extensibility in other questions of meaning, media, technology, and cognition. Important to know for differentiating terminology in the different schools of thought. Primary text reference for Barthes' Elements of Semiology.]
- Martin Irvine, "Remix and the Dialogic Engine of Culture: A Model for Generative Combinatoriality". To appear in The Routledge Companion to Remix Studies, ed. Eduardo Navas, et al. (NY: Routledge, 2014).
[This book chapter provides a synthesis of key concepts and research methods from semiotics and other disciplines that converge on the question of combinatorial meaning making. The first sections are an overview of ways to merge semiotics, generative linguistics, and cognitive science approaches to symbolic expression. I also outline a way to develop an analytical method from these disciplinary resources for understanding complex symbolic expressions like artworks and music.]
These are difficult texts presupposing the backgrounds of their contexts of argument, but it's good to grapple with the primary statements, concepts, and terms, and work through the concepts and how/whether you can mobilize them to think with.
- Ferdinand de Saussure, extracts from Course in General Linguistics
- Focus on sections: Introduction, Chap. III, and III.3, "Place of Language in Human Facts: Semiology"; and Part 1, Chap. I, "On the Nature of the Linguistic Sign."
- Web version (useful for searching key terms): Third Course of Lectures on General Linguistics (1910)
- Focus on the description of the linguistic sign (signifier, signified, sign)
- C. S. Peirce, "What is a Sign" (start in section 3). From the Peirce Edition Project, Indiana University. [Also in Charles S. Peirce, Essential Pierce: Selected Philosophical Writings (1893-1913). Edited by Nathan Houser, Christian J. W. Kloesel, and Peirce Edition Project. Vol. 2. Bloomington, IN: Indiana University Press, 1998.]
- Roland Barthes, Elements of Semiology (1964; English edition, 1968): selections. Focus on Intro through 1.1.4; 1.2.1-1.2.5; part II is an overview of the Signifier/Signified structure.
- Marcel Danesi, "Semiotics of Media and Culture," excerpt from Paul Cobley, ed. The Routledge Companion to Semiotics. New York, NY: Routledge, 2009, 135-149.
- Yuri Lotman, "On
the Semiotic Mechanism of Culture"
[A classic statement of the approach to studying culture as the generation and transmission of meanings in the "stored memory" system of cultural artefacts.]
- Roland Posner, "Basic Tasks of Cultural Semiotics". Excerpt from Gloria Withalm and Josef Wallmannsberger, eds., Signs of Power -- Power of Signs. Essays in Honor of Jeff Bernard. Vienna: INST, 2004, p. 56-89.
Online Textbooks (for reference)
- Daniel Chandler, Semiotics for Beginners, read sections 1-2. [mostly on de Saussure's model]
- Peirce's Theory of Signs (Stanford Encyclopedia of Philosophy)
Novel: William Gibson, Pattern Recognition
- William Gibson's Site for Pattern Recognition: A novel about a "cool hunter" who uses (implicitly and explicitly) semiotics to code and decode corporate branding. Fun and very engaging. By the writer who coined the term "cyberspace.
William Gibson's website | Interview with William Gibson
Weekly Seminar Discussion (Wordpress site)
This week is an introduction to thinking with semiotic theory as part of an overall method for understanding, describing, and analyzing meaning-making in all its forms and media of implementation. We will develop methods, models, and research problems in the following weeks, but this week our task is becoming intuitively clear on the basic assumptions, concepts, and lines of approach that enable useful analysis of media and culture. For your writing, think through the "meaning systems" approach in first steps toward understanding how meaning happens in common cultural forms. Consider a small part of a genre example (a TV show, movie, music composition, art work) and think through the symbolic components that make meanings possible. Examples: analyze a shot sequence in a movie, the symbolic patterns and codes in a piece of music and/or music video, an advertisement (with photography, design, typography, for example). We will refine the procedures next week; the important step is to begin thinking with the concepts and assumptions and see what opens up that you couldn't see before.
|7 Mediology and Actor-Network Theory: Mediation and Distributed Agency in Technologies||[−]|
Understanding and gaining proficiency with using the concepts and methods in mediology and Actor Network Theory.
Mediology and Actor-Network Theory (ANT) are approaches for asking new questions about mediation, communication and transmission, distributed agency in media and technology. The concepts and methods for analysis are useful for bringing the invisible forces of social relations and institutional contexts up to awareness. They represent interdisciplinary research programs, points of view, and schools of thought guided by some open hypotheses, and not "disciplines" or "fields" of study per se.
A major motivation of these approaches is undoing any easy "effects" models of technology/media (as one domain) and "culture/society" (as a separate domain). As Latour emphasizes, positing these divisions as independent spheres (that must be somehow brought into a "relation") is a social construction; it does not represent anything real or observable in things or social organization. In fact, it's quite the opposite: technologies and media are formed with and media the organized forms of distributed social agency that we call technologies.
These methods can be called a "metatheory" or way to de-blackbox social and technological systems that are often constructed ideologically to be closed from analysis (e.g., the function of hype and productizing in consumer devices and digital technologies; the reification of media and technology as "things" rather than outcomes of complex relational processes). Hence it's an approach like a Wittgenstein's ladder, a tool for climbing up out conceptual problems already worked through, to form new or better questions or to see beyond old, uninteresting questions. It's a program for posing meaningful new redescriptions of questions and problems that have reached disciplinary dead-ends (as in Rorty's method of redescription as new interpretation).
Mediology is an open-ended synthesis of approaches that allow us to expand and combine the discoveries of: systems theory, network theory, semiotics, media and mediation theory since McLuhan, the sociology of media and technology, and models of distributed, interdependent agency in the disparate fields of Actor-Network Theory, Bourdieu's social-institutional analyses, and models of human-machine relations from cybernetics.
Mediology as a synthesis of theory: With the two major criteria toward using theory we have practiced in the seminar, how do you see mediology in its heuristic and self-reflexive or self-critical potential? Does the approach lead to new discoveries, even about theory itself, and what happens when we use some of the concepts to critique questions of media, mediation, and transmission? How does mediology continue, extend, or critique other theory traditions we have examined: communication theory, semiotics/semiology, post-modernism?
Mediology and Institutions of Mediation
- Regis Debray, "What is Mediology?" Le Monde Diplomatique, Aug., 1999. Trans. Martin Irvine.
- Debray, Transmitting
Culture, trans. Eric Rauth. New York: Columbia University
Excerpts in pdf: From Chaps. 1-2; from Chap. 7, "Ways of Doing."
- Review of Transmitting Culture: Constantina Papolias, "Of Tools and Angels: Regis Debray's Mediology," Theory, Culture & Society, 21/3 (2004): 165-70.
- Martin Irvine, Intro to Mediology and Actor Network Theory: How To Hack Black Boxes (CCTP-505 presentation)
- Martin Irvine, Lecture Notes: Introduction to Mediology (web)
- Frédéric Vandenberghe, "Régis Debray and Mediation Studies, or How Does an Idea Become a Material Force?" Thesis Eleven, 89, May 2007: 23-42. [Detailed philosophical critique of mediology and traditions of thought forming Debray's context.]
- Steven Maras, "On Transmission: A Metamethodological Analysis (after Régis Debray)"
[Useful as a review of communication/transmission theories and Debray's mediological view; focus on the section on Debray and transmission.]
Introduction to Actor Network Theory: A Model for Media, Technology, and Communication
- Bruno Latour, “On Technical Mediation.” Common Knowledge 3, no. 2 (1994): 29-64.
[This is a very accessible essay that outlines Latour's approach to the "distributed agency" in technologies as used and valued socially.]
- Introduction: Sal Restivo, "Bruno Latour: The Once and Future Philosopher," excerpt from The Blackwell Companion to
Major Social Theorists, ed. George Ritzer and Jeffrey Stepinsky (Boston: Blackwell, 2010). (From Sal Restivo, Dept. of Science and Technology Studies, Rensselaer Polytechnic Institute, publications page.)
[See the Key Concepts section.]
- Bruno Latour, "On Actor-Network Theory: A few clarifications
plus more than a few
[Not as accessible to those outside "Science and Technology Studies". An attempt to summarize problems and questions in Actor-Network Theory.]
- John Law, "Notes on the Theory of the Actor Network: Ordering, Strategy and Heterogeneity" (1992, revised 2003).
[Review the section on "Society as a Heterogeneous Network".]
Mediology/ANT Case Studies for discussion:
how do we describe and analyze different forms of mediation and agency?
- The Internet and Mediology: Institutions and mediation
- TV Culture and Institutions
- Hybrid computational and media devices and software: de-blackbox an iPhone
- The Museum as mediating institution
- The University as mediating institution
Weekly Seminar Discussion (Wordpress site)
Consider a major media form (like YouTube) or an iPhone app from a mediological and ANT point of view. What invisible institutional conditions and networks of agency come into view if you open up the larger questions of mediation and distributed agency? Or, what do you discover in a common medium by considering communication and/vs. transmission? Institutions and political economy of media industries? De-blackbox a combinatorial technology for its interfaces to larger systems of mediation?
|8 Software, Code, Information: Key Concepts in Computation||[−]|
Inventing Computation: Key Ideas
This unit focuses on the key concepts in computation as the core model for software, computer and information design, and all digital media. (We will study the computer industry later in the seminar.) We will approach the questions from a non-specialist perspective, but it's important for everyone to get a conceptual grasp of the core ideas in computation because they are now pervasive throughout many sciences (including the cognitive sciences), and are behind everything we do daily with computational devices, information processing, and digital media (for example, what is happening in the Google algorithm in searches).
Readings and Background
- Martin Irvine, An Introduction to Computational Concepts
- Daniel Hillis, The Pattern On The Stone: The Simple Ideas That Make Computers Work. New York: Basic Books, 1999. (excerpts from chaps. 1-2).
- Peter Denning, et al., "What is Computation?" Symposium of state-of-the-question essays by computer science leaders and teachers, sponsored by the main computer science professional organization, ACM ("Association of Computing Machinery", but now just "ACM" since computing is about far more than machines).
This is an accessible set of statements about current ways of defining and explaining computation as any kind of abstract implementable process for interpreting representations and outputting further interpretable representations.
(This view means that computation, while requiring high order mathematical modeling, isn't "about" math or the machines [computers] that implement computational processes [programs]. It's a form of symbolic representation/interpretation that we can automate by recruiting electricity, external memory units, and math-symbolic logic to implement abstract repeatable processes.)
Read Peter Denning's Intro and John Conery on "Computation as Symbol Manipulation."
- Harvard University, CS50, Online Course and Intro Computer Science Lectures.
Video Lecture site. Begin with CS50 Week 0, and view the first few weeks, and as many as you have time for. Very good accessible lectures for online learning.
Part of Harvard's public access computer science lectures and background for courses and MOOCs.
Part of Harvard's site for computer science topics: CS50 Live.
Background on the main founders of computation theory and concepts
- Andrew Hodges, Turing (Great Philosopher's Series). London: Phoenix, 1997; New York: Routledge, 1999.
[Web version of Hodges' short, accessible book about Alan Turning's main ideas. In a fundamental sense, all computers and digital devices are "Turing Machines".]
- Background on John von Neumann's computation models:
Project: Lessons on Udacity and Code Academy
- Create an account on both sites and let the tutorials prompt you through the lessons. Go as far as you can.
- Udacity: http://www.udacity.com
Sign up for this course on Intro to Computer Science: http://www.udacity.com/course/cs101
- Code Academy: http://www.codecademy.com
Sign up for the self-paced tutorial lessons on the Python programming language "track": (http://www.codecademy.com/tracks/python)
- and if you have time and interest, try the Web Fundamentals "track":
(other tracks: http://www.codecademy.com/tracks).
- These lessons will guide you through some basic computing concepts and also let you write some basic code and see the results when it runs.
Supplementary and Advanced Readings
- Charles Schmidt (Rutgers University), Computation and Cognition (online readings for course)
- Harvey Cragon, "The Von Neumann Machine," excerpt from Computer Architecture and Implementation. Cambridge University Press, Cambridge. pp. 1-13.
Weekly Seminar Discussion (Wordpress site)
Discuss what you learned in the key concepts and the introductory lesson in software code.
|9 Mediation, Hybrid Media, and Metamedia after Digital Convergence||[−]|
Combining our conceptual modeling resources and the mediological approach for studying digital and "new" media.
How do we describe the systems of mediation, institutions of transmission, media functions, and materialities of mediation for post-digital culture?
What are digital media? How is software a medium or metamedium?
What is a digital artefact? Are there properties of "being digital" that have unique affordances and consequences?
How do we understand as fully as possible the analog-digital continuum?
"New" Media and Mediation After Digital Convergence,
Software, and Computer Interfaces
- Introduction: Media Theory, Section 7 (Irvine)
- Alan Kay and the Dynabook Concept as a Metamedium:
Alan Kay's original paper on the Dynabook concept: "A Personal Computer for Children of all Ages." Palo Alto, Xerox PARC, 1972). [Wonderful historical document. This is 1972--years before any PC, Mac, or tablet device.]
Alan Kay and Adele Goldberg, “Personal Dynamic Media” (1977), excerpt from The New Media Reader, ed. Noah Wardrip-Fruin and Nick Montfort. Originally published in Computer 10(3):31–41, March 1977. (Cambridge, MA: The MIT Press, 2003), 393–404. [Revised description of the concept for publication.]
Interview with Kay in Time Magazine (April, 2013). Interesting background on the conceptual history of the GUI, computer interfaces for "interaction," and today's computing devices.
- Lev Manovich, The Language of New Media (excerpts). Cambridge: MIT Press, 2001.
A very influential book in the field. Read selections from chap. 1 (What is New Media) and chap. 2 ("The Interface")
Note the categories Manovich set up in chap. 1 for defining "New Media."
- Author's website with supplements to the book.
- Lev Manovich: "New Media from Borges to HTML." Introduction to The New Media Reader. Edited by Noah Wardrip-Fruin and Nick Montfort. Cambridge, MA: The MIT Press, 2003, 13-25. (File from author’s site: www.manovich.net).
See especially the section on "What is New Media: Eight Propositions."
- Lev Manovich, Software Takes Command, Intro chapters; attend especially to the section on "Cultural Software".
- Jay David Bolter and Richard Grusin. Remediation: Understanding New Media. Cambridge, MA: The MIT Press, 2000. Excerpts: Intro | Chap. 1 | Conclusion
Weekly Seminar Discussion (Wordpress site)
Describe and analyze, as fully as you can, the interface(s), media functions, and mediations implemented in a PC, an iPhone/iPad, or tablet. Use whatever theory and conceptual modeling tools we have been developing in your description. Do Manovich's and Bolter and Grusin's categories of description usefully map on to the integration of media in our current platforms? How do our interfaces and software present "media"? Go beyond the specifics of the device/equipment you currently have to the underlying model being implemented, the mediations to (invisible) systems in the interface.
|10 Post-Digital Mediation: From Walter Benjamin to the Google Art Project||[−]|
Understanding the issues surrounding technical mediation and reproduction as implemented for cultural representation and transmission.
One view of the transition from the modern to post-modern era centers around the question of representation and technical mediation in mass media. This question includes the status of images; the cultural, ideological, and technical function of media; the role of photography and film; and the mass mediation of social life in general (print, film, television, advertising). The shift toward digital representation, mediation, and distribution extends these questions and adds questions specific to digital objects and artefacts.
These macro questions about the mediation and representation of culture connect to our previous week's studies of cultural semiotics and the function of the cultural encyclopedia and interpretive communities.
The statements by the writers in this unit--Benjamin, Malraux, and Baudrillard--are a chain of accruing arguments, each presupposing the earlier, and adding analyses and observations from the media of their era and schools of thought that each theorist participated in. What are the main issues? How does digital reproducibility and mediation enable us to rethink these questions today?
The Google Art Project is being positioned in a nexus of cultural, institutional, and media technology issues. How are the issues discussed by these writers and the important theory in sociological descriptions of cultural institutions played out in this recent project of cultural mediation and representation?
- Walter Benjamin, "The Work of Art in the Era of its Technological Reproducibility" (1936; rev. 1939).
This is the more accurate translation of the title of the work known in the English-speaking world as The Work of Art in the Age of Mechanical Reproduction (html) (English trans., Harry Zohn, 1968). Compare both versions.
Focus on sections: Pref., I-VI, X-XII, XV.
Benjamin's title in German is "Das Kunstwerk im Zeitalter seiner technischen Reproduzierbarkeit" = "The Work of Art in the Era of its Technical (or Technological) Reproducibility."
The first link above is to an excerpt from the new edition of Benjamin's writings (Harvard Univ. Press, 2003) with the revised title.
Benjamin was part of the Frankfurt School of neo-Marxist writers who were engaged in a serious struggle with Fascism, and Benjamin proposed his own more nuanced analyses of history and media technologies. His argument is still in the grip of presuppositions about "mass culture" as passive responses, and he was writing as at time when Fascist capitalism and state-controlled cultural economies threatened to convert all media and cultural forms into tools of ideology.
Note the troubled questions about technology and agency and a historical narrative about technological development in section VI.
For further background on Benjamin's philosophy, see Philosophical Background (Stanford Encyclopedia of Philosophy): see sections 6-8.
- André Malraux, "La Musée Imaginaire (The Imaginary Museum)".
Overview and excerpts. English translation unfortunately as "The Museum Without Walls", a chapter in The Voices of Silence, 1951). Further implications of art and culture mediated through photography, and the assumption of a global "art encyclopedia" informing the modern concept of art and art history.
- Jean Baudrillard, "Simulacra and Simulations" (excerpt in pdf; requires GU login) (excerpt also in html version).
From Simulacra and Simulation, 1981; English trans., 1988.
This essay is often cited but seldom understood. It exemplifies Baudrillard's theory and style (full of rhetorical hand grenades and provocations), and needs to be worked through and critiqued. Important points are the view of mediated "reality" as interpreted through the code of "the real," which is a mediation and style of representation, not a transparent, direct connection with a non-mediated or pre-mediated lived experience (in all it's messy, chaotic complexity).
For background, see my presentation, "Mediation and Representation: Plato to Baudrillard and Digital Media."
- Art, Cultural Capital, Symbolic Value: Brief Context-Framing through Pierre Bourdieu's Theory
The topic of the reproduction and dissemination of art images leads to the larger question of the function of art (high art) as a cultural category, the institutions that maintain, install, and reproduce this cultural category, and the learned rules for making distinctions between high and low culture that come through socialization into the accepted encyclopedia.
For an overview, see my "Introduction to the Institutional Theory of Art and the Artworld" (summary also as presentation: "The Artworld as Institutional Network") and "Cracking the Art Value Code with Bourdieu."
And as time permits, the primary background in Pierre Bourdieu, "The Forms of Capital" (also html version) and "The Production of Belief" (1980/93).
- Google Art Project: A Meta-Museum for the Post-Digital World?
Experiment with the interface and analyze the assumptions in the way representations are structured.
Choose some of the major museums for the gallery "walk throughs" and experiment with viewing the high-resolution images and making selections for your own collection.
- Emily Magnuson, "Virtual Museums," Frieze Magazine Blog, 3.8.2011.
- Discussion in Curator: The Museum Journal. [Overview of issues from a Smithsonian curator.]
- Google Art Project, Wikipedia [Useful background on the technology and institutional issues.]
- [Is the Google Art Project a "meta-museum," a virtual museum of museums? How do we interpret the interface structure, horizontal scrolling and tiling, the significance of the framing and interface design?]
Weekly Seminar Discussion (Wordpress site)
- Explore aspects of the Google Art project as a case study for tying together the questions and issues in this week's reading and theory. How are institutions and artefacts represented and mediated? Is Google's well-known "street view" 360-degree simulation/presentation a code for "the real" in networked digital media culture? What happens semiotically when artworks, as defined and constructed in the cultural category transmitted in the institutions and the Artworld collectively, are mediated as detachable objects in "user-curator" arrangements? Museums as source material of images for Pinterest-style selection and arrangement? Or a useful advance in a learning tool that enables individual learning, pattern recognition, and understanding of the museum function? All of the above?
|11 The Digital Milieu: Interfaces, Distributed Cognition, Metamedia||[−]|
Working out ways of modeling and analyzing the environment of post-digital cultural and the larger system of symbolic cognition in multiple media technologies and hybrid combinations of analog and digital.
How can we best model and describe "post-digital" culture as a meaning system (a system of interdependent mediums, mediations, technologies, and cultural genres comprising a "mediasphere" or "semiosphere")?
This week’s readings are intended to be read in our cumulative context of theory, approaches, and methods. Combined with prior weeks’ readings, the approaches this week provide a merged view of new media and software, cognitive science models for the function of symbolic artefacts and media technologies as forms of distributed cognition or extended mind, and media technologies as mediations of ideologies and social structures.
We will investigate the points of intersection in three disciplines and traditions of thought to find new conceptual models for problems that cannot be modeled with the resources of only one approach, point of view, or vocabularies of description.
Important interconnected topics: Interfaces, cultural memory, externalized memory, distributed cognition, technological and social mediation
Consider this interdisciplinary intersection of approaches, positions, and arguments
- Lev Manovich, Software Takes Command, finish book. Following up from the prior week's readings of Manovich's main arguments. Consider his expanded concept of software, computer interface as metamedium, software hybridity, and the cultural-computational view of digital media.
- Andy Clark's arguments about distributed cognition in the "extended mind" hypothesis: the function of language, symbolic systems, and technological artefacts as "cognitive scaffolding" that have shaped human cognitive functions from earliest language and artefacts to current combined technologies--with no break or rupture resulting from the industrialization of these cognitive artefacts in computational and digital devices. Macro question: what is a part of a continuum--recursive implementations of our symbolic faculty, accruing levels of complexity from combinatorial processes--and what can be defined as different in kind or in degree from earlier media?
- Andy Clark, Supersizing the Mind: Embodiment, Action, and Cognitive Extension (New York, NY: Oxford University Press, USA, 2008), excerpts from the Forward by David Chalmers, pp. ix-xi; xiv-xvi (attend especially to the comments in the last 3 pages of the Forward); Chapter 1.3: "Material Symbols" on the concept of "cognitive scaffolding."
- Janet Murray, Inventing the Medium: Principles of Interaction Design as a Cultural Practice. Cambridge, MA: MIT Press, 2012. Selections:
- In this context, recall the readings on distributed cognition from week 4:
- James Hollan, Edwin Hutchins, and David Kirsh. “Distributed Cognition: Toward a New Foundation for Human-computer Interaction Research.” ACM Transactions, Computer-Human Interaction 7, no. 2 (June 2000): 174-196.
- Jiajie Zhang and Vimla L. Patel. “Distributed Cognition, Representation, and Affordance.” Pragmatics & Cognition 14, no. 2 (July 2006): 333-341.
Weekly Seminar Discussion (Wordpress site)
After thinking through this week’s readings, what combined interdisciplinary view emerges for you as you think through the concepts, models, methods, and tools for analysis in this course? Are you starting to assemble your own heuristic “tool kit” for discovering how to describe and analyze key questions in media and technology? What do we make of the diverging and even incompatible views taking off from different levels of description and disciplinary foundations? Is there a convergence around central questions and problems? How can the approaches and ways of framing problems considered this week be used to critique the traditional "media studies" assumptions?
We have seen that two key questions about digital media and the digital milieu (in Debray's sense) are (1) the dependence on software and the encoding/decoding of digital media for interfaces, and (2) the macro question of the function of the interface, both as a technical-material implementation and as a function of culture and social organization (parallel to mediation). Interfaces seem also closely connected to what the cognitive scientists are discovering about the extended mind and distributed cognition. A useful thought project for this week: try on multiple ways to get at the function of the interface (as a function or meta-function), as we have with the closely related concept of mediation. We use the popular discourse term “interface” for features of technical artefacts, products, and software, and standardized industry concepts in human-computer-interface (HCI) design principles, but these are all implementations of an abstract machine or function that precedes any individual design-for-manufacturing. Can we make the invisible/opaque boundaries and thresholds of interfaces visible? A good case to think through with descriptive concepts from Manovich and others are the functions of a meta-interface that we use everyday like a Web browser (Internet Explorer, Google Chrome, Firefox, Safari, etc.) or the "desktop" of a PC screen. What is the larger invisible system "behind" the interface that enables everything to appear so obvious and immediate (unmediated) to us?
Weekly Seminar Discussion (Wordpress site)
|12 Distributed Cognition Case Study: Ambient / Pervasive Computing and the City||[−]|
Understanding how to analyze extended cognitive technologies in a pervasive environment like the embedded computational intelligence in city environments that include, and integrate, computing devices (large and small), wireless and wired networks, telephony, GPS, cameras, all kinds of sensors relaying information to computing devices and networked information processing.
The readings this week draw from current research and theory in urban studies and urban design, artificial intelligence and cognitive computing, and the extended field of pervasive and ambient computing. The affordances and design principles of digital networks--wired or wireless ("the network is the computer"), modular computation (mobile and embedded devices), digital sensors, GPS systems, digital cameras, and computerized control systems for the entire built environment (traffic, electrical grid, telephone systems, security systems, etc.) now all combine and intersect with a large "cumulative combinatorial" effect from distributed agency and cognitive technologies. We will open up the implications of this environment and use a synthesis of methods and concepts from the seminar for ways to interpret, describe, and analyze where we are and what may be expanded from the "permanent extensibility" of the built-in design principles of computation and digital information.
The City and/as Technology:
Local Material Spaces, Pervasive Computing, Global Networks
“I’ll begin with the following hypothesis: Society has been completely urbanized. This hypothesis implies a definition: An urban society is a society that results from a process of complete urbanization. This urbanization is virtual today, but will become real in the future.” (Henri Lefebvre, The Urban Revolution, 1970:1)
Henri Lefebvre's statement above in The Urban Revolution (1970) was written at a time when over one third of the world’s population lived in cities or metropolitan regions. Today more than one half of the world’s population is concentrated in cities and large metropolitan regions, and, soon, urban metropolitan concentration will reach two thirds of world population. Cities have always been the material, spatial, economic, technological, and social centers of the world, aggregating and concentrating resources and interactions according to the laws of network effects in dense nodes. "Complete urbanization" since the 1990s means a specific kind of urbanization in "global cities": networked, pervasive computing as the norm and city dwellers as information navigators using many layers of computational "distributed cognition" in the built environment.
Cities have always been the most efficient aggregation and concentration points for technologies enabling social transactions (transportation, communication, finance, education, government). The city exists as an information medium. The movement of people in, out, and through cities is information: human traffic flow in highways, transportation systems, streets, shopping centers, offices. From the mediological and ANT points of view, we can see that a city mediates and distributes all kinds of agency, cognitive functions, and information about itself. Cities are designed to provide interfaces for use, and design affordances for finding one's way and being part of the information environment.
From all the "orchestrated combinatoriality" in the built environment and its technologies, cities are now the major zones of "pervasive," "embedded," and "ubiquitous" computing with environmental sensors, embedded chips and RFD tags, surveillance cameras, geo-location/GPS tracking, and telecom/data wireless networks everywhere.
Examples and Background
From Built Environment (Place and Space) to Pervasive and Ubiquitous Computing
- Carlos Ramos, Juan Carlos Augusto, Daniel Shapiro, "Ambient Intelligence--The Next Step for Artificial Intelligence." IEEE, Intelligent Systems, 2008. [Accessible introduction to issues.]
- Diane J. Cook, Juan C. Augusto, and Vikramaditya R. Jakkula. “Ambient Intelligence: Technologies, Applications, and Opportunities.” Pervasive and Mobile Computing 5, no. 4 (August 2009): 277–98. [GU student login only]
[More technical, but good overview of the design models for ambient systems.]
- Nashid Nabian and Carlo Ratti, “The City to Come,” in Innovation: Perspectives for the 21st Century (OpenMind), https://www.bbvaopenmind.com/en/article/the-city-to-come/.
- Dietmar Offenhuber and Carlo Ratti, “Reading the City: Reconsidering Kevin Lynch’s Notion of Legibility in the Digital Age,” in The Digital Turn: Design in the Era of Interactive Technologies, ed. Zane Berzina, Barbara Junge, and Walter Scheiffele (Zurich: Weissensee Academy of Art, Park Books, 2012), 216-224.
- Jesper Kjeldskov et al., “Digital Urban Ambience: Mediating Context on Mobile Devices in a City,” Pervasive and Mobile Computing 9, no. 5 (October 2013): 738-749.
[Good article for latest approaches to ambient computing in urban environments, but consider the overall concepts without getting caught up in the specific products and current implementations.]
- Thad Starner, “Project Glass: An Extension of the Self,” IEEE Pervasive Computing, 2013.
[We can only begin an analysis of the kind of extended computing and augmented reality modeled in Google Glass. But it is a good case since this technical implementation presupposes the whole infrastructure of the built environment and Internet-based ubiquitous computing (wireless and wired networks, digital media, "big data," Cloud computing, etc.)].
Further Research Sources:
(1) Computer Science and Engineering
- IEEE Computing Organization: Magazine sites for Pervasive Computing and Intelligent Systems
See also the list of professional journals in the IEEE Transactions series.
[These are the more business and publicly accessible magazines, but on the technical side: important if you want to pursue the details of pervasive, ambient, and embedded computing.]
(2) City Infrastructures and Pervasive Computing Project: MIT's SENSEable City Lab:
Weekly Seminar Discussion (Wordpress site)
- A field work project after this week's readings: Go to and spend some time walking around in a major neighborhood in Washington, DC and do a "thick description" of as many layers of "distributed cognition" and "ambient computing/intelligence" that you can identify (some will not be immediately visible). For example: (1) the messaging system of the city for people navigating the space of the built environment (streets, roads, architecture) with electronic and computational systems supporting the physical structures, (2) indications of the ambient, pervasive computing-information environment, (3) visible and hidden/assumed infrastructure for utilities and communications in the daily use of technologies. Examples of dense locations: (1) intersection of 14th and U Streets, and 2-3 blocks in each direction from the intersection, (2) intersection of K Street and Connecticut Ave., and 2-3 blocks in each direction from the intersection, (3) the Washington Mall, from the Washington Monument to the Capitol Building, and views along the way. Where are people moving from/to? Road traffic, metro, walking? How many are using cell phones/smart devices? How many observable wireless Internet places/zones are there (coffee shops, bars, restaurants, more)?
|13 New Media Implementations: Concluding Combinatorial Case Studies||[−]|
Consequences of the theory and research methods in this seminar:
working with interdisciplinary theory and approaches.
Use the case that you have been assigned for working with the methods and concepts in the seminar. How can the specific example be understood in the context of the larger technology and media principles we have studied, such as:
cumulative and accruing combinations (the principle of "orchestrated combinatoriality"),
symbolic processes and their material interfaces and mediations,
what longer continuum of functions are (re)implemented or (re)mediated on this instance,
the invisible networks of distributed agency and cognition represented in any instance of media technology,
the technology design principles for extensibility and hybrid combinations (especially in software),
all the dimensions of mediation (that go beyond content to social, political, economic, and institutional conditions and forces?
Build out the fullest "thick description" that you can develop for your case.
Case Studies Overview (presentation with background and bibliography, read first)
Examples and choose one for a case study analysis:
Note: If you Google any of these topics, you're certain to get mostly products (hardware and software), not explanations of concepts (abstract machines) behind the product development.
- Digital paper, e-ink, ebooks, thin film substrates: merging page and screen functions.
- The MP3 as medium, format, system of encoding and representation analog information in digital form, industry standards for the music industry. Ecosystem of software, production, distribution, devices.
- Google Glass: augmented reality and combinatorial wireless networked device.
- Streaming Media
- The technical architecture and interface design of Google Art Project
On combinatoriality in technology design:
W. Brian Arthur, The Nature of Technology: What It Is and How It Evolves. New York, NY: Free Press, 2009. Excerpt, Chaps. 1-2. [GU student access only]
MP3 as Case Study
- Jonathan Sterne, “The MP3 as Cultural Artifact,” New Media & Society 8, no. 5 (October 1, 2006): 825-842.
- Jonathan Sterne, MP3: The Meaning of a Format (Durham, NC: Duke University Press Books, 2012), "Format Theory" (excerpt).
Weekly Seminar Discussion (Wordpress site)
Use an example for your own case study and analysis of the technologies, mediations, dependencies, and interface functions represented by the case. Try to work out as full a description as you can with the conceptual models, approaches, and methods that we have studied.
|14 Discussion and Presentation of Seminar Projects||[−]|
Final Project Essay Instructions and Resources (read instructions first)
In class: round table discussion of research projects in progress.
Due date: final project essays are due one week after the last day of class.
Archive of earlier student essays (Wiki site): scroll down to heading for final projects by semester.