As technology advances, traditional avenues for connecting with others around us have become more difficult. Personal visits and aural communication gave rise to letters, letters gave rise to phone calls, phone calls gave rise to emails, email gave rise to text messages, and text messages gave rise to Emoji texts, and so on. Each new electronic technology, from Morse code to the mobile phone, has allowed us to move further from our physical selves rather than connecting with each other face-to-face.
As we move into abstracted modes of electronic and networked communication such as texts, tweets, and Emoji, we should examine, meditate upon and explore the impact this way of communicating upon our ability to connect with each other in physical space. By highlighting Emoji as creative medium we start to unpack the millions of Emoji texts and messages sent every day, and what might be emerging as a meaning-making tool beyond the chat log.
Let us examine the Hikikomori in Japan, a group of mostly young men in their twenties in Japan who do not leave their house and still live with their parents. Often the only communication source they have to outside world is through technology. Combine this with Japan’s declining birth rate and young people who are losing interest in romantic partnerships and we begin to see the effects of technological disassociation. (Haworth 2014) I do not think it is a coincidence that Emoji is Japanese. There is a connection between people moving away from intimacy and connection as we begin to mediate our communication through more abstracted and reduced forms of electronic communication.
Figure 1: Concept Map
The artist Xu Bing explores the impact of images as text in his book, Book from the Ground: From Point to Point (See Figure 2).This book chronicles the narrative of one person’s experiences without words, using symbols. Having “read” book from the ground, I was able to follow the series of actions, or plot of the book. That fact surprised me, given that I was not expecting to be able to do that.
Figure 2: An excerpt from Xu Bing’s book
The Noun project is another crowd-sourced online repository for symbols and icons that are both free and paid. You can upload your own icons, or download public domain icons for free, or download paid icons through monthly licenses.
Both projects are bringing focus to symbols as meaning making tools to create narrative and explore cross-language possibilities of iconographic imagery. There are also a huge number of Emoji focused web and analog projects being created right now (See Figure 3).
Figure 3: Here are a few projects connected to Emoji
Emoji has reached a critical mass in our culture and a few things precipitated this. One is the induction of Emoji into the Unicode standard in 2010. “The Unicode Standard is a character coding system designed to support the worldwide interchange, processing, and display of the written texts of the diverse languages and technical disciplines of the modern world. In addition, it supports classical and historical texts of many written languages.” (Unicode, 2014) Introducing Emoji into Unicode allowed Apple and Google to share the same set of Unicode characters on their OS platforms.
I recently became one of 6 student members of Unicode.org because I wanted to get access to the body of people working on ratifying these characters for use in the communications industry. I am currently working on creating diverse Emoji that I will submit for review to Unicode.
Another factor affecting the changing relationship we have to Emoji is that we live in a culture where digital technology is changing the way people communicate. Sherry Turkle gave a talk recently at The New School about the lost art of conversation. She called this situation “Wait, What?” She described communication among young people as a series of disconnected micro-conversations where people fluidly drop out to check their devices and then enter back in saying “wait, what?” Basically, she has found that people are resorting to text-based communications more and more in lieu of face-to-face talking.
My interest in Emoji is that I believe it has become a prevalent form of mediated visual communication. We rely on it to give us signals and visual cues that faces and live expressions once did. (Derks et al., 2008, 766-785)
Leonard Shlain writes about language versus visual communication and the connection with dyslexia. He believes that TV has shifted our brains to move more toward the right/visual hemisphere of the brain. Traditionally, 90% of language centers live in the left side of the brain. He also points out that dyslexic people who are right handed have up to a 70/30 split with only 70% of language centers in the left side of the brain.
. . . A previously unrecognized affliction called dyslexia (nonexistent in ideographic China) broke out at alarming rates in classrooms all across Eurocentric TV-land. Dyslexic children, predominantly male (9:1), have difficulty deciphering the alphabet. One credible theory proposes that it is due to a failure of hemispheric dominance. Ninety percent of the language centers traditionally reside in the left hemisphere of right-handed people. In the right-handed dyslexic, the distribution of language centers may be more on the order of 80/20 or 70/30. Although we cannot be sure that dyslexia was not always among us, it seems to have erupted at the very moment that an entire generation was devaluing the left hemispheric mode of knowing. Perhaps television is the agent equilibrating the human brain’s two differing modes of perception. (Shlain 1998, chap. 25, loc. 7965)
This discovery leads me to believe that Emoji may be filling in the gaps for our lack of direct, present communication as the language of visual media is changing the way our minds work. As a right-handed dyslexic, this was really interesting to me and maybe explains why I have such an affinity for Emoji as an expressive mode. What does it mean that in China, where they use an ideographic language there is basically no issues of dyslexia? What can Emoji offer us in way of a meaning-making tool that is not affected by dyslexia?
Images have weightiness, and are structured differently than text. They evoke right brain responses. They turn on different thinking centers. The derived meanings are more up to the recipient. They are less definite, and therefore, they are capable of greater semiotic inscription. “Finally, and in more general terms, it appears increasingly more difficult to conceive a system of images and objects whose signifieds can exist independently of language: to perceive what a substance signifies is inevitably to fall back on the individuation of a language: there is no meaning which is not designated, and the world of signifieds is none other than that of language.” (Barthes and Lavers 1968, 10) Although Rolland Barthes may not have intended this reading, the language, to which with he refers, is becoming an increasingly visual one that allows for a blurring between the signified (the idea of the object or idea being represented) and the signifier (the word, sound, or image representing it). Emoji is there to fill this gap and manifest the promise of a semiotic relationship that moves past words, and constitutes a language of images. My project deals with the question, how does technology mediate our relationships with each other? Does it contribute or disrupt human closeness and connection?
The primary audience for my project is a tech-savvy American man or woman between the ages of 18 and 44 with an interest in Japanese Emoji, art, and language. They are the early adopter sort of person who uses Twitter and texting quite frequently and enjoys adding content to websites like YouTube, Vine, and Reddit. This audience will be the one who adds definitions to Emoji in the Emoji dictionary and engages with creating content on the website. They will also engage in social sharing of the content on Facebook and Twitter to engage other to participate in this crowd sourced Emoji Dictionary. I imagine this audience as the first group to engage with the content and responsible for helping spread the word about the project. As you can see in Figure 2, 64% of the audience for Twitter is between the ages of 18-44. That audience is a similar demographic group to my own.
Figure 4: Twitter user graph
My secondary audience for the project is a less tech-savvy, middle class American man or woman who is in the 30-70 year old range. This audience member will use the dictionary as a means to uncover the meaning of Emoji texts and tweets that they have received from more Emoji-versed loved ones. This audience member is generally unlikely to add to the dictionary content, but more likely to search the content for definitions to Emoji. They are an audience for a later prototype or iteration of the World Translation Foundation Emoji Dictionary.
Mara is a 21-year-old art student living in Manhattan, who loves texting and tweeting. She uses Emoji frequently and believes it helps her express things that she could not say in words. As a person active in the creative community of New York, she heard about the World Translation Foundation from a friend on Twitter and decided to check it out. She got to the homepage and clicked on her favorite Emoji. She realized that the definition for that Emoji was not complete. She then decided to add in the definition of dancers: Once she had done that, she shared her contribution on WTF
Jack is a 35-year-old writer living in Brooklyn. He is not very tech-savvy and rarely posts any content online other than the occasional Facebook status. He saw a post about the WTF Emoji Dictionary from Mara on Facebook. He clicked on it because it looked fun and he some time to kill. He searched out a few Emoji and had fun playing around with the combinations of Emoji to make up new sentences. He would not add to the dictionary, but simply explore the content.
Prototype 1: World Translation Foundation Website
I am creating a database, website, and resource hub for art and thinking around these new communication forms. The project is called The World Translation Foundation.
The World Translation Foundation believes that words often get in the way of expressing how we feel. WTF is one part web project, one part tongue-in-cheek art movement, and two parts serious scholarship. While words can be cumbersome and misleading, and worst of all, they change from country to country, the pictorial alphabet of the Emoji lexicon is ubiquitous; easy to use and understand. In a tradition that dates back to the hieroglyphs of ancient Egypt, WTF aims to educate the public in the art of letting art speak for you. Our motto is: “Let WTF transform the way you communicate, and pave the way to a quieter tomorrow!” Right now, WTF is a website hosted on a custom installation of WordPress. Contributors include Cara Rose DeFabio and Niki Selken, the author.
As a means to explore the semiotic impact of Emoji, I created the Emoji Dictionary. This is a portion of the WTF website which was built out using PHP and a MySQL database to store the Emoji codes and image locations as well as definitions. The dictionary will be a living document, allowing users to add to the definitions of each of the emoticons. Somewhere between the Noun Project and Wikipedia, the Emoji dictionary will serve as a link between pictographic communication and alphabetic, written language.
Prototype 3: A live website and MySQL database
During this phase of the project I am creating a working, hand coded and designed database with live from for user added content. The dictionary prototype started with the first 66 Emoji face characters and allow users to add definitions to each. These definitions are then be stored in the database and added to the front end of the website with PHP. I will also use this time to perform user tests of the interface design. I plan to test with my primary target audience who will fill out the definitions during this phase.
During the user testing, participants generally liked the World Translation Foundation themes and layout. There was no confusion about how to add to the dictionary or navigate the site. Many users wanted to see the Emoji Research page on the WTF blog, which was not completed. Also, there was some concern about the English-centric nature of the Emoji Dictionary. Unfortunately, due to my own language constraints the dictionary will have to launch in English. In future, I would like to use The Noun Project’s multilingual site structure as a precedent for expanding the dictionary to other languages.
The World Translation Foundation prototype is fully functional and interactive. The next steps for the project are to add the additional Emoji to the dictionary and then launch the website. A social media Campaign will accompany this launch process. I will also email various news sources, which report on Emoji such as Yahoo Tech, who has already published an article featuring the project and my thoughts on Emoji. (Yahoo Tech 2014)
By highlighting Emoji as creative medium we can start to unpack the millions of Emoji texts and messages sent every day, and what might be emerging as a meaning-making tool beyond the chat log. The World Translation Foundation is a project dedicated to exploring the impact Emoji is having on American culture. WTF takes the form of an Emoji art focused website as well as a database driven, crowd-sourced English to Emoji dictionary. By connecting with the online community through creating an interactive, fun, and informative exploration of Emoji. I hope to expand awareness of Emoji as a valuable, emerging iconographic and pictographic language.
Ideographic: Ideographic refers to a graphic symbol that represents an idea or concept. The term “ideogram” is commonly used to describe logograms in writing systems, i.e. Chinese characters.
Pictographic: Pictographic is an ideogram that conveys its meaning by it’s graphic similarity to a physical object.
The Unicode Standard: The Unicode Standard is a character coding system designed to support the worldwide interchange, processing, and display of the written texts of the diverse languages and technical disciplines of the modern world. In addition, it supports classical and historical texts of many written languages. (Unicode, 2014)
“About the Unicode Standard.” Unicode Standard. Accessed May 15, 2014.
“Americans and Text Messaging.” Pew Research Centers Internet American Life Project RSS. http://www.pewinternet.org/Reports/2011/Cell-Phone-Texting-2011/Main-Report/How-Americans-Use-Text-Messaging.aspx (accessed April 8, 2014).
Barthes, Roland, Annette Lavers, and Colin Smith. Elements of semiology. New
York: Hill and Wang, 1968.
Benenson, Fred. Emoji Dick. New York: LuLu, 2010.
Derks, Daantje, Agneta H. Fischer, and Arjan E.R. Bos. “The Role Of Emotion In Computer-mediated Communication: A Review.” Computers in Human Behavior 24, no. 3 (2008): 766-785.
“Emoji love: The science behind why we <3 emoticons.” TNW Network All Stories RSS. http://thenextweb.com/dd/2014/03/10/Emoji-love-science-behind/ (accessed April 8, 2014).
Haworth, Abigail. “Why have young people in Japan stopped having sex?.” The Observer. http://www.theguardian.com/world/2013/oct/20/young-people-japan-stopped-having-sex (accessed April 7, 2014).
Kok, B. E., K. A. Coffey, M. A. Cohn, L. I. Catalino, T. Vacharkulksemsuk, S. B. Algoe, M. Brantley, and B. L. Fredrickson. “How Positive Emotions Build Physical Health: Perceived Positive Social Connections Account for the Upward Spiral Between Positive Emotions and Vagal Tone.” Psychological Science 24, no. 7 (2013): 1123-1132.
“The Noun Project.” The Noun Project. Accessed May 15, 2014.
O’Neil, Brittney. “Mirror, Mirror on the Screen, What Does All this ASCII Mean?: A Pilot Study of Spontaneous Facial Mirroring of Emotions.” Arbutus Reveiw 4, no. No 1 (2013): 19-43.
Shlain, Leonard. The alphabet versus the goddess: the conflict between word and image. New York: Viking, 1998. Kindle Edition.
Thorhauge , Stine . “The Demographics of Twitter Users [Study] | Mindjumpers.” Mindjumpers RSS. https://www.mindjumpers.com/blog/2012/08/demographic-of-twitter (accessed April 8, 2014).
“Using Emoji in Web Apps.” iamcal.com. http://www.iamcal.com/Emoji-in-web-apps/ (accessed April 8, 2014).
“Why You Should [THUMBS UP] the Emerging Emoji Art Movement.” Why You
Should [THUMBS UP] the Emerging Emoji Art Movement. Accessed May 16, 2014. https://www.yahoo.com/tech/why-you-should-heart-smiley-face-thumbs-up-the-82103808654.html.
Xu, Bing. Book from the Ground: From Point to Point. Cambridge: MIT Press,
One Comment on “Emoji Research”
Nina Daoud reads and writes about people from the US and around the world pursuing education in America. Daoud didn t have the easiest time explaining her research in emoji, though.